Tag
论文提出 SPD-MetaFormer,一种面向小数据脑信号解码的无注意力架构,基于 SPD 流形上均匀加权 Fréchet 聚合,并发现现有流形注意力模型(MAtt、GBWAtt)的学到权重接近均匀,因此简化设计即可获得有竞争力的 EEG 基准结果。
This paper explores an alternative non-Transformer language model architecture that separates memory, deterministic execution, and contextual computation through explicit interfaces, using an attention-free Multi-Mesh model, reconstructive document memories, and a modular core with typed residual assembly. The authors frame it as a research prototype with released checkpoints, inviting independent testing and discussion rather than claiming superiority over Transformers.
TinyCast is a compact zero-shot time series foundation model with computed periodicity, enabling efficient probabilistic forecasting on edge devices like Cortex-M7.
This paper from the Kathleen series shows that an attention-free, byte-level model with ~0.5M parameters can beat a parameter-matched transformer on WikiText-103 language modeling and generation, introduces a non-parametric 'Form Distance' metric for evaluating text realism, and demonstrates that retrieval-augmented decoding from the model's own training corpus improves generation quality.
CARNet integrates global recurrent cycle information into efficient core-based interaction modeling for multivariate time series forecasting, achieving linear complexity and outperforming strong transformer baselines on real-world benchmarks.