attention-free

Tag

Cards List
#attention-free

SPD-MetaFormer is what you need for small-data brain decoding

arXiv cs.LG ↗ · 2026-10-09 Cached

论文提出 SPD-MetaFormer,一种面向小数据脑信号解码的无注意力架构,基于 SPD 流形上均匀加权 Fréchet 聚合,并发现现有流形注意力模型(MAtt、GBWAtt)的学到权重接近均匀,因此简化设计即可获得有竞争力的 EEG 基准结果。

0 favorites 0 likes
#attention-free

Beyond the Parameter Monolith: Reconstructive Memories, Executable Skills, and Residual Assembly for Language Models

Hugging Face Daily Papers ↗ · 2026-10-02 Cached

This paper explores an alternative non-Transformer language model architecture that separates memory, deterministic execution, and contextual computation through explicit interfaces, using an attention-free Multi-Mesh model, reconstructive document memories, and a modular core with typed residual assembly. The authors frame it as a research prototype with released checkpoints, inviting independent testing and discussion rather than claiming superiority over Transformers.

0 favorites 0 likes
#attention-free

TinyCast: Probabilistic Zero-Shot Forecasting with Computed Periodicity

Hugging Face Daily Papers ↗ · 2026-08-16 Cached

TinyCast is a compact zero-shot time series foundation model with computed periodicity, enabling efficient probabilistic forecasting on edge devices like Cortex-M7.

0 favorites 0 likes
#attention-free

Kathleen Writes: Autoregressive Generation and Data Scaling Without Attention

arXiv cs.CL ↗ · 2026-08-06 Cached

This paper from the Kathleen series shows that an attention-free, byte-level model with ~0.5M parameters can beat a parameter-matched transformer on WikiText-103 language modeling and generation, introduces a non-parametric 'Form Distance' metric for evaluating text realism, and demonstrates that retrieval-augmented decoding from the model's own training corpus improves generation quality.

0 favorites 0 likes
#attention-free

CARNet Cycle-Conditioned Core Aggregation and Redistribution for Multivariate Time Series Forecasting

arXiv cs.LG ↗ · 2026-07-27 Cached

CARNet integrates global recurrent cycle information into efficient core-based interaction modeling for multivariate time series forecasting, achieving linear complexity and outperforming strong transformer baselines on real-world benchmarks.

0 favorites 0 likes
← Back to home

Submit Feedback