Tag
This paper from the Kathleen series shows that an attention-free, byte-level model with ~0.5M parameters can beat a parameter-matched transformer on WikiText-103 language modeling and generation, introduces a non-parametric 'Form Distance' metric for evaluating text realism, and demonstrates that retrieval-augmented decoding from the model's own training corpus improves generation quality.
CARNet integrates global recurrent cycle information into efficient core-based interaction modeling for multivariate time series forecasting, achieving linear complexity and outperforming strong transformer baselines on real-world benchmarks.