Tag
This paper introduces Modular TTT, a framework that represents test-time training inner learners as directed acyclic graphs, enabling systematic ablation and composition of components. The authors train 410M and 1.45B parameter models on 100B tokens, achieving performance comparable to GatedDeltaNet.
Fractale-350M-base is a 386M-parameter base model pretrained from scratch with a novel trained fast-weight memory bank of 8 vectors replacing long context, fully open-sourced including weights, code, and research log.
This paper proposes a sleep-like consolidation mechanism for transformer models that uses fast weights and recurrent passes to improve long-context processing while maintaining inference speed.
FAAST proposes a forward-only method that compiles labeled examples into fast weights analytically, enabling efficient test-time supervised adaptation without backpropagation, achieving over 90% speedup and 95% memory savings while maintaining performance.