@daniel_mac8: babe, wake up. new continual learning breakthrough just dropped. fast-slow training (fst) treats model params as "slow"…
Summary
This tweet announces Fast-Slow Training (FST), a new continual learning method that treats model parameters as slow weights and optimized context as fast weights, reportedly outperforming weights-only training on math, code, and general reasoning benchmarks.
View Cached Full Text
Cached at: 05/18/26, 02:32 PM
babe, wake up.
new continual learning breakthrough just dropped.
fast-slow training (fst) treats model params as “slow” weights and optimized context as “fast weights”.
“across math, code, and general reasoning benchmarks, fst beats weights-only training on every axis we https://t.co/E3fHQKCAk0
Similar Articles
Learning, Fast and Slow: Towards LLMs That Adapt Continually [R]
This paper introduces a Fast-Slow Training framework for LLMs that combines parameter updates with optimized context to improve sample efficiency and reduce catastrophic forgetting during continual learning.
@LakshyAAAgrawal: Learning from rich textual feedback (errors, traces, partial reasoning) beats scalar reward alone for LLM optimization.…
Fast-Slow Training (FST) interleaves context optimization (via GEPA) with model weight updates via RL, achieving 3× sample efficiency over RL alone on math, code, and physics reasoning while preserving plasticity and enabling continual learning.
FAAST: Forward-Only Associative Learning via Closed-Form Fast Weights for Test-Time Supervised Adaptation
FAAST proposes a forward-only method that compiles labeled examples into fast weights analytically, enabling efficient test-time supervised adaptation without backpropagation, achieving over 90% speedup and 95% memory savings while maintaining performance.
Learning, Fast and Slow: Towards LLMs That Adapt Continually
A fast-slow learning framework for LLMs combines fixed slow weights with optimized fast context weights, achieving up to 3x better sample efficiency and reduced catastrophic forgetting in continual learning scenarios.
@AnjneyMidha: very cool a 2-3x speed up in training by essentially letting the model learn more flexibly in its early stages than rig…
A new training method achieves 2-3x speedup by allowing models to learn more flexibly in early stages, akin to homeschooling vs. factory education.