Tag
This paper introduces a trained fast-weight memory mechanism for a 3M-parameter transformer that installs never-trained rules at inference using forward-only test-time training, showing no transfer. The work is fully reproducible on a single RTX 3090.
FAAST proposes a forward-only method that compiles labeled examples into fast weights analytically, enabling efficient test-time supervised adaptation without backpropagation, achieving over 90% speedup and 95% memory savings while maintaining performance.