Tag
Recurrent Looped Transformer (RLT) is a novel architecture combining a causal encoder with a recurrent decoder to achieve latent reasoning with unbounded temporal depth, model-hardware co-design, and model-RL algorithm co-design.
This paper analyzes recurrent fast-weight memories and selective state-space models as online learning rules, deriving normalized update families that improve length extrapolation and remain competitive in language modeling.
The paper introduces Readout Feedback (RoFB), a test-time intervention that steers latent dynamics in recurrent reasoning models using their own readout probabilities, achieving performance gains on Sudoku and Maze tasks without retraining.
This paper proposes quantum-inspired recurrent models (QKAN-FWPs) for traffic-matrix forecasting, demonstrating superior accuracy with fewer parameters compared to LSTM baselines.
This paper introduces FRESCO, an Echo State Network architecture operating entirely in the frequency domain to achieve O(N) complexity for dense recurrent updates, matching state-of-the-art performance on benchmarks while reducing computational costs.
This paper introduces MF-Net, a recurrent dynamical model that represents multivariate systems through a shared field state and learns a mechanical transition for joint evolution. It achieves competitive forecasting while enabling interpretable structural readout of learned relations.
This paper introduces generic triple-latent recurrent models that compress token pair interactions into a latent state, and a gated associative retrieval variant that improves exact recall. The hybrid model outperforms Transformers on byte-level WikiText-2 and a tokenized language benchmark, achieving up to 41.9% associative recall versus 25%.
WriteSAE introduces the first sparse autoencoder that decomposes matrix cache writes in state-space and hybrid recurrent language models, enabling superior token-level interventions compared to existing methods.
This paper argues that robust state tracking in recurrent models depends on error control dynamics rather than just expressive capacity, proving that affine recurrent networks suffer from accumulating errors that limit their effective horizon.