Tag
SHiPPO extends HiPPO by transporting polynomial projection coefficients into a moving channel frame, enabling selective state-space models to recover order-sensitive memory signals. The paper provides theoretical foundations and diagnostics supporting its transported-memory prior.
This academic paper introduces finite-lag operator geometry for analyzing recurrent neural network hidden states, deriving a source-centered transport tensor and antisymmetric coordinate circulation to capture directed flow and deterministic recurrent motion beyond static snapshots.
This paper introduces StateFlow, a recurrent forecasting framework that extends the Variability-Aware Recursive Neural Network (VARNN) to long-horizon multivariate time series forecasting by using a dual-state recurrent backbone and a chunk-based decoder, achieving competitive performance against strong baselines.
This work proposes orthogonalizing the memory matrix of mLSTM recurrent models to improve their performance on noisy associative recall tasks. Experiments show that using Newton-Schulz iterations for read-only orthogonalization enhances validation accuracy compared to baseline mLSTM.
An update on Matrix Recurrent Units (MRU), a linear-time attention alternative. The author explores methods to stabilize training, finding that orthogonal matrices underperform while LDU factorization works best, and shows MRU underperforms transformers on larger datasets like TinyStories.
This paper studies functional redundancy in recurrent neural networks by using ordered real Schur coordinates to identify structured ablations that preserve task performance, finding that task-restricted symmetries vary across tasks and trained solutions.
This paper investigates memory-efficient meta-reinforcement learning architectures for adaptive safety-critical control in adversarial spacecraft proximity operations, finding that state space models like Mamba with PPO achieve superior task completion, safety, and fuel savings compared to LSTM and GRU.
This paper investigates how action information can be incorporated into recurrent neural network architectures for reinforcement learning, examining design choices and empirically evaluating them across illustrative domains.
This paper develops a local theory of gradient descent near bifurcations in dynamical models, showing that the state-space neural tangent kernel collapses to a rank-one operator that dominates learning dynamics, making optimization effectively low-dimensional and predictable from normal forms.
This paper investigates parallel-in-time algorithms for training recurrent neural networks in dynamical systems reconstruction, proposing GTF-DEER that enables stable learning over long sequences and improves reconstruction accuracy.
This paper presents Olmo Hybrid, a 7B-parameter language model that combines attention and Gated DeltaNet recurrent layers, demonstrating both theoretical and empirical advantages over pure transformers. The work shows that hybrid models have greater expressivity, scale more efficiently during pretraining, and outperform comparable transformer baselines.