Rethinking State Tracking in Recurrent Models Through Error Control Dynamics
Summary
This paper argues that robust state tracking in recurrent models depends on error control dynamics rather than just expressive capacity, proving that affine recurrent networks suffer from accumulating errors that limit their effective horizon.
View Cached Full Text
Cached at: 05/11/26, 07:18 AM
Paper page - Rethinking State Tracking in Recurrent Models Through Error Control Dynamics
Source: https://huggingface.co/papers/2605.07755
Abstract
Affine recurrent networks cannot correct errors in state tracking once state representations are preserved, leading to finite horizon solutions governed by accumulated error rather than robust tracking.
The theory ofstate trackinginrecurrent architectureshas predominantly focused onexpressive capacity: whether a fixed architecture can theoretically realize a set of symbolic transition rules. We argue that equally important iserror control, the dynamics governinghidden-state driftalong the directions that distinguish symbolic states. We prove thataffine recurrent networks, a class of models encompassingState-Space ModelsandLinear Attention, cannot correct errors alongstate-separating subspacesonce they preserve state representations. Consequently, practical affine trackers do not learn robuststate tracking; rather, they learnfinite horizon solutionsgoverned by accumulated state-relevant error. We characterize the mechanics of this failure, showing that tracking remains readable only while the accumulating within-class spread remains small relative to the initial between-class separation. We demonstrate empirically on group state-tracking tasks that this breakdown is predictable: tracking collapses when thedistinguishability ratiocrosses thereadability thresholdof the trained decoder. Across trained models, the point of this crossing predicts the horizon at which downstream accuracy fails. These results establish that robuststate trackingis determined not only by an architecture’s theoretical expressivity but crucially by itserror control.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2605\.07755
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.07755 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.07755 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.07755 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Think Shallow, Solve Deep: Controlling Recurrent Dynamics for Reliable Test-Time Depth
This paper introduces a method to control recurrent dynamics in neural networks for reliable test-time depth, analyzing dynamical regimes like settling, marginal, or drifting to improve performance on algorithmic tasks such as Sudoku and carry propagation.
Anticipatory Reinforcement Learning for Trajectory Tracking
This paper introduces a predictive formulation for deep reinforcement learning that augments the state space with future reference horizons to enable anticipatory control for trajectory tracking. Simulation results show significant error reduction, though zero-shot transfer to physical hardware reveals a sim-to-real gap.
Latent States in Neural Networks: Recovering the Temporal Structure of Drifting Data from Model Weights
This paper investigates whether temporally drifting data streams can be partitioned into discrete regimes by fitting a hidden Markov model to the trajectory of neural network weights trained on successive time windows, showing that recovered latent states correlate with transfer performance across two datasets.
Time-Varying Deep State Space Models for Sequences with Switching Dynamics
The paper proposes a class of time-varying deep state-space models where dynamics are learned via a basis function expansion, enabling adaptive modeling of switching systems. The approach outperforms time-invariant counterparts on synthetic switching data and a speech denoising task.
StateFlow: Dual-State Recurrent Modeling for Long-Horizon Time Series Forecasting
This paper introduces StateFlow, a recurrent forecasting framework that extends the Variability-Aware Recursive Neural Network (VARNN) to long-horizon multivariate time series forecasting by using a dual-state recurrent backbone and a chunk-based decoder, achieving competitive performance against strong baselines.