Rethinking State Tracking in Recurrent Models Through Error Control Dynamics
Summary
This paper argues that robust state tracking in recurrent models depends on error control dynamics rather than just expressive capacity, proving that affine recurrent networks suffer from accumulating errors that limit their effective horizon.
View Cached Full Text
Cached at: 05/11/26, 07:18 AM
Paper page - Rethinking State Tracking in Recurrent Models Through Error Control Dynamics
Source: https://huggingface.co/papers/2605.07755
Abstract
Affine recurrent networks cannot correct errors in state tracking once state representations are preserved, leading to finite horizon solutions governed by accumulated error rather than robust tracking.
The theory ofstate trackinginrecurrent architectureshas predominantly focused onexpressive capacity: whether a fixed architecture can theoretically realize a set of symbolic transition rules. We argue that equally important iserror control, the dynamics governinghidden-state driftalong the directions that distinguish symbolic states. We prove thataffine recurrent networks, a class of models encompassingState-Space ModelsandLinear Attention, cannot correct errors alongstate-separating subspacesonce they preserve state representations. Consequently, practical affine trackers do not learn robuststate tracking; rather, they learnfinite horizon solutionsgoverned by accumulated state-relevant error. We characterize the mechanics of this failure, showing that tracking remains readable only while the accumulating within-class spread remains small relative to the initial between-class separation. We demonstrate empirically on group state-tracking tasks that this breakdown is predictable: tracking collapses when thedistinguishability ratiocrosses thereadability thresholdof the trained decoder. Across trained models, the point of this crossing predicts the horizon at which downstream accuracy fails. These results establish that robuststate trackingis determined not only by an architecture’s theoretical expressivity but crucially by itserror control.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2605\.07755
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.07755 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.07755 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.07755 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Time-Varying Deep State Space Models for Sequences with Switching Dynamics
The paper proposes a class of time-varying deep state-space models where dynamics are learned via a basis function expansion, enabling adaptive modeling of switching systems. The approach outperforms time-invariant counterparts on synthetic switching data and a speech denoising task.
When Dynamics Models Read the Wrong Time Steps: Label-Free Event Credit Re-Anchoring for Robust Global Readouts
The paper identifies 'temporal credit dilution' in learned dynamics models where global readouts focus on spurious correlates rather than brief physical events. It proposes CREST, a training-free method that re-anchors pooled representations using event core estimates, improving out-of-distribution robustness.
A Theory of Online Learning with Autoregressive Chain-of-Thought Reasoning
This academic paper develops a theoretical framework for online learning with autoregressive chain-of-thought reasoning, analyzing mistake bounds under end-to-end and trajectory supervision models.
R2R2: Robust Representation for Intensive Experience Reuse via Redundancy Reduction in Self-Predictive Learning
Proposes R2R2, a regularization method for self-predictive learning in reinforcement learning to mitigate overfitting under high update-to-data ratios, achieving significant improvements on continuous control tasks.
Neural Co-state Policies: Structuring Hidden States in Recurrent Reinforcement Learning
This paper introduces Neural Co-state Policies, establishing a formal link between recurrent reinforcement learning hidden states and the Pontryagin minimum principle to improve interpretability and robustness.