Tag
This paper provides causal evidence that gradient descent in matrix memories trained on group composition tasks recruits exactly the rank required by the task's algebra, correlating with the minimal faithful representation dimensions of finite groups.
This paper tests whether large language models can maintain exact state across long sequences of dependent tool calls by having them compute MD5 hashes step by step. The study demonstrates that with proper context and voting mechanisms, LLMs can achieve high success rates in such exact, multi-step procedures.
Google introduces SKILL.state, a method that reduces token usage in AI agents by 94% during long sessions by tracking structured state instead of conversation history, achieving high accuracy with efficient resource use.
The paper introduces 'recirculation', an inference-time architectural enhancement for foundation models that markedly reduces perplexity and boosts accuracy by improving state tracking, with no additional latency during generation.
This paper introduces StateMemBench, a benchmark for evaluating state tracking in LLM agent memory systems, and proposes StateMem, a method that improves current-state accuracy in long interactions.
This thread argues that standard transformers have a topological flaw: once a state representation reaches the top layer, they cannot update beliefs over time, causing collapse as depth increases.
This paper compares xLSTM, Mamba-2, and Gated DeltaNet on complex sequence modeling tasks and finds xLSTM superior due to its enhanced state tracking and memory dynamics, validated on synthetic length-generalization tasks.
Introduces VSTAT, a new benchmark to measure how well multimodal LLMs track states in videos, revealing that frontier models struggle with tasks humans find easy.
This paper demonstrates that extended chain-of-thought reasoning degrades performance on deterministic state-tracking tasks due to information-theoretic limits of decoder-only attention, and proposes tool delegation when the reasoning horizon exceeds a threshold.
The article discusses that the main challenge for AI agents in real-world workflows is not understanding the task, but handling recovery from unexpected changes, state tracking, and knowing when to ask for human input.
The author observes that the hardest part of phone-use AI agents is tracking state changes, as mobile interfaces have more dynamic and interruptive UI changes compared to desktop, and asks for others' experience.
This paper argues that robust state tracking in recurrent models depends on error control dynamics rather than just expressive capacity, proving that affine recurrent networks suffer from accumulating errors that limit their effective horizon.