@lateinteraction: Agents often externalize some context: a repository in coding agents, a corpus in RAG, and the user prompt in an RLM. N…
Summary
New research by Joshua Gu shows that AI agents perform better when they manage a small buffer in their context window as a cache for external context, challenging the common practice of pushing context entirely out of the prompt.
View Cached Full Text
Cached at: 05/21/26, 10:14 AM
Agents often externalize some context: a repository in coding agents, a corpus in RAG, and the user prompt in an RLM.
New work by @astrogu_ shows that agents work better if they’re allowed to manage a small buffer in their context window as a “cache” for that external context.
Joshua Gu (@astrogu_): Recent agentic systems (Claude Code, Codex, RLM, etc.) push context out of the prompt and into the environment (e.g., as files). This helps them maintain long-term knowledge about their goals and functionality.
🚨 While this is a good idea, we show a surprising result: systems
Similar Articles
@astrogu_: Recent agentic systems (Claude Code, Codex, RLM, etc.) push context out of the prompt and into the environment (e.g., a…
PEEK introduces a context map as an orientation cache for LLM agents, enabling them to maintain persistent knowledge about external environments like code repositories. The system achieves 6.3-34.0% quality improvements at lower cost compared to baselines like RAG and ACE.
How we made an AI agent faster by moving stable context out of the prompt
Describes a technique to improve AI agent speed by moving stable context out of the prompt, reducing token usage and latency.
Are bigger context windows actually the wrong direction for agents?
The author questions whether the focus on expanding context windows for AI agents is counterproductive, arguing that accumulated junk slows down long sessions and suggests keeping working context small with external memory.
should agents ask for user context up front or learn it slowly?
A discussion on how AI agents should handle user context: upfront disclosure or gradual learning, with various existing approaches like project memory and chat summaries found lacking.
@GergelyOrosz: Trying to put my finger on how the more context you use in a context window (I called it context depth), over longer ru…
Gergely Orosz notes that using more context in an AI agent's context window over longer runs can increase errors and drift, suggesting shorter runs with less context for better reliability.