AI Agents Don’t Need Bigger Context Windows

This repo treats context as a managed resource: compress, cache, retrieve, and share rather than a prompt that just keeps growing.

In 8 minutes, you’ll learn how Headroom cuts agent context by up to 90% without losing access to the original information.

Watch a coding agent debug a production issue.

It searches the repo. grep returns 1,000 results. It opens the logs. 50,000 tokens. It reads the stack trace. 10,000 tokens.

It needed 3 files, 1 stack frame, 20 log lines. About 800 tokens of signal.

It consumed 77,000 to get there.

This isn’t a model problem. The model would do fine with the right 800 tokens. The problem is nobody filtered the noise before it arrived.

Context windows keep getting bigger. But bigger windows don’t fix this. They just make it more expensive.

An agent with a 200K context window that fills 60% of it with irrelevant logs is not a smarter agent. It’s a slower, costlier one. Attention diluted across 77,000 tokens finds the right signal less reliably than attention over 800.

There’s an open-source project called Headroom that sits between your agent and the LLM and compresses that noise before it ever arrives. The architecture has one idea in particular worth understanding carefully.

Most compression is a one-way door

Every existing approach to this problem shares the same flaw: once you compress, the original is gone.

Summarize a tool output and discard it, the details are gone. Truncate a log file and the model can’t recover what was cut. Provider-native compaction collapses conversation history and that’s final.

This forces an impossible tradeoff. Compress aggressively and risk losing something important. Compress conservatively and leave most of the savings on the table.

Headroom’s core architecture, called CCR (Compress-Cache-Retrieve), eliminates this tradeoff by making compression reversible.

When content is compressed, the original is stored in a local cache with a content hash. The model receives the compressed version plus a retrieval tool called headroom_retrieve.

If the model needs the full data, it calls headroom_retrieve(hash=abc123) and gets the original back in 1ms.

Nothing is permanently lost. You’re not making a one-way decision anymore. The model decides when it needs more. Headroom just makes sure nothing is gone when it does.

Three compressors, one router

Headroom’s ContentRouter classifies every incoming piece of content and sends it to the right compressor.

SmartCrusher handles JSON.

A grep returning 1,000 file paths gets reduced to the 20 most relevant. It’s selecting based on task context, not just truncating by position.

CodeCompressor is AST-aware.

It strips docstrings, collapses function bodies to signatures, removes comment blocks. The structural skeleton stays. The prose the model doesn’t need goes.

Kompress-base is Headroom’s own HuggingFace model, trained specifically on agentic traces rather than general web text.

General-purpose compressors fail on structured logs and stack traces because they weren’t trained on that distribution. Kompress-base was.

CacheAligner

A fourth component, CacheAligner, stabilizes prompt prefixes so provider KV caches actually hit. If your prefix structure shifts slightly on every request, you’re busting your cache constantly and paying the cost in latency and compute. CacheAligner normalizes it.

The numbers from real workloads: code search 92% reduction. SRE incident debugging 92%. GitHub issue triage 73%. Accuracy on GSM8K, TruthfulQA, SQuAD, and BFCL preserved within measurement noise.

The feature nobody talks about: headroom learn

The most underappreciated thing in the repo is headroom learn.

The standard loop for improving agent behavior: watch it fail, diagnose the pattern, write a correction to CLAUDE.md or AGENTS.md, repeat. It works. It’s also entirely manual and depends on a human catching every failure.

headroom learn mines failed sessions automatically.

Sessions where the agent got stuck, produced wrong output, or required intervention. It extracts patterns from the failure traces and writes corrections directly to CLAUDE.md, AGENTS.md, or GEMINI.md.

Every agent failure becomes a potential improvement to the next session without requiring anyone to manually extract the lesson. The loop is: failure happens, pattern extracted, correction written, next session starts smarter.

Most teams are still running the slow version of this loop. This closes it.

Cross-agent memory

Headroom maintains a shared compressed memory store across agents.

When Claude Code, Codex, and Cursor are all working on the same codebase, they write to and read from the same store with automatic deduplication.

The token cost of context reconstruction is one of the largest hidden costs in multi-agent systems. Every agent that re-reads the same files and re-establishes the same context is burning compute that was already spent in an earlier session.

How to use it

Install:

Pick your path:

Zero code changes, run as a proxy:

Point any OpenAI-compatible client at the proxy instead of the provider. Nothing else changes.

Wrap a coding agent:

Compression, cross-agent memory, and headroom learn in one command.

Inline in your own pipeline:

Enable CCR via MCP:

This adds headroom_retrieve to the model’s tool set. Without this step, compression is still one-way.

What it doesn’t solve

CCR only works if the model recognizes something is missing. If the compressed version is plausible enough that the model doesn’t realize it needs more, important detail can be silently lost without retrieval being triggered.

Cache adds local memory overhead. The default TTL is 5 minutes with LRU eviction, fine for most sessions, a potential bottleneck in long-running or high-volume pipelines.

And it earns its keep on tool-heavy agentic workloads. Single-turn prompts with small, tight inputs won’t see meaningful gains. The noisier the tool outputs, the bigger the payoff.

Agent workloads increasingly consist of tool outputs rather than model-generated text. As those workloads grow, the challenge shifts from expanding context windows to deciding what deserves a place inside them.

The answer was never a bigger window. It was a smarter one.