built a token-budget-aware context orchestration for long-horizon LLM agents

Reddit r/ArtificialInteligence Tools
context-orchestration llm-agents token-budget hybrid-retrieval open-source

Summary

ContextOS is an open-source token-budget-aware context orchestration layer for long-horizon LLM agents, using hybrid retrieval and deterministic planning to select memories for context with execution traces for inspection.

I built ContextOS, an open-source, token-budget-aware context orchestration layer for long-horizon LLM agents. The idea is that retrieval and context selection are different problems. ContextOS uses hybrid retrieval (dense + BM25), RRF fusion, cross-encoder reranking, and deterministic token-budget-aware planning to decide which memories actually make it into the model's context. It also records an execution trace for each decision, so you can inspect why a memory was selected or rejected, how it ranked at each stage, and how much of the context budget it consumed. I built an evaluation harness and an interactive demo to visualize the whole pipeline. GitHub: https://github.com/ayeangad/contextos
Original Article

Similar Articles

TokenPilot: Cache-Efficient Context Management for LLM Agents

Hugging Face Daily Papers

TokenPilot is a dual-granularity context management framework that reduces inference costs in long-horizon LLM sessions by stabilizing prompt prefixes and conservatively managing context segments, achieving 61-87% cost reduction on benchmarks while maintaining competitive performance.

@omarsar0: // The Efficiency Frontier // Cool paper on context management. As agents reuse the same documents and histories across…

X AI KOLs Following

This paper introduces The Efficiency Frontier, a unified framework for cost–performance optimization in LLM context management that models context strategy selection as a deployment-aware optimization problem, achieving 25% reduction in token usage and over 50% lower token cost with amortized memory compression compared to full-context prompting.

Context Recycling for Long-Horizon LLM Inference

arXiv cs.CL

This paper introduces ContextForge, a hierarchical memory architecture that treats the LLM context window as a recyclable workspace, achieving significant token and speed improvements on long-horizon tasks while maintaining accuracy on a 276-million-row enterprise benchmark.