token-compression

OmniRoute is a free, open-source AI gateway that unifies access to 268+ providers and 500+ models through a single local endpoint, featuring smart routing, token compression, and free tier aggregation to reduce costs and complexity for developers.

OmniFocus is a training-free, query-guided token compression method for omni-modal LLMs that independently estimates importance for video and audio to preserve modality-specific evidence while maintaining alignment, achieving significant inference speedups with minimal accuracy loss at low token retention ratios.

Introduces SaMer, an object-aware token merging framework that compresses image-side tokens for vision-language retrieval while preserving object-level evidence, achieving significant storage reduction and improved retrieval performance.

An open-source, self-hosted AI gateway providing a single endpoint for 237 LLM providers with auto-fallback, token compression, and routing. It has gained significant traction with 9.8K GitHub stars and 280+ contributors in 4.5 months.

OmniRoute is a free and open-source AI gateway that unifies 236 AI provider APIs into a single OpenAI-compatible interface, supporting self-hosting, automatic fallback, intelligent routing, and token compression.

OmniRoute is a trending GitHub tool that compresses AI prompts to reduce token usage by up to 95% and offers 1.6 billion free tokens per month by seamlessly routing requests across multiple providers like Claude Code, Codex, Cursor, Cline, and Copilot.

AVOC introduces a retrieval-inspired token compression method for omni-modal LLMs that effectively handles hour-long audio-video inputs by selecting informative tokens based on relevance, importance, and diversity. The framework achieves state-of-the-art results on long-form audio-video understanding benchmarks, surpassing prior methods by significant margins.

Netflix engineers open-sourced the Headroom tool, which automatically compresses LLM input context during local preprocessing, reducing token consumption by up to 95%. It is compatible with mainstream AI coding tools like Codex and Cursor, and works without any code modifications.

Headroom is a context compression layer that cuts AI agent token costs by 60–95%, supports a zero-code-change proxy mode, and does not degrade model response quality.

This article critiques RTK, a token compression tool for LLM agents, arguing that its promised 60-90% cost savings are misleading, it introduces silent failure risks, lacks rigorous accuracy benchmarks, and is structurally fragile as a standalone product.

A Netflix engineer built Headroom, an open-source proxy that compresses LLM context by 60-95% with no code changes and negligible accuracy loss. It supports major AI agents and is available on GitHub under Apache 2.0.

This paper introduces the Call Playbook dataset for classifying real-world B2B conversations and proposes methods to distill examples into compact, interpretable task instructions, achieving 99% token reduction and up to 7% AUC improvement over traditional in-context learning.

LLaVA-OneVision-2 introduces codec-stream tokenization for efficient video understanding, significantly outperforming Qwen3-VL-8B on temporal and spatial benchmarks. The model, data, and code are open-sourced.

Headroom is a GitHub tool by Netflix's Tejas Chopra that compresses inputs (tool outputs, logs, RAG chunks, etc.) before sending to an LLM, promising 60–95% fewer tokens without changing answers. It supports Python/TypeScript libraries, a local proxy, an MCP server, and wrappers for popular coding agents.

Snapcompact is a technique that renders text into dense pixel-font images to replace text tokens with cheaper image tokens, achieving near-verbatim recall at a fraction of the input cost.

InfiniteKV is an open-source KV cache technique that compresses old tokens into 104-byte searchable records stored in RAM or on disk, enabling models to handle million-token contexts beyond their trained window without discarding data. Verified working with Mistral-7B and SmolLM2.

HiLo-Token introduces an input-adaptive token compression framework for Diffusion Transformers that allocates more tokens to high-frequency regions, achieving up to 3.13x speedup in image editing tasks without quality loss.

Introduces Latent Context Language Models (LCLMs), which encode 16 tokens as 1 latent token to improve performance, speed, and memory usage.

Latent Memory introduces a compressed representation approach for external memory in question answering, reducing token consumption and storage requirements while maintaining competitive performance across text-only and multimodal benchmarks.

Headroom is an open-source tool that compresses token usage in code search results and AI conversations by up to 92% (e.g., from 17k to 1,400 tokens) while maintaining answer quality. It supports multiple platforms and runs locally for free.