Tag
本文提出 Sep-Latent，一种自适应隐式推理框架，利用分隔符 token 作为语义锚点并通过注意力衰减机制压缩推理逻辑，在 GSM8k-Aug 上以仅 5.4 个 token 达到 56.8% 准确率，显著优于固定长度的隐式推理模型。
提出 FramePack-U，一种免训练的常量内存帧打包框架，可将任意长度视频压缩为固定大小的视觉表示，直接插入现成 VLM 推理，在 Long VideoBench、MLVU 和 Video-MME 上显著超越全分辨率基线及现有 token 压缩方法。
APEX 提出异步前缀去噪方法，结合分层潜在空间，使单个 token 可同时拥有多个扩散时间步，在极高压缩比下（64 token 超越 256 token 基线）实现更优的图像生成质量。
KeyRec is a training-free framework for building bounded visual memory in vision-language models, using a recent-observation cache plus a structured event bank to compress long/streaming video context to just 10% of dense visual tokens while setting state-of-the-art results on 13 of 15 streaming and long-video benchmark settings.
Vision-RL² is a method for improving fine-grained perception in multimodal large language models by using region-level reinforcement learning to compress visual tokens and enhance performance across multiple benchmarks without full model fine-tuning.
This survey examines inference-efficiency techniques for video large language models, analyzing cost reductions across frame sampling, encoding, token compression, and language model stages while identifying evaluation gaps.
This paper introduces PAYN, a plug-and-play token compression method for MLLM-based referring expression segmentation that relies solely on position information, outperforming existing techniques by preserving spatial relational consistency.
Tevatron-Elastic presents a unified abstraction for training elastic retrievers and rerankers, enabling a single checkpoint to serve multiple model sizes across depth, token, and width axes. It generalizes prior methods like Matryoshka embeddings and early exit, and introduces Matryoshka LTC for joint token-compression training.
OmniPack proposes a training-free token compression framework for omni-modal LLMs, combining structural pre-LLM compression with task-relevant inner-LLM semantic refinement, achieving strong performance-efficiency trade-offs on multiple benchmarks.
3DZip is a three-stage token compression framework for 3D vision-language models that uses voxelization, diversity-guided anchor selection, and spatial constraint merging to reduce tokens while preserving spatial reasoning. It retains 94.7% of original performance with only 128 tokens and achieves 1.92x faster inference on 3DQA benchmarks.
JetBrains benchmarked the Caveman token-compression skill on Claude Code across 86 tasks, finding real output-token savings of about 8.5% (not the advertised 65%) with no detectable degradation in task quality.
OmniScope is a training-free token compression framework for omnimodal LLMs that estimates audio and video relevance separately using the query as a shared anchor, achieving up to 3.53x prefill speedup and over 15% GPU memory reduction with minimal accuracy loss.
OmniDelta proposes a training-free, skill-driven framework for allocating token budgets across audio and video modalities in OmniLLMs, improving accuracy-efficiency trade-offs with 22% GPU memory reduction and 1.64x speedup at 25% token retention.
OmniRoute is a free, open-source AI gateway that unifies access to 268+ providers and 500+ models through a single local endpoint, featuring smart routing, token compression, and free tier aggregation to reduce costs and complexity for developers.
OmniFocus is a training-free, query-guided token compression method for omni-modal LLMs that independently estimates importance for video and audio to preserve modality-specific evidence while maintaining alignment, achieving significant inference speedups with minimal accuracy loss at low token retention ratios.
Introduces SaMer, an object-aware token merging framework that compresses image-side tokens for vision-language retrieval while preserving object-level evidence, achieving significant storage reduction and improved retrieval performance.
An open-source, self-hosted AI gateway providing a single endpoint for 237 LLM providers with auto-fallback, token compression, and routing. It has gained significant traction with 9.8K GitHub stars and 280+ contributors in 4.5 months.
OmniRoute is a free and open-source AI gateway that unifies 236 AI provider APIs into a single OpenAI-compatible interface, supporting self-hosting, automatic fallback, intelligent routing, and token compression.
OmniRoute is a trending GitHub tool that compresses AI prompts to reduce token usage by up to 95% and offers 1.6 billion free tokens per month by seamlessly routing requests across multiple providers like Claude Code, Codex, Cursor, Cline, and Copilot.
AVOC introduces a retrieval-inspired token compression method for omni-modal LLMs that effectively handles hour-long audio-video inputs by selecting informative tokens based on relevance, importance, and diversity. The framework achieves state-of-the-art results on long-form audio-video understanding benchmarks, surpassing prior methods by significant margins.