OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models

Hugging Face Daily Papers Papers
token-compression omnimodal-llm efficiency inference-acceleration training-free audio-video

Summary

OmniScope is a training-free token compression framework for omnimodal LLMs that estimates audio and video relevance separately using the query as a shared anchor, achieving up to 3.53x prefill speedup and over 15% GPU memory reduction with minimal accuracy loss.

Existing token compression methods for omnimodal large language models typically rely on one modality to determine what to retain in the other. We show that this assumption often breaks down: for the same query, audio and video relevance often peaks at different moments. This cross-modal salience mismatch makes unidirectional guidance prone to discarding answer-critical cues under aggressive compression. We propose OmniScope, a training-free token compression framework that uses the query as a shared semantic anchor while estimating relevance separately for audio and video. OmniScope allocates modality-specific token budgets, prunes visual tokens with an anchor-delta strategy that preserves both global context and temporal changes, and merges audio tokens within each second to reduce redundancy while maintaining temporal continuity. Across four audio-video benchmarks and two Qwen2.5-Omni model scales, OmniScope achieves the best average accuracy across all compression settings. At 25% overall token retention, it delivers up to 3.53x prefill speedup and more than 15% GPU memory reduction, with only a 0.35-point drop in average accuracy. These results suggest a simple design principle for OmniLLM inference: share the query across modalities, but not the salience estimates. The code is available at https://github.com/MAC-AutoML/OmniScope.
Original Article
View Cached Full Text

Cached at: 07/31/26, 09:56 PM

Paper page - OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models

Source: https://huggingface.co/papers/2607.23193

Abstract

Existingtokencompressionmethodsforomnimodallargelanguagemodelstypicallyrelyononemodalitytodeterminewhattoretainintheother.Weshowthatthisassumptionoftenbreaksdown:forthesamequery,audioandvideorelevanceoftenpeaksatdifferentmoments.Thiscross-modalsaliencemismatchmakesunidirectionalguidancepronetodiscardinganswer-criticalcuesunderaggressivecompression.WeproposeOmniScope,atraining-freetokencompressionframeworkthatusesthequeryasasharedsemanticanchorwhileestimatingrelevanceseparatelyforaudioandvideo.OmniScopeallocatesmodality-specifictokenbudgets,prunesvisualtokenswithananchor-deltastrategythatpreservesbothglobalcontextandtemporalchanges,andmergesaudiotokenswithineachsecondtoreduceredundancywhilemaintainingtemporalcontinuity.Acrossfouraudio-videobenchmarksandtwoQwen2.5-Omnimodelscales,OmniScopeachievesthebestaverageaccuracyacrossallcompressionsettings.At25%overalltokenretention,itdeliversupto3.53xprefillspeedupandmorethan15%GPUmemoryreduction,withonlya0.35-pointdropinaverageaccuracy.TheseresultssuggestasimpledesignprincipleforOmniLLMinference:sharethequeryacrossmodalities,butnotthesalienceestimates.Thecodeisavailableathttps://github.com/MAC-AutoML/OmniScope.

View arXiv pageView PDFGitHub3Add to collection

Get this paper in your agent:

hf papers read 2607\.23193

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.23193 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.23193 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.23193 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles