Beyond Selection: Token Parameterization for Extreme Visual Token Compression

Hugging Face Daily Papers Papers
visual-token-compression vision-language-models efficiency research braco compression-algorithm

Summary

The paper introduces Braco, a lightweight token parameterization method for visual token compression in vision-language models, achieving significant efficiency gains and accuracy under extreme compression budgets.

Visual-token compression is effective for improving the efficiency of vision-language models, but under extreme compression budgets, token pruning can break visual grounding while learned resamplers increase parameter count, attention cost, and training complexity. We revisit compression through a token parameterization lens, separating (i) basis transformation and structured truncation (retained subspace/compressibility) from (ii) coordinate organization (optimization and cross-modal alignment). This view yields two coupled objectives, compressibility and learnability, which we formalize as unified functionals. Guided by these objectives, we design Braco, a lightweight four-step coder that combines transform-basis truncation, input-independent basis-coordinate embeddings, budget-dependent orthogonal re-parameterization, and learned spatial residual tokens from lightweight pooling. Experiments show that Braco forms the favorable empirical accuracy-efficiency frontier under 23times--64times compression and remains competitive at 144times, reaching 95.2% accuracy while reducing prefill FLOPs by 84.2%--86.7% relative to the uncompressed upper bound. Against prior methods, Braco matches or improves accuracy while achieving up to approximately 36% end-to-end speedup and using 16.6times/78.8times lower compressor latency/FLOPs.
Original Article
View Cached Full Text

Cached at: 09/30/26, 04:17 AM

Paper page - Beyond Selection: Token Parameterization for Extreme Visual Token Compression

Source: https://huggingface.co/papers/2609.35232

Abstract

Visual-tokencompressioniseffectiveforimprovingtheefficiencyofvision-languagemodels,butunderextremecompressionbudgets,tokenpruningcanbreakvisualgroundingwhilelearnedresamplersincreaseparametercount,attentioncost,andtrainingcomplexity.Werevisitcompressionthroughatokenparameterizationlens,separating(i)basistransformationandstructuredtruncation(retainedsubspace/compressibility)from(ii)coordinateorganization(optimizationandcross-modalalignment).Thisviewyieldstwocoupledobjectives,compressibilityandlearnability,whichweformalizeasunifiedfunctionals.Guidedbytheseobjectives,wedesignBraco,alightweightfour-stepcoderthatcombinestransform-basistruncation,input-independentbasis-coordinateembeddings,budget-dependentorthogonalre-parameterization,andlearnedspatialresidualtokensfromlightweightpooling.ExperimentsshowthatBracoformsthefavorableempiricalaccuracy-efficiencyfrontierunder23times--64timescompressionandremainscompetitiveat144times,reaching95.2%accuracywhilereducingprefillFLOPsby84.2%--86.7%relativetotheuncompressedupperbound.Againstpriormethods,Bracomatchesorimprovesaccuracywhileachievinguptoapproximately36%end-to-endspeedupandusing16.6times/78.8timeslowercompressorlatency/FLOPs.

View arXiv pageView PDFProject pageGitHub1Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.35232 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.35232 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.35232 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Compute Optimal Tokenization (2 minute read)

TLDR AI

This paper systematically derives compression-aware neural scaling laws by training nearly 1,300 models, demonstrating that the widely used heuristic of 20 tokens per parameter is an artifact of specific tokenizers. The authors propose a tokenizer-agnostic scaling law based on bytes, offering a new framework for compute-efficient training across diverse languages and modalities.