An AI4AI Framework for Visual Token Pruning
Summary
AutoPrune is a training-free framework that uses LLMs to automatically design visual-token pruning policies for multimodal LLMs via a domain-specific language and residual search, achieving high efficiency with minimal performance loss (99% performance retained while removing 94.4% of visual tokens).
View Cached Full Text
Cached at: 08/14/26, 07:26 AM
Paper page - An AI4AI Framework for Visual Token Pruning
Source: https://huggingface.co/papers/2608.07193
Abstract
AutoPrune uses large language models to automatically design visual-token pruning policies for multimodal models via a domain-specific language and residual search formulation, achieving high efficiency with minimal performance loss.
Visual-token pruningcan substantially reduce the inference cost ofmultimodal large language models(MLLMs), yet existing methods largely rely on fixed, handcrafted heuristics and costly expert trial and error. As pruning objectives, budgets, and model architectures diversify, manually navigating the expanding design space becomes increasingly difficult. This paper aims to build an AI4AI framework forvisual-token pruningby addressing a natural question: Can large language models automatically design effective visual-token reduction algorithms? Although LLMs possess broad algorithmic knowledge and strong reasoning capabilities, translating such general knowledge into effective solutions for a specialized task remains nontrivial. We argue that the key lies in designing an appropriate search-state representation that connects the internal knowledge of LLMs with the structural requirements and constraints ofvisual-token pruning. Based on this insight, we proposeAutoPrune, a training-free framework for LLM-drivenvisual-token pruningpolicy design. At its core,AutoPruneintroduces aToken Pruning Domain-Specific Language(TPDSL) comprising 131 reusable atoms for budget control, token scoring, selection constraints, and token reassembly. A key property ofTPDSLis that it represents each search state as aresidual modificationof a strongbase policy. This residual formulation narrows the search space and directs the LLM’s attention toward the policy components that are most consequential for performance. Experiments on 14 multimodal benchmarks and three MLLM backbones demonstrate the effectiveness, efficiency, and transferability ofAutoPrune. Even when removing 94.4% of visual tokens,AutoPrunepreserves more than 99% of full-token performance while reducingFLOPsby 9.9x andprefill latencyby 6.4x.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2608\.07193
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.07193 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.07193 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.07193 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Learning to Predict Middle-Layer Attention in MLLMs for Visual Token Prunin
This paper introduces a method to predict middle-layer attention in multimodal LLMs to prune visual tokens efficiently, using question-contrastive teacher selection and cross-modal attention distillation.
Researchers let AI Agents Optimize LLM Reasoning and Cut Tokens by 70%
Researchers developed AutoTTS, a framework where AI agents automatically design control policies to optimize LLM inference, cutting token consumption by approximately 70% while maintaining high reasoning accuracy.
Optimizing Korean-Centric LLMs via Token Pruning
This paper presents a systematic benchmark of token pruning—a compression technique that removes tokens and embeddings for irrelevant languages—applied to Korean-centric LLM tasks. The study evaluates popular multilingual models (Qwen3, Gemma-3, Llama-3, Aya) across different vocabulary configurations and finds that token pruning significantly improves generation stability and reduces memory footprint for domain-specific deployments.
Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training
ART (Art-based Reinforcement Training) enables parameter-efficient fine-tuning of frozen multimodal LLMs by optimizing raw visual input via gradient backpropagation, achieving performance comparable to LoRA while supporting pre-compiled computational graphs for high-throughput engines like vLLM.
Small LLMs: Pruning vs. Training from Scratch
This paper empirically compares pruning vs. training small language models from scratch, finding that pruning provides a strong advantage under limited token budgets but that the advantage diminishes as training scales, especially with coarse pruning.