attention-distillation

Tag

Cards List
#attention-distillation

Learning to Predict Middle-Layer Attention in MLLMs for Visual Token Prunin

arXiv cs.AI · yesterday Cached

This paper introduces a method to predict middle-layer attention in multimodal LLMs to prune visual tokens efficiently, using question-contrastive teacher selection and cross-modal attention distillation.

0 favorites 0 likes
← Back to home

Submit Feedback