Tag
A 2B vision-language model distilled from a tool-using agent achieves first-place performance on long egocentric video question answering by pruning its multilingual embedding table to meet parameter limits, reaching 89% accuracy of the larger pipeline with only 1.1% of parameters.
This paper introduces a compound sparsity framework for LLMs that combines static parameter pruning with dynamic token-level computation, showing that mixing both mechanisms outperforms single-dimension compression and delays performance degradation.