Tag
FutureBridge introduces a token reranker for collaborative decoding that ranks LLM-SLM candidates based on how well the SLM can continue reasoning from them, improving the Qwen3-1.7B SLM's math accuracy by 35.1% over greedy decoding.
HALO introduces a hybrid adaptive latent refinement method for frozen language models that selectively applies second-stage refinement to a subset of tokens, achieving better performance than fixed refinement steps while using less compute.
PartRep proposes a selective prompt repetition method for decoder-only LLMs that appends only the most informative tokens (selected via NLL) instead of the full prompt, reducing KV cache and prefill FLOPs while retaining most of the accuracy gains across multiple benchmarks.
SEAD introduces a competence-aware on-policy distillation method that uses entropy to guide supervision at token, temporal, and prompt levels, achieving a +4.8 average accuracy improvement on OLMo-3 across six math benchmarks.
Proposes Distribution-Aligned Self-Distillation (DASD), which dynamically filters tokens during self-distillation to preserve beneficial logical corrections while suppressing distributionally misaligned style noise, improving robust reasoning on math, code, and commonsense benchmarks.
This paper introduces a two-stage token selection framework for visual geometry transformers that reduces computational costs by restricting key/value tokens during global attention, achieving over 85% acceleration on scenes with 500 images while maintaining baseline performance.
SEATS is a training-free, stage-adaptive token selection method that reduces computational overhead in omni-modal LLMs by progressively pruning redundant visual and audio tokens, achieving a 9.3x FLOPs reduction and 4.8x prefill speedup while preserving 96.3% performance.