Tag
提出 CoTS（Competitive Token Selection）训练免调框架，在推理过程中利用 top-2 候选 token 的概率竞争来连续调节离散 token 提交与软表示探索之间的平衡，从而在六个推理基准上超越标准 CoT 和现有训练免调隐式推理方法。
该论文从蒸馏目标本身推导 token 重要性（基于局部 KL 预算下的损失可降幅度，即 logit 梯度与 Fisher 矩阵构成的效用分数），并实例化为 Fisher Utility Weighting（FUW），在七个数学推理基准和三组 Qwen3 师生模型上相对均匀 OPD 提升 1.08–4.0 个宏平均百分点。
The paper studies which token positions in natural language autoencoders are most informative to explain, finding that chat-structure-based ranking selects relevant explanations better than computational signals, and that explaining only 5% of positions retains nearly all audit success on prompt injection and concealment tasks.
论文提出 DoSA（Dots Sparse Attention），一种无损稀疏注意力方法，通过阈值证书使 indexer 打分计算变为条件式执行，同时保持精确 token 选择；GPU 实现较 SGLang 加速最多 2.74 倍，并在 GLM-5.2 和 DeepSeek-V3.2 上实现最高 1.94 倍的首 token 时间加速。
The paper demonstrates that using a shared learning rate as a control in selective on-policy distillation experiments is not neutral, leading to varying performance and conclusions, and advocates for reporting full learning-rate matrix comparisons for fair evaluation.
FutureBridge introduces a token reranker for collaborative decoding that ranks LLM-SLM candidates based on how well the SLM can continue reasoning from them, improving the Qwen3-1.7B SLM's math accuracy by 35.1% over greedy decoding.
HALO introduces a hybrid adaptive latent refinement method for frozen language models that selectively applies second-stage refinement to a subset of tokens, achieving better performance than fixed refinement steps while using less compute.
PartRep proposes a selective prompt repetition method for decoder-only LLMs that appends only the most informative tokens (selected via NLL) instead of the full prompt, reducing KV cache and prefill FLOPs while retaining most of the accuracy gains across multiple benchmarks.
SEAD introduces a competence-aware on-policy distillation method that uses entropy to guide supervision at token, temporal, and prompt levels, achieving a +4.8 average accuracy improvement on OLMo-3 across six math benchmarks.
Proposes Distribution-Aligned Self-Distillation (DASD), which dynamically filters tokens during self-distillation to preserve beneficial logical corrections while suppressing distributionally misaligned style noise, improving robust reasoning on math, code, and commonsense benchmarks.
This paper introduces a two-stage token selection framework for visual geometry transformers that reduces computational costs by restricting key/value tokens during global attention, achieving over 85% acceleration on scenes with 500 images while maintaining baseline performance.
SEATS is a training-free, stage-adaptive token selection method that reduces computational overhead in omni-modal LLMs by progressively pruning redundant visual and audio tokens, achieving a 9.3x FLOPs reduction and 4.8x prefill speedup while preserving 96.3% performance.