Tag
DoubtLess 提出一种免训练的对比解码方法，用一个更小的推理模型引导大模型输出更简洁的思维链，在七个竞赛数学、研究生科学与编程基准上将 Qwen3-32B 的 CoT 缩短 31% 且准确率不变，优于五个推理时基线。
Apple researchers' paper 'Decoding Looped Transformers Better for (Almost) Free' shows that reusing discarded intermediate inference states via Contrastive Decoding cuts compute consumption by 47% and boosts math pass@1 from 61% to 73%, with zero training cost — a shift toward algorithmic fine-tuning over massive hardware scaling.
The paper introduces LoopCD, a training-free contrastive decoding framework for looped transformers that uses intermediate recurrent states as weak/strong prediction pairs, delivering substantial gains (e.g., Ouro-2.6B-Thinking's AIME 2024 pass@1 rising from 61.88% to 73.33%) while enabling halving of recurrent loops and 22.5%-48.2% reductions in forward FLOPs.
This paper introduces Decoupled Contrastive Decoding (DCD), which uses an expert-aligned lightweight proposer for speculative decoding while keeping the contrastive signal only in verification, achieving speedups over vanilla contrastive decoding without degrading output distribution.
Proposes three attention-guided strategies for layer selection in contrastive decoding for large language models, improving factuality on TruthfulQA over the DoLa baseline.
Introduces Contrastive Weak-to-Strong Generalization (ConG), a framework that uses contrastive decoding to generate higher-quality samples from weak models for more reliable weak-to-strong generalization in LLMs, demonstrating consistent improvements across model families.
This paper introduces Contrastive Decoding Diffing (CDD), a method that recovers verbatim finetuning data from LLMs using only logit access, without requiring model weights or activations. It demonstrates high verbatim recovery across multiple model families and reveals unintended data leakage, such as the recurring fictional persona 'Dr. Elena Rodriguez' from synthetic training data.
Proposes Dual-Confidence Contrastive Decoding (DCCD), a training-free method for retrieval-augmented generation that handles intra-context conflicts in multi-document settings by combining document-level and token-level confidence signals, and introduces the DRQA benchmark for factual-conflict QA.
The paper generalizes contrastive decoding to a conflict-aware paradigm that dynamically allocates authority between external context and parametric priors, proposes the TriState-Bench evaluation protocol, and introduces Adaptive Regime Routing (ARR) to resolve asymmetry between correction and resistance.
This paper presents PCNet, a probabilistic circuit trained as a tractable density estimator on LLM residual streams to detect hallucinations as geometric anomalies. It also introduces PC-LDCD, a dynamic correction method that only intervenes on hallucinated tokens, achieving near-perfect detection and reduced corruption rates.