Tag
This paper investigates when state adaptation during inference matters for masked diffusion language models (MDMs), organizing inference into five axes and showing that selective adaptation (using lightweight detectors to identify high-opportunity states) captures a large share of oracle gains, e.g., 56.9% of opportunity while adapting only the top 10% of states on LLaDA-8B constrained JSON filling.
This paper systematically evaluates uncertainty-aware decoding with rollback mechanisms to improve code generation in large language models, demonstrating performance gains over standard methods using uncertainty signals.
Researchers at MIT CSAIL and Harvard used a modified Battleship game to study and improve language models' question-asking abilities. By applying Monte Carlo inference strategies, they significantly boosted smaller models like Llama 4 Scout's win rate from 8% to 82% against humans, outperforming larger models at a fraction of the cost.