inference-strategies

Tag

Cards List
#inference-strategies

Princeton/Meta evolved inference strategies for Qwen3 0.6B-4B: same accuracy as the best baseline at 39% lower cost, and the strategies transfer to GPT-5

Reddit r/LocalLLaMA ↗ · 5h ago

Princeton and Meta's paper shows a strategist agent can evolve inference strategies as code around small Qwen3 models (0.6B-4B), matching the best hand-built baseline's accuracy across 30 benchmarks at 39% lower inference cost, with strategies transferring unchanged to GPT-5 and Gemini 2.5.

0 favorites 0 likes
#inference-strategies

Unmask the State: When Does State Adaptation Matter for Masked Diffusion Language Models

Hugging Face Daily Papers ↗ · 2026-09-27 Cached

This paper investigates when state adaptation during inference matters for masked diffusion language models (MDMs), organizing inference into five axes and showing that selective adaptation (using lightweight detectors to identify high-opportunity states) captures a large share of oracle gains, e.g., 56.9% of opportunity while adapting only the top 10% of states on LLaDA-8B constrained JSON filling.

0 favorites 0 likes
#inference-strategies

Do Uncertainty Signals Help? A Systematic Study of Uncertainty-Aware Decoding with Rollback Mechanisms

arXiv cs.LG ↗ · 2026-08-18 Cached

This paper systematically evaluates uncertainty-aware decoding with rollback mechanisms to improve code generation in large language models, demonstrating performance gains over standard methods using uncertainty signals.

0 favorites 0 likes
#inference-strategies

Teaching AI agents to ask better questions by playing “Battleship”

MIT News — Artificial Intelligence ↗ · 2026-06-03 Cached

Researchers at MIT CSAIL and Harvard used a modified Battleship game to study and improve language models' question-asking abilities. By applying Monte Carlo inference strategies, they significantly boosted smaller models like Llama 4 Scout's win rate from 8% to 82% against humans, outperforming larger models at a fraction of the cost.

0 favorites 0 likes
← Back to home

Submit Feedback