Tag
Chopthin-Consensus Power Sampling (CCPS) is a diversity-preserving method for LLM decoding that improves reasoning accuracy without post-training by preserving distinct reasoning paths and using semantic-majority selection, as demonstrated in benchmarks.
The paper presents a theoretical framework for steering and scaling large language models via sampling algorithms, such as Sequential Monte Carlo and Replica Exchange, to improve generation quality without external supervision.
LatticeBridge proposes a twisted sequential Monte Carlo decoder for structured sequence generation that improves constraint satisfaction by treating the problem as rare-event inference, outperforming greedy and beam baselines on CommonGen, E2E NLG, and WikiBio.
Introduces Agentic Monte Carlo (AMC), a method to perform reinforcement learning-style optimization of black-box LLM agents using Sequential Monte Carlo, without requiring access to model parameters.
SMCEvolve introduces a principled framework for LLM-driven program evolution by reformulating it as sampling from a reward-tilted distribution using Sequential Monte Carlo. It provides convergence guarantees and outperforms existing methods across multiple scientific discovery benchmarks.
This paper proposes Sequential Monte Carlo Speculative Decoding (SMC-SD), a method that accelerates LLM inference by replacing token-level rejection in speculative decoding with importance-weighted resampling over draft particles, achieving 2.36× speedup over standard speculative decoding and 5.2× over autoregressive decoding while maintaining 3% accuracy loss.