Tag
The paper proposes Group Adaptive Clustering Policy Optimization (GAPO), a plug-in modification to GRPO methods that adapts the clipping boundary to rollout advantage, improving Pass@1 and Pass@k on math reasoning and coding benchmarks.
This paper introduces DiffTilt, a distributional framework that exponentially tilts a diffusion model-induced joint distribution over environments and executions to efficiently discover rare safety-critical failures in autonomous and cyber-physical systems, outperforming conditional sampling strategies on ARCH-COMP benchmarks and a new tractor-trailer benchmark.
This paper introduces Neural Non-Equilibrium Hamiltonian Monte Carlo (NHMC), a train-then-correct method for sampling from unnormalized Boltzmann densities by learning stochastic Hamiltonian-style paths and correcting them using non-equilibrium work.
Explains token-level rejection sampling for RLHF/PPO, where importance ratio M_t is the maximum over vocabulary and tokens are accepted with Bernoulli sampling based on w_t / M_t.
This paper introduces Unbounded Positive Asymmetric Optimization (UP), a universal plug-and-play objective that resolves the exploration-stability dilemma in RL-based LLM training by anchoring the policy with stop-gradient, enabling unclipped gradients for positive advantages while clipping negative ones.