Tag
This paper introduces Neural Non-Equilibrium Hamiltonian Monte Carlo (NHMC), a train-then-correct method for sampling from unnormalized Boltzmann densities by learning stochastic Hamiltonian-style paths and correcting them using non-equilibrium work.
Explains token-level rejection sampling for RLHF/PPO, where importance ratio M_t is the maximum over vocabulary and tokens are accepted with Bernoulli sampling based on w_t / M_t.
This paper introduces Unbounded Positive Asymmetric Optimization (UP), a universal plug-and-play objective that resolves the exploration-stability dilemma in RL-based LLM training by anchoring the policy with stop-gradient, enabling unclipped gradients for positive advantages while clipping negative ones.