Tag
Bellman Policy Optimization (BPO) is a critic-free reinforcement learning method that reformulates Policy Mirror Descent using the Bellman equation for autoregressive generation with terminal rewards, improving mathematical reasoning in large language models.
Gated-BEPO is a new credit assignment method for LLM agents that derives step-level credit from empirical rollout graphs using Bellman fixed-point estimation and adaptively fuses it with episode-level credit via a confidence gate. Experiments on WebShop, ALFWorld, and visual Sokoban show consistent improvements over existing critic-free methods.