progpo

ProGPO is a learned-critic-free method for step-level advantage estimation in group-based RL for LLM agents, using exact-prefix action comparisons and rollout-based state potentials to improve credit assignment on long-horizon tasks. Experiments on ALFWorld and WebShop with Qwen2.5 models show it outperforms existing agentic RL baselines.