constrained-mdp

This paper studies robust peak-cost constrained reinforcement learning, addressing limitations of standard CMDPs by controlling the maximum cost along a trajectory and considering dynamics uncertainty. The authors show zero duality gap may not hold and propose a surrogate optimization framework with robust value estimation.

This paper introduces a simple yet powerful methodology for Utility-Constrained MDPs (UCMDPs) that enables risk-sensitive constraints without fixing constraint limits in advance, outperforming baselines on Safety Gymnasium benchmarks.