multi-reward-rl

SMOPD proposes a two-stage specialize-and-merge online policy distillation method to improve multi-reward reinforcement learning, addressing issues with sparse and dense reward signals where GDPO struggles. It outperforms GDPO across 1.5B, 3B, and 7B backbones in complementary and conflicting reward settings.

This paper proposes PRISM, a multi-reward RL framework that decomposes policy space rather than mixing rewards, improving multi-reward optimization and enabling inference-time controllability. Experiments on reasoning and alignment tasks show it outperforms existing baselines.