Deterministic Pareto-Optimal Policy Synthesis for Multi-Objective Reinforcement Learning
Summary
This paper introduces a novel preference-conditioned Bellman operator based on Chebyshev scalarization to compute deterministic Pareto-optimal policies for Multi-Objective Markov Decision Processes, proving its convergence and effectiveness in capturing the entire Pareto frontier.
View Cached Full Text
Cached at: 06/26/26, 05:18 AM
# Deterministic Pareto-Optimal Policy Synthesis for Multi-Objective Reinforcement Learning Source: [https://arxiv.org/abs/2606.26397](https://arxiv.org/abs/2606.26397) [View PDF](https://arxiv.org/pdf/2606.26397) > Abstract:Real\-world decision\-making often requires balancing multiple conflicting objectives, a challenge that standard Reinforcement Learning \(RL\) frequently addresses by aggregating rewards into a single scalar signal\. While effective for simple tasks, this approach often fails to capture the full spectrum of optimal trade\-offs, known as the Pareto frontier\. In this paper, we introduce a novel preference\-conditioned Bellman operator, motivated from the Chebyshev scalarization, designed to compute deterministic Pareto\-optimal policies for Multi\-Objective Markov Decision Processes \(MOMDPs\)\. We prove that this operator satisfies an enveloping property, where the estimated value functions upper\-bound the true Pareto frontier, and demonstrate that it monotonically converges to a coverage set of this frontier\. Furthermore, we also show how to extract deterministic policies from these converged Q\-estimates\. This ensures the agent can recover a policy for any given preference, capturing the entire Pareto\-optimal frontier while guaranteeing each synthesized policy remains approximately Pareto\-optimal\. Experimental results validate that our algorithm successfully recovers complex trade\-offs, providing a solution for deterministic Pareto\-optimal policy synthesis\. ## Submission history From: Aniruddha Joshi \[[view email](https://arxiv.org/show-email/6ea3df43/2606.26397)\] **\[v1\]**Wed, 24 Jun 2026 21:28:49 UTC \(109 KB\)
Similar Articles
Preference-Conditioned Multi-Objective Reinforcement Learning for Runtime-Tunable Transit Signal Priority
This paper presents a preference-conditioned multi-objective reinforcement learning controller for transit signal priority that allows runtime tuning of the trade-off between bus priority and overall traffic delay without retraining. Experiments show it outperforms fixed-time and rule-based baselines while maintaining feasibility constraints.
From Correctness to Preference: A Framework for Personalized Agentic Reinforcement Learning
This paper proposes a unified framework for personalized agentic reinforcement learning that decouples generic task rewards from personalized preference rewards, introducing PARPO and PSGM for preference-aligned policy optimization and skill retrieval.
LEMUR: Learning to Align with Multi-Objective Reinforcement Learning from Preference Feedback
This paper introduces LEMUR, a framework that combines multi-objective reinforcement learning with preference-based learning from multiple human feedback to learn Pareto-optimal policies without predefined reward functions.
SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation
SMOPD proposes a two-stage specialize-and-merge online policy distillation method to improve multi-reward reinforcement learning, addressing issues with sparse and dense reward signals where GDPO struggles. It outperforms GDPO across 1.5B, 3B, and 7B backbones in complementary and conflicting reward settings.
Regularized Offline Policy Optimization with Posterior Hybrid Bayesian Belief
This paper introduces Posterior Hybrid Bayesian Belief (PhyB), a framework that reformulates the expectation in Bayesian RL as a convex combination over dynamics models, enabling efficient regularized offline policy optimization with bounded objective discrepancy and state-of-the-art performance.