Robust Shielding for Safe Reinforcement Learning
Summary
Introduces a novel shielding framework for robust Markov decision processes (RMDPs) that formally guarantees safety under uncertain transition dynamics, proving soundness and optimality. The approach combines with PAC guarantees for learned models, enabling safe reinforcement learning in unknown environments.
View Cached Full Text
Cached at: 06/02/26, 03:45 PM
# Robust Shielding for Safe Reinforcement Learning Source: [https://arxiv.org/abs/2606.00270](https://arxiv.org/abs/2606.00270) [View PDF](https://arxiv.org/pdf/2606.00270) > Abstract:Shielding is an effective approach to formally guarantee the safety of reinforcement learning agents in Markov decision processes \(MDPs\)\. However, existing shielding techniques typically assume knowledge of the safety\-relevant transition dynamics \- a requirement that is seldom met in practice\. To address this limitation, we introduce a novel shielding framework for robust MDPs \(RMDPs\), i\.e\., MDPs with sets of transition probabilities\. We define safety as the satisfaction of a linear temporal logic \(LTL\) formula with a certain threshold probability under the worst\-case transition probabilities of the RMDP\. We prove that our shielding framework is both sound and optimal for the RMDP: every policy admissible by the shield is safe, and conversely, every safe RMDP policy is admissible by the shield\. We combine our approach with existing sampling methods for learning transition probabilities of MDPs with probably approximately correct \(PAC\) guarantees\. This combination enables the construction of shields for MDPs that, with high confidence, guarantee safety while remaining minimally restrictive\. Our experiments show that our shields for learned RMDPs guarantee safety in unknown MDPs while recovering strong expected return as the number of samples increases\. ## Submission history From: Thom Badings \[[view email](https://arxiv.org/show-email/e8088f5e/2606.00270)\] **\[v1\]**Fri, 29 May 2026 19:01:12 UTC \(246 KB\)
Similar Articles
Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learning
This paper introduces adaptive probabilistic shielding for safe reinforcement learning, where the shield is computed from an online learned MDP model, adapting as the model becomes more accurate.
Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning
A method for contract-based compositional shielding that ensures global safety in multi-agent reinforcement learning without centralized runtime control, using local LTL obligations and a multi-armed bandit to optimize team reward.
Evolving Robustness--Exploration Trade-off in Online Reinforcement Learning via Quantile Bayesian Risk MDPs
This paper proposes a quantile Bayesian risk-aware MDP framework for online RL that adaptively balances robustness and exploration over time, providing theoretical regret bounds and demonstrating strong empirical performance.
Robust Peak-cost Constrained Reinforcement Learning
This paper studies robust peak-cost constrained reinforcement learning, addressing limitations of standard CMDPs by controlling the maximum cost along a trajectory and considering dynamics uncertainty. The authors show zero duality gap may not hold and propose a surrogate optimization framework with robust value estimation.
How Maximum Entropy makes Reinforcement Learning Robust
This article explains how incorporating Shannon entropy into reinforcement learning objectives creates more robust agents capable of handling unexpected or adversarial changes in rewards and dynamics.