Robust Shielding for Safe Reinforcement Learning
Summary
Introduces a novel shielding framework for robust Markov decision processes (RMDPs) that formally guarantees safety under uncertain transition dynamics, proving soundness and optimality. The approach combines with PAC guarantees for learned models, enabling safe reinforcement learning in unknown environments.
View Cached Full Text
Cached at: 06/02/26, 03:45 PM
# Robust Shielding for Safe Reinforcement Learning Source: [https://arxiv.org/abs/2606.00270](https://arxiv.org/abs/2606.00270) [View PDF](https://arxiv.org/pdf/2606.00270) > Abstract:Shielding is an effective approach to formally guarantee the safety of reinforcement learning agents in Markov decision processes \(MDPs\)\. However, existing shielding techniques typically assume knowledge of the safety\-relevant transition dynamics \- a requirement that is seldom met in practice\. To address this limitation, we introduce a novel shielding framework for robust MDPs \(RMDPs\), i\.e\., MDPs with sets of transition probabilities\. We define safety as the satisfaction of a linear temporal logic \(LTL\) formula with a certain threshold probability under the worst\-case transition probabilities of the RMDP\. We prove that our shielding framework is both sound and optimal for the RMDP: every policy admissible by the shield is safe, and conversely, every safe RMDP policy is admissible by the shield\. We combine our approach with existing sampling methods for learning transition probabilities of MDPs with probably approximately correct \(PAC\) guarantees\. This combination enables the construction of shields for MDPs that, with high confidence, guarantee safety while remaining minimally restrictive\. Our experiments show that our shields for learned RMDPs guarantee safety in unknown MDPs while recovering strong expected return as the number of samples increases\. ## Submission history From: Thom Badings \[[view email](https://arxiv.org/show-email/e8088f5e/2606.00270)\] **\[v1\]**Fri, 29 May 2026 19:01:12 UTC \(246 KB\)
Similar Articles
Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning
A method for contract-based compositional shielding that ensures global safety in multi-agent reinforcement learning without centralized runtime control, using local LTL obligations and a multi-armed bandit to optimize team reward.
Evolving Robustness--Exploration Trade-off in Online Reinforcement Learning via Quantile Bayesian Risk MDPs
This paper proposes a quantile Bayesian risk-aware MDP framework for online RL that adaptively balances robustness and exploration over time, providing theoretical regret bounds and demonstrating strong empirical performance.
How Maximum Entropy makes Reinforcement Learning Robust
This article explains how incorporating Shannon entropy into reinforcement learning objectives creates more robust agents capable of handling unexpected or adversarial changes in rewards and dynamics.
What are MDPs? And how can we Solve them?
This article explains the fundamentals of Markov Decision Processes (MDPs), a core framework in deep reinforcement learning, using an educational example of a student's daily decisions.
Theoretical Foundations and Effective Algorithms for Policy-Aware Simulator Learning
This paper proposes a strategic robustness objective for learning simulators in model-based reinforcement learning, formulated as a minimax game between a model player and an adversarial policy player. Theoretical guarantees and a provably convergent algorithm are provided, with experiments showing reduced prediction error and improved real-world policy transfer.