Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning
Summary
A method for contract-based compositional shielding that ensures global safety in multi-agent reinforcement learning without centralized runtime control, using local LTL obligations and a multi-armed bandit to optimize team reward.
View Cached Full Text
Cached at: 06/15/26, 09:10 AM
# Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning
Source: [https://arxiv.org/html/2606.14130](https://arxiv.org/html/2606.14130)
11institutetext:Imperial College London, London SW7 2AZ, UK22institutetext:University of Manchester, Manchester M13 9PL, UK22email:\{o\.adalat24,e\.hamel\.de\-le\-court,francesco\.belardinelli\}@imperial\.ac\.uk
###### Abstract
*Safe coordination*problems surface in multi\-agent reinforcement learning when global safety cannot be enforced by any agent unilaterally: the admissibility of one agent’s action may depend on the dynamics of other agents\. Decentralised shields can enforce safety at runtime, but purely factorised permissions often exclude optimal team behaviour that is safe only through coordination\. We study deterministic safety guarantees for agents trained and deployed under decentralised execution, recovering team\-optimal safe behaviour without centralised runtime control\. Agents have a shared global specificationϕ\\phiin the safety fragment of Linear Temporal Logic \(𝖫𝖳𝖫𝗌𝖺𝖿𝖾\\mathsf\{LTL\}\_\{\\mathsf\{safe\}\}\), and select among tuples of local𝖫𝖳𝖫𝗌𝖺𝖿𝖾\\mathsf\{LTL\}\_\{\\mathsf\{safe\}\}obligations whose conjunction implies the global specificationϕ\\phi\. Each agent may rely on the other agents’ local obligations as assumptions because the whole contract tuple is certified simultaneously and allows projection into local action masks\. At learning time, a non\-stationary multi\-armed bandit chooses among a library of local𝖫𝖳𝖫𝗌𝖺𝖿𝖾\\mathsf\{LTL\}\_\{\\mathsf\{safe\}\}obligations to select the tuple that optimises team reward, all without forgoing end\-to\-end safety\. We evaluate the approach across 6 environments and 15 algorithmic variants\.
## 1Introduction
Assuring the safety of learning cooperative agents requires reconciling two demands: agents should optimise a shared task objective, whilst satisfying safety constraints during training and deployment\. In*safe coordination*problems\[elsayed2021safe,raja2009towards\], the admissibility of one agent’s action depends on the non\-stationary policies of the other agents in the shared environment, so reasoning that treats teammate choices as arbitrary must discard behaviour that is safe only under coordinated actions\.*Multi\-Agent Reinforcement Learning*\(MARL\) offers a powerful framework for sequential decision\-making under uncertainty in a shared environment, by leveraging sampling\-based methods to iteratively refine policies in stochastic games\[littman1994markov\]\. Stochastic games are commonly studied under cooperative, competitive, and mixed strategic dynamics\. Many multi\-agent systems naturally involve cooperative tasks, such as rescue drones\[drew2021multi\]and autonomous warehouses\[wurman2008coordinating\], which makes the cooperative setting a natural target for safe MARL\. However, standard reward penalties are insufficient to verify that behaviour is safe once a policy is deployed\[ji2023safety\]\. From the formal methods standpoint,*shielding*\[konighofer2017shield\]is a popular technique for enforcing safety during both training and deployment by pre\-emptively masking out unsafe actions that may lead to a specification violation or post\-posedly replacing unsafe actions\[alshiekh2018safe\]\.
✓ centrally safe×\\timescentrally unsafe⋆\\starlocally rejected optimumenabled safe actionunsafesafe but not enabled hereSimilar Articles
Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Control
This paper proposes a hierarchical multi-agent reinforcement learning framework that enforces hard safety constraints via a constraint manifold at the low level while enabling effective coordination through high-level policy learning, providing theoretical safety guarantees and achieving near-perfect safety rates with good generalization.
Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learning
This paper introduces adaptive probabilistic shielding for safe reinforcement learning, where the shield is computed from an online learned MDP model, adapting as the model becomes more accurate.
Robust Shielding for Safe Reinforcement Learning
Introduces a novel shielding framework for robust Markov decision processes (RMDPs) that formally guarantees safety under uncertain transition dynamics, proving soundness and optimality. The approach combines with PAC guarantees for learned models, enabling safe reinforcement learning in unknown environments.
Is Per-Agent Policy Composition Safe? Rethinking Successor-Feature Transfer in Cooperative Multi-Agent Reinforcement Learning
The paper analyzes the safety of per-agent policy composition in multi-agent reinforcement learning, proving that independent composition can fail, and proposes MA-USFA, a hierarchical method that enables safe and flexible successor-feature transfer in cooperative MARL.
TRIDENT: Breaking the Hybrid-Safety-Physics Coupling for Provably Safe Multi-Agent Reinforcement Learning
TRIDENT is a novel multi-agent reinforcement learning framework that breaks the coupling between hybrid discrete-continuous actions, hard safety constraints, and physics-governed dynamics, achieving provably safe coordination with a convergence guarantee to a constrained Nash equilibrium and significant reductions in training-time violations.