DRG-MAPPO: Hierarchical Dynamic Role-Graph Multi-Agent Reinforcement Learning for Cooperative Air Combat

Hugging Face Daily Papers Papers

Summary

A hierarchical multi-agent reinforcement learning framework combining graph attention and dynamic role assignment improves tactical coordination and win rates in air combat.

Multi-Agent Reinforcement Learning (MARL) has emerged as a pivotal paradigm for complex decision-making in autonomous systems and air combat. While MARL has demonstrated significant potential in air combat, achieving sophisticated tactical coordination remains a non-trivial challenge. This difficulty is largely attributed to two primary limitations: (1) the absence of structured relational modeling hinders agents from capturing complex, time-varying interactions among battlefield entities; and (2) conventional flat architectures often lack the capability to explicitly model tactical roles, leading to ambiguous task allocation in highly dynamic environments. To address these challenges, we propose Hierarchical Dynamic Role-Graph Multi-Agent Proximal Policy Optimization (DRG-MAPPO), a novel MARL framework that integrates graph-based relational modeling with dynamic role assignment. Specifically, DRG-MAPPO constructs a graph-based representation of battlefield interactions and leverages graph attention mechanisms to extract critical relational features among allies, enemies, and threats. Subsequently, a high-level policy employs a dynamic role assignment mechanism to determine tactical responsibilities (e.g., ``leader'' and ``supporter''). Conditioned on these roles and encoded graph-relational features, a low-level policy executes discrete maneuver actions, facilitating the joint optimization of tactical strategy and collaborative execution. Furthermore, a target-priority auxiliary task is designed to foster the emergence of behaviors such as focus-fire. Experimental results demonstrate that DRG-MAPPO achieves a state-of-the-art win rate of 87%, suggesting that our framework effectively balances relational modeling, interpretability, and optimization stability for cooperative air combat.
Original Article
View Cached Full Text

Cached at: 09/11/26, 10:16 AM

Paper page - DRG-MAPPO: Hierarchical Dynamic Role-Graph Multi-Agent Reinforcement Learning for Cooperative Air Combat

Source: https://huggingface.co/papers/2609.11155

Abstract

A hierarchical multi-agent reinforcement learning framework combining graph attention and dynamic role assignment improves tactical coordination and win rates in air combat.

Multi-Agent Reinforcement Learning(MARL) has emerged as a pivotal paradigm for complex decision-making in autonomous systems and air combat. While MARL has demonstrated significant potential in air combat, achieving sophisticated tactical coordination remains a non-trivial challenge. This difficulty is largely attributed to two primary limitations: (1) the absence of structuredrelational modelinghinders agents from capturing complex, time-varying interactions among battlefield entities; and (2) conventional flat architectures often lack the capability to explicitly model tactical roles, leading to ambiguous task allocation in highly dynamic environments. To address these challenges, we propose Hierarchical Dynamic Role-Graph Multi-AgentProximal Policy Optimization(DRG-MAPPO), a novel MARL framework that integrates graph-basedrelational modelingwithdynamic role assignment. Specifically,DRG-MAPPOconstructs a graph-based representation of battlefield interactions and leveragesgraph attention mechanismsto extract critical relational features among allies, enemies, and threats. Subsequently, a high-level policy employs adynamic role assignmentmechanism to determine tactical responsibilities (e.g., ``leader’’ and ``supporter’’). Conditioned on these roles and encoded graph-relational features, a low-level policy executes discrete maneuver actions, facilitating the joint optimization of tactical strategy and collaborative execution. Furthermore, a target-priorityauxiliary taskis designed to foster the emergence of behaviors such as focus-fire. Experimental results demonstrate thatDRG-MAPPOachieves a state-of-the-art win rate of 87%, suggesting that our framework effectively balancesrelational modeling, interpretability, and optimization stability for cooperative air combat.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2609\.11155

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.11155 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.11155 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.11155 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Learning to cooperate, compete, and communicate

OpenAI Blog

OpenAI presents research on multi-agent reinforcement learning environments where agents learn to cooperate, compete, and communicate. The paper introduces MADDPG (Multi-Agent DDPG), a centralized critic approach that enables agents to learn collaborative strategies and communication protocols more effectively than traditional decentralized methods.

Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Control

arXiv cs.AI

This paper proposes a hierarchical multi-agent reinforcement learning framework that enforces hard safety constraints via a constraint manifold at the low level while enabling effective coordination through high-level policy learning, providing theoretical safety guarantees and achieving near-perfect safety rates with good generalization.

Scalable Constrained Multi-Agent Reinforcement Learning via State Augmentation and Consensus for Separable Dynamics

arXiv cs.LG

This paper presents a distributed approach for constrained multi-agent reinforcement learning that uses state-augmented policy learning and neighbor-to-neighbor consensus over dual variables to satisfy global resource constraints while scaling linearly with the number of agents. Experiments on smart grid demand response demonstrate that consensus coordination is essential for feasibility, scaling to thousands of agents unlike centralized training approaches.