ASGARD：基于强化学习的无人机动作空间防护

# Action-Space Guard for UAV Resilience via Reinforcement Learning
Source: [https://arxiv.org/html/2609.20982](https://arxiv.org/html/2609.20982)
Mohsen SalehiAffiliation:The University of British ColumbiaAffiliation:Vancouver, CanadaEmail:[msalehi@ece\.ubc\.ca](mailto:[email protected])Karthik PattabiramanAffiliation:The University of British ColumbiaAffiliation:Vancouver, CanadaEmail:[karthikp@ece\.ubc\.ca](mailto:[email protected])

###### Abstract

Reinforcement learning \(RL\) controllers have been recently adopted for Unmanned Aerial Vehicles \(UAV\) navigation and control\. However, they are susceptible to*action\-space attacks*that overwrite the action commands after the policy generates them and before the actuators execute them\. While most existing defenses target attacks on the policy’s inputs, those addressing*action\-space attacks*retrain the policy at training time and are not resilient to corrupted actions at runtime\. We proposeASGARD, a two\-phase teacher–student pipeline for making RL\-based UAV control resilient to*action\-space attacks*\. In the teacher phase, an encoder combines the UAV’s physical state with action\-attack\-related privileged information to produce an*action\-attack\-aware*latent that trains the RL control policy and a monitor that outputs corrected action commands to the actuators\. In the student phase, both the encoder and the monitor are trained via supervised learning from their teacher counterparts to run on\-board using only the UAV’s physical state history\. We evaluateASGARDacross attack scenarios targeting different action commands on UAV\. We find thatASGARDis resilient to*action\-space attacks*and completes the missions despite the attack\. We further find thatASGARDgeneralizes to unseen attacks and remains resilient against stealthy attacks\.

## IIntroduction

Unmanned Aerial Vehicles \(UAVs\) consist of control software and hardware that rely on onboard sensors to observe the vehicle’s physical state and on actuators to execute the commands that drive autonomous navigation\. The safety and correctness of this entire pipeline, from sensing and control software to actuation, are critical to mission success, yet UAVs remain vulnerable to a range of attacks\[[1](https://arxiv.org/html/2609.20982#bib.bib10)\]\. In particular,*physical attacks*such as GPS spoofing that inject noise into the physical channel\[[2](https://arxiv.org/html/2609.20982#bib.bib5),[3](https://arxiv.org/html/2609.20982#bib.bib1)\], and*action\-space attacks*that modify the*control action commands*\[[4](https://arxiv.org/html/2609.20982#bib.bib15),[5](https://arxiv.org/html/2609.20982#bib.bib4)\]\(e\.g\., roll and pitch\), threaten mission safety\.

In recent years, model\-free reinforcement learning \(RL\) has been widely adopted across the UAV stack, from low\-level control to autonomous navigation, because it adapts well to diverse and complex environments\[[6](https://arxiv.org/html/2609.20982#bib.bib3),[7](https://arxiv.org/html/2609.20982#bib.bib2),[8](https://arxiv.org/html/2609.20982#bib.bib6)\]\. Building on this line of work, robust\-RL methods such as ARMOR\[[3](https://arxiv.org/html/2609.20982#bib.bib1)\]and robust adversarial reinforcement learning \(RARL\)\[[9](https://arxiv.org/html/2609.20982#bib.bib7)\]train control policies that remain reliable under sensor spoofing, offering a defense against*physical attacks*\.

While physical attacks have been addressed by existing techniques, attackers can still modify*control action commands*after the control policy has generated the commands \- these are known as*action\-space attacks*\[[4](https://arxiv.org/html/2609.20982#bib.bib15)\]\. For example, modifying a single action command such as*roll*or*pitch*after inference can cause crashes or drive a UAV off its planned trajectory\. The attacker can achieve this through different means, such asdata\-only attacks\[[10](https://arxiv.org/html/2609.20982#bib.bib9),[11](https://arxiv.org/html/2609.20982#bib.bib24),[5](https://arxiv.org/html/2609.20982#bib.bib4)\]\(a type of software\-based memory corruption attack\), which modify the action commands*after*the control policy has generated them\. As a result, the control policy keeps producing action commands that look safe and correct on the surface, but the vehicle sees the corrupted actions and performs them\. Because these attacks occur*after*the control policy has generated the actions, they cannot be detected by input\-side resilient learning policies such as ARMOR and RARL\. Defenses that target*action\-space attacks*instead harden the policy against action perturbations encountered during training\[[12](https://arxiv.org/html/2609.20982#bib.bib16),[13](https://arxiv.org/html/2609.20982#bib.bib17),[14](https://arxiv.org/html/2609.20982#bib.bib25)\]; once deployed, they forward every action the policy produces directly to the actuators, with no mechanism to intervene if that action has been corrupted\.

To close this gap, we proposeASGARD111ASGARDstands for Action\-Space GuARD, and is named after the walled realm of the Norse gods\., an RL\-based control pipeline that makes UAV controllers resilient to*action\-space attacks*\.ASGARDis built on three innovations\.

First, inspired by prior work\[[7](https://arxiv.org/html/2609.20982#bib.bib2),[3](https://arxiv.org/html/2609.20982#bib.bib1)\],ASGARDuses a two\-phase*teacher*–*student*training scheme\.ASGARDfirst uses a*teacher encoder*with privileged information about*action\-space attacks*, such as the target action and attack duration, to generate an*action\-attack\-aware*latent, which is then used to train the*RL controller*\. The teacher encoder then supervises a*student encoder*that runs on the device at inference time, generating a matching latent from only the UAV’s physical state, so that the RL controller can operate without privileged information at deployment\.

Second,ASGARDplaces a lightweight multilayer perceptron \(MLP\), called*monitor*, between the RL controller and the actuators\. The monitor is trained under the same teacher–student scheme as the RL controller: a*teacher monitor*is trained on the teacher encoder’s output, and a*student monitor*is trained on the student encoder’s output under supervision from the teacher monitor\. Its lightweight design keeps the pipeline within the real\-time constraints of onboard UAV control\. Further, its placement shortens the attacker’s effective tampering window and introduces a trust boundary between the RL policy and the actuators\.

Third, unlike detection\-only defenses that halt or land when they suspect an attack,ASGARD’s monitor actively corrects the RL controller’s outputs at every step and forwards safe action commands to the actuators, so the UAV keeps flying under a safe control signal rather than falling back to a degraded mode or hovering and finally crashing\.

ASGARDhas three advantages over conventional techniques\. First, because it is trained end\-to\-end on both clean and attacked timesteps, the monitor learns to pass safe action commands through unchanged and repair modified ones before they reach the actuators, without a separate detection step\. Furthermore, since the teacher already sees the attack context through its privileged inputs,ASGARDdoes not need to keep generating fresh attacks during training, which keeps training costs low\. Finally, a trainedASGARDmodel can handle even attacks it did not see during training, without requiring re\-training, thereby making it robust to new attacks\.

Contributions\.We make three contributions as follows\.

- •We provide resilience against*action\-space attacks*on RL\-based UAV controllers by recovering action commands that are corrupted at runtime after they are generated by the policy, unlike existing action\-space defenses, which retrain the policy against perturbations at training time instead of recovering corrupted actions\.
- •We proposeASGARD, a two\-phase*teacher*–*student*control pipeline in which a teacher based on a Variational Autoencoder \(VAE\) with access to action\-attack\-aware privileged information supervises a runtime student, implemented as a Long Short\-Term Memory \(LSTM\) network, that adapts the trained RL controller to rely only on the history of UAV physical states\.
- •We introduce the monitor, a small multilayer perceptron \(MLP\) placed between the RL controller and the actuators that actively corrects each outgoing action command rather than merely detecting attacks\. The monitor is trained under the same teacher–student scheme: a teacher monitor conditioned on the teacher’s attack\-aware latent supervises a student monitor that relies on the student latent at deployment, letting safe commands through unchanged and repairing corrupted ones before they reach the actuators\.

The results show thatASGARDis resilient against action\-space attacks, completing 95% of missions with no crashes when a single action command is corrupted, where an RL\-only controller and ARMOR \(the state\-of\-the\-art resilient RL controller\) complete 40% and 50% respectively and crash in 47% to 36% of missions\. When all four commands are corrupted at once and both prior techniques fail every mission,ASGARDcompletes 67%\.ASGARDfurther generalizes to remain resilient to attack channels unseen during training and maintains resilience against stealthy attack patterns\.

## IIBackground

### II\-AUAV Controller

UAVs consist of control software \(controller\) and hardware, including sensors such as GPS and IMU that observe the environment, and actuators \(the motors\) that execute the action commands \(e\.g\., roll and pitch channels\) generated by the controller to complete a mission\. At each control steptt, an onboard estimator fuses these sensor measurements into a physical statests\_\{t\}\(e\.g\., positionpt=\(x,y,z\)p\_\{t\}=\(x,y,z\)and velocityvt=\(x˙,y˙,z˙\)v\_\{t\}=\(\\dot\{x\},\\dot\{y\},\\dot\{z\}\)\)\. Givensts\_\{t\}and a referencegtg\_\{t\}drawn from the mission’s waypoint list, the control policyπ\\piproduces the action commandsat=π⁡\(st,gt\)a\_\{t\}=\\pi\(s\_\{t\},g\_\{t\}\), which are sent to the actuators\. The action command is a tuple of four values: pitchatpa\_\{t\}^\{p\}, rollatra\_\{t\}^\{r\}, thrustatTa\_\{t\}^\{T\}, and gainatKa\_\{t\}^\{K\}\. The four channels together parameterize the vehicle’s intended motion:atpa\_\{t\}^\{p\}commands the forward/backward movement,atra\_\{t\}^\{r\}the lateral movement,atTa\_\{t\}^\{T\}the vertical movement, andatKa\_\{t\}^\{K\}scales the overall speed at which the UAV moves toward its destination\.

A mission is safe and successful when the UAV reaches every waypoint while staying within a mission\-specific safety radiusϵ\\epsilonof the reference, i\.e\., when the tracking errorΔ​pt=‖pt−gt‖≤ϵ\\Delta p\_\{t\}=\\\|p\_\{t\}\-g\_\{t\}\\\|\\leq\\epsilonfor alltt\.

### II\-BAction\-Space Attacks

*Action\-space attacks*intercept the RL controller’s action commands after the control policy has produced them, and modify the values before they reach the actuators\. To carry out this attack, attackers can use existing techniques such as memory corruption exploits \(e\.g\.,*data\-only attacks*\)\[[10](https://arxiv.org/html/2609.20982#bib.bib9),[5](https://arxiv.org/html/2609.20982#bib.bib4)\]\. In a UAV controller, the action commandsat=π⁡\(st,gt\)a\_\{t\}=\\pi\(s\_\{t\},g\_\{t\}\)live in memory between the moment the control policy writes them and the moment the actuator reads them, so an attacker who can overwrite them replacesata\_\{t\}with modified commandsa~t\\tilde\{a\}\_\{t\}\(e\.g\., an additive biasa~t=at\+bt\\tilde\{a\}\_\{t\}=a\_\{t\}\+b\_\{t\}chosen by the attacker\) during this window\. Thus, the control policy and any check performed on its inputs still appear normal, and the unsafe behavior appears only in the physical world\. The damage depends on which action channel is targeted\. For instance, a perturbation ofatpa\_\{t\}^\{p\}oratra\_\{t\}^\{r\}causes unintended forward or sideways motion, while tampering withatTa\_\{t\}^\{T\}oratKa\_\{t\}^\{K\}drives it up, down, or past the destination; in either case, the UAV eventually deviates from the planned trajectory or crashes\. Hardware\-level safety interlocks, if present, are designed to catch catastrophic failures \(e\.g\., complete motor loss\) and are therefore ineffective against the subtle, gradual action corruptions we consider\.

### II\-CThreat Model

We consider an adversary that can overwrite the action commandsata\_\{t\}after the control policyπ\\pihas produced them and before the actuators execute the actions\. For instance, at each control step, the adversary may replaceata\_\{t\}with modified commandsa~t=at\+bt\\tilde\{a\}\_\{t\}=a\_\{t\}\+b\_\{t\}, wherebtb\_\{t\}is an attacker\-chosen bias applied to one or more of the four action channels \(pitchatpa\_\{t\}^\{p\}, rollatra\_\{t\}^\{r\}, thrustatTa\_\{t\}^\{T\}, and gainatKa\_\{t\}^\{K\}\) with varying severity, patterns, and duration\. We assume the attacker can act stealthily by applying small biasesbtb\_\{t\}that cause gradual drift from the intended trajectory\. The encoder, the control policy, and the monitor are immutable after training, and hence we assume they reside in read\-only memory at deployment and cannot be modified by the adversary\. Action commands, in contrast, are recomputed at every control step and must hence be stored in writable memory, leaving them exposed to corruption by attackers\.

Physical attacks on sensors, such as GPS spoofing, are outside our scope; they target a different point in the pipeline and are addressed by prior work\[[3](https://arxiv.org/html/2609.20982#bib.bib1),[9](https://arxiv.org/html/2609.20982#bib.bib7)\]Attacks on the ground station, the mission plan, or the communication link between them are likewise out of scope\.

## IIIRelated work

We classify related work into two broad categories\.Model\-free Reinforcement LearningRecent advances in reinforcement learning \(RL\) have made learned policies a leading approach for robotic control, replacing hand\-tuned controllers with end\-to\-end training on interaction data\. Applications range from low\-level quadrotor stabilization\[[8](https://arxiv.org/html/2609.20982#bib.bib6)\]and championship\-level drone racing\[[15](https://arxiv.org/html/2609.20982#bib.bib13)\]to legged locomotion over challenging terrain\[[7](https://arxiv.org/html/2609.20982#bib.bib2)\]\. Extending this line of work, other approaches adapt RL to robust and safe operation: co\-training the policy against a perturbing adversary\[[9](https://arxiv.org/html/2609.20982#bib.bib7)\], learning under environmental uncertainty\[[16](https://arxiv.org/html/2609.20982#bib.bib11),[17](https://arxiv.org/html/2609.20982#bib.bib12)\]to adapt the robot’s behavior in unseen environments, or keeping the system in a safe state using manually defined, fixed boundaries of unsafe actions\[[18](https://arxiv.org/html/2609.20982#bib.bib26)\], which require anticipating unsafe regions in advance and are therefore not designed for action\-space attacks that occur after the action is generated by the control policy, the threat model considered byASGARD\.

Attacks on RobotsAdversarial attacks on RL\-based techniques span several categories\[[1](https://arxiv.org/html/2609.20982#bib.bib10),[19](https://arxiv.org/html/2609.20982#bib.bib14)\], including perturbations to observed states, manipulations of the training environment, and attacks on the action space\. Due to space constraints, we focus on the two most relevant to this work\.

1\. Physical Attacksperturb the sensor measurements that feed the policy, such as GPS spoofing\[[2](https://arxiv.org/html/2609.20982#bib.bib5)\]\. Robust Adversarial RL \(RARL\)\[[9](https://arxiv.org/html/2609.20982#bib.bib7)\]co\-trains a policy against an adversary that injects adversarial perturbations as external forces on the agent’s body, modeling physical\-layer disturbances such as mass or friction mismatch; ARMOR\[[3](https://arxiv.org/html/2609.20982#bib.bib1)\]trains a UAV controller under a teacher–student scheme with privileged information about sensor spoofing\.

2\. Action\-Space Attackstarget the action space of an RL control policy\. Lee et al\.\[[4](https://arxiv.org/html/2609.20982#bib.bib15)\]explore action\-space adversarial attacks under spatial and temporal budgets, comparing an attack that perturbs actions independently at each step against one that plans across multiple steps using the agent’s dynamics\. They show these attacks are effective at degrading a Deep RL \(DRL\) agent’s performance in simulated environments\. However, their work focuses purely on crafting stronger attacks, with no defense or resilience mechanism proposed\. Several papers have proposed robustifying DRL agents against action\-space perturbations by training the policy against adversarial or worst\-case perturbations to its actions\[[12](https://arxiv.org/html/2609.20982#bib.bib16),[13](https://arxiv.org/html/2609.20982#bib.bib17)\]\. Similarly, Lee et al\.\[[14](https://arxiv.org/html/2609.20982#bib.bib25)\]investigate black\-box targeted attacks using a learned adversarial policy and show that fine\-tuning the nominal policy via adversarial training can partially mitigate such attacks, though it does not fully eliminate their success\. However, these approaches retrain the policy against such attacks at training time rather than addressing corrupted actions at runtime after they are generated by the policy, which is the problemASGARDaddresses\. Different attack techniques can be used to corrupt the RL controller’s action commands, including*non\-control\-data attacks*\[[10](https://arxiv.org/html/2609.20982#bib.bib9)\], software\-based exploits that manipulate program data \(e\.g\., action commands in UAVs\)\.

## IVDesign:ASGARD

We first present an overview of the design ofASGARD, followed by a deep dive into the teacher and student phases\.

### IV\-AASGARD: Overview

![Refer to caption](https://arxiv.org/html/2609.20982v1/Phoenix.png)Fig\. 1:Overview ofASGARD’s operation\.\(1\) Teacher Phase:the Teacher encoder combines privileged information with the UAV’s physical state to produce an action\-attack\-aware latent that trains the RL control policy and the Teacher monitor, which outputs corrected action commands to the actuators\.\(2\) Student Phase:the Student encoder and Student monitor are trained via supervised learning from their Teacher counterparts, using only the UAV’s physical state history as input; these three components \(Student Encoder, Control Policy, Student monitor\) run together on\-board at deployment\.Figure[1](https://arxiv.org/html/2609.20982#S4.F1)shows an overview ofASGARD, a two\-phase teacher–student pipeline with three components in each phase for defending against*action\-space attacks*\. In the teacher phase, the teacher encoder consumes the UAV’s physical state \(oto\_\{t\}\) along with action\-attack\-related privileged information \(xtx\_\{t\}\) \(e\.g\., target action\) to produce an*action\-attack\-aware latent*\(lt¯\\bar\{l\_\{t\}\}\)\. This latent trains the RL control policy, which in turn outputs both an action \(at¯\\bar\{a\_\{t\}\}\) and an expected next state, jointly training the teacher monitor on the latent, the action, and the expected state to output a corrected action \(at¯~\\tilde\{\\bar\{a\_\{t\}\}\}\)\. The three components are trained end\-to\-end on a mix of clean and attacked trajectories, so the monitor learns both to forward safe commands and to repair corrupted ones \(a¯​´t\\bar\{a\}\\acute\{\}\_\{t\}\)\.

In the student phase, the pipeline is adapted for onboard deployment, where privileged information is unavailable\. The student encoder is trained via supervised learning from the teacher encoder to match its attack\-aware latent from only the UAV’s physical state history \(H\), and the student monitor is trained under supervision from the teacher monitor using the student latent \(ltl\_\{t\}\), the policy’s expected state, and the generated action commands\. The control policy is carried over from the teacher phase and now consumes the student latent to produce action commands \(ata\_\{t\}\)\. At runtime, the student encoder, control policy, and student monitor execute together on the onboard device; at each step, the monitor forwards the correct action commands to the actuators \(at~\\tilde\{a\_\{t\}\}\)\.

### IV\-BASGARD: Teacher Phase

Like prior work\[[7](https://arxiv.org/html/2609.20982#bib.bib2),[20](https://arxiv.org/html/2609.20982#bib.bib22)\], we formulate the control problem as a Markov Decision Process \(MDP\), defined by the tuple\(𝒮,𝒜,𝒫,r\)\(\\mathcal\{S\},\\mathcal\{A\},\\mathcal\{P\},r\), corresponding respectively to the state space, action space, transition probability𝒫:𝒮×𝒜→𝒮\\mathcal\{P\}:\\mathcal\{S\}\\times\\mathcal\{A\}\\to\\mathcal\{S\}, and scalar reward functionrr\. Training proceeds by selecting an actionata\_\{t\}from a control policyπ⁡\(at∣st\)\\pi\(a\_\{t\}\\mid s\_\{t\}\), receiving a rewardrtr\_\{t\}, and maximizing the expected discounted sum of rewards over time\.

In the first phase of training pipeline,ASGARDteacher assumes access to both the UAV’s physical stateoto\_\{t\}including the UAV’s position, orientation, linear velocity, and angular velocity \(ot=\[x,y,z,ϕ,θ,ψ,x˙,y˙,z˙,ϕ˙,θ˙,ψ˙\]o\_\{t\}=\\left\[x,y,z,\\phi,\\theta,\\psi,\\dot\{x\},\\dot\{y\},\\dot\{z\},\\dot\{\\phi\},\\dot\{\\theta\},\\dot\{\\psi\}\\right\]\) and privileged informationxtx\_\{t\}describing the*action\-space attacks*information and its corrupted physical states, the corruption bias currently applied, and its duration\. The teacher’s full state is thusst≔⟨ot,xt⟩s\_\{t\}\\coloneqq\\langle o\_\{t\},x\_\{t\}\\rangle\. The teacher encoderEteacherE\_\{\\text\{teacher\}\}, implemented as a variational autoencoder \(VAE\)\[[21](https://arxiv.org/html/2609.20982#bib.bib27)\], mapssts\_\{t\}\(Eteacher​\(ot,xt\)E\_\{\\text\{teacher\}\}\(o\_\{t\},x\_\{t\}\)\) to an*action\-attack\-aware*latent representationl¯t\\bar\{l\}\_\{t\}, the input reconstructionst^\\hat\{s\_\{t\}\}and the prediction of type of attackyt^\\hat\{y\_\{t\}\}\.

The control policyπ⁡\(at∣l¯t\)\\pi\(a\_\{t\}\\mid\\bar\{l\}\_\{t\}\)is trained via Proximal Policy Optimization \(PPO\)\[[22](https://arxiv.org/html/2609.20982#bib.bib23)\], usingl¯t\\bar\{l\}\_\{t\}as input to produce the intended actiona¯t\\bar\{a\}\_\{t\}\. For estimating and correcting action commands, alongside the policy, we train an auxiliary*expected\-state head*FF\(s^t\+1=F⁡\(l¯t\)\\hat\{s\}\_\{t\+1\}=F\(\\bar\{l\}\_\{t\}\)\), which predicts the physical state the UAV should reach immediately followinga¯t\\bar\{a\}\_\{t\}\. The expected state will help the monitor in the next step to correct or unchange the received action commands based on the current latent\.

FFis trained via supervised regression against the UAV’s true next state, observed one step ahead under*clean, unattacked*dynamics since policy produces action commands, it has access to uncorrupted ones:

ℒstate=‖F⁡\(l¯t\)−st\+1clean‖2\\mathcal\{L\}\_\{\\text\{state\}\}=\\left\\\|F\(\\bar\{l\}\_\{t\}\)\-s^\{\\text\{clean\}\}\_\{t\+1\}\\right\\\|^\{2\}\(1\)computed on uncorrupted commands, so thats^t\+1\\hat\{s\}\_\{t\+1\}always reflects the physically correct outcome, independent of whether an attack occurs attt\.

The teacher monitor \(MteacherM\_\{\\text\{teacher\}\}\) is the core mechanism enablingASGARDto recover from action\-attacks\. At every timestep, the attacker may overwrite the policy’s intended actiona¯t\\bar\{a\}\_\{t\}with a corrupted actiona¯​´t\\bar\{a\}\\acute\{\}\_\{t\}\.MteacherM\_\{\\text\{teacher\}\}observes the latentl¯t\\bar\{l\}\_\{t\}, the received \(possibly corrupted\) actiona¯​´t\\bar\{a\}\\acute\{\}\_\{t\}, and the expected next states^t\+1\\hat\{s\}\_\{t\+1\}, and outputs a corrected action:

at¯~=Mteacher​\(l¯t,a¯​´t,s^t\+1\)\\tilde\{\\bar\{a\_\{t\}\}\}=M\_\{\\text\{teacher\}\}\(\\bar\{l\}\_\{t\},\\bar\{a\}\\acute\{\}\_\{t\},\\hat\{s\}\_\{t\+1\}\)\(2\)MteacherM\_\{\\text\{teacher\}\}is trained via supervised regression toward the uncorrupted action using a mixture of clean timesteps \(wherea¯​´t\\bar\{a\}\\acute\{\}\_\{t\}=a¯t=\\bar\{a\}\_\{t\}\) and attacked timesteps \(wherea¯​´t\\bar\{a\}\\acute\{\}\_\{t\}≠a¯t\\neq\\bar\{a\}\_\{t\}\)\. Training on both populations with accessing to privileged action attack information helpsMteacherM\_\{\\text\{teacher\}\}to train on both scenarios and send commands consistent with the uncorrupted intent to the actuators\.

At every timestep of teacher\-phase training, the executed action isat¯~\\tilde\{\\bar\{a\_\{t\}\}\}, meaningMteacherM\_\{\\text\{teacher\}\}’s correction directly determines the UAV’s physical trajectory, not merely a diagnostic signal computed alongside it\. The rewardrtr\_\{t\}that shapes the policy’s training reflects progress toward the current target waypointgtg\_\{t\}, adopted by prior work:

rt=Rgoal⋅exp⁡\(−λ​‖pt−gt‖\)⏟sharp reward asUAV nears goal−α​‖pt−pt−1‖⏟penalize largeposition changes−β​θt⏟penalizeexcessive tilt−γ​‖at−at−1‖2⏟penalize abruptcontrol changes\\begin\{split\}r\_\{t\}=\\;&\\underbrace\{R\_\{\\text\{goal\}\}\\cdot\\exp\\left\(\-\\lambda\\\|p\_\{t\}\-g\_\{t\}\\\|\\right\)\}\_\{\\begin\{subarray\}\{c\}\\text\{sharp reward as\}\\\\ \\text\{UAV nears goal\}\\end\{subarray\}\}\-\\underbrace\{\\alpha\\\|p\_\{t\}\-p\_\{t\-1\}\\\|\}\_\{\\begin\{subarray\}\{c\}\\text\{penalize large\}\\\\ \\text\{position changes\}\\end\{subarray\}\}\\\\ &\-\\underbrace\{\\beta\\theta\_\{t\}\}\_\{\\begin\{subarray\}\{c\}\\text\{penalize\}\\\\ \\text\{excessive tilt\}\\end\{subarray\}\}\-\\underbrace\{\\gamma\\\|a\_\{t\}\-a\_\{t\-1\}\\\|^\{2\}\}\_\{\\begin\{subarray\}\{c\}\\text\{penalize abrupt\}\\\\ \\text\{control changes\}\\end\{subarray\}\}\\end\{split\}\(3\)whereptp\_\{t\}is the UAV’s position and penalty weights shown withα\\alphafor deviation from target,β\\betafor instability,γ\\gammafor sudden movement; upon reaching the final waypoint, a terminal bonus proportional to the remaining time steps in the episode rewards early, successful completion of the mission\.

### IV\-CASGARD: Student Phase

For onboard deployment, where privileged informationxtx\_\{t\}is unavailable, we train a student encoderEstudentE\_\{\\text\{student\}\}using only the UAV’s recent physical state historyHt:=\{ot−N,…,ot−1\}H\_\{t\}:=\\\{o\_\{t\-N\},\\dots,o\_\{t\-1\}\\\}, together with the UAV’s own previously executed actions\{at−N,…,at−1\}\\\{a\_\{t\-N\},\\dots,a\_\{t\-1\}\\\}, allowing the student to account for the consequences of its own recent commands when estimating the current*action\-attack\-aware*latent \(lt=Estudent​\(Ht\)l\_\{t\}=E\_\{\\text\{student\}\}\(H\_\{t\}\)\)\.

Given that the underlying data forms a sequence of time\-dependent measurements,EstudentE\_\{\\text\{student\}\}is implemented as a temporal variational autoencoder \(TVAE\) using a recurrent Long Short\-Term Memory \(LSTM\) network, trained via supervised regression to approximate the teacher’s latent\.

ℒfeat=‖lt−l¯t‖2\\mathcal\{L\}\_\{\\text\{feat\}\}=\\left\\\|l\_\{t\}\-\\bar\{l\}\_\{t\}\\right\\\|^\{2\}\(4\)We additionally minimize the discrepancy between the actions the shared control policy produces from each latent:

ℒact=‖π⁡\(lt\)−π⁡\(l¯t\)‖2\\mathcal\{L\}\_\{\\text\{act\}\}=\\left\\\|\\pi\(l\_\{t\}\)\-\\pi\(\\bar\{l\}\_\{t\}\)\\right\\\|^\{2\}\(5\)so that the student latent is optimized not only to numerically resemblel¯t\\bar\{l\}\_\{t\}, but to also yield the same downstream control decisions\. Overall, the student encoder tries to approximate the teacher encoder’s output using supervised learning, and correspondingly produce similar actions generated by the policy, by minimizing the loss:ℒfeat\+ℒact\+ℒattack\\mathcal\{L\}\_\{\\text\{feat\}\}\+\\mathcal\{L\}\_\{\\text\{act\}\}\+\\mathcal\{L\}\_\{\\text\{attack\}\}, whereℒattack\\mathcal\{L\}\_\{\\text\{attack\}\}is the attack classification loss\.

The student monitorMstudentM\_\{\\text\{student\}\}sharesMteacherM\_\{\\text\{teacher\}\}’s architecture\. Given the student latentltl\_\{t\}, the received actiona​´ta\\acute\{\}\_\{t\}, and the student’s own expected\-state prediction,MstudentM\_\{\\text\{student\}\}is trained to match the teacher monitor’s correction for the same underlying scenario:

ℒmonitorstudent=‖Mstudent​\(lt,a​´t,s^t\+1student\)−Mteacher​\(l¯t,a¯​´t,s^t\+1\)‖2\\mathcal\{L\}\_\{\\text\{monitor\}\}^\{\\text\{student\}\}=\\left\\\|M\_\{\\text\{student\}\}\(l\_\{t\},a\\acute\{\}\_\{t\},\\hat\{s\}^\{\\text\{student\}\}\_\{t\+1\}\)\-M\_\{\\text\{teacher\}\}\(\\bar\{l\}\_\{t\},\\bar\{a\}\\acute\{\}\_\{t\},\\hat\{s\}\_\{t\+1\}\)\\right\\\|^\{2\}

\(6\)
where the same attack is applied to both the teacher’s and student’s action at each training step, ensuring the two monitors are supervised on matching scenarios\. The control policyπ\\piis reused unchanged from the teacher phase, since both phases share an identical action space and reward objective\. At deployment, onlyEstudentE\_\{\\text\{student\}\},π\\pi, andMstudentM\_\{\\text\{student\}\}execute onboard: at each timestep,MstudentM\_\{\\text\{student\}\}consumes the policy’s action and either forwards it unchanged or replaces it withat~\\tilde\{a\_\{t\}\}before it reaches the actuators\.

In particular, at each control step, the student encoder computesltl\_\{t\}from the sensor and action history; the control policy producesat=π⁡\(lt\)a\_\{t\}=\\pi\(l\_\{t\}\); andMstudentM\_\{\\text\{student\}\}receives\(lt,a​´t,s^t\+1\)\(l\_\{t\},a\\acute\{\}\_\{t\},\\hat\{s\}\_\{t\+1\}\)unconditionally, on every step, regardless of whether an attack is present \(a​´ta\\acute\{\}\_\{t\}=at=a\_\{t\}ora​´ta\\acute\{\}\_\{t\}≠at\\neq a\_\{t\}\)\.MstudentM\_\{\\text\{student\}\}outputs an action command that is used directly by the actuators, adapting toward the uncorrupted command the control policy would have produced, even if an attack occurs at that step\.

## VASGARDEvaluation

This section first presents the experimental setup, and then evaluatesASGARDalong four aspects: \(i\) the training performance of the teacher–student pipeline \(§[V\-B](https://arxiv.org/html/2609.20982#S5.SS2)\), \(ii\)ASGARD’s behavior under action\-space attacks compared with prior techniques \(§[V\-C](https://arxiv.org/html/2609.20982#S5.SS3), §[V\-D](https://arxiv.org/html/2609.20982#S5.SS4)\), \(iii\) zero\-shot performance against unseen attacks \(§[V\-E](https://arxiv.org/html/2609.20982#S5.SS5)\), and \(iv\) resilience against stealthy attacks \(§[V\-F](https://arxiv.org/html/2609.20982#S5.SS6)\)\.

### V\-AExperimental Setup

For training and evaluation, we target a waypoint\-following task where a quadcopter \(shown on the right\)

![[Uncaptioned image]](https://arxiv.org/html/2609.20982v1/Images/Setup.jpeg)

reaches randomly\-selected 3D goal positions under coupled translational\-rotational dynamics\. We build the environment on*gym\-pybullet*\[[23](https://arxiv.org/html/2609.20982#bib.bib20)\], a PyBullet\-based quadrotor simulator with an OpenAI Gym interface\[[24](https://arxiv.org/html/2609.20982#bib.bib19)\]and realistic rigid\-body dynamics, matching the setup used in prior UAV RL work\[[25](https://arxiv.org/html/2609.20982#bib.bib18),[3](https://arxiv.org/html/2609.20982#bib.bib1)\]\.

ForASGARD, we extend this environment with three custom components: an action\-attack injection mechanism, a channel for privileged action\-context observations, and a monitor correction interface\. We adopt ARMOR’s publicly available implementation for the teacher encoder and control policy\[[3](https://arxiv.org/html/2609.20982#bib.bib1)\], modifying them to support our innovations in order to be resilient against*action\-space attacks*\. We implement the teacher–student encoder architecture in the training pipeline with a 15\-dimensional latent, and the student encoder takes an 80\-timestep history of the UAV’s physical state as input\. Further, the RL control policy is trained with PPO, and the monitors are implemented as lightweight MLPs\. AlthoughASGARDcan be applied to other types of vehicles, we restrict our evaluation to a single quadrotor model due to space constraints, prioritizing depth of analysis on*action\-space attacks*over breadth across platforms\.

Table[I](https://arxiv.org/html/2609.20982#S5.T1)summarizes the five attack configurations we use to evaluateASGARD’s resilience across a broad range of action\-attack scenarios, each targeting a different action channel with its own bias range and duration\. A*Pitch*attack disrupts the UAV’s forward progress, at times reversing its direction of travel outright, while a*Roll*attack turns what should be a small, controlled sideways adjustment into a hard, unintended lean\.*Thrust*attacks strike at the UAV’s climb and descent rate, disrupting vertical stability over an extended period\.*Gain*attacks act differently still, distorting not the direction of the UAV’s movement but its strength, leaving it barely responsive one moment and lurching the next\. Finally, we evaluate a combined attack that applies*all four*simultaneously, representing a worst\-case adversary that corrupts every command at once and drives the UAV off\-course along all four control dimensions in parallel\.

Target ActionBias RangeAttack DurationExplainPitch\(\-4\)\-\(4\)max 90sDisrupts or reverses forward progressRoll\(\-1\)\-\(1\)max 90sTurns a small adjustment into a hard, unintended leanThrust\(\-5\)\-\(5\)max 120sDisrupts climb/descent rateGain\(\-15\)\-\(15\)max 150sDistorts movement strengthAll\(\-1\)\-\(1\)max 60sAll four simultaneously, worst\-case scenario

TABLE I:Different types of attacks on each action command for evaluatingASGARD\.For comparison, we consider two additional techniques\. The state\-of\-the\-art RL resilience approachARMOR\[[3](https://arxiv.org/html/2609.20982#bib.bib1)\]uses a similar teacher–student training pipeline but targets*physical attacks*on the sensor inputs of the RL control policy, and forwards its action commands to the actuators without any correction step\. Other adversarial training defenses\[[26](https://arxiv.org/html/2609.20982#bib.bib8),[9](https://arxiv.org/html/2609.20982#bib.bib7)\]share the same idea, jointly training a control policy alongside an adversary that corrupts the observations fed to the control policy\. ARMOR has been shown to outperform such approaches, so we use it as the state\-of\-the\-art baseline for our comparison\. Further, as anAblation Studyto show the effectiveness ofASGARD’s two\-stage training pipeline, including the encoders and monitor, we consider aBaseline\-RL controllerthat removes both fromASGARD’s architecture in two settings: one with access to privileged action\-attack information during training, and one without\. The results show that both settings perform comparably, confirming that training with privileged information alone, without the encoder \(no latent representation in either case\) and monitor, is not sufficient for resilience against*action\-space attacks*; we therefore report both under the single labelBaseline\-RL\.

We adapt the prior work definition of metrics to evaluateASGARDand its comparisons with prior work\.Mission Success Rateis the fraction of evaluated episodes in which the UAV reaches within its designated waypoint threshold \(ϵ\\epsilon= 5m\[[27](https://arxiv.org/html/2609.20982#bib.bib21),[3](https://arxiv.org/html/2609.20982#bib.bib1)\]\) before the episode ends \(‖d​e​s​t−p​o​s‖≤ϵ\\\|dest\-pos\\\|\\leq\\epsilon, whered​e​s​tdestis the target waypoint andp​o​sposis the UAV’s current position\), without having triggered a crash or timeout termination first\.State Driftis the mean Euclidean distance, in meters, betweenposanddest, computed at each timestep during the attack and averaged across evaluation episodes\.Crash Rateis the fraction of evaluated episodes that terminate specifically because the UAV’s state exceeds a predefined safety bound, such as leaving the valid flight volume or tilting beyond a safe orientation, causing a crash\.

For brevity, we use*Teacher*and*Student*to refer toASGARD’s full pipeline \(encoder, control policy, and monitor\) at each phase\.

### V\-BASGARD Training Performance

To evaluate whetherASGARD’s Student can adapt to the Teacher’s performance, we train the baseline\-RL andASGARD’s Teacher over10×10510\\times 10^\{5\}timesteps under nominal \(attack\-free\) conditions, and then trainASGARD’s Student under the Teacher’s supervision\. All training curves in Figure[2](https://arxiv.org/html/2609.20982#S5.F2)are averaged over five runs with random seeds\. Figure[2](https://arxiv.org/html/2609.20982#S5.F2)\(a\) shows the results\. We make two observations\. First,ASGARD’s Teacher \(blue\) and Student \(yellow\) both reach the maximum episodic reward \(approximately 4000\) within about3×1053\\times 10^\{5\}timesteps, whereas the baseline\-RL approach \(green\) requires roughly7×1057\\times 10^\{5\}timesteps to reach the same level\. Second, the Student closely tracks the Teacher throughout training and both reach the same maximum reward as the baseline\-RL\. This shows that the teacher–student encoding preserves learning performance under attack\-free conditions\.

![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/NormalCondition_with500.jpg)
![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/AdversarialCondition_withactionhist.jpg)

Fig\. 2:Training performance comparison\.Left \(a\):Nominal conditions, all methods achieve similar final performance\.Right \(b\):Adversarial conditions with*action\-space attacks*, bothASGARDTeacher and Student converge faster than ARMOR while ARMOR has many fluctuations\.To evaluateASGARD’s effectiveness under*action\-space attacks*, we train bothASGARDand ARMOR under equivalent conditions, giving each teacher access to the privileged information and then supervising their respective students\. Figure[2](https://arxiv.org/html/2609.20982#S5.F2)\(b\) shows the resulting training curves, with shaded regions indicating variation across the five runs\. We can see that ARMOR’s Teacher and Student both struggle to converge, reaching a maximum episodic reward of approximately 4000 after100×105100\\times 10^\{5\}timesteps and with large fluctuations across runs\.ASGARD, in contrast, converges to a higher episodic reward of approximately 4500 in under20×10520\\times 10^\{5\}timesteps \(5×5\\timesfaster\) with much tighter variance across runs\. Overall, these results show thatASGARDis effective under both nominal and adversarial conditions, outperforming both ARMOR and the baseline\-RL, and that the Student reproduces the Teacher’s action\-attack resilience while relying on only the UAV’s physical state history\. In the remaining subsections, we focus on the Student \(the pipeline actually deployed on the device\) and refer to it asASGARD\.

### V\-CASGARD under Action Attacks

To evaluateASGARD’s effectiveness under*action\-space attacks*, we run the four single\-channel attacks \(Pitch, Roll, Thrust, Gain\) and the combined*All*attack from Table[I](https://arxiv.org/html/2609.20982#S5.T1)\. As Table[II](https://arxiv.org/html/2609.20982#S5.T2)shows,ASGARDsuccessfully completes an average of 95% of missions across the four single\-channel attacks without any crashes, while achieving the lowest state drift \(∼\\sim0\.30m on average\)\. Furthermore, under the*All*attack, which is also difficult for an attacker to carry out in practice,ASGARDsuccessfully finishes 67% of missions with only a 10% crash rate while keeping the state drift small \(∼\\sim0\.26m\)\.

To visualizeASGARD’s actions against one of the*action\-space attacks*discussed in Table[I](https://arxiv.org/html/2609.20982#S5.T1), we run the*All*action attack on a mission that follows the blue line and then turns right along the red line, shown in Figure[3](https://arxiv.org/html/2609.20982#S5.F3)together with each technique’s resulting trajectory in orange, for all three techniques: baseline\-RL \(top\), ARMOR \(middle\), andASGARD\(bottom\)\. We can see that both baseline\-RL and ARMOR cause the UAV to become unstable and crash, failing to finish the mission, whileASGARDmaintains the UAV’s flight and stays on course until the mission completes\.

![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/baseline_1.png)![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/baseline_2.png)![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/baseline_3.png)![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/baseline_4.png)![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/armor_1.png)![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/armor_2.png)![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/armor_3.png)![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/armor_4.png)![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/phoenix_1.png)![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/phoenix_2.png)![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/phoenix_3.png)![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/Environment/phoenix_4.png)Fig\. 3:Flight trajectories under*all*action attack\.Top:Baseline\-RL,Middle:ARMOR, andBottom:ASGARD, which keeps its intended trajectory despite the attack\.
### V\-DComparison with ARMOR and Baseline\-RL

We compareASGARDagainst ARMOR and baseline\-RL controller on the five*action\-space attacks*configurations from Table[I](https://arxiv.org/html/2609.20982#S5.T1), with results shown in Table[II](https://arxiv.org/html/2609.20982#S5.T2)\. Across the four single\-channel attacks, compared to the baseline\-RL controller, ARMOR increases the average success rate from 40\.5% to around 50% and reduces the average crash rate from 47% to 36\.5%, with average state drifts of∼\\sim0\.41m and∼\\sim0\.39m, respectively\. Under the*All*attack, however, both techniques cause every mission to fail \(0% success, 100% crash\), with state drifts of∼\\sim0\.36m and∼\\sim0\.37m\. In contrast,ASGARDachieves an average of 95% success with no crashes across the four single\-channel attacks, and 67% success with a 10% crash rate under the*All*attack, with the lowest average state drift of∼\\sim0\.30m and∼\\sim0\.26m, respectively, for the single channel and all attack\.

Takeaway:These results show that using only RL \(i\.e\., the baseline\-RL controller\), or its refinement with a two\-phase teacher–student pipeline as in ARMOR, is insufficient against*action\-space attacks*and leaves the UAV vulnerable to action\-space attacks\. In contrast,ASGARDmakes the UAV resilient to action\-space attacks, across all five configurations\.

TABLE II:Performance comparison of Baseline\-RL, ARMOR, andASGARDunder action attacks against five UAV action targets\.Target actionBaseline\-RLARMORASGARDSuccessCrashState Drift \(m\)SuccessCrashState Drift \(m\)SuccessCrashState Drift \(m\)Pitch18%69%0\.307±\\pm0\.01225%57%0\.287±\\pm0\.00895%0%0\.159±\\pm0\.004Roll19%80%0\.418±\\pm0\.00323%75%0\.428±\\pm0\.01094%0%0\.393±\\pm0\.003Thrust61%3%0\.608±\\pm0\.01068%0%0\.567±\\pm0\.03497%0%0\.394±\\pm0\.001Gain64%35%0\.304±\\pm0\.00185%14%0\.295±\\pm0\.00194%0%0\.267±\\pm0\.001All0%100%0\.359±\\pm0\.0010%100%0\.368±\\pm0\.00167%10%0\.259±\\pm0\.001![Refer to caption](https://arxiv.org/html/2609.20982v1/Images/attitude_err_throttle_attack.jpg)Fig\. 4:Attitude errors \(degree\) under Thrust action attack for Baseline\-RL \(left\), ARMOR \(middle\), andASGARD\(right\)\.Figure[4](https://arxiv.org/html/2609.20982#S5.F4)illustrates this trend visually, showing the attitude error of the three techniques under the Thrust attack\. Baseline\-RL’s error exceeds±10∘\\pm 10^\{\\circ\}on average, and ARMOR only marginally reduces it to around±10∘\\pm 10^\{\\circ\}, with both exhibiting large fluctuations and errors that cause crashes, consistent with the results in Table[II](https://arxiv.org/html/2609.20982#S5.T2)\.ASGARD, in contrast, keeps the average attitude error below±4∘\\pm 4^\{\\circ\}throughout the attack, demonstrating its resilience against*action\-space attacks*such as the Thrust attack\.

### V\-EASGARD Zero\-Shot Performance

We evaluated the generalization ofASGARDby testing whether it remains resilient against*action\-space attacks*on action channels that were not part of its training data\. We consider two zero\-shot scenarios: in each, we trainASGARDand ARMOR on a single randomly picked action attack \(Pitch in one scenario, Gain in the other\) and evaluate them on the attacks on the remaining three action channels from Table[I](https://arxiv.org/html/2609.20982#S5.T1)\.

Pitch\-trained\.We trainASGARDand ARMOR using Pitch as the training attack and evaluate on Roll, Thrust, and Gain, with the results in Table[III](https://arxiv.org/html/2609.20982#S5.T3)\. ARMOR finishes an average of 49% of missions with a 35% crash rate, whileASGARDfinishes 77% of missions with only 7% crashes and a lower average state drift \(∼\\sim0\.33m vs\.∼\\sim0\.44m for ARMOR\)\.

TABLE III:ASGARDand ARMOR effectiveness when trained onPitchaction attacks only and tested on unseen attacks \(Zero\-shot\)\.TechniquesMetricsRollThrustGainARMORSuccess18%50%80%Crash81%10%15%State Drift \(m\)0\.430±\\pm0\.0140\.597±\\pm0\.0360\.295±\\pm0\.001ASGARDSuccess80%60%91%Crash12%6%5%State Drift \(m\)0\.375±\\pm0\.0630\.356±\\pm0\.0720\.270±\\pm0\.016

Gain\-trained\.We use Gain as the training attack and evaluate on Pitch, Roll, and Thrust, with the results in Table[IV](https://arxiv.org/html/2609.20982#S5.T4)\. ARMOR finishes 31% of missions with a 57% crash rate, whereasASGARDfinishes around 74% of missions with only 9% crashes and a lower average state drift \(∼\\sim0\.34m vs\.∼\\sim0\.44m for ARMOR\)\.

TABLE IV:ASGARDand ARMOR effectiveness when trained onGainaction attacks only and tested on unseen attacks \(Zero\-shot\)\.TechniquesMetricsPitchRollThrustARMORSuccess16%14%64%Crash82%86%4%State Drift \(m\)0\.315±\\pm0\.0090\.436±\\pm0\.0080\.579±\\pm0\.050ASGARDSuccess44%81%96%Crash10%18%0%State Drift \(m\)0\.231±\\pm0\.0060\.385±\\pm0\.0020\.395±\\pm0\.000

Takeaway:Training on a single action attack is sufficient forASGARDto generalize to unseen attack channels, achieving roughly22–3×3\\timesARMOR’s success rate and a55–6×6\\timeslower crash rate\.

### V\-FASGARD under Stealthy Attacks

To evaluate the effectiveness ofASGARDagainst stealthy attacks, we adapt two attack profiles from prior work\[[3](https://arxiv.org/html/2609.20982#bib.bib1)\]— continuously increases the bias at a fixed rate, and increases the bias in discrete increments at fixed intervals — to*action\-space attacks*, within the ranges discussed in Table[I](https://arxiv.org/html/2609.20982#S5.T1)\. As can be seen in Table[V](https://arxiv.org/html/2609.20982#S5.T5), under these gradually accumulating attacks for Roll,ASGARDmaintains a mission success rate of 100% with no crashes, while ARMOR fails on every evaluated episode, resulting in a 0% success rate and a 100% crash rate\.

Takeaway\.ASGARD’s pipeline encodes the history of actions into the student latent; the monitor reads this latent and continuously tracks the bias as it accumulates, adjusting its corrections as the disturbance grows and preventing large deviations\. In contrast, ARMOR lacks any runtime correction mechanism and fails regardless of how gradually the attack develops\. This suggests that the absence of an active correction stage, rather than the abruptness of the disturbance, is the primary factor behind ARMOR’s vulnerability to*action\-space attacks*\.

TABLE V:ASGARDand ARMOR performance under stealthy*action\-space attacks*\.TechniquesSuccessCrashState Drift \(m\)ARMOR0%100%0\.419±\\pm0\.002ASGARD100%0%0\.392±\\pm0\.002

## VIDiscussion & Conclusion

RL controllers for UAVs are vulnerable to*action\-space attacks*, and while prior work addresses this by robustifying the policy through adversarial training, these defenses forward every action directly to the actuators once deployed — with no mechanism to intercept a command that has already been corrupted\.ASGARDcloses this gap through three innovations: an*action\-attack\-aware*latent trained with attack\-specific privileged information, a two\-phase teacher–student scheme that, as shown in Figure[2](https://arxiv.org/html/2609.20982#S5.F2)\(b\), trains more efficiently and with fewer fluctuations than prior work while distilling this capability to a student that runs from physical\-state and action history alone at deployment, and a monitor placed between the control policy and the actuators that uses this latent to intercept and correct corrupted commands before they reach the hardware\. Across attack scenarios where neither ARMOR nor an RL\-only controller is resilient,ASGARDcompletes over 95% of missions under single\-channel attacks and 67% under a combined attack on all four channels\.ASGARDfurther generalizes to attack channels unseen during training and maintains resilience against stealthy attack patterns\. However, extendingASGARDto other types of robotic systems and sim\-to\-real transfer remains an open direction for future work\.

## References

- \[1\]\(2021\)Challenges and countermeasures for adversarial attacks on deep reinforcement learning\.IEEE Transactions on Artificial Intelligence3\(2\),pp\. 90–109\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p1.1),[§III](https://arxiv.org/html/2609.20982#S3.p2.1)\.
- \[2\]T\. E\. Humphreys, B\. M\. Ledvina, M\. L\. Psiaki, B\. W\. O’Hanlon, P\. M\. Kintner,et al\.\(2008\)Assessing the spoofing threat: development of a portable gps civilian spoofer\.InProceedings of the 21st International technical meeting of the satellite division of the institute of navigation \(ION GNSS 2008\),pp\. 2314–2325\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p1.1),[§III](https://arxiv.org/html/2609.20982#S3.p3.1)\.
- \[3\]P\. Dash, E\. Chan, N\. P\. Lawrence, and K\. Pattabiraman\(2025\)ARMOR: robust reinforcement learning\-based control for uavs under physical attacks\.arXiv preprint arXiv:2506\.22423\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p1.1),[§I](https://arxiv.org/html/2609.20982#S1.p2.1),[§I](https://arxiv.org/html/2609.20982#S1.p5.1),[§II\-C](https://arxiv.org/html/2609.20982#S2.SS3.p2.1),[§III](https://arxiv.org/html/2609.20982#S3.p3.1),[§V\-A](https://arxiv.org/html/2609.20982#S5.SS1.p2.1),[§V\-A](https://arxiv.org/html/2609.20982#S5.SS1.p3.1),[§V\-A](https://arxiv.org/html/2609.20982#S5.SS1.p5.1),[§V\-A](https://arxiv.org/html/2609.20982#S5.SS1.p6.1),[§V\-F](https://arxiv.org/html/2609.20982#S5.SS6.p1.1)\.
- \[4\]X\. Y\. Lee, S\. Ghadai, K\. L\. Tan, C\. Hegde, and S\. Sarkar\(2020\)Spatiotemporally constrained action space attacks on deep reinforcement learning agents\.InProceedings of the AAAI conference on artificial intelligence,Vol\.34,pp\. 4577–4584\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p1.1),[§I](https://arxiv.org/html/2609.20982#S1.p3.1),[§III](https://arxiv.org/html/2609.20982#S3.p4.1)\.
- \[5\]H\. Alemzadeh, D\. Chen, X\. Li, T\. Kesavadas, Z\. T\. Kalbarczyk, and R\. K\. Iyer\(2016\)Targeted attacks on teleoperated surgical robots: dynamic model\-based detection and mitigation\.In2016 46th annual IEEE/IFIP international conference on dependable systems and networks \(DSN\),pp\. 395–406\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p1.1),[§I](https://arxiv.org/html/2609.20982#S1.p3.1),[§II\-B](https://arxiv.org/html/2609.20982#S2.SS2.p1.1)\.
- \[6\]D\. Chen, B\. Zhou, V\. Koltun, and P\. Krähenbühl\(2020\)Learning by cheating\.InConference on robot learning,pp\. 66–75\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p2.1)\.
- \[7\]J\. Lee, J\. Hwangbo, L\. Wellhausen, V\. Koltun, and M\. Hutter\(2020\)Learning quadrupedal locomotion over challenging terrain\.Science robotics5\(47\),pp\. eabc5986\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p2.1),[§I](https://arxiv.org/html/2609.20982#S1.p5.1),[§III](https://arxiv.org/html/2609.20982#S3.p1.1),[§IV\-B](https://arxiv.org/html/2609.20982#S4.SS2.p1.1)\.
- \[8\]J\. Hwangbo, I\. Sa, R\. Siegwart, and M\. Hutter\(2017\)Control of a quadrotor with reinforcement learning\.IEEE Robotics and Automation Letters2\(4\),pp\. 2096–2103\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p2.1),[§III](https://arxiv.org/html/2609.20982#S3.p1.1)\.
- \[9\]L\. Pinto, J\. Davidson, R\. Sukthankar, and A\. Gupta\(2017\)Robust adversarial reinforcement learning\.InInternational conference on machine learning,pp\. 2817–2826\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p2.1),[§II\-C](https://arxiv.org/html/2609.20982#S2.SS3.p2.1),[§III](https://arxiv.org/html/2609.20982#S3.p1.1),[§III](https://arxiv.org/html/2609.20982#S3.p3.1),[§V\-A](https://arxiv.org/html/2609.20982#S5.SS1.p5.1)\.
- \[10\]S\. Chen, J\. Xu, E\. C\. Sezer, P\. Gauriar, and R\. K\. Iyer\(2005\)Non\-control\-data attacks are realistic threats\.In14th USENIX Security Symposium \(USENIX Security 05\),Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p3.1),[§II\-B](https://arxiv.org/html/2609.20982#S2.SS2.p1.1),[§III](https://arxiv.org/html/2609.20982#S3.p4.1)\.
- \[11\]L\. Szekeres, M\. Payer, T\. Wei, and D\. Song\(2013\)Sok: eternal war in memory\.In2013 IEEE Symposium on Security and Privacy,pp\. 48–62\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p3.1)\.
- \[12\]C\. Tessler, Y\. Efroni, and S\. Mannor\(2019\)Action robust reinforcement learning and applications in continuous control\.InInternational Conference on Machine Learning,pp\. 6215–6224\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p3.1),[§III](https://arxiv.org/html/2609.20982#S3.p4.1)\.
- \[13\]K\. L\. Tan, Y\. Esfandiari, X\. Y\. Lee, S\. Sarkar,et al\.\(2020\)Robustifying reinforcement learning agents via action space adversarial training\.In2020 American control conference \(ACC\),pp\. 3959–3964\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p3.1),[§III](https://arxiv.org/html/2609.20982#S3.p4.1)\.
- \[14\]X\. Y\. Lee, Y\. Esfandiari, K\. L\. Tan, and S\. Sarkar\(2021\)Query\-based targeted action\-space adversarial policies on deep reinforcement learning agents\.InProceedings of the ACM/IEEE 12th international conference on cyber\-physical systems,pp\. 87–97\.Cited by:[§I](https://arxiv.org/html/2609.20982#S1.p3.1),[§III](https://arxiv.org/html/2609.20982#S3.p4.1)\.
- \[15\]E\. Kaufmann, L\. Bauersfeld, A\. Loquercio, M\. Müller, V\. Koltun, and D\. Scaramuzza\(2023\)Champion\-level drone racing using deep reinforcement learning\.Nature620\(7976\),pp\. 982–987\.Cited by:[§III](https://arxiv.org/html/2609.20982#S3.p1.1)\.
- \[16\]T\. Fan, P\. Long, W\. Liu, J\. Pan, R\. Yang, and D\. Manocha\(2020\)Learning resilient behaviors for navigation under uncertainty\.In2020 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 5299–5305\.Cited by:[§III](https://arxiv.org/html/2609.20982#S3.p1.1)\.
- \[17\]D\. Sacerdoti, F\. Benzi, and C\. Secchi\(2024\)A reinforcement learning\-based control strategy for robust interaction of robotic systems with uncertain environments\.In2024 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 5788–5794\.Cited by:[§III](https://arxiv.org/html/2609.20982#S3.p1.1)\.
- \[18\]R\. Cheng, G\. Orosz, R\. M\. Murray, and J\. W\. Burdick\(2019\)End\-to\-end safe reinforcement learning through barrier functions for safety\-critical continuous control tasks\.InProceedings of the AAAI conference on artificial intelligence,Vol\.33,pp\. 3387–3395\.Cited by:[§III](https://arxiv.org/html/2609.20982#S3.p1.1)\.
- \[19\]S\. Huang, N\. Papernot, I\. Goodfellow, Y\. Duan, and P\. Abbeel\(2017\)Adversarial attacks on neural network policies\.arXiv preprint arXiv:1702\.02284\.Cited by:[§III](https://arxiv.org/html/2609.20982#S3.p2.1)\.
- \[20\]E\. Aljalbout, F\. Frank, M\. Karl, and P\. van der Smagt\(2024\)On the role of the action space in robot manipulation learning and sim\-to\-real transfer\.IEEE Robotics and Automation Letters9\(6\),pp\. 5895–5902\.Cited by:[§IV\-B](https://arxiv.org/html/2609.20982#S4.SS2.p1.1)\.
- \[21\]D\. P\. Kingma and M\. Welling\(2013\)Auto\-encoding variational bayes\.arXiv preprint arXiv:1312\.6114\.Cited by:[§IV\-B](https://arxiv.org/html/2609.20982#S4.SS2.p2.1)\.
- \[22\]J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov\(2017\)Proximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§IV\-B](https://arxiv.org/html/2609.20982#S4.SS2.p3.1)\.
- \[23\]J\. Panerati, H\. Zheng, S\. Zhou, J\. Xu, A\. Prorok, and A\. P\. Schoellig\(2021\)Learning to fly—a gym environment with pybullet physics for reinforcement learning of multi\-agent quadcopter control\.In2021 IEEE/RSJ International Conference on Intelligent Robots and Systems \(IROS\),pp\. 7512–7519\.Cited by:[§V\-A](https://arxiv.org/html/2609.20982#S5.SS1.p2.1)\.
- \[24\]G\. Brockman, V\. Cheung, L\. Pettersson, J\. Schneider, J\. Schulman, J\. Tang, and W\. Zaremba\(2016\)Openai gym\.arXiv preprint arXiv:1606\.01540\.Cited by:[§V\-A](https://arxiv.org/html/2609.20982#S5.SS1.p2.1)\.
- \[25\]Z\. Yuan, A\. W\. Hall, S\. Zhou, L\. Brunke, M\. Greeff, J\. Panerati, and A\. P\. Schoellig\(2022\)Safe\-control\-gym: a unified benchmark suite for safe learning\-based control and reinforcement learning in robotics\.IEEE Robotics and Automation Letters7\(4\),pp\. 11142–11149\.Cited by:[§V\-A](https://arxiv.org/html/2609.20982#S5.SS1.p2.1)\.
- \[26\]F\. Fei, Z\. Tu, D\. Xu, and X\. Deng\(2020\)Learn\-to\-recover: retrofitting uavs with reinforcement learning\-assisted flight control under cyber\-physical attacks\.In2020 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 7358–7364\.Cited by:[§V\-A](https://arxiv.org/html/2609.20982#S5.SS1.p5.1)\.
- \[27\]P\. Dash, E\. Chan, and K\. Pattabiraman\(2024\)Specguard: specification aware recovery for robotic autonomous vehicles from physical attacks\.InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security,pp\. 1849–1863\.Cited by:[§V\-A](https://arxiv.org/html/2609.20982#S5.SS1.p6.1)\.