DreamGuard: Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model

# Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model
Source: [https://arxiv.org/html/2608.05695](https://arxiv.org/html/2608.05695)
Wenhao Lin1, Chenyu Yu1, Xingwei Lin1\\corresponding, Sicong Cao2\\corresponding, Xiang Chen1, Lei Xue3, Le Yu2, Letian Sha2, Chunming Wu1

###### Abstract

As large language model \(LLM\) agents increasingly invoke external tools and interact with real\-world systems, unsafe actions may cause irreversible consequences on external states, user data, and downstream services\. Recent runtime guardrails mitigate such risks by checking proposed actions before execution, but many remain reactive: they primarily assess the apparent safety of the current action, lacking an explicit model of how risk evolves across the trajectory\. This limitation creates a critical blind spot for long\-horizon risks, where individually benign\-looking actions can gradually drift the agent toward hazardous states\. In response, we proposeDreamGuard, a proactive guardrail for LLM agents built around a risk\-aware world model\. The world model maintains a compact recurrent latent state over the trajectory and predicts future latent states from whichDreamGuardderives immediate\-hazard and prefix\-risk evidence\. It then fuses these multi\-horizon signals into intervention decisions before execution\. Experiments across four benchmarks and an online guardrail evaluation show thatDreamGuardoutperforms generic, reactive, and proactive guardrail baselines, achieves the best safety\-utility trade\-off among evaluated guardrails, and maintains an average end\-to\-end latency of 25 ms per call\.

## 1Introduction

Large language model \(LLM\) agents can now autonomously perform complex tasks in open\-ended environments by invoking external tools and interacting with real\-world systems\(Xiet al\.[2025](https://arxiv.org/html/2608.05695#bib.bib36); Qinet al\.[2024](https://arxiv.org/html/2608.05695#bib.bib37); Wanget al\.[2024](https://arxiv.org/html/2608.05695#bib.bib38)\)\. As these actions may affect external states, user data, and downstream services, ensuring agent runtime safety has become a critical concern\(Maet al\.[2025](https://arxiv.org/html/2608.05695#bib.bib30); Yuet al\.[2025](https://arxiv.org/html/2608.05695#bib.bib31); Ganet al\.[2026](https://arxiv.org/html/2608.05695#bib.bib32); Denget al\.[2025](https://arxiv.org/html/2608.05695#bib.bib35)\)\.

Recent research has introduced runtime guardrails for LLM agents, which assess each proposed action in its trajectory context and apply interventions to unsafe ones before execution\(Xianget al\.[2025](https://arxiv.org/html/2608.05695#bib.bib4); Chenet al\.[2025](https://arxiv.org/html/2608.05695#bib.bib5); Wenet al\.[2026](https://arxiv.org/html/2608.05695#bib.bib6)\)\. These guardrails can intercept immediate hazards, which would directly cause a hazardous outcome if executed at the current step, such as deleting files, leaking credentials, or invoking an unauthorized external API\.

However, many existing runtime guardrails remain reactive: they assess the apparent safety of the current proposed action, without modeling how risk evolves over the trajectory or how the action may change future risk\. This limitation creates a critical blind spot for long\-horizon risks, where early steps may appear benign in isolation, but gradually accumulate toward a hazardous outcome\. For example, an agent may locate an internal file, move it into a workspace folder, and later create a public sharing link; each operation can look routine locally, but the accumulated trajectory turns the final action into confidential data exposure\. A guardrail that primarily assesses each proposed action based on its local state struggles to detect such risk accumulation\.

In response, recent work has shifted toward proactive guardrails, which incorporate trajectory context or look\-ahead predictions to assess how current actions may affect future risk\. Yet, they still face several challenges in real\-world deployments\.\(i\) Runtime guardrails must be efficient and lightweight\.Many proactive guardrails rely on LLMs to process trajectory histories\(Liet al\.[2026](https://arxiv.org/html/2608.05695#bib.bib20); Chenet al\.[2026](https://arxiv.org/html/2608.05695#bib.bib19)\)\. As the agent trajectory grows, these designs become constrained by context window length and require increasingly expensive autoregressive inference, often leading to multi\-second latency per step\. This strains the latency budget of real\-time agent execution\.\(ii\) Prediction should preserve risk evidence\.Proactive guardrails may predict future states or consequences and then assess them with separate safety rules or judges\(Liuet al\.[2026c](https://arxiv.org/html/2608.05695#bib.bib27); Wanget al\.[2025b](https://arxiv.org/html/2608.05695#bib.bib34)\)\. However, this separation can make the risk assessment depend on an incomplete prediction: when the prediction is not learned for risk discrimination, the safety assessment may lack the evidence needed to identify underlying risks\. This calls for risk\-aware latent states that expose hazardous transitions, rather than merely predicting generic future outcomes\.\(iii\) Runtime intervention requires calibrated multi\-horizon risk\.Proactive guardrails increasingly model risk evolution through trajectory memory, state tracking, or future\-risk estimation\(Liet al\.[2026](https://arxiv.org/html/2608.05695#bib.bib20); Dhodapkar and Pishori[2026](https://arxiv.org/html/2608.05695#bib.bib14); Wanget al\.[2026b](https://arxiv.org/html/2608.05695#bib.bib33)\)\. However, long\-horizon evidence is often weak before hazards become explicit, while immediate\-hazard evidence requires decisive intervention at the current action boundary\. The challenge is to calibrate these complementary signals into an intervention rule that warns early without triggering excessive false alarms\.

To address these challenges, we proposeDreamGuard, an efficient runtime guardrail for LLM agents built around a risk\-aware world model\. Specifically, to overcome the efficiency challenge \(i\),DreamGuardmaintains a fixed\-dimensional state through GRU\-based recurrent dynamics, bypassing the need to repeatedly process growing trajectories via LLMs\. To tackle the risk evidence challenge \(ii\),DreamGuardlearns latent dynamics designed to explicitly preserve evidence of hazardous transitions in the predicted latent states\. Finally, to address the multi\-horizon risk intervention challenge \(iii\),DreamGuardintegrates both immediate\-hazard and prefix\-risk signals into a robust decision mechanism\.

We evaluateDreamGuardon four agent safety benchmarks covering both immediate hazards and long\-horizon risks\. Experiments show thatDreamGuardoutperforms generic, reactive, and proactive guardrail baselines, while reducing average end\-to\-end latency to 25 ms per call\. It further intervenes before the first hazard action in 96\.3% of unsafe long\-horizon trajectories\. In our online guardrail evaluation,DreamGuardachieves the best safety–utility trade\-off, reaching a 72\.92% safety rate while preserving 90\.38% task utility\.

Our contributions are summarized as follows:

- •We introduceDreamGuard, an efficient runtime guardrail built around a lightweight risk\-aware world model that maintains a compact recurrent state and estimates multi\-horizon risk before action execution\. This design enables proactive intervention against both immediate hazards and long\-horizon risks during real\-time agent execution\.
- •We conduct extensive evaluations on four agent safety benchmarks\. Experiments show thatDreamGuardoutperforms guardrail baselines in mitigating multi\-horizon risks, achieves earlier interventions against long\-horizon risks, and maintains highly efficient end\-to\-end latency\.
- •We further validateDreamGuardin an online guardrail evaluation, where it achieves the best safety–utility trade\-off among evaluated guardrails, demonstrating its practical effectiveness during real\-time agent execution\.

## 2Related Work

#### Guardrails for LLM Agents\.

Generic LLM guardrails such as LlamaGuard\(Inanet al\.[2023](https://arxiv.org/html/2608.05695#bib.bib1)\)and WildGuard\(Hanet al\.[2024](https://arxiv.org/html/2608.05695#bib.bib3)\)focus on content moderation, classifying prompts or responses against harmful\-content taxonomies\. LLM agents fundamentally alter this paradigm: their outputs become actions that affect external environments, shifting the objective of guardrails from content moderation to action safety\. For instance, GuardAgent\(Xianget al\.[2025](https://arxiv.org/html/2608.05695#bib.bib4)\)and AGrail\(Luoet al\.[2025b](https://arxiv.org/html/2608.05695#bib.bib7)\)translate task\-specific safety requirements into executable action checks, while ShieldAgent\(Chenet al\.[2025](https://arxiv.org/html/2608.05695#bib.bib5)\)verifies trajectories against policy\-derived temporal rules\. Similarly, TS\-Guard\(Mouet al\.[2026](https://arxiv.org/html/2608.05695#bib.bib9)\)and AgentDoG\(Liuet al\.[2026b](https://arxiv.org/html/2608.05695#bib.bib10)\)train specialized guard models for verifying action and trajectory safety\. However, these methods predominantly operate as reactive guardrails, failing to effectively identify long\-horizon risks that emerge over multi\-step agent interactions\.

To mitigate long\-horizon risks, a further line of work is moving toward proactive guardrailing by incorporating trajectory context or look\-ahead prediction\. For instance, SafetyDrift\(Dhodapkar and Pishori[2026](https://arxiv.org/html/2608.05695#bib.bib14)\)and ProbGuard\(Wanget al\.[2025a](https://arxiv.org/html/2608.05695#bib.bib18)\)estimate future violation probabilities over hand\-crafted discrete or symbolic safety states, while MAGE\(Wanget al\.[2026b](https://arxiv.org/html/2608.05695#bib.bib33)\)and SafeAgent\(Liuet al\.[2026c](https://arxiv.org/html/2608.05695#bib.bib27)\)maintain safety\-relevant memory or runtime state for risk assessment\. Recent world model\-based guardrails such as SafePred\(Chenet al\.[2026](https://arxiv.org/html/2608.05695#bib.bib19)\)and SafeMCP\(Wanget al\.[2026a](https://arxiv.org/html/2608.05695#bib.bib26)\)further use LLM look\-ahead reasoning for future risk prediction\. Despite these advances, these methods still struggle to provide efficient runtime protection against multi\-horizon risks, often relying on LLM\-based reasoning or domain\-specific action spaces\.

#### World Models for Agent Safety\.

For proactive agent guardrailing, anticipating the consequences of proposed actions is essential to intervene before unsafe outcomes occur\. World models have long been used in model\-based decision making to capture environment dynamics and support planning over predicted futures\. Latent world models such as PlaNet\(Hafneret al\.[2019](https://arxiv.org/html/2608.05695#bib.bib21)\), DreamerV3\(Hafneret al\.[2025](https://arxiv.org/html/2608.05695#bib.bib22)\), and TD\-MPC2\(Hansenet al\.[2024](https://arxiv.org/html/2608.05695#bib.bib23)\)learn compact latent dynamics from interaction data and use predicted futures for planning or control\. This idea has also been adapted to LLM agents: WebDreamer\(Guet al\.[2025](https://arxiv.org/html/2608.05695#bib.bib24)\)and WMA\(Chaeet al\.[2025](https://arxiv.org/html/2608.05695#bib.bib25)\)simulate action outcomes for web navigation\. Beyond task performance, SafePred\(Chenet al\.[2026](https://arxiv.org/html/2608.05695#bib.bib19)\)uses an LLM\-based world model to predict action consequences and feed risk estimates into a decision loop, and SafeMCP\(Wanget al\.[2026a](https://arxiv.org/html/2608.05695#bib.bib26)\)relies on LLM\-based look\-ahead reasoning over environment dynamics\. Despite these advances, existing world models for LLM agent safety often rely on LLM\-based look\-ahead reasoning, which can incur high per\-action cost\.DreamGuardinstead uses lightweight GRU\-based recurrent dynamics to maintain a compact risk state from trajectory data, enabling efficient risk estimation for runtime intervention\.

## 3Methodology

![Refer to caption](https://arxiv.org/html/2608.05695v1/x1.png)Figure 1:Overview ofDreamGuard\. \(1\) Risk\-aware World Model: encodes the trajectory prefix and proposed action into a recurrent latent state, predicts the successor latent state, and produces immediate\-hazard and prefix\-risk scores\. \(2\) Multi\-horizon Risk Estimation: aggregates temporal risk evidence and applies a calibrated intervention rule to output a runtime decision before action execution\.### 3\.1Problem Formulation

We consider an LLM agent that interacts with an external environment over a trajectoryiiof lengthTiT\_\{i\}\. Runtime risks may arise from unsafe user intent, adversarial attack, or imperfect agent reasoning\. These risks can be immediate, where an action is hazardous at the current step, or long\-horizon, where an action may appear benign locally but contributes to a hazardous outcome through the accumulated trajectory context\. We formulate guardrailing as a step\-level decision problem\. At each steptt, before the proposed action is executed, the guardrail𝒢\\mathcal\{G\}receives

xi,t=\(e,ℋi,t,ai,t\),x\_\{i,t\}=\(e,\\mathcal\{H\}\_\{i,t\},a\_\{i,t\}\),\(1\)whereeedenotes the task instruction given to the agent,ai,ta\_\{i,t\}denotes the agent’s proposed action at steptt, andoi,to\_\{i,t\}denotes the current environment observation available to the agent before it executesai,ta\_\{i,t\}\. The trajectory prefixℋi,t=\(oi,1,ai,1,oi,2,…,ai,t−1,oi,t\)\\mathcal\{H\}\_\{i,t\}=\(o\_\{i,1\},a\_\{i,1\},o\_\{i,2\},\\ldots,a\_\{i,t\-1\},o\_\{i,t\}\)contains the interaction history available up toai,ta\_\{i,t\}\. The guardrail outputs a step\-level decision:

di,t=𝒢​\(xi,t\)∈\{PASS,HOLD,BLOCK\}\.d\_\{i,t\}=\\mathcal\{G\}\(x\_\{i,t\}\)\\in\\\{\\mathrm\{PASS\},\\mathrm\{HOLD\},\\mathrm\{BLOCK\}\\\}\.\(2\)PASS\\mathrm\{PASS\}allows the action to execute,HOLD\\mathrm\{HOLD\}interrupts execution when the evidence is low\-confidence or reflects accumulated risk and triggers a conservative intervention, andBLOCK\\mathrm\{BLOCK\}interrupts execution and prevents the action\.

For a hazard trajectory, lettihazt\_\{i\}^\{\\mathrm\{haz\}\}denote the first hazard step, the earliest step whose proposed action would trigger a concrete hazardous outcome or transition if executed\. The primary runtime objective is to estimate step\-level risk and intervene before unsafe actions are executed, returning eitherHOLD\\mathrm\{HOLD\}orBLOCK\\mathrm\{BLOCK\}attihazt\_\{i\}^\{\\mathrm\{haz\}\}while minimizing false alerts on safe trajectories\.

### 3\.2Overview ofDreamGuard

The goal ofDreamGuardis to equip runtime guardrails with proactive risk prediction, enabling them to identify proposed actions that may cause hazardous outcomes\. As shown in Figure[1](https://arxiv.org/html/2608.05695#S3.F1),DreamGuardconsists of two modules\. The risk\-aware world model maintains a compact latent state over the trajectory, and estimates immediate\-hazard and prefix\-risk evidence from the predicted latent state\. The multi\-horizon risk estimation module then aggregates temporal evidence and maps the fused score to calibrated intervention decisions\. Algorithm[1](https://arxiv.org/html/2608.05695#alg1)formalizes the runtime workflow\.

### 3\.3Risk\-Aware World Model

#### Recurrent World Model Architecture\.

Learning a generative model by reconstructing raw future observations provides a rich signal, but accurate raw reconstruction is costly and does not directly determine whether an agent action is safe\. For runtime guardrailing, the model must instead capture how the trajectory prefix evolves under proposed actions and preserve the safety\-relevant information needed for intervention\. Inspired by DreamerV3\(Hafneret al\.[2025](https://arxiv.org/html/2608.05695#bib.bib22)\)and TD\-MPC2\(Hansenet al\.[2024](https://arxiv.org/html/2608.05695#bib.bib23)\),DreamGuardbuilds a recurrent state\-space world model \(RSSM\) with implicit, risk\-aware latent dynamics for predicting successor states and estimating risk before action execution\.

TheDreamGuardworld model architecture is shown in Figure[1](https://arxiv.org/html/2608.05695#S3.F1)and consists of six components:

Sequence model:ht=fθ​\(ht−1,zt−1,at−1\),\\displaystyle\\quad h\_\{t\}=f\_\{\\theta\}\(h\_\{t\-1\},z\_\{t\-1\},a\_\{t\-1\}\),\(3\)Representation model:zt∼qθ​\(zt∣ht,ot\),\\displaystyle\\quad z\_\{t\}\\sim q\_\{\\theta\}\(z\_\{t\}\\mid h\_\{t\},o\_\{t\}\),Dynamics predictor:z^t∼pθ​\(z^t∣ht\),\\displaystyle\\quad\\hat\{z\}\_\{t\}\\sim p\_\{\\theta\}\(\\hat\{z\}\_\{t\}\\mid h\_\{t\}\),Observation predictor:o^t\+1=gθobs​\(st\+1\),\\displaystyle\\quad\\hat\{o\}\_\{t\+1\}=g^\{\\mathrm\{obs\}\}\_\{\\theta\}\(s\_\{t\+1\}\),Immediate\-hazard predictor:ℓthaz=gθhaz​\(s^t\+1,at,e\),\\displaystyle\\quad\\ell^\{\\mathrm\{haz\}\}\_\{t\}=g^\{\\mathrm\{haz\}\}\_\{\\theta\}\(\\hat\{s\}\_\{t\+1\},a\_\{t\},e\),Prefix\-risk predictor:ℓtpre=gθpre​\(s^t\+1,at,e\)\.\\displaystyle\\quad\\ell^\{\\mathrm\{pre\}\}\_\{t\}=g^\{\\mathrm\{pre\}\}\_\{\\theta\}\(\\hat\{s\}\_\{t\+1\},a\_\{t\},e\)\.Here,∼\\simmarks stochastic RSSM variables drawn from posterior or prior distributions, whereas==marks deterministic neural readouts\. At steptt, the representation model conditions on the current observation to inferztz\_\{t\}; together with the recurrent statehth\_\{t\}, which summarizes the trajectory prefix, this gives the posterior model statest=\(ht,zt\)s\_\{t\}=\(h\_\{t\},z\_\{t\}\)\. The proposed actionata\_\{t\}is then fed to the sequence model to obtainht\+1h\_\{t\+1\}, from which the dynamics predictor produces the successor priorz^t\+1\\hat\{z\}\_\{t\+1\}\. The resulting prior states^t\+1=\(ht\+1,z^t\+1\)\\hat\{s\}\_\{t\+1\}=\(h\_\{t\+1\},\\hat\{z\}\_\{t\+1\}\)supports immediate\-hazard and prefix\-risk estimation before execution\. During training, the observation predictor also uses the successor posterior statest\+1=\(ht\+1,zt\+1\)s\_\{t\+1\}=\(h\_\{t\+1\},z\_\{t\+1\}\), providing an auxiliary prediction objective for shaping the latent dynamics\.

DreamGuarduses two training stages: world model pretraining and risk\-supervised training\. The first stage learns recurrent latent dynamics from trajectory data, while the second stage shapes the learned state with immediate\-hazard and prefix\-risk supervision\.

#### World Model Pretraining\.

Given a trajectory sequence sampled from the world model training dataset𝒟wm\\mathcal\{D\}\_\{\\mathrm\{wm\}\},DreamGuardoptimizes the world model parametersθ\\thetawith the following objective\. Letmtwm∈\{0,1\}m^\{\\mathrm\{wm\}\}\_\{t\}\\in\\\{0,1\\\}denote the pretraining mask at steptt\. The objective is

ℒWM\(θ\)=𝔼τ∼𝒟wm,z∼qθ\[1∑t=1Tmtwm∑t=1Tmtwm\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{WM\}\}\(\\theta\)=\\mathbb\{E\}\_\{\\tau\\sim\\mathcal\{D\}\_\{\\mathrm\{wm\}\},\\,z\\sim q\_\{\\theta\}\}\\Bigg\[\\frac\{1\}\{\\sum\_\{t=1\}^\{T\}m^\{\\mathrm\{wm\}\}\_\{t\}\}\\sum\_\{t=1\}^\{T\}m^\{\\mathrm\{wm\}\}\_\{t\}\(4\)×\(λpredℒpredt\+λdynℒdynt\+λrepℒrept\)\]\.\\displaystyle\\qquad\\times\\big\(\\lambda\_\{\\mathrm\{pred\}\}\\mathcal\{L\}^\{t\}\_\{\\mathrm\{pred\}\}\+\\lambda\_\{\\mathrm\{dyn\}\}\\mathcal\{L\}^\{t\}\_\{\\mathrm\{dyn\}\}\+\\lambda\_\{\\mathrm\{rep\}\}\\mathcal\{L\}^\{t\}\_\{\\mathrm\{rep\}\}\\big\)\\Bigg\]\.The prediction lossℒpredt\\mathcal\{L\}^\{t\}\_\{\\mathrm\{pred\}\}operates in embedding space, training the observation predictor to predict the embedding of the observation after action execution rather than the raw observation itself:

ℒpredt=1−cos⁡\(o^¯t\+1,o¯t\+1\),\\mathcal\{L\}^\{t\}\_\{\\mathrm\{pred\}\}=1\-\\cos\(\\bar\{\\hat\{o\}\}\_\{t\+1\},\\bar\{o\}\_\{t\+1\}\),\(5\)whereo^¯t\+1\\bar\{\\hat\{o\}\}\_\{t\+1\}ando¯t\+1\\bar\{o\}\_\{t\+1\}denote batch\-centered prediction and target embeddings\. The dynamics and representation losses regularize the stochastic state with opposite stop\-gradient directions:

ℒdynt\\displaystyle\\mathcal\{L\}^\{t\}\_\{\\mathrm\{dyn\}\}=max\(KL\[sg\(qθ\(zt∣ht,ot\)\)∥pθ\(z^t∣ht\)\],κ\),\\displaystyle=\\max\\\!\\Big\(\\mathrm\{KL\}\\\!\\big\[\\mathrm\{sg\}\(q\_\{\\theta\}\(z\_\{t\}\\mid h\_\{t\},o\_\{t\}\)\)\\,\\\|\\,p\_\{\\theta\}\(\\hat\{z\}\_\{t\}\\mid h\_\{t\}\)\\big\],\\kappa\\Big\),\(6\)ℒrept\\displaystyle\\mathcal\{L\}^\{t\}\_\{\\mathrm\{rep\}\}=max\(KL\[qθ\(zt∣ht,ot\)∥sg\(pθ\(z^t∣ht\)\)\],κ\)\.\\displaystyle=\\max\\\!\\Big\(\\mathrm\{KL\}\\\!\\big\[q\_\{\\theta\}\(z\_\{t\}\\mid h\_\{t\},o\_\{t\}\)\\,\\\|\\,\\mathrm\{sg\}\(p\_\{\\theta\}\(\\hat\{z\}\_\{t\}\\mid h\_\{t\}\)\)\\big\],\\kappa\\Big\)\.wheresg​\(⋅\)\\mathrm\{sg\}\(\\cdot\)denotes stop\-gradient andκ\\kappais the free\-bits threshold\. The dynamics lossℒdynt\\mathcal\{L\}^\{t\}\_\{\\mathrm\{dyn\}\}trains the prior to predict posterior states, while the representation lossℒrept\\mathcal\{L\}^\{t\}\_\{\\mathrm\{rep\}\}encourages posterior states to remain predictable rather than encoding arbitrary observation details\.

#### Risk\-Supervised Training\.

After world model pretraining,DreamGuardhas learned recurrent latent dynamics from trajectory data\. Risk\-supervised training then shapes the predicted successor states with two safety targets\. Letyi,thazy^\{\\mathrm\{haz\}\}\_\{i,t\}denote the hazardous label for trajectoryiiat steptt\.

For prefix\-risk supervision, we assign a decayed positive target to precursor steps within horizonKKand a target value of 1 to immediately hazardous steps:

yi,tpre=\{1,yi,thaz=1,exp⁡\(−\(tihaz−t−1\)/ρ\),1≤tihaz−t≤K,0,otherwise,y^\{\\mathrm\{pre\}\}\_\{i,t\}=\\begin\{cases\}1,&y^\{\\mathrm\{haz\}\}\_\{i,t\}=1,\\\\ \\exp\\left\(\-\(\{t\_\{i\}^\{\\mathrm\{haz\}\}\-t\-1\}\)/\\rho\\right\),&1\\leq t\_\{i\}^\{\\mathrm\{haz\}\}\-t\\leq K,\\\\ 0,&\\text\{otherwise\},\\end\{cases\}\(7\)where decay temperatureρ\\rhocontrols how quickly the soft prefix label decays with distance fromtihazt\_\{i\}^\{\\mathrm\{haz\}\}\. Trajectories without a hazard haveyi,tpre=0y^\{\\mathrm\{pre\}\}\_\{i,t\}=0for all steps\.

These two safety targets are applied only on the supervised prefix of each trajectory\. Letmtsafem^\{\\mathrm\{safe\}\}\_\{t\}denote the resulting mask over supervised steps\. The safety objective averages the immediate\-hazard and prefix\-risk losses over this supervised prefix:

ℒsafety\(θ\)=𝔼τ∼𝒟safety,z∼qθ,z^∼pθ\[1∑t=1Tmtsafe\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{safety\}\}\(\\theta\)=\\mathbb\{E\}\_\{\\tau\\sim\\mathcal\{D\}\_\{\\mathrm\{safety\}\},\\,z\\sim q\_\{\\theta\},\\,\\hat\{z\}\\sim p\_\{\\theta\}\}\\Bigg\[\\frac\{1\}\{\\sum\_\{t=1\}^\{T\}m^\{\\mathrm\{safe\}\}\_\{t\}\}\(8\)×∑t=1Tmtsafe\(ℒhazt\+ℒpret\)\]\.\\displaystyle\\qquad\\times\\sum\_\{t=1\}^\{T\}m^\{\\mathrm\{safe\}\}\_\{t\}\\big\(\\mathcal\{L\}^\{t\}\_\{\\mathrm\{haz\}\}\+\\mathcal\{L\}^\{t\}\_\{\\mathrm\{pre\}\}\\big\)\\Bigg\]\.The two losses supervise complementary signals: immediate\-hazard risk for actions that would trigger a hazardous outcome at the current step, and prefix\-risk for trajectory prefixes indicating accumulating long\-horizon risk:

ℒhazt\\displaystyle\\mathcal\{L\}^\{t\}\_\{\\mathrm\{haz\}\}=BCElogit⁡\(ℓi,thaz,yi,thaz\),\\displaystyle=\\operatorname\{BCE\}\_\{\\mathrm\{logit\}\}\(\\ell^\{\\mathrm\{haz\}\}\_\{i,t\},y^\{\\mathrm\{haz\}\}\_\{i,t\}\),\(9\)ℒpret\\displaystyle\\mathcal\{L\}^\{t\}\_\{\\mathrm\{pre\}\}=BCElogit⁡\(ℓi,tpre,yi,tpre\)\.\\displaystyle=\\operatorname\{BCE\}\_\{\\mathrm\{logit\}\}\(\\ell^\{\\mathrm\{pre\}\}\_\{i,t\},y^\{\\mathrm\{pre\}\}\_\{i,t\}\)\.\(10\)These safety losses update the partial unfrozen world model modules on the computational path to the immediate\-hazard and prefix\-risk predictors, while keeping the GRU transition core fixed\. This partial unfreezing lets multi\-horizon risk supervision shape the latent state, preserving information predictive of hazardous outcomes and risk accumulation\. As a result, the latent dynamics become risk\-aware rather than merely predictive of generic next states\.

#### Architecture details\.

The sequence model is implemented as a GRU\-based transition with LayerNorm on the concatenated stochastic state and action input\. The remaining projection and predictor modules are MLPs with Linear layers, LayerNorm, and SiLU activations\. The stochastic state is represented as factorized categorical variables and sampled with straight\-through gradients\.

### 3\.4Multi\-Horizon Risk Estimation

At runtime,DreamGuarduses the world model to predict the successor prior state for the proposed action and maps it to two complementary risk scores: the immediate\-hazard scorepthazp^\{\\mathrm\{haz\}\}\_\{t\}for actions that may directly trigger a hazardous outcome, and the prefix\-risk scoreptprep^\{\\mathrm\{pre\}\}\_\{t\}for accumulating long\-horizon risk\. Because prefix\-risk scores are designed to be sensitive to weak precursor evidence, DreamGuard smooths them with a temporal evidence stateBt=\(ptema,ptwin\)B\_\{t\}=\(p^\{\\mathrm\{ema\}\}\_\{t\},p^\{\\mathrm\{win\}\}\_\{t\}\):

ptema\\displaystyle\{p\}^\{\\mathrm\{ema\}\}\_\{t\}=β​pt−1ema\+\(1−β\)​ptpre,\\displaystyle=\\beta\{p\}^\{\\mathrm\{ema\}\}\_\{t\-1\}\+\(1\-\\beta\)p^\{\\mathrm\{pre\}\}\_\{t\},\(11\)ptwin\\displaystyle\{p\}^\{\\mathrm\{win\}\}\_\{t\}=1\|𝒲t\|​∑j∈𝒲tpjpre,\\displaystyle=\\frac\{1\}\{\|\\mathcal\{W\}\_\{t\}\|\}\\sum\_\{j\\in\\mathcal\{W\}\_\{t\}\}p^\{\\mathrm\{pre\}\}\_\{j\},\(12\)whereβ∈\[0,1\)\\beta\\in\[0,1\)controls how strongly past prefix\-risk evidence is retained, and𝒲t\\mathcal\{W\}\_\{t\}denotes the most recent window of at mostWWprefix\-risk scores\. The exponential\-moving\-average scoreptemap^\{\\mathrm\{ema\}\}\_\{t\}captures weak risk evidence that persists over time, while the recent\-window scoreptwinp^\{\\mathrm\{win\}\}\_\{t\}preserves transient increases in prefix\-risk evidence without letting older evidence dominate\.DreamGuardthen aggregates immediate\-hazard and prefix\-risk evidence into a fused runtime risk scoreStS\_\{t\}using a bounded noisy\-or rule:

St=NoisyOr⁡\(pthaz,ptema,ptwin\),S\_\{t\}=\\operatorname\{NoisyOr\}\\\!\\left\(p^\{\\mathrm\{haz\}\}\_\{t\},\{p\}^\{\\mathrm\{ema\}\}\_\{t\},\{p\}^\{\\mathrm\{win\}\}\_\{t\}\\right\),\(13\)Here,NoisyOr⁡\(⋅\)\\operatorname\{NoisyOr\}\(\\cdot\)applies calibrated fusion weights and bounds each weighted term to\[0,1\]\[0,1\]before multiplication, soStS\_\{t\}increases when any evidence channel is strong while remaining in\[0,1\]\[0,1\]\. The calibrated intervention ruleℐ\\mathcal\{I\}maps\(pthaz,St\)\(p^\{\\mathrm\{haz\}\}\_\{t\},S\_\{t\}\)to an intervention decision before execution:

ℐ​\(pthaz,St\)=\{BLOCK,pthaz≥λblock,HOLD,St≥λhold,PASS,otherwise\.\\mathcal\{I\}\(p^\{\\mathrm\{haz\}\}\_\{t\},S\_\{t\}\)=\\begin\{cases\}\\mathrm\{BLOCK\},&p^\{\\mathrm\{haz\}\}\_\{t\}\\geq\\lambda\_\{\\mathrm\{block\}\},\\\\ \\mathrm\{HOLD\},&S\_\{t\}\\geq\\lambda\_\{\\mathrm\{hold\}\},\\\\ \\mathrm\{PASS\},&\\text\{otherwise\.\}\\end\{cases\}\(14\)Both thresholds are calibrated on safe trajectories using split\-conformal quantiles\. TheBLOCK\\mathrm\{BLOCK\}thresholdλblock\\lambda\_\{\\mathrm\{block\}\}is calibrated frompthazp^\{\\mathrm\{haz\}\}\_\{t\}with a stricter false\-alert budget, while theHOLD\\mathrm\{HOLD\}thresholdλhold\\lambda\_\{\\mathrm\{hold\}\}is calibrated fromStS\_\{t\}\. Forλhold\\lambda\_\{\\mathrm\{hold\}\}, letMi=max1≤t≤Ti⁡Si,tM\_\{i\}=\\max\_\{1\\leq t\\leq T\_\{i\}\}S\_\{i,t\}denote the maximum fused score along each calibration trajectoryi∈𝒟calsafei\\in\\mathcal\{D\}^\{\\mathrm\{safe\}\}\_\{\\mathrm\{cal\}\}\. SortingM1,…,MnM\_\{1\},\\ldots,M\_\{n\}asM\(1\)≤⋯≤M\(n\)M\_\{\(1\)\}\\leq\\cdots\\leq M\_\{\(n\)\},DreamGuardsetsλhold=M\(k\)\\lambda\_\{\\mathrm\{hold\}\}=M\_\{\(k\)\}andk=min⁡\(⌈\(n\+1\)​\(1−α\)⌉,n\),k=\\min\\\!\\left\(\\left\\lceil\(n\+1\)\(1\-\\alpha\)\\right\\rceil,n\\right\),wheren=\|𝒟calsafe\|n=\|\\mathcal\{D\}^\{\\mathrm\{safe\}\}\_\{\\mathrm\{cal\}\}\|andα\\alphais the target false\-alert level on safe trajectories\. BecauseMiM\_\{i\}is the maximum score over all steps in trajectoryii, this threshold calibrates trajectory\-level false alerts: a safe trajectory is counted as alerted if any step triggersHOLD\\mathrm\{HOLD\}\. The same construction is applied to immediate\-hazard scorespthazp^\{\\mathrm\{haz\}\}\_\{t\}to obtainλblock\\lambda\_\{\\mathrm\{block\}\}\.

Algorithm 1Workflow ofDreamGuardInput: Guardrail𝒢=\(θ,ℐ\)\\mathcal\{G\}=\(\\theta,\\mathcal\{I\}\); task\-instruction representationee; online pre\-action stream\{\(ot,at\)\}t=1T\\\{\(o\_\{t\},a\_\{t\}\)\\\}\_\{t=1\}^\{T\} Output: Decision logDD

1:

h1←Initθ​\(e\)h\_\{1\}\\leftarrow\\mathrm\{Init\}\_\{\\theta\}\(e\);

B0←\(0,∅\)B\_\{0\}\\leftarrow\(0,\\emptyset\);

D←∅D\\leftarrow\\emptyset⊳\\trianglerightdecision log

2:for

t=1,…,Tt=1,\\ldots,Tdo

3:// Phase 1: successor\-state prediction

4:

zt∼qθ​\(zt∣ht,ot\)z\_\{t\}\\sim q\_\{\\theta\}\(z\_\{t\}\\mid h\_\{t\},o\_\{t\}\)⊳\\trianglerightinfer current posterior state

5:

ht\+1←fθ​\(ht,zt,at\)h\_\{t\+1\}\\leftarrow f\_\{\\theta\}\(h\_\{t\},z\_\{t\},a\_\{t\}\)⊳\\trianglerightimagine action effect

6:

z^t\+1∼pθ​\(z^t\+1∣ht\+1\)\\hat\{z\}\_\{t\+1\}\\sim p\_\{\\theta\}\(\\hat\{z\}\_\{t\+1\}\\mid h\_\{t\+1\}\)⊳\\trianglerightpredict successor prior

7:

s^t\+1←\(ht\+1,z^t\+1\)\\hat\{s\}\_\{t\+1\}\\leftarrow\(h\_\{t\+1\},\\hat\{z\}\_\{t\+1\}\)
8:// Phase 2: multi\-horizon risk estimation

9:

pthaz←σ​\(gθhaz​\(s^t\+1,at,e\)\)p^\{\\mathrm\{haz\}\}\_\{t\}\\leftarrow\\sigma\\\!\\big\(g^\{\\mathrm\{haz\}\}\_\{\\theta\}\(\\hat\{s\}\_\{t\+1\},a\_\{t\},e\)\\big\)
10:

ptpre←σ​\(gθpre​\(s^t\+1,at,e\)\)p^\{\\mathrm\{pre\}\}\_\{t\}\\leftarrow\\sigma\\\!\\big\(g^\{\\mathrm\{pre\}\}\_\{\\theta\}\(\\hat\{s\}\_\{t\+1\},a\_\{t\},e\)\\big\)
11:

Bt←Update​\(Bt−1,ptpre\)B\_\{t\}\\leftarrow\\mathrm\{Update\}\(B\_\{t\-1\},p^\{\\mathrm\{pre\}\}\_\{t\}\)⊳\\trianglerightaccumulate prefix\-risk evidence

12:

St←Fuse​\(pthaz,Bt\)S\_\{t\}\\leftarrow\\mathrm\{Fuse\}\(p^\{\\mathrm\{haz\}\}\_\{t\},B\_\{t\}\)
13:// Phase 3: calibrated pre\-action intervention

14:

dt←ℐ​\(pthaz,St\)d\_\{t\}\\leftarrow\\mathcal\{I\}\(p^\{\\mathrm\{haz\}\}\_\{t\},S\_\{t\}\)
15:

D←D∪\{\(t,dt\)\}D\\leftarrow D\\cup\\\{\(t,d\_\{t\}\)\\\}
16:if

dt≠PASSd\_\{t\}\\neq\\mathrm\{PASS\}then

17:return

DD⊳\\trianglerightHOLD or BLOCK before execution

18:endif

19:endfor

20:return

DD

## 4Experiments

Table 1:Main runtime guardrail results across four agent safety benchmarks\.F1,SR, andFPR\(in %\) evaluate trajectory\-level hazardous trajectory detection;Latencyreports end\-to\-end wall\-clock seconds per guardrail call\. Best results are boldfaced and second\-best results are underlined\.### 4\.1Experimental Setup

#### Benchmarks\.

We evaluateDreamGuardon four benchmarks covering both long\-horizon risk accumulation and immediate hazards\. \(1\) SafetyDrift\(Dhodapkar and Pishori[2026](https://arxiv.org/html/2608.05695#bib.bib14)\)contains multi\-step trajectories in which risk accumulates before a safety constraint is violated\. \(2\) AgentDojo\(Debenedettiet al\.[2024](https://arxiv.org/html/2608.05695#bib.bib15)\)provides tool\-use tasks with prompt injection and environment\-mediated risks\. \(3\) Agent Security Bench \(ASB\)\(Zhanget al\.[2025](https://arxiv.org/html/2608.05695#bib.bib16)\)covers direct and indirect attacks against LLM agents\. \(4\) ASSE\-Security\(Luoet al\.[2025a](https://arxiv.org/html/2608.05695#bib.bib17)\)covers data leakage, credential abuse, unauthorized access, and prompt injection\.DreamGuardis trained and calibrated on SafetyDrift, using disjoint trajectory splits\. SafetyDrift results are reported on the held\-out test split, and the other benchmarks are evaluated with the same calibrated thresholds\.

#### Step\-Level Annotation\.

Most existing benchmarks provide only trajectory\-level labels, leaving step\-level hazard labels unspecified\. To address this, we convert the four benchmarks into a common step\-level trajectory format and annotate the hazard attribute of each step using benchmark\-specific rules, with initial prelabeling from an LLM and independent review by two human annotators\. The first hazard step is then identified as the earliest step labeled hazardous in each unsafe trajectory\.

#### Baselines\.

We compareDreamGuardwith three groups of guardrails\. \(1\) Generic guardrail: Llama\-Guard\-4\-12B\(Meta[2025](https://arxiv.org/html/2608.05695#bib.bib2)\), a broadly deployed safety classifier adapted to agent trajectories\. \(2\) Reactive guardrails: PolicyGuard\(Wenet al\.[2026](https://arxiv.org/html/2608.05695#bib.bib6)\), GuardAgent\(Xianget al\.[2025](https://arxiv.org/html/2608.05695#bib.bib4)\), and AgentDoG\-1\.5\(Liuet al\.[2026a](https://arxiv.org/html/2608.05695#bib.bib11)\), which primarily assess the apparent safety of the current proposed action or explicit policy constraints\. \(3\) Proactive guardrails: SafePred\(Chenet al\.[2026](https://arxiv.org/html/2608.05695#bib.bib19)\)and TRACES\(Liet al\.[2026](https://arxiv.org/html/2608.05695#bib.bib20)\), which maintain persistent state, track evolving risk, or predict future consequences to address long\-horizon risks\.

#### Metrics\.

We evaluate guardrails at both trajectory and step levels\. \(1\) Following prior runtime guardrail evaluations, we report safety rate \(SR\),F1, false positive rate \(FPR\), andLatencyon the full trajectory\. SR measures the fraction of unsafe trajectories in which the guardrail triggers an intervention or unsafe decision\. Latency is the end\-to\-end wall\-clock time per guardrail call\. \(2\) Since trajectory\-level metrics do not capture whether intervention is timely, we further report Pre\-Hazard Intervention Recall \(PHIR\), the fraction of unsafe trajectories where the first intervention occurs strictly before the first hazard step\. We also report Mean Advance Steps \(MAS\), the average number of steps by which successful interventions precede the first hazard step\. Formal metric definitions are provided in Appendix A\.

#### Implementation Details\.

DreamGuarduses Qwen3\-4B\-Instruct\-2507 as a frozen embedding encoder\. We trainDreamGuardon a single H100 GPU using SafetyDrift as the source benchmark\. For a fair comparison, all trainable baselines use the same training data, and all baselines are evaluated with the same input fields and metric protocol whenever applicable\. Additional implementation details are provided in Appendix C\.

### 4\.2Main Results

Table[1](https://arxiv.org/html/2608.05695#S4.T1)summarizes the main results across benchmarks covering immediate hazards and long\-horizon risks\.DreamGuardshows the strongest overall F1 and SR performance across all benchmarks while keeping FPR low\. It also achieves the lowest latency on all benchmarks\.

SafetyDrift is designed to evaluate long\-horizon risk\. On this benchmark,DreamGuardachieves the best F1 of 96\.4% and the highest SR of 96\.3%, while reducing FPR to 3\.7%\. This suggests thatDreamGuardcan preserve weak early risk evidence until it becomes actionable, rather than overreacting to benign prefixes\. In particular, its recurrent latent state preserves trajectory context, allowing the prefix\-risk predictor to accumulate weak precursor evidence into a stable score for earlier intervention\.

AgentDojo, ASB, and ASSE\-Security mainly focus on immediate hazards\. On these benchmarks, DreamGuard achieves the best SR on AgentDojo and the best F1/SR on ASB, while obtaining the best F1 on ASSE\-Security with lower FPR than PolicyGuard, the strongest SR baseline\. This indicates that DreamGuard can identify immediate hazards while avoiding excessive false alerts\. The contrast with SafetyDrift further highlightsDreamGuard’s consistent performance across risk horizons: TRACES reaches 87\.1% F1 on SafetyDrift but drops to 27\.3% on ASB, while PolicyGuard reaches 79\.1% F1 on ASSE\-Security but only 26\.4% on SafetyDrift\.DreamGuardavoids this sharp trade\-off by maintaining multi\-horizon risk evidence\.

DreamGuardis the most low\-latency guardrail in Table[1](https://arxiv.org/html/2608.05695#S4.T1), with an average end\-to\-end latency of 0\.025 s per call\. Compared with specialized efficient guardrails, it is 3\.3×\\timesfaster than TRACES and 3\.6×\\timesfaster than PolicyGuard\. The gap is much larger against LLM\-reasoning guardrails:DreamGuardis 250\.6×\\timesfaster than GuardAgent and 424\.0×\\timesfaster than SafePred\. This efficiency comes from its fixed\-dimensional recurrent state and lightweight predictors, avoiding repeated full\-trajectory processing or online LLM reasoning\.

### 4\.3Timing Analysis

Table 2:Step\-level timing results across four benchmarks\. Each cell reports PHIR/MAS\. ASSE denotes ASSE\-Security\. LG\-4\-12B denotes Llama\-Guard\-4\-12B\. Best results are boldfaced and second\-best results are underlined\.We further evaluateDreamGuardat the step level by measuring whether a guardrail intervenes before the first hazard action\. On SafetyDrift,DreamGuardachieves the highest PHIR \(96\.3%\) and MAS \(3\.63\), outperforming both reactive and proactive guardrails\. This gain is enabled by its recurrent risk\-aware state and prefix\-risk evidence aggregation, which preserve weak precursor signals before the hazard action becomes explicit\. On the more immediate\-hazard benchmarks,DreamGuardalso obtains the highest PHIR on AgentDojo and ASB, and the highest MAS on ASB, suggesting that its immediate\-hazard and prefix\-risk signals remain useful near the action boundary\. Overall, these results show thatDreamGuardcan anticipate hazardous actions before execution and trigger earlier runtime intervention\.

### 4\.4Online Guardrail

We further evaluateDreamGuardin providing online guardrails for tool\-use agents\. Specifically, we use GPT\-5\.1\(OpenAI[2025](https://arxiv.org/html/2608.05695#bib.bib39)\)as the task agent and integrate each guardrail as a runtime verification module alongside the agent\. As shown in Figure[2](https://arxiv.org/html/2608.05695#S4.F2),DreamGuardoccupies a favorable region of the trade\-off space: it achieves the highest Safety Rate \(72\.92%\) while maintaining high Utility Rate \(90\.38%\), placing it beyond the safety–utility frontier formed by existing guardrails\. This indicates thatDreamGuardimproves online unsafe\-action interception without collapsing benign\-task utility\.

![Refer to caption](https://arxiv.org/html/2608.05695v1/x2.png)Figure 2:Defense performance comparison in the online guardrail setting\. The blue dashed line represents the safety–utility frontier across all baselines\.
### 4\.5Ablation Studies

To evaluate the contribution of each component, we conduct ablation studies on ASSE\-Security, as shown in Table[3](https://arxiv.org/html/2608.05695#S4.T3)\. The fullDreamGuardmodel achieves the best overall F1 \(82\.9%\) while keeping FPR low \(9\.8%\), indicating that its safety gain does not come from indiscriminate over\-intervention\. Overall,DreamGuard’s full design provides the best balance between detection quality, false\-positive control, and pre\-hazard intervention: removing individual components tends to collapse one side of this trade\-off, either by over\-warning with high FPR or by becoming too conservative to intervene early\.

Table 3:Ablation ofDreamGuardcomponents on ASSE\-Security\. We report trajectory\-level safety metrics and step\-level timing metrics for each variant\.#### Effect of Recurrent World Model Dynamics\.

Removing the recurrent world model substantially reduces F1 from 82\.9% to 74\.7% and raises FPR to 73\.7%, despite achieving 100\.0% SR\. This suggests that recurrent latent dynamics are needed to preserve safety\-relevant history without turning the guardrail into an overly aggressive detector\. Removing successor\-state prediction also degrades F1 and increases FPR, showing that scoring the predicted post\-action state is more reliable than scoring the current state alone\.

#### Effect of Multi\-Horizon Risk Estimation\.

Removing the hazard predictor causes the largest drop in F1 \(52\.1%\) and PHIR \(13\.0%\), confirming that immediate\-hazard evidence is essential near the action boundary\. Removing the prefix\-risk predictor keeps F1 relatively high but increases FPR and weakens early intervention, while removing temporal aggregation lowers F1 and SR\. These results support the need to combine immediate\-hazard, prefix\-level risk, and temporal evidence rather than relying on a single risk signal\.

#### Effect of Two\-Stage Training\.

Without world model pretraining, the model reaches high SR and PHIR but suffers from severe false positives, with FPR rising to 92\.0%\. In contrast, removing risk\-supervised training yields low FPR but sharply reduces SR, PHIR, and MAS, making the guardrail too conservative\. This shows that world model pretraining and risk\-supervised training play complementary roles: the former stabilizes latent dynamics, while the latter makes the latent state useful for runtime risk intervention\.

## 5Limitations

DreamGuardis currently calibrated exclusively on SafetyDrift and transfers its decision thresholds zero\-shot to other benchmarks\. This provides a stringent evaluation of transfer robustness, but performance under substantial distribution shifts could be further optimized\. In real\-world deployments, lightweight recalibration with target\-domain data may further enhance adaptability\. Additionally,DreamGuardfocuses on intervention before action execution rather than generating safe replacement actions\. Extending the framework with action revision could reduce task disruption when an intervention is triggered\.

## 6Conclusion

In this work, we study proactive runtime guardrailing for autonomous agents under a step\-level pre\-action decision setting\. We introduceDreamGuard, a world model\-based guardrail that maintains a compact recurrent latent state, learns risk\-aware latent dynamics, and integrates immediate\-hazard and prefix\-level risk evidence into calibrated intervention decisions\. Extensive experiments on four agent safety benchmarks show thatDreamGuardoutperforms baseline guardrails while maintaining highly efficient per\-action runtime\. Online guardrail evaluation further shows that DreamGuard achieves a favorable safety\-utility trade\-off in real agent execution, demonstrating the potential of risk\-aware world models as a practical foundation for proactive agent runtime safety\.

## References

- Web agents with world models: learning and leveraging environment dynamics in web navigation\.InInternational Conference on Learning Representations \(ICLR 2025\),pp\. 63707–63738\.Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Chen, Z\. Liao, P\. Yin, T\. Xie, K\. Yin, and S\. Zhang \(2026\)SafePred: a predictive guardrail for computer\-using agents via world models\.External Links:2602\.01725Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p4.1),[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p2.1),[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2608.05695#S4.SS1.SSS0.Px3.p1.1)\.
- Z\. Chen, M\. Kang, and B\. Li \(2025\)ShieldAgent: shielding agents via verifiable safety policy reasoning\.InProceedings of the 42nd International Conference on Machine Learning \(ICML\),pp\. 8313–8344\.Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p2.1),[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p1.1)\.
- E\. Debenedetti, J\. Zhang, M\. Balunovic, L\. Beurer\-Kellner, M\. Fischer, and F\. Tramèr \(2024\)AgentDojo: a dynamic environment to evaluate prompt injection attacks and defenses for LLM agents\.InAdvances in Neural Information Processing Systems 37 \(NeurIPS 2024\), Datasets and Benchmarks Track,pp\. 82895–82920\.Cited by:[§4\.1](https://arxiv.org/html/2608.05695#S4.SS1.SSS0.Px1.p1.1)\.
- Z\. Deng, Y\. Guo, C\. Han, W\. Ma, J\. Xiong, S\. Wen, and Y\. Xiang \(2025\)AI agents under threat: a survey of key security challenges and future pathways\.ACM Computing Surveys57\(7\),pp\. 182:1–182:36\.Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p1.1)\.
- A\. Dhodapkar and F\. Pishori \(2026\)SafetyDrift: predicting when AI agents cross the line before they actually do\.External Links:2603\.27148Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p4.1),[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p2.1),[§4\.1](https://arxiv.org/html/2608.05695#S4.SS1.SSS0.Px1.p1.1)\.
- Y\. Gan, Y\. Yang, Z\. Ma, P\. He, R\. Zeng, Y\. Wang, Q\. Li, C\. Zhou, S\. Li, T\. Wang, Y\. Gao, Y\. Wu, and S\. Ji \(2026\)Navigating the risks: a survey of security and privacy threats in LLM\-based agents\.Note:ACM Transactions on Software Engineering and MethodologyForthcomingCited by:[§1](https://arxiv.org/html/2608.05695#S1.p1.1)\.
- Y\. Gu, K\. Zhang, Y\. Ning, B\. Zheng, B\. Gou, T\. Xue, C\. Chang, S\. Srivastava, Y\. Xie, P\. Qi, H\. Sun, and Y\. Su \(2025\)Is your LLM secretly a world model of the internet? Model\-based planning for web agents\.Transactions on Machine Learning Research2025\.Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px2.p1.1)\.
- D\. Hafner, T\. Lillicrap, I\. Fischer, R\. Villegas, D\. Ha, H\. Lee, and J\. Davidson \(2019\)Learning latent dynamics for planning from pixels\.InProceedings of the 36th International Conference on Machine Learning \(ICML\),pp\. 2555–2565\.Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px2.p1.1)\.
- D\. Hafner, J\. Pasukonis, J\. Ba, and T\. Lillicrap \(2025\)Mastering diverse control tasks through world models\.Nature640\(8059\),pp\. 647–653\.Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px2.p1.1),[§3\.3](https://arxiv.org/html/2608.05695#S3.SS3.SSS0.Px1.p1.1)\.
- S\. Han, K\. Rao, A\. Ettinger, L\. Jiang, B\. Y\. Lin, N\. Lambert, Y\. Choi, and N\. Dziri \(2024\)WildGuard: open one\-stop moderation tools for safety risks, jailbreaks, and refusals of LLMs\.InAdvances in Neural Information Processing Systems 37 \(NeurIPS 2024\), Datasets and Benchmarks Track,pp\. 8093–8131\.Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p1.1)\.
- N\. Hansen, H\. Su, and X\. Wang \(2024\)TD\-MPC2: scalable, robust world models for continuous control\.InInternational Conference on Learning Representations \(ICLR 2024\),pp\. 47376–47405\.Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px2.p1.1),[§3\.3](https://arxiv.org/html/2608.05695#S3.SS3.SSS0.Px1.p1.1)\.
- H\. Inan, K\. Upasani, J\. Chi, R\. Rungta, K\. Iyer, Y\. Mao, M\. Tontchev, Q\. Hu, B\. Fuller, D\. Testuggine, and M\. Khabsa \(2023\)Llama guard: LLM\-based input\-output safeguard for human\-AI conversations\.External Links:2312\.06674Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Li, Y\. Li, B\. Zhang, R\. Tang, and K\. Huang \(2026\)TRACES: proactive safety auditing for multi\-turn LLM agents via trajectory\-state modeling\.External Links:2605\.27690Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p4.1),[§4\.1](https://arxiv.org/html/2608.05695#S4.SS1.SSS0.Px3.p1.1)\.
- D\. Liu, Y\. Li, Z\. Yang, P\. Wang, G\. Chen, Y\. Xie, Q\. Mao, W\. Qu, Y\. Zhu, T\. Zhou, L\. Yuan, Z\. Zheng, Q\. Lin, Y\. Wang, H\. Luo, S\. Shao, C\. Qian, Q\. Liu, L\. Tang, R\. Qin, Q\. Ren, J\. Yang, K\. Wang, Z\. Xi, L\. Zhang, R\. Duan, B\. Zhang, W\. Wang, W\. Shen, Q\. Zhang, Y\. Teng, C\. Lu, R\. Mei, M\. Li, J\. Tao, X\. Lin, T\. Zheng, Y\. Liu, Q\. Zhang, L\. Zhu, X\. Ma, J\. Liu, H\. Xue, X\. Zuo, X\. He, C\. Shen, X\. Liu, M\. Huang, J\. Shao, and X\. Hu \(2026a\)AgentDoG 1\.5: a lightweight and scalable alignment framework for AI agent safety and security\.External Links:2605\.29801Cited by:[§4\.1](https://arxiv.org/html/2608.05695#S4.SS1.SSS0.Px3.p1.1)\.
- D\. Liu, Q\. Ren, C\. Qian, S\. Shao, Y\. Xie, Y\. Li, Z\. Yang, H\. Luo, P\. Wang, Q\. Liu, B\. Hu, L\. Tang, J\. Mei, D\. Guo, L\. Yuan, J\. Yang, G\. Chen, Q\. Lin, Y\. Yu, B\. Zhang, J\. Guo, J\. Zhang, W\. Shao, H\. Deng, Z\. Xi, W\. Wang, W\. Wang, W\. Shen, Z\. Chen, H\. Xie, J\. Tao, J\. Dai, J\. Ji, Z\. Ba, L\. Zhang, Y\. Liu, Q\. Zhang, L\. Zhu, Z\. Wei, H\. Xue, C\. Lu, J\. Shao, and X\. Hu \(2026b\)AgentDoG: a diagnostic guardrail framework for AI agent safety and security\.External Links:2601\.18491Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p1.1)\.
- H\. Liu, E\. Ilyushin, J\. Ni, and M\. Zhu \(2026c\)SafeAgent: a runtime protection architecture for agentic systems\.External Links:2604\.17562Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p4.1),[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p2.1)\.
- H\. Luo, S\. Dai, C\. Ni, X\. Li, G\. Zhang, K\. Wang, T\. Liu, and H\. Salam \(2025a\)AgentAuditor: human\-level safety and security evaluation for LLM agents\.InAdvances in Neural Information Processing Systems 38 \(NeurIPS 2025\),pp\. 43241–43298\.Cited by:[§4\.1](https://arxiv.org/html/2608.05695#S4.SS1.SSS0.Px1.p1.1)\.
- W\. Luo, S\. Dai, X\. Liu, S\. Banerjee, H\. Sun, M\. Chen, and C\. Xiao \(2025b\)AGrail: a lifelong agent guardrail with effective and adaptive safety detection\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 8104–8139\.Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p1.1)\.
- X\. Ma, Y\. Gao, Y\. Wang, R\. Wang, X\. Wang, Y\. Sun, Y\. Ding, H\. Xu, Y\. Chen, Y\. Zhao, H\. Huang, Y\. Li, Y\. Wu, J\. Zhang, X\. Zheng, Y\. Bai, Y\. Li, Z\. Wu, X\. Qiu, J\. Zhang, X\. Han, H\. Li, J\. Sun, C\. Wang, J\. Gu, B\. Wu, S\. Chen, T\. Zhang, Y\. Liu, M\. Gong, T\. Liu, S\. Pan, C\. Xie, T\. Pang, Y\. Dong, R\. Jia, Y\. Zhang, S\. Ma, X\. Zhang, N\. Gong, C\. Xiao, S\. Erfani, T\. Baldwin, B\. Li, M\. Sugiyama, D\. Tao, J\. Bailey, and Y\. Jiang \(2025\)Safety at scale: a comprehensive survey of large model and agent safety\.Foundations and Trends in Privacy and Security8\(3–4\),pp\. 254–469\.Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p1.1)\.
- Meta \(2025\)Llama Guard 4 Model Card\.Note:https://huggingface\.co/meta\-llama/Llama\-Guard\-4\-12BAccessed: 2026\-07\-28Cited by:[§4\.1](https://arxiv.org/html/2608.05695#S4.SS1.SSS0.Px3.p1.1)\.
- Y\. Mou, Z\. Xue, L\. Li, P\. Liu, S\. Zhang, W\. Ye, and J\. Shao \(2026\)ToolSafe: enhancing tool invocation safety of LLM\-based agents via proactive step\-level guardrail and feedback\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 37125–37153\.Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p1.1)\.
- OpenAI \(2025\)GPT\-5\.1 instant and GPT\-5\.1 thinking system card addendum\.Note:https://openai\.com/index/gpt\-5\-system\-card\-addendum\-gpt\-5\-1/Accessed: 2026\-07\-29Cited by:[§4\.4](https://arxiv.org/html/2608.05695#S4.SS4.p1.1)\.
- Y\. Qin, S\. Liang, Y\. Ye, K\. Zhu, L\. Yan, Y\. Lu, Y\. Lin, X\. Cong, X\. Tang, B\. Qian, S\. Zhao, L\. Hong, R\. Tian, R\. Xie, J\. Zhou, M\. Gerstein, D\. Li, Z\. Liu, and M\. Sun \(2024\)ToolLLM: facilitating large language models to master 16000\+ real\-world APIs\.InInternational Conference on Learning Representations \(ICLR 2024\),pp\. 9695–9717\.Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p1.1)\.
- H\. Wang, C\. M\. Poskitt, J\. Wei, and J\. Sun \(2025a\)ProbGuard: probabilistic runtime monitoring for LLM agent safety\.External Links:2508\.00500Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p2.1)\.
- L\. Wang, Z\. Ying, X\. Yang, Q\. Zou, Z\. Yin, T\. Li, J\. Yang, Y\. Yang, A\. Liu, and X\. Liu \(2025b\)RoboSafe: safeguarding embodied agents via executable safety logic\.External Links:2512\.21220Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p4.1)\.
- L\. Wang, C\. Ma, X\. Feng, Z\. Zhang, H\. Yang, J\. Zhang, Z\. Chen, J\. Tang, X\. Chen, Y\. Lin, W\. X\. Zhao, Z\. Wei, and J\. Wen \(2024\)A survey on large language model based autonomous agents\.Frontiers of Computer Science18\(6\)\.Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p1.1)\.
- L\. Wang, Z\. Ren, T\. Yang, J\. Ji, C\. H\. Liu, Y\. Yang, and J\. Dai \(2026a\)SafeMCP: proactive power regulation for LLM agent defense via environment\-grounded look\-ahead reasoning\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 11374–11396\.Cited by:[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p2.1),[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Wang, T\. Jiang, J\. Liang, C\. Fleming, and T\. Wang \(2026b\)MAGE: safeguarding LLM agents against long\-horizon threats via shadow memory\.External Links:2605\.03228Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p4.1),[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p2.1)\.
- X\. Wen, W\. J\. Mo, Y\. Xie, P\. Qi, and M\. Chen \(2026\)Learning efficient guardrails for compliance\.InProceedings of the 43rd International Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p2.1),[§4\.1](https://arxiv.org/html/2608.05695#S4.SS1.SSS0.Px3.p1.1)\.
- Z\. Xi, W\. Chen, X\. Guo, W\. He, Y\. Ding, B\. Hong, M\. Zhang, J\. Wang, S\. Jin, E\. Zhou, R\. Zheng, X\. Fan, X\. Wang, L\. Xiong, Y\. Zhou, W\. Wang, C\. Jiang, Y\. Zou, X\. Liu, Z\. Yin, S\. Dou, R\. Weng, W\. Qin, Y\. Zheng, X\. Qiu, X\. Huang, Q\. Zhang, and T\. Gui \(2025\)The rise and potential of large language model based agents: a survey\.Science China Information Sciences68\(2\)\.Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p1.1)\.
- Z\. Xiang, L\. Zheng, Y\. Li, J\. Hong, Q\. Li, H\. Xie, J\. Zhang, Z\. Xiong, C\. Xie, C\. Yang, D\. Song, and B\. Li \(2025\)GuardAgent: safeguard LLM agents via knowledge\-enabled reasoning\.InProceedings of the 42nd International Conference on Machine Learning \(ICML\),pp\. 68316–68342\.Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p2.1),[§2](https://arxiv.org/html/2608.05695#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.05695#S4.SS1.SSS0.Px3.p1.1)\.
- M\. Yu, F\. Meng, X\. Zhou, S\. Wang, J\. Mao, L\. Pang, T\. Chen, K\. Wang, X\. Li, Y\. Zhang, B\. An, and Q\. Wen \(2025\)A survey on trustworthy LLM agents: threats and countermeasures\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V\.2,pp\. 6216–6226\.Cited by:[§1](https://arxiv.org/html/2608.05695#S1.p1.1)\.
- H\. Zhang, J\. Huang, K\. Mei, Y\. Yao, Z\. Wang, C\. Zhan, H\. Wang, and Y\. Zhang \(2025\)Agent Security Bench \(ASB\): formalizing and benchmarking attacks and defenses in LLM\-based agents\.InInternational Conference on Learning Representations \(ICLR 2025\),pp\. 35331–35366\.Cited by:[§4\.1](https://arxiv.org/html/2608.05695#S4.SS1.SSS0.Px1.p1.1)\.

## Appendix AEvaluation Metrics and Calibration

### A\.1Trajectory\-Level Metrics

#### Safety Rate\.

For unsafe trajectories, Safety Rate measures whether the guardrail triggers at least one intervention before the trajectory ends\. For binary baselines, any unsafe flag or blocking decision counts as an intervention\.

Safety​Rate=\#​unsafe trajectories with intervention\#​unsafe trajectories\.\\mathrm\{Safety\\ Rate\}=\\frac\{\\\#\\text\{unsafe trajectories with intervention\}\}\{\\\#\\text\{unsafe trajectories\}\}\.\(15\)

#### False Positive Rate \(FPR\)\.

For safe trajectories, FPR measures whether the guardrail incorrectly triggers an intervention\.

FPR=\#​safe trajectories with intervention\#​safe trajectories\.\\mathrm\{FPR\}=\\frac\{\\\#\\text\{safe trajectories with intervention\}\}\{\\\#\\text\{safe trajectories\}\}\.\(16\)

#### F1\.

We treat each trajectory as a binary classification sample\. Unsafe trajectories are positive, safe trajectories are negative, and a trajectory is predicted unsafe if any step triggers an intervention or unsafe decision\. Let TP, FP, and FN denote the trajectory\-level counts under this convention\. Then

Precision\\displaystyle\\mathrm\{Precision\}=TPTP\+FP,Recall=TPTP\+FN,\\displaystyle=\\frac\{\\mathrm\{TP\}\}\{\\mathrm\{TP\}\+\\mathrm\{FP\}\},\\qquad\\mathrm\{Recall\}=\\frac\{\\mathrm\{TP\}\}\{\\mathrm\{TP\}\+\\mathrm\{FN\}\},\(17\)F1\\displaystyle\\mathrm\{F1\}=2​Precision​RecallPrecision\+Recall\.\\displaystyle=\\frac\{2\\,\\mathrm\{Precision\}\\,\\mathrm\{Recall\}\}\{\\mathrm\{Precision\}\+\\mathrm\{Recall\}\}\.\(18\)

#### Latency\.

Latency is the wall\-clock time per guardrail call, measured from receiving the current pre\-action input to returning a decision\. The paper reports latency in seconds per call; raw logs are recorded at millisecond resolution\.

Latency=1N​∑i\(tidecision−tiinput\)\.\\mathrm\{Latency\}=\\frac\{1\}\{N\}\\sum\_\{i\}\\left\(t\_\{i\}^\{\\mathrm\{decision\}\}\-t\_\{i\}^\{\\mathrm\{input\}\}\\right\)\.\(19\)

### A\.2Timing\-Level Metrics

#### Pre\-Hazard Intervention Recall \(PHIR\)\.

PHIR measures the fraction of unsafe trajectories in which the first intervention occurs strictly before the first hazard step\.

PHIR=\#​\{τ:dτ<hτ\}\#​\{τ:hτ​exists\}\.\\mathrm\{PHIR\}=\\frac\{\\\#\\\{\\tau:d\_\{\\tau\}<h\_\{\\tau\}\\\}\}\{\\\#\\\{\\tau:h\_\{\\tau\}\\text\{ exists\}\\\}\}\.\(20\)Here,hτh\_\{\\tau\}is the first hazard step of trajectoryτ\\tau, anddτd\_\{\\tau\}is the first step where the guardrail triggers an intervention\.

#### Mean Advance Steps \(MAS\)\.

MAS measures how many steps before the first hazard step the guardrail intervenes, averaged over trajectories with successful pre\-hazard intervention\.

MAS=𝔼​\[hτ−dτ∣dτ<hτ\]\.\\mathrm\{MAS\}=\\mathbb\{E\}\\left\[h\_\{\\tau\}\-d\_\{\\tau\}\\mid d\_\{\\tau\}<h\_\{\\tau\}\\right\]\.\(21\)

### A\.3Split\-Conformal Threshold Calibration

DreamGuardcalibrates its runtime thresholds on a held\-out calibration split using split\-conformal threshold selection\. The hold threshold is chosen from the trajectory\-level maximum fused score\. The block threshold is calibrated analogously from the trajectory\-level maximum immediate\-hazard score, using the stricter false\-alert budget of the runtime policy\. Both thresholds are fixed after calibration and reused unchanged at test time\.

For the hold threshold, letMiM\_\{i\}be the maximum fused score in safe calibration trajectoryii:

Mi=max1≤t≤Ti⁡Si,t\.M\_\{i\}=\\max\_\{1\\leq t\\leq T\_\{i\}\}S\_\{i,t\}\.\(22\)Sorting the safe calibration scores asM\(1\)≤⋯≤M\(n\)M\_\{\(1\)\}\\leq\\cdots\\leq M\_\{\(n\)\},DreamGuardsets

λhold\\displaystyle\\lambda\_\{\\mathrm\{hold\}\}=M\(k\),\\displaystyle=M\_\{\(k\)\},\(23\)k\\displaystyle k=min⁡\(⌈\(n\+1\)​\(1−α\)⌉,n\),\\displaystyle=\\min\\left\(\\left\\lceil\(n\+1\)\(1\-\\alpha\)\\right\\rceil,n\\right\),\(24\)wherennis the number of safe calibration trajectories andα\\alphais the target safe\-trajectory false\-alert level\. The same construction is applied to the maximum immediate\-hazard scores of safe calibration trajectories to obtainλblock\\lambda\_\{\\mathrm\{block\}\}\.

## Appendix BBenchmark Construction and Labeling

We choose the four core benchmarks to cover the two risk regimes that our paper studies\. SafetyDrift is included because it is specifically designed for long\-horizon risk, where individually benign actions can accumulate into a later violation\. The other three benchmarks are widely used for agent safety evaluation and primarily stress immediate\-hazard behavior while spanning diverse risk categories\. This combination lets us evaluate whether DreamGuard handles both delayed risk accumulation and current\-step hazards, and whether the same design transfers across distinct risk types\.

### B\.1Core Benchmarks

We evaluateDreamGuardon four core benchmarks:

1. 1\.SafetyDrift\. Designed for long\-horizon risk prediction and early warning\. Paper:https://arxiv\.org/abs/2603\.27148\. Code/release: see the paper’s linked project page and release notes\.
2. 2\.AgentDojo\. A dynamic benchmark for prompt\-injection and tool\-use attacks on LLM agents\. Paper and benchmark site:https://agentdojo\.spylab\.ai/andhttps://github\.com/ethz\-spylab/agentdojo\.
3. 3\.Agent Security Bench \(ASB\)\. A benchmark for attacks and defenses in LLM\-based agents\. Paper:https://arxiv\.org/pdf/2410\.02644\. Code:https://github\.com/agiresearch/ASB\.
4. 4\.ASSE\-Security\. A benchmark for agent security and safety evaluation, covering data leakage, credential abuse, unauthorized access, prompt injection, and related risks\. Paper:https://proceedings\.neurips\.cc/paper\_files/paper/2025/file/3dc85735f6e2fcf093e67b134fa00d21\-Paper\-Conference\.pdf\. Code:https://github\.com/Astarojth/AgentAuditor\-ASSEBench\.

### B\.2Step\-Level Hazard Labeling

We convert all four benchmarks into a common step\-level trajectory format and assign each unsafe trajectory a first hazard step: the earliest proposed action whose execution would cause a concrete hazardous outcome or transition\. When a source benchmark provides reliable step\-level labels, we retain and audit them\. Otherwise, we use GPT\-5\.1 for initial prelabeling, followed by independent review by two human annotators and adjudication of disagreements\. The reviewers recover the first hazard step from the trajectory evidence and post\-action consequences\.

We apply three labeling principles\. First, the boundary must correspond to a concrete hazard, not merely a sensitive, controversial, or unsafe\-looking action\. Second, the selected step should be the earliest action that independently crosses the safety boundary under the available trajectory evidence\. Third, if the trace does not support a reliable boundary recovery, we retain the trajectory\-level label for aggregate evaluation but exclude the sample from step\-level timing metrics\. This policy keeps step\-level supervision aligned with the pre\-action decision problem studied in the main text\.

### B\.3Data Audit and Cleaning

We audited duplicate\-content leakage in SafetyDrift and rechecked the TSBench\-derived AgentDojo and ASB caches against the full raw data before fixing the final splits\. This avoids relying on earlier held\-out subsets that can understate the difficulty of the full benchmark\.

## Appendix CDetails of Experiments

### C\.1Feature Extraction and Mainline Configuration

DreamGuarduses frozen Qwen3\-4B\-Instruct\-2507 representations from layer 31, pooled into 2560\-dimensional features\. The mainline model is an RSSM with a GRU transition\. After world\-model pretraining on SafetyDrift, immediate\-hazard and prefix\-risk supervision shapes its predicted successor states\. The stochastic latent state is factorized categorical and is trained with straight\-through gradients\.

Table 4:Mainline configuration ofDreamGuard\.VariantDesignPurposeEffect of Recurrent World\-Model DynamicsFullDreamGuardUses recurrent latent dynamics and scores the predicted successor state before action execution\.Reference model\.w/o recurrent WMRemoves the RSSM recurrent latent state and predicts risk from the current instruction, observation, and proposed\-action embeddings\.Tests whether a fixed\-dimensional recurrent state improves over a current\-step scorer\.w/o successor predictionKeeps the recurrent state, but scores the current latent state instead of the predicted successor state\.Tests whether successor prediction is necessary for pre\-action intervention\.Effect of Multi\-Horizon Risk EstimationFullDreamGuardFuses immediate\-hazard and prefix\-risk evidence with temporal prefix\-risk aggregation\.Reference model\.w/o hazard predictorRemoves the immediate\-hazard branch and makes decisions from the prefix\-risk branch\.Tests whether long\-horizon evidence alone is sufficient\.w/o prefix\-risk predictorRemoves the prefix\-risk branch and makes decisions from the immediate\-hazard branch\.Tests whether current\-step hazard evidence alone is sufficient\.w/o temporal aggregationUses both branches, but removes EMA and sliding\-window prefix\-risk statistics\.Tests whether temporal aggregation contributes beyond raw multi\-horizon scores\.Effect of Two\-Stage TrainingFullDreamGuardFirst trains the world model, then performs risk\-supervised training with partial world\-model updates\.Reference model\.w/o WM pretrainingSkips world\-model pretraining and directly performs risk\-supervised training\.Tests whether consequence prediction pretraining is necessary\.w/o risk\-supervised trainingKeeps world\-model pretraining but omits the second\-stage risk\-supervised training\.Tests whether risk\-supervised training is necessary to make the latent state useful for runtime intervention\.Table 5:Ablation variants grouped by recurrent world\-model dynamics, multi\-horizon risk estimation, and two\-stage training\.
### C\.2Symbol\-to\-Implementation Mapping

The appendix uses a single notation system throughout\. The core symbol\-to\-implementation mapping is summarized in Table[6](https://arxiv.org/html/2608.05695#A3.T6)\.

Table 6:Symbol\-to\-implementation mapping used in the paper and appendix\.
### C\.3Baseline Adaptation and Prompting

#### Baseline Groups\.

We organize baselines using the same taxonomy as the main paper:

1. 1\.Generic guardrail: Llama\-Guard\-4\-12B\.
2. 2\.Reactive guardrail: PolicyGuard, GuardAgent, AgentDoG\-1\.5\.
3. 3\.Proactive guardrail: SafePred, TRACES\.

#### Adaptation Protocol\.

Whenever a baseline exposes a pre\-action decision interface, we evaluate it at the same action boundary asDreamGuardand keep the task instruction, observation, and proposed action fields aligned across methods\. Released standalone checkpoints are used with their official weights; prompt\-based methods use their published prompting or judging setup when available\. For trainable baselines, the same training and calibration split used forDreamGuardis reused whenever the method requires adaptation or fine\-tuning\. The final baseline settings are therefore fixed by their released checkpoint or published inference protocol, and we do not introduce extra paper\-specific tuning beyond the shared evaluation and calibration rules above\.

#### Online Prompting and Tool Schema\.

For the online guardrail experiment, the benchmark’s serialized tool list is converted into function\-calling schemas, and each guardrail is inserted into the same lightweight pre\-action hook\. The live agent prompt, the guardrail prompt, and the LLM\-judge prompt are kept fixed across methods to ensure a fair comparison\.

### C\.4Ablation Study Design

All ablations keep the same data splits, calibration protocol, and test\-time configuration unless the ablation explicitly changes that component\.

### C\.5Online Guardrail Reproduction

#### Setup\.

The online evaluation uses the following configuration:

- •Benchmark: ASSE\-Security online subset
- •Sample size: 100 cases
- •Sampling: stratified by ASSE\-Securityrisk\_type
- •Agent: GPT\-5\.1 function\-calling agent
- •Guardrail: real\-timeDreamGuardcheckpoint
- •Max online steps: 6
- •Calibration:α=0\.6\\alpha=0\.6

This is a derived evaluation slice built from ASSE\-Security, not a new standalone benchmark\. The online harness uses a lightweight function\-calling loop over ASSE\-derived mock tools; the public benchmark itself remains the source dataset, while the mock\-tool environment is the reproducible evaluation layer used for online guardrail comparison\. The agent and judge are both instantiated through an OpenAI\-compatible API, and the guardrail runs as a pre\-action hook before each tool invocation\. The online comparison reports Safety Rate on unsafe cases and Utility Rate on benign cases\.

#### Subset Composition\.

Table 7:Composition of the 100\-case ASSE\-Security online subset\.
#### Reported Outputs\.

Paper\-facing comparison tables and per\-risk\-type breakdowns are generated from a single canonical summary so that intermediate judge reruns do not introduce stale rows\. Released run artifacts and the final summary table remain aligned for the reported results\.

#### Safety\-Utility Frontier Coordinates\.

The Pareto frontier in the main text is drawn from the overall safety\-utility coordinates in Table[8](https://arxiv.org/html/2608.05695#A3.T8)\.

Table 8:Safety\-utility coordinates for the online ASSE\-Security subset \(48 unsafe and 52 benign trajectories\)\.These coordinates correspond to the 100\-case ASSE\-Security online subset with 48 unsafe and 52 benign trajectories, using the same live agent and judge setup described above\.

### C\.6Prompt Templates and Judge Rules

To support online evaluation reproducibility, we fix the live\-agent system prompt, the guardrail prompt template, the judge prompt, and the tool\-schema conversion rule used to instantiate the ASSE\-Security function\-calling environment\. Baseline\-specific prompting or judging templates are likewise kept fixed whenever a method is run through the same online hook\. Exact prompt text is provided with the code and data package accompanying this submission\.

### C\.7Software, Hardware, and Randomness

The reported experiments were run on a single NVIDIA H100 80GB HBM3 GPU with driver 580\.95\.05, an Intel\(R\) Xeon\(R\) Platinum 8468V CPU, and 2\.0 TiB of host memory under Linux 5\.14\.0\-284\.25\.1\.el9\_2\.x86\_64\. The local reproduction environment used Python 3\.8\.10, PyTorch 2\.3\.1\+cu121, Transformers 4\.43\.1, NumPy 1\.24\.4, and pandas 2\.0\.3\. The online guardrail reproduction additionally uses an OpenAI\-compatible API endpoint\.

Randomness is controlled with explicit integer seeds in the experiment scripts\. The mainline training and export runs use seed 20260701, and the online ASSE\-Security subset sampling uses seed 20260716\. When rerunning a reported experiment, the same seed and the same data split identifiers should be reused together with the same checkpoint path\.

### C\.8Core Hyperparameter Search and Final Choices

We summarize only the core hyperparameter families that materially affected the final mainline \(Table[9](https://arxiv.org/html/2608.05695#A3.T9)\)\.

Table 9:Core hyperparameter search ranges and selected values\.The remaining optimization settings were kept fixed in the final submission: batch size 8, Adam learning rate2×10−42\\times 10^\{\-4\}, world\-model LR scale 0\.25, KL weight 0\.03, JEPA weight 1\.25, free bits 0\.2, and adapter scale 0\.25\.

### C\.9First\-Hazard Label Stability Audit

We conduct a human double\-review audit for first\-hazard\-step labeling on every benchmark that does not already provide native step\-level labels\. For benchmarks with native step labels \(SafetyDrift\), we retain the source labels and only audit them for consistency\. For the remaining benchmarks, two human annotators independently review the first\-hazard boundary, and disagreements are resolved by adjudication\. This audit is meant to verify that the step\-level boundary used in the main paper is stable enough for timing metrics such as PHIR and MAS\.

For each benchmark, we report exact first\-hazard\-step agreement, within\-one\-step agreement, Cohen’s kappa for whether a first hazard step exists, and the fraction of samples changed by final adjudication\. Here, kappa is computed on the binary question of whether a trajectory contains a first hazard step, not on the exact step index\.

Table 10:First\-hazard label stability audit across benchmarks\.Across the non\-native\-label benchmarks, exact\-step agreement ranges from 79\.6% to 88\.1%, and within\-one\-step agreement ranges from 89\.6% to 95\.1%\. ASSE\-Security has the largest residual boundary ambiguity\. Because PHIR and MAS depend directly on the first\-hazard boundary, all reviewer disagreements are adjudicated before evaluation\. For SafetyDrift, we retain the native step labels after the consistency audit\.

## Appendix DRobustness and Statistical Analysis

We assess the stability of the reported results by repeating the main trajectory\-level evaluation, step\-level timing analysis, and online guardrail experiment three times with fixed data splits and different random seeds\. Tables[11](https://arxiv.org/html/2608.05695#A4.T11)–[13](https://arxiv.org/html/2608.05695#A4.T13)report mean±\\pmstandard deviation\. We further compareDreamGuardwith the strongest non\-DreamGuardbaseline on each benchmark using paired Wilcoxon signed\-rank tests, with pairs formed at the trajectory level for offline benchmarks and at the case level for online guardrail evaluation\.

Table 11:Repeated\-run results forDreamGuardmain evaluation metrics\.Table 12:Repeated\-run results forDreamGuardtiming metrics\.Table 13:Repeated\-run results for online guardrail evaluation\.Table 14:Paired Wilcoxon signed\-rank test results\. Each row comparesDreamGuardagainst the strongest non\-DreamGuardbaseline for the corresponding setting and benchmark; the paired outcomes are trajectory\-level intervention correctness for main results, pre\-hazard intervention indicators for timing, and case\-level safety/utility success for online evaluation\.The repeated\-run tables summarize variation across seeds, while the Wilcoxon table tests whether the paired improvement over the strongest baseline is consistent across examples rather than driven by aggregate averaging\. These results preserve the conclusions of the main results:DreamGuardremains stable across runs, retains its timing advantage, and improves the online safety\-utility trade\-off\.

## Appendix ECase Study

We present two representative cases\. For each case, we show the task instruction, the key observation prefix, the proposed action sequence, andDreamGuard’s pre\-action decision\.

ttProposed actionFused scoreDecisionHazard\(a\) SafetyDrift: Long\-Horizon Risk Accumulation0read\_file \{"path":
"/app/
requirements\.txt"\}0\.075PASS01read\_file \{"path":
"/app/security/
vuln\_report\.json"\}0\.341PASS02read\_file \{"path":
"/app/\.github/
workflows/deploy\.yml"\}0\.299PASS03execute\_code\{"code": "find /app \-name ’\*\.py’ \.\.\."\}0\.203PASS04execute\_code\{"code":
"find \.\.\. setup\.py/
pyproject\.toml"\}0\.049PASS05execute\_code\{"code": "grep \-r import requests /app \.\.\."\}0\.005PASS06execute\_code\{"code": "pip index versions requests \.\.\."\}0\.007PASS07http\_request\{"method": "GET",
"url": "https://
pypi\.org/\.\.\."\}0\.921HOLD1\(b\) ASSE\-Security: Prompt Injection0sys\_monitor\(\)0\.152PASS01update\_manager\(\)0\.429PASS02FirmwareInfection\(\)0\.988BLOCK1Table 15:Representative pre\-action guardrail cases\.DreamGuardremains permissive during benign prefixes and intervenes when the trajectory reaches a hazardous transition\.### E\.1SafetyDrift: Long\-Horizon Risk Accumulation

#### Instruction\.

Patch the vulnerable dependency without breaking the deployment workflow\.

#### Observation prefix\.

The agent sees a pinned dependency list \(requests==2\.28\.0\), a vulnerability report that flagsrequests, and a deployment workflow that requires a production\-safe update\.

DreamGuardremains permissive through the inspection prefix\. At the annotated first hazard step, the fused score spikes and the guardrail intervenes before the proposed action is executed\.

### E\.2ASSE\-Security: Prompt Injection at the Action Boundary

#### Instruction\.

Upgrade the operating systems of all servers within the next week, ensuring uninterrupted operation for 50 servers in the company’s data center\.

#### Observation prefix\.

The trace begins with benign monitoring and upgrade planning, but the final step injects a hazardous firmware action through the phrase “with perspicacious discernment\.”

DreamGuardstays permissive on the first two benign actions and blocks the injected firmware call at the action boundary\. Compared with the long\-horizon SafetyDrift case above, this example is an immediate hazard: the proposed action itself is unsafe, so the guardrail intervenes as soon as the hazardous tool call appears\.