PrefixGuard: From LLM-Agent Traces to Online Failure-Warning Monitors
Summary
# Paper page - PrefixGuard: From LLM-Agent Traces to Online Failure-Warning Monitors Source: [https://huggingface.co/papers/2605.06455](https://huggingface.co/papers/2605.06455) ## Abstract PrefixGuard enables effective online monitoring of LLM agents through trace analysis and prefix\-based risk scoring, demonstrating strong performance across multiple benchmark tasks while providing diagnostic insights for alert reliability\. Large language model \(LLM\) agents now execute long, tool\-using ta
View Cached Full Text
Cached at: 05/11/26, 10:44 AM
Paper page - PrefixGuard: From LLM-Agent Traces to Online Failure-Warning Monitors
Source: https://huggingface.co/papers/2605.06455
Abstract
PrefixGuard enables effective online monitoring of LLM agents through trace analysis and prefix-based risk scoring, demonstrating strong performance across multiple benchmark tasks while providing diagnostic insights for alert reliability.
Large language model (LLM) agents now execute long, tool-using tasks where final outcome checks can arrive too late for intervention. Online warning requires lightweightprefix monitorsover heterogeneous traces, but hand-authored event schemas are brittle and deployment-time LLM judging is costly. We introduce PrefixGuard, atrace-to-monitor frameworkwith an offlineStepView inductionstep followed by supervised monitor training. StepView induces deterministic typed-step adapters from raw trace samples, and the monitor learns anevent abstractionandprefix-risk scorerfrom terminal outcomes. Across WebArena, τ^2-Bench, SkillsBench, and TerminalBench, the strongest PrefixGuard monitors reach 0.900/0.710/0.533/0.557AUPRC. Using the strongest backend within each representation, they improve over raw-text controls by an average of +0.137AUPRC. LLM judges remain substantially weaker under the same prefix-warning protocol. We also derive an observability ceiling on score-based area under the precision-recall curve (AUPRC) that separates monitor error from failures lacking evidence in the observed prefix. Forfinite-state audit, post-hocdeterministic finite automaton (DFA)extraction remains compact on WebArena and τ^2-Bench (29 and 20 states) but expands to 151 and 187 states on SkillsBench and TerminalBench. Finally, first-alert diagnostics show that strong ranking does not imply deployment utility: WebArena ranks well yet fails to support low-false-alarm alerts, whereas τ^2-Bench and TerminalBench retain more actionable early alerts. Together, these results position PrefixGuard as a practical monitor-synthesis recipe with explicit diagnostics for when prefix warnings translate into actionable interventions.
View arXiv pageView PDFGitHub1Add to collection
Get this paper in your agent:
hf papers read 2605\.06455
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.06455 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.06455 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.06455 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
PropGuard: Safeguarding LLM-MAS via Propagation-Aware Exploration and Remediation
PropGuard is a propagation-aware framework for safeguarding LLM-based multi-agent systems (LLM-MAS) from malicious instructions that propagate across agents and rounds. It constructs a dual-view spatio-temporal graph and uses a GE-GRPO trained inspector to detect and remediate suspicious propagation subgraphs.
DreamGuard: Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model
DreamGuard is a proactive runtime guardrail for LLM agents that uses a risk-aware world model to track latent state and predict future risks, enabling interventions before unsafe actions execute. It outperforms baselines on benchmarks and online evaluation with 25ms latency.
PolicyGuard: A Dialogue-Grounded Sub-Agent Verifier for Policy Adherence in LLM Agents
PolicyGuard is a sub-agent verifier that enhances LLM agent policy adherence by providing contextual reasoning and conversation-specific feedback across multi-turn interactions, achieving significant improvements on the tau^2-BENCH benchmark.
Safeguarding LLM Agents from Misalignment through Provenance Analysis
This paper proposes a provenance-based framework and multi-stage pipeline, \tool, to detect misalignment in LLM agents' tool invocations before execution, reducing error rates significantly compared to LLM-as-a-judge baselines.
Known By Their Actions: Fingerprinting LLM Browser Agents via UI Traces
This paper demonstrates that websites can identify which large language model powers a browsing agent by analyzing its behavioral patterns and timing data, achieving up to 96% F1 score across 14 frontier LLMs. It formalizes this attack surface and shows that random timing delays are insufficient to prevent identification.