Agent Safety Should Be a Runtime Contract

Hugging Face Daily Papers Papers
agent-safety runtime-contract ai-safety alignment sandboxing evidence-chain

Summary

This paper argues that AI agent safety should be enforced at runtime via preventive controls and verifiable evidence, rather than relying solely on training-time alignment. It grounds the position in audits of safety incidents, false completions, trajectory schemas, and publication trends.

The dominant paradigm treats AI safety as a property to be instilled during model training via RLHF, DPO, or Constitutional AI. We argue this is structurally insufficient for autonomous agents that execute code, mutate files, send messages, and modify databases. Agent safety should be a runtime contract enforced by the harness, and the contract has two complementary faces. The preventive face blocks dangerous actions before they happen via sandboxes, permission gates, output filters, and trajectory monitors. The evidential face requires verifiable proof that good actions actually happened, gating task submission on hard evidence such as test runs, log captures, file diffs, and citation grounding. We ground the position in four lines of public evidence, with row-level protocols and data released in the supplementary JSON files: a survey of 52 documented AI-agent and LLM safety incidents, a false-completion audit with 31 non-contested core cases plus one disputed illustrative case, a trajectory-schema audit of 12 public agent systems and harnesses, and a title-level audit of all 28,560 papers accepted at NeurIPS, ICML, and ICLR 2023-2025 showing a pooled 8-12x imbalance between training-time and deployment-time publication. Two prior communities that needed to enforce safety, computer security and the experimental sciences, converged on runtime contracts with both preventive and evidential elements; agentic AI is now under the same pressure. We formalize an Agent Trajectory Schema and Evidence Chain, state a compositional gating proposition based on standard monitor composition, and outline a research agenda. The right unit of safety in agentic AI is the trajectory-with-checkable-evidence, not the model.
Original Article
View Cached Full Text

Cached at: 08/13/26, 03:35 PM

Paper page - Agent Safety Should Be a Runtime Contract

Source: https://huggingface.co/papers/2608.11274

Abstract

Agent safety should be enforced at runtime through preventive controls and verifiable evidence rather than relying solely on training-time alignment methods.

The dominant paradigm treats AI safety as a property to be instilled during model training viaRLHF,DPO, orConstitutional AI. We argue this is structurally insufficient for autonomous agents that execute code, mutate files, send messages, and modify databases. Agent safety should be a runtime contract enforced by the harness, and the contract has two complementary faces. The preventive face blocks dangerous actions before they happen viasandboxes,permission gates,output filters, andtrajectory monitors. The evidential face requires verifiable proof that good actions actually happened, gating task submission on hard evidence such as test runs, log captures,file diffs, andcitation grounding. We ground the position in four lines of public evidence, with row-level protocols and data released in the supplementary JSON files: a survey of 52 documented AI-agent and LLM safety incidents, a false-completion audit with 31 non-contested core cases plus one disputed illustrative case, atrajectory-schema auditof 12 public agent systems and harnesses, and a title-level audit of all 28,560 papers accepted at NeurIPS, ICML, and ICLR 2023-2025 showing a pooled 8-12x imbalance between training-time and deployment-time publication. Two prior communities that needed to enforce safety, computer security and the experimental sciences, converged on runtime contracts with both preventive and evidential elements; agentic AI is now under the same pressure. We formalize anAgent Trajectory SchemaandEvidence Chain, state acompositional gatingproposition based on standardmonitor composition, and outline a research agenda. The right unit of safety in agentic AI is the trajectory-with-checkable-evidence, not the model.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2608\.11274

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.11274 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.11274 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.11274 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Agent Safety Is Action Alignment

arXiv cs.AI

This paper argues that applying content-safety refusal methods to AI agents is a category error—agentic harm lies in authority misuse rather than output—and proposes action alignment enforced outside the model via least privilege.

Agent rules need to exist where the action happens

Reddit r/AI_Agents

The article argues that AI agent safety rules should be implemented as hard workflow constraints and permissions rather than relying solely on prompt instructions. It emphasizes the need for explicit checks, approvals, and logs for sensitive or irreversible actions.

AI safety is arguing about the wrong boundary

Reddit r/AI_Agents

This article argues that the AI safety debate is misdirected, focusing on model alignment and internal controls instead of the critical boundary: external admission authority over agent execution. It warns that systems capable of self-authorizing high-impact actions (e.g., deploying code, moving money) pose a fundamental risk that logging and monitoring cannot mitigate.