Paper page - EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents

Source: https://huggingface.co/papers/2609.05903

Abstract

EvoSafeHarness optimizes deployable safety harnesses by jointly searching natural-language policies and executable logic tailored to a frozen model and target domain, improving safety-utility trade-offs across agent benchmarks.

Large Language Model (LLM) agents are turning language into real-world effects, making safety necessary against bothindirect prompt injections and direct harmful requests.System-level safety harnesses add an enforcement layer beyond model-level defenses, but existing harnesses are usually designed once by experts and applied across heterogeneous models and domains. Effective protection is deployment-dependent: models differ in how much enforcement they need before utility declines, while domains differ in the effects, state, and action sequences that must be governed. A harness that is strict enough for one model may over-block another, and a policy that transfers across domains may miss application-specific safety relations. We presentEvoSafeHarness, a safety-specific optimization framework that synthesizes a deployable harness for a frozen model in a target domain. It jointly searches anatural-language policyandexecutable code logic, guided by model behavior, domain specifications, and fresh-contextadversarial reviewto reject benchmark-specific rules. Across four agent benchmark families,EvoSafeHarnessachieves a strongersafety-utility frontierthan fixed expert-designed defenses. OnDecodingTrust-Agent, it reduces averageattack success ratefrom 45.6% to 10.0% at a 3.3-point utility cost and achieves the best score in 14 of 15 cells. OnAgentDojo, it reaches 82.8% utility at 0.0% ASR, twice CaMeL’s utility at the same operating point, and transfers unchanged to unseen AgentDyn suites. It also achieves the best score on Agent-SafetyBench for every victim and keeps mean ASR below 20% under adaptivePAIR attackswith a refinement budget of 16. Analysis shows that domain semantics determine which safety relations and trajectory state are needed, while model and runtime behavior determine how and where those relations should be enforced.

View arXiv pageView PDFProject pageGitHub1Add to collection

Get this paper in your agent:

hf papers read 2609\.05903

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.05903 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.05903 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.05903 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.