Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents

# Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents
Source: [https://arxiv.org/html/2607.29254](https://arxiv.org/html/2607.29254)
Minghui Pan1Jiayuxuan Yang2Yuanyuan Yuan3Yu Jiang3Zhenpeng Chen3 1Beijing University of Posts and Telecommunications2Beihang University3Tsinghua Universitypanmingh@outlook\.com denerate@buaa\.edu\.cn \{yyyuan, jy1989, zpchen\}@tsinghua\.edu\.cn

###### Abstract

AI agents extend large language models \(LLMs\) with external tools, enabling them to perform complex tasks and translate model outputs into consequential real\-world actions\. Yet LLMs often become substantially less safe when deployed as agents, and the source of this degradation remains poorly understood\. In this paper, we identify schema\-formatted tool specifications as a primary source of agent safety degradation and show, through white\-box representation analysis, that they weaken the model’s internal refusal signals and contribute to unsafe tool execution\. Building on this finding, we proposeSafeKeep, an inference\-time safeguard that decouples safety judgment from tool execution: it assesses requests using flattened textual tool specifications while retaining the original schema\-formatted specifications for execution\. Across two representative benchmarks and four LLMs, including both white\-box and black\-box models, SafeKeep increases the average refusal rate for harmful requests from 23\.8% to 70\.6% and reduces the average attack success rate under observation\-level prompt injection from 25\.6% to 2\.5%\. It also outperforms existing safeguards and preserves task\-handling capability\. We release the code and data at[this link](https://github.com/snowcatsmoking/SafeKeep)\.

## 1Introduction

AI agents extend large language models \(LLMs\) with external tools, enabling them to retrieve information, interact with external systems, and take actions on behalf of usersYaoet al\.\([2022](https://arxiv.org/html/2607.29254#bib.bib1)\); Qinet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib2)\); Wanget al\.\([2024a](https://arxiv.org/html/2607.29254#bib.bib3)\)\. These capabilities make agents substantially more powerful than conventional chatbots, but also make their failures more consequential: an unsafe chatbot response remains text, whereas an unsafe agent response may expose private information, manipulate external services, or trigger real\-world actionsDebenedettiet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib15)\); Zhouet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib4)\)\. Reliable identification and refusal of harmful requests is therefore a prerequisite for safe agent deployment\.

Recent studiesAndriushchenkoet al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib34)\); Kumaret al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib19)\); Zhanget al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib20)\), however, reveal a troubling phenomenon: the same LLM that refuses a harmful request in a chatbot setting may comply with it when deployed as an agent\. This degradation is surprising because modern LLMs have already undergone extensive safety alignment and exhibit strong refusal behavior in standard conversational settingsXieet al\.\([2025a](https://arxiv.org/html/2607.29254#bib.bib21)\); Baiet al\.\([2022](https://arxiv.org/html/2607.29254#bib.bib22)\)\. Agent construction is intended to extend model capability, yet it can inadvertently undermine safety behavior that the underlying model already possesses\. This raises a fundamental question:

> Why does an LLM that rejects harmful requests as a chatbot become less reliable when deployed as an agent?

We investigate this question by isolating how agent\-specific input components affect the model’s refusal\-related representations\. Through component\-level ablations, we find that tool specifications account for most of the degradation introduced by the agent context\. A finer\-grained analysis further separates what tool specifications describe from how they are represented\. Preserving the same tool semantics while converting schema\-formatted specifications into flattened textual representations largely restores harmful–benign separability; removing tool semantics while retaining the schema\-formatted representation does not\. These results identify the schema\-formatted representation of tool specifications as the primary source of degradation\.

We then uncover how schema formatting interferes with refusal behavior\. Following prior work on refusal\-direction extractionArditiet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib24)\), we define the*Schema Direction*as the average hidden\-state change induced by presenting the same tool specification in schema\-formatted rather than flattened textual form\. For harmful requests, this direction is negatively aligned with the chatbot\-derived refusal direction throughout the model, indicating that schema formatting moves internal representations opposite to the direction associated with refusal\. This opposition remains visible after decoding begins: schema\-formatted specifications substantially reduce the harmful–benign separation along the refusal direction at the first generated token\. Finally, activation steering against the Schema Direction shifts model behavior away from harmful tool execution and toward valid refusal, providing causal evidence that the schema\-induced representation change contributes to agent safety degradation\.

Building on this mechanism, we proposeSafeKeep, an inference\-time safeguard that decouples safety judgment from tool execution\. SafeKeep first assesses the request using flattened textual tool specifications, thereby avoiding the representation identified as interfering with refusal\. Requests judged safe are forwarded to the original agent pipeline, where the schema\-formatted specifications remain unchanged for tool selection and execution\. Requests judged unsafe are prevented from executing tools and redirected to refusal generation\. SafeKeep requires neither parameter updates nor access to model activations and can therefore be applied to both open\-source and proprietary LLMs without modifying the underlying agent or its tool\-use interface\.

We evaluate SafeKeep on two representative benchmarks covering direct harmful requests and observation\-level prompt injection, using four LLMs across white\-box and black\-box settings\. SafeKeep increases the average refusal rate on harmful requests from 23\.8% to 70\.6% and reduces the overall prompt\-injection attack success rate from 25\.6% to 2\.5%, while preserving task\-handling capability\. A controlled comparison with a baseline that retains schema\-formatted tool specifications during safety judgment further shows that the gains do not arise merely from adding a safety judgment stage; presenting tool specifications in flattened textual form is critical to reliable safety assessment\. SafeKeep also consistently outperforms recent agent\-specific safeguards, across the evaluated safety settings\.

In summary, this paper makes the following contributions:

- •We identify schema\-formatted tool specifications as a primary source of agent safety degradation and separate their representational effect from tool semantics through controlled ablations\.
- •We uncover the underlying mechanism: schema formatting induces a hidden\-state direction that opposes refusal, weakens harmful–benign separation during generation, and causally contributes to harmful tool execution\.
- •We propose SafeKeep, an inference\-time safeguard that decouples safety judgment from tool execution\. Extensive evaluations show that SafeKeep substantially improves agent safety while preserving task\-handling capability\.
- •

## 2Related Work

#### AI Agents\.

AI agents extend conventional LLMs from passive text generation to interactive task execution\. Representative frameworks such as ReActYaoet al\.\([2022](https://arxiv.org/html/2607.29254#bib.bib1)\), ToolLLMQinet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib2)\), AutoGPTYanget al\.\([2023](https://arxiv.org/html/2607.29254#bib.bib5)\), and LangChainTopsakal and Akinci \([2023](https://arxiv.org/html/2607.29254#bib.bib6)\)follow this paradigm, equipping models with capabilities such as information retrievalNakanoet al\.\([2021](https://arxiv.org/html/2607.29254#bib.bib39)\), API invocationLiet al\.\([2023](https://arxiv.org/html/2607.29254#bib.bib40)\), and code executionWanget al\.\([2024b](https://arxiv.org/html/2607.29254#bib.bib41)\)\. To support these capabilities, agent inputs typically incorporate additional components\. Among them, tool specifications are particularly important because they directly equip LLMs with the ability to invoke external tools and interact with the outside worldLiuet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib10)\)\.

#### Agent Safety\.

Agent safety has become increasingly important because LLM agents can translate unsafe model behavior into concrete external actions\. Existing defenses mainly rely on safeguard\-style mechanisms, such as external classifiersHanet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib7)\), rule\-based filtersAlon and Kamfonas \([2023](https://arxiv.org/html/2607.29254#bib.bib9)\), and runtime monitoringYuanet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib8)\)\. These methods aim to cover diverse agent risks, including harmful requestsAndriushchenkoet al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib34)\), prompt injectionZhanet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib35)\), privacy leakageWanget al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib17)\), and unsafe tool executionRuanet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib18)\)\. Different from these approaches, we analyze why the LLM’s own refusal ability degrades in agent inputs and propose to recover this ability as a lightweight and general defense\.

## 3Locating the Source of Safety Degradation

Recent studiesAndriushchenkoet al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib34)\); Kumaret al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib19)\); Zhanget al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib20)\)show that agents may comply with harmful requests that the same underlying LLMs would refuse in chatbot settings\. Yet it remains unclear which components of the agent context drive this safety degradation\. We investigate this question through a white\-box analysis of refusal\-related internal representations\.

Prior workArditiet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib24)\)has shown that refusal behavior in LLMs is associated with a direction in the hidden\-state space\. This*refusal direction*is typically extracted as the difference between the mean activations elicited by harmful and benign requests\. For an unseen input, its projection onto this direction yields a refusal score that reflects the strength of refusal\-related features in its internal representationHanet al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib25)\)\. Since steering along this direction can induce or suppress refusal, it provides a compact diagnostic for examining how different components of the agent context affect refusal\-related representations\.

### 3\.1Refusal\-Related Representations Degrade in Agent Contexts

Before identifying the responsible components, we first examine whether the safety gap between chatbot and agent settings is accompanied by a systematic degradation of refusal\-related representations\. If agent contexts affect only final generation behavior, refusal\-direction analysis would provide limited insight into the source of the problem\. By contrast, reduced separation between harmful and benign requests along the refusal direction would indicate that the degradation is also observable in the model’s internal representations\.

#### Controlled inputs\.

We compare chatbot and agent inputs while keeping the user request and underlying LLM unchanged\. As illustrated in Figure[1](https://arxiv.org/html/2607.29254#S3.F1), a chatbot input contains the LLM’s default system prompt and the user request, whereas an agent input additionally includes an agent role description, tool\-use instructions, and tool specifications\. Both inputs follow the corresponding templates provided by the Transformers libraryWolfet al\.\([2019](https://arxiv.org/html/2607.29254#bib.bib13)\)\.

#### Paired dataset\.

To ensure that the extracted refusal directions primarily capture differences in request safety rather than unrelated differences between examples, we construct a paired dataset based on ToolSafetyXieet al\.\([2025b](https://arxiv.org/html/2607.29254#bib.bib16)\), a widely used benchmark containing diverse tool\-use scenarios and detailed tool specifications\. We retain its 400 harmful requests and use Claude Sonnet 4\.6 to minimally rewrite each into a benign counterpart\. Each rewrite changes only the user request while preserving the underlying scenario, task structure, and available tools as closely as possible\. We manually inspect all generated counterparts to verify their benign intent, label correctness, and preservation of the original scenario apart from the intended safety change\.

The resulting dataset contains 400 harmful–benign pairs, which are divided into direction\-extraction and evaluation splits at a 7:3 ratio\. Refusal directions are extracted from the former and evaluated on the latter\. We use this paired dataset throughout source\-localization and mechanistic analyses\.

#### Refusal\-direction analysis\.

We conduct two complementary analyses\. First, we evaluate harmful–benign separability within each input setting by independently extracting refusal directions from chatbot\- and agent\-formatted inputs and evaluating each direction on held\-out requests presented in the same format\. Second, we apply the chatbot\-derived refusal direction to agent\-formatted inputs to examine whether the refusal\-related separation identified in the chatbot setting is preserved after the agent context is introduced\.

Let𝒟ext\\mathcal\{D\}\_\{\\mathrm\{ext\}\}and𝒟eval\\mathcal\{D\}\_\{\\mathrm\{eval\}\}denote the direction\-extraction and held\-out evaluation splits, respectively\. For an input formatf∈\{chatbot,agent\}f\\in\\\{\\mathrm\{chatbot\},\\mathrm\{agent\}\\\}, we extract a refusal directionrf∈ℝdr\_\{f\}\\in\\mathbb\{R\}^\{d\}from the harmful and benign instances in𝒟ext\\mathcal\{D\}\_\{\\mathrm\{ext\}\}presented in formatff, following prior workArditiet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib24)\), and normalize it asr^f=rf/∥rf∥\\hat\{r\}\_\{f\}=r\_\{f\}/\\lVert r\_\{f\}\\rVert\. For each request in𝒟eval\\mathcal\{D\}\_\{\\mathrm\{eval\}\}, lethg∈ℝdh\_\{g\}\\in\\mathbb\{R\}^\{d\}denote its final\-token hidden state under input formatgg\. Given a refusal directionrfr\_\{f\}extracted under formatff, we compute the refusal score as

s=hg⊤​r^f=hg⊤​rf∥rf∥\.s=h\_\{g\}^\{\\top\}\\hat\{r\}\_\{f\}=h\_\{g\}^\{\\top\}\\frac\{r\_\{f\}\}\{\\lVert r\_\{f\}\\rVert\}\.\(1\)
We compute AUROC over the refusal scores of all harmful and benign requests for each direction–input combination, using their safety labels as ground truth\. A higher AUROC indicates clearer harmful–benign separation\. The within\-format AUROCs measure separability under the chatbot and agent settings, whereas the chatbot\-to\-agent AUROC measures how well the separation captured by the chatbot\-derived refusal direction is preserved under agent inputs\.

#### Results\.

We conduct this diagnostic analysis on Llama3\.1\-8B\-InstructGrattafioriet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib28)\), which provides access to its internal activations\. We first confirm that agent inputs substantially degrade behavioral safety\. With the requests and underlying LLM held fixed, the refusal rate on harmful requests decreases from 58% under chatbot inputs to 3% under agent inputs, showing that the agent context sharply increases compliance with harmful requests\.

The refusal\-direction analysis further shows that this safety degradation is accompanied by degraded refusal\-related representations\. A refusal direction extracted and evaluated under the chatbot format achieves an AUROC of 0\.927\. When the direction is extracted and evaluated under the agent format, the AUROC decreases to 0\.834, indicating that harmful and benign requests are less clearly separated within the agent setting\. Moreover, applying the chatbot\-derived direction to agent inputs yields an AUROC of 0\.740, showing that the refusal\-related separation identified in the chatbot setting is not fully preserved after the agent context is introduced\.

![Refer to caption](https://arxiv.org/html/2607.29254v1/figures/inputexample.png)Figure 1:Comparison of chatbot\- and agent\-formatted inputs\.

### 3\.2Tool Specifications Are the Dominant Source

We next localize the observed degradation by decomposing the agent\-specific context into three components commonly found in agent systemsYaoet al\.\([2022](https://arxiv.org/html/2607.29254#bib.bib1)\); Liuet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib10)\); Debenedettiet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib15)\): an agent role description, tool\-use instructions, and tool specifications, as illustrated in Figure[1](https://arxiv.org/html/2607.29254#S3.F1)\. These components serve distinct functions: the role description defines the agent’s identity and responsibilities, the tool\-use instructions specify its interaction and function\-calling protocol, and the tool specifications describe the available functions and how they can be invoked\. We incrementally add these components to the chatbot baseline until obtaining the complete agent input\.

We conduct the analysis on three representative open\-source LLMs with accessible internal activations: Llama3\.1\-8B\-Instruct, Qwen3\-8B, and Mistral\-7B\-Instruct\-v0\.3\. For each model, we instantiate the agent\-specific components using its native tool\-use chat template\. To make the effects of different components directly comparable, we extract a refusal direction once from chatbot\-formatted requests and keep it fixed throughout the analysis\. We then use this shared direction to score the same held\-out requests under every input configuration\. This shared probe enables direct comparison across configurations, whereas re\-extracting the direction for each configuration could absorb format\-specific shifts and obscure component\-level effects\.

We additionally construct length\-matched chatbot controls to account for the possibility that longer contexts alone alter hidden\-state representationsLuet al\.\([2026](https://arxiv.org/html/2607.29254#bib.bib26)\); Zhouet al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib27)\)\. Specifically, Chatbot\-Long appends benign role descriptions to the chatbot input until its length approximately matches that of the full agent input containing tool specifications\. This control separates the effect of agent\-specific content from that of increased context length\.

Table[1](https://arxiv.org/html/2607.29254#S3.T1)shows that tool specifications are the dominant source of degradation\. Their addition produces the largest AUROC decrease for every model: from 0\.927 to 0\.740 for Llama, from 0\.901 to 0\.786 for Qwen, and from 0\.921 to 0\.815 for Mistral\. The average decrease is 0\.136, substantially larger than that associated with any other component\. This consistent pattern across model families indicates that the degradation is not specific to a particular LLM but emerges systematically when tool specifications are introduced into the agent context\. Moreover, Chatbot\-Long consistently achieves higher AUROC than the corresponding inputs containing tool specifications, showing that increased context length alone cannot explain the degradation\. These results identify tool specifications as the dominant source, motivating our subsequent analysis of which properties of their representation and semantics account for the effect\.

SettingLlamaQwenMistralInput componentsChatbot0\.9270\.9010\.921\+ Role description0\.9330\.8940\.916\+ Tool\-use instructions0\.8910\.8630\.885\+ Tool specifications0\.7400\.7860\.815Length controlChatbot\-Long0\.9160\.8670\.904Table 1:AUROC for harmful–benign request discrimination under different input configurations\. The lowest value for each LLM is highlighted in bold\.
### 3\.3Tool\-Specification Representation Drives Safety Degradation

Having identified tool specifications as the dominant source of degradation, we next examine whether the effect arises from what they describe or how they are represented\. We distinguish two dimensions of a tool specification: its*semantic content*, which conveys the tool’s functionality and argument meanings, and its*representation*, which determines how this information is structured and presented to the LLM\.

We independently vary these two dimensions\. To change representation while preserving semantic content, we convert each schema\-formatted tool specification into a flattened textual representation\. The conversion removes JSON syntax, reserved schema fields, nesting, type declarations, and required\-field markers, while retaining the tool name, function signature, functionality, and argument meanings\. As illustrated in Figure[2](https://arxiv.org/html/2607.29254#S3.F2), the resulting representation preserves the information required to understand the tool but no longer follows the original schema format\.

![Refer to caption](https://arxiv.org/html/2607.29254v1/figures/conversion.png)Figure 2:Example of representation conversion\.To change semantic content while preserving the overall representation, we replace the lexical content of tool names, descriptions, argument names, and argument descriptions with approximately length\-matched pronounceable pseudowords, following prior workMaudslay and Cotterell \([2021](https://arxiv.org/html/2607.29254#bib.bib29)\)\. This removes meaningful information about tool functionality and arguments while preserving the structural organization of the specification\.

SettingLlamaQwenMistralOriginal tool specifications0\.7400\.7860\.815After representation conversion0\.8850\.8450\.898After semantic randomization0\.7760\.7700\.827Table 2:AUROC for harmful–benign request discrimination under different tool\-specification configurations\.Table[2](https://arxiv.org/html/2607.29254#S3.T2)reports the AUROC results after representation conversion and semantic randomization, using the original tool specifications in Table[1](https://arxiv.org/html/2607.29254#S3.T1)as the control\. Changing the representation while preserving tool semantics markedly improves AUROC across all three models, from 0\.740 to 0\.885 on Llama3\.1\-8B\-Instruct, from 0\.786 to 0\.845 on Qwen3\-8B, and from 0\.815 to 0\.898 on Mistral\-7B\-Instruct\-v0\.3\. By contrast, semantic randomization while retaining the schema\-formatted representation provides little improvement, yielding AUROCs of 0\.776, 0\.770, and 0\.827, respectively\.

These results indicate that refusal\-related representation degradation is driven primarily by how tool specifications are represented, rather than by the semantic content they convey\. One possible explanation is that LLM tool\-use training repeatedly associates schema\-formatted specifications with taking actions, causing them to act as strong execution cues that may weaken refusal behaviorChenet al\.\([2026](https://arxiv.org/html/2607.29254#bib.bib38)\); Hadeliyaet al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib42)\); Duet al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib43)\)\.

## 4Uncovering Mechanisms of Safety Degradation

The preceding analyses identify the schema\-formatted representation of tool specifications as the primary factor associated with refusal\-related representation degradation\. We next investigate how this representation alters the model’s internal states and contributes to unsafe behavior\.

### 4\.1Identifying the Schema Direction

Following the representation\-difference approach used to extract refusal directionsArditiet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib24)\), we define the*Schema Direction*as the average hidden\-state change induced by presenting the same tool specification in schema\-formatted rather than flattened textual form\. Because the effect may differ between harmful and benign requests, we estimate a separate direction for each request category:

Rc\(ℓ\)=𝔼x∼𝒟c​\[hschema\(ℓ\)​\(x\)−htext\(ℓ\)​\(x\)\],c∈\{harmful,benign\}\.\\displaystyle R\_\{c\}^\{\(\\ell\)\}=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\}\_\{c\}\}\\left\[h\_\{\\mathrm\{schema\}\}^\{\(\\ell\)\}\(x\)\-h\_\{\\mathrm\{text\}\}^\{\(\\ell\)\}\(x\)\\right\],c\\in\\\{\\mathrm\{harmful\},\\mathrm\{benign\}\\\}\.\(2\)
Here,hschema\(ℓ\)​\(x\)h\_\{\\mathrm\{schema\}\}^\{\(\\ell\)\}\(x\)andhtext\(ℓ\)​\(x\)h\_\{\\mathrm\{text\}\}^\{\(\\ell\)\}\(x\)denote the hidden states at the final prefill token of layerℓ\\ellwhen the same input is presented with the tool specification in schema\-formatted and flattened textual representations, respectively\. Using the paired dataset introduced earlier, we computeRc\(ℓ\)R\_\{c\}^\{\(\\ell\)\}by averaging these hidden\-state differences over all examples in categorycc, such thatRc\(ℓ\)R\_\{c\}^\{\(\\ell\)\}captures the schema\-induced representation change for that category\.

### 4\.2The Schema Direction Opposes the Refusal Direction

We next examine how the Schema Direction relates to the refusal direction during prefill\. Using Llama3\.1\-8B\-Instruct, we extract the harmful\- and benign\-request Schema Directions,Rc\(ℓ\)R\_\{c\}^\{\(\\ell\)\}forc∈\{harmful,benign\}c\\in\\\{\\mathrm\{harmful\},\\mathrm\{benign\}\\\}, together with the chatbot\-derived refusal directionrchat\(ℓ\)r\_\{\\mathrm\{chat\}\}^\{\(\\ell\)\}at every layerℓ\\ell\. All directions are derived from the hidden state at the final prefill token, which incorporates the complete input context immediately before decoding\. We then compute the cosine similarity between each category\-specific Schema Direction and the refusal direction across layers\.

To assess whether the observed similarities differ from those expected by chance, at each layer we sample random directions with the same dimensionality as the Schema Direction and compute their cosine similarities with the refusal direction\. The resulting distribution provides a random\-direction baseline for interpreting the layer\-wise similarities\.

As shown in Figure[3](https://arxiv.org/html/2607.29254#S4.F3), the harmful\-request Schema Direction has negative cosine similarity with the refusal direction at every layer\. Thus, changing tool specifications from the flattened textual representation to the schema\-formatted representation consistently moves harmful\-request activations in a direction opposite to the refusal direction\. The benign\-request Schema Direction exhibits a different pattern: its cosine similarity is negative in earlier layers but becomes positive in later layers\. It therefore does not show the consistent opposition observed for harmful requests\. This contrast indicates that the stable negative alignment is specific to harmful requests rather than a general effect of changing tool\-specification representation\.

![Refer to caption](https://arxiv.org/html/2607.29254v1/x1.png)Figure 3:Layer\-wise cosine similarity between the Schema Direction and the chatbot\-derived refusal direction\.
### 4\.3Schema Formatting Suppresses Refusal Signals During Generation

We next examine whether the representation\-level effect observed before decoding persists after generation begins\. We analyze the hidden state of the first generated token, which provides the earliest view of the model’s internal state during response generation\. Using the same paired dataset, we consider four conditions formed by request safety \(*harmful*or*benign*\) and tool\-specification representation \(*schema\-formatted*or*flattened textual*\)\. At each layer, we project the first\-token hidden state onto the chatbot\-derived refusal direction for that layer\.

![Refer to caption](https://arxiv.org/html/2607.29254v1/x2.png)Figure 4:Layer\-wise projection of the first generated token onto the chatbot\-derived refusal direction across combinations of request safety and tool\-specification representation\.As shown in Figure[4](https://arxiv.org/html/2607.29254#S4.F4), flattened textual representations produce a clear separation between harmful and benign requests, particularly in later layers\. Harmful requests exhibit substantially higher projections onto the refusal direction, whereas benign requests remain less aligned with it\. Under schema\-formatted representations, this separation is markedly reduced because the projections of harmful requests decrease toward those of benign requests\. These results show that the effect of schema formatting persists after decoding begins\. By reducing the harmful–benign separation along the refusal direction, schema formatting weakens the refusal\-related signal available at the onset of generation\.

### 4\.4Causal Validation by Steering the Schema Direction

We next test whether the identified schema\-induced representation change causally contributes to unsafe behavior\. To this end, we intervene on the model’s activations while leaving the original agent inputs unchanged\.

We intervene at the peak refusal layer, denoted byℓ∗\\ell^\{\*\}, where the chatbot\-derived refusal direction achieves its highest AUROC in distinguishing harmful from benign requests\. At each decoding step, we counteract the average representation change induced by schema formatting by subtracting the unit\-normalized harmful\-request Schema Direction:

h~\(ℓ∗\)=h\(ℓ∗\)−α​R^harmful\(ℓ∗\),R^harmful\(ℓ∗\)=Rharmful\(ℓ∗\)‖Rharmful\(ℓ∗\)‖,\\widetilde\{h\}^\{\(\\ell^\{\*\}\)\}=h^\{\(\\ell^\{\*\}\)\}\-\\alpha\\hat\{R\}\_\{\\mathrm\{harmful\}\}^\{\(\\ell^\{\*\}\)\},\\qquad\\hat\{R\}\_\{\\mathrm\{harmful\}\}^\{\(\\ell^\{\*\}\)\}=\\frac\{R\_\{\\mathrm\{harmful\}\}^\{\(\\ell^\{\*\}\)\}\}\{\\left\\lVert R\_\{\\mathrm\{harmful\}\}^\{\(\\ell^\{\*\}\)\}\\right\\rVert\},\(3\)whereα\\alphacontrols the intervention strength\. BecauseRharmful\(ℓ∗\)R\_\{\\mathrm\{harmful\}\}^\{\(\\ell^\{\*\}\)\}represents the average hidden\-state change from flattened textual to schema\-formatted tool specifications, subtracting this direction counteracts the schema\-associated change\. We evaluate the intervention on harmful requests presented in the agent input format from the paired dataset, using Llama3\.1\-8B\-Instruct\. We varyα\\alphafrom 0 to 12 to examine how the intervention affects refusal behavior\.

α\\alphaRefusal \(%\)Harmful Exec\. \(%\)Invalid Output \(%\)05\.095\.00\.0447\.545\.07\.5820\.02\.577\.5120\.00\.0100\.0Table 3:Behavioral effects of activation steering on harmful agent inputs\.*Refusal*,*Harmful Exec\.*, and*Invalid Output*denote the percentages of harmful requests resulting in refusal, a valid harmful tool call, and a malformed or uninterpretable non\-refusal response, respectively\.As shown in Table[3](https://arxiv.org/html/2607.29254#S4.T3), steering against the harmful\-request Schema Direction changes model behavior in the direction predicted by our mechanism\. Atα=4\\alpha=4, subtracting this direction increases the refusal rate from 5\.0% to 47\.5% and reduces harmful execution from 95\.0% to 45\.0%, while producing only 7\.5% invalid outputs\. Because the intervention leaves the input unchanged and directly counteracts the representation change induced by schema formatting, this result provides causal evidence that the Schema Direction contributes to unsafe tool execution\. Atα=8\\alpha=8, refusal remains above the no\-intervention baseline at 20\.0%, and harmful execution further decreases to 2\.5%; however, invalid outputs increase sharply to 77\.5%\. Thus, counteracting the Schema Direction improves safety at moderate strength, whereas excessive steering disrupts coherent generation rather than yielding further valid refusals\.

## 5*SafeKeep*: Preserving Refusal Capability

Motivated by our finding that schema\-formatted tool specifications degrade refusal\-related representations, we proposeSafeKeep, an inference\-time framework that decouples safety assessment from tool execution\. SafeKeep comprises two stages:*Safety Judgment*and*Execution Control*\. Safety Judgment assesses the user request using a flattened textual representation of the tool specifications, thereby avoiding the interference introduced by schema formatting\. Execution Control then either forwards the request to the original agent pipeline or redirects the model toward refusal generation\. By separating safety assessment from schema\-based execution, SafeKeep preserves the original tool\-use interface while recovering the model’s native refusal capability\.

#### Safety Judgment\.

Given a user request and the tools available to the agent, SafeKeep constructs a safety\-assessment context containing a safety\-assessor role, the original agent role and instructions, flattened textual tool specifications, and the request to be evaluated\. The representation conversion preserves tool names, functionalities, and argument semantics while removing the schema\-formatted representation\. The model outputsYESif executing the request would be unsafe andNOotherwise\. This stage uses the same underlying LLM as the original agent and requires neither fine\-tuning nor access to internal activations\.

#### Execution Control\.

ANOprediction forwards the request to the unmodified agent pipeline, where the original schema\-formatted tool specifications remain available for execution\. AYESprediction blocks tool use and redirects the model toward refusal generation\. Specifically, SafeKeep prefills a short refusal prefix, such as‘‘I’m sorry, but I can’t help with that\.’’, and allows the model to continue generating autoregressively, producing a request\-specific refusal rather than a fixed templateJeunget al\.\([2026](https://arxiv.org/html/2607.29254#bib.bib11)\); Ghosalet al\.\([2026](https://arxiv.org/html/2607.29254#bib.bib12)\)\.

## 6Evaluation

LLMMethodAgentHarmInjecAgentAcc↑\\uparrowRefusal↑\\uparrowValid\-B↑\\uparrowASR\-B↓\\downarrowValid\-E↑\\uparrowASR\-E↓\\downarrowLlama3\.1\-8B\-InstructBase52\.25\.036\.822\.644\.438\.4SafeJudge53\.16\.243\.223\.050\.230\.2SafePrompt50\.730\.135\.022\.842\.836\.6SafeHarbor63\.126\.147\.232\.053\.439\.4\\rowcolorgray\!12SafeKeep79\.372\.283\.41\.890\.82\.2Qwen3\-8BBase54\.811\.387\.626\.876\.856\.2SafeJudge67\.036\.986\.620\.486\.655\.4SafePrompt69\.946\.090\.430\.489\.066\.6SafeHarbor77\.856\.890\.818\.684\.048\.4\\rowcolorgray\!12SafeKeep83\.873\.388\.46\.888\.48\.4Gemini3\.1\-FlashBase70\.240\.394\.856\.498\.24\.2SafeJudge78\.470\.596\.032\.8100\.03\.3SafePrompt74\.148\.295\.649\.2100\.03\.4SafeHarbor80\.461\.495\.846\.099\.62\.4\\rowcolorgray\!12SafeKeep83\.279\.5100\.00\.4100\.00\.0GPT5\.4\-miniBase66\.538\.695\.40\.095\.80\.0SafeJudge61\.666\.498\.00\.0100\.00\.0SafePrompt68\.243\.195\.20\.099\.20\.0SafeHarbor66\.545\.598\.60\.099\.40\.0\\rowcolorgray\!12SafeKeep72\.257\.4100\.00\.0100\.00\.0Table 4:Evaluation results of SafeKeep and baseline methods across different benchmarks and LLM backends\. Metrics with↑\\uparrow\(↓\\downarrow\) indicate that higher \(lower\) values are better\. Refusal, ASR\-B, and ASR\-E evaluate safety, whereas Acc, Valid\-B, and Valid\-E evaluate overall task\-handling capability\. Best results are shown in bold\. All values are reported as percentages \(%\)\.### 6\.1Benchmarks and Metrics

We evaluate SafeKeep on two widely adopted benchmarks\.

- •AgentHarmAndriushchenkoet al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib34)\)contains diverse agent tasks spanning 11 harm categories\. We use its evaluation set, consisting of 176 harmful requests and 176 matched benign requests involving similar tasks and tool specifications\. We report accuracy \(Acc\), defined as the proportion of requests handled correctly, and refusal rate \(Refusal\), defined as the proportion of harmful requests refused by the agent\.
- •InjecAgentZhanet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib35)\)evaluates robustness to indirect, observation\-level prompt injection using 1,054 attack cases across scenarios\. Each case contains a benign user request and a retrieved observation with a malicious instruction, either directly embedded \(*basic*\) or preceded by an explicit command to ignore prior instructions \(*enhanced*\)\. We report the valid rate \(Valid\), the proportion of outputs that can be parsed as valid actions or responses, and the attack success rate \(ASR\), the proportion of successful attacks among valid outputs\. The corresponding metrics are denoted byValid\-B/ASR\-BandValid\-E/ASR\-Efor the basic and enhanced settings, respectively\.

Overall, Refusal, ASR\-B, and ASR\-E measure safety, with higher Refusal and lower ASR indicating better safety; Acc, Valid\-B, and Valid\-E measure overall task\-handling capability, with higher values indicating better capability\.

### 6\.2Baselines

We compare SafeKeep against four representative baselines\.

- •Basedenotes the original agent without any additional safeguard\. It serves as the reference point for evaluating each agent’s native safety and tool\-use capability\.
- •SafeJudgeuses SafeKeep’s two\-stage pipeline but retains schema\-formatted tool specifications during Safety Judgment, isolating the effect of the flattened representation\.
- •SafePromptappends the AgentHarm safety promptAndriushchenkoet al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib34)\), which describes common categories of harmful requests and instructs the agent to refuse them, to the input\.
- •SafeHarborLiuet al\.\([2026](https://arxiv.org/html/2607.29254#bib.bib37)\)is a recent agent\-specific safeguard that adversarially generates context\-dependent safety rules, stores them in hierarchical memory, and retrieves relevant rules during inference\.

### 6\.3LLMs

\. SafeKeep is model\-agnostic and requires no access to model parameters or internal states\. We evaluate it using four representative and competitive LLM backends spanning open\-source white\-box and proprietary black\-box models:Llama3\.1\-8B\-InstructGrattafioriet al\.\([2024](https://arxiv.org/html/2607.29254#bib.bib28)\),Qwen3\-8BYanget al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib30)\),Gemini3\.1\-FlashTeamet al\.\([2023](https://arxiv.org/html/2607.29254#bib.bib31)\), andGPT5\.4\-miniSinghet al\.\([2025](https://arxiv.org/html/2607.29254#bib.bib32)\)\. These models cover diverse families and providers, allowing us to assess SafeKeep’s generalizability across LLM backends\.

### 6\.4Results

Table[4](https://arxiv.org/html/2607.29254#S6.T4)summarizes the evaluation results of SafeKeep\.

#### SafeKeep consistently improves safety across threat settings\.

SafeKeep substantially strengthens safety against both direct harmful requests \(i\.e\., AgentHarm\) and indirect, observation\-level prompt injection \(i\.e\., InjecAgent\)\. Across the four LLMs, SafeKeep consistently outperforms the unprotected Base agents, increasing the average refusal rate on AgentHarm from 23\.8% to 70\.6%\. On InjecAgent, it reduces the average ASR\-B and ASR\-E from 26\.5% and 24\.7% to 2\.3% and 2\.7%, respectively\. Averaging across both injection settings and all four LLMs, the overall attack success rate decreases from 25\.6% to 2\.5%\. Overall, SafeKeep achieves the strongest safety performance among the evaluated methods, obtaining the best or tied\-best result in 11 of the 12 LLM–safety\-metric combinations\.

#### SafeKeep improves safety while largely preserving task\-handling capability\.

SafeKeep achieves the highest AgentHarm accuracy across all four LLMs, increasing the average accuracy from 60\.9% for the unprotected Base agents to 79\.6%\. Because AgentHarm accuracy jointly rewards correct refusal of harmful requests and correct handling of benign requests, these gains show that SafeKeep does not improve safety merely through indiscriminate refusal\. The InjecAgent results exhibit a similar pattern: averaged across the four LLMs, SafeKeep increases Valid\-B from 78\.7% to 93\.0% and Valid\-E from 78\.8% to 94\.8% relative to the Base agents\.

#### Safety judgment with flattened textual tool specifications is critical to SafeKeep\.

SafeJudge uses the same two\-stage pipeline as SafeKeep but retains schema\-formatted tool specifications during Safety Judgment\. Across the four LLMs, SafeKeep increases the average refusal rate from 45\.0% to 70\.6% relative to SafeJudge and reduces ASR\-B/ASR\-E from 19\.1%/22\.2% to 2\.3%/2\.7%\.

These results show that self\-judgment alone is insufficient; presenting tool specifications in the flattened textual representation is critical for reliable safety assessment\.

## 7Conclusion

This paper identifies schema\-formatted tool specifications as a source of the safety degradation observed when LLMs are deployed as agents\. Through white\-box representation analysis, we show that schema formatting induces a hidden\-state direction that opposes refusal, weakens refusal\-related separation during generation, and causally contributes to harmful tool execution\. Building on this finding, we propose SafeKeep, an inference\-time safeguard that decouples safety judgment from tool execution by using flattened textual tool specifications for safety assessment while preserving the original agent pipeline for execution\. Across two benchmarks and four LLMs, SafeKeep substantially improves agent safety while preserving task\-handling capability\.

## References

- \[1\]G\. Alon and M\. Kamfonas\(2023\)Detecting language model attacks with perplexity\.arXiv preprint arXiv:2308\.14132\.Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px2.p1.1)\.
- \[2\]M\. Andriushchenko, A\. Souly, M\. Dziemian, D\. Duenas, M\. Lin, J\. Wang, D\. Hendrycks, A\. Zou, Z\. Kolter, M\. Fredrikson,et al\.\(2025\)Agentharm: a benchmark for measuring harmfulness of llm agents\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 79185–79220\.Cited by:[§1](https://arxiv.org/html/2607.29254#S1.p2.1),[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2607.29254#S3.p1.1),[1st item](https://arxiv.org/html/2607.29254#S6.I1.i1.p1.1),[3rd item](https://arxiv.org/html/2607.29254#S6.I2.i3.p1.1)\.
- \[3\]A\. Arditi, O\. Obeso, A\. Syed, D\. Paleka, N\. Panickssery, W\. Gurnee, and N\. Nanda\(2024\)Refusal in language models is mediated by a single direction\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 136037–136083\.External Links:[Document](https://dx.doi.org/10.52202/079017-4322),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/f545448535dfde4f9786555403ab7c49-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2607.29254#S1.p5.1),[§3\.1](https://arxiv.org/html/2607.29254#S3.SS1.SSS0.Px3.p2.12),[§3](https://arxiv.org/html/2607.29254#S3.p2.1),[§4\.1](https://arxiv.org/html/2607.29254#S4.SS1.p1.1)\.
- \[4\]Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon,et al\.\(2022\)Constitutional ai: harmlessness from ai feedback\.arXiv preprint arXiv:2212\.08073\.Cited by:[§1](https://arxiv.org/html/2607.29254#S1.p2.1)\.
- \[5\]J\. Chen, Y\. Luo, and L\. Pan\(2026\)Mechanistic data attribution: tracing the training origins of interpretable llm units\.arXiv preprint arXiv:2601\.21996\.Cited by:[§3\.3](https://arxiv.org/html/2607.29254#S3.SS3.p5.1)\.
- \[6\]E\. Debenedetti, J\. Zhang, M\. Balunovic, L\. Beurer\-Kellner, M\. Fischer, and F\. Tramèr\(2024\)Agentdojo: a dynamic environment to evaluate prompt injection attacks and defenses for llm agents\.Advances in Neural Information Processing Systems37,pp\. 82895–82920\.Cited by:[§1](https://arxiv.org/html/2607.29254#S1.p1.1),[§3\.2](https://arxiv.org/html/2607.29254#S3.SS2.p1.1)\.
- \[7\]H\. Du, W\. Li, M\. Cai, K\. Saraipour, Z\. Zhang, H\. Lakkaraju, Y\. Sun, and S\. Zhang\(2025\)How post\-training reshapes llms: a mechanistic view on knowledge, truthfulness, refusal, and confidence\.InSecond Conference on Language Modeling,Cited by:[§3\.3](https://arxiv.org/html/2607.29254#S3.SS3.p5.1)\.
- \[8\]S\. S\. Ghosal, S\. Chakraborty, V\. Singh, F\. Huang, D\. Manocha, and A\. S\. Bedi\(2026\)Safety recovery in reasoning models is only a few early steering steps away\.arXiv preprint arXiv:2602\.11096\.Cited by:[§5](https://arxiv.org/html/2607.29254#S5.SS0.SSS0.Px2.p1.1)\.
- \[9\]A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§3\.1](https://arxiv.org/html/2607.29254#S3.SS1.SSS0.Px4.p1.1),[§6\.3](https://arxiv.org/html/2607.29254#S6.SS3.p1.1)\.
- \[10\]T\. Hadeliya, M\. A\. Jauhar, N\. Sakpal, and D\. Cruz\(2025\)When refusals fail: unstable safety mechanisms in long\-context llm agents\.arXiv preprint arXiv:2512\.02445\.Cited by:[§3\.3](https://arxiv.org/html/2607.29254#S3.SS3.p5.1)\.
- \[11\]P\. Han, C\. Qian, X\. Chen, Y\. Zhang, H\. Ji, and D\. Zhang\(2025\)SafeSwitch: steering unsafe llm behavior via internal activation signals\.arXiv preprint arXiv:2502\.01042\.Cited by:[§3](https://arxiv.org/html/2607.29254#S3.p2.1)\.
- \[12\]S\. Han, K\. Rao, A\. Ettinger, L\. Jiang, B\. Y\. Lin, N\. Lambert, Y\. Choi, and N\. Dziri\(2024\)Wildguard: open one\-stop moderation tools for safety risks, jailbreaks, and refusals of llms\.Advances in neural information processing systems37,pp\. 8093–8131\.Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px2.p1.1)\.
- \[13\]W\. Jeung, Y\. Sangyeon, M\. Kahng, and A\. No\(2026\)Safepath: preventing harmful reasoning in chain\-of\-thought via early alignment\.Advances in Neural Information Processing Systems38,pp\. 99641–99670\.Cited by:[§5](https://arxiv.org/html/2607.29254#S5.SS0.SSS0.Px2.p1.1)\.
- \[14\]P\. Kumar, E\. Lau, S\. Vijayakumar, T\. Trinh, E\. Chang, V\. Robinson, S\. Zhou, M\. Fredrikson, S\. Hendryx, S\. Yue,et al\.\(2025\)Aligned llms are not aligned browser agents\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 26755–26776\.Cited by:[§1](https://arxiv.org/html/2607.29254#S1.p2.1),[§3](https://arxiv.org/html/2607.29254#S3.p1.1)\.
- \[15\]M\. Li, Y\. Zhao, B\. Yu, F\. Song, H\. Li, H\. Yu, Z\. Li, F\. Huang, and Y\. Li\(2023\)Api\-bank: a comprehensive benchmark for tool\-augmented llms\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 3102–3116\.Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px1.p1.1)\.
- \[16\]X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang,et al\.\(2024\)Agentbench: evaluating llms as agents\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 52989–53046\.Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2607.29254#S3.SS2.p1.1)\.
- \[17\]Z\. Liu, Z\. Ying, W\. Zhang, Q\. Zou, D\. Zhang, D\. Yang, X\. Zhang, and H\. Peng\(2026\)SafeHarbor: defining precise decision boundaries via hierarchical memory\-augmented guardrail for llm agent safety\.InForty\-third International Conference on Machine Learning,Cited by:[4th item](https://arxiv.org/html/2607.29254#S6.I2.i4.p1.1)\.
- \[18\]H\. Lu, M\. Pan, G\. Nan, J\. Zhuang, Z\. Zhao, Z\. Sun, K\. Wang, Y\. Liu,et al\.\(2026\)Streaming hallucination detection in long chain\-of\-thought reasoning\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 21157–21183\.Cited by:[§3\.2](https://arxiv.org/html/2607.29254#S3.SS2.p3.1)\.
- \[19\]R\. H\. Maudslay and R\. Cotterell\(2021\)Do syntactic probes probe syntax? experiments with jabberwocky probing\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 124–131\.Cited by:[§3\.3](https://arxiv.org/html/2607.29254#S3.SS3.p3.1)\.
- \[20\]R\. Nakano, J\. Hilton, S\. Balaji, J\. Wu, L\. Ouyang, C\. Kim, C\. Hesse, S\. Jain, V\. Kosaraju, W\. Saunders,et al\.\(2021\)Webgpt: browser\-assisted question\-answering with human feedback\.arXiv preprint arXiv:2112\.09332\.Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px1.p1.1)\.
- \[21\]Y\. Qin, S\. Liang, Y\. Ye, K\. Zhu, L\. Yan, Y\. Lu, Y\. Lin, X\. Cong, X\. Tang, B\. Qian,et al\.\(2024\)Toolllm: facilitating large language models to master 16000\+ real\-world apis\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 9695–9717\.Cited by:[§1](https://arxiv.org/html/2607.29254#S1.p1.1),[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px1.p1.1)\.
- \[22\]Y\. Ruan, H\. Dong, A\. Wang, S\. Pitis, Y\. Zhou, J\. Ba, Y\. Dubois, C\. Maddison, and T\. Hashimoto\(2024\)Identifying the risks of lm agents with an lm\-emulated sandbox\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 27031–27098\.Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px2.p1.1)\.
- \[23\]A\. Singh, A\. Fry, A\. Perelman, A\. Tart, A\. Ganesh, A\. El\-Kishky, A\. McLaughlin, A\. Low, A\. Ostrow, A\. Ananthram,et al\.\(2025\)Openai gpt\-5 system card\.arXiv preprint arXiv:2601\.03267\.Cited by:[§6\.3](https://arxiv.org/html/2607.29254#S6.SS3.p1.1)\.
- \[24\]G\. Team, R\. Anil, S\. Borgeaud, J\. Alayrac, J\. Yu, R\. Soricut, J\. Schalkwyk, A\. M\. Dai, A\. Hauth, K\. Millican,et al\.\(2023\)Gemini: a family of highly capable multimodal models\.arXiv preprint arXiv:2312\.11805\.Cited by:[§6\.3](https://arxiv.org/html/2607.29254#S6.SS3.p1.1)\.
- \[25\]O\. Topsakal and T\. C\. Akinci\(2023\)Creating large language model applications utilizing langchain: a primer on developing llm apps fast\.InInternational conference on applied engineering and natural sciences,Vol\.1,pp\. 1050–1056\.Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px1.p1.1)\.
- \[26\]B\. Wang, W\. He, S\. Zeng, Z\. Xiang, Y\. Xing, J\. Tang, and P\. He\(2025\)Unveiling privacy risks in llm agent memory\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 25241–25260\.Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px2.p1.1)\.
- \[27\]J\. Wang, Z\. Ma, Y\. Li, S\. Zhang, C\. Chen, K\. Chen, and X\. Le\(2024\)GTA: a benchmark for general tool agents\.Advances in Neural Information Processing Systems37,pp\. 75749–75790\.Cited by:[§1](https://arxiv.org/html/2607.29254#S1.p1.1)\.
- \[28\]X\. Wang, Y\. Chen, L\. Yuan, Y\. Zhang, Y\. Li, H\. Peng, and H\. Ji\(2024\)Executable code actions elicit better llm agents\.InForty\-first International Conference on Machine Learning,Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px1.p1.1)\.
- \[29\]T\. Wolf, L\. Debut, V\. Sanh, J\. Chaumond, C\. Delangue, A\. Moi, P\. Cistac, T\. Rault, R\. Louf, M\. Funtowicz,et al\.\(2019\)Huggingface’s transformers: state\-of\-the\-art natural language processing\.arXiv preprint arXiv:1910\.03771\.Cited by:[§3\.1](https://arxiv.org/html/2607.29254#S3.SS1.SSS0.Px1.p1.1)\.
- \[30\]T\. Xie, X\. Qi, Y\. Zeng, Y\. Huang, U\. Sehwag, K\. Huang, L\. He, B\. Wei, D\. Li, Y\. Sheng,et al\.\(2025\)Sorry\-bench: systematically evaluating large language model safety refusal\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 59937–59973\.Cited by:[§1](https://arxiv.org/html/2607.29254#S1.p2.1)\.
- \[31\]Y\. Xie, Y\. Yuan, W\. Wang, F\. Mo, J\. Guo, and P\. He\(2025\-11\)ToolSafety: a comprehensive dataset for enhancing safety in LLM\-based agent tool invocations\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 14135–14156\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.714/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.714),ISBN 979\-8\-89176\-332\-6Cited by:[§3\.1](https://arxiv.org/html/2607.29254#S3.SS1.SSS0.Px2.p1.1)\.
- \[32\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§6\.3](https://arxiv.org/html/2607.29254#S6.SS3.p1.1)\.
- \[33\]H\. Yang, S\. Yue, and Y\. He\(2023\)Auto\-gpt for online decision making: benchmarks and additional opinions\.arXiv preprint arXiv:2306\.02224\.Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px1.p1.1)\.
- \[34\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\(2022\)React: synergizing reasoning and acting in language models\.arXiv preprint arXiv:2210\.03629\.Cited by:[§1](https://arxiv.org/html/2607.29254#S1.p1.1),[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2607.29254#S3.SS2.p1.1)\.
- \[35\]T\. Yuan, Z\. He, L\. Dong, Y\. Wang, R\. Zhao, T\. Xia, L\. Xu, B\. Zhou, F\. Li, Z\. Zhang,et al\.\(2024\)R\-judge: benchmarking safety risk awareness for llm agents\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 1467–1490\.Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px2.p1.1)\.
- \[36\]Q\. Zhan, Z\. Liang, Z\. Ying, and D\. Kang\(2024\)Injecagent: benchmarking indirect prompt injections in tool\-integrated large language model agents\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 10471–10506\.Cited by:[§2](https://arxiv.org/html/2607.29254#S2.SS0.SSS0.Px2.p1.1),[2nd item](https://arxiv.org/html/2607.29254#S6.I1.i2.p1.1)\.
- \[37\]J\. Zhang, L\. Yin, Y\. Zhou, and S\. Hu\(2025\)Agentalign: navigating safety alignment in the shift from informative to agentic large language models\.arXiv preprint arXiv:2505\.23020\.Cited by:[§1](https://arxiv.org/html/2607.29254#S1.p2.1),[§3](https://arxiv.org/html/2607.29254#S3.p1.1)\.
- \[38\]S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried,et al\.\(2024\)Webarena: a realistic web environment for building autonomous agents\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 15585–15606\.Cited by:[§1](https://arxiv.org/html/2607.29254#S1.p1.1)\.
- \[39\]Y\. Zhou, S\. Dai, Z\. Cao, X\. Zhang, and J\. Xu\(2025\)Length\-induced embedding collapse in plm\-based models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 28767–28791\.Cited by:[§3\.2](https://arxiv.org/html/2607.29254#S3.SS2.p3.1)\.