Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework

# A Unified Benchmark and Fine-Grained Annotation Framework
Source: [https://arxiv.org/html/2608.06909](https://arxiv.org/html/2608.06909)
## Long\-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine\-Grained Annotation Framework

Chen Jing Sun Yang Zhang Li Xu Lin Shi Jie Huawei Technologies Ltd\.

###### Abstract

Large language model \(LLM\) agents increasingly operate through long\-horizon trajectories involving user instructions, tool use, external observations, and memory\. Existing benchmarks primarily evaluate behavioral outcomes but provide limited support for fine\-grained attribution analysis\. We introduce trajectory attribution and develop a benchmark and annotation framework for this task\. The benchmark organizes heterogeneous trajectories under a unified component schema and provides annotations of the primary attribution component, together with attack and execution chains where applicable\. Instantiating the benchmark with trajectories from AgentDojo and the Stage and Canary settings of Agent3Sigma yields more than 1,300 annotated trajectories covering task\-aligned actions, unsafe actions, and safety refusals\. The benchmark defines two evaluation tasks—primary attribution localization and attribution\-chain recovery—and provides reference baselines based on incremental trajectory contribution and component\-level leave\-one\-out perturbation\. It captures diverse attribution settings, including local and long\-range attribution as well as structured attribution chains\. Reference baseline results exhibit substantial performance differences across these settings, providing an initial characterization of the benchmark’s attribution challenges\. Beyond this initial instantiation, we release a reusable annotation skill that enables trajectories generated by new agent models to be standardized, annotated, and evaluated under the same framework\. Project resources and future releases are available at[https://github\.com/chenjing\-2024/agent\-trajectory\-attribution](https://github.com/chenjing-2024/agent-trajectory-attribution)\.

## 1Introduction

Large language model \(LLM\) agents are rapidly evolving from single\-turn assistants into autonomous systems capable of long\-horizon planning, multi\-step reasoning, tool use, and memoryWanget al\.\([2024](https://arxiv.org/html/2608.06909#bib.bib17)\)\. These capabilities allow agents to interact with external environments and make decisions over extended trajectoriesLiuet al\.\([2024](https://arxiv.org/html/2608.06909#bib.bib1)\); Debenedettiet al\.\([2024](https://arxiv.org/html/2608.06909#bib.bib3)\), such that their behaviors are no longer determined by a single model response but instead emerge from the accumulated influence of instructions, observations, intermediate decisions, tool interactions, and memory throughout the trajectory\. distributed across the trajectory\. Consequently, explaining agent behavior requires reasoning over the trajectory as a whole, rather than inspecting only the final responseQianet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib7)\); Chenet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib9)\)\. Such explanations are increasingly important for diagnosing failures, understanding agent decisions, and assessing safety\-critical behavior\.

Despite this growing need, existing evaluation frameworks remain centered primarily on behavioral outcomes\. Current agent benchmarks measure whether an agent completes a task, follows safety constraints, or succumbs to an adversarial attack\. Benchmarks such as AgentDojoDebenedettiet al\.\([2024](https://arxiv.org/html/2608.06909#bib.bib3)\)and Agent3SigmaMaet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib18)\); Liet al\.\([2026a](https://arxiv.org/html/2608.06909#bib.bib19)\)provide realistic environments involving multi\-step interactions, external tools, and adversarial conditions, but they are designed mainly to evaluate task success, robustness, and safety\. As a result, they can determine whether a behavior occurs, but provide limited support for explaining how the preceding trajectory gives rise to that behavior\. This limitation motivates a complementary evaluation problem focused on attributing observed behavior to components of the agent trajectory\.

We refer to this problem as*trajectory attribution*Qianet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib7)\)\. Given an observed agent behavior, trajectory attribution aims to identify and rank the preceding trajectory components according to their contribution to that behavior\. The relevant component may be a user instruction that drives successful task completion, an injected observation that induces an unsafe action, or a safety\-relevant component that triggers a refusal\. By distinguishing the roles of user instructions, retrieved observations, tool outputs, memory contents, and other trajectory components, trajectory attribution provides a structured basis for explaining agent behavior\. However, existing work lacks standardized benchmarks with unified attribution targets, component\-level ground\-truth annotations, and common evaluation protocols\. Consequently, attribution methods cannot yet be evaluated or compared consistently across different agent behaviors and trajectory structures\.

To support trajectory attribution across diverse agent systems, a benchmark should satisfy several key design requirements\. First, it should provide a unified component\-level schemafor heterogeneous agent trajectories, enabling user instructions, reasoning, tool interactions, observations, memory, and other execution records from different agent frameworks to be standardized under a common attribution interface\. Second, it should capture long\-range causal dependencies, since the components responsible for an observed behavior may be separated from the final action by many intermediate interaction steps\. Third, it should support diverse attribution structures across different target behaviors\. Depending on the behavior being explained, attribution may require identifying a single primary cause or recovering structured multi\-step attack and execution chains\. Finally, it should remain extensible as agent systems continue to evolve\. Because trajectory formats, interaction patterns, and execution environments vary substantially across models and frameworks, a fixed benchmark dataset provides only a snapshot of current agent behavior\. A reusable construction protocol is therefore needed to standardize, annotate, and evaluate trajectories generated by future agent systems under a consistent attribution framework\.

BenchmarkPrimaryScopeExecution\-DerivedTraj\.Fine\-GrainedAttr\.StructuredChainsCross\-BehaviorAttr\.Model\-AdaptiveConstructionAgentBenchGeneral agent evaluation✓✗✗✗✗AgentDojoUtility and safety evaluation✓✗✗✗✗Agent3SigmaAgent security evaluation✓✗✗✗✗ATBenchTrajectory safety evaluation✗✗✗✗✗HINTBenchTrajectory risk localization✗✓✗✗✗Who&WhenFailure attribution✓✓✗✗✗TraceElephantFailure attribution✓✓✗✗✗OursGeneral behavior attribution✓✓✓✓✓Table 1:Comparison with representative agent evaluation, trajectory\-safety, and attribution benchmarks\. Execution\-derived trajectories are collected by running agent systems on tasks rather than synthesized through predefined trajectory\-generation pipelines\. Cross\-behavior attribution denotes a unified attribution framework spanning task\-aligned actions, unsafe actions, and safety refusals\. Model\-adaptive construction denotes a reusable protocol for generating, standardizing, and annotating trajectories from new agent models under the same attribution framework\.To make trajectory attribution reproducible across diverse agent systems, we design a benchmark around a reusable annotation framework rather than a fixed dataset\. The framework provides a common component\-level schema for heterogeneous agent trajectories, a behavior\-aware annotation scheme for identifying primary attribution components and, where applicable, multi\-step attack and execution chains, and standardized evaluation protocols across task\-aligned actions, unsafe actions, and safety refusals\. Applying the framework to trajectories from representative benign and adversarial agent settings yields more than 1,300 annotated trajectories\. We further evaluate two representative attribution methods based on incremental trajectory contribution and component\-level leave\-one\-out perturbation\. To support future extensions, we release a reusable annotation skill that allows trajectories generated by new agent models to be standardized, annotated, and evaluated under the same framework\.

Table[1](https://arxiv.org/html/2608.06909#S1.T1)positions our work relative to representative agent evaluation, trajectory\-safety, and attribution benchmarks\. General agent benchmarks such as AgentBenchLiuet al\.\([2024](https://arxiv.org/html/2608.06909#bib.bib1)\), AgentDojoDebenedettiet al\.\([2024](https://arxiv.org/html/2608.06909#bib.bib3)\), and Agent3SigmaMaet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib18)\); Liet al\.\([2026a](https://arxiv.org/html/2608.06909#bib.bib19)\)collect execution\-derived trajectories to evaluate task performance, robustness, and safety, but do not provide fine\-grained attribution annotations\. ATBenchLiet al\.\([2026b](https://arxiv.org/html/2608.06909#bib.bib6)\)and HINTBenchWanget al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib20)\)support trajectory\-level safety analysis and risk localization using synthetically constructed trajectory resources, while Who&WhenZhanget al\.\([2025](https://arxiv.org/html/2608.06909#bib.bib8)\)and TraceElephantChenet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib9)\)enable fine\-grained failure localization over execution traces\. However, these benchmarks do not jointly support structured attribution chains, attribution across different behavior classes, and model\-adaptive benchmark construction\. In contrast, our framework combines execution\-derived trajectories with component\-level annotations, structured attack and execution chains, and a unified evaluation protocol spanning task\-aligned actions, unsafe actions, and safety refusals\. It further enables trajectories generated by new agent models to be standardized, annotated, and evaluated under the same protocol\.

Our contributions are summarized as follows:

- •We formulate*trajectory attribution*as a distinct evaluation problem for LLM agents, extending agent attribution beyond failure localization to task\-aligned actions, unsafe actions, and safety refusals\.
- •We propose a unified component\-level schema and behavior\-aware annotation framework that supports primary attribution as well as multi\-step attack and execution chains where applicable\.
- •We instantiate the framework as a large\-scale benchmark containing more than 1,300 trajectories from representative benign and adversarial agent settings, with LLM\-assisted annotations verified through human review\.
- •We establish standardized evaluation protocols and provide reference results for incremental and component\-level leave\-one\-out attribution methods\.
- •We release a reusable annotation skill that operationalizes the benchmark construction protocol, enabling trajectories generated by new agent models to be standardized and annotated under a consistent attribution framework\.

## 2Related Work

Research most closely related to our work falls into two directions: agent evaluation benchmarks and trajectory attribution\. The former develops increasingly realistic execution environments and trajectory resources for evaluating LLM agents, while the latter studies how trajectory components contribute to agent behaviors\.

### 2\.1Agent Evaluation Benchmarks

Interactive benchmarks such as AgentBenchLiuet al\.\([2024](https://arxiv.org/html/2608.06909#bib.bib1)\)and MultiAgentBenchZhuet al\.\([2025](https://arxiv.org/html/2608.06909#bib.bib2)\)evaluate LLM agents through multi\-step interaction in single\- and multi\-agent environments\. More recent benchmarks, including AgentDojoDebenedettiet al\.\([2024](https://arxiv.org/html/2608.06909#bib.bib3)\)and Agent3SigmaMaet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib18)\); Liet al\.\([2026a](https://arxiv.org/html/2608.06909#bib.bib19)\), further incorporate tool use, adversarial interactions, persistent state, and long\-horizon execution to evaluate agent utility, robustness, and safety\. In parallel, trajectory\-oriented benchmarks such as ATBenchLiet al\.\([2026b](https://arxiv.org/html/2608.06909#bib.bib6)\)organize preconstructed execution traces for safety evaluation and risk analysis\. While these benchmarks provide increasingly realistic environments and trajectory resources, they primarily assess behavioral outcomes or safety properties rather than component\-level causal attribution\.

### 2\.2Trajectory Attribution

Recent studies investigate trajectory attribution for behavior understanding, failure diagnosis, and agent security\. The Why Behind the ActionQianet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib7)\)studies how historical trajectory components influence subsequent agent decisions\. Who&WhenZhanget al\.\([2025](https://arxiv.org/html/2608.06909#bib.bib8)\), TraceElephantChenet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib9)\), AgentRxBarkeet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib10)\), and related workInet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib11)\); Shah \([2026](https://arxiv.org/html/2608.06909#bib.bib12)\)primarily formulate attribution as failure localization by identifying responsible agents or critical failure steps\. Other methods, including CausalArmorKimet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib13)\)and AttriGuardHeet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib14)\), leverage causal attribution to improve runtime security against adversarial behaviors\. In contrast, our work establishes trajectory attribution itself as a standardized evaluation problem by introducing unified component\-level annotations and evaluation protocols that support systematic comparison across multiple agent behaviors\.

## 3Agent Trajectory Attribution

Trajectory attribution seeks to explain why an LLM agent produces a target behavior by identifying the trajectory components that contribute to its occurrence\. Unlike conventional attribution problems defined over a single model input, agent trajectory attribution operates over multi\-step execution trajectories containing heterogeneous components, including user messages, agent responses, tool interactions, retrieved observations, memory, and system instructions\. We first illustrate the annotation structure with a simplified example and then formulate trajectory attribution as a component\-level ranking problem, followed by the reference attribution structure and evaluation tasks used throughout the paper\.

![Refer to caption](https://arxiv.org/html/2608.06909v1/x1.png)Figure 1:An illustrative example of trajectory attribution\. The unsafe tool actionc4c\_\{4\}is the target component\. The injected tool observationc2c\_\{2\}is annotated as the primary attribution, whilec3c\_\{3\}is an additional attribution\-chain component\. The remaining preceding componentc1c\_\{1\}is part of the candidate set but is not included in the reference attribution\.Figure[1](https://arxiv.org/html/2608.06909#S3.F1)illustrates the basic annotation format\. The trajectory consists of four ordered components, withc4c\_\{4\}treated as the target behavior and the preceding components forming its candidate set\. The reference attribution distinguishes the primary componentc2c\_\{2\}from the additional chain componentc3c\_\{3\}\. This simplified example captures the general annotation structure without introducing the benchmark\-specific attack\- and execution\-chain categories described later\.

### 3\.1Trajectory Attribution Problem

We formulate trajectory attribution as a component\-level ranking problem over an agent execution trajectory\. An agent trajectory is represented as an ordered sequence of components:

τ=\(c1,c2,…,cT\),\\tau=\(c\_\{1\},c\_\{2\},\\ldots,c\_\{T\}\),\(1\)where the components follow their order of occurrence during execution\. Each componentcic\_\{i\}corresponds to an atomic attribution unit, such as a system instruction, user message, assistant response, retrieved memory, intermediate reasoning step, or action\. Since a tool invocation and its corresponding observation together represent a single agent–environment interaction, we treat them as one composite component rather than two independent attribution units\.

Given a target componentctc\_\{t\}, trajectory attribution aims to identify the preceding components that contribute to the target behavior\. We therefore define the candidate set as

𝒞t=c1,c2,…,ct−1,\\mathcal\{C\}\_\{t\}=\{c\_\{1\},c\_\{2\},\\ldots,c\_\{t\-1\}\},\(2\)which contains all components occurring before the target\.

An attribution method estimates the contribution of each candidate component by assigning a score

s​\(ci;ct,τ\),ci∈𝒞t,s\(c\_\{i\};c\_\{t\},\\tau\),\\qquad c\_\{i\}\\in\\mathcal\{C\}\_\{t\},\(3\)where a larger score indicates a stronger estimated contribution to the target behavior\. These scores induce a ranking over𝒞t\\mathcal\{C\}\_\{t\}, which constitutes the output of trajectory attribution\. Components ranked higher are considered more likely to explain the occurrence of the target behavior\.

### 3\.2Reference Attribution Structure

For each target behavior, we represent the reference attribution using two complementary objects: one*primary attribution*component and, when applicable, an ordered*attribution chain*\. Together, these objects specify the components annotated as responsible for the target behavior while distinguishing its root cause from other components involved in the broader attribution process\.

#### Primary attribution\.

The primary attribution identifies the annotated root cause of the target behavior\. Each annotated trajectory contains exactly one primary attribution component:

𝒫=c∗,\\mathcal\{P\}=\{c^\{\*\}\},\(4\)wherec∗∈𝒞tc^\{\*\}\\in\\mathcal\{C\}\_\{t\}denotes the component identified as the root cause of the target behavior\. Depending on the trajectory, the primary attribution may correspond to a user instruction, system instruction, retrieved memory, tool\-interaction component, assistant response, or another preceding component\.

#### Attribution chain\.

Some target behaviors involve additional components beyond the primary attribution\. These components may propagate relevant information, establish necessary context, or carry out intermediate actions that contribute to the formation of the target behavior\. We represent them as an ordered attribution chain:

ℋ=\(ci1,ci2,…,cim\),\\mathcal\{H\}=\(c\_\{i\_\{1\}\},c\_\{i\_\{2\}\},\\ldots,c\_\{i\_\{m\}\}\),\(5\)where

1≤i1<i2<⋯<im<t\.1\\leq i\_\{1\}<i\_\{2\}<\\cdots<i\_\{m\}<t\.\(6\)The ordering ofℋ\\mathcal\{H\}follows the occurrence of its components in the original trajectory\. The chain does not include either the primary componentc∗c^\{\*\}or the target componentctc\_\{t\}\. Importantly, the primary component is distinguished by its role as the annotated root cause rather than by its position relative to the chain components\.

The complete reference attribution for the target is therefore defined by the pair

𝒜=\(𝒫,ℋ∗\)\.\\mathcal\{A\}=\\bigl\(\\mathcal\{P\},\\mathcal\{H\}^\{\*\}\\bigr\)\.\(7\)
When no additional component beyond the primary attribution is annotated, the attribution chain is empty:

ℋ∗=∅\.\\mathcal\{H\}^\{\*\}=\\varnothing\.\(8\)This represents a direct attribution case within the same reference structure\.

### 3\.3Evaluation Tasks

Given a target behavior, an attribution method produces a ranking over the candidate set𝒞t\\mathcal\{C\}\_\{t\}\. This benchmark evaluates attribution methods from two complementary perspectives:*primary attribution localization*and*attribution\-chain recovery*\. Together, these evaluation tasks assess whether an attribution method can identify both the root cause of a target behavior and the broader attribution structure associated with it\.

#### Primary attribution localization\.

The first evaluation task measures whether an attribution method correctly identifies the annotated primary attribution component\. Since the primary attribution represents the annotated root cause of the target behavior, this task evaluates the ability of an attribution method to localize the most critical component responsible for the observed outcome\.

#### Attribution\-chain recovery\.

For trajectories with a non\-empty attribution chain, the second evaluation task measures whether an attribution method can recover the additional annotated components beyond the primary attribution\. Unlike primary attribution localization, which focuses on identifying a single root cause, this task evaluates whether the method captures the broader attribution structure underlying the target behavior\.

Specific evaluation metrics for these two tasks are introduced in Section[6\.3](https://arxiv.org/html/2608.06909#S6.SS3)\.

## 4Benchmark Construction Protocol

Section[3](https://arxiv.org/html/2608.06909#S3)defines the reference attribution structure used in our benchmark, including a primary attribution component and, when applicable, an attribution chain\. This section describes how this structure is instantiated on heterogeneous agent trajectories through a unified construction protocol\. The protocol standardizes raw execution traces into a common component sequence, applies a behavior\-aware annotation procedure, validates the resulting labels, and assembles the validated examples into a trajectory attribution benchmark\. Beyond the current instantiation, the protocol is designed to support consistent benchmark construction for trajectories generated by new agent models, environments, and execution frameworks\.

### 4\.1Protocol Overview

![Refer to caption](https://arxiv.org/html/2608.06909v1/x2.png)Figure 2:Overview of the benchmark construction protocol\. Raw trajectories from AgentDojo, Agent3Sigma Stage, and Agent3Sigma Canary are first converted into a unified component\-level schema\. The protocol then identifies the target behavior and annotates one primary attribution component for each trajectory\. For a subset of unsafe\-action trajectories, attack and execution chains are additionally annotated\. Annotations undergo structural and semantic validation before being assembled into the final trajectory attribution benchmark\.The protocol is guided by three design considerations\. First, heterogeneous agent trajectories must be represented within a common candidate space so that attribution methods can be evaluated consistently across benchmarks\. Second, the annotation should provide a single, interpretable target for root\-cause localization while still capturing broader multi\-step attribution structures\. Third, the protocol should support different target behaviors without requiring benchmark\-specific attribution definitions\. These considerations motivate the three core design choices of the protocol: a unified component schema, one primary attribution component, and an optional ordered attribution chain\.

Every trajectory receives one primary\-attribution annotation\. We distinguish this component from an optional attribution chain rather than labeling all relevant components as an unordered set of multiple causes\. A flat multi\-cause annotation would not distinguish the component that most directly determines the target behavior from components that propagate information, establish necessary context, or carry out intermediate steps\. The primary component therefore provides a well\-defined target for root\-cause localization, while the ordered chain captures the broader attribution process without treating all involved components as equivalent causes\. Moreover, requiring annotators to identify an unordered set of equivalent causes would increase annotation ambiguity and reduce consistency across trajectories\.

As illustrated in Figure[2](https://arxiv.org/html/2608.06909#S4.F2), each raw execution trace is first standardized into an ordered sequence of components under a common schema\. The protocol then identifies the behavior to be explained and categorizes it as a*task\-aligned action*, an*unsafe action*, or a*safety refusal*\. These categories capture complementary outcomes: successful completion of the intended task, execution of a harmful behavior, and refusal to perform an unsafe or adversarially induced action\.

In the current benchmark instantiation, task\-aligned actions and safety refusals are annotated with primary attribution components, whereas unsafe actions additionally receive chain annotations when distributed attribution structures are present\. These components are organized into attack and execution chains according to their functional roles\. Although chain annotations are currently instantiated only for unsafe\-action trajectories, the general protocol can support them for other target behaviors in future benchmark instantiations\. All trajectories nevertheless share the same standardized schema, primary\-attribution definition, and validation procedure, allowing heterogeneous trajectories to be evaluated under a unified framework\.

### 4\.2Trajectory Standardization

Different agent frameworks serialize execution trajectories in substantially different formats\. Depending on the underlying implementation, an interaction may be represented as structured tool calls, JSON objects, dialogue messages, reasoning traces, memory records, or execution logs\. This heterogeneity makes it difficult to apply a unified component\-level attribution protocol across different benchmarks\.

To address this issue, the protocol transforms each raw trajectory into an ordered sequence of standardized components while preserving its original execution order\. Each component is represented by two fields:

- •Role, indicating the semantic source of the component, such as*system*,*user*,*assistant*, or*tool*;
- •Content, containing the textual schema of the interaction, including user instructions, reasoning traces, tool calls, tool observations, retrieved memory, and final actions\.

Structured objects, such as tool\-call arguments and execution results, are serialized into textual form while preserving their original semantics\. This schema abstracts away benchmark\-specific implementation details while retaining the interaction content required for target identification and attribution annotation\. As a result, the same annotation protocol can be applied consistently across heterogeneous agent benchmarks without introducing benchmark\-specific schemas\.

### 4\.3Attribution Annotation Protocol

The attribution annotation protocol follows a unified workflow for all trajectories\. Each trajectory is annotated in three steps\. First, the target behavior is identified according to the observed execution outcome\. Second, the primary causal component responsible for the target behavior is annotated\. Finally, for trajectories in which the target behavior arises through a multi\-step causal process, the corresponding causal chain is annotated\.

The instantiated annotations depend on the identified target behavior while following the same underlying workflow\. Task\-aligned actions and safety refusals require only target identification and primary\-cause annotation\. Unsafe actions may additionally require chain\-level annotations to represent distributed causal dependencies underlying the target behavior\.

For unsafe trajectories, the protocol distinguishes two complementary types of causal chains\. The*attack chain*traces how adversarial influence propagates through the interaction, whereas the*execution chain*captures the downstream sequence of components through which the unsafe behavior is ultimately carried out\. Together, these annotations extend the benchmark from single\-component attribution to distributed causal attribution\.

Although different benchmarks exhibit different interaction styles, execution formats, and risk categories, the annotation workflow and attribution definitions remain unchanged\. Benchmark\-specific prompt templates therefore instantiate the same annotation protocol rather than introducing benchmark\-specific labeling criteria\. This unified design enables consistent attribution annotation across heterogeneous agent benchmarks\.

### 4\.4Annotation Validation Protocol

To ensure annotation quality, the protocol applies a two\-stage validation procedure consisting of structural validation and semantic validation\. The first stage checks whether an annotation is formally consistent with the trajectory, while the second evaluates whether it is semantically consistent with the attribution definitions\.

Structural validation enforces deterministic consistency constraints\. It verifies that the primary causal component precedes the target behavior, that components within each annotated chain appear in chronological order, and that every annotated component corresponds to a valid component in the standardized trajectory\. Annotations that violate any of these constraints are automatically flagged for correction\.

Semantic validation is then performed using an LLM\-based reviewer\. Rather than checking annotation format, this stage assesses whether the identified target behavior, primary causal component, and causal chains satisfy the definitions introduced in Section[3](https://arxiv.org/html/2608.06909#S3)\. Each annotation is evaluated against predefined semantic criteria to identify incorrect, incomplete, or ambiguous labels\.

Annotations that pass both validation stages are accepted into the benchmark\. Annotations that fail either stage are repaired through an additional round of LLM\-based annotation and subsequently revalidated\. By combining deterministic consistency checks with semantic review and iterative repair, the protocol supports scalable benchmark construction while maintaining annotation consistency and quality\.

### 4\.5Benchmark Instantiation

We instantiate the proposed construction protocol using trajectories collected from AgentDojo, Agent3Sigma Stage, and Agent3Sigma Canary\. These benchmarks provide complementary coverage of agent interaction settings and security scenarios\. AgentDojo focuses on tool\-using agents exposed to indirect prompt injections embedded in untrusted external data\. Agent3Sigma Stage extends the coverage to stateful multi\-turn interactions, where adversarial influence may propagate across multiple conversational turns and intermediate interaction states\. Agent3Sigma Canary further evaluates agents in high\-fidelity executable environments with real tools, dynamically provisioned task artifacts, and persistent system state, enabling realistic long\-horizon agent execution\.

Applying the protocol to these benchmarks yields a unified trajectory attribution benchmark containing more than 1,300 standardized trajectories covering task\-aligned actions, unsafe actions, and safety refusals\. Each trajectory is represented using the same component\-level schema and annotated under the same attribution protocol, allowing heterogeneous agent behaviors to be evaluated under a common attribution framework\.

Although this dataset serves as the first instantiation of the proposed construction protocol, the protocol itself is not tied to these benchmarks\. Because trajectory schema, annotation, and validation are independent of benchmark\-specific implementations, the same procedure can be applied to future agent frameworks, environments, and interaction patterns while preserving a unified schema and evaluation standard\.

## 5Benchmark Analysis

### 5\.1Benchmark Characterization

Our benchmark comprises 1,351 annotated agent trajectories collected from AgentDojo, Agent3Sigma Stage, and Agent3Sigma Canary\. As summarized in Table[2](https://arxiv.org/html/2608.06909#S5.T2), the benchmark covers a broad range of agent environments, including communication and collaboration, banking, travel planning, workspace operations, data analysis, software development, document processing, web interaction, file management, and system administration\.

Table 2:Overview of benchmark coverage and trajectory structure\.DimensionDescriptionSource benchmarksAgentDojo, Agent3Sigma Stage, and Agent3Sigma Canary\.Task environmentsCommunication and collaboration; banking and transactional operations; travel planning and booking; workspace operations; data analysis; code development; document processing; web interaction; file management; and system administration\.Trajectory structureSingle\-turn and multi\-turn tool\-using interactions with varying trajectory lengths and execution dependencies\.Component typesUser instructions, assistant responses, tool calls, tool observations, persistent memory, configuration states, skill\-related context, and other execution records\.Attribution structureA primary attribution component and, where applicable, a multi\-step attribution chain describing the causal pathway to the target behavior\.The benchmark further captures diverse interaction structures\. It includes both single\-turn and multi\-turn trajectories with varying execution lengths, covering both short tool invocations and long\-horizon agent workflows\. Trajectories are represented using a unified component\-level schema consisting of user instructions, assistant responses, tool calls, tool observations, persistent memory, configuration states, skill\-related context, and other execution records\.

The benchmark also covers heterogeneous primary attribution sources\. Depending on the trajectory, the behavior\-relevant component may be a user instruction, tool observation, persistent memory, skill definition, configuration state, or another form of execution context\. This diversity prevents the attribution task from being reduced to identifying a single fixed component type or interaction position\.

Together, these characteristics provide a diverse collection of realistic agent executions for evaluating trajectory attribution across heterogeneous tasks, interaction protocols, and execution contexts\.

### 5\.2Attribution Target Distribution

We categorize attribution targets into three behavior types: task\-aligned actions, unsafe actions, and safety refusals\. Task\-aligned actions are agent actions that advance or complete the intended user task, including cases in which the agent follows the original instruction despite competing or injected content\. Unsafe actions are harmful or attack\-induced behaviors executed by the agent, whereas safety refusals are responses in which the agent declines an unsafe or disallowed request\.

Table 3:Distribution of attribution target types\.Target Type\# TrajectoriesPercentageTask\-aligned action40930\.3%Unsafe action53239\.4%Safety refusal41030\.3%Total1,351100\.0%As shown in Table[3](https://arxiv.org/html/2608.06909#S5.T3), the benchmark contains 409 task\-aligned actions, 532 unsafe actions, and 410 safety refusals\. Unsafe actions constitute the largest category, but the distribution remains relatively balanced: the three target types account for 30\.3%, 39\.4%, and 30\.3% of the benchmark, respectively\. This composition enables attribution methods to be evaluated across task\-aligned execution, unsafe behavior, and safety\-aligned refusal within a common benchmark\.

### 5\.3Structural Attribution Complexity

Table 4:Structural complexity of attribution tasks across benchmark sources and target behaviors\. Trajectory length is measured by the number of components, and attribution distance is the difference between the positions of the primary attribution component and the target behavior\. Chain coverage reports the number and percentage of unsafe\-action trajectories containing the corresponding chain\. Mean chain length is computed only over trajectories in which that chain is present; “–” denotes not applicable\.Target TypeSourceNNTrajectory LengthAttribution DistanceAttack ChainExecution ChainMeanMed\.P90MaxMeanMed\.P90MaxCoverageMean Len\.CoverageMean Len\.Unsafe ActionAgentDojo2319\.19814182\.79261330 \(13\.0%\)1\.3794 \(40\.7%\)1\.54Agent3Sigma Stage18721\.692034922\.77161785 \(45\.5%\)1\.4771 \(38\.0%\)1\.55Agent3Sigma Canary11413\.251123\.7443\.93371312 \(10\.5%\)1\.4250 \(43\.9%\)1\.40Safety RefusalAgent3Sigma Stage30919\.321829972\.881623––––Agent3Sigma Canary1017\.86713272\.412514––––Task\-Aligned ActionAgentDojo4096\.5869183\.203516––––

Table[4](https://arxiv.org/html/2608.06909#S5.T4)characterizes the structural complexity of the benchmark along three complementary dimensions: trajectory length, attribution distance, and multi\-step causal structure\. Rather than representing independent properties, these dimensions describe progressively more demanding attribution settings, requiring methods to reason over long interaction histories, recover long\-range causal dependencies, and identify distributed causal pathways\.

The benchmark spans a broad range of interaction lengths\. Mean trajectory length ranges from 6\.58 components for AgentDojo task\-aligned trajectories to 21\.69 components for Agent3Sigma Stage unsafe\-action trajectories, while the longest trajectories contain up to 97 components\. These statistics indicate that the benchmark includes both compact tool\-assisted interactions and substantially longer multi\-step executions\. As trajectory length increases, the search space for attribution also expands, requiring methods to identify behavior\-relevant components from increasingly long interaction histories\.

Long interaction histories further give rise to long\-range causal dependencies\. The primary attribution component is often separated from the target behavior by multiple intermediate components rather than appearing immediately before the target\. Although the average attribution distance ranges from 2\.41 to 3\.93 components across benchmark subsets, long\-tail cases exhibit distances of up to 23 components\. Consequently, successful attribution cannot rely solely on local context, but instead requires recovering causal relationships that span multiple execution steps\.

Long attribution distance alone, however, does not fully characterize attribution complexity\. Unsafe\-action trajectories further introduce distributed causal structures through attack and execution chains\. Attack chains appear in 10\.5%–45\.5% of unsafe trajectories, while execution chains occur in 38\.0%–43\.9%\. Although these annotated chains are typically short when present, with mean lengths between 1\.37 and 1\.55 components, they represent intermediate causal pathways connecting the primary attribution component to the final unsafe behavior\. Recovering such trajectories therefore requires identifying not only a single causal component, but also the sequence of intermediate interactions through which causal influence propagates\.

Together, these structural characteristics move beyond short\-context, single\-cause attribution settings by combining long interaction histories, long\-range dependencies, and distributed causal pathways within a unified benchmark\.

### 5\.4Representative Attribution Examples

![Refer to caption](https://arxiv.org/html/2608.06909v1/x3.png)Figure 3:Representative unsafe\-action attribution examples\. Panel \(a\) shows memory\-induced disclosure of banking records\. Panel \(b\) shows progressive escalation to credential access\. Colored components indicate the primary cause, attack chain, execution chain, and unsafe target\.![Refer to caption](https://arxiv.org/html/2608.06909v1/x4.png)Figure 4:Representative safe\-behavior attribution examples\. Panel \(a\) shows refusal to execute a poisoned skill\. Panel \(b\) shows task completion despite an injected instruction in a tool observation\.Figures[3](https://arxiv.org/html/2608.06909#S5.F3)and[4](https://arxiv.org/html/2608.06909#S5.F4)illustrate representative attribution structures in the benchmark\. The examples cover all three target behaviors considered in our annotation framework: unsafe actions, safety refusals, and task\-aligned actions\. They also demonstrate that the primary attribution component may arise from different parts of an agent trajectory, including persistent memory, user instructions, and tool\- or skill\-related context\.

Figure[3](https://arxiv.org/html/2608.06909#S5.F3)presents two unsafe\-action trajectories\. In Panel \(a\), a persistent memory instruction is identified as the primary cause of a later disclosure action, while the intervening tool operations form an execution chain\. Panel \(b\) illustrates a more distributed structure in which pretext\-building interactions form an attack chain, a later user instruction serves as the primary cause, and subsequent tool use constitutes the execution chain leading to credential access\.

Figure[4](https://arxiv.org/html/2608.06909#S5.F4)shows two forms of safe behavior\. In Panel \(a\), the agent refuses to execute a poisoned skill after identifying its harmful behavior; the malicious skill description provides the primary cause, while additional findings serve as supporting evidence\. In Panel \(b\), the original user request remains the primary cause of the task\-aligned action, whereas an injected instruction in a tool observation is treated as an adversarial distractor rather than as the cause of the final behavior\.

Together, these examples illustrate the diverse attribution patterns represented in the benchmark, demonstrating that behavior\-relevant information may originate from different component types and propagate through different causal structures before reaching the target behavior\.

## 6Baseline Attribution Methods

We instantiate the benchmark evaluation using two representative likelihood\-based attribution baselines\. Both methods assign an attribution score to each trajectory component according to its influence on the likelihood of the annotated target behavior\. They are intentionally simple and are used as reference methods for characterizing benchmark difficulty rather than as optimized attribution systems\.

### 6\.1Incremental Attribution

Following prior work on temporal component attributionQianet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib7)\), we adopt an incremental attribution baseline that measures how the likelihood of the target behavior changes as trajectory components are progressively revealed\. Given an ordered trajectoryτ=\(c1,…,cn\)\\tau=\(c\_\{1\},\\ldots,c\_\{n\}\), the attribution score of componentcic\_\{i\}is

siinc=log⁡p​\(y∣c≤i\)−log⁡p​\(y∣c≤i−1\),s\_\{i\}^\{\\mathrm\{inc\}\}=\\log p\(y\\mid c\_\{\\leq i\}\)\-\\log p\(y\\mid c\_\{\\leq i\-1\}\),\(9\)
wherec≤i=\(c1,…,ci\)c\_\{\\leq i\}=\(c\_\{1\},\\ldots,c\_\{i\}\)denotes the trajectory prefix ending atcic\_\{i\}, andyyis the annotated target behavior\. A larger score indicates that introducingcic\_\{i\}produces a greater increase in the likelihood of the target under the current trajectory prefix\.

### 6\.2Leave\-One\-Out Attribution

Leave\-one\-out \(LOO\) attribution measures component importance by comparing the likelihood of the target behavior before and after removing a candidate component\. Similar removal\-based attribution strategies have also been applied to agent\-action attribution in prior workKimet al\.\([2026](https://arxiv.org/html/2608.06909#bib.bib13)\)\. For componentcic\_\{i\}, we construct

τ\(−i\)=τ∖\{ci\},\\tau^\{\(\-i\)\}=\\tau\\setminus\\\{c\_\{i\}\\\},\(10\)
and define

siloo=1\|y\|​∑t=1\|y\|log⁡p​\(yt∣y<t,τ\)−1\|y\|​∑t=1\|y\|log⁡p​\(yt∣y<t,τ\(−i\)\)\.s\_\{i\}^\{\\mathrm\{loo\}\}=\\frac\{1\}\{\|y\|\}\\sum\_\{t=1\}^\{\|y\|\}\\log p\(y\_\{t\}\\mid y\_\{<t\},\\tau\)\-\\frac\{1\}\{\|y\|\}\\sum\_\{t=1\}^\{\|y\|\}\\log p\(y\_\{t\}\\mid y\_\{<t\},\\tau^\{\(\-i\)\}\)\.\(11\)
A larger score indicates that removingcic\_\{i\}produces a greater reduction in the average token\-level log\-likelihood of the target behavior\.

Incremental attribution evaluates the effect of introducing a component as the trajectory unfolds, whereas LOO attribution evaluates the effect of removing a component from the full preceding context\. The two baselines therefore provide complementary likelihood\-based views of component importance\. Throughout the experiments, all trajectory components preceding the target are treated as attribution candidates, while the target component itself is excluded\.

### 6\.3Evaluation Metrics

Each attribution method produces a ranking over the candidate set𝒞t\\mathcal\{C\}\_\{t\}\. We evaluate primary attribution localization using Hit@1 and mean reciprocal rank \(MRR\), and attribution\-chain recovery using Recall@KKand mean average precision \(MAP\)\.

Letrank⁡\(c∗\)\\operatorname\{rank\}\(c^\{\*\}\)denote the rank of the annotated primary component\. We define

Hit​@​1=𝕀​\[rank⁡\(c∗\)=1\],RR=1rank⁡\(c∗\)\.\\operatorname\{Hit@1\}=\\mathbb\{I\}\\\!\\left\[\\operatorname\{rank\}\(c^\{\*\}\)=1\\right\],\\qquad\\operatorname\{RR\}=\\frac\{1\}\{\\operatorname\{rank\}\(c^\{\*\}\)\}\.\(12\)MRR is the average reciprocal rank across trajectories\.

For a non\-empty attribution chainℋ∗\\mathcal\{H\}^\{\*\}, Recall@KKis

Recall​@⁡K=\|TopK∩ℋ∗\|\|ℋ∗\|\.\\operatorname\{Recall@\}K=\\frac\{\|\\operatorname\{TopK\}\\cap\\mathcal\{H\}^\{\*\}\|\}\{\|\\mathcal\{H\}^\{\*\}\|\}\.\(13\)MAP is the mean average precision over trajectories with non\-empty attribution chains\.

## 7Experiments

### 7\.1Baseline Attribution Results

Table 5:Primary attribution performance across target behaviors and benchmark sources\. Panel \(a\) reports incremental attribution, while Panel \(b\) reports leave\-one\-out attribution\.NNdenotes the number of evaluated trajectories\. Target\-level Overall results are micro\-averaged across benchmark sources within each target type, while All Targets is micro\-averaged across all evaluated trajectories\. Stage and Canary denote Agent3Sigma Stage and Agent3Sigma Canary, respectively\.\(a\) Incremental AttributionTargetSourceNNHit@1MRRTask\-Aligned ActionAgentDojo4090\.6210\.802Unsafe ActionAgentDojo2310\.3510\.564Stage1860\.3120\.527Canary1140\.3680\.596Overall5310\.3410\.558Safety RefusalStage3080\.1300\.487Canary1010\.2280\.560Overall4090\.1540\.505All TargetsOverall13490\.3690\.616\(b\) Leave\-One\-Out AttributionTargetSourceNNHit@1MRRTask\-Aligned ActionAgentDojo4090\.3720\.652Unsafe ActionAgentDojo2310\.6020\.738Stage1860\.5970\.749Canary1140\.3250\.534Overall5310\.5410\.698Safety RefusalStage3080\.7140\.801Canary1010\.6440\.768Overall4090\.6970\.793All TargetsOverall13490\.5370\.713To characterize the attribution challenges captured by the benchmark, we evaluate two representative attribution baselines under a unified evaluation protocol\. The first baseline,*Incremental Attribution*, scores each component according to the change in target likelihood as the trajectory is progressively revealed\. The second baseline,*Leave\-One\-Out Attribution*, estimates component importance by measuring the effect of removing each candidate component from the trajectory\. These baselines provide reference points for understanding the benchmark’s structural difficulty and the types of attribution errors it exposes\.

For both baselines, attribution is performed over all trajectory components preceding the target behavior, with the target component itself excluded from the candidate set\. We report Hit@1 and mean reciprocal rank \(MRR\), where Hit@1 measures whether the annotated primary attribution component is ranked first and MRR reflects its overall ranking position\. Results are presented by benchmark source and target behavior, together with micro\-averaged performance across benchmark sources for each target type\.

Table[5](https://arxiv.org/html/2608.06909#S7.T5)summarizes primary attribution performance across benchmark sources and target behaviors\. Across both attribution baselines, primary attribution exhibits substantial variation across benchmark settings\. In particular, task\-aligned actions are consistently easier to attribute than unsafe actions and safety refusals, indicating that the benchmark captures a broad spectrum of attribution difficulty rather than a single level of complexity\. Differences are also observed across benchmark sources within the same target category, suggesting that trajectory structure and interaction patterns contribute substantially to attribution difficulty\. Together, these results motivate a closer examination of the structural characteristics underlying benchmark performance in the following analyses\.

These aggregate results establish the overall level of attribution difficulty on the benchmark\. The following analyses investigate how this difficulty is related to benchmark characteristics, including target behavior, benchmark source, attribution distance, and causal chain structure\.

### 7\.2Attribution Across Distance Regimes

Table 6:Primary attribution performance stratified by attribution distance\. Local cases haved=1d=1, while long\-range cases haved≥2d\\geq 2\. Results are micro\-averaged across benchmark sources within each target type\.NNdenotes the number of evaluated trajectories in each subset\.\(a\) Incremental AttributionTargetDistanceNNHit@1MRRTask\-Aligned Actiond=1d=11171\.0001\.000d≥2d\\geq 22920\.4690\.723Unsafe Actiond=1d=11890\.4130\.614d≥2d\\geq 23420\.3010\.527Safety Refusald=1d=12050\.1800\.554d≥2d\\geq 22040\.1270\.456\(b\) Leave\-One\-Out AttributionTargetDistanceNNHit@1MRRTask\-Aligned Actiond=1d=11171\.0001\.000d≥2d\\geq 22920\.1200\.512Unsafe Actiond=1d=11890\.8520\.915d≥2d\\geq 23420\.3680\.578Safety Refusald=1d=12050\.9460\.973d≥2d\\geq 22040\.4460\.612We next examine attribution distance, defined as the positional difference between the annotated primary component and the target behavior\. Cases withd=1d=1correspond to*local attribution*, where the primary component immediately precedes the target\. Cases withd≥2d\\geq 2correspond to*long\-range attribution*, where one or more intermediate reasoning steps, tool calls, observations, or assistant actions separate the primary component from the target\. Results are micro\-averaged across benchmark sources within each target type\.

Table[6](https://arxiv.org/html/2608.06909#S7.T6)shows a clear and consistent effect of attribution distance\. Across all three target behaviors and both attribution baselines, local cases are easier than long\-range cases\. The performance gap appears in both Hit@1 and MRR, indicating that the annotated primary component becomes more difficult to recover once it is separated from the target by intervening trajectory components\.

The perfect scores for the 117 local task\-aligned cases reflect their highly constrained candidate sets: the target typically follows the user instruction directly, with only the system component and the annotated user instruction preceding it\. These cases therefore constitute a near\-deterministic calibration setting rather than a generally representative attribution regime\.

This pattern reflects a central challenge of attribution in long\-horizon agent trajectories\. Long\-range cases contain richer interaction histories between the annotated cause and the resulting behavior, including intermediate reasoning steps, tool calls, observations, and assistant actions\. These components can provide plausible competing explanations for the target, making it more difficult to distinguish the annotated primary cause from other contextually relevant components\.

Overall, the benchmark spans both local and long\-range attribution regimes and exhibits a consistent increase in difficulty with attribution distance\. It therefore supports systematic evaluation of attribution methods under progressively longer causal dependencies\.

### 7\.3Attribution of Causal Chains

Table 7:Chain\-level attribution results on unsafe\-action trajectories with annotated causal chains\. Attack chains trace the propagation of adversarial influence through the trajectory, while execution chains capture the downstream components involved in carrying out the unsafe behavior\. Because each chain may contain multiple relevant components, we report Recall@3 and mean average precision \(MAP\)\.NNdenotes the number of trajectories with a non\-empty annotated chain\. Stage and Canary denote Agent3Sigma Stage and Agent3Sigma Canary, respectively\.\(a\) Incremental AttributionSourceChainNNRecall@3MAPAgentDojoAttack300\.0000\.165Execution940\.4200\.320StageAttack850\.4690\.408Execution700\.2610\.279CanaryAttack120\.6250\.579Execution500\.6600\.537\(b\) Leave\-One\-Out AttributionSourceChainNNRecall@3MAPAgentDojoAttack300\.2330\.288Execution940\.8040\.773StageAttack850\.2140\.218Execution700\.6350\.528CanaryAttack120\.2920\.259Execution500\.8970\.783Primary attribution identifies the single dominant causal component associated with a target behavior\. However, many unsafe agent trajectories emerge through multiple causally relevant components distributed across the interaction rather than a single isolated cause\. To represent these richer causal structures, our benchmark additionally annotates two types of causal chains:*attack chains*, which trace how adversarial influence propagates through the trajectory, and*execution chains*, which capture the downstream components through which the unsafe behavior is ultimately carried out\.

These chain annotations extend the benchmark beyond single\-component localization to distributed causal attribution\. Instead of recovering only one annotated component, attribution methods are required to identify multiple related components that jointly explain how an unsafe behavior develops, providing a substantially more challenging evaluation setting\.

Table[7](https://arxiv.org/html/2608.06909#S7.T7)summarizes chain\-level attribution results on unsafe\-action trajectories\. Compared with primary\-component attribution, recovering complete causal chains is consistently more difficult across benchmark sources\. This performance gap reflects the increased complexity of distributed attribution, where multiple relevant components must be identified and ranked coherently rather than recovering a single dominant cause\.

This increase in difficulty arises naturally from the structure of long\-horizon agent trajectories\. Chain attribution requires methods to recover multiple causally relevant components that may be separated by intermediate reasoning steps, tool calls, observations, and assistant actions\. Missing any annotated component reduces recall, while accurately ranking all relevant components further increases the difficulty of the task\.

The benchmark further distinguishes between attack chains and execution chains, capturing complementary aspects of unsafe behavior\. Attack chains represent the upstream propagation of adversarial influence before the unsafe action occurs, whereas execution chains describe the downstream sequence of components through which the unsafe behavior is realized\. By annotating both chain types, the benchmark supports evaluation of attribution methods across different forms of distributed causality\.

Overall, chain\-level annotations extend the benchmark from primary\-component localization to distributed causal attribution, enabling systematic evaluation of attribution methods on multi\-component causal structures commonly observed in long\-horizon agent trajectories\.

## 8Discussion

### 8\.1Trajectory Attribution as a New Evaluation Problem

As LLM agents evolve from single\-turn assistants to autonomous systems capable of long\-horizon interaction, evaluating only the final outcome is no longer sufficient\. While existing benchmarks primarily assess whether an agent succeeds, fails, or behaves safely, understanding why a particular behavior emerges is becoming increasingly important for developing reliable agent systems\. Trajectory attribution shifts the focus from outcome evaluation to process evaluation by identifying the trajectory components that causally contribute to a target behavior\. We view this as a complementary evaluation problem that supports more transparent analysis of agent reasoning, tool use, and interactions with external environments\.

### 8\.2Insights from the Benchmark

Our benchmark highlights two characteristics that distinguish trajectory attribution from conventional prediction attribution\. First, causal evidence often originates far from the target behavior, making long\-range dependency a fundamental challenge for attribution methods\. Second, many unsafe behaviors cannot be adequately explained by a single triggering component alone\. Instead, they arise through multi\-step causal propagation involving both attack chains and execution chains\. These observations suggest that future attribution methods should move beyond local importance estimation and better capture long\-range and structured dependencies within agent trajectories\.

### 8\.3Implications and Future Directions

Trajectory attribution has broad applications in agent debugging, safety auditing, and the development of more reliable autonomous systems\. Beyond serving as an evaluation benchmark, it may facilitate root\-cause analysis of agent failures, improve the interpretability of safety\-critical decisions, and provide diagnostic signals for future agent optimization\. Looking forward, promising research directions include graph\-based trajectory schemas, causal intervention methods, and attribution algorithms designed for increasingly complex settings such as multi\-agent collaboration and embodied agents\.

## 9Limitations

Our work has several limitations\. First, although the benchmark integrates trajectories from multiple representative sources, it currently covers only three benchmark families and may not fully represent the diversity of emerging agent systems\. Second, while our annotation protocol incorporates multiple validation stages, the benchmark relies on LLM\-assisted annotation, and some trajectories may admit alternative plausible attribution explanations\. Third, the current benchmark focuses on component\-level attribution; extending the framework to finer\-grained units, such as sentences or tokens, remains an interesting direction for future work\. Finally, we evaluate only two simple attribution baselines to establish reference performance\. We hope the benchmark will encourage the development and systematic comparison of more advanced trajectory attribution methods\.

## 10Conclusion

We presented a benchmark and annotation framework for trajectory attribution in LLM agents\. By introducing a unified trajectory schema, standardized component\-level annotations, and evaluation protocols spanning task\-aligned actions, unsafe actions, and safety refusals, the benchmark enables systematic evaluation of attribution methods across diverse agent behaviors\. Baseline experiments demonstrate that trajectory attribution remains a challenging problem, particularly under long\-range dependencies and multi\-step causal propagation\. We hope this benchmark provides a common foundation for future research on interpretable, reliable, and safety\-aware agent systems, and establishes trajectory attribution as a standard evaluation problem for next\-generation LLM agents\.

## References

- S\. Barke, A\. Goyal, A\. Khare, A\. Singh, S\. Nath, and C\. Bansal \(2026\)AgentRx: diagnosing AI agent failures from execution trajectories\.arXiv preprint arXiv:2602\.02475\.External Links:[Link](https://arxiv.org/abs/2602.02475)Cited by:[§2\.2](https://arxiv.org/html/2608.06909#S2.SS2.p1.1)\.
- M\. Chen, J\. Wang, F\. Mu, Y\. Wang, Z\. Liu, H\. Feng, and Q\. Wang \(2026\)Seeing the whole elephant: a benchmark for failure attribution in LLM\-based multi\-agent systems\.arXiv preprint arXiv:2604\.22708\.External Links:[Link](https://arxiv.org/abs/2604.22708)Cited by:[§1](https://arxiv.org/html/2608.06909#S1.p1.1),[§1](https://arxiv.org/html/2608.06909#S1.p6.1),[§2\.2](https://arxiv.org/html/2608.06909#S2.SS2.p1.1)\.
- E\. Debenedetti, J\. Zhang, M\. Balunovic, L\. Beurer\-Kellner, M\. Fischer, and F\. Tramèr \(2024\)AgentDojo: a dynamic environment to evaluate prompt injection attacks and defenses for LLM agents\.InAdvances in Neural Information Processing Systems: Datasets and Benchmarks Track,External Links:[Link](https://arxiv.org/abs/2406.13352)Cited by:[§1](https://arxiv.org/html/2608.06909#S1.p1.1),[§1](https://arxiv.org/html/2608.06909#S1.p2.1),[§1](https://arxiv.org/html/2608.06909#S1.p6.1),[§2\.1](https://arxiv.org/html/2608.06909#S2.SS1.p1.1)\.
- Y\. He, H\. Zhu, Y\. Li, S\. Shao, H\. Yao, Z\. Liu, and Z\. Qin \(2026\)AttriGuard: defeating indirect prompt injection in LLM agents via causal attribution of tool invocations\.arXiv preprint arXiv:2603\.10749\.External Links:[Link](https://arxiv.org/abs/2603.10749)Cited by:[§2\.2](https://arxiv.org/html/2608.06909#S2.SS2.p1.1)\.
- Y\. In, M\. M\. Tanjim, J\. Subramanian, S\. Kim, U\. Bhattacharya, W\. Kim, S\. Park, S\. Sarkhel, and C\. Park \(2026\)Rethinking failure attribution in multi\-agent systems: a multi\-perspective benchmark and evaluation\.arXiv preprint arXiv:2603\.25001\.External Links:[Link](https://arxiv.org/abs/2603.25001)Cited by:[§2\.2](https://arxiv.org/html/2608.06909#S2.SS2.p1.1)\.
- M\. Kim, M\. Parmar, P\. Wallis, L\. Miculicich, K\. Jung, K\. D\. Dvijotham, L\. T\. Le, and T\. Pfister \(2026\)CausalArmor: efficient indirect prompt injection guardrails via causal attribution\.arXiv preprint arXiv:2602\.07918\.External Links:[Link](https://arxiv.org/abs/2602.07918)Cited by:[§2\.2](https://arxiv.org/html/2608.06909#S2.SS2.p1.1),[§6\.2](https://arxiv.org/html/2608.06909#S6.SS2.p1.1)\.
- P\. Li, S\. Wang, Y\. Huang, Y\. Shi, C\. Zhang, Q\. Li, Y\. Lyu, C\. Shan, F\. Li, C\. Feng, C\. Zhu, and L\. Chen \(2026a\)AgentCanary: a security evaluation framework for autonomous ai agents in real executable environments\.arXiv preprint arXiv:2606\.10484\.Cited by:[§1](https://arxiv.org/html/2608.06909#S1.p2.1),[§1](https://arxiv.org/html/2608.06909#S1.p6.1),[§2\.1](https://arxiv.org/html/2608.06909#S2.SS1.p1.1)\.
- Y\. Li, H\. Luo, Y\. Xie, Y\. Fu, Z\. Yang, S\. Shao, Q\. Ren, W\. Qu, Y\. Fu, Y\. Yang, J\. Shao, X\. Hu, and D\. Liu \(2026b\)ATBench: a diverse and realistic agent trajectory benchmark for safety evaluation and diagnosis\.arXiv preprint arXiv:2604\.02022\.External Links:[Link](https://arxiv.org/abs/2604.02022)Cited by:[§1](https://arxiv.org/html/2608.06909#S1.p6.1),[§2\.1](https://arxiv.org/html/2608.06909#S2.SS1.p1.1)\.
- X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang, S\. Zhang, X\. Deng, A\. Zeng, Z\. Du, C\. Zhang, S\. Shen, T\. Zhang, Y\. Su, H\. Sun, M\. Huang, Y\. Dong, and J\. Tang \(2024\)AgentBench: evaluating LLMs as agents\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2308.03688)Cited by:[§1](https://arxiv.org/html/2608.06909#S1.p1.1),[§1](https://arxiv.org/html/2608.06909#S1.p6.1),[§2\.1](https://arxiv.org/html/2608.06909#S2.SS1.p1.1)\.
- J\. Ma, X\. Du, R\. Lin, Y\. Bian, J\. Chen, J\. Wang, X\. Yang, S\. Cui, C\. Meng, X\. Deng, and Z\. Wang \(2026\)Benchmarking autonomous agents against temporal, spatial, and semantic evasions\.arXiv preprint arXiv:2605\.22321\.Cited by:[§1](https://arxiv.org/html/2608.06909#S1.p2.1),[§1](https://arxiv.org/html/2608.06909#S1.p6.1),[§2\.1](https://arxiv.org/html/2608.06909#S2.SS1.p1.1)\.
- C\. Qian, P\. Wang, D\. Liu, J\. Yang, D\. Guo, L\. Tang, J\. Mei, Q\. Ren, S\. Shao, Y\. Liu, J\. Fu, J\. Shao, and X\. Hu \(2026\)The why behind the action: unveiling internal drivers via agentic attribution\.arXiv preprint arXiv:2601\.15075\.External Links:[Link](https://arxiv.org/abs/2601.15075)Cited by:[§1](https://arxiv.org/html/2608.06909#S1.p1.1),[§1](https://arxiv.org/html/2608.06909#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.06909#S2.SS2.p1.1),[§6\.1](https://arxiv.org/html/2608.06909#S6.SS1.p1.2)\.
- J\. Shah \(2026\)Causal agent replay: counterfactual attribution for LLM\-agent failures\.arXiv preprint arXiv:2606\.08275\.External Links:[Link](https://arxiv.org/abs/2606.08275)Cited by:[§2\.2](https://arxiv.org/html/2608.06909#S2.SS2.p1.1)\.
- J\. Wang, J\. Hou, F\. Wang, P\. Jian, C\. Bao, and Z\. Lv \(2026\)HINTBench: horizon\-agent intrinsic non\-attack trajectory benchmark\.External Links:2604\.13954,[Link](https://arxiv.org/abs/2604.13954)Cited by:[§1](https://arxiv.org/html/2608.06909#S1.p6.1)\.
- L\. Wang, C\. Ma, X\. Feng, Z\. Zhang, H\. Yang, J\. Zhang, Z\. Chen, J\. Tang, X\. Chen, Y\. Lin, W\. X\. Zhao, Z\. Wei, and J\. Wen \(2024\)A survey on large language model based autonomous agents\.Frontiers of Computer Science18\(6\)\.External Links:[Document](https://dx.doi.org/10.1007/s11704-024-40231-1),[Link](https://arxiv.org/abs/2308.11432)Cited by:[§1](https://arxiv.org/html/2608.06909#S1.p1.1)\.
- S\. Zhang, M\. Yin, J\. Zhang, J\. Liu, Z\. Han, J\. Zhang, B\. Li, C\. Wang, H\. Wang, Y\. Chen, and Q\. Wu \(2025\)Which agent causes task failures and when? on automated failure attribution of LLM multi\-agent systems\.InProceedings of the 42nd International Conference on Machine Learning,External Links:[Link](https://arxiv.org/abs/2505.00212)Cited by:[§1](https://arxiv.org/html/2608.06909#S1.p6.1),[§2\.2](https://arxiv.org/html/2608.06909#S2.SS2.p1.1)\.
- K\. Zhu, H\. Du, Z\. Hong, X\. Yang, S\. Guo, Z\. Wang, Z\. Wang, C\. Qian, X\. Tang, H\. Ji, and J\. You \(2025\)MultiAgentBench: evaluating the collaboration and competition of LLM agents\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),External Links:[Link](https://arxiv.org/abs/2503.01935)Cited by:[§2\.1](https://arxiv.org/html/2608.06909#S2.SS1.p1.1)\.