基于 LLM 的智能体系统中自动失败归因的机制研究

# A Mechanistic Study of Automated Failure Attribution in LLM-based Agentic Systems
Source: [https://arxiv.org/html/2610.28811](https://arxiv.org/html/2610.28811)
Liang WuAffiliation:NokiaYinhan HeAffiliation:University of VirginiaMayank DarbariAffiliation:NokiaJundong LiAffiliation:University of VirginiaLiangjie HongAffiliation:Nokia

###### Abstract

Agentic systems have demonstrated strong capabilities on complex tasks by enabling multiple interacting large language model \(LLM\)\-based agents to collaborate and execute subtasks in parallel\. However, this collaborative structure introduces new challenges for reliably evaluating and diagnosing system failures, particularly failure attribution: tracing a system’s execution to identify the agent step responsible for an incorrect final outcome\. Because manually performing this task is time\- and labor\-intensive, recent work has turned to LLMs as judges to automate the process, referred to as automated failure attribution \(AFA\)\. These approaches assume that a sufficiently capable model can infer the failing step directly from an interaction transcript\. In practice, however, direct prompting often performs poorly, which motivates follow\-up work that explores various ways to improve AFA performance, including alternative prompting strategies and inference\-time interventions\. In this paper, we examine a more fundamental question: whether this weak behavioral performance reflects a genuine capability limitation\. Using linear probes applied to internal activations, we uncover a significant*knows\-but\-won’t\-say gap:*models encode sufficient information to attribute a failure correctly but fail to explicitly output the responsible step\. Building on this finding, we propose*grafting*, a simple manual intervention that exploits knowledge already encoded in the model to improve AFA performance without requiring ground\-truth labels at inference time\. Extensive evaluations across multiple LLMs and benchmark settings demonstrate that grafting consistently outperforms direct\-prompting approaches, underscoring the potential of leveraging latent model knowledge for more reliable failure attribution\. Our code is available at[https://github\.com/wendyzheng22/mi4afa](https://github.com/wendyzheng22/mi4afa)\.

## 1Introduction

Agentic systems extend the capabilities of large language models \(LLMs\) by decomposing complex tasks and distributing them across multiple specialized agents \(e\.g\., planners, retrievers, coders, and verifiers\), that collaborate and execute subtasks in parallel\([Acharya et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib2);[Bandi et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib3);[Wu et al\., 2023](https://arxiv.org/html/2610.28811#bib.bib7)\)\. This collaborative paradigm enables agentic systems to tackle problems that would exceed the capacity of a single model working alone\. However, as the deployment of such systems expands into high\-stakes settings, reliable evaluation methods become essential\([Tran et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib4);[Gridach et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib5)\)\. One key evaluation task is failure attribution: identifying the earliest step in a system’s trajectory that caused the final output to fail\. Performing this task manually is expensive, since agentic systems can involve many interacting agents, and the space of possible interactions grows exponentially with system size\. This challenge motivates Automated Failure Attribution \(AFA\)\([Zhang et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib1)\), which employs an LLM as a judge to identify the failing step directly from a system’s interaction transcript\. However, as demonstrated by the Who&When benchmark\([Zhang et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib1)\), even large, capable LLMs perform poorly on this task\.

This weak performance has spurred a line of follow\-up work that seeks to improve AFA, such as refined prompting strategies\([West et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib6);[Sun et al\., 2026](https://arxiv.org/html/2610.28811#bib.bib9)\)or inference\-time interventions\([Ma et al\., 2026](https://arxiv.org/html/2610.28811#bib.bib8)\)\. These approaches share a common underlying assumption: the capability for failure attribution already exists within the model and simply needs to be elicited more effectively\. However, the persistently poor performance of direct prompting raises a more fundamental question: do LLMs actually possess the capability required for AFA? To answer this, we look beyond the model’s generated output and instead examine the information encoded in its internal representations using simple linear probes[Alain and Bengio \(2016\)](https://arxiv.org/html/2610.28811#bib.bib10)\. Surprisingly, these probes recover substantially more information about the correct failing step than the model’s generated answer\. We refer to this discrepancy as the*knows\-but\-won’t\-say gap*: LLMs encode considerably more AFA\-relevant information internally than they are able to express behaviorally\. This finding indicates that poor AFA performance does not necessarily reflect an absence of underlying capability; rather, the relevant information may already be present within the model but not effectively accessed during generation\.

This observation then raises a second question: if models already encode the information needed for AFA, can this latent capability be translated into better behavioral performance? To investigate this question, we propose*grafting*, a simple manual intervention that connects internal components encoding information about the failing step to the components involved in generating the model’s output\. Rather than teaching the model new failure\-attribution knowledge, grafting improves how existing information is routed during generation, enabling the model to more effectively express knowledge it already possesses\. In this way, grafting converts latent, decodable AFA information into improved behavioral predictions without requiring ground\-truth labels at inference time\.

We evaluate direct prompting, linear probing, and grafting across multiple open\-weight LLMs and benchmark settings, and find a consistent progression: direct prompting provides a weak behavioral baseline; linear probes reveal that substantially more information about the correct failing step is encoded in model activations; and grafting converts a significant portion of this latent information into improved behavioral predictions\. Together, these findings suggest that the limitations of AFA stem not solely from whether models contain the relevant information, but also from whether that information can be effectively accessed\. Our contributions are summarized as follows:

- •Knowledge–behavior gap\.We show that linear probes consistently outperform direct prompting across model–dataset combinations, revealing a substantial gap between the AFA\-relevant information encoded in model activations and the information expressed through behavioral predictions\. This provides evidence that LLMs possess considerably more AFA\-relevant capability than their direct responses reflect\.
- •Grafting\.We introduce*grafting*, a simple manual intervention that bridges components that encode information about the failing step to those involved in output generation\. Grafting consistently improves AFA performance over direct prompting without requiring ground\-truth labels at inference time\.

## 2Background and Preliminaries

In this section, we formally define the Automated Failure Attribution \(AFA\) task and establish the performance of direct prompting as a baseline\. These results motivate the central question of this work: whether the poor behavioral performance of LLMs on AFA reflects a genuine lack of capability, or merely a failure to express information already encoded within the model\.

### 2\.1Automated Failure Attribution

Consider an agentic system composed of one or more interacting agents that is given a task but produces an incorrect final answer\. LetH=\(h1,h2,…,hn\)H=\(h\_\{1\},h\_\{2\},\\ldots,h\_\{n\}\)denote the resulting interaction transcript, where eachhih\_\{i\}corresponds to the output of one agent at stepii\. Each transcript is annotated with the earliest stepe∈\{1,…,n\}e\\in\\\{1,\\ldots,n\\\}whose error ultimately causes the system to fail, which referred to as the*mistake step*\(equivalently, the*failure step*\)\. Given the transcriptHH, the goal of AFA is to predict the failing stepe^∈\{1,…,n\}\\hat\{e\}\\in\\\{1,\\ldots,n\\\}\. An AFA method succeeds whene^=e\\hat\{e\}=e, correctly identifying the earliest mistake responsible for the incorrect final output\.

Following[Zhang et al\. \(2025\)](https://arxiv.org/html/2610.28811#bib.bib1), we report two evaluation metrics throughout the paper: \(1\)Step accuracymeasures the probability of correctly identifying the failing step,P⁡\(e^=e\)P\(\\hat\{e\}=e\), and \(2\)Agent accuracymeasures whether the agent at the predicted mistake stepe^\\hat\{e\}matches the ground\-truth failing agent, regardless of which occurrence of that agent is selected\. Because a single agent may speak multiple times over the course of a transcript, a prediction can identify the wrong step while still attributing failure to the correct agent\. Agent accuracy is therefore a strictly coarser metric than step accuracy, and the two do not necessarily move together\.

### 2\.2Direct Prompting

We first evaluate direct prompting to characterize the behavioral baseline of LLMs on the AFA task, using the Who&When benchmark\([Zhang et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib1)\)\. This benchmark consists of two subsets: theAlgorithm\-generated \(Algo\.\)subset, containing relatively short execution traces produced automatically, and theHand\-crafted \(Craft\.\)subset, containing substantially longer and more complex agent trajectories designed by human annotators\.

Table[1](https://arxiv.org/html/2610.28811#S2.T1)summarizes the performance of several open\-weight and closed\-source LLMs under direct prompting\. The closed\-source LLMs are accessed via Cursor Python SDK\([Anysphere, 2026](https://arxiv.org/html/2610.28811#bib.bib24)\)\. Two observations are notable\. First, consistent with prior findings, every model achieves less desirable step accuracy on both subsets, confirming that reliably identifying the failing step remains difficult regardless of model family\. Second, performance does not scale consistently with model size or capability: within the Qwen family, the 1\.5B model underperforms the 0\.5B variant on step accuracy for Algo\., and the closed\-source models only modestly outperform substantially smaller open\-weight models on Craft\. This absence of a clear capacity trend suggests that increasing model size alone is insufficient to close the gap in behavioral performance on AFA, motivating the investigation, in the following sections, of what the model encodes internally rather than what direct prompting can elicit from it\.

Table 1:Direct Prompting Results on the Who&When benchmark

## 3Method

In this section, we introduce two methods to address the questions posed in Section[1](https://arxiv.org/html/2610.28811#S1): \(1\)*probing*, which determines whether LLMs internally encode sufficient information to identify the failing step, and \(2\)*grafting*, which investigates whether this latent information can be exposed to improve the model’s behavioral performance\.

### 3\.1Probing: Is the failing step decodable from model activations?

Poor performance under direct prompting does not necessarily mean that an LLM lacks the capability required for AFA\. The relevant information may instead be encoded in its hidden representations without being reflected in its output distribution\. To distinguish between these possibilities, we train linear probes to test whether the failing step can be linearly decoded from intermediate activations\. Our probing pipeline has three stages: activation extraction, probe training, and prediction\.

#### Activation extraction\.

For each transcript, we perform a single forward pass through the frozen LLM and extract hidden representations from every transformer layer at two sites: the post\-attention residual stream and the post\-MLP residual stream\. For a model withLLlayers, this gives2​L2Lcandidate representation sites\. Because AFA is a step\-level prediction problem, each step is represented by the hidden statexxat the final token of that step\.

#### Probe training\.

We evaluate each of the2​L2Lrepresentation sites independently\. For a fixed site, we pool the hidden statesxxfrom every step in every training transcript into a single dataset\. Each hidden state receives a binary label indicating whether its corresponding step is the ground\-truth failing step:

y=\{1,if the step is the ground\-truth failing step,0,otherwise\.y=\\begin\{cases\}1,&\\text\{if the step is the ground\-truth failing step\},\\\\ 0,&\\text\{otherwise\}\.\\end\{cases\}
We train a linear logistic probe

s⁡\(x\)=w⊤​x^\+b,s\(x\)=w^\{\\top\}\\hat\{x\}\+b,\(1\)where

x^=x−μσ,\\hat\{x\}=\\frac\{x\-\\mu\}\{\\sigma\},\(2\)andμ\\muandσ\\sigmaare computed only on the training split\. The probe minimizes

ℒ⁡\(w,b\)=𝔼⁡\[BCE⁡\(s⁡\(x^\),y,π\)\]\+λ​‖w‖22,\\mathcal\{L\}\(w,b\)=\\mathbb\{E\}\\left\[\\mathrm\{BCE\}\(s\(\\hat\{x\}\),y;\\pi\)\\right\]\+\\lambda\\\|w\\\|\_\{2\}^\{2\},where

π=N−N\+N\+\\pi=\\frac\{N\-N\_\{\+\}\}\{N\_\{\+\}\}is the positive\-class weight used to compensate for the severe class imbalance created by each transcript having exactly one labeled failure step\.

#### Prediction\.

At evaluation, the trained probe independently scores every step in a transcript using the corresponding hidden representation\. The predicted failing step is the highest\-scoring step:e^=arg⁡maxk⁡s⁡\(xk\)\\hat\{e\}=\\arg\\max\_\{k\}s\(x\_\{k\}\)\. Evaluating probes at every residual location allows us to identify where failure\-attribution information is most linearly accessible within the model\.

### 3\.2Grafting: Making the model say what it locates

Linear probing reveals what information is encoded in the model but leaves its behavior unchanged\. The goal now is to improve the model’s own predictions\. Our approach is motivated by a simple hypothesis about the functional organization of transformer attention heads: some heads specialize in locating the failing step \(locator heads\), while others specialize in writing the model’s predicted answer into the output distribution \(writer heads\)\. If these two groups are only weakly coupled, the model may internally identify the correct failing step without effectively communicating this information to the output\. Grafting aims to bridge this gap using a three step process: \(1\) identify locator heads, \(2\) identify writer heads, and \(3\) connect them through an inference\-time intervention\.

#### Step 1: Identify locator heads

We define locator heads as attention heads whose attention patterns reliably identify the failing step\. Intuitively, if a head consistently places most of its attention on the ground\-truth mistake step when producing the final answer, then that head provides a useful signal about where the failure occurred\. Thus, for each head\(ℓ,h\)\(\\ell,h\), we compute the attention distribution at the last token position:

z\(ℓ,h\)=softmax⁡\(qlast\(ℓ,h\)​K\(ℓ,h\)dh\),z^\{\(\\ell,h\)\}=\\mathrm\{softmax\}\\\!\\left\(\\frac\{q^\{\(\\ell,h\)\}\_\{\\text\{last\}\}K^\{\(\\ell,h\)\}\}\{\\sqrt\{d\_\{h\}\}\}\\right\),\(3\)
wheredhd\_\{h\}represents the head dimension\. Summing the attention mass over each step’s token span gives the total weight assigned to stepss:

α\(ℓ,h\)s=∑t=τtτt\+1−1zt\(l,h\),s=1,…,n,\\alpha^\{\(\\ell,h\)\}\_\{s\}\\;=\\;\\sum\_\{t=\\tau\_\{t\}\}^\{\\tau\_\{t\+1\}\-1\}z\_\{t\}^\{\(l,h\)\},\\qquad s=1,\\dots,n,\(4\)whereτs\\tau\_\{s\}indexes the token boundaries of stepss\. Each head casts a vote for the step receiving the largest attention mass,

arg⁡maxs⁡αs\(ℓ,h\)\.\\arg\\max\_\{s\}\\alpha^\{\(\\ell,h\)\}\_\{s\}\.
A head’s*locator score*is the proportion of training examples on which its vote matches the ground\-truth failing step\. We rank heads by this score and retain at most five heads from the top 10% of the ranking, excluding heads in the final transformer layer \(a final\-layer head has no downstream computation left to route its signal into, so it cannot itself act as a useful locator for the graft\)\.

#### Step 2: Identify writer heads

Whereas locator heads determine*which*step is important, writer heads determine*what*the model ultimately outputs\. We therefore identify these using direct logit attribution\([nostalgebraist, 2020](https://arxiv.org/html/2610.28811#bib.bib11)\), which measures a component’s direct contribution to a particular output logit\. Each attention and MLP layer’s output is added into the residual stream, so the final residual streamresid\(L\)\\text\{resid\}^\{\(L\)\}is a linear sum of every component’s contribution\. Projecting a component’s output through the model’s final normalization scaleggand the unembedding matrixWUW\_\{U\}gives that component’s direct, linear contribution to any vocabulary logitvv:

DLA\(ℓ,h\)​\(v\)=\(WO\(ℓ,h\)​zlast\(ℓ,h\)\)⊤​r​\(v,C\)rms⁡\(residlast\(L\)\)\.\\mathrm\{DLA\}^\{\(\\ell,h\)\}\(v\)\\;=\\;\\frac\{\\big\(W\_\{O\}^\{\(\\ell,h\)\}z^\{\(\\ell,h\)\}\_\{\\text\{last\}\}\\big\)^\{\\top\}r\(v,C\)\}\{\\mathrm\{rms\}\\big\(\\text\{resid\}^\{\(L\)\}\_\{\\text\{last\}\}\\big\)\}\.\(5\)wherezlast\(ℓ,h\)z^\{\(\\ell,h\)\}\_\{\\text\{last\}\}is the output of an attention head at the last token \(before the output projectionWOW\_\{O\}\), andr⁡\(v,C\)r\(v,C\)is a contrastive readout direction defined as

r⁡\(v,C\)=g⊙\(WU​\[v\]−1\|C\|​∑c∈CWU​\[c\]\),r\(v,C\)=g\\odot\\Big\(W\_\{U\}\[v\]\-\\tfrac\{1\}\{\|C\|\}\\textstyle\\sum\_\{c\\in C\}W\_\{U\}\[c\]\\Big\),\(6\)whereCCis the set of candidate step\-index tokens for the transcript\. Subtracting the mean unembedding vector over the candidate tokens removes directions shared across candidates and emphasizes the dimensions that distinguish the target token from the other possible step indices\. Writer heads are scored by their DLA toward the step index token that the model emits on the clean forward pass, instead of the ground truth mistake step\. We intentionally measure contribution toward the model’s own predicted answer because the objective is to identify the components currently responsible for expressing the model’s internal belief, irrespective of whether that belief is correct\. Writer heads are identified on the training set, keeping up to five heads from the top 10%, excluding heads in the first layer \(a first\-layer head has not yet had access to enough contextual computation to serve as a meaningful writer\)\.

#### Step 3: Construct the graft

Once the locator and writer heads have been identified, grafting combines them at inference time without any supervision or ground\-truth labels:

1. 1\.Read the locators\.Each selected locator head votes for its top\-attended step; we take the plurality winnert^\\hat\{t\}as the model’s internal guess\. We use a vote across heads rather than relying on a single head because individual heads are not perfectly reliable\.
2. 2\.Form the target direction\.Mapt^\\hat\{t\}to its index token and computer⁡\(t^\)r\(\\hat\{t\}\)by Equation[6](https://arxiv.org/html/2610.28811#S3.E6)\.
3. 3\.Pull it back into head space\.For each writer headii, we compute the direction in its value space that maximally increases the target logit after the head’s output projection:vi=WO\(i\)⊤​r​\(t^\)v\_\{i\}=W\_\{O\}^\{\(i\)\\top\}r\(\\hat\{t\}\)\.
4. 4\.Inject\.At the final token position, addviv\_\{i\}to the writer’s value vector beforeWOW\_\{O\}, scaled relative to the norm of the full concatenated head outputzlastz\_\{\\text\{last\}\}at that position: zi←zi\+β​‖zlast‖‖vi‖​vi\.z\_\{i\}\\;\\leftarrow\\;z\_\{i\}\\;\+\\;\\beta\\,\\frac\{\\\|z\_\{\\text\{last\}\}\\\|\}\{\\\|v\_\{i\}\\\|\}\\,v\_\{i\}\.\(7\)Here,β\\betacontrols the strength of the intervention, while the norm ratio scales the injected direction relative to the magnitude of the model’s existing head output\.

The intervention is applied in the writer heads’ value space rather than directly to the residual stream\. As a result, the injected signal passes through the same output projectionWOW\_\{O\}that the model normally uses to write information from the head into the residual stream\. Together, these steps create a bridge between the model’s internal localization and its output behavior\.

## 4Experiments

Table 2:Performance comparison between different prompting strategies, probing, and graftingIn this section, we first describe our experimental setup and then present results comparing the performance of probing and grafting to the behavioral prompting baselines\.

### 4\.1Experiment Setup

We evaluate seven open\-weight LLMs spanning three model families and multiple model sizes: Qwen 2\.5 0\.5B, 1\.5B, and 7B Instruct variants\([Yang et al\., 2024](https://arxiv.org/html/2610.28811#bib.bib31);[Team, 2024](https://arxiv.org/html/2610.28811#bib.bib30);[Qwen, 2024a](https://arxiv.org/html/2610.28811#bib.bib27);[Qwen, 2024b](https://arxiv.org/html/2610.28811#bib.bib28);[Qwen, 2024c](https://arxiv.org/html/2610.28811#bib.bib29)\); Llama 3\.2 1B and 3B variants\([Meta, 2024a](https://arxiv.org/html/2610.28811#bib.bib25);[Meta, 2024b](https://arxiv.org/html/2610.28811#bib.bib26)\); and Gemma 3 1B and 4B variants\([Deepmind, 2025a](https://arxiv.org/html/2610.28811#bib.bib32);[Deepmind, 2025b](https://arxiv.org/html/2610.28811#bib.bib33);[Team et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib34)\)\. This selection allows us to examine AFA performance across both model scale and model family\.

Evaluation Tasks\.We adopt the Who&When benchmark\([Zhang et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib1)\), which contains two subsets with different trajectory lengths\. The Algorithm Generation \(Algo\.\) subset contains transcripts with an average of 9 steps, while the Hand Crafted \(Craft\.\) subset contains substantially longer transcripts with an average of 29 steps\. This variation allows us to evaluate AFA across agentic trajectories of different complexity\.

Baselines\.We compare against three prompting\-based baselines\. All\-at\-Once provides the entire transcript to the model at once, and Step\-by\-Step incrementally presents each step to the model to determine whether it constitutes the failure; both strategies were introduced by[Zhang et al\. \(2025\)](https://arxiv.org/html/2610.28811#bib.bib1)\. Sequential decomposes attribution into two stages, first identifying the responsible agent and subsequently predicting the failing step conditioned on that agent\.

Computation Resources\.All experiments were performed on a cluster of four A100 80GB PCIe GPUs running Cuda version 13\.0\.

### 4\.2Failure\-Attribution Information is Linearly Decodable

Table[2](https://arxiv.org/html/2610.28811#S4.T2)compares the performance of all five methods\. We first examine whether the poor performance of direct prompting reflects a genuine absence of failing\-step information within the model\. Across the evaluated models, linear probes substantially outperform the prompting baselines on step accuracy\. For Qwen 0\.5B on Algo\., for example, the strongest prompting baseline reaches 23\.81% step accuracy, compared with 36\.51% for probing\. For Qwen 1\.5B and Qwen 7B, probing reaches 44\.44% and 47\.62%, respectively, compared with 15\.87% and 14\.29% for the strongest prompting baseline\. Comparable gains are observed for Llama and Gemma\.

These results indicate that failing\-step information is substantially more accessible from the model’s internal representations than from its generated answers\. A simple linear classifier can recover information about the correct step that the model’s own output does not reliably express\. We refer to this discrepancy between internally encoded latent information and behavioral performance as the*knows\-but\-won’t\-say*gap\.

At the same time, the results reveal substantial variation in the behavioral baselines across the different configurations\. No prompting strategy consistently performs best, and increasing model size does not uniformly translate into better failure attribution\. This variability further suggests that the primary challenge may not simply be acquiring failure\-attribution information, but effectively accessing and expressing information that is already encoded in the model\.

### 4\.3Grafting Exploits Latent Information for Improved Performance

The probing results establish that LLMs encode information about the failing step, but probing itself does not provide a mechanism for using this information during generation\. This motivates our next approach, grafting, which aims to leverage the model’s own output mechanism to express its predicted failing step\. As shown in Table[2](https://arxiv.org/html/2610.28811#S4.T2), grafting consistently outperforms the prompting baselines and substantially narrows the gap between behavioral prediction and the information revealed by probing\. For example, on the Algo\. subset, grafting achieves the highest step accuracy for Qwen 0\.5B \(38\.10%\), while also substantially outperforming all prompting methods for Qwen 1\.5B \(33\.33% vs\. 15\.87% for Step\-by\-Step\) and Qwen 7B \(30\.16% vs\. 14\.29% for Sequential\)\. Similar improvements are observed for the Llama and Gemma models\. On the Craft\. subset, grafting also improves step accuracy over prompting for several models, including Qwen 0\.5B \(28\.57%\), Gemma 1B \(21\.43%\), and Llama 3B \(21\.43%\)\.

Importantly, grafting does not simply reproduce the performance of the linear probe: probing remains stronger in many settings\. Rather, the results show that a substantial portion of the information that is linearly decodable from the model can be converted into improved behavioral predictions through a targeted intervention on the model’s internal computation\. This provides evidence that the knowledge–behavior gap is not merely an observation about representational capacity, but can be actively exploited to improve AFA\.

## 5Related Works

#### Automated Failure Attribution

Automated Failure Attribution \(AFA\) aims to identify the agent and step responsible for a task failure in an LLM\-based multi\-agent system\([Zhang et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib1)\)\. The Who&When benchmark demonstrates a substantial gap between agent\-level and step\-level attribution\([Zhang et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib1)\)\. Thus, subsequent work aims to improve AFA performance through diverse strategies, including decomposing attribution into scope delineation and localization\([Sun et al\., 2026](https://arxiv.org/html/2610.28811#bib.bib9)\), causal counterfactual reasoning\([West et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib6)\), and intervention\-based verification of failure hypotheses\([Ma et al\., 2026](https://arxiv.org/html/2610.28811#bib.bib8)\)\. Other recent approaches further investigate automated localization and hierarchical attribution\([Zhang et al\., 2026](https://arxiv.org/html/2610.28811#bib.bib15);[Banerjee et al\., 2025](https://arxiv.org/html/2610.28811#bib.bib16)\)\. More recently, TraceElephant evaluates attribution under full execution observability, showing that access to complete execution traces can substantially improve attribution accuracy\([Chen et al\., 2026](https://arxiv.org/html/2610.28811#bib.bib17)\)\. These approaches primarily seek to improve an evaluator’s ability to produce correct attribution judgments from trajectory information\. We instead ask whether AFA\-relevant information is already encoded in the evaluator’s internal representations and whether it can be routed into generation\.

#### Latent Knowledge and Probing

Prior work has shown that language models can encode information that is not reliably expressed in their outputs\. Probing classifiers provide a standard method for testing whether properties are linearly recoverable from intermediate representations\([Alain and Bengio, 2016](https://arxiv.org/html/2610.28811#bib.bib10);[Belinkov, 2022](https://arxiv.org/html/2610.28811#bib.bib18)\)\. In particular, latent knowledge can be recovered from activations even when models are prompted to produce incorrect answers\([Burns et al\., 2022](https://arxiv.org/html/2610.28811#bib.bib19)\), and hidden states encode signals related to truthfulness\([Azaria and Mitchell, 2023](https://arxiv.org/html/2610.28811#bib.bib20)\)with approximately linear structure\([Marks and Tegmark, 2023](https://arxiv.org/html/2610.28811#bib.bib21)\)\. These results motivate the distinction between information that is decodable from representations and information that is actually expressed or used during generation\. We apply this distinction to AFA, using linear probes to quantify attribution information encoded in intermediate representations and testing whether this information can be made behaviorally accessible during generation\.

#### Activation Intervention and Representation Engineering

A complementary literature studies interventions on internal representations to with the goal of altering model behavior\. Activation engineering and representation engineering manipulate the model’s intermediate representations to steer its outputs\([Turner et al\., 2023](https://arxiv.org/html/2610.28811#bib.bib14);[Zou et al\., 2023](https://arxiv.org/html/2610.28811#bib.bib22)\), while inference\-time intervention uses directions identified from activation\-space classifiers to steer models toward more truthful outputs\([Li et al\., 2023](https://arxiv.org/html/2610.28811#bib.bib23)\)\. Activation patching and causal tracing provide complementary methods for localizing model components involved in specific computations through causal interventions\([Meng et al\., 2022](https://arxiv.org/html/2610.28811#bib.bib12);[Zhang and Nanda, 2024](https://arxiv.org/html/2610.28811#bib.bib13)\)\. Grafting builds on these ideas but targets a different problem: rather than steering toward a predefined behavioral attribute, it connects representations from which a task\-specific variable \(i\.e\., the failing step\)is decodable to components involved in output generation\. Thus, grafting tests whether information already decodable from a model’s representations can be translated into improved task behavior\.

## 6Conclusion

In this paper, we study Automated Failure Attribution \(AFA\) from the perspective of the gap between information encoded in LLM representations and information expressed through their behavior\. We show that linear probes consistently recover substantially more information about the failing step than direct prompting, revealing a*knows\-but\-won’t\-say gap*in LLM\-based failure attribution\. Motivated by this observation, we propose*grafting*, a simple inference\-time intervention that connects components encoding failing\-step information to those responsible for output generation, allowing the model to more effectively express its internally encoded knowledge\. Experiments across multiple LLMs and benchmark settings demonstrate that grafting improves behavioral AFA performance over direct prompting, showing that latent failure\-attribution information can be translated into improved predictions without requiring ground\-truth labels at inference time\. More broadly, these findings suggest that improving LLM\-based evaluators may depend not only on scaling models or refining prompts, but on developing methods that allow models to more effectively access and express knowledge they already possess\.

## References

- Acharyaet al\.\(2025\)D\. B\. Acharya, K\. Kuppan, and B\. DivyaAgentic ai: autonomous intelligence for complex goals—a comprehensive survey\.IEEe Access13,pp\. 18912–18936\.Cited by:[§1](https://arxiv.org/html/2610.28811#S1.p1.1)\.
- Alain and Bengio \(2016\)G\. Alain and Y\. BengioUnderstanding intermediate layers using linear classifier probes\.arXiv preprint arXiv:1610\.01644\.Cited by:[§1](https://arxiv.org/html/2610.28811#S1.p2.1),[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px2.p1.1)\.
- Anysphere \(2026\)Inc\. AnysphereCursor\-sdk: the official python sdk for cursor agent workflows\.PyPI\.External Links:[Link](https://pypi.org/project/cursor-sdk/)Cited by:[§2\.2](https://arxiv.org/html/2610.28811#S2.SS2.p2.1)\.
- Azaria and Mitchell \(2023\)A\. Azaria and T\. MitchellThe internal state of an llm knows when it’s lying\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 967–976\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px2.p1.1)\.
- Bandiet al\.\(2025\)A\. Bandi, B\. Kongari, R\. Naguru, S\. Pasnoor, and S\. V\. VilipalaThe rise of agentic ai: a review of definitions, frameworks, architectures, applications, evaluation metrics, and challenges\.Future Internet17\(9\),pp\. 404\.Cited by:[§1](https://arxiv.org/html/2610.28811#S1.p1.1)\.
- Banerjeeet al\.\(2025\)A\. Banerjee, A\. Nair, and T\. BorogovacWhere did it all go wrong? a hierarchical look into multi\-agent error attribution\.arXiv preprint arXiv:2510\.04886\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px1.p1.1)\.
- Belinkov \(2022\)Y\. BelinkovProbing classifiers: promises, shortcomings, and advances\.Computational Linguistics48\(1\),pp\. 207–219\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px2.p1.1)\.
- Burnset al\.\(2022\)C\. Burns, H\. Ye, D\. Klein, and J\. SteinhardtDiscovering latent knowledge in language models without supervision\.arXiv preprint arXiv:2212\.03827\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2026\)M\. Chen, J\. Wang, F\. Mu, Y\. Wang, Z\. Liu, H\. Feng, and Q\. WangSeeing the whole elephant: a benchmark for failure attribution in llm\-based multi\-agent systems\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 19888–19905\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px1.p1.1)\.
- Deepmind \(2025a\)G\. DeepmindGemma\-3\-1b\-it\.Hugging Face\.External Links:[Link](https://huggingface.co/google/gemma-3-1b-it)Cited by:[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p1.1)\.
- Deepmind \(2025b\)G\. DeepmindGemma\-3\-4b\-it\.Hugging Face\.External Links:[Link](https://huggingface.co/google/gemma-3-4b-it)Cited by:[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p1.1)\.
- Gridachet al\.\(2025\)M\. Gridach, J\. Nanavati, K\. Z\. E\. Abidine, L\. Mendes, and C\. MackAgentic ai for scientific discovery: a survey of progress, challenges, and future directions\.arXiv preprint arXiv:2503\.08979\.Cited by:[§1](https://arxiv.org/html/2610.28811#S1.p1.1)\.
- Liet al\.\(2023\)K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. WattenbergInference\-time intervention: eliciting truthful answers from a language model\.Advances in neural information processing systems36,pp\. 41451–41530\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px3.p1.1)\.
- Maet al\.\(2026\)M\. Ma, J\. Zhang, F\. Yang, Y\. Kang, Q\. Lin, S\. Rajmohan, and D\. ZhangDover: intervention\-driven auto debugging for llm multi\-agent systems\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 28059–28087\.Cited by:[§1](https://arxiv.org/html/2610.28811#S1.p2.1),[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px1.p1.1)\.
- Marks and Tegmark \(2023\)S\. Marks and M\. TegmarkThe geometry of truth: emergent linear structure in large language model representations of true/false datasets\.arXiv preprint arXiv:2310\.06824\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px2.p1.1)\.
- Menget al\.\(2022\)K\. Meng, D\. Bau, A\. Andonian, and Y\. BelinkovLocating and editing factual associations in gpt\.Advances in neural information processing systems35,pp\. 17359–17372\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px3.p1.1)\.
- Meta \(2024a\)MetaLlama\-3\.2\-1b\.Hugging Face\.External Links:[Link](https://huggingface.co/meta-llama/Llama-3.2-1B)Cited by:[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p1.1)\.
- Meta \(2024b\)MetaLlama\-3\.2\-3b\.Hugging Face\.External Links:[Link](https://huggingface.co/meta-llama/Llama-3.2-3B)Cited by:[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p1.1)\.
- nostalgebraist \(2020\)nostalgebraistInterpreting gpt: the logit lens\.LessWrong\.External Links:[Link](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens)Cited by:[§3\.2](https://arxiv.org/html/2610.28811#S3.SS2.SSS0.Px2.p1.1)\.
- Qwen \(2024a\)QwenQwen2\.5\-0\.5b\-instruct\.Hugging Face\.External Links:[Link](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct)Cited by:[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p1.1)\.
- Qwen \(2024b\)QwenQwen2\.5\-1\.5b\-instruct\.Hugging Face\.External Links:[Link](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)Cited by:[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p1.1)\.
- Qwen \(2024c\)QwenQwen2\.5\-7b\-instruct\.Hugging Face\.External Links:[Link](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)Cited by:[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p1.1)\.
- Sunet al\.\(2026\)K\. Sun, W\. Li, B\. Dong, Y\. Lin, J\. Zhang, and B\. ShiScope delineation before localization: a two\-stage framework for enhancing failure attribution in multi\-agent systems\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 33108–33116\.Cited by:[§1](https://arxiv.org/html/2610.28811#S1.p2.1),[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px1.p1.1)\.
- Teamet al\.\(2025\)G\. Team, A\. Kamath, J\. Ferret, S\. Pathak, N\. Vieillard, R\. Merhej, S\. Perrin, T\. Matejovicova, A\. Ramé, M\. Rivière,et al\.Gemma 3 technical report\.arXiv preprint arXiv:2503\.19786\.Cited by:[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p1.1)\.
- Team \(2024\)Q\. TeamQwen2\.5: a party of foundation models\.External Links:[Link](https://qwenlm.github.io/blog/qwen2.5/)Cited by:[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p1.1)\.
- Tranet al\.\(2025\)K\. Tran, D\. Dao, M\. Nguyen, Q\. Pham, B\. O’Sullivan, and H\. D\. NguyenMulti\-agent collaboration mechanisms: a survey of llms\.arXiv preprint arXiv:2501\.06322\.Cited by:[§1](https://arxiv.org/html/2610.28811#S1.p1.1)\.
- Turneret al\.\(2023\)A\. M\. Turner, L\. Thiergart, G\. Leech, D\. Udell, J\. J\. Vazquez, U\. Mini, and M\. MacDiarmidSteering language models with activation engineering\.arXiv preprint arXiv:2308\.10248\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px3.p1.1)\.
- Westet al\.\(2025\)A\. West, Y\. Weng, M\. Zhu, Z\. Lin, Z\. Ning, and Y\. ZhangAbduct, act, predict: scaffolding causal inference for automated failure attribution in multi\-agent systems\.arXiv preprint arXiv:2509\.10401\.Cited by:[§1](https://arxiv.org/html/2610.28811#S1.p2.1),[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px1.p1.1)\.
- Wuet al\.\(2023\)Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu,et al\.Autogen: enabling next\-gen llm applications via multi\-agent conversation\.arXiv preprint arXiv:2308\.08155\.Cited by:[§1](https://arxiv.org/html/2610.28811#S1.p1.1)\.
- Yanget al\.\(2024\)A\. Yang, B\. Yang, B\. Hui, B\. Zheng, B\. Yu, C\. Zhou, C\. Li, C\. Li, D\. Liu, F\. Huang, G\. Dong, H\. Wei, H\. Lin, J\. Tang, J\. Wang, J\. Yang, J\. Tu, J\. Zhang, J\. Ma, J\. Xu, J\. Zhou, J\. Bai, J\. He, J\. Lin, K\. Dang, K\. Lu, K\. Chen, K\. Yang, M\. Li, M\. Xue, N\. Ni, P\. Zhang, P\. Wang, R\. Peng, R\. Men, R\. Gao, R\. Lin, S\. Wang, S\. Bai, S\. Tan, T\. Zhu, T\. Li, T\. Liu, W\. Ge, X\. Deng, X\. Zhou, X\. Ren, X\. Zhang, X\. Wei, X\. Ren, Y\. Fan, Y\. Yao, Y\. Zhang, Y\. Wan, Y\. Chu, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. FanQwen2 technical report\.arXiv preprint arXiv:2407\.10671\.Cited by:[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p1.1)\.
- Zhang and Nanda \(2024\)F\. Zhang and N\. NandaTowards best practices of activation patching in language models: metrics and methods\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 1651–1678\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px3.p1.1)\.
- Zhanget al\.\(2026\)G\. Zhang, J\. Wang, J\. Chen, W\. Zhou, K\. Wang, and S\. YanAgentracer: who is inducing failure in the llm agentic systems?\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 11377–11399\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2025\)S\. Zhang, M\. Yin, J\. Zhang, J\. Liu, Z\. Han, J\. Zhang, B\. Li, C\. Wang, H\. Wang, Y\. Chen,et al\.Which agent causes task failures and when? on automated failure attribution of llm multi\-agent systems\.arXiv preprint arXiv:2505\.00212\.Cited by:[§1](https://arxiv.org/html/2610.28811#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.28811#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2610.28811#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p2.1),[§4\.1](https://arxiv.org/html/2610.28811#S4.SS1.p3.1),[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px1.p1.1)\.
- Zouet al\.\(2023\)A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski,et al\.Representation engineering: a top\-down approach to ai transparency\.arXiv preprint arXiv:2310\.01405\.Cited by:[§5](https://arxiv.org/html/2610.28811#S5.SS0.SSS0.Px3.p1.1)\.