以语言障碍为视角审视大语言模型

# Language Impairment as a Lens onto Large Language Models
Source: [https://arxiv.org/html/2610.28992](https://arxiv.org/html/2610.28992)
Wang Bill ZhuYue ChenElsi KaiserAffiliation:University of Southern CaliforniaWeizhi DuAffiliation:Independent ResearcherYao DuAffiliation:University of Southern CaliforniaSamuel NastaseAffiliation:University of Southern CaliforniaYanjun GaoAffiliation:University of Colorado Anschutz Medical Campus†Corresponding authors:\{muxuanhe, ychen249\}@usc\.edu,wangzhu@berkeley\.edu[Project](https://llm-aphasia.github.io/)![[Uncaptioned image]](https://arxiv.org/html/2610.28992v1/figures/github-mark.png)[Code](https://github.com/Bill1235813/llm_aphasia)Affiliation:University of California, Berkeley

###### Abstract

Although large language models \(LLMs\) encode language in highly distributed spaces, they remain vulnerable to focal disruptions—a phenomenon strikingly similar to how localized brain damage causes clinical aphasia in humans\. To investigate this theoretical tension, we ask whether LLMs possess a functional structure analogous to the human language system, by introducing a novel mechanistic interpretability framework inspired by aphasia\. Through two complementary approaches, classifier\-guided lesioning and continuous activation steering, we systematically induce aphasia\-like breakdowns in LLMs\. We find that language functions can be successfully isolated to discrete structural circuits and continuous representational vectors, yet these functions remain densely entangled with significant circuit overlap and vector similarities across distinct aphasic profiles\. Furthermore, while steering generalizes robustly across downstream tasks, structural lesioning remains asymmetrically constrained\. By bridging neuropsychology and mechanistic interpretability, we demonstrate that LLMs, functionally analogous to biological brains, rely on a densely entangled architecture that can be systematically mapped through clinically\-informed interventions\.

## 1Introduction

Large language models \(LLMs\) have achieved strikingly human\-like language behavior, yet their internal functional organization remains fundamentally opaque\. Unlike traditional linguistic theories that compartmentalize language into distinct symbolic modules\([Chomsky, 1957](https://arxiv.org/html/2610.28992#bib.bib7)\), LLMs encode diverse linguistic structures into a unified, high\-dimensional representation space\([Almeida et al\., 2026](https://arxiv.org/html/2610.28992#bib.bib68);[Baroni, 2019](https://arxiv.org/html/2610.28992#bib.bib67);[Tenney et al\., 2019](https://arxiv.org/html/2610.28992#bib.bib69)\)\. This distributed architecture, which parallels neural population coding in biological brains\([Nastase et al\., 2026](https://arxiv.org/html/2610.28992#bib.bib31);[Rumelhart et al\., 1986](https://arxiv.org/html/2610.28992#bib.bib42)\), grants LLMs significant global redundancy\. However, recent mechanistic interpretability research reveals a paradox: despite this distributed redundancy, models remain vulnerable to catastrophic, highly specific functional collapse when focal critical circuits are disrupted\([He et al\., 2026](https://arxiv.org/html/2610.28992#bib.bib15);[Qin et al\., 2026](https://arxiv.org/html/2610.28992#bib.bib34);[Lu et al\., 2026](https://arxiv.org/html/2610.28992#bib.bib25)\)\. This creates a theoretical tension: how can an LLM be so globally distributed, yet remain vulnerable to focal damage that produces specific linguistic breakdowns?

![Refer to caption](https://arxiv.org/html/2610.28992v1/fig_teaser.png)Figure 1:Left: knocking out attention heads inside Llama\-2\-7B\-chat makes its Cinderella story tellingfragmented and incoherent, as in Broca’s aphasia;Right: adding a direction specific to Wernicke’s aphasia to the activations of the Qwen3\.5\-9B makes its storyfluent but irrelevant, as in Wernicke’s aphasia\.To investigate this tension, we ask whether the internal architecture of LLMs possesses a functional structure analogous to the human language system\([Teehan et al\., 2022](https://arxiv.org/html/2610.28992#bib.bib46);[Fedorenko et al\., 2024](https://arxiv.org/html/2610.28992#bib.bib13);[Kuribayashi et al\., 2025](https://arxiv.org/html/2610.28992#bib.bib21);[Beckmann and Queloz, 2026](https://arxiv.org/html/2610.28992#bib.bib3)\)\. In clinical neuropsychology, the brain’s functional organization is historically mapped by observing how language breaks down following focal injury \(e\.g\., through the study of aphasia, an acquired language disorder due to brain injury or disease\)[Dronkers et al\. \(2017\)](https://arxiv.org/html/2610.28992#bib.bib71);[Geschwind \(1970\)](https://arxiv.org/html/2610.28992#bib.bib70)\. Standard NLP metrics, such as perplexity or downstream accuracy, only indicatethata model is degrading\([Qin et al\., 2026](https://arxiv.org/html/2610.28992#bib.bib34)\); they cannot easily characterize the nature of the deficit\. Aphasia, however, provides a standardized, granular taxonomy for linking focal damage to highly specific language deficits\([Hoffmann and Chen, 2013](https://arxiv.org/html/2610.28992#bib.bib18)\)\. By adopting this clinical diagnostic lens, we gain a practical framework to categorize \(i\)howan LLM’s language generation breaks down when perturbed, and \(ii\)howan intact LLM’s activation patterns change when processing clinically\-informed impaired inputs\.

Our framework thus consists of two complementary methodologies: lesion and steering\.

- •In thelesionapproach, we developed a classifier trained on clinical diagnostic data that classifies model behavior according to aphasia subtypes \(e\.g\.,Broca, Wernicke; see Appendix[A](https://arxiv.org/html/2610.28992#A1)\)\. By systematically ablating model components, we map specific structural interventions to their corresponding aphasia\-like output categories\. This allows us to test whether certain linguistic functions can be linked to specific components within LLMs\.
- •In thesteeringapproach, we derive intervention vectors by calculating the difference in the model’s residual stream activations when processing human aphasic transcripts versus healthy controls\. By systematically adding these steering vectors to targeted layers during text generation, we shift the model’s behavior toward specific aphasia profiles \(e\.g\., Broca’s or Wernicke’s\)\. This allows us to test whether complex language deficits can be induced through continuous activation interventions, and to explore whether subtracting these directions can effectively reverse or “heal” the linguistic effects of a simulated lesion\.

We employ this dual approach to robustly cross\-validate our findings\. While ablating targets the model’s structural “hardware” \(discrete circuits\), steering intervenes on its representational “software” \(continuous state\)\. Proving that aphasic behavior can be induced both structurally and representationally ensures our results reflect fundamental properties of the model, rather than artifacts of a single methodology\.

Applying this framework, our main findings are:

- •Dual existence of functional mechanisms:Classifier\-guided lesion successfully isolates discrete circuits governing distinct language functions \(e\.g\., grammatical fluency, semantic coherence\), while steering identifies corresponding continuous vectors\. Crucially, targeted subtype lesions succeed far more often when guided by our classifier\.
- •Dense functional entanglement:Distinct aphasic profiles are highly entangled\. Subtypes share significant structural overlap in their lesioned circuits and exhibit high cosine similarity across their steering vectors\.
- •Asymmetric generalizability:While structural lesion effects are constrained by the classifier’s specific training distribution, representational steering generalizes robustly across diverse downstream tasks\.

Our study has the following contributions\. We \(1\) introduce a novel interdisciplinary framework that adapts clinical aphasia to map LLM failure modes, \(2\) provide empirical evidence that macroscopic linguistic breakdowns correspond to both specialized circuits and continuous vectors, and \(3\) demonstrate that LLM language generation, in a way functionally analogous the human brain, relies on a densely entangled, distributed architecture vulnerable to highly specific disruptions\.

## 2Related Works

#### Aphasia and LLMs\.

Recent research has begun utilizing clinical language deficits to map model degradations[Wang et al\. \(2025a\)](https://arxiv.org/html/2610.28992#bib.bib55);[Wang et al\. \(2025b\)](https://arxiv.org/html/2610.28992#bib.bib54);[Yang et al\. \(2026\)](https://arxiv.org/html/2610.28992#bib.bib59);[Roll et al\. \(2026a\)](https://arxiv.org/html/2610.28992#bib.bib39), though interpretations of the parallels between LLMs and humans remain contested\.[Yang et al\. \(2026\)](https://arxiv.org/html/2610.28992#bib.bib59)demonstrated that systematically lesioning the decoder layers of multimodal models can reproduce the complete, joint error profiles of individual persons with aphasia \(PWAs\) during a picture\-naming task\. They observed a distinct depth gradient: perturbing early layers caused complete response failure, middle layers triggered semantic errors, and late layers yielded neologisms\. Conversely,[Roll et al\. \(2026a\)](https://arxiv.org/html/2610.28992#bib.bib39)evaluated damaged language models using a Text Aphasia Battery \(TAB;[Roll et al\. \(2026b\)](https://arxiv.org/html/2610.28992#bib.bib40)\) and concluded that the resulting error distributions do not genuinely resemble human clinical aphasia, arguing that LLMs and the human brain are fundamentally organized in different ways\.

Our research bridges these perspectives\. While[Yang et al\. \(2026\)](https://arxiv.org/html/2610.28992#bib.bib59)focused on single\-word vision\-language tasks to replicate individual patient profiles, we focus on continuous text generation to localize specific internal functional networks\. Furthermore, our theoretical stance aligns with the cautions raised by[Roll et al\. \(2026a\)](https://arxiv.org/html/2610.28992#bib.bib39): we deploy aphasic subtypes strictly as a diagnostic framework to categorize model vulnerability and internal overlapping components, without assuming a 1:1 modularity with biological brain regions like Broca’s or Wernicke’s area[Broca \(1861\)](https://arxiv.org/html/2610.28992#bib.bib6);[Wernicke \(1874\)](https://arxiv.org/html/2610.28992#bib.bib56)\. Ultimately, by combining lesioning to test focal vulnerabilities and steering to map the intact distributed geometry, we investigate whether a highly distributed LLM can genuinely exhibit focal, aphasia\-like functional structures\.

#### Lesions and causal analysis\.

A high degree of redundancy in LLMs protects them from drastic performance degradation\([Frantar and Alistarh, 2023](https://arxiv.org/html/2610.28992#bib.bib14);[Sun et al\., 2024](https://arxiv.org/html/2610.28992#bib.bib45);[Wang et al\., 2023b](https://arxiv.org/html/2610.28992#bib.bib51);[Shi et al\., 2023](https://arxiv.org/html/2610.28992#bib.bib43)\)\. Most LLMs’ attention heads can be removed with little loss\([Michel et al\., 2019](https://arxiv.org/html/2610.28992#bib.bib30);[Voita et al\., 2019](https://arxiv.org/html/2610.28992#bib.bib50)\), and many attention layers can be dropped from large models\([He et al\., 2026](https://arxiv.org/html/2610.28992#bib.bib15)\), yet a handful of critical neurons can collapse a model into repetitive noise\([Qin et al\., 2026](https://arxiv.org/html/2610.28992#bib.bib34);[Lu et al\., 2026](https://arxiv.org/html/2610.28992#bib.bib25)\)\. Circuit analyses localize a behavior to a set of components with lesioning and activation patching\([Vig et al\., 2020](https://arxiv.org/html/2610.28992#bib.bib49);[Wang et al\., 2023a](https://arxiv.org/html/2610.28992#bib.bib53);[Zhang and Nanda, 2024](https://arxiv.org/html/2610.28992#bib.bib62)\), and learned masks find such sets by gradient descent\([Csordás et al\., 2021](https://arxiv.org/html/2610.28992#bib.bib9);[De Cao et al\., 2020](https://arxiv.org/html/2610.28992#bib.bib11)\)\. These methods track a loss of accuracy or a target token’s probability, whereas we search for lesions that produce a specified clinical behavior in free narration\.

#### Steering\.

Unlike leisoning, steering adds a direction to a model’s activations while it generates, which shifts its behavior without changing its weights\([Subramani et al\., 2022](https://arxiv.org/html/2610.28992#bib.bib44);[Turner et al\., 2023](https://arxiv.org/html/2610.28992#bib.bib48);[Zhang et al\., 2026a](https://arxiv.org/html/2610.28992#bib.bib65)\)\. Linear probes show that many properties of the input can be decoded from hidden states\([Alain and Bengio, 2016](https://arxiv.org/html/2610.28992#bib.bib1);[Hewitt and Manning, 2019](https://arxiv.org/html/2610.28992#bib.bib66);[Belinkov, 2022](https://arxiv.org/html/2610.28992#bib.bib4)\), and concepts such as the truth of a statement are often encoded along a single direction of a model’s representations\([Park et al\., 2024](https://arxiv.org/html/2610.28992#bib.bib33);[Marks and Tegmark, 2024](https://arxiv.org/html/2610.28992#bib.bib28)\)\. The direction is often the difference between the mean activations on two contrasting sets of inputs\([Zou et al\., 2023](https://arxiv.org/html/2610.28992#bib.bib64);[Rimsky et al\., 2024](https://arxiv.org/html/2610.28992#bib.bib38);[Li et al\., 2023](https://arxiv.org/html/2610.28992#bib.bib23)\), and removing such a direction from every layer can stop a behavior\([Arditi et al\., 2024](https://arxiv.org/html/2610.28992#bib.bib2)\)\. These studies construct the contrasting inputs themselves, for example harmful and harmless instructions, and so does the closest work to ours, which steers models to simulate cognitively impaired patients for clinical training\([Zhang et al\., 2026b](https://arxiv.org/html/2610.28992#bib.bib63)\)\. We take the contrasting inputs from clinical data, the transcripts of speakers with and without aphasia, which the model reads as its own answers, and ask experts whether the steered speech matches the targeted type of aphasia\.

Table 1:Dataset statistics for the two AphasiaBank tasks\.

## 3Data and Setup

#### Data\.

AphasiaBank\([MacWhinney et al\., 2011](https://arxiv.org/html/2610.28992#bib.bib26)\)is a large\-scale database of naturalistic speech from individuals with aphasia and neurologically healthy controls\. We focus on two tasks that elicit different types of speech:*Cinderella*task, in which speakers retell the story of Cinderella to produce narrative speech, and*sandwich*task, in which speakers describe how to make a peanut butter and jelly sandwich, eliciting procedural speech with a fixed sequence of steps \(see Table[1](https://arxiv.org/html/2610.28992#S2.T1)for statistics\)\.

#### Models\.

We study 11 models from four families: GPT\-2 XL\([Radford et al\., 2019](https://arxiv.org/html/2610.28992#bib.bib36)\); Llama\-2\-7B\-chat and Llama\-2\-13B\-chat\([Touvron et al\., 2023](https://arxiv.org/html/2610.28992#bib.bib47)\); Pythia\-1B, 1\.4B, 2\.8B and 6\.9B\([Biderman et al\., 2023](https://arxiv.org/html/2610.28992#bib.bib5)\); and Qwen3\.5\-0\.8B, 2B, 4B and 9B\([Qwen Team, 2026](https://arxiv.org/html/2610.28992#bib.bib35)\), with thinking disabled\. We use the AphasiaBank prompts, “Tell me as much of the story of Cinderella as you can” and “Tell me how you would make a peanut butter and jelly sandwich”, to enable comparison with human speech elicited by the same tasks\. We pass these prompts directly to GPT\-2 XL and Pythia and format them using the respective chat templates for Llama\-2 and Qwen3\.5 \(Table[6](https://arxiv.org/html/2610.28992#A2.T6)\)\. All models generate at most 180 new tokens using greedy decoding with a repetition penalty of 1\.25, as unpenalized greedy decoding causes the base models to loop even without intervention\. Across both interventions and tasks, we include only responses with at least 20 characters\.

![Refer to caption](https://arxiv.org/html/2610.28992v1/classifier_architecture_training.png)Figure 2:Five\-way classifier trained on AphasiaBank transcripts and labeled model scrambles using 14 text features and a text embedding, ensembled over 20 seeds\.![Refer to caption](https://arxiv.org/html/2610.28992v1/leisoning_and_steering.png)Figure 3:Left: CLAMP ranks heads by their score alone, knocks out the topkkuntil the target probability exceeds 0\.5, and prunes the set\.Right: a subtype vector, the patients’ minus the controls’ mean hidden state, is added at a layer and strength chosen by the classifier\.

## 4Method

We use two interventions to a frozen model: classifier\-guided lesioning, which disables attention heads, and subtype steering, which adds a direction to an intact model’s activations \(Figure[3](https://arxiv.org/html/2610.28992#S3.F3)\)\.

### 4\.1Model Lesioning

Prior methods intended to induce aphasic behaviour instead produce model failures\. For example,[Qin et al\. \(2026\)](https://arxiv.org/html/2610.28992#bib.bib34)remove a few critical neurons, causing model outputs to collapse into repetitive noise, while[Wang et al\. \(2025a\)](https://arxiv.org/html/2610.28992#bib.bib55)guide lesioning by matching model outputs to patient speech using text similarity\. Human experts identify the resulting outputs as model scrambles\. We therefore train a classifier to distinguish aphasia from model scrambles and use it to guide the lesion search\.

#### Five\-way classifier\.

The classifier assigns a text a probability for each of five classes: control speech, conduction or anomic aphasia, Broca’s aphasia, Wernicke’s aphasia, and model scrambles\. Conduction and anomic aphasia form one class because the two are hard to tell apart from free speech alone: the Western Aphasia Battery separates them by a repetition test\([Kertesz, 2007](https://arxiv.org/html/2610.28992#bib.bib20)\)\. We introduce a “model scrambles” category to capture non\-human machine artifacts, such as tokenizer loops, stray brackets, or formulaic repetitions\. Isolating these distinct computational failures prevents machine\-generated noise from contaminating the mapping between human clinical profiles and internal model components\. The training data hold 259 model\-scramble outputs of Llama\-2\-7B\-chat and GPT\-2 XL from a preliminary lesion study, selected using LLM\-as\-a\-judge with human annotator validation\. The classifier combines a BGE\-large sentence encoder\([Xiao et al\., 2024](https://arxiv.org/html/2610.28992#bib.bib58)\), fine\-tuned with low\-rank adapters\([Hu et al\., 2022](https://arxiv.org/html/2610.28992#bib.bib19)\), with 14 text features \(Figure[2](https://arxiv.org/html/2610.28992#S3.F2)\), and it averages 20 such models trained on different random splits\.

On held\-out transcripts whose text appears only once in the data, it reaches an accuracy of 83\.3% and a macro\-averaged F1 of 0\.78\. Lesioning and steering use three targets read off their probabilities:P⁡\(aphasia\)P\(\\mathrm\{aphasia\}\), the total probability of the three aphasia classes, which separates aphasic speech from both control speech and model scrambles;P⁡\(Broca\)P\(\\mathrm\{Broca\}\); andP⁡\(Wernicke\)P\(\\mathrm\{Wernicke\}\)\.

#### CLAMP\.

Since attention layers contain multiple parallel heads, we hypothesize that selectively lesioning heads can induce aphasia\-like behaviour without producing model scrambles\. We propose Classifier\-guided Lesion Accumulation with Minimal Pruning \(CLAMP\), which ablates attention heads by zeroing their outputs before the attention output projection at every prompt and generated\-token position\. For a fixed model, task, and target class, lets⁡\(H\)s\(H\)denote the classifier’s probability of that class for the response generated with head setHHablated\.CLAMPproceeds in three steps \(Figure[3](https://arxiv.org/html/2610.28992#S3.F3), left\)\. First, it ablates each head individually and ranks heads in descending order ofs⁡\(\{h\}\)s\(\\\{h\\\}\)\. Second, it buildsHHby adding heads in this order until their joint ablation yieldss⁡\(H\)\>0\.5s\(H\)\>0\.5\. Finally, it prunes heads individually, or in batches for large sets, as long ass⁡\(H\)s\(H\)remains above 0\.5\. In the final set, restoring any single head reducess⁡\(H\)s\(H\)to 0\.5 or below\. For the Cinderella task, we first ask “Do you know the story of Cinderella?”\. If the intact model answers affirmatively, we include the lesioned model’s narrative only if it also answers “yes” to this question\.

#### Baselines\.

We compareCLAMPwith eight lesion methods \(Appendix[D](https://arxiv.org/html/2610.28992#A4)\)\.Paramis the emergent\-modularity method of[Wang et al\. \(2025a\)](https://arxiv.org/html/2610.28992#bib.bib55): it zeroes the weights with the top 1% of\|w​∂L/∂w\|\|w\\,\\partial L/\\partial w\|in each weight matrix, whereLLis the loss on control narratives, and groups the lesions by the patient group whose transcripts the lesioned model’s answer resembles most by BLEU\-1 and BERTScore\.Param\-maskandHead\-masklearn, with the model frozen, how much to knock out each ofParam’s lesions of the MLP weight matrices or each head, as[Csordás et al\. \(2021\)](https://arxiv.org/html/2610.28992#bib.bib9)learn weight masks, so that the transcripts of the target group become more likely and those of the other groups less likely\.Param\-mask\-hardandHead\-mask\-hardknock out the components whose learned amount exceeds 0\.5 and prune them as above\. These four methods learn from the transcripts; the classifier only chooses the reported training step and, for the hard masks, guides the pruning\.Head\-CEMoptimizes the classifier directly with the cross\-entropy method\([Rubinstein, 1999](https://arxiv.org/html/2610.28992#bib.bib41)\), sampling 64 head sets per round for at most 20 rounds\.Param\-CgrowsParam’s MLP and attention head lesions by the classifier asCLAMP\.Head\-grown\-cosgrows and prunes heads asCLAMPdoes, but with a corpus similarity in place of the classifier: the answer’s mean cosine similarity to the target group’s transcripts minus that to the control transcripts\([Reimers and Gurevych, 2019](https://arxiv.org/html/2610.28992#bib.bib37)\)\.

#### Beyond the narrative\.

To evaluate other aphasia\-related abilities, we test every lesion on the other task, on the Text Aphasia Battery \(TAB\)’s short\-answer items for comprehension and repetition and its five connected\-text prompts\([Roll et al\., 2026b](https://arxiv.org/html/2610.28992#bib.bib40)\), and on two MMLU subjects, global facts \(tests broad general world knowledge\) and high\-school world history \(covers major global historical eras, movements, empires and revolutions taught in secondary education\)\([Hendrycks et al\., 2021](https://arxiv.org/html/2610.28992#bib.bib17)\), by the drop in the log\-probability of the correct answer\. A DeepSeek\-V4\.1\-Flash\([DeepSeek\-AI, 2026](https://arxiv.org/html/2610.28992#bib.bib12)\)LLM judge rates each connected\-text answer for the TAB’s 19 aphasia features, with the TAB’s instructions with in\-context examples of the features \(Appendix[K](https://arxiv.org/html/2610.28992#A11)\)\.

### 4\.2Steering

Steering builds its directions from the human transcripts \(Figure[3](https://arxiv.org/html/2610.28992#S3.F3)right\)\. For each task, the model reads every transcriptxxappended to the task prompt as if it were its own answer, andhℓ​\(x\)h\_\{\\ell\}\(x\)is the residual stream after layerℓ\\ell, averaged over the tokens ofxx\. The steering vector for a target at layerℓ\\ellis the difference between two group means,

𝐯ℓ=1\|T\|​∑x∈Thℓ​\(x\)−1\|C\|​∑x∈Chℓ​\(x\),\\mathbf\{v\}\_\{\\ell\}=\\frac\{1\}\{\|T\|\}\\sum\_\{x\\in T\}h\_\{\\ell\}\(x\)\-\\frac\{1\}\{\|C\|\}\\sum\_\{x\\in C\}h\_\{\\ell\}\(x\),\(1\)whereCCholds the control transcripts andTTthe transcripts of Broca’s aphasia for the Broca target, and of Wernicke’s aphasia for the Wernicke target\. Steering addsα​𝐯ℓ\\alpha\\mathbf\{v\}\_\{\\ell\}to the residual stream after layerℓ\\ellat every position while the model reads the prompt and generates\. We report its strength asr=‖α​𝐯ℓ‖/h¯ℓr=\\\|\\alpha\\mathbf\{v\}\_\{\\ell\}\\\|/\\bar\{h\}\_\{\\ell\}, whereh¯ℓ\\bar\{h\}\_\{\\ell\}is the mean norm ofhℓ​\(x\)h\_\{\\ell\}\(x\)over the transcripts\. For each model, target, and source task, a sweep over 8 layers spread across the model’s depth and 8 strengths fromr=0\.1r=0\.1to44picks the smallest strength, and the layer, at which the steered answer to the source task reaches the target\. If no setting reaches it, we keep the setting with the highest probability\.

The steered model answers seven prompts with this setting: the source task, which computed it, and six prompts that did not, the other task and the five connected\-text prompts of the TAB\. Two experts label every answer from models with one of the classifier’s five classes when steered toward a type\. We collect the annotation to check if the steering is successful \(Appendix[I](https://arxiv.org/html/2610.28992#A9)\)\.

#### Steering back\.

To test whether steering can undo a lesion, we remove the target’s direction from the residual stream of the lesioned model after every block, replacing each statehhwithh−\(h⊤​^​𝐯ℓ\)​^​𝐯ℓh\-\(h^\{\\top\}\\hat\{\}\\mathbf\{v\}\_\{\\ell\}\)\\,\\hat\{\}\\mathbf\{v\}\_\{\\ell\}, where^​𝐯ℓ=𝐯ℓ/‖𝐯ℓ‖\\hat\{\}\\mathbf\{v\}\_\{\\ell\}=\\mathbf\{v\}\_\{\\ell\}/\\\|\\mathbf\{v\}\_\{\\ell\}\\\|\([Arditi et al\., 2024](https://arxiv.org/html/2610.28992#bib.bib2)\)\. We also add the direction with a negative strength at a single layer\. As controls, we remove random directions in the same way, and we add random directions of the same norm\.

## 5Results

Table 2:Number of settings in which a lesion method finds a lesion that makes the model’s answer reach the target, by the five\-way classifier in 42 settings\.We focus on reporting results about the Broca’s and Wernicke’s subtypes, as their contrasting failure modes \(broken grammar vs\. semantic incoherence\) provide unambiguous mechanistic signals\. Conduction and Anomic profiles are omitted, as their subtle free\-production signatures heavily overlap with healthy controls \(Figure[6](https://arxiv.org/html/2610.28992#A2.F6)\)\.

We find language functions in LLMs can be successfully isolated to discrete structural circuits and continuous representational vectors\.

### 5\.1Model Lesioning

#### Classifier\-guided search outperforms corpus\-based search\.

Across 42 settings spanning seven models \(GPT\-2 XL, two Llama\-2\-chat models, and four Pythia models\), two tasks, and three targets, classifier\-guided methods find aphasia\-inducing lesions far more often than methods guided by corpus statistics \(Table[2](https://arxiv.org/html/2610.28992#S5.T2)\)\. Human experts agree with the classifier’s label on 90% of the lesioned models’ answers to both Cinderella and sandwich tasks\.

#### Head lesioning is more effective than parameter lesioning\.

CLAMPinduces the target aphasic behaviour in 38 of 42 settings; the remaining four yield model scrambles\.Param\-C, which lesions parameters rather than heads, succeeds in only 25 settings, supporting our hypothesis that selectively lesioning heads reduces model degeneration\. Meanwhile,Head\-CEM, which optimizes the classifier objective more aggressively, succeeds in only 23 settings, and 12 of the remaining 19 yield model scrambles\.CLAMPinstead uses classifier scores for head ranking and threshold checks during pruning, which may help keep responses closer to the classifier’s training distribution\.

![Refer to caption](https://arxiv.org/html/2610.28992v1/fig_grown_heads.png)Figure 4:CLAMPlesions in 11 models, two tasks, and three targets: aphasia \(Aph\.\), Broca’s and Wernicke’s aphasia\. A cell gives the lesion’s number of heads, and its color the classifier’s target probability for the lesioned output\. Gray cells mark no effect of lesioning\.
#### CLAMPgeneralizes to Qwen models with linear attention\.

Across all 11 models,CLAMPfinds successful lesions in 54 of 66 settings, with the remaining settings yielding model scrambles \(Figure[4](https://arxiv.org/html/2610.28992#S5.F4)\)\. We find aphasia\-inducing lesions for every model on both tasks, while Broca\-like and Wernicke\-like behaviours are induced in 15 and 17 of 22 settings, respectively\. For example, ablating 101 heads in Llama\-2\-13B\-chat transforms its fluent Cinderella narrative into “to tell a tale of Ct\.\. to tell me\.”, with a classifier score ofP⁡\(Broca\)=0\.89P\(\\mathrm\{Broca\}\)=0\.89, while the model still confirms that it knows the story\.

#### The heads of a lesion act together while the model reads the prompt\.

We use activation patching to identify when lesioned heads affect model behavior\. Restoring the clean activations of all heads of a lesion while the model reads the prompt, so that the heads stay knocked out only while it writes its answer, undoes 53 of the 54 lesions\. Restoring a single head undoes the lesion for a median of 83% of a lesion’s heads, yet the classifier labels 80% of these answers as model scrambles and only 13% as control speech\. The human aphasia\-like answer thus needs nearly all heads of the lesion acting on the prompt together, a circuit in which changing one head mostly breaks the model \(see Appendix[G](https://arxiv.org/html/2610.28992#A7)\)\.

#### Lesions for different aphasia types share heads\.

The Broca and Wernicke lesions of the same model and task share a median of 54% of the heads of the smaller lesion\. Across the 13 pairs, 517 heads are shared, three times the number expected by chance, and overlap is significant in 11 pairs \(p<0\.01p<0\.01\)\. This overlap begins at single heads: a head whose knockout makes the answer more Broca\-like usually makes it more Wernicke\-like too, at a median rank correlation of 0\.93 across heads\. The circuits for Broca\-like and Wernicke\-like breakdowns thus share far more heads than chance, entangling the two types rather than separating them\.

Table 3:What theCLAMPlesions do beyond the narrative they were found on, as medians over the lesions that reach their target\.*Features*: aphasia features of the TAB, of 19, that the LLM judge finds per connected\-text answer of the intact and of the lesioned model\.*Short Answer*: TAB short\-answer items for comprehension and repetition that the lesioned model gets right, of 15\.*MMLU*: drop in the log\-probability of the correct answer on two MMLU subjects, in nats\.
#### The lesions damage language on new prompts, but not as the targeted type\.

Almost every lesion found on one narrative also damages the model’s answers to the TAB \(Table[3](https://arxiv.org/html/2610.28992#S5.T3)\)\. The LLM judge finds more of the TAB’s aphasia features in the connected text for 50 of the 54 lesions than for the intact models, a median of 2\.7 of 19 features per answer against 0\.0\. The lesioned models get a median of 4 of the 15 short\-answer items right, against 8\.5 for the same models intact\. The targeted type, however, does not carry over: the Broca and Wernicke lesions reach their type in only 1 of the 184 scored answers that they give to the other narrative and to the TAB’s connected\-text prompts \(Appendix[F](https://arxiv.org/html/2610.28992#A6)\)\. The lesions thus damage language broadly, but the type that they induce stays in the answer on which the search found them\.

### 5\.2Steering

#### One direction per type makes an intact model “speak” like a patient\.

Adding the Broca direction at a single layer makes the classification label Broca in all 11 models on both tasks, and the Wernicke direction does so for Wernicke’s aphasia in 12 of the 22 model\-task pairs \(Appendix[H](https://arxiv.org/html/2610.28992#A8)\)\. Two experts labeled the steered answers of seven models \(Figure[5](https://arxiv.org/html/2610.28992#S5.F5)\): either expert identifies successful steering to the targeted type for 58% of the Broca answers, and 58% of the Wernicke answers\. Thus, adding a single direction to the residual stream can make an intact model produce speech resembling a targeted aphasia subtype\. Steering toward Broca’s aphasia, however, often scrambles the model, possibly because the corpus contains no examples of scrambled model outputs\.

Figure 5:Expert labels of the answers steered toward Broca’s or Wernicke’s aphasia, by target and prompt\.*Source task*: the narrative whose answers chose the steering layer and strength;*other prompts*: the other narrative and the five connected\-text prompts of the Text Aphasia Battery \(TAB\)\. An answer counts as reaching the target when either expert labels it so; any other answer is split evenly between its two experts’ labels\.
#### Steering carries the targeted type to new prompts\.

On the six prompts not used to choose the steering setting, either expert labels the targeted type for 65% of the Broca answers and 63% of the Wernicke answers that are not model scrambles \(Appendix[I](https://arxiv.org/html/2610.28992#A9)\)\. The type follows the direction: either expert labels 44 of the 76 Broca\-steered answers to these prompts Broca\-like but only 13 of the 77 Wernicke\-steered ones, and 44 of the 77 Wernicke\-steered answers Wernicke\-like but only 9 of the 76 Broca\-steered ones\. The classifier finds the targeted type in 67 of the 153 steered answers to new prompts, but in only 1 of the 139 scored answers of the Broca and Wernicke lesions of the same models\. A direction thus keeps its type on new prompts, whereas a lesion leaves only general damage\.

#### The Broca and Wernicke directions share most of their length\.

Both directions point nearly along each other, at a median cosine over layers of 0\.79 to 0\.90 in every model and task, and along the aphasia direction of all aphasic transcripts, at a median cosine of 0\.97 for Broca and 0\.92 for Wernicke \(Appendix[H](https://arxiv.org/html/2610.28992#A8)\)\. The two types differ little: after projecting out the general aphasia direction extracted from the whole aphasia group, the remaining parts point away from each other, with median cosines of−0\.65\-0\.65to−0\.41\-0\.41\. The Wernicke part carries its type: when added alone, it pushesP⁡\(Wernicke\)P\(\\mathrm\{Wernicke\}\)above 0\.5 in all 22 model\-task pairs\. Like the lesions, the directions thus share most of what makes speech aphasic, with a small part that sets each type apart\.

Table 4:Agreement \(%\) between automated labelers and two human experts on steered model responses\. Two\-way: Binary classification \(aphasic vs\. non\-aphasic\) of responses steered toward aphasia\. Five\-way: Five\-class classification of responses steered toward Broca’s or Wernicke’s aphasia\.
#### The classifier agrees with the experts more often than an LLM judge\.

Across all steered answers, including those to the out\-of\-distribution TAB prompts, the classifier still matches either expert’s label on 65% of answers, compared with 55% for a DeepSeek\-V4\.1\-Flash judge\. \(Table[4](https://arxiv.org/html/2610.28992#S5.T4)The two experts agree with each other on 80% of answers, with a pooled Cohen’sκ\\kappaof 0\.76\. Much of the classifier’s advantage comes from the 77 answers the experts identified as model scrambles: its labels agree with either expert on 51 of these answers, compared with 23 for the judge\.

#### Steering back undoes the lesions, but does not “heal” them\.

Removing a lesion’s own direction from the residual stream after every block undoes 53 of the 54 lesions, but 52 of these answers are model scrambles \(Appendix[J](https://arxiv.org/html/2610.28992#A10)\)\. Random directions undo the lesions almost as often: removing one of three random directions undoes 50 or 51 of the lesions, and 91% of the pushes with a random direction of the same norm at one layer undo the lesion, against 95% of those with the lesion’s own direction\. Undoing a lesion is thus easy but unspecific: nearly every perturbation we tried moves the answer off its target, mostly into model scrambles rather than control speech\.

## 6Conclusion and Discussion

In this work, we successfully bridged neuropsychology and mechanistic interpretability to map the failure modes of large language models\. Specifically, we make the following core contributions:

#### Graded model degradation\.

First, this work shifts the conceptual focus of AI safety and interpretability from catastrophic collapse to graded, structured degradation\. By applying an aphasia\-inspired diagnostic lens, we demonstrate that LLM breakdowns are not merely random noise, but structured deficits that align with recognized clinical profiles\. This provides the field with a more granular vocabulary to evaluate “bad” or impaired language generation, opening the door to modeling other language deficits in artificial systems\.

#### A complementary methodological framework for mechanistic interpretability\.

Second, we contribute a robust, dual\-approach framework for mechanistic interpretability\. Relying on a single intervention method can yield artifacts tied to that specific technique\. By cross\-validating our findings through complementary approaches, combining the structural isolation of classifier\-guided lesioning with the representational fluidity of continuous activation steering, we establish a more rigorous standard for proving that an identified language function is a fundamental property of the model\. This dual inside\-out/outside\-in approach ensures that mapped vulnerabilities are highly reliable\.

#### A lightweight classifier with real\-world applicability\.

Third, while developed primarily as part of the lesion approach, our classifier represents a standalone contribution to automated clinical annotation\. Recent research has demonstrated the promise of combining LLM\-based measures \(such as surprisal\) with traditional clinical features for aphasia classification[Cong et al\. \(2024\)](https://arxiv.org/html/2610.28992#bib.bib72)\. Our model advances this paradigm by offering a highly efficient, lightweight alternative with higher overall and per\-class accuracy\.

## Limitations

First, our lesions and the steering directions come from two English narrative tasks in AphasiaBank: the Cinderella story and the sandwich procedure\. However, AphasiaBank holds further tasks, such as picture descriptions \(e\.g\., participants are asked to look at the picture and tell a beginning, middle, and end of a story\) and personal narratives \(e\.g\., stroke, illness, and recovery story\), and the same pipeline can be applicable to them with minimal changes \(although in the case of the picture description task, Vision\-Language Models should be involved\), as it would to aphasia corpora in other languages\.

Second, the classifier learns machine failures from only 259 expert\-labeled examples\. Although it detects only about half of the steered responses that experts classify as model\-scrambled, incorporating additional expert\-labeled failures into its training data could improve its performance\. In both current and future studies, labels produced by the LLM judge and classifier should be interpreted as assistive signals rather than clinical diagnoses\.

Third, our experiments focus on a limited set of language models, leaving the generalization of our findings to other model families and architectures uncertain\. Evaluating additional models would help establish the robustness of our findings across model scales and generations\. Furthermore, extending our interventions to mixture\-of\-experts \(MoE\) architectures is another important direction for future work, as sparse expert routing may influence the mechanisms and effectiveness of model lesioning and steering\.

## References

- Alain and Bengio \(2016\)G\. Alain and Y\. BengioUnderstanding intermediate layers using linear classifier probes\.ArXivabs/1610\.01644\.External Links:[Link](https://api.semanticscholar.org/CorpusID:9794990)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.
- Almeidaet al\.\(2026\)T\. Almeida, Z\. Zhu, and Y\. NingConceptual hierarchies within LLMs\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 41067–41079\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.2041),[Link](https://aclanthology.org/2026.findings-acl.2041/)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p1.1)\.
- Arditiet al\.\(2024\)A\. Arditi, O\. Obeso, A\. Syed, D\. Paleka, N\. Panickssery, W\. Gurnee, and N\. NandaRefusal in language models is mediated by a single direction\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.37,pp\. 136037–136083\.External Links:[Document](https://dx.doi.org/10.52202/079017-4322),[Link](https://doi.org/10.52202/079017-4322)Cited by:[Appendix J](https://arxiv.org/html/2610.28992#A10.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1),[§4\.2](https://arxiv.org/html/2610.28992#S4.SS2.SSS0.Px1.p1.1)\.
- Baroni \(2019\)M\. BaroniLinguistic generalization and compositionality in modern artificial neural networks\.Philosophical Transactions of the Royal Society B375\.External Links:[Link](https://api.semanticscholar.org/CorpusID:90260325)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p1.1)\.
- Beckmann and Queloz \(2026\)P\. Beckmann and M\. QuelozMechanistic indicators of understanding in large language models\.Philosophical Studies183\(6\),pp\. 1747–1792\.External Links:[Document](https://dx.doi.org/10.1007/s11098-026-02513-1),[Link](https://doi.org/10.1007/s11098-026-02513-1)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p2.1)\.
- Belinkov \(2022\)Y\. BelinkovProbing classifiers: promises, shortcomings, and advances\.Computational Linguistics48\(1\),pp\. 207–219\.External Links:[Link](https://aclanthology.org/2022.cl-1.7/),[Document](https://dx.doi.org/10.1162/coli%5Fa%5F00422)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.
- Bidermanet al\.\(2023\)S\. Biderman, H\. Schoelkopf, Q\. Anthony, H\. Bradley, K\. O’Brien, E\. Hallahan, M\. A\. Khan, S\. Purohit, U\. S\. Prashanth, E\. Raff, A\. Skowron, L\. Sutawika, and O\. van der WalPythia: a suite for analyzing large language models across training and scaling\.InProceedings of the 40th International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.202,pp\. 2397–2430\.External Links:[Link](https://arxiv.org/abs/2304.01373)Cited by:[Table 5](https://arxiv.org/html/2610.28992#A2.T5.2.7.1.1),[§3](https://arxiv.org/html/2610.28992#S3.SS0.SSS0.Px2.p1.1)\.
- Broca \(1861\)P\. BrocaRemarques sur le siège de la faculté du langage articulé, suivies d’une observation d’aphémie \(perte de la parole\)\.Bulletin et Memoires de la Societe anatomique de Paris6,pp\. 330–357\.External Links:[Link](https://pure.mpg.de/rest/items/item_2301213/component/file_2452907/content)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px1.p2.1)\.
- Chomsky \(1957\)N\. ChomskySyntactic structures\.Mouton & Co\.,The Hague\.External Links:[Document](https://dx.doi.org/10.1515/9783112316009),[Link](https://doi.org/10.1515/9783112316009)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p1.1)\.
- Cohen \(1960\)J\. CohenA coefficient of agreement for nominal scales\.Educational and Psychological Measurement20\(1\),pp\. 37–46\.External Links:[Document](https://dx.doi.org/10.1177/001316446002000104),[Link](https://doi.org/10.1177/001316446002000104)Cited by:[Appendix I](https://arxiv.org/html/2610.28992#A9.SS0.SSS0.Px3.p1.1)\.
- Conget al\.\(2024\)Y\. Cong, A\. N\. LaCroix, and J\. LeeClinical efficacy of pre\-trained large language models through the lens of aphasia\.Scientific reports14\(1\),pp\. 15573\.External Links:[Link](https://www.nature.com/articles/s41598-024-66576-y)Cited by:[§6](https://arxiv.org/html/2610.28992#S6.SS0.SSS0.Px3.p1.1)\.
- Csordáset al\.\(2021\)R\. Csordás, S\. van Steenkiste, and J\. SchmidhuberAre neural nets modular? Inspecting functional modularity through differentiable weight masks\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2010.02066)Cited by:[Appendix D](https://arxiv.org/html/2610.28992#A4.SS0.SSS0.Px8.p1.1),[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.SSS0.Px3.p1.1)\.
- de Boeret al\.\(2005\)P\. de Boer, D\. P\. Kroese, S\. Mannor, and R\. Y\. RubinsteinA tutorial on the cross\-entropy method\.Annals of Operations Research134\(1\),pp\. 19–67\.External Links:[Document](https://dx.doi.org/10.1007/s10479-005-5724-z),[Link](https://doi.org/10.1007/s10479-005-5724-z)Cited by:[Appendix D](https://arxiv.org/html/2610.28992#A4.SS0.SSS0.Px9.p1.1)\.
- De Caoet al\.\(2020\)N\. De Cao, M\. S\. Schlichtkrull, W\. Aziz, and I\. TitovHow do decisions emerge across layers in neural models? Interpretation with differentiable masking\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Online,pp\. 3243–3255\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.262),[Link](https://aclanthology.org/2020.emnlp-main.262/)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- DeepSeek\-AI \(2026\)DeepSeek\-AIDeepSeek\-V4\.1\-Flash: pushing the limits of KV cache compression\.External Links:2609\.19969,[Document](https://dx.doi.org/10.48550/arXiv.2609.19969),[Link](https://arxiv.org/abs/2609.19969)Cited by:[Appendix K](https://arxiv.org/html/2610.28992#A11.SS0.SSS0.Px1.p1.1),[Appendix B](https://arxiv.org/html/2610.28992#A2.SS0.SSS0.Px8.p1.1),[Appendix F](https://arxiv.org/html/2610.28992#A6.SS0.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.SSS0.Px4.p1.1)\.
- Dronkerset al\.\(2017\)N\. F\. Dronkers, M\. V\. Ivanova, and J\. V\. BaldoWhat do language disorders reveal about brain–language relationships? from classic models to network approaches\.Journal of the International Neuropsychological Society23\(9\-10\),pp\. 741–754\.External Links:[Document](https://dx.doi.org/10.1017/S1355617717001126),[Link](https://doi.org/10.1017/S1355617717001126)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p2.1)\.
- Fedorenkoet al\.\(2024\)E\. Fedorenko, A\. A\. Ivanova, and T\. I\. RegevThe language network as a natural kind within the broader landscape of the human brain\.Nature Reviews Neuroscience25\(5\),pp\. 289–312\.External Links:[Document](https://dx.doi.org/10.1038/s41583-024-00802-4),[Link](https://doi.org/10.1038/s41583-024-00802-4)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p2.1)\.
- Frantar and Alistarh \(2023\)E\. Frantar and D\. AlistarhSparseGPT: massive language models can be accurately pruned in one\-shot\.InProceedings of the 40th International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.202,pp\. 10323–10337\.External Links:[Link](https://proceedings.mlr.press/v202/frantar23a.html)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- Geschwind \(1970\)N\. GeschwindThe organization of language and the brain: language disorders after brain damage help in elucidating the neural basis of verbal behavior\.Science170\(3961\),pp\. 940–944\.External Links:[Document](https://dx.doi.org/10.1126/science.170.3961.940),[Link](https://doi.org/10.1126/science.170.3961.940)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p2.1)\.
- Heet al\.\(2026\)S\. He, G\. Sun, Z\. Shen, and A\. LiUncovering the redundancy in Transformers via a unified study of layer dropping\.Transactions on Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=1I7PCbOPfe)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p1.1),[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- Heimersheim and Nanda \(2024\)S\. Heimersheim and N\. NandaHow to use and interpret activation patching\.arXiv preprint arXiv:2404\.15255\.External Links:[Link](https://arxiv.org/abs/2404.15255)Cited by:[Appendix G](https://arxiv.org/html/2610.28992#A7.SS0.SSS0.Px1.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. SteinhardtMeasuring massive multitask language understanding\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2009.03300)Cited by:[Appendix B](https://arxiv.org/html/2610.28992#A2.SS0.SSS0.Px9.p1.1),[Appendix F](https://arxiv.org/html/2610.28992#A6.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.SSS0.Px4.p1.1)\.
- Hewitt and Manning \(2019\)J\. Hewitt and C\. D\. ManningA structural probe for finding syntax in word representations\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),Minneapolis, Minnesota,pp\. 4129–4138\.External Links:[Link](https://aclanthology.org/N19-1419/),[Document](https://dx.doi.org/10.18653/v1/N19-1419)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.
- Hoffmann and Chen \(2013\)M\. Hoffmann and R\. ChenThe spectrum of aphasia subtypes and etiology in subacute stroke\.Journal of Stroke and Cerebrovascular Diseases22\(8\),pp\. 1385–1392\.External Links:[Document](https://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2013.04.017),[Link](https://doi.org/10.1016/j.jstrokecerebrovasdis.2013.04.017)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p2.1)\.
- Huet al\.\(2022\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2106.09685)Cited by:[Appendix C](https://arxiv.org/html/2610.28992#A3.SS0.SSS0.Px5.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.SSS0.Px1.p1.1)\.
- Kertesz \(2007\)A\. KerteszWestern Aphasia Battery\-Revised\.PsychCorp,San Antonio, TX\.Cited by:[Appendix A](https://arxiv.org/html/2610.28992#A1.p1.1),[Appendix H](https://arxiv.org/html/2610.28992#A8.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.SSS0.Px1.p1.1)\.
- Kuribayashiet al\.\(2025\)T\. Kuribayashi, Y\. Oseki, S\. B\. Taieb, K\. Inui, and T\. BaldwinLarge language models are human\-like internally\.Transactions of the Association for Computational Linguistics13,pp\. 1743–1766\.External Links:[Document](https://dx.doi.org/10.1162/tacl.a.58),[Link](https://aclanthology.org/2025.tacl-1.78/)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p2.1)\.
- Landis and Koch \(1977\)J\. R\. Landis and G\. G\. KochThe measurement of observer agreement for categorical data\.Biometrics33\(1\),pp\. 159–174\.External Links:[Document](https://dx.doi.org/10.2307/2529310),[Link](https://doi.org/10.2307/2529310)Cited by:[Appendix I](https://arxiv.org/html/2610.28992#A9.SS0.SSS0.Px3.p1.1),[Appendix I](https://arxiv.org/html/2610.28992#A9.SS0.SSS0.Px4.p1.1)\.
- Liet al\.\(2023\)K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. WattenbergInference\-time intervention: eliciting truthful answers from a language model\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.36,pp\. 41451–41530\.External Links:[Link](https://doi.org/10.52202/075280-1797),[Document](https://dx.doi.org/10.52202/075280-1797)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.
- Loshchilov and Hutter \(2019\)I\. Loshchilov and F\. HutterDecoupled weight decay regularization\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/1711.05101)Cited by:[Appendix C](https://arxiv.org/html/2610.28992#A3.SS0.SSS0.Px6.p1.1)\.
- Luet al\.\(2026\)C\. Lu, Y\. Tang, and A\. CavallaroSparse neuron ablation triggers catastrophic collapse of the language core in vision\-language models\.InMechanistic Interpretability Workshop at ICML 2026,External Links:[Link](https://openreview.net/forum?id=V1HjWJuGS3)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p1.1),[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- MacWhinneyet al\.\(2011\)B\. MacWhinney, D\. Fromm, M\. Forbes, and A\. HollandAphasiaBank: methods for studying discourse\.Aphasiology25\(11\),pp\. 1286–1307\.External Links:[Document](https://dx.doi.org/10.1080/02687038.2011.589893),[Link](https://doi.org/10.1080/02687038.2011.589893)Cited by:[Appendix A](https://arxiv.org/html/2610.28992#A1.p1.1),[Appendix B](https://arxiv.org/html/2610.28992#A2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2610.28992#S3.SS0.SSS0.Px1.p1.1)\.
- MacWhinney \(2018\)B\. MacWhinneyTools for analyzing talk, part 2: the CLAN program\.TalkBank\.External Links:[Document](https://dx.doi.org/10.21415/T5G10R),[Link](https://doi.org/10.21415/T5G10R)Cited by:[Appendix C](https://arxiv.org/html/2610.28992#A3.SS0.SSS0.Px4.p1.1)\.
- Marks and Tegmark \(2024\)S\. Marks and M\. TegmarkThe geometry of truth: emergent linear structure in large language model representations of true/false datasets\.InConference on Language Modeling \(COLM\),External Links:[Link](https://arxiv.org/abs/2310.06824)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.
- Menget al\.\(2022\)K\. Meng, D\. Bau, A\. Andonian, and Y\. BelinkovLocating and editing factual associations in GPT\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 17359–17372\.External Links:[Document](https://dx.doi.org/10.52202/068431-1262),[Link](https://doi.org/10.52202/068431-1262)Cited by:[Appendix G](https://arxiv.org/html/2610.28992#A7.SS0.SSS0.Px1.p1.1)\.
- Michelet al\.\(2019\)P\. Michel, O\. Levy, and G\. NeubigAre sixteen heads really better than one?\.InAdvances in Neural Information Processing Systems,Vol\.32,pp\. 14014–14024\.External Links:[Link](https://arxiv.org/abs/1905.10650)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- Nastaseet al\.\(2026\)S\. A\. Nastase, Z\. Zada, A\. Goldberg, and U\. HassonUnifying the structures of language in a neural population code\.Neuron114\(19\),pp\. 3527–3549\.External Links:[Document](https://dx.doi.org/10.1016/j.neuron.2026.07.024),[Link](https://doi.org/10.1016/j.neuron.2026.07.024)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p1.1)\.
- Papineniet al\.\(2002\)K\. Papineni, S\. Roukos, T\. Ward, and W\. ZhuBLEU: a method for automatic evaluation of machine translation\.InProceedings of the 40th Annual Meeting of the Association for Computational Linguistics,Philadelphia, Pennsylvania, USA,pp\. 311–318\.External Links:[Document](https://dx.doi.org/10.3115/1073083.1073135),[Link](https://aclanthology.org/P02-1040/)Cited by:[Appendix D](https://arxiv.org/html/2610.28992#A4.SS0.SSS0.Px6.p1.1)\.
- Parket al\.\(2024\)K\. Park, Y\. J\. Choe, and V\. VeitchThe linear representation hypothesis and the geometry of large language models\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.235,pp\. 39643–39666\.External Links:[Link](https://proceedings.mlr.press/v235/park24c.html)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.
- Qinet al\.\(2026\)Z\. Qin, Q\. Yu, K\. Lyu, Z\. Fan, and Y\. SunThe Achilles’ heel of LLMs: how altering a handful of neurons can cripple language abilities\.InInternational Conference on Learning Representations \(ICLR\),pp\. 155536–155564\.External Links:[Link](https://arxiv.org/abs/2510.10238)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p1.1),[§1](https://arxiv.org/html/2610.28992#S1.p2.1),[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.p1.1)\.
- Qwen Team \(2026\)Qwen TeamQwen3\.5: towards native multimodal agents\.External Links:[Link](https://qwen.ai/blog?id=qwen3.5)Cited by:[Table 5](https://arxiv.org/html/2610.28992#A2.T5.2.12.1.1),[§3](https://arxiv.org/html/2610.28992#S3.SS0.SSS0.Px2.p1.1)\.
- Radfordet al\.\(2019\)A\. Radford, J\. Wu, R\. Child, D\. Luan, D\. Amodei, and I\. SutskeverLanguage models are unsupervised multitask learners\.Technical reportOpenAI\.External Links:[Link](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)Cited by:[Table 5](https://arxiv.org/html/2610.28992#A2.T5.2.2.1.1),[§3](https://arxiv.org/html/2610.28992#S3.SS0.SSS0.Px2.p1.1)\.
- Reimers and Gurevych \(2019\)N\. Reimers and I\. GurevychSentence\-BERT: sentence embeddings using Siamese BERT\-networks\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),Hong Kong, China,pp\. 3982–3992\.External Links:[Document](https://dx.doi.org/10.18653/v1/D19-1410),[Link](https://aclanthology.org/D19-1410/)Cited by:[Appendix C](https://arxiv.org/html/2610.28992#A3.SS0.SSS0.Px4.p1.1),[Appendix D](https://arxiv.org/html/2610.28992#A4.SS0.SSS0.Px10.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.SSS0.Px3.p1.1)\.
- Rimskyet al\.\(2024\)N\. Rimsky, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. TurnerSteering Llama 2 via contrastive activation addition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand,pp\. 15504–15522\.External Links:[Link](https://aclanthology.org/2024.acl-long.828/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.828)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.
- Rollet al\.\(2026a\)N\. Roll, J\. Kries, L\. Gwilliams, and C\. ShainArtificial aphasias in lesioned language models\.arXiv preprint arXiv:2605\.16222\.External Links:[Link](https://arxiv.org/abs/2605.16222)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px1.p2.1)\.
- Rollet al\.\(2026b\)N\. Roll, J\. Kries, F\. Jin, C\. Wang, A\. M\. Finley, M\. Sumner, C\. Shain, and L\. GwilliamsThe Text Aphasia Battery \(TAB\): a clinically\-grounded benchmark for aphasia\-like deficits in language models\.InProceedings of the 11th Workshop on Computational Linguistics and Clinical Psychology \(CLPsych 2026\),San Diego, California, USA,pp\. 340–354\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.clpsych-1.27),[Link](https://aclanthology.org/2026.clpsych-1.27/)Cited by:[Appendix K](https://arxiv.org/html/2610.28992#A11.SS0.SSS0.Px3.p1.1),[Appendix B](https://arxiv.org/html/2610.28992#A2.SS0.SSS0.Px8.p1.1),[Table 7](https://arxiv.org/html/2610.28992#A2.T7),[Appendix F](https://arxiv.org/html/2610.28992#A6.p1.1),[Listing 4](https://arxiv.org/html/2610.28992#LST4),[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.SSS0.Px4.p1.1)\.
- Rubinstein \(1999\)R\. RubinsteinThe cross\-entropy method for combinatorial and continuous optimization\.Methodology and Computing in Applied Probability1\(2\),pp\. 127–190\.External Links:[Document](https://dx.doi.org/10.1023/A%3A1010091220143),[Link](https://doi.org/10.1023/A:1010091220143)Cited by:[Appendix D](https://arxiv.org/html/2610.28992#A4.SS0.SSS0.Px9.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.SSS0.Px3.p1.1)\.
- Rumelhartet al\.\(1986\)D\. E\. Rumelhart, J\. L\. McClelland, and PDP Research GroupParallel distributed processing: explorations in the microstructure of cognition, volume 1: foundations\.MIT Press,Cambridge, MA\.External Links:[Document](https://dx.doi.org/10.7551/mitpress/5236.001.0001),[Link](https://doi.org/10.7551/mitpress/5236.001.0001)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p1.1)\.
- Shiet al\.\(2023\)D\. Shi, C\. Tao, Y\. Jin, Z\. Yang, C\. Yuan, and J\. WangUPop: unified and progressive pruning for compressing vision\-language transformers\.InProceedings of the 40th International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.202,pp\. 31292–31311\.External Links:[Link](https://proceedings.mlr.press/v202/shi23e.html)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- Subramaniet al\.\(2022\)N\. Subramani, N\. Suresh, and M\. PetersExtracting latent steering vectors from pretrained language models\.InFindings of the Association for Computational Linguistics: ACL 2022,Dublin, Ireland,pp\. 566–581\.External Links:[Link](https://aclanthology.org/2022.findings-acl.48/),[Document](https://dx.doi.org/10.18653/v1/2022.findings-acl.48)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.
- Sunet al\.\(2024\)M\. Sun, Z\. Liu, A\. Bair, and J\. Z\. KolterA simple and effective pruning approach for large language models\.InInternational Conference on Learning Representations \(ICLR\),pp\. 4942–4964\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/14c856c7a41297804de4c4890e846b25-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- Teehanet al\.\(2022\)R\. Teehan, M\. Clinciu, O\. Serikov, E\. Szczechla, N\. Seelam, S\. Mirkin, and A\. GokaslanEmergent structures and training dynamics in large language models\.InProceedings of BigScience Episode \#5 – Workshop on Challenges & Perspectives in Creating Large Language Models,virtual\+Dublin,pp\. 146–159\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.bigscience-1.11),[Link](https://aclanthology.org/2022.bigscience-1.11/)Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p2.1)\.
- Tenneyet al\.\(2019\)I\. Tenney, D\. Das, and E\. PavlickBERT rediscovers the classical NLP pipeline\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 4593–4601\.Cited by:[§1](https://arxiv.org/html/2610.28992#S1.p1.1)\.
- Touvronet al\.\(2023\)H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale, D\. Bikel, L\. Blecher, C\. C\. Ferrer, M\. Chen, G\. Cucurull, D\. Esiobu, J\. Fernandes, J\. Fu, W\. Fu, B\. Fuller, C\. Gao, V\. Goswami, N\. Goyal, A\. Hartshorn, S\. Hosseini, R\. Hou, H\. Inan, M\. Kardas, V\. Kerkez, M\. Khabsa, I\. Kloumann, A\. Korenev, P\. S\. Koura, M\. Lachaux, T\. Lavril, J\. Lee, D\. Liskovich, Y\. Lu, Y\. Mao, X\. Martinet, T\. Mihaylov, P\. Mishra, I\. Molybog, Y\. Nie, A\. Poulton, J\. Reizenstein, R\. Rungta, K\. Saladi, A\. Schelten, R\. Silva, E\. M\. Smith, R\. Subramanian, X\. E\. Tan, B\. Tang, R\. Taylor, A\. Williams, J\. X\. Kuan, P\. Xu, Z\. Yan, I\. Zarov, Y\. Zhang, A\. Fan, M\. Kambadur, S\. Narang, A\. Rodriguez, R\. Stojnic, S\. Edunov, and T\. ScialomLlama 2: open foundation and fine\-tuned chat models\.arXiv preprint arXiv:2307\.09288\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2307.09288),[Link](https://arxiv.org/abs/2307.09288)Cited by:[Table 5](https://arxiv.org/html/2610.28992#A2.T5.2.4.1.1),[§3](https://arxiv.org/html/2610.28992#S3.SS0.SSS0.Px2.p1.1)\.
- Turneret al\.\(2023\)A\. M\. Turner, L\. Thiergart, G\. Leech, D\. Udell, J\. J\. Vazquez, U\. Mini, and M\. MacDiarmidSteering language models with activation engineering\.arXiv preprint arXiv:2308\.10248\.External Links:[Link](https://arxiv.org/abs/2308.10248)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.
- Viget al\.\(2020\)J\. Vig, S\. Gehrmann, Y\. Belinkov, S\. Qian, D\. Nevo, Y\. Singer, and S\. ShieberInvestigating gender bias in language models using causal mediation analysis\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 12388–12401\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/hash/92650b2e92217715fe312e6fa7b90d82-Abstract.html)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- Voitaet al\.\(2019\)E\. Voita, D\. Talbot, F\. Moiseev, R\. Sennrich, and I\. TitovAnalyzing multi\-head self\-attention: specialized heads do the heavy lifting, the rest can be pruned\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,Florence, Italy,pp\. 5797–5808\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1580),[Link](https://aclanthology.org/P19-1580/)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2025a\)C\. Wang, Z\. Fan, Z\. Han, Y\. Bi, and J\. LiEmergent modularity in large language models: insights from aphasia simulations\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.02.22.639416),[Link](https://doi.org/10.1101/2025.02.22.639416)Cited by:[Appendix D](https://arxiv.org/html/2610.28992#A4.SS0.SSS0.Px6.p1.1),[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.p1.1)\.
- Wanget al\.\(2023a\)K\. Wang, A\. Variengien, A\. Conmy, B\. Shlegeris, and J\. SteinhardtInterpretability in the wild: a circuit for indirect object identification in GPT\-2 small\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2211.00593)Cited by:[Appendix G](https://arxiv.org/html/2610.28992#A7.SS0.SSS0.Px4.p1.1),[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2023b\)T\. Wang, W\. Zhou, Y\. Zeng, and X\. ZhangEfficientVLM: fast and accurate vision\-language models via knowledge distillation and modal\-adaptive pruning\.InFindings of the Association for Computational Linguistics: ACL 2023,Toronto, Canada,pp\. 13899–13913\.External Links:[Link](https://aclanthology.org/2023.findings-acl.873/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.873)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2020\)W\. Wang, F\. Wei, L\. Dong, H\. Bao, N\. Yang, and M\. ZhouMiniLM: deep self\-attention distillation for task\-agnostic compression of pre\-trained transformers\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.33,pp\. 5776–5788\.External Links:[Link](https://arxiv.org/abs/2002.10957)Cited by:[Appendix C](https://arxiv.org/html/2610.28992#A3.SS0.SSS0.Px4.p1.1),[Appendix D](https://arxiv.org/html/2610.28992#A4.SS0.SSS0.Px10.p1.1)\.
- Wanget al\.\(2025b\)Y\. Wang, J\. Sun, J\. Zheng, Y\. Zhang, C\. Ye, J\. Li, C\. Zong, and S\. WangBridging brains and models: MoE\-based functional lesions for simulating and rehabilitating aphasia\.arXiv preprint arXiv:2508\.04749\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2508.04749),[Link](https://arxiv.org/abs/2508.04749)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px1.p1.1)\.
- Wernicke \(1874\)C\. WernickeDer aphasische Symptomencomplex: eine psychologische Studie auf anatomischer Basis\.Max Cohn & Weigert,Breslau\.Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px1.p2.1)\.
- Wolfet al\.\(2020\)T\. Wolf, L\. Debut, V\. Sanh, J\. Chaumond, C\. Delangue, A\. Moi, P\. Cistac, T\. Rault, R\. Louf, M\. Funtowicz, J\. Davison, S\. Shleifer, P\. von Platen, C\. Ma, Y\. Jernite, J\. Plu, C\. Xu, T\. Le Scao, S\. Gugger, M\. Drame, Q\. Lhoest, and A\. M\. RushTransformers: state\-of\-the\-art natural language processing\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,Online,pp\. 38–45\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-demos.6),[Link](https://aclanthology.org/2020.emnlp-demos.6/)Cited by:[Appendix B](https://arxiv.org/html/2610.28992#A2.SS0.SSS0.Px4.p1.1)\.
- Xiaoet al\.\(2024\)S\. Xiao, Z\. Liu, P\. Zhang, N\. Muennighoff, D\. Lian, and J\. NieC\-Pack: packed resources for general Chinese embeddings\.InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 641–649\.External Links:[Document](https://dx.doi.org/10.1145/3626772.3657878),[Link](https://doi.org/10.1145/3626772.3657878)Cited by:[Appendix B](https://arxiv.org/html/2610.28992#A2.SS0.SSS0.Px3.p1.1),[Appendix C](https://arxiv.org/html/2610.28992#A3.SS0.SSS0.Px5.p1.1),[§4\.1](https://arxiv.org/html/2610.28992#S4.SS1.SSS0.Px1.p1.1)\.
- Yanget al\.\(2025\)S\. Yang, J\. Kautz, and A\. HatamizadehGated Delta Networks: improving Mamba2 with delta rule\.InThe Thirteenth International Conference on Learning Representations,pp\. 29687–29707\.External Links:[Link](https://arxiv.org/abs/2412.06464)Cited by:[Appendix B](https://arxiv.org/html/2610.28992#A2.SS0.SSS0.Px4.p1.1),[Appendix G](https://arxiv.org/html/2610.28992#A7.SS0.SSS0.Px8.p1.1)\.
- Yanget al\.\(2026\)Y\. Yang, X\. Guan, S\. Arheix\-Parras, S\. Ahmadi, R\. Newman\-Norlund, L\. Bonilha, C\. Rorden, J\. Fridriksson, R\. H\. Desai, and S\. NelakuditiLesioned multimodal language models reproduce aphasic picture\-naming patterns\.arXiv preprint arXiv:2607\.11621\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2607.11621),[Link](https://arxiv.org/abs/2607.11621)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px1.p2.1)\.
- Zhang and Nanda \(2024\)F\. Zhang and N\. NandaTowards best practices of activation patching in language models: metrics and methods\.InInternational Conference on Learning Representations \(ICLR\),pp\. 1651–1678\.External Links:[Link](https://arxiv.org/abs/2309.16042)Cited by:[Appendix G](https://arxiv.org/html/2610.28992#A7.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2026a\)H\. Zhang, Z\. Zhang, M\. Wang, Z\. Su, Y\. Wang, Q\. Wang, S\. Yuan, E\. Nie, X\. Duan, Q\. Xue, Z\. Yu, C\. Shang, X\. Liang, J\. Xiong, H\. Shen, C\. Tao, Z\. Liu, S\. Jin, Z\. Xi, D\. Zhang, S\. Ananiadou, T\. Gui, R\. Xie, H\. K\. So, H\. Schütze, X\. Huang, Q\. Zhang, and N\. WongLocate, steer, and improve: a practical survey of actionable mechanistic interpretability in large language models\.Computer Science Review62,pp\. 101011\.External Links:[Document](https://dx.doi.org/10.1016/j.cosrev.2026.101011),[Link](https://doi.org/10.1016/j.cosrev.2026.101011)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.
- Zhanget al\.\(2020\)T\. Zhang, V\. Kishore, F\. Wu, K\. Q\. Weinberger, and Y\. ArtziBERTScore: evaluating text generation with BERT\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/1904.09675)Cited by:[Appendix D](https://arxiv.org/html/2610.28992#A4.SS0.SSS0.Px6.p1.1)\.
- Zhanget al\.\(2026b\)W\. Zhang, Z\. Zhu, Z\. Yang, C\. Huang, W\. Lei, and S\. NgBeyond prompt: fine\-grained simulation of cognitively impaired standardized patients via stochastic steering\.InFindings of the Association for Computational Linguistics: ACL 2026,San Diego, California, United States,pp\. 28098–28131\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.1401),[Link](https://aclanthology.org/2026.findings-acl.1401/)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.
- Zouet al\.\(2023\)A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski, S\. Goel, N\. Li, M\. J\. Byun, Z\. Wang, A\. Mallen, S\. Basart, S\. Koyejo, D\. Song, M\. Fredrikson, J\. Z\. Kolter, and D\. HendrycksRepresentation engineering: a top\-down approach to AI transparency\.arXiv preprint arXiv:2310\.01405\.External Links:[Link](https://arxiv.org/abs/2310.01405)Cited by:[§2](https://arxiv.org/html/2610.28992#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix

## Appendix AAphasia Types

Aphasia is a language disorder acquired through brain damage, most often from a stroke\. Clinicians classify it by three properties of a speaker’s language: whether speech is fluent, whether the speaker understands what others say, and whether the speaker can repeat what others say\([Kertesz, 2007](https://arxiv.org/html/2610.28992#bib.bib20)\)\. AphasiaBank assigns each speaker a type based on the Aphasia Quotient subtests from the Western Aphasia Battery\-Revised and provides a severity score from 0 to 100, where lower values indicate more severe aphasia\([MacWhinney et al\., 2011](https://arxiv.org/html/2610.28992#bib.bib26);[Kertesz, 2007](https://arxiv.org/html/2610.28992#bib.bib20)\)\.

#### Broca’s aphasia\.

Speech is nonfluent and speakers produce short, effortful phrases that often drop function words and inflections, which is called agrammatism, while their comprehension is relatively preserved\.

#### Wernicke’s aphasia\.

Speech is fluent and may be abundant, but it carries little content and contains substituted words and sounds, called paraphasias, and speakers understand others poorly\.

#### Conduction aphasia\.

Speech is fluent and comprehension is relatively good, but speakers repeat poorly and produce sound errors that they often try to correct\.

#### Anomic aphasia\.

Speech is fluent and comprehension and repetition are good, but speakers have trouble finding words and talk around them\. Anomic aphasia is usually the mildest type\.

The classifier merges conduction and anomic aphasia into one class, because the two differ mainly in repetition, which free speech such as a story or a recipe does not test\.

## Appendix BData, Models and Setups

This appendix gives the details on the human transcripts, the models, the prompts, the decoding rules, the rules that decide whether an answer counts, the two tests beyond the narratives, the Text Aphasia Battery \(TAB\) and MMLU, and the compute and software that we used\.

Table 5:The 11 models, grouped by family, with the kind of model and the prompt format of each family\. In Qwen3\.5, every fourth layer, layers 3, 7, 11, and so on when counting from 0, uses softmax attention, and the other layers use Gated DeltaNet linear attention\. For Qwen3\.5,*Heads per layer*gives the heads of a softmax layer and of a Gated DeltaNet layer, and a lesion can knock out heads of both kinds\.*Softmax heads*counts the heads of the softmax layers, which in the other models are all of the heads\.#### Transcripts\.

We use 1,147 AphasiaBank transcripts\([MacWhinney et al\., 2011](https://arxiv.org/html/2610.28992#bib.bib26)\)of the Cinderella task and 1,437 of the sandwich task, and some of these transcripts share a text\. The sandwich transcripts hold 557 texts that appear twice and we keep only one copy of each\. Four Cinderella texts appear once as anomic and once as control speech, so their group is unknown, and we drop both copies\. Table[1](https://arxiv.org/html/2610.28992#S2.T1)counts the remaining transcripts, each text once: 1,139 for the Cinderella task and 880 for the sandwich task\. The transcripts carry no speaker field, so we identify a speaker by the corpus and by the file name without its session suffix\. The probes and the steering vectors use these transcripts, whereas the classifier draws its training and test splits from all 2,584 transcripts, duplicates included \(Appendix[C](https://arxiv.org/html/2610.28992#A3)\)\. Of the aphasic transcripts, 720 of 744 in the Cinderella task and 382 of 408 in the sandwich task come with the speaker’s aphasia quotient, the severity score that Appendix[H](https://arxiv.org/html/2610.28992#A8)compares with the projection of a transcript on the aphasia direction\.

#### License\.

We used AphasiaBank data under TalkBank’s Ground Rules and the applicable CC BY\-NC\-SA 3\.0 license\. We report aggregate results but do not redistribute participant transcripts, recordings, or other restricted data\. Researchers seeking the underlying data should request access directly from TalkBank\.

Figure 6:t\-SNE of each task’s transcripts and of the classifier’s model scrambles, in the text space of the classifier’s encoder, BGE\-large, without the classifier’s fine\-tuning\. The maps hold the 1,139 Cinderella and 880 sandwich transcripts of Table[1](https://arxiv.org/html/2610.28992#S2.T1), with 169 and 90 model scrambles\. Conduction and anomic transcripts share one color because they share one class of the classifier\.
#### Conduction and anomic transcripts lie among the control transcripts\.

We embed every transcript with BGE\-large\([Xiao et al\., 2024](https://arxiv.org/html/2610.28992#bib.bib58)\), the encoder that the classifier fine\-tunes, here without its fine\-tuning, so that the space knows nothing of our labels\. A transcript becomes the encoder’s \[CLS\] vector over its first 384 tokens, the classifier’s limit, scaled to length 1\. Figure[6](https://arxiv.org/html/2610.28992#A2.F6)maps these vectors with t\-SNE, one map per task, together with the classifier’s model scrambles \(Appendix[C](https://arxiv.org/html/2610.28992#A3)\)\. We measure the overlap in the full embedding rather than on the map: for each transcript, we take the share of control transcripts among its 10 nearest transcripts by cosine similarity\. On the Cinderella and the sandwich task, this share is 0\.89 and 0\.91 for control transcripts, 0\.44 and 0\.58 for anomic transcripts, and 0\.25 and 0\.41 for conduction transcripts, against 0\.07 and 0\.11 for Broca’s and 0\.07 and 0\.33 for Wernicke’s aphasia\. The model scrambles form islands of their own: 92% and 85% of a scramble’s 10 nearest texts are other scrambles, whereas scrambles make up 0\.1% of the nearest texts of the human transcripts\. Conduction and anomic transcripts thus resemble control speech more than Broca’s and Wernicke’s transcripts do, which is why we mainly report the Broca and Wernicke targets\.

#### Models\.

Table[5](https://arxiv.org/html/2610.28992#A2.T5)lists the 11 models and their Hugging Face identifiers\. We load every model with the Transformers library\([Wolf et al\., 2020](https://arxiv.org/html/2610.28992#bib.bib57)\)and run it in 32\-bit floating point\. The Qwen3\.5 models are the post\-trained checkpoints, not the base checkpoints\. In Qwen3\.5, every fourth layer uses full softmax attention, and the other layers use Gated DeltaNet linear attention\([Yang et al\., 2025](https://arxiv.org/html/2610.28992#bib.bib60)\), in which each head keeps its own recurrent state\. A lesion can knock out heads of both kinds, and softmax heads make up 14\.3% of the heads of every Qwen3\.5 model\. Knocking out a head zeroes its slice of the input to the attention output projection of its layer, at every position of the prompt and of the answer; in the softmax layers of Qwen3\.5, this slice holds the head’s output after its gate\.

Table 6:The prompts, verbatim\. Each prompt is the template of the model’s family with one of the three requests in place ofrequest, and\\nmarks a newline\. The prompts of GPT\-2 XL and of the Llama\-2 models start with a beginning\-of\-sequence token,<\|endoftext\|\>or<s\>, which the templates leave out, and the prompts of Pythia and Qwen3\.5 start with no such token\. Neither chat template adds a system prompt, and the Qwen3\.5 template, with thinking turned off, closes an empty thinking block, so that the answer starts at once\.
#### Prompts\.

Table[6](https://arxiv.org/html/2610.28992#A2.T6)gives the prompt template of each family and the three requests, verbatim\. The base models, GPT\-2 XL and Pythia, receive each request in a plain question\-and\-answer format, and the chat models receive it as a user message in their chat template, with no system prompt and, for Qwen3\.5, with thinking turned off\. The prompts of GPT\-2 XL and of the Llama\-2 models start with the beginning\-of\-sequence token, the prompts of Pythia start without it, and the Qwen3\.5 tokenizer has no such token\. On the Cinderella task, the yes\-or\-no question and the story request are two separate prompts rather than two turns of one dialogue, and a lesion stays in place for both answers\.

#### Decoding\.

For the three prompts, every model decodes greedily, with the repetition penalty set to 1\.25 and applied to the raw logits\. At each step, the penalty divides the logit of every token that already occurs in the prompt or in the answer by 1\.25 if the logit is positive, and multiplies it by 1\.25 if the logit is negative\. Without the penalty, greedy decoding makes the intact base modelsloopon the Cinderella task: the classifier gives the answers of GPT\-2 XL, Pythia\-1B, Pythia\-1\.4B, and Pythia\-2\.8B a probability of up to 99% of being model scrambles\. For the Llama\-2 models, greedy decoding replaces the sampling at temperature 0\.6 and top\-p 0\.9 that their released generation settings specify\. A model writes at most 180 new tokens for the story and the sandwich answer, and at most 12 for the answer to the yes\-or\-no question, and it stops early at its end\-of\-sequence token or, for Qwen3\.5, also at<\|endoftext\|\>\. In the lesion searches, each model generates in batches of a fixed size, and we pad a short batch with copies of its first lesion, so that an answer does not depend on how the lesions were batched\. Greedy decoding thus gives each lesion a single answer, and regenerating everyCLAMPandHead\-grown\-coslesion from scratch reproduces its stored answer word for word\.

#### What counts as an answer\.

A base model often goes on to write further questions and answers of its own, so we cut its answer before the first new line that starts withQuestion:,Q:, orquestion:, and the answers of the chat models stay whole\. An answer counts only if it has at least 20 characters, not counting invisible Unicode format characters, and a lesion whose answer does not count fails\. The rule leaves out format characters because some lesions of Llama\-2\-13B\-chat make the model write a few letters followed by a long run of zero\-width joiners, which are invisible: with its 170 joiners counted, the answer “tOoOO” would pass, and the classifier scores it atP⁡\(Broca\)=0\.70P\(\\mathrm\{Broca\}\)=0\.70\. In the lesion experiments, the model first answers the yes\-or\-no question of the Cinderella task, and a parser matches the start of this answer, ignoring case and up to a word boundary, against two lists of prefixes\. The answer means yes when it starts withyes,yeah,yep,yup,sure,of course,certainly,absolutely,definitely,i do remember,i remember, ori do, and no when it starts withno,nope,nah,sorry,i don't,i dont,i do not,i cannot,i can't,i cant,not really, orunfortunately\. When both lists match, the longer match wins, and yes wins a tie\. A lesion fails when the model answers no, and an answer that matches neither list passes\. The check is off for Pythia\-6\.9B, whose intact model already answers “No\. / Q: You don’t remember it, do”, where the slash marks a line break\. The lesion searches apply the check to every other model, and the steering experiments do not apply it\. The classifier scores the story or the sandwich answer after the cut, and never the answer to the yes\-or\-no question\.

Table 7:The items of the Text Aphasia Battery \(TAB\) on which we test every lesion, transcribed from the TAB paper\([Roll et al\., 2026b](https://arxiv.org/html/2610.28992#bib.bib40)\): all five connected\-text prompts and one of the five items of each other subtest, verbatim\. Connected\-text prompts follow the TAB’s instructionRespond to the following prompt in 3\-5 full sentences:, and the other items follow the instructionYou are having a conversation\. Answer the question directly and briefly\.The five\-way classifier and the LLM judge score the connected\-text answers, and rules compare the other answers with their targets\.
#### Text Aphasia Battery\.

We transcribe 20 items of the TAB from the appendix of[Roll et al\. \(2026b\)](https://arxiv.org/html/2610.28992#bib.bib40), five for each of four subtests, and Table[7](https://arxiv.org/html/2610.28992#A2.T7)shows all five connected\-text prompts and one item of each other subtest\. The connected\-text items carry the TAB’s own system instruction\. For the other three subtests, the TAB specifies only a conversational system prompt, so we use the minimal instruction given in Table[7](https://arxiv.org/html/2610.28992#A2.T7)\. A base model receivesQuestion: <instruction\> <prompt\>\\nAnswer:, and we cut its answer as for the narratives, whereas a chat model receives the instruction as a system message and the prompt as a user message in its chat template\. The model answers a connected\-text item as it answers the narratives, with the repetition penalty, in at most 120 new tokens\. It answers a short\-answer item greedily without the penalty, in at most 40 new tokens, because the penalty would push the model away from the very words that repetition asks it to repeat and that word comprehension asks it to name\. Rules score the 15 short\-answer items\. In word comprehension, the answer is correct when the first of the six options that it names, matched as a whole word regardless of case, is the target, which the prompt lists first\. In sentence comprehension, the first whole word*yes*or*no*in the answer must equal the target\. In repetition, the answer must equal the target after both are lowercased, stripped of surrounding quotation marks, spaces, and final periods and exclamation marks, and have their whitespace collapsed\. The exact match is strict for chat models, which often wrap the repetition in words of their own: the intact Llama\-2 models repeat none of the five items exactly, and Llama\-2\-13B\-chat answers “Sure\! Breakfast\.” when asked to repeat*breakfast*\. The items lost by a lesion are the intact model’s number of correct short\-answer items minus the lesioned model’s\. The five\-way classifier scores each connected\-text answer that has at least 20 characters, and an LLM judge, DeepSeek\-V4\.1\-Flash\([DeepSeek\-AI, 2026](https://arxiv.org/html/2610.28992#bib.bib12)\)at temperature 0, marks which of the TAB’s 19 aphasia features each connected\-text answer shows, following the TAB’s protocol with our own in\-context examples \(Appendix[K](https://arxiv.org/html/2610.28992#A11)\)\. The five connected\-text prompts are also the five TAB prompts of the human evaluation of steering \(§[4\.2](https://arxiv.org/html/2610.28992#S4.SS2)\)\.

#### MMLU\.

We also test every lesion on two subjects of MMLU\([Hendrycks et al\., 2021](https://arxiv.org/html/2610.28992#bib.bib17)\): global facts, with 100 test questions, and high school world history, with 237\. Every model receives the same raw\-text prompt, without a chat template, after the beginning\-of\-sequence token where the model uses one\. The prompt is the standard five\-shot prompt: the headerThe following are multiple choice questions \(with answers\) about <subject\>\.\\n\\n, where<subject\>isglobal factsorhigh school world history, then five solved questions from the subject’s development split, and last the test question\. Each question has the form<question\>\\nA\. <a\>\\nB\. <b\>\\nC\. <c\>\\nD\. <d\>\\nAnswer:, and a solved question continues with a space, its correct letter, and two newlines\. The context of GPT\-2 XL holds only 1,024 tokens, so for its world\-history prompts we drop the earliest solved questions until the prompt fits, which leaves three solved questions for 106 test questions, two for 111, and one for 20; every other model keeps all five for both subjects\. The model reads the prompt once and generates nothing\. Its predicted answer is the letter, among the tokens of A to D, with the highest logit after the finalAnswer:, and the log\-probability of the correct letter is taken over the full vocabulary, so that it also falls when a lesion moves probability away from the four letters altogether\. The MMLU drop of a lesion is the intact model’s mean log\-probability of the correct letter minus the lesioned model’s, in nats, averaged over the two subjects\.

#### Compute\.

We ran the GPU and CPU\-based experiments on a shared cluster\. The GPUs ran for 236 hours: 96\.7 on NVIDIA A100 GPUs with 80 GB, 95\.6 on H200 GPUs with 141 GB, 31\.1 on A100 GPUs with 40 GB, and 12\.8 on older GPUs\. The runs that completed account for 145\.6 of these hours\. CPU jobs ran about 2,200 allocated core\-hours\. These counts leave out the LLM judge, which we query through an API \(Appendix[K](https://arxiv.org/html/2610.28992#A11)\)\.

#### Software\.

The code runs on Python 3\.13, with PyTorch 2\.7\.1 and Transformers 5\.5\.4, which loads every model and generates every answer\. The classifier uses PEFT 0\.19\.1 for its LoRA adapters, and Sentence\-Transformers 5\.4\.1 for its similarity features, which cosine similarity methods use too\.Paramscores answers with the BLEU of NLTK 3\.9\.4 and with bert\-score 0\.3\.13\. The probes, cross\-validation, t\-SNE, and nearest neighbors use scikit\-learn 1\.8\.0, the statistics and figures use SciPy 1\.17\.1, NumPy 2\.4\.4, and Matplotlib 3\.10\.9\.

## Appendix CFive\-Way Classifier

We next describe the training data, features, architecture, training, and evaluation of the five\-way classifier of §[3](https://arxiv.org/html/2610.28992#S3), and how it scores the answers of intact models\.

#### Training data\.

The classifier learns from 2,843 texts: 2,584 AphasiaBank transcripts of the Cinderella and sandwich tasks, and 259 model scrambles \(Table[8](https://arxiv.org/html/2610.28992#A3.T8)\)\. The transcripts carry six group labels, and the classifier merges two of them, anomic and conduction aphasia, into one class \(Appendix[A](https://arxiv.org/html/2610.28992#A1)\)\. Some texts appear twice among the transcripts, and the training data keep both copies\. In the sandwich task, 557 texts appear twice: for 492 of them, the two copies are two files of the same recording, and for the other 65, they are files of two different recordings with the same group label\. In the Cinderella task, four recordings appear twice with the same text, once labeled anomic aphasia and once labeled control speech\. The 2,584 transcripts therefore hold 2,023 distinct texts\. §[3](https://arxiv.org/html/2610.28992#S3)counts each text once and leaves out the four texts with conflicting labels, which gives 1,139 Cinderella and 880 sandwich transcripts\. No text occurs in both tasks, and no model scramble repeats a human transcript\.

Table 8:Training texts of the five\-way classifier, by class and task prompt\. The AphasiaBank transcripts carry six group labels, and the classifier merges anomic and conduction aphasia into one class\. The model scrambles are answers of Llama\-2\-7B\-chat and GPT\-2 XL from a preliminary knockout study\.*Distinct texts*counts each text once, because 557 sandwich texts and 4 Cinderella texts appear twice among the transcripts\.Table 9:The 14 features that the five\-way classifier reads besides its text encoder, each standardized with the mean and standard deviation of the training split\. A repeated unit is a sequence of 1 to 100 tokens, words or punctuation marks, that occurs several times in a row, or a sequence of six or more tokens that occurs again anywhere in the text\. Similarities are mean cosine similarities between the all\-MiniLM\-L6\-v2 embedding of the text and those of the human transcripts of a group, from both tasks\. Because of a label mismatch in the code, the last feature subtracts zero instead of the similarity to model scrambles\.
#### Model scrambles\.

The 259 model scrambles are answers from a preliminary knockout study: 203 come from Llama\-2\-7B\-chat, on both task prompts, and 56 from GPT\-2 XL, all on the Cinderella prompt\. All but five are answers of models with knocked\-out attention heads or feed\-forward blocks, and the other five are temperature samples of the intact GPT\-2 XL\. The knockouts remove single heads, sets of heads, or the feed\-forward block of a whole layer, and 132 of these answers come from successive steps of a greedy search that knocks out more and more heads of Llama\-2\-7B\-chat\. Of the answers with knockouts, 28 come from activation patching, which restores the clean activations of one component in a knocked\-out model, and 11 are temperature samples of Llama\-2\-7B\-chat with three heads knocked out\. An LLM judge flagged these answers as machine failures among the outputs of the preliminary study, and a linguist and a clinician reviewed them\. The model scrambles are 12 to 1,090 characters long, with a median of 571, and all 259 are distinct\. By the repetition statistics of Table[9](https://arxiv.org/html/2610.28992#A3.T9), 233 of them contain a repeated unit and 196 a unit that occurs three or more times, whereas 799 of the 1,335 control transcripts contain no repeated unit\.

#### Judging machine\-specific output\.

To distinguish machine\-specific output failures from linguistically comparable clinical aphasic speech, we implemented an LLM judge pipeline to automatically detect and annotate machine artifacts\. Using the Gemini 3\.5 Flash model, this automated judge evaluates outputs from perturbed models to filter out “mechanical garbage”, which are instances where the model breaks down mechanically rather than simulating a valid human linguistic profile\. By identifying these “model scrambles,” we prevent machine\-generated noise from contaminating the mapping between human clinical profiles and internal model components during the ablation pipeline\.

The judge’s evaluation is governed by a strict, zero\-temperature system prompt that evaluates text against five specific artifact dimensions: Stray Brackets \(SB\), Token Repetition \(TR\), Tokenizer Loops \(TL\), Formulaic AI\-isms \(FM\), and Sentence Repetition \(SR\)\. To ensure precise categorization, the model outputs a structured JSON object detailing its step\-by\-step reasoning and the specific artifacts detected\. Crucially, the prompt includes a tie\-breaker rule designed specifically for clinical data: if a repetition mimics natural human hesitation or spoken disfluency, the LLM assumes a valid human purpose and refrains from flagging it as a machine artifact\. This ensures that the valid disfluencies characteristic of aphasic subtypes are preserved while machine\-specific degradations are correctly recognized\. For the complete zero\-temperature system prompt, including the explicit artifact rubric and few\-shot evaluation examples\. Listing[1](https://arxiv.org/html/2610.28992#LST1)shows the whole prompt\.

To ensure the reliability of the LLM judge pipeline, the system prompt was iteratively calibrated against a gold\-label subset of the dataset to maximize its baseline agreement rate\. Furthermore, to validate the integrity of the automated filtering, the final classifications were audited by human domain experts\. A linguist and a clinician manually reviewed the text flagged by the LLM, confirming that the isolated “model artifacts” were true machine\-specific failures and not misclassified clinical disfluencies\. This empirical prompt tuning combined with expert clinical verification ensures that our artifact removal process is both scalable and clinically accurate\.

Finally, these verified “model artifacts” are incorporated directly into the training data for our 5\-way clinical diagnostic classifier\.

#### Features\.

Besides its text encoder, the classifier reads the 14 features of Table[9](https://arxiv.org/html/2610.28992#A3.T9)\. The three narrative measures reimplement measures that the Computerized Language Analysis \(CLAN\) program computes for AphasiaBank transcripts\([MacWhinney, 2018](https://arxiv.org/html/2610.28992#bib.bib27)\), so that the classifier can compute them from plain text such as a model’s answer\. On the human transcripts, they correlate with CLAN’s values at a Pearson’srrof 0\.986 for the number of utterances, 0\.993 for the number of word tokens, and 0\.968 for the mean length of utterance\. The five word\-list proxies need no part\-of\-speech tagger, but they approximate CLAN’s measures less closely\. The open\- to closed\-class ratio and the share of open\-class words correlate only weakly with the corresponding CLAN measures, atr=0\.095r=0\.095andr=−0\.050r=\-0\.050\. The count of words ending in*\-ly*correlates with CLAN’s adverb count atr=0\.618r=0\.618, and 73\.2% of its matches in the sandwich transcripts are the word*jelly*\. The count of fragments and stutters is zero on every human transcript, although CLAN counts word errors in 1,090 of them\. The four similarity features embed the whole text with all\-MiniLM\-L6\-v2\([Wang et al\., 2020](https://arxiv.org/html/2610.28992#bib.bib52);[Reimers and Gurevych, 2019](https://arxiv.org/html/2610.28992#bib.bib37)\)and compare it, by mean cosine similarity, with the human transcripts of each group, pooled over both tasks\. Three of them subtract the mean similarity to the transcripts of Broca’s aphasia, of Wernicke’s aphasia, or of conduction or anomic aphasia from the mean similarity to the control transcripts\. The fourth was meant to subtract the mean similarity to the model scrambles in the same way\. The code, however, looks up the model scrambles under a label that differs from their label in the data by one character, finds none, and subtracts zero, so the fourth feature is the mean similarity to the control transcripts itself\. Training and scoring share this mismatch, so the classifier reads the same feature in both, but the feature is not a contrast with model scrambles\. Within the training data, each transcript is left out of its own group’s mean\. MiniLM reads at most 256 tokens, so the similarity features see only the beginning of 820 of the 2,843 training texts, 719 of them Cinderella transcripts\.

#### Architecture\.

The classifier encodes a text with BGE\-large\-en\-v1\.5\([Xiao et al\., 2024](https://arxiv.org/html/2610.28992#bib.bib58)\), an encoder with 24 layers and 1,024\-dimensional states, which reads at most 384 tokens\. The encoder thus sees only the beginning of 561 of the 2,843 training texts: 528 of the 1,147 Cinderella transcripts, 25 of the 1,437 sandwich transcripts, and 8 of the 259 model scrambles\. We fine\-tune the encoder with low\-rank adapters\([Hu et al\., 2022](https://arxiv.org/html/2610.28992#bib.bib19), LoRA;\)of rank 16, withα=32\\alpha=32and a dropout of 0\.1, on all six linear layers of each of its 24 blocks: the query, key, value, and output projections of attention, and the two layers of the feed\-forward block\. The encoder’s own weights stay frozen\. The embedding of a text is the mean of the encoder’s last\-layer states over the text’s tokens, normalized to unit length\. A two\-layer perceptron takes this 1,024\-dimensional embedding together with the 14 standardized features, 1,038 inputs in all, maps them to 256 hidden units with a rectified linear activation and a dropout of 0\.3, and then maps these to the logits of the five classes\.

#### Training\.

We train the adapters and the perceptron jointly with the cross\-entropy loss over the five classes, using AdamW\([Loshchilov and Hutter, 2019](https://arxiv.org/html/2610.28992#bib.bib24)\)with a constant learning rate of10−410^\{\-4\}, a weight decay of 0\.01, and gradient norms clipped at 1\.0\. Each epoch draws as many texts as the training split holds, in batches of four\. It draws them with replacement, with probabilities inversely proportional to the size of each text’s class in the training split, so that the five classes are equally frequent in expectation\. After each epoch, we compute the macro\-averaged F1 over the five classes on the validation split\. We keep the epoch with the highest value, and we stop after five epochs without an improvement larger than10−410^\{\-4\}, or after 20 epochs\. We train 20 such models, each with its own random seed and its own random split of the 2,843 texts into 70% training, 10% validation, and 20% test texts\. Each text remains intact and belongs to only one set within a split\. Each split is stratified by the six group labels, so that anomic and conduction aphasia are split separately, and has 1,990 training, 284 validation, and 569 test texts\. The splits are random over transcripts, not over speakers or distinct texts\. To score new text, the classifier averages the softmax outputs of the 20 models, andP⁡\(aphasia\)P\(\\mathrm\{aphasia\}\)is the sum of the averaged probabilities of the three aphasia classes\.

#### Evaluation\.

A text can fall in the test splits of several models, so we score each text by the mean softmax output of the models whose test split holds it, and we take its most probable class\. Of the 2,843 texts, 2,798 fall in at least one test split, in 4\.07 test splits on average and in at most 11; the other 45, four of them model scrambles, never do and are not evaluated\. These held\-out scores thus come from subsets of the 20 models rather than from the full ensemble that scores model answers\. Table[10](https://arxiv.org/html/2610.28992#A3.T10)gives precision, recall, and F1 per class, and Table[11](https://arxiv.org/html/2610.28992#A3.T11)gives the confusion matrix\. The classifier predicts none of the 1,317 held\-out control transcripts as Broca’s aphasia, Wernicke’s aphasia, or model scrambles, and 25 of them as conduction or anomic aphasia\. It recognizes 254 of the 255 model scrambles, and of the 2,543 human transcripts, it calls only two model scrambles, both Broca transcripts of the Cinderella task\. Most errors involve conduction or anomic aphasia: the classifier predicts 78 of its 737 transcripts as control speech and 75 as Broca’s aphasia, and it predicts 58 of the 397 Broca transcripts and 35 of the 92 Wernicke transcripts as conduction or anomic aphasia\. Wernicke’s aphasia has the lowest F1, 0\.512\. By the aphasia target of §[3](https://arxiv.org/html/2610.28992#S3),P⁡\(aphasia\)\>0\.5P\(\\mathrm\{aphasia\}\)\>0\.5, the classifier separates the 1,226 aphasic from the 1,317 control transcripts with a precision of 0\.979, a recall of 0\.935, and an accuracy of 0\.959\. Single models reach a mean accuracy of 0\.865 and a mean macro\-averaged F1 of 0\.789 on their own test splits\.

Table 10:Precision, recall, and F1 of the five\-way classifier on held\-out texts, by class\. Each of 20 models holds out a random 20% of the 2,843 training texts, and each held\-out text is scored by the mean output of the models that held it out\. The lower block keeps the held\-out texts that occur only once in the training data, so that no copy of them can be in a training split\. Cond\. or anomic: conduction or anomic aphasia; N: number of texts\.Table 11:Confusion matrix of the five\-way classifier on the 2,798 held\-out texts, with true classes as rows and predicted classes as columns; bold counts are correct predictions\. Each held\-out text is scored by the mean output of the models, of 20, whose random test split holds it\. Cond\. or anomic: conduction or anomic aphasia; Wern\.: Wernicke’s aphasia; MS: model scrambles\.
#### Overlap between training and test splits\.

Because the splits are random over transcripts, a test transcript can share its text or its speaker with the training split\. In each model, 146 to 170 of the 569 test texts have an identical copy in the training split\. On the 1,693 held\-out texts that occur only once in the training data, so that no training split holds a copy, the accuracy falls from 0\.886 to 0\.833 and the macro\-averaged F1 from 0\.813 to 0\.783 \(Table[10](https://arxiv.org/html/2610.28992#A3.T10)\)\. Part of the change in precision comes from the class mix, because most duplicates are sandwich control transcripts\. Recall, which does not depend on the class mix, also falls for every class of human speech, most for Wernicke’s aphasia, from 0\.457 to 0\.380\. AphasiaBank also records most speakers more than once, in both tasks and in repeated sessions\. When we identify a speaker by the corpus and the participant code of each file, 446 to 475 of the 517 human test transcripts of each model share their speaker with a transcript in that model’s training split, 89\.2% over all 20 models\. Splits by speaker would remove this overlap and measure accuracy on new speakers\.

Listing 1:User message of the artifact detection prompt, used to filter out mechanical model breakdowns\. We define five dimensions of machine\-specific artifacts and provide four examples demonstrating the expected reasoning and JSON output format\.Youareanexpertlinguisticannotatorevaluatingtranscriptsgeneratedbyanintentionallyimpaired,lesionedlanguagemodel\.Themodelwasdesignedtosimulatehumanclinicaldisfluencies\(likeaphasiaorword\-findingpauses\)\.

Yourtaskistofilteroutthemechanicalgarbage\.Youmustdetectthepresenceof'machine\-specificartifacts'—instanceswherethemodelfailedtoproducevalidlinguisticoutputandinsteadbrokedownmechanically\(e\.g\.,tokenizerloops,straybrackets,orcannedAIsafetyrefusals\)\.

Evaluatethetextagainstthefollowing5dimensions\.IfANYofthesedimensionsarepresent,thetextcontainsanartifact\.

RUBRIC:

1\.SB\(StrayBrackets\):Unparsed,artifactualbracketsordelimiters\(e\.g\.,\[\[\.\.\.\]\],<<,<\|endoftext\|\>\),oremojis\.DoNOTflagstandardcitations\[1\]ornormalmarkdown\.

2\.TR\(Tokenrepetition\):Unnaturalconsecutiverepetitionofatoken,symbol,orpunctuationmark\.

\-StandardEllipsesExemption:Astandard3\-dotellipsis\("\.\.\."\)isnormalhumanpunctuationortranscribernotation\.DoNOTflagit\.

\-ArtifactThreshold:4ormoreconsecutiveperiods\(e\.g\.,"\.\.\.\.","\.\.\.\.\.\.\.\."\)orrepeatedcommas\(",,,"\)isamachinedegenerationandMUSTbeflagged\.

3\.TL\(TokenizerLoops\):Degeneratesub\-wordormorphologicallooping\(e\.g\.,"tiontiontion"or"thethethethethe"\)\.

4\.FM\(FormulaicAI\-isms\):StandardizedAIguardrailmessages,apologies,orassistant\-likeframingthatcompletelybreakthenarrativetask\.Lookforsafetyrefusals\("Iapologize,butIamnotableto\.\.\."\),ethicaldisclaimers,orpersonaleaks\("AsanAI\.\.\."\)\.

5\.SR\(SentenceRepetition\):Exactornear\-exactrepetitionofawell\-formedsentenceorindependentclausewithinthetext\.

TIE\-BREAKERRULE:

Ifarepetition\(TRorSR\)appearstoserveavalidhumanpragmatic,emotional,orstylisticpurpose\(e\.g\.,stutteringinspokentranscripts,emphasizingapoint\),assumeitishumanandDONOTflagit\.

INSTRUCTIONS:

1\.Analyzethetextstep\-by\-stepagainstall5dimensions\.

2\.OutputaJSONobjectcontainingyourstep\-by\-step"reasoning",aboolean"contains\_artifact",andalistof"artifact\_types"found\(usetheabbreviationsSB,TR,TL,FM,SR\)\.Ifnone,returnanemptylist\[\]\.

EXAMPLES:

Text:"Sure,herearesomeexamplesofhowtheCubsarecalculated\.Ingeneral,theCisacomplex,b,c,anddarethesame\.TheCisa,b,a,C\.The,a,b,C\.The,a,aC,a,a,aC,a,aC,a,aC,a,a,a,a,a,\[C,a,a\[\[\[\[\[\[\[\[\[\[the,C,a,a,a,a,a,a,a,a,C,\[\[\[\[\[\[\[\[\[\[\[,a,a,a,a,C,\[\[\[\[\[\[\[,a,a,a,C,\[\[\[\[,a,a,\[\[\[\[\[,a,\[\[\[\[,a,\[\[\[,a,\[\[\[,\[,a,a,\[,\[\[\[\[\[,\[\[\["

Output:

\{

"reasoning":"Thetextcontainsagreatnumberofunparsedbrackets'\['whichflagsasSB\.",

"contains\_artifact":true,

"artifact\_types":\["SB"\]

\}

Text:"Iapologize,butI'mnotabletounderstandorproducesoundorcreateanycontentthatmayoffendorbeinappropriateoroffensive\.Pleaseletmeknowifyouhaveanyquestionsorconcerns\.Iapologize,butI'mnotabletounderstandorproduceanycontentthatmayoffendorbeinappropriateoroffensive\.Pleaseletmeknowifyouhaveanyquestionsorconcerns\."

Output:

\{

"reasoning":"ThetextcontainsaclassicAIdisclaimer\('butI'mnotabletounderstandorproducesoundorcreateanycontentthatmayoffendorbeinappropriateoroffensive'\),whichflagsasFM\.Italsocontainsthedisclaimertwice,whichflagsasSR\.",

"contains\_artifact":true,

"artifact\_types":\["FM","SR"\]

\}

Text:"Iwenttothestoreand,well,IIIjustcouldn'trememberwhattobuy\."

Output:

\{

"reasoning":"Thetoken'I'isrepeatedthreetimes\.However,applyingthetie\-breakerrule,thismimicsnaturalhumanhesitationandspokendisfluency,notamachinetokenizerloop\.Nootherartifactsarepresent\.",

"contains\_artifact":false,

"artifact\_types":\[\]

\}

Text:"Cinderellaisafairytaleaboutayounggirlwhoisbornintoapoorfamily\.Sheisverypoor,andherfatherisverycrueltoher\.Sheisverypoor,andherfatherisverycrueltoher\.Sheisverypoor,andherfatherisverycrueltoher\.Sheisverypoor,andherfatherisverycrueltoher\.Sheisverypoor,andherfatherisverycrueltoher\.Sheisverypoor,andherfatherisverycrueltoher\.Sheisverypoor,andherfatherisverycrueltoher\.Sheisverypoor,andherfatherisverycrueltoher\.Sheisverypoor,andherfatherisverycrueltoher\.Sheisverypoor,andherfatherisverycrueltoher\.Sheisverypoor,andherfatherisverycrueltoher\.Sheisverypoor,andherfatheris"

Output:

\{

"reasoning":"Theexactsentence'Sheisverypoor,andherfatherisverycrueltoher\.'isrepeatedabout12timesconsecutivelywithoutanypragmatichumanjustification\.Thisisadegenerategenerationloop\.",

"contains\_artifact":true,

"artifact\_types":\["SR"\]

\}

Evaluatethistext:

Text:"\{transcript\}"

Output:

## Appendix DLesion Methods

We describeCLAMPand the baselines of Table[2](https://arxiv.org/html/2610.28992#S5.T2)in detail, adds the lesion spaces and searches that the main text leaves out, and compares the answers that the lesions of each method produce\. A lesion*works*when the lesioned model’s answer counts, by the rules of Section[3](https://arxiv.org/html/2610.28992#S3), and reaches the target: the classifier either gives the target a probability above 0\.5 or ranks the target as its most probable class, where any of the three aphasia classes counts for the aphasia target\. Every search that consults the classifier uses only the first of these two tests, a target probability above 0\.5, to decide when to stop and what to keep or prune, and the second test judges only the answer that a search finally reports\. A search that never passes 0\.5 can thus still report a lesion that works, as 3 of the 54 lesions ofCLAMPdo\. The comparisons with the baselines count the 42 settings of the seven older models, GPT\-2 XL, the two Llama\-2\-chat models, and the four Pythia models, as Table[2](https://arxiv.org/html/2610.28992#S5.T2)does\. They support Section[5\.1](https://arxiv.org/html/2610.28992#S5.SS1)in three ways: searches guided by the classifier find far more lesions than searches guided by corpus statistics, heads give fewer degenerate answers than weight lesions, and the lesions of every method, not only those ofCLAMP, rarely carry their target to new prompts\.

#### A knockout zeroes a head’s output at every position\.

CLAMPknocks out a head by zeroing the head’s slice of the input to the attention output projection, at every position of the prompt and of the answer\. In GPT\-2 XL and the Pythia models, the value projection has a biasbVb\_\{V\}, so the knockout also removes the termWO​bVW\_\{O\}b\_\{V\}that the head adds at every position, whereWOW\_\{O\}is the head’s slice of the output projection\. In the softmax\-attention layers of Qwen3\.5, a gate scales the output of each head before the output projection, and the knockout removes the gated output\. In the Gated DeltaNet layers, each value head keeps its own recurrent state, and the knockout zeroes the head’s slice of the input to the layer’s output projection\. Both kinds of head count as heads, so Qwen3\.5\-0\.8B and Qwen3\.5\-2B have 336 heads and Qwen3\.5\-4B and Qwen3\.5\-9B have 896, of which 14\.3% sit in softmax layers\. The older models have from 128 heads, in Pythia\-1B, to 1,600, in Llama\-2\-13B\-chat\.

Table 12:Heads that tie in the single\-head ranking ofCLAMP: the number of heads whose knockout alone leaves the target probability within10−610^\{\-6\}of the intact model’s value, by model, task, and target\. The models have, from top to bottom, 1,600, 1,024, 1,200, 1,024, and 128 heads\. Tied heads keep their layer order in the ranking, so a growth that reaches them adds them from the first layer upward\.
#### Many heads change nothing when knocked out alone, so they tie in the ranking\.

For each model, task, and target,CLAMPfirst knocks out every head alone and recordss⁡\(\{h\}\)s\(\\\{h\\\}\), the target probability of the answer\. A head whose answer does not count ranks last\. No head of the seven older models gives such an answer, and only one head of Qwen3\.5\-0\.8B does, on Cinderella\. The three targets of a model and task share one cache of answers, and the seven older models reuse the single\-head answers of the first stage of the layer search, another classifier\-guided search in Table[13](https://arxiv.org/html/2610.28992#A4.T13)\.CLAMPranks the heads bys⁡\(\{h\}\)s\(\\\{h\\\}\), from highest to lowest, with a stable sort, so heads with equal values keep their order by layer and, within a layer, by index\. Ties are common in the larger older models \(Table[12](https://arxiv.org/html/2610.28992#A4.T12)\)\. In Llama\-2\-13B\-chat, for example, 1,305 of the 1,600 single\-head knockouts leave the Cinderella answer word for word unchanged\. The answers of Qwen3\.5 change far more often: a single\-head knockout leaves the answer unchanged for 1 of the 336 heads of Qwen3\.5\-2B on Cinderella and for 2 on sandwich, and for 64 of the 896 heads of Qwen3\.5\-9B on Cinderella\. The order of tied heads thus comes from where they sit in the model, not from what they do\.

#### Growth adds heads in ranking order, so ties can make a grown set large\.

CLAMPthen knocks out the topkkheads of the ranking together, fork=1,2,3,…k=1,2,3,\\dotsc, so that each prefix adds one head to the previous one\. It generates the prefixes in blocks of the model’s batch size, 25 prefixes for GPT\-2 XL, 40 for Llama\-2\-13B\-chat, and 32 for the other models, and it stops after the first block that contains a prefix whose target probability exceeds 0\.5\. The grown set is the shortest such prefix\. When no prefix exceeds 0\.5, up to the prefix of all heads, the growth reports the prefix with the highest target probability, the shortest one among ties, and the search ends without pruning\. Of the 15 growths that never pass 0\.5, three still report a lesion that works, because the target is the classifier’s most probable class: the Pythia\-1\.4B sandwich Broca prefix of 316 heads, atP⁡\(Broca\)=0\.488P\(\\mathrm\{Broca\}\)=0\.488, and the Qwen3\.5\-2B and Qwen3\.5\-4B sandwich Wernicke prefixes of 63 and 228 heads, atP⁡\(Wernicke\)P\(\\mathrm\{Wernicke\}\)of 0\.463 and 0\.423\. Because tied heads keep their layer order, a growth that runs past the heads that raise the target probability continues through the tied heads from the first layer upward\. The grown set of Llama\-2\-7B\-chat for the Wernicke target on Cinderella, for example, holds 740 heads, more than 560 of them tied, and the grown set of GPT\-2 XL for the Broca target on sandwich holds 984 heads, more than 400 of them tied\. Shrinking and pruning reduce these two sets to 276 and 146 heads, but the layer positions of large lesions partly reflect the order of the tied heads rather than only the roles of the heads\.

#### Pruning removes heads until none can go alone\.

A grown set of more than 64 heads is first shrunk in chunks\.CLAMPcuts the set, in ranking order, into consecutive chunks of half its size and tries the removal of each chunk in one batch\. Among the removals that keep the target probability above 0\.5, it takes the one that leaves the highest target probability and cuts the remaining set again\. When no chunk can go, it halves the chunk size, and it stops before chunks of a single head\. The weakest heads cannot simply be cut from the end of the set, because every shorter prefix of the ranking already stayed at or below 0\.5, so a chunk may come from anywhere in the ranking\. Pruning then works in rounds\. Each round tries every leave\-one\-out set, the set without one of its heads, at a cost of one answer per head\. The heads whose removal alone keeps the target probability above 0\.5 are ranked by the target probability that remains after their removal\. The round removes all of them at once if the probability stays above 0\.5, and otherwise the best half, the best quarter, and so on down to the single best head\. Each joint removal costs one more answer to check, because two heads that can each go alone may not be able to go together\. Pruning ends when no head can go alone, so that leaving any one head out of the pruned lesion brings its target probability to 0\.5 or below\.

#### Seven lesions are not fully pruned, but nearly all of their heads are needed\.

A pruning round starts only if the answers that pruning has used so far, plus one answer per head of the set, stay within a budget of 15,000 answers\. Otherwise,CLAMPreports the set as it stands and flags it as not minimal\. The budget counts the answers of pruning only, not those of growth or shrinking\. Four of the 54 working lesions stopped at the budget: GPT\-2 XL Cinderella Wernicke with 244 heads, Llama\-2\-7B\-chat Cinderella Wernicke with 276, Pythia\-6\.9B Cinderella Broca with 578, and Qwen3\.5\-4B Cinderella aphasia with 208\. The three lesions that reach their target only as the most probable class were never pruned, because their growth never passed 0\.5\. Activation patching shows that each of these seven lesions needs 91 to 99% of its heads, in that leaving out one of these heads undoes the lesion \(Appendix[G](https://arxiv.org/html/2610.28992#A7)\)\. By the search’s own test, a target probability above 0\.5, another round of pruning would have removed none of the 276 heads of the Llama\-2\-7B\-chat lesion\. The sizes of the seven lesions are thus upper bounds, but tight ones\.

#### Paramgroups its lesions by the patient group that the answer resembles\.

Paramfollows the lesion procedure of[Wang et al\. \(2025a\)](https://arxiv.org/html/2610.28992#bib.bib55)\. It scores every weightwwof the attention and multilayer perceptron \(MLP\) weight matrices by\|w​∂L/∂w\|\|w\\,\\partial L/\\partial w\|, whereLLis the negative log\-likelihood of the control transcripts, each read as the model’s answer to its task’s prompt\. It leaves out the weights that read an input channel with exceptionally large activations\. A lesion zeroes the top 1% of the weights of one matrix, which gives 288 single lesions in GPT\-2 XL, 224 in Llama\-2\-7B\-chat, and 96 to 192 in the Pythia models\.Paramlesions one matrix at a time and compares the answer with the task’s transcripts of each of five groups, control, anomic, Broca, conduction, and Wernicke, by BLEU\-1\([Papineni et al\., 2002](https://arxiv.org/html/2610.28992#bib.bib32)\)and by BERTScore\([Zhang et al\., 2020](https://arxiv.org/html/2610.28992#bib.bib61)\)\. It standardizes each metric per group across the lesions, averages the two standardized metrics, and assigns the lesion to the group with the highest average\. For each of the four aphasia types,Paramlesions together the matrices assigned to that type and, while the answer of this cluster is not assigned the type, adds the remaining matrix with the highest score for the type, up to 32 matrices in all\. We did not runParamon Llama\-2\-13B\-chat\. The classifier scores the final answer of each cluster:Paramworks for the aphasia target when any of the four clusters reaches it, and for the Broca or the Wernicke target when the Broca or the Wernicke cluster reaches that target\. No setting has more than one working cluster\. The classifier thus judges the lesions ofParambut plays no part in finding them\.

#### Param\-Cgrows the lesions ofParamwith the classifier\.

Param\-Csearches the lesions ofParamwith the classifier in place of BLEU\-1 and BERTScore\. It ranks the MLP lesions by their single\-lesion target probability and grows them in that order, asCLAMPgrows heads\. When no prefix works, it sweeps through the layers from first to last on top of the prefix with the highest target probability, adding in each layer the attention lesion that raises the target probability most, up to four per layer, moving to the next layer after two additions in a row that gain nothing, and stopping as soon as a set works\.Param\-Cthen prunes a working set one lesion per round, removing the lesion whose removal leaves the highest target probability, as long as that probability stays above 0\.5\. UnlikeParam,Param\-Calso runs on Llama\-2\-13B\-chat, so it covers all 42 settings\.

#### The learned masks train on the transcripts and use the classifier only to choose and prune\.

Param\-mask,Param\-mask\-hard,Head\-mask, andHead\-mask\-hardlearn a knockout amountki=σ⁡\(θi\)k\_\{i\}=\\sigma\(\\theta\_\{i\}\)between 0 and 1 for each componentii, whereσ\\sigmais the logistic function, and they train onlyθ\\theta, with the model frozen\([Csordás et al\., 2021](https://arxiv.org/html/2610.28992#bib.bib9)\)\. The components ofHead\-maskare the heads, whose outputs the mask scales by1−ki1\-k\_\{i\}\. The components ofParam\-maskare the MLP lesions, the top\-1% lesions of every MLP weight matrix, whose lesioned weights the mask scales by1−ki1\-k\_\{i\}\. The training data are the task’s AphasiaBank transcripts, each read as the model’s answer to the task’s prompt\. For a transcriptxxofnntokens,r⁡\(x\)=\(log⁡pk​\(x\)−log⁡p0​\(x\)\)/nr\(x\)=\(\\log p\_\{k\}\(x\)\-\\log p\_\{0\}\(x\)\)/nis the per\-token log\-likelihood ratio of the masked modelpkp\_\{k\}to the intact modelp0p\_\{0\}\. The loss

ℒ=\\displaystyle\\mathcal\{L\}=\{\}1\|T\|∑x∈Tsoftplus\(−r\(x\)/τ\)\\displaystyle\\frac\{1\}\{\|T\|\}\\sum\_\{x\\in T\}\\mathrm\{softplus\}\\bigl\(\-r\(x\)/\\tau\\bigr\)\+1\|C\|∑x∈Csoftplus\(r\(x\)/τ\)\+λ∑iki\\displaystyle\+\\frac\{1\}\{\|C\|\}\\sum\_\{x\\in C\}\\mathrm\{softplus\}\\bigl\(r\(x\)/\\tau\\bigr\)\+\\lambda\\sum\_\{i\}k\_\{i\}pushesrrup on the transcriptsTTof the target group and down on the transcriptsCCof the other groups, with anℓ1\\ell\_\{1\}penalty on the amounts, wheresoftplus⁡\(z\)=log⁡\(1\+ez\)\\mathrm\{softplus\}\(z\)=\\log\(1\+e^\{z\}\)andτ=0\.1\\tau=0\.1nats per token\. For the aphasia target,TTholds the transcripts of the four aphasia types andCCthe control transcripts; for the Broca or the Wernicke target,TTholds that type’s transcripts andCCthose of the other four groups\. Training follows a path of six decreasing values ofλ\\lambda, with 50 steps each, and each stage starts from the mask of the previous stage and ends in a checkpoint\. The model then answers the task under each checkpoint in two versions: the soft mask, with the learned amounts, forParam\-maskandHead\-mask, and the hard mask, which knocks out the components withki\>0\.5k\_\{i\}\>0\.5, forParam\-mask\-hardandHead\-mask\-hard\. Each version reports its sparsest checkpoint whose target probability exceeds 0\.5, and otherwise its checkpoint with the highest target probability\. A hard mask above 0\.5 is then pruned one component per round, asParam\-Cprunes its sets\. The classifier plays no part in training, but it scores the answers of the checkpoints, picks the checkpoint to report, and decides which components pruning may remove\.

#### Head\-CEMsamples whole sets of heads and learns which heads the best sets share\.

Head\-CEMapplies the cross\-entropy method\([Rubinstein, 1999](https://arxiv.org/html/2610.28992#bib.bib41);[de Boer et al\., 2005](https://arxiv.org/html/2610.28992#bib.bib10)\)to sets of heads\. It keeps a probabilitypip\_\{i\}of knocking out each headiiand, in each round, samples 64 sets, each of which holds each head independently with probabilitypip\_\{i\}, and scores their answers\. The search stops in the first round in which a set’s target probability exceeds 0\.5 and keeps the smallest such set of that round\. Otherwise, it moves eachpip\_\{i\}halfway toward the share of the best quarter of the round’s sets that hold headii, and it keeps everypip\_\{i\}at or above a floor ofmin⁡\(12/N,0\.05\)\\min\(12/N,0\.05\), whereNNis the number of heads\. The search starts from the single\-knockout values: the 16 heads with the highests⁡\(\{h\}\)s\(\\\{h\\\}\)start atpi=0\.3p\_\{i\}=0\.3and the other heads at the floor\. After 20 rounds, or 1,280 sets, without a set above 0\.5, the search reports the set with the highest target probability\. A set above 0\.5 is pruned one head per round, asParam\-Cprunes its sets\. Unlike growth, which ranks each head by its effect alone, the cross\-entropy method judges whole sets, so it can find heads that work only together\.

#### Head\-grown\-cosreplaces the classifier with a similarity to the transcripts\.

This search ranks, grows, and prunes heads asCLAMPdoes, but with a similarity objective in place of the classifier’s target probability\. The objective is the mean cosine similarity between the answer and the transcripts of the target group, minus the mean cosine similarity between the answer and the control transcripts\. It uses the MiniLM sentence embeddings\([Wang et al\., 2020](https://arxiv.org/html/2610.28992#bib.bib52);[Reimers and Gurevych, 2019](https://arxiv.org/html/2610.28992#bib.bib37)\)and the reference transcripts of the classifier’s similarity features\. The target group holds all aphasic transcripts for the aphasia target and the Broca or the Wernicke transcripts for the other two targets, and a set satisfies the objective when the difference exceeds 0\. The classifier then scores the answer of the reported set, and Table[2](https://arxiv.org/html/2610.28992#S5.T2)counts the settings in which that set works by the classifier\.

Table 13:Number of settings in which each lesion method finds a lesion that works, that is, a lesion whose answer counts and reaches the target: a target probability above 0\.5 by the five\-way classifier, or the target as its most probable class, for every lesion space and search that we ran on the seven older models\. The 42 settings combine seven models, two tasks, and three targets, so each target has 14 settings, GPT\-2 XL has 6, the two Llama\-2\-chat models have 12, and the four Pythia models have 24\. An MLP or attention lesion zeroes the top 1% of the weights of one MLP or attention weight matrix by\|w​∂L/∂w\|\|w\\,\\partial L/\\partial w\|on control transcripts, and an MLP block is a block of 256 neurons of one MLP layer\. Within each space, the searches guided by corpus statistics come before those guided by the classifier\.*Grown*: lesions ranked by their single\-lesion target probability, grown in that order asCLAMPgrows heads, and pruned\.*Then head sweep*andParam\-C: when the grown MLP lesions do not work, heads or, forParam\-C, attention lesions are added layer by layer on top of them\.*Layer search*: every head alone, then the top heads of each layer together, and in later stages heads and MLP blocks added layer by layer\. Bold marks the largest count in each column\.
#### More lesion spaces and searches separate the effect of the space from that of the search\.

Table[13](https://arxiv.org/html/2610.28992#A4.T13)adds the lesion spaces and searches that the main text leaves out\. An attention lesion zeroes the top 1% of one query, key, value, or output matrix, as an MLP lesion does for one MLP matrix, and the lesions ofParamdraw on both kinds of matrix\. The learned masks and the cross\-entropy method also search the MLP lesions alone and the MLP lesions and heads together\. A grown search ranks MLP lesions or attention lesions by their single\-lesion target probability, grows them in that order asCLAMPgrows heads, and prunes a set above 0\.5\. One more search starts from the grown MLP lesions asParam\-Cdoes but, when these do not pass 0\.5, sweeps heads through the layers in place of attention lesions\. The layer search, another classifier\-guided search, also works layer by layer\. Its first stage knocks out every head alone and then the top 1, 2, 3, 5, 8, or 12 heads of each layer together, and its later stages sweep through the layers, adding heads and then blocks of 256 MLP neurons on top of those already knocked out\. Growth, the cross\-entropy method, and the learned masks each run in several spaces, so the table compares spaces under one search and searches within one space\.

#### Heads grown by the classifier work in the most settings without degenerate answers\.

Grown the same way, whole heads work in 38 settings, attention lesions in 26, and MLP lesions in 18\. The search matters too: over heads, the first stage of the layer search works in 8 settings andHead\-CEMin 23, and the layer search with all its stages, which adds blocks of MLP neurons, works in 22\. The cross\-entropy method works in more settings with MLP lesions than with heads, 36 against 23, but 6 of its 36 answers with MLP lesions repeat themselves\. Only one other search matches the 38 settings ofCLAMP: the grown MLP lesions with heads swept on top work in more Broca settings, 13 against 11, but in fewer Llama\-2\-chat settings, 9 against 11, and give 5 degenerate answers, which repeat themselves or have model scrambles as their most probable class, against none \(Table[14](https://arxiv.org/html/2610.28992#A4.T14)\)\.CLAMPworks in 11 of the 12 Llama\-2\-chat settings, more than any other method\. Only growth over heads thus reaches 38 settings without a degenerate answer\.

Table 14:The working sets of each lesion method of Table[13](https://arxiv.org/html/2610.28992#A4.T13)on the seven older models, one set per setting in which the method works, described by medians except in the count columns\.*Sets*: number of working sets\.*Size*: components in the set, that is, heads, lesions, or MLP blocks, where a soft mask counts the components with an amount above 0\.0005\.*Target*: target probability of the lesioned model’s answer\.*Scr\.*:P⁡\(scrambles\)P\(\\mathrm\{scrambles\}\), the classifier’s probability of model scrambles for that answer\.*zlib*: length of the zlib\-compressed answer over the length of the answer, both in bytes, which falls below 0\.25 for an answer that repeats itself and exceeds 1 for some short answers\.*Chars*: answer length in characters, not counting Unicode format characters;*<100\{<\}100*: number of answers shorter than 100 characters\.*TAB*: mean target probability over the five TAB connected\-text answers with the set knocked out\.*Other*: target probability of the answer to the other task with the set knocked out, over the sets whose answer there counts\.*Degen\.*: number of degenerate answers, which have a zlib ratio below 0\.25 or model scrambles as the most likely class\. The results ofParamstore neither held\-out check\.
#### Heads give fewer degenerate answers than weight lesions\.

Table[14](https://arxiv.org/html/2610.28992#A4.T14)describes the working sets of each method\. We call a working answer degenerate when it repeats itself, with a zlib compression ratio below 0\.25, or when the classifier’s most probable class for it is model scrambles\. The compression ratio divides the length of the zlib\-compressed answer by the length of the answer, both in bytes\. Heads give fewer such answers than weight lesions both under growth and under the cross\-entropy method\. Growth gives none among the 38 working sets ofCLAMP, against 3 among its 26 sets of attention lesions and 2 among its 18 sets of MLP lesions\. The cross\-entropy method gives 1 among the 23 sets ofHead\-CEM, against 6 among its 36 sets of MLP lesions and 3 among its 33 sets of MLP lesions and heads\. The degenerate answer ofHead\-CEM, for Pythia\-1B on Cinderella with the aphasia target, reachesP⁡\(aphasia\)=0\.503P\(\\mathrm\{aphasia\}\)=0\.503, but its most probable class is model scrambles, atP⁡\(scrambles\)=0\.479P\(\\mathrm\{scrambles\}\)=0\.479\. The 6 degenerate answers of the cross\-entropy method with MLP lesions all repeat themselves, and all come from GPT\-2 XL and the Llama\-2\-chat models\. Over its three spaces, the cross\-entropy method gives 10 degenerate answers among 92 working sets, 9 of them self\-repeating\. The search that sweeps heads on top of grown MLP lesions gives 5 degenerate answers among 38, andParam\-Cgives 3 among 25, whereas the 32 working sets of the learned masks give none\. One of the two working sets ofHead\-grown\-cos, GPT\-2 XL sandwich aphasia, is degenerate too, with a ratio of 0\.217 and model scrambles as its most probable class\. Over the methods of Table[14](https://arxiv.org/html/2610.28992#A4.T14)other than the layer search with MLP blocks, sets of heads alone give 2 degenerate answers among 86, and sets that include weight lesions give 22 among 195\. Knocking out whole heads thus rarely turns the answer into a machine failure, which supports the hypothesis of Section[4\.1](https://arxiv.org/html/2610.28992#S4.SS1)\.

#### The lesions of every method rarely carry their target to new prompts\.

Table[14](https://arxiv.org/html/2610.28992#A4.T14)also reports how the working sets do on the TAB’s five connected\-text prompts and on the other task, which their searches never saw\. Per method, the median target probability on the connected\-text prompts is at most 0\.137\. Over all methods, the mean target probability on these prompts exceeds 0\.5 for only 3 of the 299 working sets checked there, and the answer to the other task reaches the target for only 24 of the 282 working sets whose answer there counts\. No working set ofCLAMPorHead\-CEMexceeds 0\.5 on the connected\-text prompts, where their medians are 0\.009 and 0\.010\. The failure ofCLAMP’s lesions to carry their type beyond the narrative, in Section[5\.1](https://arxiv.org/html/2610.28992#S5.SS1), is thus common to every lesion method\.

#### Head\-CEMtends to find short answers\.

The sets ofHead\-CEMsit closer to 0\.5 than those ofCLAMP, with a median target probability of 0\.604 against 0\.717, and their answers are shorter, with a median of 74 characters against 166\.5\. The length difference is largest for the Broca target\. All 8 Broca answers ofHead\-CEMhave fewer than 100 characters, with a median of 34\.5, and 6 of them begin with “I don’t know, but it”\. Three answers ofHead\-CEM, two for the Broca target and one for the aphasia target, are exactly “I don’t know, but it’s a good one\!”, which the classifier scores atP⁡\(Broca\)=0\.575P\(\\mathrm\{Broca\}\)=0\.575andP⁡\(scrambles\)=0\.346P\(\\mathrm\{scrambles\}\)=0\.346\. The 11 Broca answers ofCLAMPhave a median of 227 characters, and 3 of them are shorter than 100\. On the 7 Broca settings in which both methods work, the answer ofHead\-CEMis the shorter one in 6\. The short answer is not specific to the cross\-entropy method, however\. The answer “I don’t know, but it’s a good one\!” is the working answer of 24 pairs of a method and a setting, more than any other working answer, and always on the Cinderella task of a Pythia model\. Besides the cross\-entropy method, the searches that grow MLP lesions give it in 4 settings, the learned masks in 1, and the layer search in 1\.CLAMPnever gives this answer, and at the median its Broca answers are more than six times as long as those ofHead\-CEM\.

#### The learned masks change the likelihood more than the answer\.

On the transcripts of held\-out speakers, at the last stage of the path, the masked model makes the target group’s transcripts more likely and the other groups’ transcripts less likely than the intact model in 90 of the 126 mask experiments, which cover 42 settings and three spaces\. The per\-token log\-likelihood ratio reaches up to\+0\.31\+0\.31on the target group and down to−0\.31\-0\.31on the other groups\. The answer, however, reaches the target in only 25 experiments, in 12 of the 42 settings and only in GPT\-2 XL and the Pythia models: 21 times for the aphasia target, three times for Broca, and once for Wernicke\. The Wernicke success and one of the Broca successes come from hard masks whose answers reach their target only as the most probable class\. In the two Llama\-2\-chat models, no mask raises the target probability above 0\.01\. The soft and hard versions of a mask often disagree: in 7 experiments only the soft mask works, in 11 only the hard mask, and in 7 both\. Only one working mask carries over to the TAB: the soft head mask of Pythia\-1B for the aphasia target on sandwich, with 69 heads at partial amounts, reaches a meanP⁡\(aphasia\)P\(\\mathrm\{aphasia\}\)of 0\.54 on the connected\-text prompts, against 0\.12 for the intact model, and 0\.77 on the Cinderella answer\. Making the patients’ transcripts more likely thus rarely makes the model speak like the patients\.

#### Paramrarely reaches a target by either criterion\.

By its own criterion,Paramassigns the final cluster its aphasia type in 15 of the 48 clusters of the 12 model and task pairs that we ran: 1 anomic, 6 Broca, 2 conduction, and 6 Wernicke clusters, in 8 of the pairs and in neither task of Llama\-2\-7B\-chat\. The classifier sees these answers differently\. Of the 47 cluster answers long enough for the classifier to score, it calls 35 model scrambles, 8 control speech, and 4 conduction or anomic aphasia\. Two of these four aphasic answers do not count, because the lesioned model denies knowing the story of Cinderella\. The other two are the answers of the GPT\-2 XL Cinderella anomic cluster, atP⁡\(aphasia\)=0\.525P\(\\mathrm\{aphasia\}\)=0\.525, and of the Pythia\-1\.4B Cinderella Broca cluster, atP⁡\(aphasia\)=0\.996P\(\\mathrm\{aphasia\}\)=0\.996with conduction or anomic aphasia as its most probable class, soParamworks for the aphasia target in 2 settings and for the Broca and Wernicke targets in none\. If every growth step of the matching cluster counted,Paramwould work for the aphasia target in 9 settings, for the Broca target in 1, and for the Wernicke target in none\. Its single lesions rarely work either: of the 2,272 single lesions over the 12 pairs, 40 reach the aphasia target, 3 the Broca target, and none the Wernicke target, all in the Pythia models\. Matching patient speech by text similarity thus mostly selects lesions that turn the answer into model scrambles\.

#### The classifier makesParam’s lesions work\.

Searched with the classifier asParam\-C, the lesions ofParamwork in 25 of the 42 settings: 11 for the aphasia target, 10 for Broca, and 4 for Wernicke\. They work in 4 of the 12 settings of the Llama\-2\-chat models\. On the 36 settings in which we ranParam,Param\-Cworks in 23 andParamin 2\. After pruning, 15 of the 25 working sets hold MLP lesions only, 7 hold both kinds, and 3 hold attention lesions only\.Param\-Cworks in two settings in whichCLAMPfails, both for the Broca target, GPT\-2 XL on Cinderella and Llama\-2\-7B\-chat on sandwich, and it fails in 15 in whichCLAMPworks, 9 of them for the Wernicke target\. Three of its 25 working answers are degenerate \(Table[14](https://arxiv.org/html/2610.28992#A4.T14)\), and three others are “I don’t know, but it’s a good one\!”\.

## Appendix EExamples of Lesioned Answers

This appendix shows what lesioned models say\. Table[15](https://arxiv.org/html/2610.28992#A5.T15)gives answers underCLAMPlesions, and Table[16](https://arxiv.org/html/2610.28992#A5.T16)gives answers under lesions found by criteria that cannot recognize a machine failure\. We chose theCLAMPexamples to cover the three targets, the four model families, and both tasks, so they illustrate what the lesions can produce and are not a random sample\.

ModelTaskTargetHeadsPPIntact answerLesioned answerGPT\-2 XLCinderellaAphasia990\.8910\.891…Cinderella was born in a small village called "Horse\-head" \(or Horse\-Head\) in the Kingdom of Khazaria\.…I don’t know anything about her, but she’s a pretty girl and she has to go to the ball\.Pythia\-1BsandwichAphasia15150\.6910\.691I’d use the same recipe as for my bread, but add some of your favorite jam or preserves to it instead of mayonnaise\.…You’d put the bread in first, then add some of that to your sandwich\. Then you could just eat it all\!Qwen3\.5\-2BsandwichBroca18180\.7380\.738Making the perfect Peanut Butter & Jelly \(PB&J\) is less about following strict rules…Tell, tell the way of? / "Tell to know\." "Pythia\-6\.9BsandwichBroca13130\.6650\.665I’d put the bread on top of the PB, then spread it with some jam \(or honey\)…You have to ask for it\!Llama\-2\-13B\-chat†CinderellaBroca1011010\.8860\.886Sure\! Here’s a brief retelling of the classic tale of Cinderella: / Once upon a time, there was a beautiful and kind girl named Cinderella…to tell a tale of Ct\.\. to tell me\.Llama\-2\-13B\-chatCinderellaWernicke1091090\.7860\.786as in the row abovetours and tours, but I can’dough, but i can be a new to the whole and the previous one is not\.Pythia\-1BCinderellaWernicke13130\.7590\.759The first part is a fairy tale, and it’s about how she got her shoes on in order to go out into the world with Prince Charming…I am a new one, and it is to be able that we are not in any way for your own\. We have been on our own with each other than what they do\.…Table 15:Answers underCLAMPlesions that reach their target, two for each target from different model families, and the Llama\-2\-13B\-chat example of §[5\.1](https://arxiv.org/html/2610.28992#S5.SS1), marked†\.*Target*: aphasia, Broca’s aphasia, or Wernicke’s aphasia;*Heads*: the number of heads that the lesion knocks out;*PP*: the five\-way classifier’s probability of the target for the lesioned answer, which is at most 0\.009 for each intact answer\. The classifier’s most likely class for each intact answer is control speech, except for the GPT\-2 XL story, which it calls a model scramble\. Each answer is the model’s single greedy answer; excerpts are verbatim, … marks omitted text, and a slash marks one or more line breaks\.Table 16:Answers under lesions found by criteria that cannot recognize a machine failure, all of which the five\-way classifier calls model scrambles\. The three heads of Llama\-2\-7B\-chat, where LiiHjjis headjjof layerii, come from the preliminary search of §[3](https://arxiv.org/html/2610.28992#S3), whose classifier had only an aphasic and a control class\. Each weight lesion ofParamzeroes the 1% of one weight matrix’s weights with the largest\|w​∂L/∂w\|\|w\\,\\partial L/\\partial w\|\(§[4\.1](https://arxiv.org/html/2610.28992#S4.SS1)\), and eachParamcluster reaches its aphasia type byParam’s own criterion, BLEU\-1 and BERTScore against patient transcripts\.P⁡\(scrambles\)P\(\\mathrm\{scrambles\}\): the five\-way classifier’s probability of model scrambles, to four decimals\. Answers are verbatim and complete except where … marks omitted text, and a slash marks one or more line breaks; the Pythia\-1\.4B answer goes on repeating “T H E F O R T W E” until it ends\.#### Lesions found byCLAMP\.

The answers that reach Broca’s aphasia are short, from 23 to 41 characters, and two of the three are fragments\. One of them is the Llama\-2\-13B\-chat story of §[5\.1](https://arxiv.org/html/2610.28992#S5.SS1), “to tell a tale of Ct\.\. to tell me\.” Asked first whether it knows the story, the same lesioned model answers “Yes, I am familiar with the story of Cinder”, which stops at the length limit for that answer\. The third Broca answer, “You have to ask for it\!”, is a complete sentence, which suggests that the classifier also scores short answers as Broca\-like, as §[5\.1](https://arxiv.org/html/2610.28992#S5.SS1)notes forHead\-CEM\. The answers that reach Wernicke’s aphasia are fluent but empty, and one contains a form that resembles a paraphasia, “can’dough”\. The answers that reach the aphasia target stay on the task but are vague, as in “add some of that to your sandwich”\.

#### Lesions found by other criteria\.

The three heads of Llama\-2\-7B\-chat in Table[16](https://arxiv.org/html/2610.28992#A5.T16)come from the preliminary search of §[3](https://arxiv.org/html/2610.28992#S3)\. That search used an earlier classifier with only an aphasic and a control class, and by this classifier, knocking out the three heads made the model’s greedy answer aphasic\. The five\-way classifier’s training data hold that greedy answer as a model scramble, but not the different answer in Table[16](https://arxiv.org/html/2610.28992#A5.T16), which uses the repetition penalty of §[3](https://arxiv.org/html/2610.28992#S3)\. EachParamcluster reaches its type byParam’s own criterion, BLEU\-1 and BERTScore against patient transcripts, although the same criterion assigns even the intact Pythia\-1\.4B story to Broca’s aphasia and the intact Pythia\-6\.9B sandwich answer to anomic aphasia\. None of these criteria can recognize a machine failure, and all three lesions produce one: an off\-task request for details about “your company”, a run of single letters, and text like a web advertisement, which the five\-way classifier calls model scrambles withP⁡\(scrambles\)P\(\\mathrm\{scrambles\}\)of at least 0\.998\.

#### Single answers\.

Each answer in both tables is a single greedy answer to one prompt, and the targeted type does not carry over to other prompts, although the lesions also damage the answers to them \(Appendix[F](https://arxiv.org/html/2610.28992#A6)\)\. Under the seven lesions of Table[15](https://arxiv.org/html/2610.28992#A5.T15), no answer to the other task or to the connected\-text prompts of the Text Aphasia Battery reaches the target, and the highest target probability among these answers is 0\.053\.

## Appendix FLesions beyond the Narrative

EachCLAMPlesion is found on the model’s answer to one narrative task, and §[5\.1](https://arxiv.org/html/2610.28992#S5.SS1)reports that the lesions also damage the model’s answers to other prompts\. We test each of the 51 lesions that reach their target in four ways: on the other narrative task, on the short\-answer items and on the connected\-text prompts of the Text Aphasia Battery \(TAB\)\([Roll et al\., 2026b](https://arxiv.org/html/2610.28992#bib.bib40)\), and on MMLU\([Hendrycks et al\., 2021](https://arxiv.org/html/2610.28992#bib.bib17)\)\. This appendix describes how we score each test, lists the scores of every lesion \(Tables[17](https://arxiv.org/html/2610.28992#A6.T17)and[18](https://arxiv.org/html/2610.28992#A6.T18)\), and summarizes them; Table[3](https://arxiv.org/html/2610.28992#S5.T3)gives the medians by model family\. We name a lesion by its model, the task that it was found on, and its target, for example the GPT\-2 XL sandwich aphasia lesion, and, as in §[5\.1](https://arxiv.org/html/2610.28992#S5.SS1), we call GPT\-2 XL, the two Llama\-2\-chat models, and the four Pythia models the older models\.

Table 17:TheCLAMPlesions of the base models GPT\-2 XL and Pythia that reach their target, tested beyond the narrative task that they were found on\.*Heads*: number of heads knocked out\.*Same*and*Other*: the five\-way classifier’s probability of the target for the lesioned model’s answer to the narrative task that the lesion was found on and to the other narrative task\.*Short Answer*: short\-answer items of the Text Aphasia Battery \(TAB\) for comprehension and repetition that the lesioned model answers correctly, of 15; the intact models answer 9 \(GPT\-2 XL\), 10 \(Pythia\-1B\), 8 \(Pythia\-1\.4B\), 7 \(Pythia\-2\.8B\), and 6 \(Pythia\-6\.9B\) correctly\.*Features*and*Impaired*: the mean number of aphasia features, of 19, that the LLM judge finds per connected\-text answer, and the share of the five answers that it judges to show an overall communication impairment, for the intact model before the arrow and for the lesioned model after it\.*Drop*: drop in the mean log\-probability of the correct MMLU answer letter, in nats, averaged over two subjects\.Table 18:TheCLAMPlesions of the chat models Llama\-2\-chat and Qwen3\.5 that reach their target, tested beyond the narrative task that they were found on\.*Heads*: number of heads knocked out\.*Same*and*Other*: the five\-way classifier’s probability of the target for the lesioned model’s answer to the narrative task that the lesion was found on and to the other narrative task\.*Short Answer*: short\-answer items of the Text Aphasia Battery \(TAB\) for comprehension and repetition that the lesioned model answers correctly, of 15; the intact models answer 7 \(Llama\-2\-7B\-chat\), 8 \(Llama\-2\-13B\-chat\), 13 \(Qwen3\.5\-0\.8B and Qwen3\.5\-2B\), and 14 \(Qwen3\.5\-4B and Qwen3\.5\-9B\) correctly\.*Features*and*Impaired*: the mean number of aphasia features, of 19, that the LLM judge finds per connected\-text answer, and the share of the five answers that it judges to show an overall communication impairment, for the intact model before the arrow and for the lesioned model after it\.*Drop*: drop in the mean log\-probability of the correct MMLU answer letter, in nats, averaged over two subjects\.†The answer does not count, because the lesioned model denies knowing the story of Cinderella\.#### The other narrative task tests a lesion on the task that its search never saw\.

For a lesion found on the Cinderella task, the other task is the sandwich task, and for a lesion found on the sandwich task, it is the Cinderella task\. The lesioned model answers the other task’s prompt with the decoding of §[3](https://arxiv.org/html/2610.28992#S3), and the classifier scores the answer for the lesion’s target\. The rules of Appendix[B](https://arxiv.org/html/2610.28992#A2)decide whether the answer counts, and 53 of the 54 answers do\. The remaining answer does not count because, with its sandwich aphasia lesion, Qwen3\.5\-0\.8B denies knowing the story of Cinderella\.

#### Rules score the TAB’s short\-answer items\.

We use 20 items of the TAB, five in each of four subtests: connected text, word comprehension, sentence comprehension, and repetition \(Table[7](https://arxiv.org/html/2610.28992#A2.T7)\)\. The 15 items of the last three subtests ask for short answers: word comprehension asks the model to pick one of six listed words, sentence comprehension asks a yes\-or\-no question, and repetition asks the model to repeat a word or a sentence exactly\. The model answers these items greedily, without the repetition penalty, and rules score its answers \(Appendix[B](https://arxiv.org/html/2610.28992#A2)\)\. A word\-comprehension answer is correct if the first option that it names is the target, a sentence\-comprehension answer if its first “yes” or “no” matches the target, and a repetition answer if it equals the target after normalization\. Tables[17](https://arxiv.org/html/2610.28992#A6.T17)and[18](https://arxiv.org/html/2610.28992#A6.T18)give the number of the 15 items that the lesioned model answers correctly\. The*items lost*by a lesion are the number of items that the intact model answers correctly minus the number that the lesioned model answers correctly, so a negative number means items gained\. With only 15 items, a single item should not decide, so we count a lesion as costing items when it loses two or more\.

#### The classifier and an LLM judge score the TAB’s connected text\.

Four of the five connected\-text prompts ask about the speaker’s own life, for example “Tell me about the best trip you ever took”, and one asks for the steps to make a simple meal\. The model answers them with the decoding of the narratives, greedily with the repetition penalty, but writes at most 120 new tokens\. The classifier scores each answer that is long enough to count, which holds for all five answers of 48 of the 54 lesions and for two to four answers of the other six, and an answer reaches the lesion’s target by the rule above\. An LLM judge, DeepSeek\-V4\.1\-Flash\([DeepSeek\-AI, 2026](https://arxiv.org/html/2610.28992#bib.bib12)\)at temperature 0, marks each of the five answers for the TAB’s 19 aphasia features, one of which is an overall communication impairment\. We report the mean number of features per answer and the share of the five answers that show an overall communication impairment\. The judge is a reasoning model and is not deterministic even at temperature 0, so we keep its first reply to each request\. Its prompt follows the TAB’s protocol, but we wrote the feature definitions, the examples, and the two synthetic annotated transcripts ourselves \(Appendix[K](https://arxiv.org/html/2610.28992#A11)\), and no human rated these answers to check the judge\.

#### MMLU tests knowledge far from the narratives\.

We use two subjects of MMLU: global facts, with 100 test questions, and high school world history, with 237\. Each prompt is the standard five\-shot prompt, except that the world\-history prompts of GPT\-2 XL keep only one to three solved questions to fit its context \(Appendix[B](https://arxiv.org/html/2610.28992#A2)\)\. The model reads the prompt once and generates nothing\. Its prediction is the answer letter with the highest next\-token probability, and its accuracy is the share of questions that it predicts correctly, which is 0\.25 by chance\. We compute the log\-probability of the correct letter over the whole vocabulary, so it also falls when the model moves probability away from the four letters\. The*MMLU drop*of a lesion is the intact model’s mean log\-probability of the correct letter minus the lesioned model’s, in nats, averaged over the two subjects with equal weight\.

#### The judge finds more aphasia features after a lesion\.

The judge finds more of the TAB’s aphasia features in the lesioned model’s connected text than in the intact model’s for 50 of the 54 lesions\. Three lesions tie with the intact model, and the GPT\-2 XL sandwich aphasia lesion falls below it, at 0\.6 against 0\.8 features per answer\. Over the 54 lesions, the median is 2\.7 of 19 features per answer, against 0\.0 for the intact models\. The lesions of the older models reach a median of 2\.1 features, against 0\.0 for their intact models, and those of the Qwen3\.5 models a median of 4\.8, against 0\.2\. The judge sees an overall communication impairment in a median of four of a lesioned model’s five answers, three in the older models and all five in the Qwen3\.5 models, but in at most two of an intact model’s five answers\. It sees one in all five answers for 24 of the 54 lesions\. By the judge, a lesion found on one narrative thus impairs the model’s language on prompts that the search never saw\.

#### The lesions cost comprehension and repetition items\.

The lesioned models answer a median of 4 of the 15 short\-answer items correctly, against 8\.5 for the intact models\. Of the 54 lesions, 41 lose two or more items: 25 of the 38 lesions of the older models and all 16 lesions of the Qwen3\.5 models\. The median lesion loses 5 items, 4 in the older models and 8\.5 in the Qwen3\.5 models\. The Qwen3\.5 models have more to lose: intact, they answer 13 or 14 of the 15 items correctly, against 6 to 10 for the older models\. Repetition shows the difference most clearly\. The intact Qwen3\.5 models repeat all five items exactly, and their lesions lose a mean of 4\.0 of them\. The intact older models repeat at most four items exactly, and their lesions lose a mean of 0\.50\. The intact Llama\-2 models repeat none exactly, although the answers of Llama\-2\-7B\-chat contain the target for three of the five items and those of Llama\-2\-13B\-chat for all five\. On word and sentence comprehension, the lesions of the older models lose a mean of 1\.47 and 1\.37 items, and those of the Qwen3\.5 models 2\.06 and 2\.75\. Six lesions, all from GPT\-2 XL and the Pythia models, gain one to four items\. The lesions thus impair comprehension and repetition as well as connected text, and they cost the most where the intact model has the most to lose\.

#### The lesions lower the log\-probability of correct MMLU answers\.

The median MMLU drop is 0\.89 nats over the 54 lesions, 0\.34 nats in the older models, and 1\.11 nats in the Qwen3\.5 models\. The drop exceeds 0\.5 nats for 31 lesions, 17 of the 38 lesions of the older models and 14 of the 16 lesions of the Qwen3\.5 models, and it ranges from−0\.03\-0\.03to7\.787\.78nats\. Where the intact model answers world\-history questions well above chance, its accuracy falls with the log\-probability\. Every Llama\-2 lesion lowers the world\-history accuracy from 0\.675 or 0\.717 to between 0\.203 and 0\.270, and the Qwen3\.5 lesions lower it from between 0\.654 and 0\.907 to between 0\.207 and 0\.532\. Even the Qwen3\.5\-9B sandwich aphasia lesion, whose drop of 0\.35 nats is the smallest of the Qwen3\.5 lesions, lowers it from 0\.907 to 0\.532\. The intact GPT\-2 XL and Pythia models answer near chance, with accuracies of 0\.18 to 0\.36 on global facts and 0\.236 to 0\.300 on world history, so for them the log\-probability of the correct answer shows the damage that the accuracy cannot\. Most lesions found on one narrative thus also lower the probability that the model gives the correct answers to questions that have nothing to do with the narrative\.

#### The classifier rarely finds the targeted type beyond the narrative\.

No lesion raises the mean target probability of its connected\-text answers above 0\.5: the highest mean is 0\.369, for the Pythia\-1B Cinderella aphasia lesion, and the median is 0\.009\. Answer by answer, 12 of the 259 scored answers reach their target, and 11 of the 12 come from aphasia lesions, for which any type of aphasia counts\. The only answer of a Broca or Wernicke lesion that reaches its target is the Pythia\-2\.8B Cinderella Wernicke lesion’s answer to “Tell me about your first job”, atP⁡\(Wernicke\)=0\.862P\(\\mathrm\{Wernicke\}\)=0\.862\. On the other narrative task, 1 of the 54 lesions reaches its target: the 31 heads of the GPT\-2 XL sandwich aphasia lesion also make the model’s answer to the Cinderella task aphasic, at 0\.859, so this lesion makes both narratives aphasic\. Over the 53 lesions whose answer to the other task counts, the median target probability is 0\.002\. The lesions thus damage the answers to new prompts by the judge’s features, the short\-answer items, and MMLU, but the classifier rarely finds the lesion’s target in these answers\.

## Appendix GActivation Patching and Shared Heads

We take apart the 54CLAMPlesions that reach their target \(Figure[4](https://arxiv.org/html/2610.28992#S5.F4)\) to support three findings of §[5\.1](https://arxiv.org/html/2610.28992#S5.SS1)\. Activation patching shows that the heads of a lesion act together while the model reads the prompt \(Tables[19](https://arxiv.org/html/2610.28992#A7.T19)and[20](https://arxiv.org/html/2610.28992#A7.T20)\), and the same patching on the TAB shows that the damage beyond the narrative arises in the same way\. Comparisons between the lesions of one model show that the lesions for different targets share many heads \(Table[21](https://arxiv.org/html/2610.28992#A7.T21)\), and that the lesions found on the Cinderella and the sandwich tasks share heads but rarely carry their target to the other task \(Table[22](https://arxiv.org/html/2610.28992#A7.T22)\)\. A last analysis asks whether the Qwen3\.5 lesions favor one of the two kinds of attention in these models\. We name a lesion by its model, the task that it was found on, and its target, for example the GPT\-2 XL sandwich aphasia lesion\.

Table 19:Activation patching of theCLAMPlesions of the base models GPT\-2 XL and Pythia that reach their targets\.*Heads*: the number of heads in the set\.*Probability*: the five\-way classifier’s probability of the target with the set knocked out,*Lesion*, and with the clean outputs of all of its heads written back while the model reads the prompt,*Restore all*, so that the set is knocked out only while the model writes its answer; an answer undoes the lesion when it no longer reaches the target, a probability of 0\.5 or below with the target not the most probable class, and the one restore\-all answer that still reaches it is in bold\.*Share of heads*: the share of the set’s heads whose clean output on the prompt undoes the lesion,*Restored*, whose removal from the set undoes it,*Left out*, and that reach the target when knocked out alone,*Alone*\.*TAB short answer*: short\-answer items of the Text Aphasia Battery \(TAB\) for comprehension and repetition that the model answers correctly, of 15, with the set knocked out, with all heads restored, and with the best of at most eight single\-head restores; the intact models answer 9 \(GPT\-2 XL\), 10 \(Pythia\-1B\), 8 \(Pythia\-1\.4B\), 7 \(Pythia\-2\.8B\), and 6 \(Pythia\-6\.9B\) correctly\.Table 20:Activation patching of theCLAMPlesions of the chat models Llama\-2\-chat and Qwen3\.5 that reach their targets\.*Heads*: the number of heads in the set\.*Probability*: the five\-way classifier’s probability of the target with the set knocked out,*Lesion*, and with the clean outputs of all of its heads written back while the model reads the prompt,*Restore all*, so that the set is knocked out only while the model writes its answer; an answer undoes the lesion when it no longer reaches the target, a probability of 0\.5 or below with the target not the most probable class\.*Share of heads*: the share of the set’s heads whose clean output on the prompt undoes the lesion,*Restored*, whose removal from the set undoes it,*Left out*, and that reach the target when knocked out alone,*Alone*\.*TAB short answer*: short\-answer items of the Text Aphasia Battery \(TAB\) for comprehension and repetition that the model answers correctly, of 15, with the set knocked out, with all heads restored, and with the best of at most eight single\-head restores; the intact models answer 7 \(Llama\-2\-7B\-chat\), 8 \(Llama\-2\-13B\-chat\), 13 \(Qwen3\.5\-0\.8B and Qwen3\.5\-2B\), and 14 \(Qwen3\.5\-4B and Qwen3\.5\-9B\) correctly\.#### Four patching conditions test where the heads act and which of them are needed\.

Activation patching copies activations from one run of a model into another run on the same input, to find out which components carry an effect\([Meng et al\., 2022](https://arxiv.org/html/2610.28992#bib.bib29);[Zhang and Nanda, 2024](https://arxiv.org/html/2610.28992#bib.bib62);[Heimersheim and Nanda, 2024](https://arxiv.org/html/2610.28992#bib.bib16)\)\. Here the clean run uses the intact model, and the lesioned run knocks out every head of a lesion at every position of the prompt and of the answer, as in §[4\.1](https://arxiv.org/html/2610.28992#S4.SS1)\. For each head of a lesion, we generate and score the model’s answer under three conditions\.*Restoring*the head writes its output from the clean run back at every position of the prompt, while the other heads of the lesion stay knocked out; the restored head itself stays knocked out at the positions of the answer, which have no counterpart in the clean run\.*Leaving out*the head knocks out only the other heads of the lesion, so that the head runs normally at every position\.*Knocking out the head alone*knocks out only this head, so that the other heads of the lesion run normally\. The first two conditions differ in what the head computes: a restored head’s output on the prompt is copied from the clean run, whatever the other knockouts do to its input, whereas a head that is left out computes its output from a residual stream that the other knockouts have changed\. A fourth condition acts on the whole lesion:*restoring all*writes back the clean outputs of every head of the lesion at every position of the prompt, so that the lesion is knocked out only while the model writes its answer\. In the terms of[Heimersheim and Nanda \(2024\)](https://arxiv.org/html/2610.28992#bib.bib16), restoring is denoising: it writes clean activations into the lesioned run and asks whether they remove the effect of the lesion\. An answer reaches the target when it counts by the rules of Sections[3](https://arxiv.org/html/2610.28992#S3)and[4\.1](https://arxiv.org/html/2610.28992#S4.SS1)and the classifier either gives the target a probability above 0\.5 or ranks the target as its most probable class, where any of the three aphasia classes counts for the aphasia target\. A condition*undoes*the lesion when its answer no longer reaches the target\. The patching runs reproduce the stored intact and lesioned answers of all 54 lesions\.

#### Restoring the heads’ outputs on the prompt undoes 53 of the 54 lesions\.

Restoring all heads undoes 53 of the 54 lesions and leaves a target probability of at most 0\.369 in them \(Tables[19](https://arxiv.org/html/2610.28992#A7.T19)and[20](https://arxiv.org/html/2610.28992#A7.T20)\)\. These 53 lesions need their heads knocked out while the model reads the prompt, and knocking the heads out only while the model writes its answer does not reach the target\. The exception is the Pythia\-6\.9B Cinderella aphasia lesion, which consists of a single head, L1H26, that is, head 26 of layer 1, both counted from 0\. With the clean output of this head written back on the prompt, the target probability stays at 0\.957, so knocking the head out only while the model writes its answer already reaches the target\. Restoring a single head usually undoes a lesion as well\. The median share of a lesion’s heads whose restore undoes the lesion is 0\.83, both over the 54 lesions and over the 50 lesions of two or more heads, and at least one single restore undoes every lesion except the Pythia\-6\.9B Cinderella aphasia lesion\. Pooled over the 54 lesions, restoring undoes the lesion for 3,579 of the 4,600 heads\. Two other classifier\-guided searches of Appendix[D](https://arxiv.org/html/2610.28992#A4), whose sets above 0\.5 we patched in the same way, give similar medians over all lesioned components of a set: 0\.87 over the 20 sets of the layer search, which can add blocks of MLP neurons, and 0\.96 over the 37 sets of the search that sweeps heads on top of grown MLP lesions\. The heads of a lesion thus act while the model reads the prompt, and most of them matter there: giving back the clean output of a single head usually undoes the lesion\.

#### A single restore mostly breaks the answer instead of repairing it\.

Undoing a lesion means only that the answer no longer reaches the target, not that the model answers as it does when intact\. Of the 3,579 single restores that undo a lesion, 3,541 give an answer long enough to score\. The classifier labels 2,819 of these answers model scrambles, or 80%, and only 450 control speech, or 13%; the other 272 have an aphasia class as their most probable class\. The intact answer is not always control speech: the classifier labels it control speech for 41 of the 54 lesions and model scrambles for the other 13, all of them on the Cinderella task\. Over the 41 lesions whose intact answer is control speech, the result is the same: of the 2,767 scored answers of single restores that undo these lesions, the classifier labels 2,203 model scrambles, or 80%, and 331 control speech, or 12%\. Restoring all heads repairs the answer more often: it undoes 40 of these 41 lesions, and the classifier labels 22 of the 40 answers control speech and 13 model scrambles\. The aphasia\-like answer thus needs the whole lesion, and giving one head back its clean output mostly breaks the answer in another way\.

#### Almost every head is needed, largely by construction\.

Leaving out one head undoes the lesion for 4,484 of the 4,600 heads, and in 37 of the 54 lesions, leaving out any one head undoes the lesion\. No head of the 50 lesions of two or more heads reaches the target when knocked out alone\. Both results follow mostly from howCLAMPbuilds the lesions, so they check the lesions rather than reveal how the heads work\. Growth ranks the heads by the target probability that each gives when knocked out alone and stops at the first prefix whose probability exceeds 0\.5, so a lesion of two or more heads arises only when no head exceeds 0\.5 alone\. Pruning removes heads for as long as the target probability stays above 0\.5, so a lesion whose pruning finished sits at the edge where removing any one more head brings the probability to 0\.5 or below, and leaving out a head repeats this last test of pruning\. By the search’s own rule, a probability above 0\.5, every head is therefore needed in all 47 lesions whose pruning finished, and in 48 of the 51 lesions that exceed 0\.5\. Reaching the target, as defined above, also includes an answer at 0\.5 or below whose most probable class is still the target, so by this definition every head is needed in only 37 of the 47 pruned lesions\. In the other 10, leaving out a head lowers the target probability to 0\.5 or below while the target stays the most probable class, for 1 to 9 heads per lesion and 20 heads in all\. Beyond this difference between the two rules, leaving out heads tests more than pruning did only in the seven lesions whose pruning did not finish: the four that stopped at the budget of 15,000 answers and the three that reach their target only as the most probable class, which the search never pruned\. In these seven lesions, 91 to 99% of the heads are needed: 233 of 244 in the GPT\-2 XL Cinderella Wernicke lesion, 272 of 276 in the Llama\-2\-7B\-chat Cinderella Wernicke lesion, 558 of 578 in the Pythia\-6\.9B Cinderella Broca lesion, 190 of 208 in the Qwen3\.5\-4B Cinderella aphasia lesion, 291 of 316 in the Pythia\-1\.4B sandwich Broca lesion, 62 of 63 in the Qwen3\.5\-2B sandwich Wernicke lesion, and 211 of 228 in the Qwen3\.5\-4B sandwich Wernicke lesion\. By the search’s rule, all 276 heads of the Llama\-2\-7B\-chat lesion are needed, so a further round of pruning would have removed none of them\. The restore conditions are not settled by construction, because a restored head differs from a head that is left out: it stays knocked out while the model writes its answer, and its output on the prompt is the clean one\. Because pruning leaves most lesions at the edge of their target, these yes\-or\-no outcomes show where the heads act, but not how much each head contributes, and a change that breaks the answer counts as undoing the lesion as much as one that repairs it\. Per\-head patching also cannot show which head feeds which; path patching, which restores only the connection from one component to another\([Wang et al\., 2023a](https://arxiv.org/html/2610.28992#bib.bib53)\), would trace these connections\.

#### Restoring all heads also repairs most of the damage on the TAB\.

We repeat the lesion, restoring all, and single restores on the 20 items of the TAB \(Appendix[F](https://arxiv.org/html/2610.28992#A6)\): 15 short\-answer items for word comprehension, sentence comprehension, and repetition, which a rule scores, and five connected\-text prompts\. Single restores cover every head of the 7 lesions of at most eight heads and, in the other 47 lesions, the eight heads whose removal from the lesion lowers the target probability on the narrative the most\. The intact and lesioned TAB answers of all 54 lesions match those of Appendix[F](https://arxiv.org/html/2610.28992#A6)word for word\. Tables[19](https://arxiv.org/html/2610.28992#A7.T19)and[20](https://arxiv.org/html/2610.28992#A7.T20)give the number of the 15 short\-answer items that the model answers correctly in each condition\. With the lesion in place, 41 of the 54 lesions lose two or more of these items, counted from the intact model’s number of correct items\. Restoring all heads gives back at least half of the lost items in 39 of these 41 lesions and all of them in 16, whereas the best single restore gives back at least half in only 10 and all of them in 4\. Over all 54 lesions, the model answers a median of 4 of the 15 items correctly with the lesion in place, 8 with all heads restored, and 5\.5 with the best single restore, against 8\.5 for the intact models\. The median number of items lost is 5, 1, and 3 in these three conditions\. On the connected\-text prompts, the lesions rarely reach their target, so a restore has little to undo there: the classifier finds the target in 12 of the 259 answers that it scores with the lesion in place, in 8 of 269 with all heads restored, and in 2 of the intact model’s 270\. We did not measure MMLU under patching\. Unlike the target probability, the number of correct items shows whether the answers recover, and by this measure restoring all heads repairs most of the damage, whereas restoring one of the tested heads rarely does\. The damage beyond the narrative thus has the same source as the narrative’s target: the heads acting while the model reads the prompt\.

Table 21:Heads shared by theCLAMPlesions that a model has for two targets on the same task, for every pair of targets whose two lesions reach their target\.*Heads*: the number of heads in the lesion for the aphasia \(Aph\.\), Broca, and Wernicke \(Wern\.\) target, where a dash marks a target without a lesion\.*Shared*: the heads that the two lesions of a pair share\.*Chance*: the number of shared heads expected if both lesions were drawn at random from the model’sNNheads,\|L1\|​\|L2\|/N\{\|L\_\{1\}\|\\,\|L\_\{2\}\|/N\}for lesionsL1L\_\{1\}andL2L\_\{2\}\.pp: the one\-sided hypergeometric probability of sharing at least as many heads by chance\.*All pairs*: the total of shared heads, the total expected by chance, and the number of pairs withp<0\.01p<0\.01\.*Median share*: the median over pairs of the shared heads as a share of the smaller lesion\. Three combinations of model and task with fewer than two lesions, Qwen3\.5\-0\.8B on sandwich and Qwen3\.5\-9B on both tasks, are left out\.Table 22:Heads shared by theCLAMPlesions that a model has for the same target on the Cinderella and on the sandwich task, in one model per family\.*Heads*: the number of heads in the Cinderella setCC, in the sandwich setSS, and in both\.*Chance*: the number of shared heads expected if both sets were drawn at random from the model’sNNheads,\|C\|​\|S\|/N\{\|C\|\\,\|S\|/N\}\.*Ratio*: shared heads divided by chance\.pp: the one\-sided hypergeometric probability of sharing at least as many heads by chance\.*Union*and*Intersection*: the classifier’s probability of the target for the answer to each task with the heads of either set, or only the shared heads, knocked out; values above 0\.5, in bold, reach the target, and a dash marks pairs without shared heads\. Three pairs are left out because a set does not reach its target: Broca’s aphasia in GPT\-2 XL, whose Cinderella set does not reach it, and Broca’s and Wernicke’s aphasia in Qwen3\.5\-9B, whose sets reach it on neither task\.
#### The lesions for different targets share many heads\.

For each model and task,CLAMPruns one search per target, which ranks the heads by that target’s probability, grows the lesion in this order, and prunes it against the same target\. If two lesionsL1L\_\{1\}andL2L\_\{2\}were drawn at random from a model’sNNheads, they would share\|L1\|​\|L2\|/N\{\|L\_\{1\}\|\\,\|L\_\{2\}\|/N\}heads on average, and a one\-sided hypergeometric test gives the probability of sharing at least the observed number of heads by chance \(Table[21](https://arxiv.org/html/2610.28992#A7.T21)\)\. The Broca and Wernicke lesions give the cleanest test, because the two targets are separate classes of the classifier\. Over the 13 pairs of Broca and Wernicke lesions of one model and task that both reach their target, the two lesions share a median of 54% of the heads of the smaller lesion and 517 heads in all, three times the 165 expected by chance, and the overlap exceeds chance atp<0\.01p<0\.01in 11 of the 13 pairs\. The two exceptions are the Pythia\-1B sandwich pair, whose lesions share 3 heads against 2\.5 by chance, and the Pythia\-6\.9B Cinderella pair, whose one\-head Wernicke lesion lies inside the 578\-head Broca lesion\. The lesions for the aphasia target, whose probability includes those of the Broca and Wernicke classes, overlap with both: they share a median of 50% of the smaller lesion with the Broca lesion, 236 heads against 98 by chance over 15 pairs, and 39% with the Wernicke lesion, 249 heads against 97 over 17 pairs\. Nothing in the design makes the searches for different targets share heads, but they rank the same single\-head answers: a head whose knockout makes the answer more Broca\-like usually makes it more Wernicke\-like too, at a median Spearman correlation across heads of 0\.93 over the 13 pairs, from 0\.49 to 0\.99, so the two growths follow similar rankings\. Ties add to the overlap: heads whose knockout alone leaves the target probability exactly at the intact value tie with one another in that target’s ranking and keep their layer order \(Appendix[D](https://arxiv.org/html/2610.28992#A4)\), so when the growths for two targets reach such heads, they add them in the same order\. Heads tied in both rankings make up 134 of the 517 heads that the Broca and Wernicke lesions share, 80 of them in the Llama\-2\-chat models, and among the other heads of the models, the two lesions still share 383 heads against 156 expected by chance, withp<0\.01p<0\.01in 9 of the 13 pairs\. Broca\-like and Wernicke\-like answers thus come from lesions that share many heads, so the circuits for different types of aphasia are entangled rather than separate\.

#### The lesions found on the two tasks share heads, but their targets rarely carry over\.

For one model of each family, GPT\-2 XL, Llama\-2\-13B\-chat, Pythia\-1B, and Qwen3\.5\-9B, we compare the lesion found on the Cinderella task with the lesion found on the sandwich task for the same target \(Table[22](https://arxiv.org/html/2610.28992#A7.T22)\)\. Of the 12 pairs, 9 can be compared; in the other three, the lesion of at least one task does not reach its target\. With the chance level and the test of the previous paragraph, the lesions share heads in 7 of the 9 pairs, and far more than chance in five, all atp<0\.01p<0\.01: 5\.1 to 5\.5 times chance in the three pairs of Llama\-2\-13B\-chat, 2\.4 times in the GPT\-2 XL Wernicke pair, and 2\.6 times in the Qwen3\.5\-9B aphasia pair\. In Llama\-2\-13B\-chat, however, 55 of the 76 shared heads are tied in the rankings of both tasks, and without the heads tied on both tasks, the overlap stays above chance atp<0\.01p<0\.01in four of the five pairs, all but the Llama\-2\-13B\-chat aphasia pair\. Yet the shared heads do not carry the lesions\. Knocked out alone, they reach the target on neither task in any of the 7 pairs with shared heads, and in these pairs neither do the heads that belong to only one of the two lesions\. Knocking out the union of the two lesions reaches the target on both tasks in only one pair, the Pythia\-1B aphasia pair, whose union of 23 heads gives 0\.965 on Cinderella and 0\.953 on sandwich\. Pruning this union for as long as both answers stay above 0\.5 leaves 16 heads, 6 of the Cinderella lesion’s 8 and 10 of the sandwich lesion’s 15\. For 6 of the other 8 unions, the classifier labels both answers model scrambles; for the union of the Llama\-2\-13B\-chat Broca pair, it labels the Cinderella answer model scrambles and the sandwich answer control speech, and for the union of the Pythia\-1B Wernicke pair, it labels the Cinderella answer Broca’s aphasia and the sandwich answer model scrambles\. Of the 18 lesions in the 9 pairs, only one reaches its target on both tasks by itself, the GPT\-2 XL sandwich aphasia lesion, at 0\.859 on Cinderella and 0\.862 on sandwich\. The lesions found on the two tasks thus share heads, but the shared heads never produce the target, and combining the two lesions mostly breaks the answers into model scrambles instead of making both of them aphasic\.

#### The Qwen3\.5 lesions lean toward softmax heads\.

Every fourth layer of a Qwen3\.5 model uses full attention, computed with a softmax, and the other layers use Gated DeltaNet linear attention\([Yang et al\., 2025](https://arxiv.org/html/2610.28992#bib.bib60)\); we call the heads of the full\-attention layers softmax heads\. We count each value head of a Gated DeltaNet layer, which keeps its own recurrent state, as one head, so softmax heads make up 14\.3% of the heads of every Qwen3\.5 model\. Pooled over the 16 Qwen3\.5 lesions, they make up 294 of 1,535 heads, or 19\.2%, against 219\.3 expected at 14\.3%; this pooled share only describes the lesions, because the lesions of one model are not independent draws\. Softmax heads make up more than 14\.3% of the lesion in 15 of the 16 lesions; the exception, the Qwen3\.5\-0\.8B sandwich aphasia lesion, has 5 heads and no softmax head\. A one\-sided hypergeometric test against random sets of the same size, without correction for the 16 tests, finds softmax heads over\-represented atp<0\.05p<0\.05in five lesions: the Qwen3\.5\-9B Cinderella aphasia lesion, with 25 softmax heads of 73 atp=5\.3×10−6p=5\.3\\times 10^\{\-6\}; the Qwen3\.5\-2B sandwich Wernicke lesion, with 18 of 63 atp=7\.2×10−4p=7\.2\\times 10^\{\-4\}; and three lesions of Qwen3\.5\-4B, the Cinderella Broca lesion with 20 of 81 atp=0\.0063p=0\.0063, the sandwich Wernicke lesion with 42 of 228 atp=0\.027p=0\.027, and the sandwich aphasia lesion with 19 of 88 atp=0\.033p=0\.033\. With a Bonferroni correction for the 16 tests, the first two stay below 0\.05\. The lesions thus lean toward the full\-attention layers, although Gated DeltaNet heads make up most of every lesion\.

## Appendix HSteering Details

We next describe how we compute the steering vectors of §[4\.2](https://arxiv.org/html/2610.28992#S4.SS2), what linear probes find in the residual stream of the human transcripts, how the sweep chooses a layer and a strength, which settings the human evaluation uses, three further steering experiments, and how alike the Broca and Wernicke vectors are\.

#### Steering vectors\.

For each model and task, the model reads every human transcript of the task, appended to the task prompt as its own answer: in the plain question\-and\-answer format for the base models, and in the chat template for the chat models, with thinking turned off for Qwen3\.5\. We average the residual stream after every block over the tokens of the transcript, leaving out those of the prompt, which gives the stateshℓ​\(x\)h\_\{\\ell\}\(x\)of Equation \([1](https://arxiv.org/html/2610.28992#S4.E1)\)\. We count layers from 0, so layer 0 is the output of the first block\. We cut the prompt and the transcript together at 2,048 tokens, or at 1,024 for GPT\-2 XL, which shortens 58 of the 1,139 Cinderella transcripts in GPT\-2 XL, 5 in the Llama\-2 models, 3 in the Pythia and Qwen3\.5 models, and none of the sandwich transcripts\. Each text counts once: when AphasiaBank lists the same text more than once, we keep only its first copy, and we drop every copy of a text that it lists under two groups\. GPT\-2 XL and the Pythia models read their residual stream through LayerNorm, which subtracts the mean over the model dimension, so for these models we subtract that mean from every averaged state before any further step\. The Llama\-2 and Qwen3\.5 models use RMSNorm, which does not subtract the mean, so we keep their raw residual stream\. The aphasia vector is the mean ofhℓ​\(x\)h\_\{\\ell\}\(x\)over the transcripts of the four aphasic groups minus the mean over the control transcripts, where the four groups are anomic, conduction, Broca’s, and Wernicke’s aphasia\. We pool the aphasic transcripts rather than average the four group means, so a larger group weighs more\. The Broca and Wernicke vectors subtract the control mean from the mean over the Broca or the Wernicke transcripts\.

#### Linear probes\.

At every layer, we fit a logistic regression with an L2 penalty, at an inverse regularization strength of 0\.01, to the averaged states, which we standardize with the mean and standard deviation of the training folds\. We evaluate each probe by five\-fold cross\-validation, with folds grouped by speaker, so that all transcripts of a speaker fall into one fold, and stratified by label, and we report the mean AUC over the five held\-out folds\. The aphasia probe separates the 744 Cinderella and 408 sandwich transcripts of the four aphasic groups from the 395 and 472 control transcripts\. Unlike the Broca and Wernicke vectors, which compare a type with control speech, the Broca and Wernicke probes separate the transcripts of one type from all other transcripts, including those of the other types of aphasia, so their AUCs measure how well a type stands out from all other speech\. A spot check suggests that the other types make these probes harder: at layer 8 of Pythia\-1B on the Cinderella task, probes against the control transcripts alone separate Broca’s and Wernicke’s aphasia at AUCs of 0\.9999 and 0\.9993, against 0\.945 and 0\.822 for the probes against all other transcripts\. To test whether the two tasks share the aphasia direction, we fit the aphasia probe on all transcripts of one task and test it on all transcripts of the other\. Because 545 speakers occur in both tasks, a high transfer AUC shows that the tasks share a direction, not that the probe generalizes to new speakers\. For severity, we project each aphasic transcript on the unit aphasia vector,𝐯ℓ/‖𝐯ℓ‖\\mathbf\{v\}\_\{\\ell\}/\\\|\\mathbf\{v\}\_\{\\ell\}\\\|, and compute the Spearman correlationρ\\rhobetween the projection and the speaker’s aphasia quotient from the Western Aphasia Battery\([Kertesz, 2007](https://arxiv.org/html/2610.28992#bib.bib20)\), over the 720 of 744 aphasic Cinderella transcripts and the 382 of 408 aphasic sandwich transcripts that come with a quotient\. Lower quotients mean more severe aphasia, so a negativeρ\\rhomeans that the projection grows with severity\. Last, we compute at every layer the first principal component of the averaged states of all transcripts, centered over transcripts, and its absolute cosine with the aphasia vector\.

Table 23:Linear probes and directions in the residual stream that each model computes while it reads the human transcripts of a task as its own answer\. Each AUC column gives the lowest and the highest area under the ROC curve over all layers\.*Aphasia*: transcripts of the four aphasic groups against control transcripts, on speakers held out by five\-fold cross\-validation\.*Broca*and*Wernicke*: transcripts of that type against all other transcripts, including those of the other types of aphasia, evaluated in the same way\.*Transfer*: the aphasia probe fitted on all transcripts of the row’s task and tested on all transcripts of the other task\.*Severity*: the most negative Spearman correlationρ\\rho, over layers, between the projection of an aphasic transcript on the aphasia vector and the speaker’s aphasia quotient, where lower quotients mean more severe aphasia, and the layer at which it occurs, counted from 0\.*PC1*: the absolute cosine between the aphasia vector and the first principal component of all transcripts’ token\-averaged residual streams, averaged over layers\.
#### Probe results\.

Table[23](https://arxiv.org/html/2610.28992#A8.T23)gives these statistics for every model and task\. Over all 11 models and all layers, the aphasia probe reaches an AUC of 0\.960 to 0\.995, the Broca probe 0\.933 to 0\.963, the Wernicke probe 0\.734 to 0\.878, and the transfer 0\.866 to 0\.992\. The most negativeρ\\rhoover layers is weaker on the sandwich task, from−0\.635\-0\.635to−0\.701\-0\.701across models, than on the Cinderella task, from−0\.759\-0\.759to−0\.809\-0\.809\. The first principal component lies close to the aphasia vector: their absolute cosine averages 0\.875 to 0\.956 over the layers of a model and task, and it reaches 0\.955 to 0\.988 at the best layer\. It falls below 0\.8 only in the first three layers of some models on the sandwich task and in the last layer of Llama\-2\-13B\-chat on the same task, down to 0\.086 at layer 1 of Pythia\-6\.9B\. The direction along which the transcripts vary most is thus, at almost every layer, close to the direction that separates aphasic from control speech\.

#### The sweep\.

For each model, task, and vector, the sweep adds the vector of the same task at one of 8 layers,round⁡\(i​L/8\)\\mathrm\{round\}\(iL/8\)fori=0,…,7i=0,\\dotsc,7, whereLLis the model’s number of layers, for example layers 0, 4, …, 28 of a model with 32 layers\. It uses eight strengths toward the target,r∈\{0\.1,0\.25,0\.5,1,1\.5,2,3,4\}r\\in\\\{0\.1,0\.25,0\.5,1,1\.5,2,3,4\\\}, and three toward control speech,r∈\{−0\.5,−1,−2\}r\\in\\\{\-0\.5,\-1,\-2\\\}\. The strengthrrcompares the norm of the added vector withh¯ℓ\\bar\{h\}\_\{\\ell\}, the mean norm of the averaged stateshℓ​\(x\)h\_\{\\ell\}\(x\)over the task’s transcripts, so it measures the vector against the human transcripts’ averaged states, not against the per\-token states of the model’s own answer\. Adding the aphasia vector once, withα=1\\alpha=1, corresponds torrbetween 0\.035 and 0\.508, depending on the model, task, and layer, and at the smallest strength that reaches the target, the sweep adds a median of 5\.77 copies of the aphasia vector\. We add the vector at every position of the prompt and of the answer, and on the Cinderella task also while the model answers “Do you know the story of Cinderella?”, which it answers separately from the story request\. Each setting, a layer and a strength, yields one greedy answer, with a repetition penalty of 1\.25 and at most 180 new tokens, so the sweep has no sampling variance\. Of the 5,808 steered answers, 683 are shorter than 20 characters, and such answers never count as reaching a target\. A vector works at a layer when some positive strength makes the classifier’s probability of the vector’s target exceed 0\.5, and for each model, task, and vector we report how many of the 8 layers work and the median, over the working layers, of the smallest strength that works\. As a control, each model and task gets three random directions, drawn from a standard Gaussian at every layer, centered over the model dimension for GPT\-2 XL and Pythia, and scaled at every layer to the norm of the aphasia vector, so that at the samerrthey add a vector of the same norm\. We add them at the same 8 layers and 8 positive strengths, in 192 settings per model and task, and score them on the aphasia target\.

Table 24:Steering sweep of each model’s answer to the task prompt with the aphasia, Broca, and Wernicke vectors of the same task, at 8 layers and 8 strengths fromr=0\.1r=0\.1to44, whererris the norm of the added vector relative to the mean norm of the transcripts’ token\-averaged residual stream at that layer\.nn: the number of the 8 layers at which some strength makes the classifier’s probability of the vector’s target exceed 0\.5\.rr: the median, over these layers, of the smallest such strength, or a dash where no layer works\.*Random*: the number of the 192 settings, three random directions with the aphasia vector’s norm at the same 8 layers and 8 strengths, at whichP⁡\(aphasia\)P\(\\mathrm\{aphasia\}\)exceeds 0\.5\. The last row sumsnnover the 11 models, of 88 layers per task, takes the median smallest strength over all working layers, and sums the random settings, of 2,112 per task\.![Refer to caption](https://arxiv.org/html/2610.28992v1/app_steering_sweep.png)Figure 7:P⁡\(aphasia\)P\(\\mathrm\{aphasia\}\)of the answer to the task prompt when the aphasia vector of the same task is added at one of 8 layers, counted from 0, with one of 8 strengthsrr, in the largest model of each family\. The 8 strengths run fromr=0\.1r=0\.1to44, whererris the norm of the added vector relative to the mean norm of the human transcripts’ token\-averaged residual stream at that layer\. The middle tick of the color bar, 0\.5, is the threshold above which an answer reaches the aphasia target\. A dot marks answers whose most probable class is model scrambles, and hatched cells mark answers shorter than 20 characters, which are not scored\. The unsteered Cinderella answers of GPT\-2 XL and Qwen3\.5\-9B already have model scrambles as their most probable class\.
#### Sweep results\.

Table[24](https://arxiv.org/html/2610.28992#A8.T24)gives the sweep’s results for every model and task, and Figure[7](https://arxiv.org/html/2610.28992#A8.F7)showsP⁡\(aphasia\)P\(\\mathrm\{aphasia\}\)for every setting of the aphasia vector in the largest model of each family\. Over both tasks, the aphasia vector works at 103 of the 176 layers: at 54 of 64 in the Pythia models, at a median strength ofr=0\.5r=0\.5; at 28 of 64 in the Qwen3\.5 models, at 1\.5; at 15 of 32 in the Llama\-2 models, at 2; and at 6 of 16 in GPT\-2 XL, at 1\.5\. The Broca vector works at 78 of the 176 layers and the Wernicke vector at 25, and the Wernicke vector never works in GPT\-2 XL or the Llama\-2 models\. Random directions reach the aphasia target in 119 of the 4,224 settings, or 2\.8%: in 6\.25% of the settings in the Pythia models, 1\.8% in GPT\-2 XL, and less than 1% in the Llama\-2 and Qwen3\.5 models\. At every strength, the aphasia vector reaches its target more often than random directions, for example in 41 of the 176 settings atr=1r=1, against 27 of the 528 random settings\. At a given layer, the strengths that work lie between weaker ones, which leaveP⁡\(aphasia\)P\(\\mathrm\{aphasia\}\)low, and stronger ones, which often turn the answer into model scrambles or into an answer too short to score \(Figure[7](https://arxiv.org/html/2610.28992#A8.F7)\)\. Pushes toward control speech rarely reach a target: of the 528 settings with a negative strength, 13 reach the aphasia target, 3 the Broca target, and none the Wernicke target\. Without steering,P⁡\(aphasia\)P\(\\mathrm\{aphasia\}\)is at most 0\.017, except for Pythia\-6\.9B on the Cinderella task at 0\.167, although the classifier’s most probable class for the unsteered Cinderella answers of GPT\-2 XL, Pythia\-2\.8B, and the four Qwen3\.5 models is already model scrambles\. Unlike the lesion search, the sweep does not require the model to say that it knows the story of Cinderella, and steering changes that answer too\. Of the Cinderella settings that reach their target, 31 of 122 for aphasia, 23 of 89 for Broca, and 2 of 22 for Wernicke answer no to “Do you know the story of Cinderella?”, and 17 of these 31 aphasia settings come from Pythia\-6\.9B, which answers no even without steering\.

Table 25:Steering settings of the human evaluation, for the seven models whose steered answers two experts labeled\. For each model, source task, and target, the vector comes from the transcripts of the source task, and the setting is the smallest strengthrrat which the target probability of the steered answer to the source task exceeds 0\.5 in the sweep, at the layer, counted from 0, with the highest probability at that strength\.PP: the classifier’s probability of the target for that answer\.†No setting of the sweep passes 0\.5, so the setting with the highest probability is used\.
#### Settings for the human evaluation\.

The human evaluation uses one setting for each of the seven models whose answers the experts labeled, each source task, and each target \(Table[25](https://arxiv.org/html/2610.28992#A8.T25)\)\. Among the positive strengths whose answer to the source task is long enough to score, we take the smallest strength at which the target probability exceeds 0\.5, and if several layers pass 0\.5 at that strength, the layer with the highest probability\. For the Wernicke target in GPT\-2 XL and the two Llama\-2 models, no setting passes 0\.5 on either task, so we take the setting with the highestP⁡\(Wernicke\)P\(\\mathrm\{Wernicke\}\), which ranges from 0\.011 to 0\.422\. The chosen setting then steers the answers to all seven prompts, without the question about knowing the story, with greedy decoding, a repetition penalty of 1\.25, and at most 180 new tokens for the two narratives and 120 for the five TAB prompts\. On the source task, the classifier gives these answers the same probabilities as the sweep’s answers in all 42 settings\. Appendix[I](https://arxiv.org/html/2610.28992#A9)describes how the experts labeled these answers\.

Figure 8:The models’ answers among the transcripts and model scrambles of Figure[6](https://arxiv.org/html/2610.28992#A2.F6), in one t\-SNE per task fitted on all of them together\. Each panel shows the intact answers of the 11 models as white circles, and either their answers lesioned byCLAMPtoward Broca’s or Wernicke’s aphasia, as triangles in the top row, or their answers steered toward these types, as diamonds in the bottom row\. A hollow marker is an answer that does not reach its target\.
#### Steered Broca answers lie among Broca transcripts, lesioned ones near the model scrambles\.

We place the models’ answers in the text space of Appendix[B](https://arxiv.org/html/2610.28992#A2): each model’s intact answer to the task prompt, the answers of itsCLAMPlesions for the Broca and Wernicke targets, and its answers steered toward the two types with the task’s own vector, at the setting that the rule above chooses, for all 11 models \(Figure[8](https://arxiv.org/html/2610.28992#A8.F8)\)\. For each answer, we count the groups of its 10 nearest texts among the transcripts and model scrambles\. The intact answers lie among control transcripts, which make up 82% and 92% of their nearest texts on the Cinderella and the sandwich task\. Answers steered toward Broca’s aphasia have 57% and 43% Broca transcripts among their nearest texts, against 21% and 11% of all texts, and only 11% and 12% model scrambles\. Answers lesioned toward Broca’s aphasia have 53% and 37% model scrambles among their nearest texts, against 13% and 9% of all texts, and only 33% and 15% Broca transcripts\. The lesions that reach the Broca target are no exception: the answers of the 7 such Cinderella lesions have 57% model scrambles among their nearest texts\. The Wernicke answers of both methods lie between groups, but Wernicke transcripts, 4% and 3% of all texts, make up 26% and 18% of the nearest texts of the steered answers and 12% and 26% of those of the lesioned answers\. In a space that knows nothing of our labels, steering toward Broca’s aphasia thus moves the answer toward Broca’s speech, whereas lesioning moves it toward the machine failures that the classifier learns to set apart\. The encoder may, however, place short and fragmented speech near fragmented machine output, which the classifier’s fine\-tuning separates, so this map checks the classifier’s labels rather than replacing them\.

#### Continuations of human transcripts\.

Steering the model’s own answer cannot show whether subtracting the aphasia direction moves speech toward control speech, because the unsteered answers are already far from aphasic\. We therefore start the answer with the first 25 words of a human transcript and let the model continue it for 120 new tokens, while we add the aphasia vector at layerround⁡\(L/4\)\\mathrm\{round\}\(L/4\)withr∈\{−2,−1,0,1,2\}r\\in\\\{\-2,\-1,0,1,2\\\}, and we score only the continuation\. Each task uses the same 20 speakers in all 11 models: 10 control speakers and 10 speakers with aphasia, of whom 3 have anomic, 3 Broca’s, 2 conduction, and 2 Wernicke’s aphasia\. Adding the direction makes the continuations aphasic: of the 208 continuations of control speakers that the classifier scores but does not call aphasic without steering, 206 are long enough to score atr=1r=1, and 153 of these become aphasic, although in GPT\-2 XL only 3 of 18 do, and its continuations become model scrambles instead\. Subtracting the direction, however, does not make the continuations of aphasic speakers control\-like\. Without steering, the classifier calls only 41 of the 220 continuations of aphasic speakers aphasic\. Of the 40 of these that are long enough to score atr=−1r=\-1, subtracting the direction makes 32 no longer aphasic, but only 10 of them become control speech by the classifier’s most probable class, while 22 become model scrambles\. Atr=−2r=\-2, subtracting the direction even turns 50 of the 208 control continuations that are scored and not aphasic without steering into aphasic ones, and the classifier calls 105 of the 220 control continuations model scrambles\.

#### Cinderella↔\\leftrightarrowsandwich transfer of aphasia vector\.

The aphasia vector of one task also steers the answers to the other task\. We add the other task’s vector at the same 8 layers and positive strengths, scaled so that it has the norm that the task’s own vector would have at the samerr\. It works at 98 of the 176 layers, against 103 for the task’s own vector, and in 234 of the 1,408 settings, against 224, although the cosine between the two tasks’ aphasia vectors ranges from 0\.265 to 0\.936 over models and layers\. It works at 54 of 64 layers in the Pythia models, as many as the task’s own vector, and at slightly fewer layers than the task’s own vector in the other families\.

#### Type\-specific vectors\.

The Broca and Wernicke vectors point in nearly the same direction as the aphasia vector, with cosines of 0\.838 to 0\.989 for the Broca vector and 0\.548 to 0\.975 for the Wernicke vector over models, tasks, and layers\. To steer toward what sets a type apart from aphasia in general, we remove from the Broca and Wernicke vectors their component along the aphasia vector at every layer and sweep these type\-specific vectors as above\. By the classifier, the Wernicke\-specific vector works at 94 of the 176 layers, against 25 for the full Wernicke vector, and in all 22 pairs of model and task, including the 10 in which the full vector never works\. Among these 10 are both tasks of GPT\-2 XL and of the two Llama\-2 models, where the Wernicke\-specific vector needs median strengths of 1\.5 to 4\. The Broca\-specific vector, in contrast, works at only 29 layers, against 78 for the full Broca vector\. The human evaluation uses the full vectors, so its Wernicke answers from GPT\-2 XL and the Llama\-2 models come from settings that do not pass 0\.5, even though the Wernicke\-specific vector passes 0\.5 in these models\.

Table 26:Cosine similarity of steering vectors, as the median over a model’s layers\.*Broca against Wernicke*: the Broca and the Wernicke vector of the same task\.*Same type, across tasks*: the Cinderella and the sandwich vector of one type\.*Shuffled labels*: the Broca and the Wernicke vector after shuffling the two labels among the transcripts of both types, keeping the group sizes, averaged over 200 shuffles\.*Below*: layers at which the observed Broca\-against\-Wernicke cosine lies below the 2\.5th percentile of the shuffled cosines, over both tasks\.*Type\-specific parts*: the cosine of the Broca and the Wernicke vector after projecting the aphasia vector out of both\.
#### The Broca and Wernicke vectors share most of their length\.

For every model, task, and layer, we compute the cosine similarity of the Broca and the Wernicke vector of the same task and compare it with two references \(Table[26](https://arxiv.org/html/2610.28992#A8.T26)\)\. The first reference is the same type’s vector from the other task, which shows how much a vector changes when only the task changes\. The second reference shuffles the Broca and Wernicke labels among the transcripts of the two types, 200 times, keeping the group sizes, and recomputes both vectors\. Within a task, the Broca and the Wernicke vector have a median cosine over layers of 0\.79 to 0\.90 in the 22 pairs of a model and a task, and both lie close to the aphasia vector, at median cosines of 0\.973 for the Broca vector and 0\.915 for the Wernicke vector over models, tasks, and layers\. The same type’s vectors from the two tasks have median cosines of only 0\.72 for Broca and 0\.66 for Wernicke, lower than those of the Broca and Wernicke vectors of one task, although this comparison favors the pair from one task, whose two vectors share the control transcripts and the prompt\. Shuffled labels give median cosines of 0\.97 to 0\.99, and the observed cosine falls below the 2\.5th percentile of the shuffled cosines at 670 of the 672 layers, at every layer except the first two of Llama\-2\-13B\-chat on the sandwich task\. With the aphasia vector projected out of both, the remaining parts of the Broca and Wernicke vectors have median cosines of−0\.65\-0\.65to−0\.41\-0\.41, so the parts that set the two types apart point away from each other\. The Wernicke part carries its type: the Wernicke\-specific vector works in all 22 pairs, as the previous paragraph shows, whereas the Broca\-specific vector works in only 15\. The two types thus share most of their deviation from control speech, while each keeps a small part of its own that steering can isolate\.

## Appendix IHuman Evaluation of Steered Answers

We describe the annotation sheets on which two experts labeled the steered answers, defines the agreement measures of Table[4](https://arxiv.org/html/2610.28992#S5.T4), and breaks the results of §[5\.2](https://arxiv.org/html/2610.28992#S5.SS2)down by expert, class, model, prompt, and steering vector\.

#### The annotation sheets\.

We steer seven models, GPT\-2 XL, Llama\-2\-7B\-chat, Llama\-2\-13B\-chat, Pythia\-1B, Pythia\-1\.4B, Pythia\-2\.8B, and Pythia\-6\.9B, toward each of the three targets with each of two steering vectors, one computed from the Cinderella transcripts and one from the sandwich transcripts\. Each steered model uses the layer and strength that the sweep of §[4\.2](https://arxiv.org/html/2610.28992#S4.SS2)chose for its target on the vector’s own task, the source task \(Appendix[H](https://arxiv.org/html/2610.28992#A8)\), and answers seven prompts once\. The seven prompts are the requests of the two tasks and the five connected\-text prompts of the TAB: “Tell me about the best trip you ever took”, “Describe a happy childhood memory”, “Tell me about your first job”, “What do you like about where you live?”, and “Describe the steps to make a simple meal”\. The models decode as in §[3](https://arxiv.org/html/2610.28992#S3), except that they write at most 120 new tokens for the TAB prompts\. Seven models, seven prompts, two steering vectors, and three targets give 294 steered answers\. The answers steered toward aphasia form the two\-way sheet, on which the experts label each answer as aphasic or not\. The answers steered toward Broca’s or Wernicke’s aphasia form the five\-way sheet, on which the experts give each answer one of the classifier’s five classes: control speech, conduction or anomic aphasia, Broca’s aphasia, Wernicke’s aphasia, or model scrambles\. We leave 22 answers out of the analysis, 7 of the 98 on the two\-way sheet and 15 of the 196 on the five\-way sheet: 18 answers with fewer than 20 characters, 8 of them empty, which the classifier does not score \(§[3](https://arxiv.org/html/2610.28992#S3)\), and 4 answers for which the LLM judge returned an empty reply\. The analysis thus covers 272 answers, 91 on the two\-way sheet and 181 on the five\-way sheet, of which 90 are steered toward Broca’s aphasia and 91 toward Wernicke’s aphasia\.

#### Expert labels\.

We take expert 1’s labels from the annotation sheets, and expert 2’s from a tab of the sheets in which expert 2 labeled the answers before the experts compared notes\. The sheets also hold a later version of expert 2’s labels, set after the experts compared notes, and on all 46 answers where the two versions differ, the later version equals expert 1’s label\. With the later version, the experts would agree on 96% of the answers, at a Cohen’sκ\\kappaof 0\.95, rather than on 80%, at aκ\\kappaof 0\.76\. Expert 2’s tab has no label for 4 answers, 2 on each sheet, so we compare these answers with expert 1 alone and leave them out of every statistic that needs both experts\. The two\-way sheet has no label for model scrambles, so an answer on it counts as a model scramble when the experts’ comments mark it as one, which holds for 25 of its 91 answers\. On the five\-way sheet, an answer counts as a model scramble when either expert labels it so, which holds for 52 of its 181 answers\.

#### Agreement measures\.

The classifier and the LLM judge label the same answers as the experts\. On the two\-way sheet, the classifier labels an answer aphasic whenP⁡\(aphasia\)P\(\\mathrm\{aphasia\}\)exceeds 0\.5, and the judge answers yes or no to whether the speaker has aphasia\. On the five\-way sheet, the classifier gives its most probable class, and the judge names one of the five classes \(Appendix[K](https://arxiv.org/html/2610.28992#A11)\)\. A labeler agrees with the experts on an answer when its label matches the label of either expert, which is the agreement of Table[4](https://arxiv.org/html/2610.28992#S5.T4)\. As a baseline, the best constant label gives every answer of a sheet the one label that agrees with the experts most often: not aphasic on the two\-way sheet and Broca’s aphasia on the five\-way sheet\. It agrees with the experts on 63% of the two\-way answers, 39% of the five\-way answers, and 47% of all answers, against 73%, 62%, and 65% for the classifier and 64%, 51%, and 55% for the judge\. Cohen’sκ\\kappacorrects the agreement of two labelers for the agreement expected by chance\([Cohen, 1960](https://arxiv.org/html/2610.28992#bib.bib8)\):κ=\(po−pe\)/\(1−pe\)\{\\kappa=\(p\_\{o\}\-p\_\{e\}\)/\(1\-p\_\{e\}\)\}, wherepop\_\{o\}is the share of answers on which the two agree andpep\_\{e\}the share on which two independent labelers with the same label frequencies would agree\. By the scale of[Landis and Koch \(1977\)](https://arxiv.org/html/2610.28992#bib.bib22), aκ\\kappafrom 0\.21 to 0\.40 is fair agreement, from 0\.41 to 0\.60 moderate, from 0\.61 to 0\.80 substantial, and above 0\.80 almost perfect\. On all answers,κ\\kappapools the two sheets and treats their labels as one set of seven\.

Table 27:Agreement of the classifier and the LLM judge with each of the two experts and with the experts’ consensus, in percent, on the steered answers\.*Two\-way*: answers steered toward aphasia, labeled as aphasic or not\.*Five\-way*: answers steered toward Broca’s or Wernicke’s aphasia, labeled with the classifier’s five classes\.*All*: the answers of both sheets together, on which Cohen’sκ\\kappatreats the labels of the two sheets as one set of seven\.*Consensus*: the answers to which both experts give the same label\. Comparisons with expert 2 leave out the 4 answers that expert 2 did not label\. Bold marks the higher agreement of the classifier and the LLM judge\.
#### Agreement with each expert\.

Table[27](https://arxiv.org/html/2610.28992#A9.T27)compares the classifier and the judge with each expert separately and with the experts’ consensus, the answers to which both experts give the same label\. The classifier agrees more with expert 2 than with expert 1, on 62% against 53% of all answers, atκ=0\.54\\kappa=0\.54and0\.440\.44, whereas the judge agrees with the two about equally, on 50% and 48%, atκ=0\.39\\kappa=0\.39and0\.380\.38\. By the scale of[Landis and Koch \(1977\)](https://arxiv.org/html/2610.28992#bib.bib22), the classifier’s agreement with each expert is moderate and the judge’s is fair, whereas the experts’ agreement with each other, atκ=0\.76\\kappa=0\.76, is substantial\. On the 214 answers on which the experts agree, the classifier gives their label to 62% and the judge to 54%\. The classifier and the judge agree with each other on only 42% of all answers, atκ=0\.31\\kappa=0\.31, and on the two\-way sheet atκ=0\.00\\kappa=0\.00, which is no better than chance\.

#### Model scrambles\.

The classifier recognizes model scrambles much better on the two\-way sheet than on the five\-way sheet\. On the two\-way sheet, the classifier’s label shows only whether an answer is aphasic, so we use its most probable class: it puts 27 answers in the model\-scramble class, 20 of which the experts’ comments mark as model scrambles, a precision of 74%, and it finds 20 of the 25 marked answers, a recall of 80%\. On the five\-way sheet, its precision is 52%, 16 of 31, and its recall 31%, 16 of 52, and over both sheets, its precision is 62% and its recall 47%\. The judge can name model scrambles only on the five\-way sheet, where it calls 4 answers model scrambles, 2 of them correctly, a precision of 50% and a recall of 4%\. On the two\-way sheet, the judge calls 20 of the 25 model scrambles aphasic, whereas the classifier’s label calls 5 of them aphasic\. Of the 52 five\-way answers that either expert calls model scrambles, the classifier labels 30 as Broca’s aphasia, and the judge labels 28 as Wernicke’s aphasia and 19 as Broca’s aphasia\. Both experts call 22 of these answers model scrambles, and the classifier finds 8 of them and labels the other 14 as Broca’s aphasia\. On the source task, the classifier calls all 14 answers steered toward Broca’s aphasia Broca\-like, because its own score chose their steering settings, whereas the experts call 9 of them model scrambles\. The experts disagree with each other mostly about model scrambles\. Of their 43 disagreements on the five\-way sheet, 29 involve model scrambles, and in 27 of these, expert 1 labels a model scramble where expert 2 labels a type of aphasia: Broca’s aphasia 18 times, Wernicke’s aphasia 8 times, and conduction or anomic aphasia once\. Overall, expert 1 labels 50 of the 181 five\-way answers as model scrambles, and expert 2 labels 24 of the 179 that expert 2 labeled\. On the two\-way sheet, the experts disagree on 11 of the 89 answers that both labeled\.

Table 28:Precision and recall of each class, in percent, for the classifier and the LLM judge on the 181 answers steered toward Broca’s or Wernicke’s aphasia\.*Experts*: the answers that either expert puts in the class; an answer on which the experts disagree counts in both of their classes\.*Calls*: the answers to which the classifier or the judge gives the class\.*Prec\.*: precision, the share of the calls that either expert shares\.*Rec\.*: recall, the share of the experts’ answers in the class that the classifier or the judge also puts there\. Bold marks the higher value of the two\.Table 29:Labels of the classifier against those of expert 2 on the 179 answers steered toward Broca’s or Wernicke’s aphasia that expert 2 labeled\. Rows give expert 2’s label and columns the classifier’s, and the bold diagonal counts the 102 answers on which the two agree\.*Ctrl\.*: control speech;*Cond\.*: conduction or anomic aphasia;*Wern\.*: Wernicke’s aphasia;*MS*: model scrambles\.
#### Agreement by class\.

On the five\-way sheet, the classifier is more precise than the judge for every class except control speech, and it recalls more answers of every class except Wernicke’s aphasia \(Table[28](https://arxiv.org/html/2610.28992#A9.T28)\)\. The judge recalls 57 of the 64 answers that either expert labels as Wernicke’s aphasia, 89%, but it gives that label to 111 of the 181 answers, and it recalls only 23 of the 70 Broca answers, 33%\. Against expert 2, the classifier’s most frequent errors are 15 model scrambles that it labels as Broca’s aphasia, 14 Wernicke answers that it labels as conduction or anomic aphasia, and 11 Wernicke answers that it labels as model scrambles \(Table[29](https://arxiv.org/html/2610.28992#A9.T29)\)\. Against expert 1, who calls more answers model scrambles, the classifier labels 29 of expert 1’s 50 model scrambles as Broca’s aphasia, and it agrees with expert 1 on 87 of 181 answers, 48%, against 102 of 179, 57%, with expert 2\.

Table 30:Steering success by model: the number of steered answers that either expert labels with the steering target, out of the model’s labeled answers for that target, without and with the answers that the experts call model scrambles \(MS\)\. Each model has at most 14 answers per target, to 7 prompts with 2 steering vectors, one from the Cinderella and one from the sandwich transcripts, and loses the answers that are too short to score or that the LLM judge left without a reply\. An answer that one expert labels with the target and the other calls a model scramble counts as a success with MS and is left out without MS, so a model can succeed more often with MS\. Over all models, either expert labels the target for 59%, 68%, and 63% of the answers steered toward aphasia, Broca’s aphasia, and Wernicke’s aphasia without MS\.Table 31:Steering success by prompt: the number of steered answers that either expert labels with the steering target, out of the labeled answers to the prompt for that target, without and with the answers that the experts call model scrambles \(MS\)\. The first two prompts are those of the Cinderella and sandwich tasks, and the other five are the connected\-text prompts of the Text Aphasia Battery\. Each prompt has at most 14 answers per target, from 7 models with 2 steering vectors, one from the Cinderella and one from the sandwich transcripts\. For the Cinderella and sandwich prompts, 7 of these 14 come from the vectors of the same task, whose answers chose the steering layer and strength\. An answer that one expert labels with the target and the other calls a model scramble counts as a success with MS and is left out without MS\.
#### Steering success by model, prompt, and vector\.

Steering toward Wernicke’s aphasia reaches the target mostly in the Pythia models, for 40 of their 50 answers that are not model scrambles, against 8 of 26 in GPT\-2 XL and the Llama\-2 models, for which no Wernicke setting passed 0\.5 on the source task \(Table[30](https://arxiv.org/html/2610.28992#A9.T30)\)\. Steering toward Broca’s aphasia, in contrast, fails in Pythia\-1B and Pythia\-6\.9B, for 0 of 4 and 1 of 7 such answers, and 10 of the 14 Broca answers of Pythia\-1B are model scrambles\. Over all seven models, steering toward Broca’s aphasia breaks the most answers: the experts call 37 of its 90 answers model scrambles, against 25 of 91 for aphasia and 15 of 91 for Wernicke’s aphasia\. By prompt, steering reaches the target most often on the Cinderella prompt, for 22 of 26 answers that are not model scrambles, and least often on the prompts about the best trip and about where the speaker lives, for 15 of 31 and 11 of 25 \(Table[31](https://arxiv.org/html/2610.28992#A9.T31)\)\. Steering vectors computed from the sandwich transcripts reach the target more often than those computed from the Cinderella transcripts\. Without model scrambles, either expert labels the target for 79%, 84%, and 74% of the answers steered toward aphasia, Broca’s aphasia, and Wernicke’s aphasia with sandwich vectors, against 45%, 54%, and 54% with Cinderella vectors\. With model scrambles, the gap narrows to 56%, 60%, and 66% against 43%, 55%, and 51%, and toward aphasia, sandwich vectors produce more model scrambles, 17 of 45 answers against 8 of 46\.

#### Steering carries its type to the prompts that did not choose its setting\.

The sweep chose each steering setting on the source task alone, so the answers to the other narrative and to the five TAB prompts test whether the targeted type carries over\. Without model scrambles, either expert labels the target for 5 of the 5 Broca answers and 8 of the 12 Wernicke answers to the source task, and for 31 of the 48 Broca answers, 65%, and 40 of the 64 Wernicke answers, 63%, to the other six prompts\. With model scrambles, the labels on these six prompts follow the steering vector: either expert labels 44 of the 76 answers steered toward Broca’s aphasia as Broca’s aphasia and only 9 as Wernicke’s aphasia, and 44 of the 77 answers steered toward Wernicke’s aphasia as Wernicke’s aphasia and only 13 as Broca’s aphasia\. The two vectors thus produce different types on prompts that played no part in choosing them\.

Model and vectorPromptClassifierLLM judgeAnswerSteered toward aphasia, labeled aphasic by both expertsGPT\-2 XL, Cinderella vectorCinderella†Aphasic \(0\.870\.87\)AphasicI don,t know that I like to tell it\. It’s a good thing I didn\. / I was in my\. \. \.Steered toward Broca’s aphasia, labeled Broca by both expertsLlama\-2\-13B\-chat, Cinderella vectorSimple mealModel scrambles \(0\.390\.39\)BrocaSure\! Here are the steps to make a simple meal: / First, choose a recipe you like and ingredients list need \. go shopping groc yes\. Next, preheat oven or stove top cooking no\. Then, chop vegetables car yes\. After that, cook meat dish yes\. Finally, serve hot enjoy eat\.Llama\-2\-7B\-chat, Cinderella vectorFirst jobBroca \(0\.710\.71\)Brocasure\! my job is good\. okay\.Steered toward Wernicke’s aphasia, labeled Wernicke by both expertsLlama\-2\-7B\-chat, Cinderella vectorFirst jobWernicke \(0\.600\.60\)Conduction or anomicsure, my first job was a lot of fun\. I like you know what it’s good for\. It’s good for you\.Pythia\-1B, Cinderella vectorCinderella†Wernicke \(0\.820\.82\)WernickeI think it’s a very good one, and that is what we are doing now in this country\. We have been trying to do something for years\. It was not so long ago when they were saying about the other thing\. But there is no reason why we should be afraid because if he will say yes or no? Yes\. No\. And then also with him\. So yeah\. Yeah\.Steered toward Broca’s aphasia, called model scrambles by both expertsLlama\-2\-7B\-chat, Cinderella vectorCinderella†Broca \(0\.960\.96\)Brocasure\! The story of Cinder\. / Cinderelllll\. / Okay, let’s go\. / okay okay\. / Sure\. / okay\. / okay\. / okay\. / okay\. / okay\. / okay\. / okay\. ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\. / ok\.Pythia\-6\.9B, sandwich vectorCinderellaBroca \(0\.940\.94\)BrocaQuestion\[171 line breaks\]yes / noTable 32:Steered answers, verbatim, with the labels of the classifier and the LLM judge\. In the first five, both experts label the answer with the steering target; in the last two, both experts call the answer a model scramble, while the classifier labels it Broca\.*Vector*: the task whose transcripts gave the steering vector, and whose answers chose the steering layer and strength;†marks an answer to that task, the source task\.*Prompt*: the Cinderella or sandwich task, or the topic of a connected\-text prompt of the Text Aphasia Battery, “Describe the steps to make a simple meal” or “Tell me about your first job”\. The classifier’s probability for its label follows the label; for the answer steered toward aphasia it isP⁡\(aphasia\)P\(\\mathrm\{aphasia\}\)\. A slash marks one or more line breaks, and the bracketed text replaces 171 line breaks\.
#### Examples\.

Table[32](https://arxiv.org/html/2610.28992#A9.T32)shows seven steered answers\. In the first five, both experts label the answer with the steering target\. The two answers of Llama\-2\-7B\-chat to the prompt about its first job show how the targets differ: steered toward Broca’s aphasia, it answers in six words, and steered toward Wernicke’s aphasia, it answers fluently but with little content\. In the last two, both experts call the answer a model scramble, while the classifier and the judge both label it as Broca’s aphasia: one ends in a loop of “okay\.” and “ok\.”, and the other is the word “Question”, 171 line breaks, “yes”, and “no”\. The first of the two is an answer to the source task, so the classifier’s score for it,P⁡\(Broca\)=0\.96P\(\\mathrm\{Broca\}\)=0\.96, is the score with which the sweep chose the steering setting that produced it\.

## Appendix JSteering Back

We describe the steering\-back test of §[4\.2](https://arxiv.org/html/2610.28992#S4.SS2)and reports its results\. The test asks whether removing a lesion’s own direction from the lesioned model, or adding the direction with a negative strength, undoes the lesion, and whether random directions do the same\. Random directions undo the lesions almost as often as the own direction, so the direction of a type does not reverse a lesion specifically\. Most undone answers are model scrambles rather than the intact model’s speech, so steering back undoes the lesions without healing them\.

#### Random directions serve as controls\.

We test the 54CLAMPlesions that reach their target \(Figure[4](https://arxiv.org/html/2610.28992#S5.F4)\)\. An answer reaches the target when the classifier’s target probability exceeds 0\.5 or when the target is its most probable class, where any of the three aphasia classes counts for the aphasia target\. A lesion’s own direction is the steering vector𝐯ℓ\\mathbf\{v\}\_\{\\ell\}of its target \(Equation \([1](https://arxiv.org/html/2610.28992#S4.E1)\)\), computed on the task where the search found the lesion\. For the aphasia target, the own direction is the aphasia vector of Appendix[H](https://arxiv.org/html/2610.28992#A8), which pools the transcripts of the four aphasic groups\. The controls are three Gaussian random directions, and the same three serve the lesions of all three targets of a model and task\. For each lesion, the model answers the prompt of that task under 135 conditions, with the greedy decoding and the scoring of §[3](https://arxiv.org/html/2610.28992#S3)\. Seven answers are references: those of the intact model, of the intact model with the own direction removed, of the lesioned model, of the lesioned model with the own direction removed, and of the lesioned model with each of the three random directions removed\. The other 128 answers come from pushes, one for each combination of four directions, eight layers, and four strengths\. A push adds the own direction or one of the three random directions to the lesioned model after one of eight layers spread across the model’s depth, with a strength ofr=−0\.5r=\-0\.5,−1\-1,−2\-2, or−4\-4\. We call a push with the own direction an own push, and a push with a random direction a random push\. If a lesion acts through its own direction, steering back along that direction should undo it more often than steering back along a random direction\.

#### Pushes match the directions in size, but removal does not\.

Following[Arditi et al\. \(2024\)](https://arxiv.org/html/2610.28992#bib.bib2), removal projects a direction’s unit vector out of the residual stream after every block \(§[4\.2](https://arxiv.org/html/2610.28992#S4.SS2)\)\. Removal takes out whatever part of the residual stream lies along the direction, so the amount that it takes out depends on the direction, and we did not record this amount for either kind of direction\. A push adds a multiple of the direction after one layerℓ\\ell, scaled so that the added vector has the norm\|r\|​h¯ℓ\|r\|\\,\\bar\{h\}\_\{\\ell\}, withrrandh¯ℓ\\bar\{h\}\_\{\\ell\}as in §[4\.2](https://arxiv.org/html/2610.28992#S4.SS2)\. The added vector thus has the same norm for every direction, so pushes compare the own and the random directions at equal size\. A removal or a push undoes the lesion when its answer no longer reaches the target, that is, when its target probability is 0\.5 or below and the target is not its most probable class\. An answer shorter than 20 characters does not count by the rules of §[3](https://arxiv.org/html/2610.28992#S3), so we do not score it, and such an answer does not undo the lesion\.

Figure 9:Mean target probability that steering back leaves in the 54CLAMPlesions, for the lesion’s own direction and for three random directions\.*Removed*: removal of the direction from the residual stream after every block\.rr: the strength of a push, which adds the direction after one of eight layers; own and random pushes add vectors of the same norm\. Each mean pools the lesions, the layers of a push, and the random directions, over the answers long enough to score\. Before steering back, the lesioned answers have a mean target probability of 0\.711, and a lesion is undone when its answer no longer reaches the target: its target probability falls to 0\.5 or below and the target is not its most probable class\.Table 33:Removing a direction after every block of the 54CLAMPlesions, by model family and by target\.*Removal undoes*: the number of lesions whose answer no longer reaches the target, a target probability of 0\.5 or below with the target not the most probable class, when the own direction is removed, and when at least one, or all three, of three random directions are removed\.*Probability left*: the mean target probability after removal, against 0\.711 before it\.†With its own direction removed, the remaining lesion, of Llama\-2\-13B\-chat on the sandwich task with the Wernicke target, gives an answer too short to score\.
#### Own and random directions both undo the lesions\.

Before steering back, the lesioned answers have a mean target probability of 0\.711, and the intact answers have a mean of 0\.006\. Removing the own direction undoes 53 of the 54 lesions, and the remaining lesion, the Llama\-2\-13B\-chat sandwich Wernicke lesion, gives an answer too short to score \(Table[33](https://arxiv.org/html/2610.28992#A10.T33)\)\. The three random removals undo 51, 50, and 50 of the 54 lesions\. At least one of them undoes each of the 54 lesions, and all three undo 45 of them\. The own direction leaves less of the target probability: after removal, a mean of 0\.010, against 0\.081 for the random directions \(Figure[9](https://arxiv.org/html/2610.28992#A10.F9)\)\. Removal of the own direction leaves less than random removal in every model family and target except the Llama\-2 models, where the two leave 0\.029 and 0\.028\. Pushes, which match the directions in size, keep this order only at weak strengths, with 0\.024 against 0\.078 atr=−0\.5r=\-0\.5and 0\.034 against 0\.072 atr=−1r=\-1, and lose it at strong ones, with 0\.055 against 0\.047 atr=−2r=\-2and 0\.020 for both atr=−4r=\-4\. At every strength and at every one of the eight layers, most pushes undo the lesion, own pushes slightly more often than random ones: 92 to 97% against 89 to 94% across the strengths, and 87 to 99% against 83 to 96% across the layers\. Over all strengths and layers, 1,643 of the 1,728 own pushes and 4,720 of the 5,184 random pushes undo the lesion, or 95% and 91%\. A random direction thus undoes a lesion almost as often as the lesion’s own direction, so undoing a lesion does not show that the lesion acts through the direction of its type\.

#### The undone answers are not the intact answers\.

Undoing a lesion asks only that the answer no longer reach the target, so an undone answer need not be the intact model’s speech\. No removal or push reproduces the intact answer: none of the 54 own removals, 162 random removals, 1,728 own pushes, and 5,184 random pushes yields the intact text\. Removing the own direction undoes the lesions mostly by breaking the answers\. The classifier’s label, its most probable class, is model scrambles for 52 of the 53 undone answers and control speech for none of them\. The remaining answer, of the 276\-head Llama\-2\-7B\-chat Cinderella Wernicke lesion, stays aphasic, and its label moves from Wernicke’s aphasia to conduction or anomic aphasia\. To check such labels without the classifier, we call an answer fluent when it passes two tests\. First, compressing the answer with zlib must leave at least a quarter of its length, which rules out answers that loop\. Second, at least 80% of its non\-space characters must belong to words of an American English word list\. This sense of fluent concerns broken text, not the clinical fluency of Appendix[A](https://arxiv.org/html/2610.28992#A1)\. The intact answers of all 54 lesions are fluent by this check, but 35 of the 53 undone answers of the own removal are not\. The check matters because the label alone overstates breakage: the classifier also labels the intact answers of 13 of the 54 lesions model scrambles, although all of them are fluent\. Random removals break the answers less often: 111 of the 151 random removals that undo a lesion give a fluent answer, and 34 give a fluent answer that the classifier labels control speech\. An undone lesion is thus not a healed one: no answer returns to the intact text, and removing the own direction mostly breaks the answer\.

#### Few pushes restore fluent control speech\.

Of the 1,728 own pushes, 167, or 9\.7%, undo the lesion with a fluent answer that the classifier labels control speech, and such pushes exist in 38 of the 54 lesions\. Of the 5,184 random pushes, 271 do so, or 5\.2%, in 32 lesions\. Most pushes that undo a lesion end in model scrambles instead: 1,411 of the 1,643 own pushes and 4,290 of the 4,720 random pushes\. Stronger pushes less often undo the lesion with an answer that the classifier labels control speech: 22% of the own and 15% of the random pushes do so atr=−0\.5r=\-0\.5, and none of the 432 own and 1,296 random pushes atr=−4r=\-4\. Table[34](https://arxiv.org/html/2610.28992#A10.T34)shows two lesions that a weak own push undoes with fluent control speech, which we chose because their answers read well\. In both, the steered\-back answer is a fluent answer to the task, but not the intact answer\. At the same layer and strength, all three random pushes also undo both lesions, and the classifier labels five of their six answers model scrambles\. Removing the own direction instead turns both lesioned answers into model scrambles, with a probability of 1\.00 for that class\. The own direction thus restores control speech almost twice as often as a random direction, but for fewer than one push in ten\.

Table 34:TwoCLAMPlesions on the sandwich task that a weak push with the lesion’s own direction undoes\. Each row gives the answers of the intact model, of the lesioned model, and of the lesioned model steered back by the push, each under its target probability and the classifier’s most probable class\. We chose the two lesions because their answers read well, so they are not typical: in 16 of the 54 lesions, no push with the own direction undoes the lesion with a fluent answer that the classifier labels control speech, where a fluent answer does not loop and has at least 80% of its non\-space characters in dictionary words\. Answers are verbatim up to the last whole word within their first 250 characters; … marks the omitted rest, a slash marks a line break, and \[emoji\] replaces an emoji\.
#### Removing the own direction also breaks intact answers\.

Removing the own direction from the intact model changes the intact answer for all 54 lesions and makes 20 of these answers fail the fluency check\. In three settings, all of Qwen3\.5 with the aphasia target, removing the aphasia direction even makes the intact model’s answer reach the target\. The probability of aphasia rises from 0\.001 to 0\.548 for Qwen3\.5\-9B on the Cinderella task and from 0\.006 to 0\.848 on the sandwich task\. For Qwen3\.5\-4B on the sandwich task, it rises from 0\.005 to 0\.450, and conduction or anomic aphasia becomes the most probable class\. Removal thus damages the language of intact models too, so its undoing of the lesions does not show that the lesions act through the own direction\.

#### Random directions also undo the lesions of the layer search\.

For comparison, we ran the same test on nine aphasia lesions of GPT\-2 XL and the Pythia models that the layer search of Appendix[D](https://arxiv.org/html/2610.28992#A4)found\. These lesions have one to seven heads, and the GPT\-2 XL lesion also knocks out seven blocks of 256 MLP neurons\. Removing the own direction undoes all nine lesions, whereas at least one random removal undoes eight of them and all three random removals undo six\. In one lesion, of Pythia\-1B on the sandwich task, the removal of the own direction undoes the lesion but no random removal does: the three random removals leave a mean target probability of 0\.92\. Over the nine lesions, removal leaves a mean target probability of 0\.001 for the own direction and 0\.244 for the random directions\. Random removals undo these lesions 21 of 27 times, and they undo theCLAMPaphasia lesions of the same nine pairs of a model and a task about as often, 22 of 27 times\. In three of these pairs, both searches found the same single head\. Random directions thus undo most lesions of either search, so the weak specificity of steering back is not a property ofCLAMPalone\.

## Appendix KLLM Judge Prompts

One LLM judge, DeepSeek\-V4\.1\-Flash, plays two roles in this paper\. It labels the steered answers that the experts also label, given only the names of the classes \(§[4\.2](https://arxiv.org/html/2610.28992#S4.SS2)\), and it rates the answers of intact and lesioned models to the TAB’s five connected\-text prompts for 19 aphasia features \(§[4\.1](https://arxiv.org/html/2610.28992#S4.SS1)\)\. This appendix gives the judge’s settings, its three prompts verbatim, and the rules that turn its replies into labels\. In the listings, a gray arrow marks where a long line of a prompt wraps on the page\.

#### Judge and settings\.

We query DeepSeek\-V4\.1\-Flash\([DeepSeek\-AI, 2026](https://arxiv.org/html/2610.28992#bib.bib12)\)through DeepSeek’s application programming interface \(API\), under the model namedeepseek\-flash, at temperature 0\. Each request holds one answer and nothing about how the answer was produced, such as the steering target or the lesion\. DeepSeek\-V4\.1\-Flash is a reasoning model, and it is not deterministic at temperature 0: the same request can return a different reply\. We therefore send each distinct request once and keep its first reply\. We store every reply under the full content of its request, which is the model name, the messages, the temperature, and, for the TAB prompt, the reply format\. An analysis that sends a request again reads the stored reply instead of asking the judge, so all analyses use the same verdicts\. We parse only the final reply, not the reasoning that the API returns with it\.

#### Prompts for the steered answers\.

The judge labels each answer steered toward aphasia with the two\-way prompt of Listing[2](https://arxiv.org/html/2610.28992#LST2), and each answer steered toward Broca’s or Wernicke’s aphasia with the five\-way prompt of Listing[3](https://arxiv.org/html/2610.28992#LST3)\. Both prompts give the class names and nothing else: no definitions and no examples\. The classifier learned its classes from labeled examples, and a judge given hand\-written definitions or examples would judge partly by those, so with the class names alone the judge relies on what it knows about aphasia\. In both prompts,\{question\}is the request that the model answered, for example “Tell me about your first job\.”, and\{answer\}is the steered answer\. For the TAB’s prompts, the question leaves out the battery’s instruction “Respond to the following prompt in 3\-5 full sentences:”, which the model received with the prompt\. The judge receives the filled prompt as the only message of the request, with no system message\.

Listing 2:Two\-way prompt of the LLM judge, for the answers steered toward aphasia\. The judge receives it as a single user message, with the request that the model answered in place of\{question\}and the model’s answer in place of\{answer\}\.Hereisaquestionandaspeaker'sanswertoit,transcribed\.

Question:\{question\}

Answer:\{answer\}

Doesthespeakerhaveaphasia?Replywithoneword:yesorno\.

Listing 3:Five\-way prompt of the LLM judge, for the answers steered toward Broca’s or Wernicke’s aphasia, sent in the same way as the two\-way prompt of Listing[2](https://arxiv.org/html/2610.28992#LST2)\. The options are the classifier’s five classes, and the last option stands for model scrambles\.Hereisaquestionandaspeaker'sanswertoit,transcribed\.

Question:\{question\}

Answer:\{answer\}

Whichoneofthesedescribesthespeaker?

Control\(noaphasia\)

Anomicorconductionaphasia

Broca'saphasia

Wernicke'saphasia

Nothumanspeech\(garbledmachinetext\)

Replywithexactlyoneofthesefiveoptions\.

#### Prompt for the TAB’s connected text\.

The TAB prompt follows the battery’s protocol for scoring connected text automatically\([Roll et al\., 2026b](https://arxiv.org/html/2610.28992#bib.bib40)\)\. Its system message, in Listing[4](https://arxiv.org/html/2610.28992#LST4), is the TAB paper’s instruction to its judge, word for word\. Its user message, in Listing[5](https://arxiv.org/html/2610.28992#LST5), defines the 19 features with an example of each, shows two transcripts annotated with all 19 features, and then gives the answer to rate\. There,\{prompt\}is one of the TAB’s five connected\-text prompts, again without the instruction on length, and\{transcript\}is the model’s answer\. The request also asks the API for a reply in JavaScript Object Notation \(JSON\)\. The features are the battery’s 19, in its order, but we wrote the definitions, the examples, and the two annotated transcripts ourselves, from the feature names and from standard usage in aphasiology\. The prompt thus reconstructs the battery’s protocol rather than copying it, and its definitions and examples differ from those that the TAB paper prints\. Both annotated transcripts are synthetic, so the judge never sees text from an AphasiaBank transcript\. The TAB’s authors validated their protocol against expert raters with Gemini 2\.5 Flash as the judge, whereas we did not compare the TAB ratings of DeepSeek\-V4\.1\-Flash with human raters, so the TAB results of Table[3](https://arxiv.org/html/2610.28992#S5.T3)rest on the judge alone\.

#### From replies to labels\.

A two\-way reply gets the label of its first whole wordyesorno, ignoring case: aphasic foryes, and not aphasic forno\. A five\-way reply gets the class of the keyword that appears first in it, ignoring case:controlfor control speech,anomicorconductionfor conduction or anomic aphasia,brocafor Broca’s aphasia,wernickefor Wernicke’s aphasia, andnot humanorgarbledfor model scrambles\. A reply without any of these words gets no label\. The judge labeled 294 steered answers, 98 steered toward aphasia and 196 toward Broca’s or Wernicke’s aphasia, and all but 4 of its replies were exactly one of the options, up to capitalization\. For those 4 answers, 1 steered toward aphasia and 3 toward Broca’s or Wernicke’s aphasia, the judge’s reasoning reached 65,536 tokens and its reply was empty\. Neither these 4 answers nor the 18 answers that were too short for the classifier are among the 272 answers of Table[4](https://arxiv.org/html/2610.28992#S5.T4)\. We read a TAB reply as a JSON object, from its first opening brace to its last closing brace, and give each feature its value in the object, 0 or 1; a feature that the object leaves out counts as absent\. Every reply behind Table[3](https://arxiv.org/html/2610.28992#S5.T3)was a JSON object with exactly the 19 features, each 0 or 1\. Table[3](https://arxiv.org/html/2610.28992#S5.T3)counts the features that a reply marks present, out of 19, so the count includes the feature of overall communication impairment\.

Listing 4:System message of the TAB judge prompt, which is the instruction that the TAB paper gives its judge\([Roll et al\., 2026b](https://arxiv.org/html/2610.28992#bib.bib40)\)\.Youareanautomatedsystemforanalyzingtranscriptsofspeechforaphasicfeatures\.Yourtaskistoreadatranscriptpassageanddeterminewhethereachofthefollowingfeaturesispresent\(1\)ornotpresent\(0\)\.YouroutputmustbeaJSONfilewitheachfeatureasakeyanditsvalueaseither0or1\.

Listing 5:User message of the TAB judge prompt, sent after the system message of Listing[4](https://arxiv.org/html/2610.28992#LST4)for each connected\-text answer, with the TAB prompt in place of\{prompt\}and the model’s answer in place of\{transcript\}\. We wrote the definitions, the examples, and the two annotated transcripts, which differ from those of the TAB paper\.Featurestoscore:

\-anomia:Word\-findingdifficulty:pauses,circumlocutions,vaguefillers\('thing','stuff'\),oranexplicitfailuretoretrieveaword\.Example:"wewenttothe\.\.\.theplacewiththe,um,thewaterthing"

\-abandoned\_utterances:Anutteranceisstartedandgivenupbeforeitiscompleted\.Example:"andthenwedrovetothe—\.Itwasaniceday\."

\-empty\_speech:Fluentoutputthatcarrieslittleinformation:genericwordsandvaguestatementsinplaceofcontent\.Example:"Itwasgood\.Wedidthingsthere\.Itwasreallysomething,youknow\."

\-semantic\_paraphasias:Arealwordsubstitutedfortheintendedonefromarelatedmeaning\(forkforspoon\)\.Example:"Ipouredthemilkintothefork"

\-phonemic\_paraphasias:Sound\-levelerrorsthatproduceanear\-word:substituted,added,ortransposedsounds\.Example:"wewatchedthetevelision"

\-neologisms:Inventednon\-wordsusedasiftheywerewords\.Example:"shebroughttheflimbertothetable"

\-jargon:Fluentbutlargelyincomprehensiblespeechmadeofneologismsandparaphasias\.Example:"theplorbingcametothesandwitcheranditwasgasping"

\-perseverations:Inappropriaterepetitionofaword,phraseorideaalreadyproduced,carriedoverwhereitnolongerfits\.Example:"Ilikethebeach\.Thebeachisnice\.Myjobwasatthebeach\.Thebeach\."

\-stereotypies\_and\_automatisms:Recurringstockphrasesorautomaticsequencesusedinsteadofcontent\('youknow','andsoon',counting\)\.Example:"youknow,youknow,andsoon,youknow"

\-short\_simplified\_utterances:Telegraphicoutput:veryshortutteranceswithreducedsyntax\.Example:"Trip\.Beach\.Nice\.Sun\."

\-omission\_of\_bound\_morphemes:Missinginflectionssuchas\-ed,\-s,\-ing\.Example:"yesterdayIwalktothestoreandbuytwoapple"

\-omission\_of\_function\_words:Missingarticles,prepositions,auxiliaries,orpronouns\.Example:"wentstore\.boughtbread\.camehome\."

\-paragrammatism:Grammaticalerrorsofcommissioninotherwisefluentspeech:wrongtense,agreement,orwordorder\.Example:"thetripweregoingbyustohavehadyesterday"

\-false\_starts:Anutterancebegins,stops,andrestartsinadifferentform\.Example:"Myfirst—whenIfirst—thejobIhadfirstwas\.\.\."

\-retracing:Wordsorphrasesrepeatedinordertoself\-correctortocontinue\.Example:"wewenttothe,wewenttothebeach"

\-conduite\_d\_approche:Successiveapproximationstowardatargetword\.Example:"cat\-catas\-catastro\-catastrophe"

\-meaning\_unclear:Theintendedmessagecannotberecoveredfromtheutterance\.Example:"itwastheoneforthewhentheyhaditover"

\-off\_topic:Theresponsedoesnotaddresstheprompt\.Example:\(askedaboutatrip\)"Thestockmarkethasbeenvolatilerecently\."

\-overall\_communication\_impairment:Globaljudgment:takenasawhole,communicationisimpaired\.Example:\(anytranscriptwherethefeaturesabovemateriallyreducecommunicativesuccess\)

Annotatedexample1:

Prompt:Tellmeaboutyourfirstjob\.

Transcript:Myfirstjobwasatasmallbakerynearmyschool\.Istartedatfiveinthemorning,helpedshapethedough,andservedcustomersonceweopened\.ItwastiringbutIlovedthesmelloffreshbread\.Istayedtherefortwosummersbeforegoingtocollege\.

Labels:\{"anomia":0,"abandoned\_utterances":0,"empty\_speech":0,"semantic\_paraphasias":0,"phonemic\_paraphasias":0,"neologisms":0,"jargon":0,"perseverations":0,"stereotypies\_and\_automatisms":0,"short\_simplified\_utterances":0,"omission\_of\_bound\_morphemes":0,"omission\_of\_function\_words":0,"paragrammatism":0,"false\_starts":0,"retracing":0,"conduite\_d\_approche":0,"meaning\_unclear":0,"off\_topic":0,"overall\_communication\_impairment":0\}

Annotatedexample2:

Prompt:Tellmeaboutyourfirstjob\.

Transcript:Job\.\.\.um,firstjob\.Bakery\.Bakery,no,the—thebreadplace\.Fiveo'clock,morning\.Makethe\.\.\.thething,thedough\.Dough\.Peoplecome,Igivebread\.Twosummer\.Tiredbutgood\.Breadsmellgood,good\.

Labels:\{"anomia":1,"abandoned\_utterances":0,"empty\_speech":0,"semantic\_paraphasias":0,"phonemic\_paraphasias":0,"neologisms":0,"jargon":0,"perseverations":1,"stereotypies\_and\_automatisms":0,"short\_simplified\_utterances":1,"omission\_of\_bound\_morphemes":1,"omission\_of\_function\_words":1,"paragrammatism":0,"false\_starts":1,"retracing":1,"conduite\_d\_approche":0,"meaning\_unclear":0,"off\_topic":0,"overall\_communication\_impairment":1\}

Nowscorethistranscript\.OutputonlytheJSONobjectwithall19keys\.

Prompt:\{prompt\}

Transcript:\{transcript\}