间接证据在哪里？用影响函数研究语法泛化

# Where’s the Indirect Evidence? Investigating Grammatical Generalization with Influence Functions
Source: [https://arxiv.org/html/2610.22919](https://arxiv.org/html/2610.22919)
William DaiZhenghao Herbert ZhouAffiliation:Department of Linguistics, Yale UniversityMaya ViswanathanAffiliation:Department of Computer Science, Yale UniversityRobert FrankAffiliation:Department of Linguistics, Yale UniversityAffiliation:Wu Tsai Institute, Yale University\{william\.dai, herbert\.zhou, maya\.viswanathan,

###### Abstract

Language models generalize beyond their training corpus to learn linguistic phenomena\([Patil et al\., 2024](https://arxiv.org/html/2610.22919#bib.bib16);[Misra and Mahowald, 2024](https://arxiv.org/html/2610.22919#bib.bib13)\), showing that there exists indirect evidence driving learning\. Prior work has inferred that indirect evidence exists without identifying its character or considered specific hypotheses about what it might be\. We explore an alternative bottom\-up method for identifying indirect evidence\. We use influence functions\([Koh and Liang, 2017](https://arxiv.org/html/2610.22919#bib.bib10)\)to measure how individual training sentences affect model predictions in 15 grammatical phenomena from BLiMP\([Warstadt et al\., 2020](https://arxiv.org/html/2610.22919#bib.bib21)\)\. We filter out a phenomenon’s direct evidence using hand\-constructed rules and take the 60,000 most influential remaining sentences as indirect evidence\. This indirect evidence substantially affects learning: retraining without it lowers held\-out accuracy more than removing a matched random group for every phenomenon across five models \(by a median of1212points\)\. We characterize the indirect evidence by its similarity to our direct evidence filters: it \(partially\) meets the filters’ structural conditions at a median of4\.74\.7times the rate of a matched random group\. Finally, clustering phenomena by indirect\-evidence overlap recovers three of the five BLiMP categories that include multiple phenomena, suggesting that this method can reveal linguistic relationships\.

## 1Introduction

Figure 1:Schematic overview of our methodology, shown for one phenomenon \(c\-command\) and one reference model\. \(A\) We rank every sentence in the training corpus using influence functions with an attribution objective capturing the model’s grammatical preferences in the phenomenon’s BLiMP minimal pairs\. We filter out the phenomenon’s direct evidence and take the 60,000 most influential remaining sentences as indirect evidence\. \(B\) We remove the indirect evidence or an equally sized control group, retrain without the group, and then evaluate on held\-out minimal pairs to validate our methodology\. \(C\) We characterize the indirect evidence\.Language models are capable of learning grammatical phenomena that are rare or even absent in the training data\([Patil et al\., 2024](https://arxiv.org/html/2610.22919#bib.bib16);[Misra and Mahowald, 2024](https://arxiv.org/html/2610.22919#bib.bib13)\)\.111However, other works find limits on how language models can generalize\. For instance,[Oba et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib14)find weak generalization from linguistic evidence, and generalization fails for some constructions\([Lan et al\., 2026](https://arxiv.org/html/2610.22919#bib.bib11);[Howitt et al\., 2024](https://arxiv.org/html/2610.22919#bib.bib7);[Zhou et al\., 2026](https://arxiv.org/html/2610.22919#bib.bib23)\)\.This fact suggests that they generalize to the unattested forms using*indirect evidence*: sentences which do not directly contain a phenomenon, but which have properties that are informative about it\.

Unlike the input a child receives, the training data of a language model is fully observed and controllable\. As a result, the study of language models offers an opportunity to investigate what indirect evidence can drive the learning of a range of phenomena\([Warstadt and Bowman, 2022](https://arxiv.org/html/2610.22919#bib.bib20)\)\.[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)individually remove direct evidence for a range of phenomena and show that learning is still possible, establishing that indirect evidence must exist but not what such evidence consists of\.[Misra and Mahowald \(2024\)](https://arxiv.org/html/2610.22919#bib.bib13)usefully explore the role of particular sources of indirect evidence in the case of the AANN construction\. However, their approach can only confirm or disprove specific prior hypotheses about the nature of indirect evidence\. It would be useful to understand the nature of indirect evidence in a broad range of cases, and in the absence of specific hypotheses about what data might be relevant for generalization\. We would like to be able to determine empirically what features or cues are most relevant for learning a phenomenon\.

In order to do this, we propose a bottom\-up procedure for identifying indirect evidence \(Figure[1](https://arxiv.org/html/2610.22919#S1.F1)\)\. We use influence functions\([Koh and Liang, 2017](https://arxiv.org/html/2610.22919#bib.bib10)\)to evaluate, for a range of grammatical phenomena, how much each training sentence supports a model’s preference for the grammatical sentences in BLiMP minimal pairs\([Warstadt et al\., 2020](https://arxiv.org/html/2610.22919#bib.bib21)\)\. We then filter out sentences that constitute direct evidence using hand\-constructed rules, and take the remaining sentences that are judged most influential as the relevant indirect evidence\. We then test the significance of this evidence for learning each phenomenon by retraining without it\. Finally, we characterize the indirect evidence by its lexical and structural similarity to each phenomenon\.

This paper makes the following contributions:222Our code will be made publicly available upon acceptance\.

- •We give an operationalbottom\-up definition of indirect evidencefor syntactic capability, and apply it to 15 phenomena on five reference models\.
- •Wevalidate the method through retraining: in all 15 phenomena and all five reference models, removing a phenomenon’s 60,000 indirect evidence sentences lowers held\-out accuracy more than removing a matched random group\.
- •Wecharacterize the indirect evidence\. Most of the top 60,000 sentences in the full\-corpus influence ranking are indirect evidence\. We find that while lexical similarity plays a modest role in establishing a sentence as indirect evidence, structural similarity plays a larger role\.
- •Werecover relationships between grammatical phenomena: the overlap in indirect evidence that we identify for grammatically similar phenomena is twice that of unrelated phenomena\. Clustering of shared evidence largely recovers broad BLiMP categories, such as agreement and binding\.

## 2Methodology

### 2\.1Influence functions

Influence functions have been proposed as a way of measuring how a training example affects a model’s behavior\([Koh and Liang, 2017](https://arxiv.org/html/2610.22919#bib.bib10)\)\. Influence functions have been successfully exploited in attributing LLM behavior in reasoning\([Ruis et al\., 2025](https://arxiv.org/html/2610.22919#bib.bib19)\), factual recall\([Chang et al\., 2025](https://arxiv.org/html/2610.22919#bib.bib4)\), and generalization more broadly\([Grosse et al\., 2023](https://arxiv.org/html/2610.22919#bib.bib5)\)\. In this study, we apply influence functions to the study of what supports linguistic phenomena in syntax and semantics\.

Influence functions estimate how a differentiable functionf⁡\(θ\)f\(\\theta\)of the model parametersθ\\thetachanges when a training sentence’s weight changes\. Letθ^\\hat\{\\theta\}be the model’s fitted parameters,ℓi​\(θ\)\\ell\_\{i\}\(\\theta\)be the language\-modeling loss for training sentenceziz\_\{i\}andf⁡\(θ\)f\(\\theta\)the attribution objective\. We define the gradientsgi=∇θℓi​\(θ^\)g\_\{i\}=\\nabla\_\{\\theta\}\\ell\_\{i\}\(\\hat\{\\theta\}\)andq=∇θf​\(θ^\)q=\\nabla\_\{\\theta\}f\(\\hat\{\\theta\}\)\. We compute an estimate of the impact of theiith sentence on the attribution objective as follows:

f⁡\(θ^−i\)−f⁡\(θ^\)≈1N​qT​H−1​gif\(\\hat\{\\theta\}\_\{\-i\}\)\-f\(\\hat\{\\theta\}\)\\approx\\frac\{1\}\{N\}q^\{T\}H^\{\-1\}g\_\{i\}whereNNis the number of sentences in the training corpus,θ^−i\\hat\{\\theta\}\_\{\-i\}denotes the model’s parameters fitted withoutziz\_\{i\}, andHHis the Hessian of the average training loss atθ^\\hat\{\\theta\}\. For a strictly convex loss at its minimum, this cheaply approximates leave\-one\-out retraining\. For neural networks, loss is not strictly convex and influence functions instead approximate the proximal Bregman response function\([Bae et al\., 2022](https://arxiv.org/html/2610.22919#bib.bib2)\)\. We can therefore interpret influence as how much the trained model’s grammatical preferences respond to up\-weighting the sentence, rather than a direct prediction of retraining without it\.

This exact computation cannot be feasibly done for LLMs because of the difficulty of computing the inverse Hessian\. Instead, scaling to LLMs requires approximations and other approaches, such as EK\-FAC\([Grosse et al\., 2023](https://arxiv.org/html/2610.22919#bib.bib5)\), linearization of the model with the neural tangent kernel\([Park et al\., 2023](https://arxiv.org/html/2610.22919#bib.bib15)\), or random projection of the gradients\([Lin et al\., 2024](https://arxiv.org/html/2610.22919#bib.bib12)\)\. We approximate the inverse Hessian termH−1H^\{\-1\}with damped EK\-FAC\([Grosse et al\., 2023](https://arxiv.org/html/2610.22919#bib.bib5)\), computed with Kronfluence \(Appendix[I](https://arxiv.org/html/2610.22919#A9)\)\. Then, lettingBBbe the resulting approximation to the inverse Hessian operator, the relative\-influence score is the following:

ri=qT​B​gigiT​B​gir\_\{i\}=\\frac\{q^\{T\}Bg\_\{i\}\}\{\\sqrt\{g\_\{i\}^\{T\}Bg\_\{i\}\}\}where we apply a normalization term\([Barshan et al\., 2020](https://arxiv.org/html/2610.22919#bib.bib3)\); it normalizes by the training gradient’s norm with the EK\-FAC approximation\. Normalization favors training sentences whose gradients align with the gradient of the attribution objective, and keeps sentences with large gradients from dominating\.

### 2\.2Attribution objective

We now construct an attribution objective that captures a language model’s grammatical preferences for linguistic phenomena\. To evaluate a language model’s grammatical preferences on a particular linguistic phenomenon, we use minimal pairs: two sentences that differ minimally but show a contrast in grammaticality because of adherence \(or not\) to some grammatical regularity\. For example,Most adults who left Victoria noticed themselvesand \*…noticed herselfconstitute a minimal pair for the phenomenon ofc\-command\. However, we cannot simply attribute on minimal pair accuracy because it is a binary metric which either has zero gradient or is discontinuous, making it unsuitable for influence functions computation\. Instead, we need an attribution objective that is smooth\.

Letx\+x^\{\+\}andx−x^\{\-\}be the grammatical and ungrammatical members of a minimal pair\. For a model with parametersθ\\theta, we define themarginas follows:

mθ​\(x\+,x−\)=sθ​\(x\+\)−sθ​\(x−\)m\_\{\\theta\}\(x^\{\+\},x^\{\-\}\)=s\_\{\\theta\}\(x^\{\+\}\)\-s\_\{\\theta\}\(x^\{\-\}\)wheresθ​\(x\)s\_\{\\theta\}\(x\)is the log\-probability score ofxx\. A positive margin means the model \(correctly\) prefers the grammatical sentence in the pair\. Then, using the margin, we define thelogistic margin loss:

ℓpair​\(θ,x\+,x−\)=−log⁡σ⁡\(mθ​\(x\+,x−\)\)\\ell\_\{\\text\{pair\}\}\(\\theta;x^\{\+\},x^\{\-\}\)=\-\\log\\sigma\(m\_\{\\theta\}\(x^\{\+\},x^\{\-\}\)\)This loss has the property that its gradient,∇θℓpair=−σ⁡\(−mθ\)​∇θmθ\\nabla\_\{\\theta\}\\ell\_\{\\text\{pair\}\}=\-\\sigma\(\-m\_\{\\theta\}\)\\nabla\_\{\\theta\}m\_\{\\theta\}, weights each pair byσ⁡\(−mθ\)\\sigma\(\-m\_\{\\theta\}\)\. This means that the weight is small for pairs that have high margin and thus are confidently predicted correctly \(so those individual examples are less likely to be impactful\), while it is larger for pairs that have low margin \(so they are more likely to be affected\)\. Hence, reducing this logistic margin loss targets weak grammatical preferences\.

The logistic margin loss can be extended from minimal pairs to entire phenomena\. For each phenomenon, we have a query set of minimal pairs for influence function scoring; a separate held\-out set of minimal pairs is used for evaluating the effects of removal \(Section[3\.2](https://arxiv.org/html/2610.22919#S3.SS2)\)\. Let𝒫t\\mathcal\{P\}\_\{t\}be the BLiMP paradigms \(the minimal pair types\) associated with phenomenonttand𝒬p\\mathcal\{Q\}\_\{p\}be the query minimal pairs from paradigmpp\. The attribution objective then is:

ℒt=1\|𝒫t\|​∑p∈𝒫t1\|𝒬p\|​∑\(x\+,x−\)∈𝒬pℓpair​\(θ,x\+,x−\)\\mathcal\{L\}\_\{t\}=\\frac\{1\}\{\|\\mathcal\{P\}\_\{t\}\|\}\\sum\_\{p\\in\\mathcal\{P\}\_\{t\}\}\\frac\{1\}\{\|\\mathcal\{Q\}\_\{p\}\|\}\\sum\_\{\(x^\{\+\},x^\{\-\}\)\\in\\mathcal\{Q\}\_\{p\}\}\\ell\_\{\\text\{pair\}\}\(\\theta;x^\{\+\},x^\{\-\}\)
This phenomenon\-level loss averages over the minimal pairs within each paradigm, then equally over paradigms so that paradigm has equal weighting even if they have a differing number of retained pairs\.

## 3Experimental Setup

### 3\.1Model and corpus

#### Corpus\.

We train, attribute, and retrain using the English Wikipedia corpus of[Gulordava et al\. \(2018\)](https://arxiv.org/html/2610.22919#bib.bib6), also used by[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16), which uses a 50k\-word vocabulary with rare words being mapped to<unk\>\. Also following[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16), we uniformly subsample 80% of the 3,052,726 training sentences\. We use this subsample for attribution, removal, and characterization\.333Hereafter, we use the term “full corpus” to refer to this 80% subsample\.

#### Architecture\.

Following[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16), we explore OPT\-style decoder\-only transformers\([Zhang et al\., 2022](https://arxiv.org/html/2610.22919#bib.bib22)\)\. These models have 8 layers, 8 attention heads, hidden and feed\-forward dimension 768, a word\-level tokenizer with a vocabulary of size 50,002, and∼\{\\sim\}67M parameters\. Models are all trained for 40 epochs\. Appendix[D](https://arxiv.org/html/2610.22919#A4)lists details of hardware and compute\.

#### Reference models\.

We train five reference models from scratch following the same architecture, training corpus, and hyperparameters as[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)\(see Appendix[A](https://arxiv.org/html/2610.22919#A1)\)\. Our reference models are trained on the full corpus\. We perform attribution on each model at the checkpoint with lowest validation loss\. For each phenomenon, we identify supporting sentences that are outside of the direct evidence filter in models trained on the full corpus\.

### 3\.2Phenomena and splits

We study the fifteen*phenomena*of[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)\(Table[4](https://arxiv.org/html/2610.22919#A2.T4), Appendix[B](https://arxiv.org/html/2610.22919#A2)\), each with one or more BLiMP paradigms that evaluate it\. For filtering direct evidence, we use[Patil et al\.](https://arxiv.org/html/2610.22919#bib.bib16)’s twelve parse\-based filters and three Negative Polarity Item \(NPI\) surface filters \(Appendix[E](https://arxiv.org/html/2610.22919#A5)\)\. Theregular/irregular plural,determiner–noun, andpassivefilters test lexical generalization and remove a construction only with BLiMP’s words\([Patil et al\., 2024](https://arxiv.org/html/2610.22919#bib.bib16), §3\.2\.2\)\. BLiMP also groups its 67 paradigms into 12 broader classes, which we call*categories*\. For each phenomenon, we randomly split its minimal pairs into a*query*set and a*held\-out*set\. We randomly draw500500query pairs \(with an equal proportion from each paradigm testing the phenomenon\) and hold out the rest of the pairs for evaluation\.444The query set is smaller overall than the held\-out set because the query set only has to estimate one phenomenon\-level query gradient, whereas the held\-out set has to detect possibly small intervention effects\.The minimal pairs in the query set are used to define the attribution objective \(Section[2\.2](https://arxiv.org/html/2610.22919#S2.SS2)\)\. Meanwhile, the held\-out pairs are used to measure the effect of a data intervention in Section[4](https://arxiv.org/html/2610.22919#S4)\. Because evidence is selected using only the query set, its effect on the held\-out minimal pairs’ logistic margin loss does not follow from how influence is defined\. Additionally, we exclude minimal pairs containing<unk\>as well as 6 pairs whose two sentences are identical \(Appendix[B](https://arxiv.org/html/2610.22919#A2)\)\. In total, this leaves 5,826 query pairs \(with a range of 307 to 462 per phenomenon\) and 18,416 held\-out pairs\.

### 3\.3Control groups

For each phenomenonttand reference model, we take theindirect evidencegroup to be the 60,000 most influential sentences which do not matchtt’s filter \(about 2\.5% of the corpus\)\.555Appendix[G](https://arxiv.org/html/2610.22919#A7.SS0.SSS0.Px5)compares the result if we consider the 20,000 most influential sentences\.We compare the effects of the indirect evidence training data to the following control groups \(details are elaborated in Appendix[F](https://arxiv.org/html/2610.22919#A6)\):

- •Matched random\.For each indirect evidence sentence, we draw a random sentence of the same length and similar mean log word frequency from the remaining non\-direct sentences\.
- •BM25\.The purpose of this control group is to compare our influence\-based selection against lexical selection\. We identify the top 60,000 sentences by BM25 score, a measure of lexical similarity\([Robertson and Zaragoza, 2009](https://arxiv.org/html/2610.22919#bib.bib18)\), against the vocabulary oftt’s query pairs\.
- •Direct Evidence\.The sentences that matchtt’s filter\. For characterization, we use all of them\. For retraining, we remove a random sample of 60,000, the same size as the other groups\. Only five phenomena have that many direct\-evidence sentences \(relational\-noun distractor,relative\-clause distractor,regular/irregular plural,superlative, andpassive\), so retraining covers only those five\.

We draw every group except direct evidence from the pool of sentences that do not match the direct evidence filter\. The groups therefore differ only in how they are selected: by influence, at random, or by lexical similarity\.

## 4Causal Validation

To test whether the identified indirect evidence is causally relevant, we remove it from the training corpus and retrain from scratch \(Panel B in Figure[1](https://arxiv.org/html/2610.22919#S1.F1)\)\. For each phenomenon and reference model, we remove one group, either indirect evidence or one of the controls \(Section[3\.3](https://arxiv.org/html/2610.22919#S3.SS3)\), from the training corpus and then train a new model using the reference model’s initialization, data order, and hyperparameters\. We then evaluate on the held\-out pairs at the same epoch as the reference model’s best\-validation checkpoint \(we test alternative choices for this in Appendix[G](https://arxiv.org/html/2610.22919#A7.SS0.SSS0.Px4)\); every comparison is exactly paired between the control and a reference model with the same initialization\. We expect model performance to significantly degrade when the indirect evidence is removed and to not degrade as much when the control groups are removed\.

Figure 2:Accuracy on held\-out BLiMP minimal pairs after removing each group\. Extra drop is the additional drop in accuracy, averaged over the five initializations, when removing indirect evidence compared to matched random\. Points show means over the five models; bars are the range\. Ranges narrower than a marker are hidden behind it\. Comparing against direct evidence \(a random 60,000 of its sentences\) was done only for the five phenomena with at least 60,000 direct\-evidence sentences\.#### Outcome\.

Figure[2](https://arxiv.org/html/2610.22919#S4.F2)shows the results of removing indirect evidence and each control group\. Removing indirect evidence lowers held\-out BLiMP accuracy more than removing a matched random group for all 15 phenomena in each of the five reference models\. On held\-out loss, an exact sign\-flip test across the five reference models givesp=1/32p=1/32, its smallest possible value \(Appendix[G](https://arxiv.org/html/2610.22919#A7)\)\. Relative to the matched random group and averaged over the five initializations, the extra accuracy drop ranges from3\.23\.2to44\.444\.4percentage points, with a median of about1212points\.666Percentage points are on an absolute scale and not relative\.Both groups are the same size and matched on sentence length and mean word frequency\. Hence, the larger accuracy drop shows that influence functions select for more influential sentences than random\. We also see larger accuracy drops for indirect evidence as compared to the BM25 control group and, perhaps surprisingly, the direct evidence group\. The BM25 results demonstrate that lexical similarity alone does not imply causal importance in retraining\. The direct evidence results could be explained by the fact that the removal of all direct evidence in[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)generally results in modest effects, and we only remove a fraction of the direct evidence; this may also suggest direct evidence does not need to be highly influential\.

The largest drops from removing indirect evidence generally come from phenomena that are not robustly learned by the reference models, such asreconstruction\(55% held\-out accuracy\) andmatrix question\(31% held\-out accuracy\)\. We discuss this further in Section[6](https://arxiv.org/html/2610.22919#S6.SS0.SSS0.Px2)\.

#### Same\-category vs\. different\-category removal effects\.

We want to understand the effect of a phenomenon’s indirect evidence on other phenomena, and if this effect differs between same\-category and different\-category phenomena\. Relative to the reference model and averaged over the five initializations, removing a phenomenon’s own indirect evidence lowers its held\-out accuracy by a median of11\.411\.4points while removing another phenomenon’s indirect evidence lowers accuracy by a median of0\.30\.3points \(Figure[3](https://arxiv.org/html/2610.22919#S4.F3)\)\. Thus, the indirect evidence does not consist of sentences that are useful for every phenomenon\. However, there does exist a causal relationship between phenomena in the same BLiMP category: same\-category phenomena pairs have an average accuracy drop of2\.22\.2points while different\-category phenomena pairs have an average accuracy drop of0\.40\.4points\. Indirect evidence for a phenomenon is thus more important for phenomena in the same category than for phenomena in other categories, showing how indirect evidence may link together same\-category phenomena\.

![Refer to caption](https://arxiv.org/html/2610.22919v1/specificity_column.png)Figure 3:Held\-out accuracy lost \(points\) after removing each phenomenon’s indirect evidence \(rows\), evaluated on every phenomenon \(columns\), relative to the reference model and averaged over five initializations\. Numbers give each phenomenon’s own accuracy drop\. Black boxes outline the BLiMP categories\.

## 5Characterizing Indirect Evidence

Retraining helps to validate that our definition of indirect evidence yields data that supports the model’s grammatical preferences for the relevant phenomenon\. Unlike previous studies, the indirect evidence is not a priori characterized by prior hypotheses about what data might matter, so characterizing it is far from trivial\. So, to further understand why the indirect evidence may aid in generalization, we ask how much of the top evidence is indirect and analyze the indirect evidence along three dimensions: lexical similarity, structural coverage, and overlap between phenomena\. We also compare the indirect evidence to the control groups in Section[3\.3](https://arxiv.org/html/2610.22919#S3.SS3)\. These analyses illustrate what sentence\-level evidence makes possible\. To foreshadow the results, we find that the indirect evidence tends to share structure with its phenomenon and to be shared by related phenomena\.

#### How much of the evidence is direct?

To measure the share of indirect evidence, we rank every training sentence by its influence and compute the proportion of each phenomenon’s 60,000 most influential sentences that are direct evidence as identified by its filter \(Figure[4](https://arxiv.org/html/2610.22919#S5.F4)\)\. Influence favors direct evidence: it is enriched in the top 60,000 sentences compared to its share in the corpus\. Still, direct evidence is at most34\.1%34\.1\\%for any phenomenon’s top 60,000 \(relational\-noun distractor\) and under19%19\\%for the other1414phenomena\. Table[5](https://arxiv.org/html/2610.22919#A3.T5)in Appendix[C](https://arxiv.org/html/2610.22919#A3)gives other cutoffs\. Most of each phenomenon’s top 60,000 is therefore indirect evidence\.

Figure 4:Direct evidence among the 60,000 most influential sentences compared to its proportion of the entire training corpus\. Direct share is how much of the top 60,000 is direct evidence\. Enrichment is direct share divided by how much of the corpus is direct evidence\.
#### Is the indirect evidence lexically similar to the query pairs?

A deflationary hypothesis for why indirect evidence might be influential is that it shares lexical items with the query pairs\. To test this hypothesis, we measure a group’s*lexical similarity*as the mean BM25 score of the sentences in that group against the vocabulary of the query pairs\. The indirect evidence is slightly more lexically similar to the query pairs than the matched random group \(1\.151\.15–1\.33×1\.33\\timesits mean BM25 score\), but in every phenomenon it has lexical similarity about half that of the BM25 group\. Indirect evidence is also less lexically similar than the direct evidence for 14 of the 15 phenomena \(Figure[5](https://arxiv.org/html/2610.22919#S5.F5)\)\. Lexical similarity as measured by BM25 does not explain why the indirect evidence is influential\.

Figure 5:Lexical similarity of the indirect evidence, direct evidence, and the BM25 group, relative to the matched random group’s lexical similarity\. Points show means over the five models; bars are the range\.Figure 6:Structural coverage of the indirect evidence and the BM25 group relative to the matched random group \(log scale\), for the 11 phenomena whose coverage varies among non\-direct sentences; Appendix[E](https://arxiv.org/html/2610.22919#A5)reports the other four\. Points are means over five models and bars are ranges; the right columns give the matched random group’s mean coverage and the indirect evidence’s enrichment\.
#### Is the indirect evidence structurally similar to the phenomenon?

Another hypothesis for why indirect evidence impacts learning is that indirect evidence shares syntactic structure with the phenomenon\. While indirect evidence does not match all of the conditions in a phenomenon’s filter, it may match some of the filter’s structural conditions\. We define a sentence’s*structural coverage*as the fraction of the filter’s structural conditions it meets\. To aid in understanding this definition, Table[1](https://arxiv.org/html/2610.22919#S5.T1)illustrates three examples of indirect evidence and the structural and lexical conditions they do or do not meet \(Table[7](https://arxiv.org/html/2610.22919#A5.T7)in Appendix[E](https://arxiv.org/html/2610.22919#A5)enumerates all filters’ conditions\)\. In all 11 phenomena, the indirect evidence has higher structural coverage than the matched random group, ranging from1\.3×1\.3\\timesto62×62\\timeswith a median of4\.7×4\.7\\times\(Figure[6](https://arxiv.org/html/2610.22919#S5.F6)\)\. The indirect evidence’s structural coverage is also higher than the BM25 group’s in 9 \(all butbinding domainandbinding case\)\. Where coverage varies, indirect evidence is enriched for the structure its phenomenon’s filter looks for, even though it is not direct evidence\.

Table 1:Examples of indirect evidence, with a minimal pair for each phenomenon \(\* marks the ungrammatical sentence\)\. Each is among the 60,000 most influential non\-direct sentences of all five reference models\. / indicate the filter conditions it satisfies or misses\. Structural conditions count toward structural coverage, while lexical \(word\-list\) conditions do not\.Figure 7:Average\-linkage clustering of the 15 phenomena by shared indirect evidence\. Distance between two phenomena is one minus overlap\. The horizontal axis \(log scale\) gives the average overlap where two branches join\. Colors are added after clustering and mark branches whose phenomena all belong to one BLiMP category\. The “x/5” numbers at joins show how many of the five reference models produce the same cluster when they are clustered separately\.
#### Do related phenomena share indirect evidence?

If indirect evidence carries structure that is useful beyond a single phenomenon, phenomena in the same category should share it\([Misra and Mahowald, 2024](https://arxiv.org/html/2610.22919#bib.bib13)\)\. For a pair of phenomena, we define*overlap*as the size of the intersection between the two phenomena’s indirect evidence set divided by the size of indirect evidence set \(which is always 60,000\), averaged over the five reference models\. On average, phenomena in the same category have an overlap of11\.7%11\.7\\%against5\.8%5\.8\\%for phenomena in different categories\. The exception is the two quantifier phenomena, which have an overlap of5\.0%5\.0\\%, less than the5\.8%5\.8\\%average for different\-category pairs\. We also see thatc\-commandandrelative\-clause distractorhave an overlap of14\.1%14\.1\\%, likely because the sentences from BLiMP that evaluate both phenomena involve relative clauses \(Appendix[H](https://arxiv.org/html/2610.22919#A8)shows results for every pair\)\. We also perform average\-linkage clustering on the phenomena by their overlap \(Figure[7](https://arxiv.org/html/2610.22919#S5.F7)\)\. The clustering puts three of the five BLiMP categories with more than one phenomenon on their own branches: determiner–noun agreement, subject–verb agreement, and binding\. However, the clustering also links together some different\-category phenomena: for instance,sentential negationandexistential there\. This may be because both phenomena involve negation\. Overall, this suggests we can recover linguistic categories using the indirect evidence\.

## 6Discussion and Conclusion

#### Recovering category relationships\.

The retraining results in[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)demonstrate the existence of indirect evidence, but do not establish possible relationships between different phenomena in the same category\. Indeed, in their results for transformer models, removing one phenomenon’s direct evidence does not, on average, lower accuracy on same\-category phenomenon more than on different\-category phenomenon\. Thus, removing direct evidence may not be enough to reveal relationships between same\-category phenomena\. This may be because direct evidence makes up a modest portion of what the model uses to learn, a median of5\.7%5\.7\\%of a phenomenon’s 60,000 most influential sentences and at most34\.1%34\.1\\%\(Figure[4](https://arxiv.org/html/2610.22919#S5.F4)\)\. In contrast, investigating identified indirect evidence does reveal same\-category relationships \(Figure[7](https://arxiv.org/html/2610.22919#S5.F7), Figure[3](https://arxiv.org/html/2610.22919#S4.F3), Figure[8](https://arxiv.org/html/2610.22919#A8.F8)in Appendix[H](https://arxiv.org/html/2610.22919#A8)\)\. These same\-category phenomena could be linked by structurally enriched sentences that are not direct evidence for either phenomenon\.

#### Removal sensitivity as a diagnostic\.

The magnitude of the accuracy drop when a fixed amount of indirect evidence is removed indicates how concentrated the support is for the phenomenon\. Across the 15 phenomena, the extra accuracy drop is higher when the reference model has lower competence on the phenomenon \(Spearmanρ=−0\.78\\rho=\-0\.78between reference model accuracy and extra accuracy drop\)\. This indicates that these barely learned phenomena rely on a small set of sentences or on fragile surface regularities\. When accuracy drop and BLiMP performance diverge, such as when the models score82%82\\%ononlybut removal of indirect evidence leads to a drop of4242points relative to matched random, this may suggest that the phenomenon is actually fragile despite good BLiMP performance\. Influence\-guided removal or analysis of influence score concentration could mark phenomena that appear to be robustly learned, but are actually fragile\.

#### A testbed for training data attribution\.

It may be hard to judge whether an attribution method is working because the theory for which training examples should matter for a capability is often not developed or independent\. In fact\-tracing benchmarks measuring factual knowledge in a language model, the relevance of a training example could be defined through fact\-entailment\([Akyurek et al\., 2022](https://arxiv.org/html/2610.22919#bib.bib1)\)or causally, for instance, through tail\-patching or retraining\. These two accounts of relevance may disagree, leading to high influence examples that do not entail a fact\([Chang et al\., 2025](https://arxiv.org/html/2610.22919#bib.bib4)\)\. When an influential training example does not entail a fact, due to the lack of a developed theory, it is unclear whether it is meaningful or noise from a poor attribution method; even if causally influential, it is hard to interpret\. Meanwhile, linguistics is a developed source of theories for generalization, and this could help us interpret influential training examples\. Thus, we suggest syntactic generalization could be a testbed for attribution methods\. Influential training examples that do not directly contain a linguistic phenomenon could be interpreted using descriptive terms from linguistic theory\. This provides another check that our attribution method has reasonable results\. For example, recovering BLiMP categories in Figure[7](https://arxiv.org/html/2610.22919#S5.F7)helps show that data attribution is finding sensible structure\.

#### Open questions\.

Our study leaves several questions open\. First, our structural coverage metric is deliberately generic, so it does not give any account of which structures matter for a given phenomenon\. Taking the relations in a sentence’s dependency parse as features would show which structures are over\-represented in the indirect evidence\. Such an analysis could indicate whether the patterns of generalization adhere to the landscape of linguistic theory, for instance whether binding phenomena share indirect evidence containing reflexives\. Second, pieces of evidence might play competing roles: sentences supporting one phenomenon might interfere with others\. This could be revealed by measuring negative influence on other phenomena\. Third, performing this study on developmentally plausible corpora could show whether child\-directed speech provides a different profile of indirect evidence that enables rapid generalization\.

## Limitations

Our study inherits the weaknesses of other studies applying influence functions to LLMs\. Influence functions approximate the proximal Bregman response rather than leave\-one\-out retraining\([Bae et al\., 2022](https://arxiv.org/html/2610.22919#bib.bib2)\)\. The validation experiment does not mean the influence scores predict the magnitude of the retraining’s impact\. The Hessian must be approximated for influence functions to scale to LLMs\. Influence functions may not capture redundancy in a group of training sentences where the removal of the entire group is needed for a measurable effect\. Our methodology also requires fixing the parameterkkfor determining what counts as an influential sentence, and we do not perform a systematic study over how changingkkmay change our results\.

Our language model \(67M parameters\) and training corpus are relatively small\. To validate our methodology, we trained270270models with different groups of data removed\. This retraining\-based validation would be infeasible on a larger scale with our compute budget\. However,[Khaddaj et al\. \(2025\)](https://arxiv.org/html/2610.22919#bib.bib9)find that training data tends to affect small and large models in correlated ways, so the method may remain valid at larger scale\. On the other hand,[Grosse et al\. \(2023\)](https://arxiv.org/html/2610.22919#bib.bib5)find that the influential data for larger models tend to be less lexically similar to the queries\. We may expect the lexical and structural profiles to change if we trained larger models\.

The scope of our exploration is also limited\. We study only English, investigate only the 15 BLiMP phenomena that[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)filtered, train only on one Wikipedia corpus, and investigate one architecture\. BLiMP’s minimal pairs are generated from templates with a restricted vocabulary, so held\-out accuracy only measures performance on these templates rather than each phenomenon in general\. For phenomena the models have not robustly learned, removing indirect evidence may just shift preferences related to the BLiMP minimal pairs templates’ frames\.

Influence functions attribution is relatively automatic, but our definition of indirect evidence still requires hand\-written filters for direct evidence\. These filters are imperfect and may leak direct examples of the phenomena; our study’s filters inherit the same possible flaws as[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)’s filters\. We did not audit a sample of the indirect evidence by hand, so the extent of the filter leakage is unknown, though we should expect relatively little leakage;[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)intentionally favored stronger filters which filter out more data, though at the cost of false positives\. Our structural coverage metric also inherits the filters’ design and conditions, and therefore is affected by particular design choices in its logic\.

## Broader Impact and Ethical Considerations

This study had no human participants\. The validation of attribution via retraining was computationally expensive, and took about 5,700 GPU\-hours\. Thus, this type of validation through retraining can be environmentally damaging; computationally lighter methods may reduce this harm\. We do not foresee any risks arising from this research in particular beyond the general risks associated with machine learning research or data attribution research; moreover, the specific use of data attribution here is removed from potential harms such as data poisoning or adversarial manipulation of a dataset\.

## References

- Akyurek et al\. \(2022\)Ekin Akyurek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, and Kelvin Guu\. 2022\.[Towards Tracing Knowledge in Language Models Back to the Training Data](https://doi.org/10.18653/v1/2022.findings-emnlp.180)\.In*Findings of the Association for Computational Linguistics: EMNLP 2022*, pages 2429–2446, Abu Dhabi, United Arab Emirates\. Association for Computational Linguistics\.
- Bae et al\. \(2022\)Juhan Bae, Nathan Ng, Alston Lo, Marzyeh Ghassemi, and Roger Grosse\. 2022\.If Influence Functions are the Answer, Then What is the Question?In*Proceedings of the 36th International Conference on Neural Information Processing Systems*, NIPS ’22, Red Hook, NY, USA\. Curran Associates Inc\.
- Barshan et al\. \(2020\)Elnaz Barshan, Marc\-Etienne Brunet, and Gintare Karolina Dziugaite\. 2020\.[RelatIF: Identifying Explanatory Training Samples via Relative Influence](https://proceedings.mlr.press/v108/barshan20a.html)\.In*Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics*, volume 108 of*Proceedings of Machine Learning Research*, pages 1899–1909\. PMLR\.
- Chang et al\. \(2025\)Tyler A\. Chang, Dheeraj Rajagopal, Tolga Bolukbasi, Lucas Dixon, and Ian Tenney\. 2025\.[Scalable Influence and Fact Tracing for Large Language Model Pretraining](https://openreview.net/forum?id=gLa96FlWwn)\.In*The Thirteenth International Conference on Learning Representations*\.
- Grosse et al\. \(2023\)Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamilė Lukošiūtė, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, and Samuel R\. Bowman\. 2023\.[Studying Large Language Model Generalization with Influence Functions](http://arxiv.org/abs/2308.03296)\.
- Gulordava et al\. \(2018\)Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni\. 2018\.[Colorless Green Recurrent Networks Dream Hierarchically](https://doi.org/10.18653/v1/N18-1108)\.In*Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long Papers\)*, pages 1195–1205, New Orleans, Louisiana\. Association for Computational Linguistics\.
- Howitt et al\. \(2024\)Katherine Howitt, Sathvik Nair, Allison Dods, and Robert Melvin Hopkins\. 2024\.[Generalizations across filler\-gap dependencies in neural language models](https://doi.org/10.18653/v1/2024.conll-1.21)\.In*Proceedings of the 28th Conference on Computational Natural Language Learning*, pages 269–279, Miami, FL, USA\. Association for Computational Linguistics\.
- Jumelet et al\. \(2021\)Jaap Jumelet, Milica Denic, Jakub Szymanik, Dieuwke Hupkes, and Shane Steinert\-Threlkeld\. 2021\.[Language Models Use Monotonicity to Assess NPI Licensing](https://doi.org/10.18653/v1/2021.findings-acl.439)\.In*Findings of the Association for Computational Linguistics: ACL\-IJCNLP 2021*, pages 4958–4969, Online\. Association for Computational Linguistics\.
- Khaddaj et al\. \(2025\)Alaa Khaddaj, Logan Engstrom, and Aleksander Madry\. 2025\.[Small\-to\-Large Generalization: Training Data Influences Models Consistently Across Scale](https://openreview.net/forum?id=79ZkWgY2FI)\.In*The Thirteenth International Conference on Learning Representations*\.
- Koh and Liang \(2017\)Pang Wei Koh and Percy Liang\. 2017\.[Understanding Black\-box Predictions via Influence Functions](https://proceedings.mlr.press/v70/koh17a.html)\.In*Proceedings of the 34th International Conference on Machine Learning*, volume 70 of*Proceedings of Machine Learning Research*, pages 1885–1894\. PMLR\.
- Lan et al\. \(2026\)Nur Lan, Emmanuel Chemla, and Roni Katzir\. 2026\.[Large Language Models and the Argument from the Poverty of the Stimulus](https://doi.org/10.1162/ling_a_00533)\.*Linguistic Inquiry*, 57\(2\):315–342\.
- Lin et al\. \(2024\)Huawei Lin, Jikai Long, Zhaozhuo Xu, and Weijie Zhao\. 2024\.[Token\-wise Influential Training Data Retrieval for Large Language Models](https://doi.org/10.18653/v1/2024.acl-long.48)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 841–860, Bangkok, Thailand\. Association for Computational Linguistics\.
- Misra and Mahowald \(2024\)Kanishka Misra and Kyle Mahowald\. 2024\.[Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs](https://doi.org/10.18653/v1/2024.emnlp-main.53)\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 913–929, Miami, Florida, USA\. Association for Computational Linguistics\.
- Oba et al\. \(2024\)Miyu Oba, Yohei Oseki, Akiyo Fukatsu, Akari Haga, Hiroki Ouchi, Taro Watanabe, and Saku Sugawara\. 2024\.[Can Language Models Induce Grammatical Knowledge from Indirect Evidence?](https://doi.org/10.18653/v1/2024.emnlp-main.1146)In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 20591–20603, Miami, Florida, USA\. Association for Computational Linguistics\.
- Park et al\. \(2023\)Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, and Aleksander Mądry\. 2023\.TRAK: Attributing Model Behavior at Scale\.In*Proceedings of the 40th International Conference on Machine Learning*, ICML’23\. JMLR\.org\.
- Patil et al\. \(2024\)Abhinav Patil, Jaap Jumelet, Yu Ying Chiu, Andy Lapastora, Peter Shen, Lexie Wang, Clevis Willrich, and Shane Steinert\-Threlkeld\. 2024\.[Filtered Corpus Training \(FiCT\) Shows that Language Models Can Generalize from Indirect Evidence](https://doi.org/10.1162/tacl_a_00720)\.*Transactions of the Association for Computational Linguistics*, 12:1597–1615\.
- Qi et al\. \(2020\)Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D\. Manning\. 2020\.[Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://doi.org/10.18653/v1/2020.acl-demos.14)\.In*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations*, pages 101–108, Online\. Association for Computational Linguistics\.
- Robertson and Zaragoza \(2009\)Stephen Robertson and Hugo Zaragoza\. 2009\.[The Probabilistic Relevance Framework: BM25 and Beyond](https://doi.org/10.1561/1500000019)\.*Found\. Trends Inf\. Retr\.*, 3\(4\):333–389\.
- Ruis et al\. \(2025\)Laura Ruis, Maximilian Mozes, Juhan Bae, Siddhartha Rao Kamalakara, Dwaraknath Gnaneshwar, Acyr Locatelli, Robert Kirk, Tim Rocktäschel, Edward Grefenstette, and Max Bartolo\. 2025\.[Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models](https://openreview.net/forum?id=1hQKHHUsMx)\.In*The Thirteenth International Conference on Learning Representations*\.
- Warstadt and Bowman \(2022\)Alex Warstadt and Samuel R Bowman\. 2022\.What Artificial Neural Networks Can Tell Us About Human Language Acquisition\.*Algebraic Structures in Natural Language*, 17\.
- Warstadt et al\. \(2020\)Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng\-Fu Wang, and Samuel R\. Bowman\. 2020\.[BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://doi.org/10.1162/tacl_a_00321)\.*Transactions of the Association for Computational Linguistics*, 8:377–392\.
- Zhang et al\. \(2022\)Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer\. 2022\.[OPT: Open Pre\-trained Transformer Language Models](http://arxiv.org/abs/2205.01068)\.
- Zhou et al\. \(2026\)Zhenghao Zhou, William Dai, Maya Viswanathan, Simon Charlow, R\. Thomas McCoy, and Robert Frank\. 2026\.[What Exactly do Children Receive in Language Acquisition? A Case Study on CHILDES with Automated Detection of Filler\-Gap Dependencies](https://doi.org/10.18653/v1/2026.conll-main.42)\.In*Proceedings of the 30th Conference on Computational Natural Language Learning*, pages 692–706, San Diego, California, USA\. Association for Computational Linguistics\.

## Appendix AReference Models

We train five reference models with the same architecture, corpus, and hyperparameters as[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)\(Table[2](https://arxiv.org/html/2610.22919#A1.T2)\)\. We reconstruct[Patil et al\.](https://arxiv.org/html/2610.22919#bib.bib16)’s \([2024](https://arxiv.org/html/2610.22919#bib.bib16)\) training sample by rerunning their sampling code \(seed 42, dropping sentences longer than 512 words\)\. The resulting subsampled corpus has the same token count as the corpus in their released results\. Our validation retrains each validation model from the same initialization as its corresponding reference model \(§[4](https://arxiv.org/html/2610.22919#S4)\)\. Because initializations are not fixed in[Patil et al\.](https://arxiv.org/html/2610.22919#bib.bib16)’s \([2024](https://arxiv.org/html/2610.22919#bib.bib16)\) code, we do not replicate their models exactly\.

Moreover, there is a slight difference in how data order is handled\.[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)read the corpus in shards with parallel loaders\. In our training, we shuffle all sentences once per epoch\. We use the same shuffled data order and the same seed for training randomness for our five reference models; our reference models differ only in their initialization\. This is to keep data order fixed for comparisons\.

Table 2:Training settings of the reference and validation models\.For every trained model used for validation, we start from its corresponding reference model’s initialization\. Then, the validation model sees the data in the same order as the corresponding reference model, except without the removed sentences\. When reporting held\-out accuracy or loss, we evaluate the validation model at its reference model’s selected epoch rather than its own best epoch\. This is so the validation model has the same number of passes over the data between evaluations in order to make comparison between models fair\.

We provide some more details and elaboration on our metrics\. For a sentencexx,sθ​\(x\)s\_\{\\theta\}\(x\)is the sum of its token log\-probabilities from the second word through an appended end\-of\-sentence token, without length normalization\. We consider the language model to be correct on a minimal pair whensθ​\(x\+\)\>sθ​\(x−\)s\_\{\\theta\}\(x^\{\+\}\)\>s\_\{\\theta\}\(x^\{\-\}\), wherex\+x^\{\+\}is the grammatical sentence andx−x^\{\-\}is the ungrammatical sentence\. Our held\-out accuracy and loss \(the logistic margin loss of §[2\.2](https://arxiv.org/html/2610.22919#S2.SS2)\) metrics are calculated by averaging pairs within each paradigm, then averaging over paradigms with equal weight\.

We also demonstrate comparable performance with the published models in[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)\. Table[3](https://arxiv.org/html/2610.22919#A1.T3)compares our reference models with[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)’s published full\-corpus models\. Both our reference models and the[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)models are scored with our evaluation code on our held\-out minimal pairs\. The difference in BLiMP performance is generally minimal between our models and[Patil et al\.](https://arxiv.org/html/2610.22919#bib.bib16)’s \([2024](https://arxiv.org/html/2610.22919#bib.bib16)\) models\. There is a median difference of0\.50\.5percentage points, and 8 phenomena have a difference less than one point\. Moreover, the accuracy difference is larger than the standard deviation of either set of models only on two phenomena:binding domain\(\+3\.2\+3\.2\) andsentential negation\(\+3\.7\+3\.7\)\. Even for these phenomena, the difference is still less than two standard deviations of at least one of the sets of models\.

Table 3:Held\-out BLiMP accuracy \(%\) of our reference models and[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)’s published full\-corpus models \(mean±\\pmstandard deviation over five models\)\. Note that differences are computed between means before rounding occurs\.
## Appendix BPhenomena

Table[4](https://arxiv.org/html/2610.22919#A2.T4)displays the fifteen phenomena with their BLiMP paradigms, the example pair[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)provide for each paradigm, and each paradigm’s number of query and held\-out pairs\. In addition to the pairs that contain<unk\>, we exclude 6 pairs whose two sentences are identical in the released BLiMP data\. Table[5](https://arxiv.org/html/2610.22919#A3.T5)\(last column\) gives the share of the training corpus that each phenomenon’s filter marks as direct evidence\.

Phenomenon\([Patil et al\., 2024](https://arxiv.org/html/2610.22919#bib.bib16)\)BLiMP paradigmExampleQueryHeld\-outDeterminer–noun agreementDeterminer–noundet\_noun\_agr\_1Craig explored that grocery store/\*stores\.102706det\-noundet\_noun\_agr\_2Carl cures those/\*that horses\.108732det\_noun\_agr\_irr\_1Phillip was lifting this mouse/\*mice\.84608det\_noun\_agr\_irr\_2Those ladies walk through those/\*that oases\.96665Determiner–adj\.–noundet\_noun\_agr\_with\_adj\_1Tracy praises those lucky guys/\*guy\.104725det\-adj\-noundet\_noun\_agr\_with\_adj\_2Some actors buy these/\*this gray books\.104735det\_noun\_agr\_with\_adj\_irr\_1He shouldn’t criticize this upset child/\*children\.88562det\_noun\_agr\_with\_adj\_irr\_2That adult has brought that/\*those purple octopus\.92701Subject–verb agreementRegular/irregular pluralregular\_plural\_sv\_agr\_1Jeffrey hasn’t/\*haven’t criticized Donald\.100703agr\-re\-irr\-svregular\_plural\_sv\_agr\_2The dress/\*dresses crumples\.98673irr\_plural\_sv\_agr\_1This goose isn’t/\*weren’t bothering Edward\.85582irr\_plural\_sv\_agr\_2The woman/\*women cleans every public park\.109737Relative\-clause distractor
agr\-rel\-cldistractor\_agr\_relative\_clauseBoys that aren’t disturbing Natalie suffer/\*suffers\.328337Relational\-noun distractor
agr\-pp\-moddistractor\_agr\_relational\_nounA sketch of lights doesn’t/\*don’t appear\.340343Argument structurePassivepassive\_1Jeffrey’s sons are insulted/\*smiled by Tina\.150455passivepassive\_2Most cashiers are disliked/\*flirted\.157500BindingDomainprinciple\_A\_domain\_1Carlos said that Lori helped him/\*himself\.150729binding\-domainprinciple\_A\_domain\_2Mark imagines Erin might admire herself/\*himself\.138722principle\_A\_domain\_3Nancy could say every guy hides himself\. / \*Every guy could say Nancy hides himself\.153751C\-command
binding\-c\-commandprinciple\_A\_c\_commandA lot of actresses that thought about Alice healed themselves/\*herself\.395392Caseprinciple\_A\_case\_1Tara thinks that she/\*herself sounded like Wayne\.218633binding\-caseprinciple\_A\_case\_2Anna imagines herself praising/\*praises this boy\.210630Reconstruction
binding\-reconstructionprinciple\_A\_reconstructionIt’s herself who Karen criticized / \*criticized Karen\.462479QuantifiersSuperlativesuperlative\_quantifiers\_1No man has revealed more than/\*at least 5 forks\.213653quantifier\-superlativesuperlative\_quantifiers\_2An/\*No actor arrived at at most 6 lakes\.220659Existential there
quantifier\-existential\-thereexistential\_there\_quantifiers\_1There aren’t many/\*all lights darkening\.414404NPI licensingOnlyonly\_npi\_licensor\_presentOnly/\*Even Bill would ever complain\.191576npi\-onlyonly\_npi\_scopeOnly those doctors who Karla respects ever … / \*Those doctors who only Karla respects ever …158496Sentential negationsentential\_negation\_npi\_licensor\_presentThose banks had not/\*really ever lied\.202613npi\-sent\-negsentential\_negation\_npi\_scopeThe turtles that are boring me could not ever … / \*The turtles that are not boring me could ever …158508Matrix question
npi\-sim\-quesmatrix\_question\_npi\_licensor\_presentShould I ever join? / \*I should ever join\.399407Total5,82618,416Table 4:The fifteen phenomena, grouped by BLiMP category, with the specific names used in[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16), BLiMP paradigms, and example pairs taken exactly from[Patil et al\. \(2024, Table 1\)](https://arxiv.org/html/2610.22919#bib.bib16)\(\* marks the ungrammatical sentence\)\. Paradigm names are abbreviated: agr = agreement, adj = adjective, det = determiner, irr = irregular, sv = subject–verb\.
## Appendix CVarying Top\-kk

Table[5](https://arxiv.org/html/2610.22919#A3.T5)gives the share of direct evidence among each phenomenon’skkmost influential sentences for other values ofkk\.

Table 5:Share of direct evidence among each phenomenon’skkmost influential sentences, as a mean over five reference models\. Bold numbers indicate a majority\. The last column is the share of all training sentences in the corpus that are marked as direct evidence\. This last column figure is used as the baseline number for the enrichment calculation in Figure[4](https://arxiv.org/html/2610.22919#S5.F4)\.
## Appendix DReproducibility, Compute, and AI Usage

#### Hardware\.

Training for each reference and validation model was performed on a single NVIDIA H100 GPU\. EK\-FAC attribution was performed with four L40S GPUs per model\. Characterization was performed with a mix of H100 GPUs and L40S GPUs\.

#### Compute\.

We trained 275 models for 40 epochs\. This includes the five reference models and 270 validation models \(150 indirect evidence and matched random models removing 60,000 sentences, 75 BM25 models, 25 direct\-evidence models, and 20 models removing 20,000 sentences\)\. One 40\-epoch training run took 13 to 27 H100\-hours \(median 23\), and training all 275 models took about 5,700 H100\-hours\. Superseded and preempted runs add about 270 more H100\-hours\. EK\-FAC fitting and influence functions scoring used 190 L40S\-hours\. Finally, the characterization analysis used about 8 GPU\-hours\. Causal validation took the vast majority of compute, while the attribution and characterization steps were relatively cheap\.

#### Licenses\.

The Wikipedia corpus of[Gulordava et al\. \(2018\)](https://arxiv.org/html/2610.22919#bib.bib6)is released under CC BY\-NC 4\.0, BLiMP of[Warstadt et al\. \(2020\)](https://arxiv.org/html/2610.22919#bib.bib21)under CC BY 4\.0, and the Filtered Corpus Training code of[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)under GPL\-3\.0\. Our usage of these resources for non\-commercial academic research complies with these licenses\.

#### Software\.

We train our language models with PyTorch 2\.5\.1 and Transformers 4\.46\.3\. We perform attribution using EK\-FAC influence with Kronfluence 1\.0\.1\. We perform parsing with Stanza 1\.13\. We score BM25 with rank\-bm25 0\.2\.2\.

#### AI usage\.

We used Claude and ChatGPT as AI assistants for coding and writing\. AI assisted with coding under the authors’ direction and supervision\. Relatedly, AI was used to critique the authors’ proposed analyses\. The authors had the final decision on which analyses to run\. AI was used for editing and proofreading of the paper during drafting and revision, for formatting, and for producing TikZ diagrams\. Similarly, AI was also used to help while drafting low\-novelty sections of the appendix \(compute, experimental details, controls\) using our code, configurations, and run records\. The authors take full responsibility for the content of this paper and its code\.

## Appendix EDirect\-Evidence Filters

We reproduce the twelve parse\-based filters used in[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)following their codebase\. We also reproduce their three NPI surface filters as described in[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)\. Each parse\-based filter utilizes a sentence’s Stanza dependency parse\([Qi et al\., 2020](https://arxiv.org/html/2610.22919#bib.bib17)\)\. These parse\-based filters perform filtering based on dependency relations, morphological features, and, for some phenomena, lexical lists\. One slight deviation is that we parse with Stanza 1\.13 and remove each sentence’s end\-of\-sentence token, whereas[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)used Stanza 1\.5 and kept the end\-of\-sentence token\.

For computing the structural coverage metric in Figure[6](https://arxiv.org/html/2610.22919#S5.F6), we split the direct evidence filter’s logic into necessary conditions that must all be true for the filter to hold, and when the original logic has a disjunction, we have alternative branching conditions where at least one of these branches should be satisfied for direct evidence\. We list these filter conditions in Table[7](https://arxiv.org/html/2610.22919#A5.T7)\. Our filters mark direct\-evidence shares of the training corpus within 0\.33 percentage points of the figures reported in[Patil et al\. \(2024, Table 1\)](https://arxiv.org/html/2610.22919#bib.bib16)for every phenomenon exceptpassive\(4\.30% vs\. 2\.67%\)\. This difference for the passive construction may be caused by the updated Stanza version’s parsing differences or by our removal of the end\-of\-sentence token\.

#### Structural coverage\.

Figure[6](https://arxiv.org/html/2610.22919#S5.F6)omits four phenomena due to their filters structurally leaving little to no variance in coverage, meaning that comparison would be uninformative\. For completeness of results, we provide results for these four phenomena in Table[6](https://arxiv.org/html/2610.22919#A5.T6)\. Forrelative\-clause distractor,relational\-noun distractorandreconstruction, the reason for zero variance is that any sentence that has the structural condition is direct evidence\. Therefore, for these three phenomena, all indirect evidence must have structural coverage of00\. Forregular/irregular plural, the only structural condition in the filter is having a subject, which nearly every sentence in the corpus has, so every group’s structural coverage is near11, making this phenomenon also uninformative for comparison\.

Table 6:Structural coverage for the four phenomena omitted from Figure[6](https://arxiv.org/html/2610.22919#S5.F6), as means computed over 5 reference models\.PhenomenonStructural conditions \(counted\)Lexical conditions \(not counted\)Max\.Determiner–noun agreementDeterminer–noundemonstrative determiner \(this, that, these, those\)its noun is on the BLiMP noun list \(450 words\)1Determiner–adj\.–noundemonstrative determiner \(this, that, these, those\); the demonstrative is directly followed by a non\-noun \(e\.g\., an adjective\)–0\.5Subject–verb agreementRegular/irregular pluralthe sentence has a subjecta subject is on the BLiMP subject list \(729 words, including names\)1Relative\-clause distractora relative clause modifies the subject or a noun modifying it–0Relational\-noun distractora prepositional modifier of the subject stands between the subject and its predicate–0Argument structurePassive\(a\)a passive verbthe verb is on the BLiMP passive list \(127 verbs\)1\(b\)a copula with a predicate \(e\.g\.,are dislikedparsed as an adjective\)the predicate is on the same listBindingDomain\(a\)a complementizer \(e\.g\.,that\) introducing a complement clause; an object or oblique of the clause’s head, after itthe object is a pronoun1\(b\)a reflexive depending on the head of a complement clause \(ccomporxcomp\)a reflexive \(\-self\)C\-commanda reflexive whose head also has a subject; a relative clause inside that subject, before the reflexivea reflexive \(\-self\)0\.5Case\(a\)a complementizer \(e\.g\.,that\) introducing a complement clause; the clause’s subject directly follows the complementizerthe subject is a pronoun1\(b\)a reflexive directly followed by a clause attached to the reflexive’s head \(xcomporadvcl\)a reflexive \(\-self\)Reconstructiona reflexive heading a clause with a subject \(It’s herself who …\)a reflexive \(\-self\)0QuantifiersSuperlativea comparative or superlative word \(e\.g\.,more,fewer\); that word is inside an object or oblique–0\.5Existential thereexistentialtherewithbe; thatbehas a subject with a determiner or modifierthe subject’s quantifier is one of BLiMP’s ten \(a, an, no, some, few, many, all, most, every, each\)1NPI licensingOnlyonlyoccursan NPI from the list follows it1Sentential negationa negation \(not,cannot,n’t\)an NPI from the list follows it1Matrix questionthe sentence ends in a question markthe sentence contains an NPI from the list1Table 7:Conditions for each direct\-evidence filter\. A sentence is considered direct evidence by the filter if it meets all conditions in at least one branch \(a, b\)\. The structural coverage of a sentence is the fraction of a branch’s structural conditions it meets, taking the best branch\. Lexical conditions \(BLiMP and NPI word lists, and whether a word is a pronoun or a reflexive\) do not count towards structural coverage\. Max\. is the highest coverage of any sentence outside the direct\-evidence filter, and thus indicates an upper limit for the structural coverage metric\. We reproduce the parse\-based filters from[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)and split them into conditions\. The three NPI filters used are surface filters following[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16), and their structural coverage is the presence of the licensing cue\.
#### NPI filters\.

We elaborate more on the construction of the surface filters for the NPIs\.[Patil et al\. \(2024, §3\.2\.1\)](https://arxiv.org/html/2610.22919#bib.bib16)describe the three NPI filters:npi\-onlyremoves sentences with an NPI afteronly,npi\-sent\-negsentences with a negation and an NPI, andnpi\-sim\-quesquestions with NPIs\. As they do, we take NPIs from[Jumelet et al\. \(2021\)](https://arxiv.org/html/2610.22919#bib.bib8), which are the 40 expressions in its Appendix A \(e\.g\.,any,ever,yet\)\. The filter checks for matches in the lowercased sentence with a space on each side\. Following that paper’s filtering rule, a sentence is direct evidence whenonlyor a negation \(not,cannot,n’t\) occurs anywhere before an NPI or, formatrix question, when it contains an NPI and ends in a question mark\.

#### Examples of indirect evidence\.

We elaborate more on the examples of indirect evidence provided in Table[1](https://arxiv.org/html/2610.22919#S5.T1)\. In general, some indirect evidence sentences are the phenomenon itself with words outside the filter’s lists:those productsisdeterminer–noun agreement, butproductsis not on the BLiMP noun list\. This is because of the lexical generalization design of[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)\. Other indirect evidence sentences are related constructions: inSeveral of ESPN’s personalities also portray themselves, the reflexive agrees with the subject, like thec\-commandpairs, but the relevant noun is in a prepositional phrase rather than a relative clause\.

## Appendix FControl groups

#### Matched random\.

We match indirect evidence sentences by their length in tokens and its mean word frequency: the average over its tokens oflog⁡c⁡\(w\)\+1N\+V\\log\\frac\{c\(w\)\+1\}\{N\+V\}, the add\-one smoothed log probability of tokenww, wherec⁡\(w\)c\(w\)is the tokenww’s count in the training corpus,NNthe corpus size in tokens, andVVthe vocabulary size\. Possible candidates for the matched random control group are the non\-direct sentences outside the indirect evidence\. Candidates are selected without replacement and each is used at most once in the matched random control group\. For each indirect evidence sentence, we randomly pick uniformly one of the 10 candidates of the same length with the closest mean frequency\. If no candidate of that exact length remains in the pool, we choose one of the 10 training sentences which are nearest on both covariates\. In the resulting matched random control group, sentence lengths are matched almost exactly, and the standardized mean difference in mean frequency is about 0\.02 \(0\.04 for the 20,000\-sentence groups of Appendix[G](https://arxiv.org/html/2610.22919#A7.SS0.SSS0.Px5)\)\.

#### BM25\.

We compute BM25 scores for each training sentence with BM25Okapi from rank\-bm25 0\.2\.2 \(k1=1\.5k\_\{1\}=1\.5,b=0\.75b=0\.75,ϵ=0\.25\\epsilon=0\.25\)\. We split sentences on whitespace\. We consider a phenomenon’s query to be the set of words in its query pairs, from both of the sentences in a minimal pair\. The BM25 group is the 60,000 highest\-scoring non\-direct sentences\. Note that the BM25 control group does not depend on the reference model and is shared between reference models; the trained BM25 models differ because of different initializations\. The lexical similarity in Figure[5](https://arxiv.org/html/2610.22919#S5.F5)is a group’s mean BM25 score\. Thus, by definition, the BM25 group has the highest lexical similarity of any 60,000 sentences out of the pool of non\-direct sentences\.

#### Direct evidence\.

For retraining, the direct\-evidence group is a uniform random sample of 60,000 sentences that match the direct\-evidence filter\. This control group does not depend on the reference model and is shared between reference models\. There are five phenomena with at least 60,000 direct\-evidence sentences:relational\-noun distractor,relative\-clause distractor,regular/irregular plural,superlative, andpassive\. Importantly, we are comparing removals of equal size rather than removing all direct evidence as was done in[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16); the direct evidence control group removes between 13% \(relational\-noun distractor\) and 87% \(relative\-clause distractor\) of their direct evidence\. Our direct evidence retraining results should not be taken to be comparable to[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16)’s results\.

## Appendix GAdditional validation results

#### Results by phenomenon\.

Table[8](https://arxiv.org/html/2610.22919#A7.T8)gives the held\-out accuracies behind Figure[2](https://arxiv.org/html/2610.22919#S4.F2)\.

Table 8:Held\-out BLiMP accuracy \(%\) of the reference models and of the models retrained without each group, as mean±\\pmstandard deviation over the five models\. Direct evidence is a random 60,000 of its sentences, removed only for the five phenomena with at least that many\. Extra drop is the accuracy lost by removing the indirect evidence relative to the matched random group: mean±\\pmstandard deviation of the paired difference over the five models\.
#### Significance test\.

Our reported statistical test in Section[4](https://arxiv.org/html/2610.22919#S4)is an exact one\-sided sign\-flip test over the five reference models\. The sign\-flip test is on held\-out loss, the logistic margin loss of §[2\.2](https://arxiv.org/html/2610.22919#S2.SS2)\. For each reference model, we average over the 15 phenomena the extra loss from removing the indirect evidence relative to removing the matched random group\. We compare the observed mean paired difference with all25=322^\{5\}=32sign assignments\. Thus, the smallest possibleppfrom this sign\-flip test is1/321/32, which we achieve\.

#### Specificity\.

Figure[3](https://arxiv.org/html/2610.22919#S4.F3)shows how much each phenomenon’s held\-out accuracy drops when any phenomenon’s indirect evidence is removed\. The diagonal gives each phenomenon’s own drop \(median 11\.4 points\)\. The median of off\-diagonal cells is 0\.3\. The three largest off\-diagonal drops are within categories\. For example,determiner–nounanddeterminer–adjective–noun agreementhurt each other by 8\.3 and 8\.2 points \(the two differ structurally only by the intervening adjective, and their indirect evidence overlaps by38%38\\%\)\.

#### Checkpoint\.

In Section[4](https://arxiv.org/html/2610.22919#S4), we evaluated each validation model at its reference model’s selected epoch\. One may ask if this choice of when to evaluate the validation model could materially impact the results\. We find that it does not change our conclusions\. Our results do not change much if we evaluate the validation model instead at its own lowest\-validation\-loss epoch\. Removing the indirect evidence lowers accuracy more than the matched random group in all 75 model pairs\. Averaged over the five initializations, the range of the extra drop is 3\.3 to 45\.6 points \(median 12\.0\)\. Similarly, our results also do not change much if we evaluate after all4040epochs\. When evaluating after4040epochs, removing the indirect evidence lowers accuracy more than the matched random group in all 75 model pairs\. Averaged over the five initializations, the range of the extra drop is 3\.2 to 44\.1 points \(median 12\.4\)\. Thus, this suggests our findings are robust to changing to various choices of when we evaluate the validation models\.

#### Removal size\.

We want to understand the effect of varying the hyperparameterkk, which is the size of the indirect evidence set that we study\. For two phenomena, we also removed the 20,000 most influential non\-direct sentences and a matched random group of the same size\. Table[9](https://arxiv.org/html/2610.22919#A7.T9)shows a smaller but still positive extra drop in accuracy and extra loss compared to a matched random group of the same size\. While we only performed this comparison for two phenomena, this result does suggest that our methodology should find causally important indirect evidence sets at variouskk\.

Table 9:A comparison of the effect of removing the most influential indirect evidence at two different sizes, as means over five models\. The displayed extra drop for held\-out loss and held\-out accuracy are relative to the matched random control group of the same size\.

## Appendix HShared indirect evidence

Figure[8](https://arxiv.org/html/2610.22919#A8.F8)shows the overlap between the indirect evidence of every pair of phenomena\. The clustering in Figure[7](https://arxiv.org/html/2610.22919#S5.F7)uses this same overlap metric\.

![Refer to caption](https://arxiv.org/html/2610.22919v1/family_membership_all_targets_column.png)Figure 8:Overlap between phenomena’s indirect evidence\. Each cell in the heatmap is the share of one phenomenon’s 60,000 indirect evidence sentences that are also in the other’s indirect evidence, averaged over five reference models\. Black outlines mark BLiMP categories\.
## Appendix IInfluence estimation details

Table[10](https://arxiv.org/html/2610.22919#A9.T10)lists the EK\-FAC settings, shared by all five reference models\.

Table 10:EK\-FAC settings, shared by all five reference models\.#### Fisher matrix\.

The EK\-FAC method uses the Fisher matrix to approximate the curvature term in the efficient computation of influence functions scores\. There are two methods of calculating the Fisher matrix: the empirical Fisher and the true Fisher\. In this study, we use the empirical Fisher which uses gradients for the observed next tokens in the corpus, while the true Fisher uses gradients for next tokens sampled from the model’s prediction distribution\. The two methods of calculating the Fisher matrix are the same in expectation when the model’s prediction distributions match the data distribution\. In a preliminary comparison on a published model from[Patil et al\. \(2024\)](https://arxiv.org/html/2610.22919#bib.bib16), we scoreddeterminer–noun agreementagainst a 500,000\-sentence sample of the corpus with both variants and fit the factors on 8,192 sentences\. We obtained almost identical rankings \(Spearmanρ=0\.998\\rho=0\.998\), with the top 60,000 influential sentences having an overlap of97%97\\%\. This result indicates that our choice of how to calculate the Fisher matrix does not materially impact our study; we only depend on the ranking of the training sentences obtained from influence functions scores, and we do not directly utilize the magnitude of influence functions scores for our analysis\.