Evaluating Causal Reasoning in LLMs: A Case Study in the Challenges of Procedural Generation

# Evaluating Causal Reasoning in LLMs:A Case Study in the Challengesof Procedural Generation
Source: [https://arxiv.org/html/2610.38100](https://arxiv.org/html/2610.38100)
Isha KarnPatrick MurrayAffiliation:KJ HillDavid JensenAffiliation:University of Massachusetts AmherstAffiliation:jbclarke@umass\.edu,ikarn@umass\.edu,psmurray@umass\.eduAffiliation:kjhill424@gmail\.com,jensen@umass\.edu

###### Abstract

Large language models \(LLMs\) appear to represent and reason about a large range of phenomena\. In particular, recent studies have examined whether LLMs can accurately reason about causation\. One of the most commonly used benchmarks for evaluating formal causal reasoning in LLMs is CLadder, a framework for procedural generation of benchmark data sets that was released in 2023\. Experiments with CLadderappear to indicate that LLMs can perform formal causal reasoning in a large percentage of generated examples\. In this paper, we examine what can be reliably inferred when LLMs perform well on test sets produced by CLadder\. We identify several examples of superficial and highly predictive patterns that the CLaddergenerator embeds in data sets it generates\. We find that even simple models such as random forests perform well on CLadder\-generated data sets after training on a sample of generated data\. We also find that the performance of many commercial LLMs on CLaddertest sets is extremely brittle and that removing large sections of the text that are critical for formal causal reasoning has almost no effect on performance\. Given these results, the most likely hypothesis for the strong performance of state\-of\-the\-art LLMs on CLadderis that they are exploiting superficial patterns learned from exposure to training data from the generator\. Any inference that LLMs are exhibiting more complex causal reasoning must rule out this hypothesis\.

## 1Introduction

One of the most important and challenging questions about large language models \(LLMs\) is the extent to which these models exhibit complex reasoning\. On the one hand, LLMs regularly respond to prompts in ways that appear to require accurate reasoning about logic, mathematics, physics, and causation, as well as knowledge about the natural, physical, and social worlds\([Hendrycks et al\., 2021](https://arxiv.org/html/2610.38100#bib.bib32);[Wei et al\., 2022](https://arxiv.org/html/2610.38100#bib.bib33);[Chen et al\., 2021](https://arxiv.org/html/2610.38100#bib.bib34);[Kiciman et al\., 2024](https://arxiv.org/html/2610.38100#bib.bib29)\)\. On the other hand, much of this apparent knowledge and reasoning could be recalled verbatim from the massive amounts of data used to train LLMs, as well as some lightweight taxonomic knowledge gleaned from how humans produce natural language\([Carlini et al\., 2022](https://arxiv.org/html/2610.38100#bib.bib35);[Collins et al\., 2022](https://arxiv.org/html/2610.38100#bib.bib36)\)\. In short, researchers have sought to determine where LLMs sit between the extremes of artificial general intelligence and stochastic parrots\([Zečević et al\., 2023](https://arxiv.org/html/2610.38100#bib.bib18)\)\.

![Refer to caption](https://arxiv.org/html/2610.38100v1/intro_graphic_final.png)Figure 1:This paper examines how and why LLMs perform well on CLadder, a dynamic benchmark for evaluating causal reasoning, and what can be inferred from the results\.A major challenge in resolving this question has concerned evaluation — how to create large sets of questions and \(correct\) answers to evaluate specific types of reasoning\. Early evaluations often repurposed existing questions originally designed to assess human knowledge and reasoning, including college entrance examinations, intelligence tests, and textbook exercises\([Achiam et al\., 2023](https://arxiv.org/html/2610.38100#bib.bib7);[Hendrycks et al\., 2021](https://arxiv.org/html/2610.38100#bib.bib32), e\.g\., \)\. However, because many such benchmarks were publicly available before contemporary LLMs were trained, researchers raised concerns that their content may have appeared in model training corpora and been memorized by LLMs\([Xu et al\., 2024](https://arxiv.org/html/2610.38100#bib.bib8);[Sainz et al\., 2023](https://arxiv.org/html/2610.38100#bib.bib16)\)\. These concerns are particularly difficult to resolve for closed models whose training data are not publicly documented\([Deng et al\., 2024](https://arxiv.org/html/2610.38100#bib.bib37)\)\. Subsequent analyses have found evidence of benchmark\-specific overfitting and possible memorization in both open and closed models\.\([Zhang et al\., 2024](https://arxiv.org/html/2610.38100#bib.bib13), e\.g\., \)\. As one researcher has quipped: “Pretraining on the test set is all you need”\([Schaeffer, 2023](https://arxiv.org/html/2610.38100#bib.bib27)\)\.

Given the problems with static benchmark datasets, an alternative approach to evaluating LLM reasoning has gained prominence:procedural generation\. With procedural generation, test sets are generated algorithmically rather than being generated manually or gathered from the real world\([Chen et al\., 2025](https://arxiv.org/html/2610.38100#bib.bib5);[Risi and Togelius, 2020](https://arxiv.org/html/2610.38100#bib.bib6)\)\. In the context of evaluating LLM reasoning, procedural generation allows new test sets to be generated as needed, making it highly unlikely that test set instances could have been memorized\([Zhu et al\., 2024](https://arxiv.org/html/2610.38100#bib.bib38);[White et al\., 2025](https://arxiv.org/html/2610.38100#bib.bib39)\)\. Such “dynamic” test sets appear to overcome some of the most obvious problems associated with earlier “static” test sets\. Procedural generation is best suited for reasoning tasks that have been sufficiently formalized such that questions can be randomly sampled from an extremely large \(potentially infinite\) space and their correct answers can be computed or verified automatically\([Wei et al\., 2025](https://arxiv.org/html/2610.38100#bib.bib40)\)\. Fortunately, many reasoning tasks have well\-established formal theories that allow for the construction of procedural generators\. These include mathematics, statistics, logic, and \(most importantly for this paper\) causal reasoning\([Saxton et al\., 2019](https://arxiv.org/html/2610.38100#bib.bib41);[Wei et al\., 2025](https://arxiv.org/html/2610.38100#bib.bib40);[Jin et al\., 2023](https://arxiv.org/html/2610.38100#bib.bib1), e\.g\.,\)\.

In this work, we examine CLadder, a particularly prominent and important procedural generator for evaluating the causal reasoning abilities of LLMs\. Specifically, we focus on a single research question:What can be reliably inferred about the reasoning ability of an LLM when it performs well on a CLadder\-generated test set?We focus on this question because a large range of papers have implicitly or explicitly concluded that high performance on CLadder\-generated test sets implies that a given LLM has learned to perform formal causal inference, and that such reasoning capabilities will generalize to other scenarios\. In contrast, our findings strongly indicate that such inferences are unwarranted\. A far more likely conclusion is that high\-performing models have been exposed to one or more CLadderdata sets and that the models have learned a set of superficial patterns that allow for high performance\. While the ability to learn such patterns is impressive, it is not indicative of formal causal reasoning, and such performance is unlikely to generalize beyond data sets generated by CLadder\.

As summarized in Figure[1](https://arxiv.org/html/2610.38100#S1.F1), we show that:

- •CLadderembeds superficial predictive patterns in the data it produces— We show that CLadder’s data generating process explicitly embeds superficial patterns in every data set that it produces\. These patterns are simple and predictive of correct answers\.
- •These patterns are learnable from exposure to a single CLadderdata set— We show that a variety of simple and complex models can learn these superficial patterns and perform well on subsequent test sets\. Simple models such as random forests can learn them when given superficial features derived from the queries\. Open\-weight LLMs can also learn these predictive patterns from Fine\-tuning on CLadder\-generated queries and answers\.
- •Exposure to CLadderdatasets is likely for many modern LLMs— CLadder\-generated data has been popular and has proliferated widely since the paper’s initial publication\. CLadderdata is likely to be included in the training corpus of many LLMs\.
- •Seemingly essential information is not required for high performance— Some information in CLadderqueries is known to be essential for formal causal reasoning\. We show that removing such information does not substantially impair LLM performance\.
- •Seemingly incidental information is critical to model performance— We show that omitting seemingly incidental tokens from specific CLadderqueries can dramatically affect the answers provided by LLMs\.
- •High\-performance on CLadderdoes not generalize— Models that achieve high accuracy on CLadderperform no better than random on CounterBench, a recently released benchmark for causal reasoning with a very similar overall design and phrasing of causal queries\.

## 2CLadder: Background, Design, and Prior Results

Causal reasoning is often identified as one of the most important and complex forms of reasoning that humans do, and thus causal reasoning has become an explicit target of efforts to evaluate the wider reasoning capabilities of LLMs\. Several prominent researchers have explicitly identified causal reasoning as a key gap in the reasoning capabilities of LLMs\([Zhang et al\., 2023](https://arxiv.org/html/2610.38100#bib.bib28);[Wu et al\., 2024](https://arxiv.org/html/2610.38100#bib.bib20)\)\.

Early attempts to empirically evaluate causal reasoning in LLMs concentrated on commonsense causality, assessing whether models correctly assess everyday causal knowledge expressed in natural language\([Roemmele et al\., 2011](https://arxiv.org/html/2610.38100#bib.bib30)\)\. However, such evaluations conflate causalknowledgewith causalreasoning, since a model could answer correctly by manipulating linguistic patterns rather than reasoning about the abstract causal concepts expressed in that language\. Surveys of this work\([Wu et al\., 2024](https://arxiv.org/html/2610.38100#bib.bib20), e\.g\., \)note that while models can generate causal language that sounds accurate, they fall short of genuine causal reasoning\.

The introduction of the CLadderbenchmark\([Jin et al\., 2023](https://arxiv.org/html/2610.38100#bib.bib1)\)was a substantial milestone in empirical evaluations of causal reasoning in LLMs\. CLadderleverages a formal theory of causal reasoning\([Pearl, 2009](https://arxiv.org/html/2610.38100#bib.bib9)\)to create a procedural generator of question\-answer pairs that range across ten distinct types of causal reasoning \(average treatment effect, backdoor adjustment, collider bias, correlation, deterministic counterfactual, effect of treatment on the treated, explaining away, marginal effect, natural direct effect, and natural indirect effect\) and that span Pearl’s three rungs of causation \(association, intervention, and counterfactual reasoning\)\.

### 2\.1What is CLadder?

CLadderis a procedural generator of question\-answer pairs\. Questions are generated by randomly sampling from several sets of data structures that completely specify a causal reasoning task \(e\.g\., problem type, graphical model, probability values, etc\.\) and then converting this information into a query in natural language\. Correct answers to the query are also produced automatically by construction, based on the formal data structures\. All queries have binary \(yes/no\) answers\. Full details can be found in the original paper\([Jin et al\., 2023](https://arxiv.org/html/2610.38100#bib.bib1)\)\.

Such a procedural generation approach would appear to entirely escape the problems of prior “static” test sets that could have appeared in the training data of an LLM\. Consequently, the results in the original paper \(and subsequent papers that have used CLadder\) have far more credibility than prior work that used static test sets\. It is worth noting that, in addition to the CLaddergenerator,[Jin et al\.](https://arxiv.org/html/2610.38100#bib.bib1)released several demonstration test sets that illustrate the ability of the generator to produce data with different properties \(e\.g\., use of nonsense words in place of common nouns and causal dependencies that defied “commonsense” expectations\)\. The full demonstration dataset contains 41,748 queries\.

To avoid memorization issues, we generated a new dataset for our experiments with 9866 unique samples that includes all the combinations of options available using the standard implementation of the CLaddergenerator\. The dataset contains 1200 queries for each query type, unless there are fewer than 1200 unique possibilities, in which case we include them all\. See Appendix[A\.3](https://arxiv.org/html/2610.38100#A1.SS3)for further details on our generated dataset\.

### 2\.2Impact of CLadder

The CLadderpaper had a strong effect on both the methods and the conclusions of the AI research community\. In less than three years since its publication, the paper has been cited over 380 times, and CLadderdata has been used in more than 20 other empirical evaluations of LLM reasoning\([Yu et al\., 2025](https://arxiv.org/html/2610.38100#bib.bib42);[He et al\., 2026](https://arxiv.org/html/2610.38100#bib.bib43);[Momentè et al\., 2025](https://arxiv.org/html/2610.38100#bib.bib44);[Mao and Venkat, 2026](https://arxiv.org/html/2610.38100#bib.bib45);[Schrader et al\., 2024](https://arxiv.org/html/2610.38100#bib.bib46), e\.g\., \)\. In some cases, those studies used the CLaddergenerator\. In others, however, the original demonstration test sets were used as static benchmarks\.

The results in the original paper and subsequent papers that used CLadderhave been widely cited as evidence that LLMs can perform effective causal reasoning\. It is worth noting that[Jin et al\.](https://arxiv.org/html/2610.38100#bib.bib1)were quite cautious in the original paper, saying that “our work suggests that LLMs may still be far from reasoning reliably about causality\.” That said, they also claim that some LLM prompting methods presented in the paper “elicit sound causal reasoning in LLMs and help them solve challenging causality questions\.” Subsequent authors have made substantially stronger claims based on results from CLadderin the original and subsequent papers\([Gui and Ji, 2025](https://arxiv.org/html/2610.38100#bib.bib31);[Chen et al\., 2026a](https://arxiv.org/html/2610.38100#bib.bib19), e\.g\., \)\.

Only a few papers have attempted to understand the underlying source of the observed performance in ways that could clarify what can be inferred from high performance\.[Jin et al\.](https://arxiv.org/html/2610.38100#bib.bib1)themselves attempted to evaluate the degree to which surface linguistic information in queries affected the answers provided by LLMs\. They substituted nonsense words for nouns in causal queries and observed essentially no change in performance\. Similarly,[Pacchiardi et al\.](https://arxiv.org/html/2610.38100#bib.bib4)analyzed a variety of LLM reasoning benchmarks for surface linguistic features that predict correct answers, and conclude that CLaddertest sets are among the least susceptible to learning of generator\-specific shortcuts\.

## 3Experiments and Key Findings

In this section, we report the findings from analyses and experiments that inform our view on what can be inferred when a model accurately answers queries in CLadder\-generated data sets\.

### 3\.1Highly Predictive Patterns are Directly Embedded by the Generator

The methods used to generate CLadderqueries introduce multiple patterns into the data that can be learned and exploited by a model\. The data generator produces each query by combining elements from different templates depending on the specified variables and causal graph\. This method of generating queries leads to predictable elements that are repeated in any data produced by the generator, such as the structure and specific phrasing of the queries \(see Appendix[A\.7](https://arxiv.org/html/2610.38100#A1.SS7)for additional details\)\. In many cases, these patterns provide shortcuts that allow a model to correctly answer the queries without the need for any causal reasoning\.

For example, after providing the graph structure, many of the average treatment effect queries have the form ”P⁡\(X=1\|Y=0\)=p1,P⁡\(X=1\|Y=1\)=p2\.P\(X=1\|Y=0\)=p\_\{1\},P\(X=1\|Y=1\)=p\_\{2\}\.Willd​o​\(Y=1\)do\(Y=1\)increase the chance ofX=1X=1?” This query can be trivially solved by comparingp1p\_\{1\}andp2p\_\{2\}and checking the polarity of the query \(i\.e\. whether it is phrased as ”increase” or ”decrease”\)\. Once that pattern is apparent, other elements of the query, including the structure of the graph and names of the variables, are no longer necessary\. In Figure[2](https://arxiv.org/html/2610.38100#S3.F2), we plot the first probability given in the average treatment effect queries versus the second given probability\. We can see that for each polarity the queries can be classified nearly perfectly using only the two probability values\. This is not an isolated case: A very similar pattern can be observed in queries that ask about the effect of treatment on the treated\. We note that some of the examples shown in Figure[2](https://arxiv.org/html/2610.38100#S3.F2)have as many as seven probability values provided in the query, even though this highly predictive pattern uses only the first two\.

The patterns above are fairly easy to identify, but we have identified other patterns in CLadder\-generated data that are far more subtle and difficult to detect by inspection alone\. Without a way to mitigate this issue, there is a risk that any model exposed to the data during training may learn superficial shortcuts associated with the data generation procedure\. This will likely lead to strong performance on the CLadderdatasets but poor generalization\.

Figure 2:Scatter plots of the two key conditional probabilities provided in average treatment effect \(ATE\) and effect of treatment on the treated \(ETT\) queries, colored by correct answer\. Within each polarity condition, yes and no answers are nearly linearly separable\. A model that learns this simple comparison rule can achieve near\-perfect accuracy on these query types without any formal causal reasoning\.
### 3\.2Simple models can learn the predictive patterns

We have shown that predictive patterns exist within CLadderdata, but can these patterns be learned by models and, if so, how well do they work? To answer these questions, we created two sets of superficial features derived directly from the query text: a small set of surface\-level features that characterize the query text, as well as a fixed\-length BERT embedding\([Devlin et al\., 2019](https://arxiv.org/html/2610.38100#bib.bib3)\)of each query\. The BERT embeddings do not represent the probabilities that appear in the query, so they are appended to the embedding \(see Appendix[A\.8](https://arxiv.org/html/2610.38100#A1.SS8)for feature descriptions\)\.

We used each set of features to learn a random forest classifier that predicts the binary answer for each query\. The cross\-validated accuracy estimates for each feature set, disaggregated by query type, are shown in Figure[3](https://arxiv.org/html/2610.38100#S3.F3)\. For some query types \(e\.g\., collider bias and marginal\), the random forest classifier was able to predict the answer with very high accuracy \(100% and 94%, respectively\)\. Averaged across all query types, the classifier achieved a cross\-validated accuracy of 87%, roughly comparable to the accuracy achieved by current language models\.

![Refer to caption](https://arxiv.org/html/2610.38100v1/variant_heatmap_v2.png)Figure 3:Model accuracies by query type\. The first two rows show accuracy by query type on our generated CLaddertest set for a random forest classifier trained on superficial features derived from the query text or BERT embeddings\. We report the mean accuracy over 10\-fold cross\-validation\. The third row shows the performance of OLMo\-3\.1\-32B after fine\-tuning on CLadder\. We also show gpt\-5\.6\-sol with and without reasoning for comparison\.To explore the effect of CLadderexposure on LLMs, we also investigate whether supervised fine\-tuning on CLadderqueries produces significant improvements in performance\. We select four instruction\-tuned models for which prior exposure to CLadderis unlikely based on their release dates or the availability of information about their training data: OLMo\-3\.1\-32B\-Instruct, OLMo\-3\-7B\-Instruct, Llama\-3\.1\-8B\-Instruct, and Qwen2\.5\-3B\-Instruct\. Consistent with limited or no prior exposure, zero\-shot performance on the generated queries \(n = 9,866\) is close to chance for all four models, with accuracies of 63\.9±\\pm0\.9 %, 55\.3±\\pm1\.0, 55\.7±\\pm1\.0 and 52\.1±\\pm1\.0, respectively\.

When trained on 2,000 labeled examples, all fine\-tuned models exceeded 80% overall accuracy\. When trained on 16,000 examples, every model exceeded 94% overall accuracy\. These results, shown in Appendix[A\.5](https://arxiv.org/html/2610.38100#A1.SS5), indicate that all models improve at approximately the same rate, which is not what we would expect from models of such different sizes \(Qwen2\.5\-3B vs OLMo\-3\.1\-32B\)\. The performance of all the models scales with the number of labeled training examples the model has been exposed to, which is consistent with learning superficial features\.

Taken together, these results show that combinations of superficial features are sufficient to perform at state\-of\-the\-art levels on CLaddertest sets\. Even limited exposure to CLadderdata is sufficient for models to learn generator\-specific shortcuts that transfer across CLadderdatasets\. At the very least, these results should create a new, minimum baseline for surprising performance of LLM\-based models\.

### 3\.3Models may have been exposed to CLadderdata

The ability of relatively simple models to quickly acquire the knowledge necessary for high performance in CLaddercasts substantial doubt on the conclusion that evaluations using the benchmark indicate any degree of formal causal reasoning\. This is particularly true if datasets from CLadderhave been included in the training data of LLMs\. Can we rule out such exposure?

CLadderhas been publicly available since its original release\. It has been copied, forked, and redistributed across a large and still\-growing number of public repositories\. We do not claim that any single evaluation, model, or copy of the dataset is the source of this exposure risk, since any one of these many copies is sufficient to contaminate a model’s training data regardless of whether it was ever exposed to the canonical release\. That alone is enough to complicate the interpretation of any post\-release evaluation’s results, because high performance cannot cleanly be attributed to improvements in model reasoning abilities when exposure to the data \(in some form, through some channel\) cannot be ruled out\.

We evaluate the performance of several models of varying size and complexity on the dataset we produced using the CLaddergenerator[2\.1](https://arxiv.org/html/2610.38100#S2.SS1)\. Figure[4](https://arxiv.org/html/2610.38100#S3.F4)shows the performance of each model versus the date of the model’s public release\. We observe a clear trend of performance improving over time, with early evaluations finding results slightly above random, and a steady increase in performance leading up to current ”reasoning” models, which regularly achieve over 90% accuracy\. But what is the cause of this improvement in performance?

Figure 4:Model accuracy compared to the date of the model’s release\. Open circles represent the published ”knowledge cutoff” dates for each model\. The results for the text\-davinci models come from the original CLadderpaper\.In general, model size and complexity has increased significantly over the time frame shown in Figure[4](https://arxiv.org/html/2610.38100#S3.F4)\. However, many of these models were trained after the public release of the CLaddergenerator and example datasets\. Without access to the training corpus, there is no way to verify whether the models trained after the dataset’s release were exposed during training\. In light of our findings above, it appears likely that these models have learned the patterns embedded in the published datasets, confounding any evaluation of their causal reasoning abilities using CLadderas a benchmark\.

### 3\.4Removing essential information has little effect on performance

Above we show that the use of simple features is sufficient to explain high performance on CLadder\. Next, we take a different approach: We examine whether LLMs still perform well on CLadderqueries when they lack the information that is necessary for formal causal reasoning\. Specifically, we remove key components of the queries that leave them unsolvable by formal causal reasoning, and we find that such ablations produce no substantial drop in LLM performance\.

Each query begins with a statement establishing a set of background conditions and specifying the structure of the causal graph used to generate the query \(see Figure[5](https://arxiv.org/html/2610.38100#S3.F5)for an example\)\. Together, these elements are necessary to make the causal query “identifiable” \(answerable with a unique quantitative solution\)\. Our experiments remove each of these elements individually, and then together\. In all cases, we observe minimal reductions in accuracy\. Across five evaluated frontier models, these interventions resulted in reductions in overall accuracy of only 0\.6 to 4\.2 percentage points relative to the baseline performance on the unaltered queries\.

![Refer to caption](https://arxiv.org/html/2610.38100v1/intervention_figure_mockup-Fig4-Var-B.drawio.png)Figure 5:Model accuracy under interventions that remove elements of the queries that are critical for formal causal reasoning\. A full query\-wise breakdown in Appendix[A\.6](https://arxiv.org/html/2610.38100#A1.SS6)\. The 95% Wilson confidence intervals are not visible in this plot due to large sample size\.Next, we perform a set of even more aggressive interventions, resulting in queries where it is impossible to calculate the answer with the available information\. We begin by removing only the first marginal or conditional probability value provided in the query, followed by removing only the second value provided\. Finally, we remove all the marginal or conditional probability values that are provided with the query \(identified as the ”given\_info” in the CLadderqueries\)\. The result in all cases are queries with insufficient information to calculate the correct answer\.

These interventions, shown in Figure[6](https://arxiv.org/html/2610.38100#S3.F6), result in a larger reduction in accuracy than those in Figure[5](https://arxiv.org/html/2610.38100#S3.F5), which makes sense due to the critical nature of the information being removed\. However, we would expect performance to collapse to 50% in all three cases, as models should not be able to perform better than random chance\. Instead, we see the accuracy reduced, but still remaining well above 50%\. Across all three versions of the intervention, the models perform 11 to 21 percentage points above random\.

![Refer to caption](https://arxiv.org/html/2610.38100v1/intervention_figure_mockup-Fig6-Var-B.drawio.png)Figure 6:Model accuracy under interventions that remove elements of the queries that make it impossible to calculate the answer\. Full query\-wise breakdown in Appendix[A\.6](https://arxiv.org/html/2610.38100#A1.SS6)\. The 95% Wilson confidence intervals are not visible in this plot due to large sample size\.Taken together, these results indicate that the models are unlikely to be performing formal causal reasoning\. A model that was doing so should not perform any better than random after such critical information is removed\. In fact, these results show that large sections of the text that are critical for formal causal reasoning actually have a negligible effect on model performance\. Instead, the models appear to be exploiting other shortcuts to correctly answer the queries, including ones that do not rely on elements of the query that formal causal reasoning methods would require\.

### 3\.5Seemingly incidental information greatly affects performance

The results above indicate that models largely ignore information that is essential for formal causal reasoning\. That raises a natural question: What informationdomodels use? In the next set of experiments, we aimed to localize the elements of the query that have the largest impact on model performance\. We establish the model’s baseline accuracy for each query, and then we re\-process the query with a single token removed and observe any change in performance\. The models we tested proved to be very brittle under these types of interventions, and highly sensitive to the presence or absence of specific tokens\. We often see a large change in performance when tokens are removed that are incidental to a human’s ability to solve the problem, and we find extreme variations in both baseline performance and sensitivity to individual tokens for nearly identical queries\.

We find a surprising amount of variation in which tokens affect model performance, even between queries of the same type\. Figure[7](https://arxiv.org/html/2610.38100#S3.F7)shows the model’s sensitivity to the removal of the token in each position for a set of queries drawn from the “ marginal” queries\. We run each query 100 times to establish a baseline accuracy, then 100 times for each token that is removed\. We can see that despite being nearly identical in difficulty and appearance from a human’s perspective, there is a significant variation in the response to the removal of each token\. There are several tokens whose removal produces a complete collapse in accuracy \(Δ=−1\.0\\Delta=\-1\.0\), suggesting the model relies heavily on specific tokens\. See Appendix[A\.10](https://arxiv.org/html/2610.38100#A1.SS10)for additional examples\.

![Refer to caption](https://arxiv.org/html/2610.38100v1/main_token_occ.png)Figure 7:Token ablation results for a set of marginal queries on gpt\-5\.2\. Each row corresponds to a single prompt, and each cell represents one token\. Red cells denote tokens whose removalreducesmodel performance, while green cells denote tokens whose removalincreasesperformance\. Numerical tokens \(\#\), percentage tokens \(%\), and the polarity indicator are always labeled\. Otherwise, labels are shown only for tokens exceeding a salience threshold \(\|Δ\|≥0\.04\|\\Delta\|\\geq 0\.04\)\.Looking closely at Figure[7](https://arxiv.org/html/2610.38100#S3.F7)we can see the surprising variation we find in the model’s response to nearly identical queries\. For the first query, labeledQ​17820Q17820, where the baseline accuracy is 100%, removing the single token “hard” in the compound word “hard\-working” can reliably flip the model’s answer such that its accuracy on the query is approximately 0%\. The second query in the figure, labeledQ​7131Q7131, is nearly identical aside from the variable names\. Except in this case we observe a baseline accuracy of only 8\.0%, even though these queries would be of equal difficulty for a human\. Additionally, for the query with low baseline performance we find there are many tokens whose removal results in a significant increase in performance to nearly 100% accuracy\. And again, we see that removing tokens that would appear inconsequential to a human seems to have a large impact on the model’s performance\.

Variation in the model’s response to similar queries, as well as the surprising impact of individual tokens on model performance, are both strong indicators that the models are exploiting superficial patterns to solve the queries, and that disrupting these patterns can cause dramatic shifts in performance\.

### 3\.6Model Performance on CLadderDoes Not Generalize

The experiments described above establish that high performance on CLadderdatasets is achievable through mechanisms that do not require formal causal reasoning\. If models were performing formal causal reasoning, or even exploiting broadly generalizable patterns, we would expect these abilities to generalize to structurally similar causal reasoning tasks\. If instead their performance is dependent on patterns that are specific to CLadder’s data generation process, we would not expect those patterns to generalize to an independent benchmark\.

We evaluate several frontier models on CounterBench\([Chen et al\., 2026b](https://arxiv.org/html/2610.38100#bib.bib47)\), a recently released benchmark for counterfactual causal reasoning that uses abstract variable names and was independently produced\. Table[4](https://arxiv.org/html/2610.38100#A1.T4)in Appendix[A\.9](https://arxiv.org/html/2610.38100#A1.SS9)shows the accuracy for each model on CounterBench\. All frontier models score near chance on CounterBench \(48\.6% – 52\.2%\)\. The gap between CLadderand CounterBench performance for different models ranges from 29 to 44 percentage points\.

Surprisingly, the random forest classifier trained exclusively on superficial features derived from query text transfers substantially better to CounterBench, achieving 79\.1%, as shown in Table[4](https://arxiv.org/html/2610.38100#A1.T4)\. It performed substantially above every frontier LLM we tested, suggesting that the frontier LLMs are not relying on the same transferable combination of features captured by the random forests or the fine\-tuned models\.

## 4Discussion and Conclusion

The converging lines of evidence in Section 3 strongly suggest that high performance on CLadderdoes not reliably imply that LLMs are performing some type of formal causal reasoning\. Instead, exposure to CLadderdata and superficial pattern learning seems much more likely\. Clearly, this has direct implications for what we should conclude from the results of prior evaluations that employed CLadderdata sets and how the generator should be used in future studies\.

More broadly, our results illustrate a limitation of procedural generation as a strategy for evaluating LLM reasoning\. Many of the patterns in CLadderare not easily detected by inspection, and they appear to be an artifact of the generation process itself rather than resulting from design flaws that could be easily fixed\. The combination of procedural generation and public examples of generated data may be incompatible with evaluating powerful machine learning models\. While such an approach largely escapes the problem of a model memorizing training data that will also appear as test data, it does not escape the problem of a model learning agenerator’sintrinsic patterns\.

This suggests at least three possible ways forward\. First, in theory, a procedural generator could be constructed that is so faithful to the intended domain that its only embedded patterns are those that will generalize to real world systems\. This seems unlikely, unless real\-world systems are directly used to generate data\. Second, a less faithful procedural generator could be constructed and then withheld from public view\. Without public access to either the generator itself or data from the generator, its intrinsic patterns could not easily be learned\. This strategy has already been taken in other evaluations of causal inference methods that have been somewhat successful \(e\.g\.,[Dorie et al\. \(2019\)](https://arxiv.org/html/2610.38100#bib.bib2)\)\. Third, if neither of these approaches is practical, then LLMs should always be compared to a baseline of a simpler model directly trained on the relevant public data set\. This will, at least, prevent some types of unwarranted conclusions about the complex reasoning capabilities of LLMs\.

### AI use statement

We used LLMs during the preparation of this paper for code assistance\. All experimental design, analysis, and interpretation of the results were done by the authors\. No text in the paper was generated by AI\.

### Ethics statement

This work uses two existing public benchmark datasets \(CLadderand CounterBench\) and a newly generated dataset produced with CLadder’s own procedural generator\. No data involves human subjects, personally identifiable information, or sensitive content\.

### Reproducibility statement

The codebase for this study includes intervention scripts, simple classifier code, and fine\-tuning configurations\. Generated parameters and fine\-tuning hyperparameters are documented in the appendix\.

## References

- Achiamet al\.\(2023\)J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p2.1)\.
- Carliniet al\.\(2022\)N\. Carlini, D\. Ippolito, M\. Jagielski, K\. Lee, F\. Tramer, and C\. ZhangQuantifying memorization across neural language models\.InThe Eleventh International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p1.1)\.
- Chang \(2026\)E\. Y\. ChangRight for the wrong reasons: epistemic regret minimization for causal rung collapse in llms\.arXiv preprint arXiv:2602\.11675\.Cited by:[§A\.1](https://arxiv.org/html/2610.38100#A1.SS1.p1.1),[§A\.2](https://arxiv.org/html/2610.38100#A1.SS2.p1.1)\.
- Chenet al\.\(2026a\)J\. Chen, S\. Chen, and C\. LuCan post\-training transform llms into causal reasoners?\.arXiv preprint arXiv:2602\.06337\.Cited by:[§A\.2](https://arxiv.org/html/2610.38100#A1.SS2.p1.1),[§2\.2](https://arxiv.org/html/2610.38100#S2.SS2.p2.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. D\. O\. Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman,et al\.Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p1.1)\.
- Chenet al\.\(2025\)S\. Chen, Y\. Chen, Z\. Li, Y\. Jiang, Z\. Wan, Y\. He, D\. Ran, T\. Gu, H\. Li, T\. Xie,et al\.Benchmarking large language models under data contamination: a survey from static to dynamic evaluation\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 10091–10109\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p3.1)\.
- Chenet al\.\(2026b\)Y\. Chen, V\. K\. Singh, J\. Ma, and R\. TangCounterBench: evaluating and improving counterfactual reasoning in large language models\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 30350–30358\.Cited by:[§3\.6](https://arxiv.org/html/2610.38100#S3.SS6.p2.1)\.
- Collinset al\.\(2022\)K\. M\. Collins, C\. Wong, J\. Feng, M\. Wei, and J\. TenenbaumStructured, flexible, and robust: benchmarking and improving large language models towards more human\-like behavior in out\-of\-distribution reasoning tasks\.InProceedings of the Annual Meeting of the Cognitive Science Society,Vol\.44\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p1.1)\.
- de Leeuwet al\.\(2025\)N\. de Leeuw, M\. Nahon, M\. Reymond, R\. Chatila, and M\. KhamassiSemantic deception: when reasoning models can’t compute an addition\.arXiv preprint arXiv:2512\.20812\.Cited by:[§A\.2](https://arxiv.org/html/2610.38100#A1.SS2.p1.1)\.
- Denget al\.\(2024\)C\. Deng, Y\. Zhao, X\. Tang, M\. Gerstein, and A\. CohanInvestigating data contamination in modern benchmarks for large language models\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 8706–8719\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p2.1)\.
- Dettmerset al\.\(2023\)T\. Dettmers, A\. Pagnoni, A\. Holtzman, and L\. ZettlemoyerQLoRA: efficient finetuning of quantized llms\.External Links:2305\.14314,[Link](https://arxiv.org/abs/2305.14314)Cited by:[§A\.5](https://arxiv.org/html/2610.38100#A1.SS5.p1.1)\.
- Devlinet al\.\(2019\)J\. Devlin, M\. Chang, K\. Lee, and K\. ToutanovaBert: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 \(long and short papers\),pp\. 4171–4186\.Cited by:[§3\.2](https://arxiv.org/html/2610.38100#S3.SS2.p1.1)\.
- Dorieet al\.\(2019\)V\. Dorie, J\. Hill, U\. Shalit, M\. Scott, and D\. CervoneAutomated versus do\-it\-yourself methods for causal inference: lessons learned from a data analysis competition\.Statistical Science34\(1\),pp\. 43–68\.Cited by:[§4](https://arxiv.org/html/2610.38100#S4.p3.1)\.
- Duet al\.\(2025\)J\. Du, L\. Chen, X\. Xian, A\. Luo, F\. Tian, G\. Wang, C\. Doss, X\. Shen, and J\. DingIce cream doesn’t cause drowning: benchmarking llms against statistical pitfalls in causal inference\.arXiv preprint arXiv:2505\.13770\.Cited by:[§A\.1](https://arxiv.org/html/2610.38100#A1.SS1.p1.1)\.
- Geirhoset al\.\(2020\)R\. Geirhos, J\. Jacobsen, C\. Michaelis, R\. Zemel, W\. Brendel, M\. Bethge, and F\. A\. WichmannShortcut learning in deep neural networks\.Nature Machine Intelligence2\(11\),pp\. 665–673\.External Links:ISSN 2522\-5839,[Link](http://dx.doi.org/10.1038/s42256-020-00257-z),[Document](https://dx.doi.org/10.1038/s42256-020-00257-z)Cited by:[§A\.2](https://arxiv.org/html/2610.38100#A1.SS2.p1.1)\.
- Genget al\.\(2026\)L\. Geng, A\. Ouyang, T\. Wu, D\. Barretto, M\. J\. Hayes, R\. Cooper, Y\. Zeng, S\. Vijay, G\. Ancone, A\. Rai,et al\.Causalt5k: diagnosing and informing refusal for trustworthy causal reasoning of skepticism, sycophancy, detection\-correction, and rung collapse\.arXiv preprint arXiv:2602\.08939\.Cited by:[§A\.1](https://arxiv.org/html/2610.38100#A1.SS1.p1.1)\.
- Gui and Ji \(2025\)S\. Gui and S\. JiMitigating spurious correlations in llms via causality\-aware post\-training\.External Links:2506\.09433,[Link](https://arxiv.org/abs/2506.09433)Cited by:[§2\.2](https://arxiv.org/html/2610.38100#S2.SS2.p2.1)\.
- Heet al\.\(2026\)P\. He, Y\. Huang, M\. Sachan, and Z\. JinUncovering hidden correctness in LLM causal reasoning via symbolic verification\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),V\. Demberg, K\. Inui, and L\. Marquez \(Eds\.\),Rabat, Morocco,pp\. 1231–1250\.External Links:[Link](https://aclanthology.org/2026.eacl-long.56/),[Document](https://dx.doi.org/10.18653/v1/2026.eacl-long.56),ISBN 979\-8\-89176\-380\-7Cited by:[§2\.2](https://arxiv.org/html/2610.38100#S2.SS2.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. SteinhardtMeasuring massive multitask language understanding\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p1.1),[§1](https://arxiv.org/html/2610.38100#S1.p2.1)\.
- Huet al\.\(2021\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.External Links:2106\.09685,[Link](https://arxiv.org/abs/2106.09685)Cited by:[§A\.5](https://arxiv.org/html/2610.38100#A1.SS5.p1.1)\.
- Jinet al\.\(2023\)Z\. Jin, Y\. Chen, F\. Leeb, L\. Gresele, O\. Kamal, Z\. Lyu, K\. Blin, F\. Gonzalez Adauto, M\. Kleiman\-Weiner, M\. Sachan,et al\.Cladder: assessing causal reasoning in language models\.Advances in Neural Information Processing Systems36,pp\. 31038–31065\.Cited by:[§A\.3](https://arxiv.org/html/2610.38100#A1.SS3.p1.1),[§1](https://arxiv.org/html/2610.38100#S1.p3.1),[§2\.1](https://arxiv.org/html/2610.38100#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2610.38100#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2610.38100#S2.SS2.p2.1),[§2\.2](https://arxiv.org/html/2610.38100#S2.SS2.p3.1),[§2](https://arxiv.org/html/2610.38100#S2.p3.1)\.
- Kicimanet al\.\(2024\)E\. Kiciman, R\. Ness, A\. Sharma, and C\. TanCausal reasoning and large language models: opening a new frontier for causality\.Transactions on Machine Learning Research\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p1.1)\.
- Li and Russo \(2026\)Z\. Li and F\. RussoLeveraging large language models for causal discovery: a constraint\-based, argumentation\-driven approach\.arXiv preprint arXiv:2602\.16481\.Cited by:[§A\.1](https://arxiv.org/html/2610.38100#A1.SS1.p2.1)\.
- Maaschet al\.\(2025a\)J\. Maasch, J\. Kalantari, and K\. KhezeliCausalARC: abstract reasoning with causal world models\.arXiv preprint arXiv:2509\.03636\.Cited by:[§A\.1](https://arxiv.org/html/2610.38100#A1.SS1.p1.1)\.
- Maaschet al\.\(2025b\)J\. R\. Maasch, A\. Hüyük, X\. Xu, A\. V\. Nori, and J\. GonzalezCompositional causal reasoning evaluation in language models\.arXiv preprint arXiv:2503\.04556\.Cited by:[§A\.1](https://arxiv.org/html/2610.38100#A1.SS1.p1.1)\.
- Mao and Venkat \(2026\)Z\. Mao and A\. VenkatRecurrent confidence chain: temporal\-aware uncertainty quantification in large language models\.InICASSP 2026 \- 2026 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),Vol\.,pp\. 18562–18566\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP55912.2026.11461479)Cited by:[§2\.2](https://arxiv.org/html/2610.38100#S2.SS2.p1.1)\.
- McCoyet al\.\(2019\)R\. T\. McCoy, E\. Pavlick, and T\. LinzenRight for the wrong reasons: diagnosing syntactic heuristics in natural language inference\.InProceedings of the 57th annual meeting of the association for computational linguistics,pp\. 3428–3448\.Cited by:[§A\.2](https://arxiv.org/html/2610.38100#A1.SS2.p1.1)\.
- Momentèet al\.\(2025\)F\. Momentè, A\. Suglia, M\. Giulianelli, A\. Ferrari, A\. Koller, O\. Lemon, D\. Schlangen, R\. Fernández, and R\. BernardiTriangulating LLM progress through benchmarks, games, and cognitive tests\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 20051–20072\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.1092/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.1092),ISBN 979\-8\-89176\-335\-7Cited by:[§2\.2](https://arxiv.org/html/2610.38100#S2.SS2.p1.1)\.
- Pacchiardiet al\.\(2024\)L\. Pacchiardi, M\. Tesic, L\. G\. Cheke, and J\. Hernández\-OralloLeaving the barn door open for clever hans: simple features predict llm benchmark answers\.arXiv preprint arXiv:2410\.11672\.Cited by:[§2\.2](https://arxiv.org/html/2610.38100#S2.SS2.p3.1)\.
- Pearl \(2009\)J\. PearlCausality\.Cambridge University Press,Cambridge\.Cited by:[§2](https://arxiv.org/html/2610.38100#S2.p3.1)\.
- Risi and Togelius \(2020\)S\. Risi and J\. TogeliusIncreasing generality in machine learning through procedural content generation\.Nature Machine Intelligence2\(8\),pp\. 428–436\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p3.1)\.
- Roemmeleet al\.\(2011\)M\. Roemmele, C\. A\. Bejan, and A\. S\. GordonChoice of plausible alternatives: an evaluation of commonsense causal reasoning\.\.InAAAI spring symposium: logical formalizations of commonsense reasoning,pp\. 90–95\.Cited by:[§2](https://arxiv.org/html/2610.38100#S2.p2.1)\.
- Sainzet al\.\(2023\)O\. Sainz, J\. Campos, I\. García\-Ferrero, J\. Etxaniz, O\. L\. de Lacalle, and E\. AgirreNLP evaluation in trouble: on the need to measure llm data contamination for each benchmark\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 10776–10787\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p2.1)\.
- Saxtonet al\.\(2019\)D\. Saxton, E\. Grefenstette, F\. Hill, and P\. KohliAnalysing mathematical reasoning abilities of neural models\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p3.1)\.
- Schaeffer \(2023\)R\. SchaefferPretraining on the test set is all you need\.arXiv preprint arXiv:2309\.08632\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p2.1)\.
- Schraderet al\.\(2024\)T\. P\. Schrader, L\. Lange, S\. Razniewski, and A\. FriedrichQUITE: quantifying uncertainty in natural language text in bayesian reasoning scenarios\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 2634–2652\.Cited by:[§2\.2](https://arxiv.org/html/2610.38100#S2.SS2.p1.1)\.
- Weiet al\.\(2025\)A\. Wei, Y\. Wu, Y\. Wan, T\. Suresh, H\. Tan, Z\. Zhou, S\. Koyejo, K\. Wang, and A\. AikenSATBench: benchmarking LLMs’ logical reasoning via automated puzzle generation from SAT formulas\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 33832–33849\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1716/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1716),ISBN 979\-8\-89176\-332\-6Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p3.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p1.1)\.
- Whiteet al\.\(2025\)C\. White, S\. Dooley, M\. Roberts, A\. Pal, B\. Feuer, S\. Jain, R\. Shwartz\-Ziv, N\. Jain, K\. Saifullah, S\. Dey,et al\.Livebench: a challenging, contamination\-limited llm benchmark\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 91595–91631\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p3.1)\.
- Wuet al\.\(2024\)A\. Wu, K\. Kuang, M\. Zhu, Y\. Wang, Y\. Zheng, K\. Han, B\. Li, G\. Chen, F\. Wu, and K\. ZhangCausality for large language models\.arXiv preprint arXiv:2410\.15319\.Cited by:[§2](https://arxiv.org/html/2610.38100#S2.p1.1),[§2](https://arxiv.org/html/2610.38100#S2.p2.1)\.
- Xuet al\.\(2024\)C\. Xu, S\. Guan, D\. Greene, M\. Kechadi,et al\.Benchmark data contamination of large language models: a survey\.arXiv preprint arXiv:2406\.04244\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p2.1)\.
- Yamaokaet al\.\(2026\)K\. Yamaoka, S\. Mukherjee, T\. Gärtner, D\. A\. Selby, S\. Konigorski, E\. Hüllermeier, V\. Bengs, and S\. J\. VollmerLinear\-llm\-scm: benchmarking llms for coefficient elicitation in linear\-gaussian causal models\.arXiv preprint arXiv:2602\.10282\.Cited by:[§A\.1](https://arxiv.org/html/2610.38100#A1.SS1.p2.1)\.
- Yuet al\.\(2025\)L\. Yu, D\. Chen, S\. Xiong, Q\. Wu, D\. Li, Z\. Chen, X\. Liu, and L\. PanCausaleval: towards better causal reasoning in language models\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 12512–12540\.Cited by:[§2\.2](https://arxiv.org/html/2610.38100#S2.SS2.p1.1)\.
- Zečevićet al\.\(2023\)M\. Zečević, M\. Willig, D\. S\. Dhami, and K\. KerstingCausal parrots: large language models may talk causality but are not causal\.arXiv preprint arXiv:2308\.13067\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p1.1)\.
- Zhanget al\.\(2023\)C\. Zhang, S\. Bauer, P\. Bennett, J\. Gao, W\. Gong, A\. Hilmkil, J\. Jennings, C\. Ma, T\. Minka, N\. Pawlowski,et al\.Understanding causality with large language models: feasibility and opportunities\.arXiv preprint arXiv:2304\.05524\.Cited by:[§2](https://arxiv.org/html/2610.38100#S2.p1.1)\.
- Zhanget al\.\(2024\)H\. Zhang, J\. Da, D\. Lee, V\. Robinson, C\. Wu, W\. Song, T\. Zhao, P\. Raja, C\. Zhuang, D\. Slack, Q\. Lyu, S\. Hendryx, R\. Kaplan, M\. Lunati, and S\. YueA careful examination of large language model performance on grade school arithmetic\.External Links:2405\.00332,[Link](https://arxiv.org/abs/2405.00332)Cited by:[§A\.2](https://arxiv.org/html/2610.38100#A1.SS2.p1.1),[§1](https://arxiv.org/html/2610.38100#S1.p2.1)\.
- Zhuet al\.\(2024\)K\. Zhu, J\. Chen, J\. Wang, N\. Gong, D\. Yang, and X\. XieDyval: dynamic evaluation of large language models for reasoning tasks\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 18091–18128\.Cited by:[§1](https://arxiv.org/html/2610.38100#S1.p3.1)\.

## Appendix AAppendix

### A\.1Critiques of CLadderand Synthetic Benchmarks

Several recent papers have identified limitations of CLadderfrom directions complementary to ours\.[Geng et al\. \[2026\]](https://arxiv.org/html/2610.38100#bib.bib23)and[Chang \[2026\]](https://arxiv.org/html/2610.38100#bib.bib17)argue that CLadder’s template\-heavy queries have low linguistic diversity and fail to capture failure modes that appear in naturalistic causal judgments, positioning their own benchmarks as testing more realistic reasoning scenarios\.[Maasch et al\. \[2025a\]](https://arxiv.org/html/2610.38100#bib.bib24)explicitly cite benchmark leakage as a motivation for designing CausalARC around randomly sampled tasks that cannot be memorized, noting that o4\-mini offered unprompted CLadder\-style outputs suggestive of training exposure\.[Maasch et al\. \[2025b\]](https://arxiv.org/html/2610.38100#bib.bib25)extend this concern with a compositional framework, finding that only o1 does not degrade with reasoning complexity, which is consistent with our findings that apparent causal competence collapses under targeted perturbations\.[Du et al\. \[2025\]](https://arxiv.org/html/2610.38100#bib.bib26)show that CLadder\-style benchmarks do not test statistical pitfalls such as survivorship bias and collider conditioning in realistic scenarios, and find that model conclusions shift based on semantic label choice\.

More recent work has examined specific subproblems:[Li and Russo \[2026\]](https://arxiv.org/html/2610.38100#bib.bib21)use LLMs for causal discovery via graph structure identification, while[Yamaoka et al\. \[2026\]](https://arxiv.org/html/2610.38100#bib.bib22)benchmark coefficient elicitation in linear Gaussian Structural Causal Models\. These complementary directions highlight that causal competence is multidimensional and that strong performance on one subproblem does not imply general causal reasoning ability\.

### A\.2Shortcut Learning and Benchmark Contamination

The phenomenon of models achieving high benchmark accuracy through spurious correlations rather than the intended reasoning process is well documented\[[Geirhos et al\., 2020](https://arxiv.org/html/2610.38100#bib.bib12),[McCoy et al\., 2019](https://arxiv.org/html/2610.38100#bib.bib14)\]\.[Chang \[2026\]](https://arxiv.org/html/2610.38100#bib.bib17)formalize this as Rung Collapse — the systematic substitution of associational reasoning for interventional queries\.[de Leeuw et al\. \[2025\]](https://arxiv.org/html/2610.38100#bib.bib15)demonstrate analogous surface exploitation in arithmetic, showing that LLMs default to learned semantic associations over abstract reasoning when tested on novel symbol systems\. On the contamination side,[Zhang et al\. \[2024\]](https://arxiv.org/html/2610.38100#bib.bib13)show that models exhibit inflated performance on publicly available math benchmarks relative to held\-out equivalents, with accuracy gaps correlating with the probability of generating training examples\.[Chen et al\. \[2026a\]](https://arxiv.org/html/2610.38100#bib.bib19)show that fine\-tuning on synthetic SCM data produces dramatic accuracy gains similar to our LoRA experiments, but do not investigate whether those gains reflect genuine reasoning or shortcut learning\.

### A\.3Generated dataset

We generated a CLadderdataset using the original codebase of[Jin et al\. \[2023\]](https://arxiv.org/html/2610.38100#bib.bib1), runningfig generateacross the configurations \(final/commonandfinal/det\-common\)\.

This produced 9,866 unique \(question, answer\) pairs, which we used for all of our experiments\. We use this generated split rather than the original v1 release for our evaluation as our test set for the fine\-tuning experiment\.

Table 1:Query\-type distribution in the generated dataset\.Query\-types vary in count because the underlying causal graph templates vary in richness and would generate duplicates\. The original CLadderreleases included surface\-level interventions as a robustness test\. We did not apply these during our dataset generation, as we found no significant difference in model performance between these dataset split\.

To confirm our generated data matches the difficulty of the original CLadderv1 release, we evaluate gpt\-5\.5 \(medium reasoning\) zero\-shot on our generated set and each of the four v1 splits\. Table[2](https://arxiv.org/html/2610.38100#A1.T2)shows nearly identical performance on all splits \(allp\>0\.05p\>0\.05, none significant after Bonferroni correction\)\.

Table 2:Zero\-shot GPT\-5\.5 \(medium reasoning\) accuracy on our generated dataset \(Generated\) and eachCLadderv1 split individually\. No split is significantly harder than the generated dataset \(pp\-values from two\-proportionzz\-tests, Bonferroni\-corrected thresholdp<0\.0125p<0\.0125, none significant\)\.
### A\.4Model Performance on Generated CLadderData

Table 3:Model accuracy on generatedCladderqueries \(%\\%,±\\pmWilson 95 % CI half\-width\)\. Bold = best per column\. † Back\-Adj hasn=74n\{=\}74; all other typesn≥432n\{\\geq\}432\.
### A\.5Fine\-tuning Details

We fine\-tune four open\-weight models \( OLMo\-3\-7B\-Instruct, OLMo\-3\.1\-32B\-Instruct, Llama\-3\.1\-8B\-Instruct, and Qwen2\.5\-3B\) on the public CLadderdataset, stratified by query type at 80/20 split for train and val\. We use the generated CLadderdataset as our test set\. We perform parameter\-efficient fine\-tuning via LoRA\[[Hu et al\., 2021](https://arxiv.org/html/2610.38100#bib.bib10)\]\. To study the effects of training data size, we vary the number of training examplesN∈50,200,500,1​k,2​k,4​k,8​k,16​kN\\in\{50,200,500,1\\text\{k\},2\\text\{k\},4\\text\{k\},8\\text\{k\},16\\text\{k\}\}and evaluate on the held\-out test set\. We use rankr=16r=16, scaling factorα=32\\alpha=32, dropout0\.050\.05, targeting both attention projection layers \(q\_proj,k\_proj,v\_proj,o\_proj\) and feed\-forward layers \(gate\_proj,up\_proj,down\_proj\), following[Dettmers et al\. \[2023\]](https://arxiv.org/html/2610.38100#bib.bib11)\. Models are trained for 5 epochs with AdamW, cosine learning rate2×10−42\\times 10^\{\-4\}, effective batch size1616, bf16 precision with gradient checkpointing, and prompt tokens masked to−100\-100so only the answer token contributes to training loss\.

Figure 8:Model performance as a function of the number of publicly\-available CLaddersamples the model is exposed to during fine\-tuning\.Figure 9:Model performance \(per query\-type\) as a function of the number of publicly\-available CLaddersamples the model is exposed to during fine\-tuning\.
### A\.6Query\-wise Accuracy After Performing Interventions

We abbreviate query types present in CLadderas follows: Marginal \(MAR\), Correlation \(COR\), Back\-door Adjustment \(BKA\), Average Treatment Effect \(ATE\), Effect of Treatment on the Treated \(ETT\), Natural Indirect Effect \(NIE\), Natural Direct Effect \(NDE\), Deterministic Counterfactual \(DCF\), Explaining Away \(EXP\), and Collider Bias \(COL\)\. Figures[10](https://arxiv.org/html/2610.38100#A1.F10)and[11](https://arxiv.org/html/2610.38100#A1.F11)show the per\-query\-type accuracy change for the six interventions that were discussed in Section[3\.4](https://arxiv.org/html/2610.38100#S3.SS4)\.

![Refer to caption](https://arxiv.org/html/2610.38100v1/intervention_figure_mockup-Fig4-Var-A.drawio.png)Figure 10:Per\-query\-type model accuracy under interventions that remove elements of the queries that are critical for formal causal reasoning\. We show 95% Wilson CI\.![Refer to caption](https://arxiv.org/html/2610.38100v1/intervention_figure_mockup-Fig6-Var-A.drawio.png)Figure 11:Per\-query\-type model accuracy under interventions that remove elements of the queries that make it impossible to calculate the answer\. We show 95% Wilson CI\.
### A\.7Sources of Determinism in CLadderData

To understand the mechanisms underlying model performance on CLadder, we conducted a systematic analysis of the dataset’s generation codebase\. This analysis reveals three distinct levels at which answer determinism is introduced into the benchmark, each of which provides a potential shortcut a model could exploit without performing genuine causal reasoning\.

#### A\.7\.1Parametric determinism

CLadder’sDifficultyBuildergenerates questions by sampling Structural Causal Model parameters from predefined specification lists stored in per\-story configuration files\. In the easy split, these specifications assign consistent sign directions to causal mechanisms across all 37 non\-nonsense stories, implying the direction of the causal effect can be typically determined by reading a single mechanism edge in the prompt, without computing anything\. The hard split systematically reverses these signs\. In both cases, knowing which story a query comes from is often sufficient to predict the correct answer regardless of the specific probability values provided\.

#### A\.7\.2Structural determinism

The collision graph topology \(X→V←YX\\rightarrow V\\leftarrow Y\), Y has no parents and is therefore statistically independent of X by construction\. Any intervention on X cannot affect Y, so the average treatment effectE⁡\[Y\|d​o​\(X=1\)\]−E⁡\[Y\|d​o​\(X=0\)\]=0E\[Y\|do\(X=1\)\]\-E\[Y\|do\(X=0\)\]=0for every parameter setting\. All queries pairing a collision graph with an ATE, ETT or COR query type therefore have a predetermined answer that can be read from the graph structure\.

#### A\.7\.3Formula\-level determinism

The COL query type computes the correct answer as\(−1\)p​o​l​a​r​i​t​y\(\-1\)^\{polarity\}, where polarity is a flag that controls whether the question is phrased as ”does X affect Y” \(answer is alwaysno\) or ”does X not affect Y” \(answer is alwaysyes\)\. This is independent of the causal graph, the Structural Causal Model parameters, and the numerical correlation value shown in the prompt\. The correlation value is computed and displayed to the model, but is never used to determine the ground truth answer\.

### A\.8Random Forest Classifier Features

When training the random forest classifier we use the following features:

- •Boolean variable indicating whether the query asks if one quantity is ”greater than” or ”less than” another quantity\.
- •The number of characters in the query\.
- •The number of probability values that appear in the query\.
- •The first probability that appears in the query\. This feature is set to 0 for queries with no probability values\.
- •The second probability that appears in the query\. This feature is set to 0 for queries with less than two probability values\.
- •The last probability that appears in the query\. This feature is set to 0 for queries with no probability values\. Some query types contain 2 or less probability values, so a probability value that appears first or second in the query could also be the last\.
- •Boolean variable indicating whether the first probability value in the query is larger than the second\.
- •Boolean variable indicating whether the first probability value in the query is larger than the last\.
- •The product of the first and second probabilities that appear in the query\.
- •The product of the first and last probabilities that appear in the query\.
- •The product of the second and last probabilities that appear in the query\.
- •Boolean variables indicating the presence or absence of the following strings: ”does it mean,” ”does not effect,” ”method,”, ”case by case,” ”chance of,” ”increase,” ”decrease,” ”overall,” ”more likely,” ”less likely,” ”when observing,” ”larger,” ”smaller,” ”does the chance of,” ”does the probability of,” ”does the likelihood of,” ”disregard the mediation effect,” ”positively affect,” ”negatively affect,” ”through,” ”would it be,” ”instead of,” ”not,” ”and,” ”or,” ”doesn’t,” and ”correlation\.”
- •Boolean variables indicating if the query starts with the words ”for” or ”would\.”

### A\.9Model performance on CounterBench

ModelTypeAcc\. \(%\)Simple models \(n=1200n=1200\)Random ForestClassifier79\.1±2\.379\.1\\pm 2\.3Commercial models \(one\-word prompt,n=1200n=1200\)GPT\-3\.5\-turboBase48\.6±2\.848\.6\\pm 2\.8GPT\-4\.1Base48\.8±2\.848\.8\\pm 2\.8GPT\-4oBase52\.2±2\.852\.2\\pm 2\.8GPT\-5\-nanoBase51\.2±2\.851\.2\\pm 2\.8GPT\-5\.2Base50\.8±2\.850\.8\\pm 2\.8GPT\-5\.5Base51\.9±2\.851\.9\\pm 2\.8GPT\-5\.5Med\. reason50\.3±2\.850\.3\\pm 2\.8GPT\-5\.6 SolBase49\.3±2\.849\.3\\pm 2\.8GPT\-5\.6 SolMed\. reason52\.1±2\.852\.1\\pm 2\.8Open\-weight base models \(n=1200n=1200\)OLMo\-3\-7B\-InstructBase50\.4±2\.850\.4\\pm 2\.8OLMo\-3\.1\-32B\-InstructBase50\.4±2\.850\.4\\pm 2\.8LLama\-3\.1\-8B\-InstructBase50\.8±2\.850\.8\\pm 2\.8Qwen2\.5\-3B\-InstructBase50\.5±2\.850\.5\\pm 2\.8Fine\-tuned models,N=4000N=4000\(n=1200n=1200\)OLMo\-3\-7B\-InstructFine\-tuned63\.0±2\.763\.0\\pm 2\.7OLMo\-3\.1\-32B\-InstructFine\-tuned71\.5±2\.571\.5\\pm 2\.5LLama\-3\.1\-8B\-InstructFine\-tuned68\.9±2\.668\.9\\pm 2\.6Qwen2\.5\-3B\-InstructFine\-tuned59\.2±2\.859\.2\\pm 2\.8Table 4:CounterBench Alpha accuracy±\\pmhalf\-width of Wilson 95% CI \(chance = 50%\)\. All commercial GPT models perform at or near chance regardless of generation\. Fine\-tuned open\-weight models and the random forest outperform all commercial models\.
### A\.10Expanded Token Ablation Experiment

![Refer to caption](https://arxiv.org/html/2610.38100v1/appendix_token_occ.png)Figure 12:Token ablation analysis on gpt\-5\.2 for a random sample of 25 evaluated marginal queries\. Same visualization as Figure[7](https://arxiv.org/html/2610.38100#S3.F7)\. Color encoding, label conventions, and salience threshold are identical to the main figure\.