对智能体因果发现进行基准测试与增强

# Benchmarking and Enhancing Agentic Causal Discovery
Source: [https://arxiv.org/html/2610.22846](https://arxiv.org/html/2610.22846)
Rui DingAffiliation:MicrosoftAffiliation:Email:[juding@microsoft\.com](mailto:)Shuai WangAffiliation:The Hong Kong UniversityAffiliation:of Science and TechnologyEmail:[shuaiw@cse\.ust\.hk](mailto:)

###### Abstract

LLM agents can now run causal discovery algorithms on a dataset and report what comes out\. Whether they do more than that, such as noticing where an algorithm went wrong and fixing it with further tests, is hard to tell from current benchmarks\. Most of them give the variables meaningful names, and an LLM shown only the names of the Tübingen cause\-effect pairs already gets 99% of the directions right\. With the names removed, the data are usually clean enough that picking a better algorithm than PC closes most of the gap\. We introduceHeuresis, a benchmark of anonymous simulated datasets whose mechanisms range from linear Gaussian to heavy\-tailed\. For each family of datasets it separates what a better existing algorithm can gain from what only a closer look at the data can gain, and it traces every edge an agent changes\. Every family leaves a gap that none of the algorithms, the router and the two causal\-discovery foundation models we ran can close, so a score above the best chosen algorithm has to come from using the data\. We instantiate the benchmark with an agent harness and three LLMs, GPT\-6\.1 Sol, Claude Opus 5\.5 and Qwen 3\.8 Flash\. GPT\-6\.1 Sol surpasses the best chosen algorithm on all six families, by 14% adjacency F1 on the most difficult one, and Claude Opus 5\.5 does so by 7\.4% there\. A third of the true edges the two agents restore on the two hardest families appear in no algorithm’s output\. The code and data will be publicly released to facilitate future work\.

Figure 1:Overview ofHeuresis\.## 1Introduction

Early work asked LLMs for causal relations from the names of the variables\([Kıcıman et al\., 2024](https://arxiv.org/html/2610.22846#bib.bib19)\), which tests what a model has read rather than whether it can find a structure nobody knows yet\. The setting that matters in practice is an LLM agent that receives a dataset and carries out causal discovery itself, by running statistical tests, discovery algorithms and code on the data\. Causal\-Copilot\([Wang et al\., 2025](https://arxiv.org/html/2610.22846#bib.bib38)\)picks, tunes and interprets an algorithm;[Zheng et al\. \(2026\)](https://arxiv.org/html/2610.22846#bib.bib44)argue that agents should coordinate the tools while the causal claims stay grounded in data, and[Wu et al\. \(2025\)](https://arxiv.org/html/2610.22846#bib.bib39)that LLMs should stay out of the decisions altogether\.

In this setting the agent decides at several granularities: which*algorithm*to run, such as PC, GES or LiNGAM; which*component*to use inside it, such as the conditional\-independence \(CI\) test of PC; which*local*steps of a run to revisit, such as a test, a conditioning set or a single deletion; and how to revise the*graph*with further evidence\. At the finest grain the LLM may even judge a dependence itself, in place of a small tool\. The levels form no strict hierarchy, and each can be driven by the data: an agent that switches to a kernel test because the relations look nonlinear uses evidence as much as one that repairs an edge\. A conventional pipeline fixes these decisions before it sees the data, whereas an agent can revisit any of them\. We call this capability*fine\-grained, evidence\-driven adaptation*\. Take PC\([Spirtes et al\., 2000](https://arxiv.org/html/2610.22846#bib.bib35)\), which deletes an edge as soon as one CI test misses a dependence and never reconsiders the deletion\. When a true edge is lost this way, can the agent find the test, re\-test the pair and restore the edge?

Current benchmarks cannot tell, for three reasons\. ➀*Leakage*\. Most LLM evaluations of causal discovery name the variables\([Kıcıman et al\., 2024](https://arxiv.org/html/2610.22846#bib.bib19);[Zhou et al\., 2024](https://arxiv.org/html/2610.22846#bib.bib45);[Babakov et al\., 2025](https://arxiv.org/html/2610.22846#bib.bib2)\): LLMs reproduce graphs such as Asia and Cancer of the BNLearn benchmarks from the names alone\([Srivastava et al\., 2025](https://arxiv.org/html/2610.22846#bib.bib36)\), and given no data at all an LLM gets 99% of the Tübingen directions right \(Figure[1](https://arxiv.org/html/2610.22846#S0.F1)a\)\. ➁*Saturation*\. With the names removed, the data are usually clean enough that choosing the algorithm settles almost everything: on linear\-Gaussian data the best of PC and GES is only∼5%\\sim 5\\%below an ideal oracle that restores every true edge PC wrongly deleted \(Figure[1](https://arxiv.org/html/2610.22846#S0.F1)b\), which leaves the finer decisions no room to show\. ➂*Opacity*\. Existing benchmarks score only the final graph, so they cannot tell whether a gain came from a better existing algorithm or from analysis inside the run\. Agents may well make fine\-grained decisions already; what is missing is a measurement that credits a gain to them\.

Heuresismeasures this adaptation without leakage, saturation or opacity\. Its datasets are anonymous and drawn from six families of mechanisms, from linear Gaussian to heavy\-tailed, so that substantial headroom remains after the algorithm is chosen\. For every family it fixes reference points before any agent is scored: the best algorithm chosen globally and per family, and an ideal oracle that repairs every wrong deletion of PC \(Figure[1](https://arxiv.org/html/2610.22846#S0.F1)c\)\. Choosing per family gains nothing on five of the six families, where PC with a kernel CI test \(KCI\) is already best, while the gap to the oracle stays substantial on every family and is largest where every algorithm scores lowest\. We probe the finer levels through the one decision type whose correctness the ground truth fixes exactly, PC’s edge deletions: each is logged with its test and conditioning set, the ground truth says which were wrong, and every edge an agent restores can be traced to the calls before it\. The benchmark therefore reports the gain over existing algorithms and selection strategies separately from the behavior in the trajectory that produced it\.

We run a tool\-enabled agent harness with GPT\-6\.1 Sol, Claude Opus 5\.5 and Qwen 3\.8 Flash, under one toolbox and budget and without telling them the family\. GPT\-6\.1 Sol improves the best chosen algorithm on every family, by 14% onwild, the most difficult family, and Claude Opus 5\.5 by 7\.4% there; Qwen 3\.8 Flash reaches the best chosen algorithm on linear\-Gaussian data\. Their traces show that nokk\-vote merge of the algorithms’ outputs, withkkchosen per family in hindsight, reproduces these gains, and that a third of the true edges they restore on the two hardest families appear in no algorithm’s output at all\. Our contributions are:

- •We frame agentic causal discovery as evidence\-driven decisions at several granularities and show that leakage, saturation and opacity keep existing benchmarks from crediting a gain to the decisions behind it\.
- •We proposeHeuresis, anonymous simulated data with headroom beyond algorithm choice, reference points for choosing and for repairing, a traced probe on PC’s deletions, and an agent harness that records every call\.
- •We find that two of the three LLMs surpass the best chosen algorithm by re\-testing what PC deleted, and trace the edges they restore to the calls that produced them\.

## 2Related Work

#### LLMs and causal discovery\.

[Kıcıman et al\. \(2024\)](https://arxiv.org/html/2610.22846#bib.bib19)show that LLMs recover causal directions from variable names alone\. Corr2Cause and CLadder remove the names but pose closed\-form questions with no data\([Jin et al\., 2024](https://arxiv.org/html/2610.22846#bib.bib14);[Jin et al\., 2023](https://arxiv.org/html/2610.22846#bib.bib13)\), while CausalBN\-Bench and CausalGraphBench compare LLMs with classical algorithms on named BNLearn graphs\([Zhou et al\., 2024](https://arxiv.org/html/2610.22846#bib.bib45);[Babakov et al\., 2025](https://arxiv.org/html/2610.22846#bib.bib2)\), which LLMs have largely memorized\([Srivastava et al\., 2025](https://arxiv.org/html/2610.22846#bib.bib36);[Yu and Zhou, 2026](https://arxiv.org/html/2610.22846#bib.bib41)\)\. A second line uses the LLM as a component of a classical algorithm, for instance to answer or prioritize its independence tests from the variable names\([Cohrs et al\., 2024](https://arxiv.org/html/2610.22846#bib.bib6);[Hiremath et al\., 2025](https://arxiv.org/html/2610.22846#bib.bib11)\), or to check and orient the edges it learned\([Ban et al\., 2023](https://arxiv.org/html/2610.22846#bib.bib3);[Khatibi et al\., 2024](https://arxiv.org/html/2610.22846#bib.bib18);[Lyu et al\., 2025](https://arxiv.org/html/2610.22846#bib.bib25)\)\. These works primarily evaluate the causal knowledge of LLMs w\.r\.t\. the variable names instead of purely statistics\-driven causal discovery\.

#### Agents for causal and statistical analysis\.

Causal\-Copilot and CausalAgent let an LLM run a whole analysis\([Wang et al\., 2025](https://arxiv.org/html/2610.22846#bib.bib38);[Han et al\., 2024](https://arxiv.org/html/2610.22846#bib.bib10)\), and QRData, DSBench, CauSciBench and P\-Bench score the final answer on real data\([Liu et al\., 2024](https://arxiv.org/html/2610.22846#bib.bib23);[Jing et al\., 2025](https://arxiv.org/html/2610.22846#bib.bib15);[Acharya et al\., 2026](https://arxiv.org/html/2610.22846#bib.bib1);[Miao et al\., 2026](https://arxiv.org/html/2610.22846#bib.bib27)\)\. A final answer cannot show whether the agent chose a better method or used the evidence, and agent scores move with the budget and the harness\([Kapoor et al\., 2024](https://arxiv.org/html/2610.22846#bib.bib17);[Kapoor et al\., 2026](https://arxiv.org/html/2610.22846#bib.bib16)\)\.

#### Simulated discovery benchmarks for agents\.

CausalDS, CausaLab and CausalArena generate data from hidden causal graphs and score the graph an agent recovers\([Leban and Sun, 2026](https://arxiv.org/html/2610.22846#bib.bib20);[Yang et al\., 2026](https://arxiv.org/html/2610.22846#bib.bib40);[Li et al\., 2026a](https://arxiv.org/html/2610.22846#bib.bib21)\); CausalArena also finds that which algorithm works best depends on the kind of data\. The closest benchmark is ACDB\([Deb et al\., 2026](https://arxiv.org/html/2610.22846#bib.bib7)\), which gives LLM agents anonymized simulated data and compares them with PC\. Its data are simply linear Gaussian with few variables and it scores only the final graph\.

## 3Background

#### Causal discovery from data\.

A causal discovery algorithm takes a table of observations overddvariables and returns a graph whose edges mean direct influence; the true graph is a directed acyclic graph \(DAG\)\. We score an output graph mainly on its*skeleton*, the edges without directions, by*adjacency F1*, the F1 score of the output edges against the true edges\. Directions are scored by the*structural Hamming distance*\(SHD\), the number of edge edits to the true graph\.

#### PC and wrong deletions\.

PC\([Spirtes et al\., 2000](https://arxiv.org/html/2610.22846#bib.bib35)\)is the standard algorithm in this paper \(Figure[2](https://arxiv.org/html/2610.22846#S3.F2)\)\. It starts from a fully connected graph and removes the edge between two variables when a statistical test, a Fisher\-zztest or a kernel\-based conditional independence \(KCI\) test\([Zhang et al\., 2011](https://arxiv.org/html/2610.22846#bib.bib42)\), declares them independent given some other variables, the*conditioning set*\. PC reaches the data only through this test, and a deletion is final: when the test misses a real dependence, a true edge is lost\. These*wrong deletions*are the primary source of PC’s errors\. PC records every deletion with the test and the conditioning set behind it, the*deletion log*\. We focus on whether agents can repair wrong deletions, the most direct way to use the evidence to improve the output\.

Figure 2:Running example of PC\.
#### Example\.

Figure[2](https://arxiv.org/html/2610.22846#S3.F2)gives a running example\. PC starts from the complete graph and deletes an edge \(dashed\) when a test finds its two variables independent given a setSSof others\. In steps 1 and 2 the hidden graph agrees \(✓\)\. In step 3 it does not:X5X\_\{5\}depends onX2X\_\{2\}through a V\-shaped map, which a Fisher\-zztest, blind to non\-linear dependence, reads as independence\. The true edge is lost and never revisited\.

#### Other algorithms and the ideal oracle\.

GES\([Chickering, 2002](https://arxiv.org/html/2610.22846#bib.bib5)\)searches over graphs with a score instead of tests and suits linear\-Gaussian data; FCI, LiNGAM\([Shimizu et al\., 2006](https://arxiv.org/html/2610.22846#bib.bib34)\), additive\-noise methods\([Hoyer et al\., 2008](https://arxiv.org/html/2610.22846#bib.bib12)\)and NOTEARS rest on other assumptions\. Real data violate many of these assumptions, so no single algorithm is best\. The*ideal oracle*, which restores every wrongly deleted edge of PC, is the ceiling for repairing deletions, not for the task: an agent can also gain outside PC’s deletions \(§[5\.2](https://arxiv.org/html/2610.22846#S5.SS2)\)\.

Figure 3:AHeuresisepisode and its evaluation\.Table 1:Existing benchmarks against the design requirements D1–D4, as described in their papers\. ✓ yes,∼\\simpartly,×\\timesno, – not reported\.*nonlinear**input\-dep\. noise**non\-monotone**t3t\_\{3\}noise*difficultyFamilylinearMLPpiecewisesigmoidMLPsigmoidquadratic\|⋅\|\|\\cdot\|FourierlinearWrongZeroHeadroomlg⚫·········2\.423%\.121anm·⚫········1\.426%\.078hetero····⚫·····1\.039%\.049wild⚫⚫··⚫·⚫⚫⚫⚫3\.05%\.169mixed⚫⚫··⚫·····1\.236%\.059gauntlet··⚫⚫·⚫⚫⚫⚫·2\.49%\.134Table 2:Mechanism families ofHeuresis

## 4Heuresis

The three failures of §[1](https://arxiv.org/html/2610.22846#S1)lead to four design requirements ofHeuresis\.*D1:anonymous variables\.*Nothing can be recalled from a name\.*D2:both kinds of headroom measured per family\.*For every family, the benchmark measures how much of the gap above the global best chosen algorithm a better existing algorithm closes and how much no existing algorithm or the router recovers, before any agent is scored\.*D3:one gain per reference\.*Gains over the global best chosen algorithm and over the best chosen algorithm of the family must separate an agent that only calls the right algorithm, one that edits the graph blindly and one that improves on the right algorithm\.*D4:every graph decomposed and every call recorded*, so that a score can be explained without further experiments\. Table[1](https://arxiv.org/html/2610.22846#S3.T1)sets existing benchmarks against these four requirements\.

### 4\.1Task and mechanism families

Figure 4:Running example ofHeuresisinstance\.We separate three levels: a mechanism familyϕ\\phi, the SCMsSSdrawn from it, and the dataXXsampled from one SCM\.

###### Definition 1\(SCM\)\.

A*mechanism*m=\(f,ν\)m=\(f,\\nu\)pairs a functionffwith a noise distributionν\\nu\. A*structural causal model*\(SCM\)S=\(G,\(mj\)j=0d−1\)S=\\bigl\(G,\(m\_\{j\}\)\_\{j=0\}^\{d\-1\}\\bigr\)consists of a DAGGGoverddvariables and a mechanismmj=\(fj,νj\)m\_\{j\}=\(f\_\{j\},\\nu\_\{j\}\)for each variable, which generates the variables as

Xj=fj​\(XpaG​\(j\),εj\),εj∼νj,X\_\{j\}=f\_\{j\}\\bigl\(X\_\{\\mathrm\{pa\}\_\{G\}\(j\)\},\\varepsilon\_\{j\}\\bigr\),\\quad\\varepsilon\_\{j\}\\sim\\nu\_\{j\},\(1\)forj=0,…,d−1j=0,\\ldots,d\-1, where the noise termsεj\\varepsilon\_\{j\}are independent andpaG​\(j\)\\mathrm\{pa\}\_\{G\}\(j\)denotes the parents ofjjinGG\. Drawingnnindependent samples of\(X0,…,Xd−1\)\(X\_\{0\},\\ldots,X\_\{d\-1\}\)fromSSgives a data matrixX∈ℝn×dX\\in\\mathbb\{R\}^\{n\\times d\}\.

An*instance*ofHeuresisis the data matrixXXof a hidden SCMS∼𝒮ϕS\\sim\\mathcal\{S\}\_\{\\phi\}, withSSand its DAGGGkept as the ground truth \(Figure[4](https://arxiv.org/html/2610.22846#S4.F4)\)\. The agent sees onlyXX, whose columns are anonymous and in random order\. In an episode, the agent is asked to recoverGGfromXX\. As one can expect, the difficulty of the instance largely depends on the familyϕ\\phifrom whichSSis drawn, and we define it as follows\.

###### Definition 2\(Mechanism family\)\.

A*mechanism family*ϕ\\phiis a distribution over mechanisms\. An SCM of familyϕ\\phi, writtenS∼𝒮ϕS\\sim\\mathcal\{S\}\_\{\\phi\}, is drawn in two steps: first, the DAGGGis drawn from a random\-graph prior \(Appendix[C](https://arxiv.org/html/2610.22846#A3)\), and then each variable draws its own mechanismmj∼ϕm\_\{j\}\\sim\\phiindependently of the others, withfjf\_\{j\}taking as input the\|paG​\(j\)\|\|\\mathrm\{pa\}\_\{G\}\(j\)\|parents ofjj\.

InHeuresis, we introduce six families of mechanisms, from linear Gaussian to heavy\-tailed, as summarized in Table[2](https://arxiv.org/html/2610.22846#S3.T2)\. The families are designed to stress the assumptions of the standard algorithms, and they are ordered by which assumption each family removes, not by difficulty\. The first three families \(lg,anmandhetero\) are single\-mechanism families, where every variable draws its mechanism from the same distribution \(i\.e\., linear Gaussian, MLP mean, or MLP mean and noise scale, respectively\) \. The last three families \(wild,mixedandgauntlet\) are mixtures, where every variable draws its mechanism from a different distribution\. In particular, we deemwildthe most difficult family, because it involves highly sophisticated mechanisms that are nonlinear, non\-monotone and heavy\-tailed\. Inwild, every variable can have a different mechanism and the assumptions of the standard algorithms fail in multiple ways\.gauntletdraws each variable from the widest set of nonlinear, non\-monotone and input\-dependent\-noise mechanisms \(Table[2](https://arxiv.org/html/2610.22846#S3.T2)\), so that a single instance breaks several assumptions at once\. The benchmark is designed to be extensible, so that new families can be added in the future\. We report the full details of these mechanism families in Appendix[C](https://arxiv.org/html/2610.22846#A3)\.

### 4\.2Agent harness

Figure 5:The working graph over an episode\.An episode is an iterative edit of a*working graph*\(Figure[5](https://arxiv.org/html/2610.22846#S4.F5)\)\. The agent starts from an initial graph and, in each turn, reads evidence from the data and writes changes to the graph, until it submits\. The harness is built around two requirements: evidence is available on demand, and each decision can be traced back to the evidence behind it\.

#### Interaction\.

The state of an episode is the working graph, which assigns each pair of variables one relation: no edge, an undirected edge, or an edge in either direction\. In each turn the agent takes one of four kinds of actions: it runs a structure\-learning algorithm, queries evidence, analyzes the data, or edits and submits the graph\. The observations are the structured results of these tools, and the submitted graph is completed and scored as in §[4\.3](https://arxiv.org/html/2610.22846#S4.SS3)\. The unit of decision is a single variable pair rather than the whole graph, which is what makes the repair headroomHϕrH^\{\\mathrm\{r\}\}\_\{\\phi\}reachable by an agent at all\.

#### Evidence at three granularities\.

The toolbox offers evidence at three levels\. ➀ At the*graph level*, a menu of structure\-learning algorithms returns a complete graph per call: PC with a Fisher\-zztest and with KCI, FCI, GES, DirectLiNGAM, an additive\-noise method and linear NOTEARS\([Zheng et al\., 2024](https://arxiv.org/html/2610.22846#bib.bib43)\)\. Choosing among them is the selection headroomHϕsH^\{\\mathrm\{s\}\}\_\{\\phi\}of §[4\.3](https://arxiv.org/html/2610.22846#S4.SS3), which most existing causal discovery agents target\. ➁ At the*edge level*, the agent can run a conditional\-independence test, Fisher\-zzor KCI\([Zhang et al\., 2011](https://arxiv.org/html/2610.22846#bib.bib42);[Strobl et al\., 2019](https://arxiv.org/html/2610.22846#bib.bib37)\), with a conditioning set of its own choice, and check how stable a verdict is under resampling of the data\. This is the kind of evidence PC consumes when it deletes an edge, so the agent can repeat, replace or overturn a deletion on the same footing as the algorithm\. ➂ At the*mechanism level*, the agent can summarize the data, inspect how well a variable is explained by a candidate set of parents, probe the direction of a pair, and write short numerical programs \(numpyandscipy\) that run in an isolated process\. These tools let it test whether the assumptions of an algorithm hold on this instance, for instance linearity, a constant noise scale or light tails, and decide whose output to trust\. The menu is thus the coarse end of the toolbox, and the two finer levels are where the decisions inside a run are made\.

#### Resources and termination\.

Evidence is charged by what it costs to produce\. Calls that re\-run a statistical test draw on a test budget, counted in the same unit as the cost of PC itself, while diagnostics and analysis are bounded only by computation\. The number of turns, the output per turn, the number of algorithm calls and the total running time are also limited, and all limits are visible to the agent \(values in Appendix[H](https://arxiv.org/html/2610.22846#A8)\)\. When any of them is exhausted, the agent gets one last turn in which it can only submit\. Every episode therefore returns a graph, which is the precondition for reporting every cell over all episodes \(§[4\.3](https://arxiv.org/html/2610.22846#S4.SS3), Appendix[H](https://arxiv.org/html/2610.22846#A8.SS0.SSS0.Px1)\)\.

#### What the agent sees\.

A test returns a verdict with a coarse strength grade, not a rawpp\-value, so that decisions rest on interpreting the evidence instead of tuning against a threshold\. The identity of each algorithm is masked within an episode, so that the agent chooses a tool for what the data suggest and not for its name\. The default prompt is neutral: it states neither the metric nor the conditions under which an algorithm applies \(variants in §[5\.4](https://arxiv.org/html/2610.22846#S5.SS4)\)\. Analysis code has access to the data only, not to the ground truth or the state of the harness\.

#### Record\.

The harness records every call with its type, arguments, result and cost\. Each edge of a submitted graph can thus be traced to the edits and the evidence behind it, and the records are the source of the behavioral statistics in §[4\.3](https://arxiv.org/html/2610.22846#S4.SS3)and the trace\-level attribution in §[5\.3](https://arxiv.org/html/2610.22846#S5.SS3)\.

#### Guided vs\. unguided\.

The two conditions differ only in the starting point of the working graph and in what the agent is shown\. In the*guided*condition, the working graph starts from the output graph of PC\-KCI and the agent receives the deletion log of PC\-KCI: for each deleted pair, the conditioning set, the test and the strength of the evidence that removed it, but not the truth label\. In the*unguided*condition, the agent starts from the complete undirected graph and edits it based on its own decisions\. The difference between the two is the joint effect of a good starting point and of knowing how PC decided, so an agent whose guided and unguided scores coincide gains from neither\. Guided episodes also give the precision and recall of the agent’s repairs per deletion \(§[5\.3](https://arxiv.org/html/2610.22846#S5.SS3)\)\.

### 4\.3Evaluation protocol

#### Scores\.

Every submitted graph is first completed with the standard orientation rules\([Meek, 1995](https://arxiv.org/html/2610.22846#bib.bib26)\)and then scored by adjacency F1, the primary score, and by normalized SHD \(nSHD\) against the true graph \(§[3](https://arxiv.org/html/2610.22846#S3)\) as a secondary metric\. Adjacency F1 is the primary score because the skeleton is what observational data identify under faithfulness\. Every episode must submit a graph\. Scoring only the episodes that finish would keep the easy instances and could mimic a gain beyond algorithm choice \(Appendix[H](https://arxiv.org/html/2610.22846#A8.SS0.SSS0.Px1)\), so we report every cell over all episodes and claim a capability only when finished episodes agree\.

#### Headroom Estimation\.

Before any agent is scored, we estimate for every familyϕ\\phihow much a better algorithm and how much further evidence could add\. To this end, we run a collection of existing algorithms on every instance and select, on development instances, the global best chosen algorithm \(BCAg\), the one with the best mean adjacency F1 across all families, and the best chosen algorithm for each family \(BCAϕ\), the one with the best mean on that family\. As introduced in §[3](https://arxiv.org/html/2610.22846#S3), PC deletes edges sequentially and a deletion is final, so we further define an*ideal oracle*that restores every wrongly deleted edge of PC and leaves the rest of the output unchanged\. With these references, we estimate two kinds of headroom\. The*selection headroom*

Hϕs=F1⁡\(BCAϕ\)−F1⁡\(BCAg\),H^\{\\mathrm\{s\}\}\_\{\\phi\}=\\mathrm\{F1\}\(\\mathrm\{BCA\}\_\{\\phi\}\)\-\\mathrm\{F1\}\(\\mathrm\{BCA\}\_\{\\mathrm\{g\}\}\),\(2\)is what choosing the right algorithm for the family gains and is the focus of many existing causal discovery agents, and the*repair headroom*

Hϕr=F1⁡\(oracle\)−F1⁡\(BCAϕ\),H^\{\\mathrm\{r\}\}\_\{\\phi\}=\\mathrm\{F1\}\(\\text\{oracle\}\)\-\\mathrm\{F1\}\(\\mathrm\{BCA\}\_\{\\phi\}\),\(3\)is what remains for decisions inside a run, measured on PC’s deletions, and is of interest in this paper\. We report both headrooms in our experiments\.

#### Main Metrics\.

With the two headrooms defined, we can now define the metrics that place an agent against these references\. LetF1⁡\(A\)\\mathrm\{F1\}\(A\)be the adjacency F1 of an agentAAon a familyϕ\\phi\. We define*gain*asΔϕs​\(A\)=F1⁡\(A\)−F1⁡\(BCAg\)\\Delta^\{\\mathrm\{s\}\}\_\{\\phi\}\(A\)=\\mathrm\{F1\}\(A\)\-\\mathrm\{F1\}\(\\mathrm\{BCA\}\_\{\\mathrm\{g\}\}\), which is the improvement of the agent over the global best chosen algorithm, and*residual gain*asΔϕr​\(A\)=F1⁡\(A\)−F1⁡\(BCAϕ\)\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}\(A\)=\\mathrm\{F1\}\(A\)\-\\mathrm\{F1\}\(\\mathrm\{BCA\}\_\{\\phi\}\), which is the improvement of the agent over the best chosen algorithm for that family\.

#### Drill\-down Analysis\.

SinceHeuresisholds the ground truth, it can decompose every submitted graph into the edges recovered and harmed in specific episodes\. A submitted graph is then compared with the deletion log of PC: the wrongly deleted pairs it contains as edges \(*recovered*\), the rightly deleted pairs it contains as edges \(*harmful*\), the edges of PC it dropped, and the pairs it re\-oriented\. Three behavioral statistics come from the tool\-call record: the*shortcut rate*, the share of episodes whose final graph is within one SHD of the output of the first call to a PC variant; the number of distinct algorithms called; and*tool\-choice dependence*, the mutual information between the family and the tools called\.

## 5Experiments

#### Setup\.

All instances haved=10d=10andn=600n=600, with 100 development and 32 test instances per family\. Reference algorithms are chosen on development instances and every result is on test instances\. Then, we instantiate our harness with three LLMs, GPT\-6\.1 Sol, Claude Opus 5\.5 and Qwen 3\.8 Flash, and run them on the test instances of all six families\. The agents are given the same toolbox and budget, without telling them the family\. We report the adjacency F1 of every agent on every family, against the best chosen algorithm for that family and the ideal oracle\. Intervals in the figures are 95% paired bootstrap intervals over instances; the full numbers are in Appendix[G](https://arxiv.org/html/2610.22846#A7)\.

### 5\.1Headroom above BCA onHeuresis

#### Baselines\.

Besides the best chosen algorithm for each family \(BCAϕ\), we compare three baselines that claim to adapt to diverse data: ➀router, a non\-LLM adaptive controller that sees the same data and CI tests as the agents, screens the data, routes to a suitable algorithm and re\-tests the deleted pairs \(Appendix[E](https://arxiv.org/html/2610.22846#A5)\), and the two recent causal\-discovery foundation models ➁CDFM\([Qiao et al\., 2026](https://arxiv.org/html/2610.22846#bib.bib30)\)and ➂TabCausal\([Li et al\., 2026b](https://arxiv.org/html/2610.22846#bib.bib22)\), run as released with default thresholds and no family label \(Appendix[E](https://arxiv.org/html/2610.22846#A5)\)\. We run them on the test instances of all six families and report their adjacency F1 relative to BCAϕ\(Table[3](https://arxiv.org/html/2610.22846#S5.T3)\); Appendix[F](https://arxiv.org/html/2610.22846#A6)adds scripted repairs of PC’s output, up to a gradient\-boosted classifier\.

Table 3:F1 of the best chosen algorithm \(BCAϕ\) and the oracle, the two headrooms, and the residual gainΔϕr\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}of three baselines\.An oracle that restores only the one to three true edges PC wrongly deletes on an instance, out of roughly ten \(Table[2](https://arxiv.org/html/2610.22846#S3.T2)\), scores \.97 to \.99 on every family, whereas the best chosen algorithm lags 5% to 16% behind, most onwild\(16%\) andgauntlet\(13%\)\. Non\-monotone relations, not heavy\-tailed noise, open the gap \(Figure[1](https://arxiv.org/html/2610.22846#S0.F1)b\)\.

Choosing the best algorithm per family gains \.07 onlgand nothing beyond±\.007\\pm\.007elsewhere \(Figure[1](https://arxiv.org/html/2610.22846#S0.F1)c\)\. The hand\-written router,CDFMandTabCausalnever beat the best chosen algorithm by more than \.03, andCDFMis 10% to 19% below it \(Table[3](https://arxiv.org/html/2610.22846#S5.T3)\)\. Repairing the output of PC from its own log does not help\. Where the deleting test has little power, its recordedpp\-value says almost nothing about whether the deletion was wrong \(Proposition[1](https://arxiv.org/html/2610.22846#Thmproposition1), Appendix[A](https://arxiv.org/html/2610.22846#A1)\), and restoring edges by signals that need new tests does not raise F1 either \(Appendix[F](https://arxiv.org/html/2610.22846#A6)\)\. Closing the gap therefore takes evidence from the individual dataset, which we test with agents in §[5\.2](https://arxiv.org/html/2610.22846#S5.SS2)\.

Takeaway\.Heuresisleaves room that evidence can fill: an oracle that repairs PC’s wrong deletions scores \.97 to \.99 on every family, the best chosen algorithm lags 5% to 16% behind, and none of the algorithms, the router, the two foundation models and the scripted repairs we ran closes the gap\.

### 5\.2Agent Gains over the Best Chosen Algorithm

Figure 6:Residual gain onHeuresisof LLM agents\.Figure[6](https://arxiv.org/html/2610.22846#S5.F6)showsΔϕr\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}of each agent on every family, against the two kinds of headroom \(full results in Table[6](https://arxiv.org/html/2610.22846#A7.T6), Appendix[G](https://arxiv.org/html/2610.22846#A7)\)\. GPT\-6\.1 Sol improves the best chosen algorithm on all six families, and its largest gain, 14%, is onwild, the most difficult family\. Onlgits \.978 lies above the oracle’s \.973, since it also gains outside PC’s deletions\. Claude Opus 5\.5 improves the best chosen algorithm onwildby 7\.4%, about a third of that family’s repair headroom, is on par with the ideal oracle onlg, and loses on no family\. Qwen 3\.8 Flash reaches the best chosen algorithm onlgwithout being told the family, which closes the selection headroom, but it stops there and closes none of the repair headroom\. It is on par with PC\-KCI onanm,wildandmixed, slightly below it onhetero, and recovers only a small part of the headroom above PC\-KCI ongauntlet\. Over all families \(row all\), theΔϕr\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}of GPT\-6\.1 Sol is about twice that of Claude Opus 5\.5\. Repeating GPT\-6\.1 Sol and Claude Opus 5\.5 three times onlg,anm,heteroandwildleaves the ordering of the two agents, and the families on whichΔϕr\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}lies above zero, unchanged \(Table[13](https://arxiv.org/html/2610.22846#A8.T13)\)\. GPT\-6\.1 Sol earns this gain at about $0\.19 per episode, twice the gain of Claude Opus 5\.5 at an eighth of its cost \(Table[12](https://arxiv.org/html/2610.22846#A8.T12)\)\. On the DAG target both agents also lower the nSHD of PC\-KCI’s output on every family \(Table[7](https://arxiv.org/html/2610.22846#A7.T7)\)\.

Takeaway\.LLM\-based agents achieve positive residual gainsΔϕr\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}onHeuresis: the meanΔϕr\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}of every agent is above zero over all families, and on all six families for GPT\-6\.1 Sol, five for Claude Opus 5\.5 and three for Qwen 3\.8 Flash\.

Table 4:Trace attribution\.*merge∗*is the best F1 reached by adding to the PC\-KCI output every edge that at leastkkof the other algorithms the agent called propose, withkkchosen per family in hindsight\.*Unsupp\.*: true edges missing from the PC\-KCI output that the agent restores although no algorithm it called proposes them, out of all such edges it restores\.
### 5\.3Trace\-Level Attribution

To see how the agents reach their gains, we trace every edge they add to the PC\-KCI output they obtained \(Table[4](https://arxiv.org/html/2610.22846#S5.T4)\)\.

A straightforward hypothesis is that the agents merge the outputs of several algorithms, and that the edges they restore are those that at leastkkof the algorithms propose, withkkchosen per family in hindsight\. The traces rule this out\. As shown in Table[4](https://arxiv.org/html/2610.22846#S5.T4), even the bestkk\-vote merge of the outputs the agent called falls far below the agent’s own score onHeuresis\. Over*merge∗*, for instance, GPT\-6\.1 Sol gains 8\.9% on average\. Onwildandgauntlet, where the merge gains nothing over PC\-KCI, 35 of the 104 true edges that GPT\-6\.1 Sol restores, about a third, appear in no algorithm’s output\.

Figure 7:\(a\) Recall of the wrongly deleted edges\. \(b\) Gain in F1 of the guided episode over the unguided episode on the same instance\.As a second hypothesis, the agents could be reading the deletion log of PC\-KCI and restoring the edges it wrongly deleted\. We run the guided condition with the full toolbox and with a minimal toolbox that lacks the code tool, the stability test and the orientation probes\. The results are reported in Figure[7](https://arxiv.org/html/2610.22846#S5.F7)\(see full results in Table[8](https://arxiv.org/html/2610.22846#A7.T8), Appendix[G](https://arxiv.org/html/2610.22846#A7)\)\. We observe that GPT\-6\.1 Sol, the best\-performing LLM, restores nearly all of the wrongly deleted edges onlg\. Handing them both the output of PC and its deletion log adds nothing for GPT\-6\.1 Sol and Claude Opus 5\.5 \(the interval of the guided minus unguided gain contains zero on every family\), so neither channel accounts for their gains\. The toolbox does matter: without the code tool, the stability test and the orientation probes, GPT\-6\.1 Sol’s recall onwildfalls from \.70 to \.47\.

Takeaway\.GPT\-6\.1 Sol and Claude Opus 5\.5 go beyond BCAϕthrough evidence they gather themselves: they test individual edges and probe the mechanism with code, and neither a merge of the algorithms’ outputs nor the output and deletion log of PC reproduces the gain\.

### 5\.4Robustness to Harness Settings

The residual gains do not depend on how the harness is set\. Onlg, the family with selection headroom, andwild, the family with the most repair headroom, GPT\-6\.1 Sol and Claude Opus 5\.5 keep theirΔϕr\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}across the four prompt levels and the three stated targets \(none, CPDAG, DAG; Table[9](https://arxiv.org/html/2610.22846#A7.T9), Appendix[G](https://arxiv.org/html/2610.22846#A7)\); their cells differ by less than the intervals\. Raising the token budget per turn nearly eightfold changes nothing either \(Table[10](https://arxiv.org/html/2610.22846#A7.T10)\)\. Only halving the wall matters, and only for Qwen 3\.8 Flash: a third of its episodes then reach the wall and itsΔϕr\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}onwilddrops from \.032 to \.016, while the other two models never reach even the shorter wall\.

Takeaway\.The residual gains are robust to the prompt, the stated target and the token budget; only a shorter wall lowers the gain of Qwen 3\.8 Flash, whose episodes are the longest\.

## 6Conclusion

We framed agentic causal discovery as evidence\-driven adaptation at several granularities\.Heuresisremoves the names, stresses existing algorithms with six families, and places every agent against fixed references\. With its harness, GPT\-6\.1 Sol surpasses the best chosen algorithm on every family and, together with Claude Opus 5\.5, restores true edges that no algorithm proposes\.

## Limitations

Simulated data\.Real data with a trusted ground\-truth causal graph are rare, and the headroom and the per\-edge decomposition ofHeuresisneed an exact one, so all instances are simulated\. Whether the gains carry over to real data remains open and can be tested as such data become available\. The six families, the graph priors and the continuous variables atd=10d=10andn=600n=600define the setting of every number in this paper; the generator takesdd,nnand the graph density as parameters, and new families can be added\.

Models\.Agent numbers come from three models under the neutral prompt, on 32 test instances per family, with repeats on four families \(Appendix[H](https://arxiv.org/html/2610.22846#A8)\); the guided condition and the prompt and budget sweeps \(on 16 test instances per family\) were run once per model\. Each gain is a statement about that model with this toolbox, and every row records its model, provider and last\-turn setting\. Hosted LLM APIs are not deterministic, so token counts and cost are recorded per episode\.

Reference points\.The collection of existing algorithms is of our own construction and is released with the benchmark, and both headrooms are re\-measured whenever algorithms are added\. The repair headroom is defined on PC’s deletions, so the decomposition explains a graph only through where it differs from the output of PC\. Scores are adjacency F1 with nSHD as the secondary metric\. The code\-analysis tool runs in a child process with a cleaned environment and is not a security sandbox\.

## Ethics Statement and Responsible NLP Checklist Items

The data are synthetic numeric tables with no human subjects, no personal data and no annotators; Appendix[C](https://arxiv.org/html/2610.22846#A3)gives a datasheet\. The benchmark is intended for evaluating the statistical judgment of LLM agents on causal discovery; it should not be used to certify a system for causal conclusions about real populations\. The code and the development instances will be publicly released, the code under the MIT License and the data under CC BY 4\.0\. Compute and API cost, the number of runs per cell and the model versions are reported in Appendix[H](https://arxiv.org/html/2610.22846#A8)\. We used Claude Code for coding and for drafting the paper; the authors verified every number against the result files\.

## References

- Acharya et al\. \(2026\)Sawal Acharya, Terry Jingchen Zhang, Andrew Kim, Rahul Babu Shrestha, Xianlin Sun, Pepijn Cobben, Maximilian Mordig, Jacob T\. Emmerson, et al\. 2026\.[CauSciBench: Can LLMs automate causal inference in real\-world scientific research?](https://proceedings.mlr.press/v306/acharya26a.html)In*Proceedings of the 43rd International Conference on Machine Learning \(ICML\)*, volume 306 of*Proceedings of Machine Learning Research*, pages 332–384\.
- Babakov et al\. \(2025\)Nikolay Babakov, Ehud Reiter, and Alberto Bugarín\. 2025\.[CausalGraphBench: a benchmark for evaluating language models capabilities of causal graph discovery](https://aclanthology.org/2025.acl-srw.16)\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 4: Student Research Workshop\)*, pages 240–258\.
- Ban et al\. \(2023\)Taiyu Ban, Lyuzhou Chen, Derui Lyu, Xiangyu Wang, and Huanhuan Chen\. 2023\.[Causal structure learning supervised by large language model](https://arxiv.org/abs/2311.11689)\.*arXiv preprint arXiv:2311\.11689*\.
- Bender and Friedman \(2018\)Emily M\. Bender and Batya Friedman\. 2018\.[Data statements for natural language processing: Toward mitigating system bias and enabling better science](https://aclanthology.org/Q18-1041/)\.*Transactions of the Association for Computational Linguistics*, 6:587–604\.
- Chickering \(2002\)David Maxwell Chickering\. 2002\.Optimal structure identification with greedy search\.*Journal of Machine Learning Research*, 3:507–554\.
- Cohrs et al\. \(2024\)Kai\-Hendrik Cohrs, Gherardo Varando, Emiliano Diaz, Vasileios Sitokonstantinou, and Gustau Camps\-Valls\. 2024\.[Large language models for constrained\-based causal discovery](https://arxiv.org/abs/2406.07378)\.*arXiv preprint arXiv:2406\.07378*\.
- Deb et al\. \(2026\)Sagar Deb, Devam Shah, and Ashwanth Krishnan\. 2026\.[Active causal discovery benchmark: Evaluating LLM agents under budgeted interventions](https://arxiv.org/abs/2609.31675)\.*arXiv preprint arXiv:2609\.31675*\.
- Gebru et al\. \(2021\)Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford\. 2021\.[Datasheets for datasets](https://arxiv.org/abs/1803.09010)\.*Communications of the ACM*, 64\(12\)\.
- Hafemeister and Satija \(2019\)Christoph Hafemeister and Rahul Satija\. 2019\.Normalization and variance stabilization of single\-cell RNA\-seq data using regularized negative binomial regression\.*Genome Biology*, 20\(1\):296\.
- Han et al\. \(2024\)Kairong Han, Kun Kuang, Ziyu Zhao, Junjian Ye, and Fei Wu\. 2024\.[Causal agent based on large language model](https://arxiv.org/abs/2408.06849)\.*arXiv preprint arXiv:2408\.06849*\.
- Hiremath et al\. \(2025\)Sujai Hiremath, Dominik Janzing, Philipp Faller, Patrick Blöbaum, Elke Kirschbaum, Shiva Prasad Kasiviswanathan, and Kyra Gan\. 2025\.[From Guess2Graph: When and how can unreliable experts safely boost causal discovery in finite samples?](https://arxiv.org/abs/2510.14488)*arXiv preprint arXiv:2510\.14488*\.
- Hoyer et al\. \(2008\)Patrik O\. Hoyer, Dominik Janzing, Joris M\. Mooij, Jonas Peters, and Bernhard Schölkopf\. 2008\.Nonlinear causal discovery with additive noise models\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Jin et al\. \(2023\)Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, Zhiheng Lyu, Kevin Blin, Fernando Gonzalez Adauto, Max Kleiman\-Weiner, Mrinmaya Sachan, and Bernhard Schölkopf\. 2023\.[CLadder: Assessing causal reasoning in language models](https://arxiv.org/abs/2312.04350)\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Jin et al\. \(2024\)Zhijing Jin, Jiarui Liu, Zhiheng Lyu, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona Diab, and Bernhard Schölkopf\. 2024\.[Can large language models infer causation from correlation?](https://arxiv.org/abs/2306.05836)In*International Conference on Learning Representations \(ICLR\)*\.
- Jing et al\. \(2025\)Liqiang Jing, Zhehui Huang, Xiaoyang Wang, Wenlin Yao, Wenhao Yu, Kaixin Ma, Hongming Zhang, Xinya Du, and Dong Yu\. 2025\.[DSBench: How far are data science agents from becoming data science experts?](https://arxiv.org/abs/2409.07703)In*International Conference on Learning Representations \(ICLR\)*\.
- Kapoor et al\. \(2026\)Sayash Kapoor, Benedikt Stroebl, Peter Kirgis, Nitya Nadgir, Zachary S\. Siegel, et al\. 2026\.[Holistic agent leaderboard: The missing infrastructure for AI agent evaluation](https://arxiv.org/abs/2510.11977)\.In*International Conference on Learning Representations \(ICLR\)*\.
- Kapoor et al\. \(2024\)Sayash Kapoor, Benedikt Stroebl, Zachary S\. Siegel, Nitya Nadgir, and Arvind Narayanan\. 2024\.[AI agents that matter](https://arxiv.org/abs/2407.01502)\.*arXiv preprint arXiv:2407\.01502*\.
- Khatibi et al\. \(2024\)Elahe Khatibi, Mahyar Abbasian, Zhongqi Yang, Iman Azimi, and Amir M\. Rahmani\. 2024\.[ALCM: Autonomous LLM\-augmented causal discovery framework](https://arxiv.org/abs/2405.01744)\.*arXiv preprint arXiv:2405\.01744*\.
- Kıcıman et al\. \(2024\)Emre Kıcıman, Robert Ness, Amit Sharma, and Chenhao Tan\. 2024\.[Causal reasoning and large language models: Opening a new frontier for causality](https://arxiv.org/abs/2305.00050)\.*Transactions on Machine Learning Research*\.
- Leban and Sun \(2026\)Andrej Leban and Yuekai Sun\. 2026\.[CausalDS: Benchmarking causal reasoning in data\-science agents](https://arxiv.org/abs/2607.08093)\.*arXiv preprint arXiv:2607\.08093*\.
- Li et al\. \(2026a\)Zi\-Rong Li, Si\-Yang Liu, Tian\-Zuo Wang, and Han\-Jia Ye\. 2026a\.[CausalArena: Benchmarking causal discovery in the foundation model era](https://arxiv.org/abs/2609.11897)\.*arXiv preprint arXiv:2609\.11897*\.
- Li et al\. \(2026b\)Zi\-Rong Li, Si\-Yang Liu, Tian\-Zuo Wang, and Han\-Jia Ye\. 2026b\.[TabCausal: Pretraining across causal environments for tabular causal discovery](https://arxiv.org/abs/2605.31156)\.*arXiv preprint arXiv:2605\.31156*\.
- Liu et al\. \(2024\)Xiao Liu, Zirui Wu, Xueqing Wu, Pan Lu, Kai\-Wei Chang, and Yansong Feng\. 2024\.[Are LLMs capable of data\-based statistical and causal reasoning? benchmarking advanced quantitative reasoning with data](https://arxiv.org/abs/2402.17644)\.In*Findings of the Association for Computational Linguistics: ACL 2024*\.
- Love et al\. \(2014\)Michael I\. Love, Wolfgang Huber, and Simon Anders\. 2014\.Moderated estimation of fold change and dispersion for RNA\-seq data with DESeq2\.*Genome Biology*, 15\(12\):550\.
- Lyu et al\. \(2025\)Ruiqi Lyu, Alistair Turcan, Martin Jinye Zhang, and Bryan Wilder\. 2025\.[Improving constraint\-based discovery with robust propagation and reliable LLM priors](https://arxiv.org/abs/2509.23570)\.*arXiv preprint arXiv:2509\.23570*\.
- Meek \(1995\)Christopher Meek\. 1995\.Causal inference and causal explanation with background knowledge\.In*Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence \(UAI\)*\.
- Miao et al\. \(2026\)Jiacheng Miao, Jin Mu, Guanhua Chen, and James Zou\. 2026\.[Fisher\-r1: Training LLM agents for reliable hypothesis testing](https://arxiv.org/abs/2608.07437)\.*arXiv preprint arXiv:2608\.07437*\.
- Mirzadeh et al\. \(2025\)Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar\. 2025\.[GSM\-Symbolic: Understanding the limitations of mathematical reasoning in large language models](https://arxiv.org/abs/2410.05229)\.In*International Conference on Learning Representations \(ICLR\)*\.
- Mooij et al\. \(2016\)Joris M\. Mooij, Jonas Peters, Dominik Janzing, Jakob Zscheischler, and Bernhard Schölkopf\. 2016\.Distinguishing cause from effect using observational data: methods and benchmarks\.*Journal of Machine Learning Research*, 17\(32\):1–102\.
- Qiao et al\. \(2026\)Jie Qiao, Ruichu Cai, Zijian Li, Weilin Chen, Pengfei Hua, Boyan Xu, Zhengming Chen, Zhifeng Hao, and Peng Cui\. 2026\.[CDFM: Towards a general\-purpose causal discovery foundation model](https://arxiv.org/abs/2607.11508)\.*arXiv preprint arXiv:2607\.11508*\.
- Reisach et al\. \(2021\)Alexander G\. Reisach, Christof Seiler, and Sebastian Weichwald\. 2021\.[Beware of the simulated DAG\! causal discovery benchmarks may be easy to game](https://arxiv.org/abs/2102.13647)\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.
- Sachs et al\. \(2005\)Karen Sachs, Omar Perez, Dana Pe’er, Douglas A\. Lauffenburger, and Garry P\. Nolan\. 2005\.Causal protein\-signaling networks derived from multiparameter single\-cell data\.*Science*, 308\(5721\):523–529\.
- Scutari \(2022\)Marco Scutari\. 2022\.Bayesian network repository\.[https://www\.bnlearn\.com/bnrepository/](https://www.bnlearn.com/bnrepository/)\.Page last updated November 2022\.
- Shimizu et al\. \(2006\)Shohei Shimizu, Patrik O\. Hoyer, Aapo Hyvärinen, and Antti Kerminen\. 2006\.A linear non\-Gaussian acyclic model for causal discovery\.*Journal of Machine Learning Research*, 7:2003–2030\.
- Spirtes et al\. \(2000\)Peter Spirtes, Clark Glymour, and Richard Scheines\. 2000\.*Causation, Prediction, and Search*, 2nd edition\.MIT Press\.
- Srivastava et al\. \(2025\)Ashutosh Srivastava, Lokesh Nagalapatti, Gautam Jajoo, Aniket Vashishtha, Parameswari Krishnamurthy, and Amit Sharma\. 2025\.[Realizing LLMs’ causal potential requires science\-grounded, novel benchmarks](https://arxiv.org/abs/2510.16530)\.*arXiv preprint arXiv:2510\.16530*\.
- Strobl et al\. \(2019\)Eric V\. Strobl, Kun Zhang, and Shyam Visweswaran\. 2019\.Approximate kernel\-based conditional independence tests for fast non\-parametric causal discovery\.*Journal of Causal Inference*, 7\(1\)\.
- Wang et al\. \(2025\)Xinyue Wang, Kun Zhou, Wenyi Wu, Har Simrat Singh, Fang Nan, Songyao Jin, Aryan Philip, Saloni Patnaik, Hou Zhu, Shivam Singh, Parjanya Prashant, Qian Shen, and Biwei Huang\. 2025\.[Causal\-copilot: An autonomous causal analysis agent](https://arxiv.org/abs/2504.13263)\.*arXiv preprint arXiv:2504\.13263*\.
- Wu et al\. \(2025\)Xingyu Wu, Kui Yu, Jibin Wu, and Kay Chen Tan\. 2025\.[LLM cannot discover causality, and should be restricted to non\-decisional support in causal discovery](https://arxiv.org/abs/2506.00844)\.*arXiv preprint arXiv:2506\.00844*\.
- Yang et al\. \(2026\)Junlin Yang, Dylan Zhang, Xiangchen Song, Qirun Dai, Xiao Liu, Yuen Chen, Aniket Vashishtha, Jing Shi, Chenhao Tan, and Hao Peng\. 2026\.[CausaLab: A scalable environment for interactive causal discovery toward AI scientists](https://arxiv.org/abs/2605.26029)\.*arXiv preprint arXiv:2605\.26029*\.
- Yu and Zhou \(2026\)Zhenyu Yu and Shuigeng Zhou\. 2026\.[Caliper: Probing lexical anchors versus causal structure in LLMs](https://arxiv.org/abs/2606.04915)\.*arXiv preprint arXiv:2606\.04915*\.
- Zhang et al\. \(2011\)Kun Zhang, Jonas Peters, Dominik Janzing, and Bernhard Schölkopf\. 2011\.Kernel\-based conditional independence test and application in causal discovery\.In*Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence \(UAI\)*\.
- Zheng et al\. \(2024\)Yujia Zheng, Biwei Huang, Wei Chen, Joseph Ramsey, Mingming Gong, Ruichu Cai, Shohei Shimizu, Peter Spirtes, and Kun Zhang\. 2024\.[Causal\-learn: Causal discovery in Python](https://arxiv.org/abs/2307.16405)\.*Journal of Machine Learning Research*, 25\(60\)\.
- Zheng et al\. \(2026\)Yujia Zheng, Vishal Verma, Mantej Gill, Haoyue Dai, Peter Spirtes, and Kun Zhang\. 2026\.[Causal discovery in the era of agents](https://arxiv.org/abs/2606.23608)\.*arXiv preprint arXiv:2606\.23608*\.
- Zhou et al\. \(2024\)Yu Zhou, Xingyu Wu, Beicheng Huang, Jibin Wu, Liang Feng, and Kay Chen Tan\. 2024\.[CausalBench \(CausalBN\-Bench\): A comprehensive benchmark for causal learning capability of LLMs](https://arxiv.org/abs/2404.06349)\.*arXiv preprint arXiv:2404\.06349*\.

## Appendix AProposition[1](https://arxiv.org/html/2610.22846#Thmproposition1)and its Proof

###### Proposition 1\.

Letα\\alphabe the level of the test that deleted a pair andβ\\betaits power against the dependence that exists\. Condition on deletion and letp0p\_\{0\}be the recordedpp\-value\. For a wrongly deleted true edge the law ofp0p\_\{0\}on\(α,1\]\(\\alpha,1\]has densitygβg\_\{\\beta\}; for a rightly deleted non\-edge it is uniform\. Ifgβ→1g\_\{\\beta\}\\to 1uniformly as the dependence vanishes, so thatβ↓α\\beta\\downarrow\\alpha, then the likelihood ratio between the two classes tends to 1 uniformly, and every ranker that is a function ofp0p\_\{0\}has AUC→12\\to\\tfrac\{1\}\{2\}for detecting wrong deletions\.

#### Setting\.

PC deletes a pair at the first set in its enumeration whose test returns app\-value aboveα\\alpha\. Condition on deletion and writep0p\_\{0\}for the recordedpp\-value, which lies in\(α,1\]\(\\alpha,1\]\. Two assumptions idealize the two classes\.

*\(A1\) Right deletions\.*For a deleted non\-edge, the accepting set is a true separating set, sop0p\_\{0\}is a draw from a true null conditioned on exceedingα\\alpha: its density on\(α,1\]\(\\alpha,1\]isfr​\(p\)=1/\(1−α\)f\_\{r\}\(p\)=1/\(1\-\\alpha\)\.

*\(A2\) Wrong deletions\.*For a deleted true edge, every test is under the alternative\. Index the alternative by an effect sizeδ≥0\\delta\\geq 0\. Thepp\-value of the accepting test has densitygδg\_\{\\delta\}on\(0,1\]\(0,1\], withg0≡1g\_\{0\}\\equiv 1, and powerβδ=∫0αgδ\\beta\_\{\\delta\}=\\int\_\{0\}^\{\\alpha\}g\_\{\\delta\}\. Conditioned on deletion, the density ofp0p\_\{0\}on\(α,1\]\(\\alpha,1\]isfw​\(p\)=gδ​\(p\)/\(1−βδ\)f\_\{w\}\(p\)=g\_\{\\delta\}\(p\)/\(1\-\\beta\_\{\\delta\}\), because1−βδ=∫α1gδ1\-\\beta\_\{\\delta\}=\\int\_\{\\alpha\}^\{1\}g\_\{\\delta\}\. We assumegδ→1g\_\{\\delta\}\\to 1uniformly on\(α,1\]\(\\alpha,1\]asδ→0\\delta\\to 0\.

#### Claim\.

Asδ→0\\delta\\to 0,βδ→α\\beta\_\{\\delta\}\\to\\alpha, the likelihood ratioΛδ=fw/fr\\Lambda\_\{\\delta\}=f\_\{w\}/f\_\{r\}satisfiessup\(α,1\]\|Λδ−1\|→0\\sup\_\{\(\\alpha,1\]\}\|\\Lambda\_\{\\delta\}\-1\|\\to 0, and for every measurable scores⁡\(p0\)s\(p\_\{0\}\)the AUC for detecting wrong deletions satisfies\|AUC⁡\(s\)−12\|≤12​sup\|Λδ−1\|\|\\mathrm\{AUC\}\(s\)\-\\tfrac\{1\}\{2\}\|\\leq\\tfrac\{1\}\{2\}\\sup\|\\Lambda\_\{\\delta\}\-1\|\.

#### Proof\.

By uniform convergence on a bounded interval,1−βδ=∫α1gδ→1−α1\-\\beta\_\{\\delta\}=\\int\_\{\\alpha\}^\{1\}g\_\{\\delta\}\\to 1\-\\alpha, soβδ→α\\beta\_\{\\delta\}\\to\\alpha\. ThenΛδ​\(p\)=gδ​\(p\)​\(1−α\)/\(1−βδ\)\\Lambda\_\{\\delta\}\(p\)=g\_\{\\delta\}\(p\)\(1\-\\alpha\)/\(1\-\\beta\_\{\\delta\}\)is a ratio of uniformly convergent positive functions with limit 1, hence converges to 1 uniformly\. The total variation distance between the two classes isTV=12​∫α1\|fw−fr\|=12​𝔼fr​\|Λδ−1\|≤12​sup\|Λδ−1\|\\mathrm\{TV\}=\\tfrac\{1\}\{2\}\\int\_\{\\alpha\}^\{1\}\|f\_\{w\}\-f\_\{r\}\|=\\tfrac\{1\}\{2\}\\,\\mathbb\{E\}\_\{f\_\{r\}\}\|\\Lambda\_\{\\delta\}\-1\|\\leq\\tfrac\{1\}\{2\}\\sup\|\\Lambda\_\{\\delta\}\-1\|\. LetX∼fwX\\sim f\_\{w\},Y∼frY\\sim f\_\{r\}andX′∼frX^\{\\prime\}\\sim f\_\{r\}be independent, and leth\(x,y\)=𝟏\{s\(x\)\>s\(y\)\}\+12𝟏\{s\(x\)=s\(y\)\}∈\[0,1\]h\(x,y\)=\\mathbf\{1\}\\\{s\(x\)\>s\(y\)\\\}\+\\tfrac\{1\}\{2\}\\mathbf\{1\}\\\{s\(x\)=s\(y\)\\\}\\in\[0,1\]\. ThenAUC⁡\(s\)=𝔼​h​\(X,Y\)\\mathrm\{AUC\}\(s\)=\\mathbb\{E\}\\,h\(X,Y\)and, by symmetry,𝔼​h​\(X′,Y\)=12\\mathbb\{E\}\\,h\(X^\{\\prime\},Y\)=\\tfrac\{1\}\{2\}\. Becausehhtakes values in\[0,1\]\[0,1\],\|𝔼​h​\(X,Y\)−𝔼​h​\(X′,Y\)\|≤TV\|\\mathbb\{E\}\\,h\(X,Y\)\-\\mathbb\{E\}\\,h\(X^\{\\prime\},Y\)\|\\leq\\mathrm\{TV\}\.□\\square

#### Scope of the proposition\.

The result concerns rankers that are functions ofp0p\_\{0\}only and approachesAUC=12\\mathrm\{AUC\}=\\tfrac\{1\}\{2\}as power vanishes\. Signals that use other information achieve AUCs from 0\.64 to 0\.90 in the first campaign \(Appendix[F](https://arxiv.org/html/2610.22846#A6)\)\. A wrong deletion also requires some test in the enumeration to miss a real dependence, which is likely exactly whereβδ\\beta\_\{\\delta\}is small; this is the sense in which the errors and the flat evidence share one cause\. Assumption \(A1\) is an idealization, since a non\-edge can also be deleted at a non\-separating set by a test that happens to miss a dependence\.

## Appendix BInstrument verification

Table[5](https://arxiv.org/html/2610.22846#A2.T5)lists the checks run on the instruments as they now stand \(31 cluster nodes, about7×1057\\times 10^\{5\}CPU\-seconds\)\.

CheckReferenceScaleResultGraph primitivesd\-separationtwo independent implementations \(moralized ancestral graph; path blocking\)2\.16×1062\.16\\times 10^\{6\}random DAGs,4\.3×1084\.3\\times 10^\{8\}queries0 disagreementsDAG to CPDAGbrute\-force enumeration of the Markov equivalence class; Chickering’s edge labeling; causal\-learn3\.4×1063\.4\\times 10^\{6\}DAGs \(enumeration\),5\.8×1065\.8\\times 10^\{6\}DAGs \(labeling, causal\-learn\)0 disagreementsExhaustived≤6d\\leq 6causal\-learn: d\-separation, CPDAG and oracle PCall3,811,3553\{,\}811\{,\}355labeled DAGs \(equals OEIS A003024\)0 disagreementsPCOracle PCtrue CPDAG returned, first\-set and union rules6×1056\\times 10^\{5\}random DAGs plus the exhaustive set0 failuresSkeleton and separating setsclean\-headroom PC\-stable; causal\-learn896,000896\{,\}000lginstances,d∈\{6,…,30\}d\\in\\\{6,\\dots,30\\\},n∈\{300,600,2000\}n\\in\\\{300,600,2000\\\}identical on allSeed PC \(union rule\)causal\-learnpc\(stable, uc\_priority=2\)same instancesidentical CPDAG on892,563892\{,\}563\(99\.62%\); the other3,4373\{,\}437have a cyclic causal\-learn outputScoringScorergadjid SHD1\.2×1051\.2\\times 10^\{5\}graph pairs0 disagreementsInvariances“no repair” equals PC; oracle PC under relabeling1,8001\{,\}800instancespassConditional\-independence testsDeterminismseeded runs across 14 nodes, 8 hash seeds, 1 to all BLAS threadsFisher\-zzand kernel testsFisher\-zzbit\-identical; kernellog10⁡p\\log\_\{10\}pwithin10−1010^\{\-10\}, 0 verdict flipsTable 5:Instrument verification checks and results, grouped by the component under test\.
## Appendix CGenerator specification and datasheet

#### Motivation and intended use\.

Heuresismeasures whether an LLM agent goes beyond calling PC when it discovers causal structure from data alone, and, in a guided condition, what it adds to the output of PC\. The benchmark targets statistical adaptation in this tool setting, alongside separate questions about general causal reasoning and coding ability\.

#### Composition\.

Each instance is a tuple of an anonymous data matrixX∈ℝn×dX\\in\\mathbb\{R\}^\{n\\times d\}, for the guided condition the PC seed graph and the deletion log, and a hidden record of the SCMSS\(true DAGGG, CPDAG, per\-deletion labels\)\. Instances per family: 100 development and 32 test instances, all withd=10d=10andn=600n=600\.

#### Mechanisms\.

Linear:Xj=∑iwi​j​Xi\+εjX\_\{j\}=\\sum\_\{i\}w\_\{ij\}X\_\{i\}\+\\varepsilon\_\{j\},wi​j∼±U⁡\[0\.5,2\]w\_\{ij\}\\sim\\pm U\[0\.5,2\]\. MLP:Xj=fj​\(pa⁡\(j\)\)\+εjX\_\{j\}=f\_\{j\}\(\\mathrm\{pa\}\(j\)\)\+\\varepsilon\_\{j\},fjf\_\{j\}a random one\-hidden\-layer MLP with 10 tanh units\. MLP with input\-dependent noise:Xj=fj​\(pa⁡\(j\)\)\+gj​\(pa⁡\(j\)\)​εjX\_\{j\}=f\_\{j\}\(\\mathrm\{pa\}\(j\)\)\+g\_\{j\}\(\\mathrm\{pa\}\(j\)\)\\,\\varepsilon\_\{j\},gj=softplus⁡\(MLP\)\+0\.2g\_\{j\}=\\mathrm\{softplus\}\(\\mathrm\{MLP\}\)\+0\.2,εj∼N⁡\(0,1\)\\varepsilon\_\{j\}\\sim N\(0,1\)\. The scalar maps \(piecewise linear with a random knot and two signed slopes, sigmoid, random Fourier features, quadratic, absolute value\) act onzj=wj⊤​Xpa⁡\(j\)z\_\{j\}=w\_\{j\}^\{\\top\}X\_\{\\mathrm\{pa\}\(j\)\}with a random unit\-normwjw\_\{j\}; inwildthey are standardized\. Sigmoid with input\-dependent noise istanh⁡\(a​zj\)\+\(softplus⁡\(b​zj\)\+0\.2\)​εj\\tanh\(az\_\{j\}\)\+\(\\mathrm\{softplus\}\(bz\_\{j\}\)\+0\.2\)\\,\\varepsilon\_\{j\}\. Linear witht3t\_\{3\}noise replacesεj\\varepsilon\_\{j\}by scaled Student\-t3t\_\{3\}noise\. Noise scales:σ∼U⁡\[0\.5,2\]\\sigma\\sim U\[0\.5,2\]for the linear nodes oflgandmixed;U⁡\[0\.5,1\.5\]U\[0\.5,1\.5\]for MLP nodes and the linear nodes ofwild;U⁡\[0\.15,0\.35\]U\[0\.15,0\.35\]for the maps ofwild;U⁡\[0\.15,0\.40\]U\[0\.15,0\.40\]ingauntlet, whose root nodes areN⁡\(0,1\)N\(0,1\)\.

#### Collection process\.

Instances are generated by the code in the repository from a seed\. Graphs are Erdős–Rényi or Barabási–Albert over a uniformly random causal order, with expected degree 2 by default, so aboutddedges\. Variables are named in random order\. Columns of continuous families are standardized to mean 0 and variance 1, which removes a known shortcut of simulated data, where the variance of a variable reveals its position in the causal order\([Reisach et al\., 2021](https://arxiv.org/html/2610.22846#bib.bib31)\); after standardization this shortcut is at chance level\. Development and test seeds are disjoint\. Every reference algorithm is chosen on development instances and reported on test instances, and we report distributions over instances rather than single draws\([Mirzadeh et al\., 2025](https://arxiv.org/html/2610.22846#bib.bib28)\)\.

#### Design rationale\.

The families beyondlgare motivated by properties that real biological data are known to have\. Protein\-signaling data show nonlinear dependences between the measured proteins\([Sachs et al\., 2005](https://arxiv.org/html/2610.22846#bib.bib32)\);anm,wildandgauntletmodel such dependences with nonlinear maps\. Gene\-expression data have noise whose scale depends on the signal\([Love et al\., 2014](https://arxiv.org/html/2610.22846#bib.bib24);[Hafemeister and Satija, 2019](https://arxiv.org/html/2610.22846#bib.bib9)\);heteromodels this by letting the noise scale depend on the parents\. Each family beyondlgtherefore stresses an assumption of the standard algorithms that real data of this kind also fail to meet\.

#### Data statement\.

The data are synthetic numeric tables\. There is no human language, no human subject, no personal data and no annotator population, so there is nothing to report on speakers or annotators\([Bender and Friedman, 2018](https://arxiv.org/html/2610.22846#bib.bib4);[Gebru et al\., 2021](https://arxiv.org/html/2610.22846#bib.bib8)\)\.

#### Maintenance\.

Frozen development instances are public\. Test seeds are drawn after the tag that fixes the prompt, tool schema, budgets, model list and analysis code; they are sealed and rotated on a published schedule\.

## Appendix DDetails of the probes in Figure[1](https://arxiv.org/html/2610.22846#S0.F1)

Both probes use Qwen 3\.8 Flash in thinking mode at temperature 0\.

#### Leakage \(Figure[1](https://arxiv.org/html/2610.22846#S0.F1)a\)\.

We give the model only the variable names, with no data, and ask for the skeleton and the edge directions of seven BNLearn networks\([Scutari, 2022](https://arxiv.org/html/2610.22846#bib.bib33)\)\(Asia, Cancer, Earthquake, Sachs, Child, Insurance, Alarm\) and, for each of the 101 Tübingen pairs with a parseable description\([Mooij et al\., 2016](https://arxiv.org/html/2610.22846#bib.bib29)\), which variable causes which\. As controls we replace the names with indices in random order and, for the networks, add eightHeuresisinstances, whose variables never had names\. With the names, the model gets 99% of the Tübingen directions right and most of the BNLearn skeletons; without them it falls to chance, and on theHeuresisinstances it is near zero\.

#### Saturation on the BNLearn structures\.

We simulate linear\-Gaussian data on the anonymized BNLearn structures and run PC with a Fisher\-zztest and with the kernel conditional\-independence test KCI\([Zhang et al\., 2011](https://arxiv.org/html/2610.22846#bib.bib42)\), GES, and the ideal oracle that restores every true edge PC wrongly deleted\. GES alone recovers from 31% \(Insurance\) to all \(Cancer, Earthquake\) of what the ideal oracle gains over PC\-KCI\. An agent given these instances does no better than the best algorithm: Qwen 3\.8 Flash finishes on par with GES, and in 28% of its episodes it returns the output of PC unchanged\. The continuous deviations from the linear\-Gaussian case in Figure[1](https://arxiv.org/html/2610.22846#S0.F1)b are described in §[4\.1](https://arxiv.org/html/2610.22846#S4.SS1)and §[5\.1](https://arxiv.org/html/2610.22846#S5.SS1)\.

## Appendix EThe router baseline

Therouteris a hand\-crafted rule that decides, from the data of one instance and without the family label, whose output to submit\. It has three steps\.

#### Screen\.

The router first runs PC with the Fisher\-zztest atα=0\.05\\alpha=0\.05and records the pairs that PC deletes\. For each deleted pair it computes the gap between the distance correlation and the absolute Pearson correlation of the two variables, on a fixed random subsample of at most 300 rows\. A gap of at least 0\.15 flags the pair as dependent in a way that a linear test cannot see\.

#### Route\.

If the flagged pairs make up at least a fractionτ=0\.2\\tau=0\.2of the deleted pairs, the router switches to PC with the kernel\-based conditional independence test \(KCI\)\. Otherwise it keeps the output of PC with the Fisher\-zztest\.

#### Re\-test\.

On the KCI branch the router re\-tests every pair that PC with KCI deleted, and restores a pair only if two conditions hold: KCI rejects independence at the conditioning set that deleted the pair, with a Holm correction across all deleted pairs of the instance, and no alternative conditioning set that the PC graph implies for the pair returns independence at levelα\\alpha\. A pair is a true non\-edge if some set separates it, so a test at another set can only confirm a deletion\. On the full sample, the test at the original set is the same test on the same data that deleted the pair; it returns the samepp\-value aboveα\\alphaand cannot reject\. The KCI branch therefore returns the graph of PC with KCI, and the router equals PC with KCI on the instances it routes there and PC with the Fisher\-zztest on the others\.

#### Foundation models\.

CDFM\([Qiao et al\., 2026](https://arxiv.org/html/2610.22846#bib.bib30)\)runs from its released checkpoint \(DMIRLAB/CDFM\) and binarizes its edge logits with the threshold that the checkpoint calibrates from the logits themselves\.TabCausal\([Li et al\., 2026b](https://arxiv.org/html/2610.22846#bib.bib22)\)runs from its released checkpoint with its exponential\-moving\-average weights and the default edge threshold of 0\.5\. Both read the same\(n,d\)\(n,d\)data matrix as the agents in one forward pass; nothing is tuned onHeuresis, and neither sees the family label or a development split\.

## Appendix FRepairing the output of PC from its own log

A script can also try to find PC’s mistakes without a model of the data\. Figure[8](https://arxiv.org/html/2610.22846#A6.F8)a ranks the signals that the deletion log of PC\-KCI offers for telling a wrongly deleted edge from a rightly deleted one\. The two signals available from the log alone, thepp\-value of the deleting test and the size of its conditioning set, rank wrong deletions only moderately, and a classifier on the four such features restores nothing at any threshold chosen on development instances\. For thepp\-value this is what Proposition[1](https://arxiv.org/html/2610.22846#Thmproposition1)predicts where the deleting test has little power: a ranker that depends on the recordedpp\-value alone tends to chance as power vanishes\. Signals that need new tests rank better: the minimum KCIpp\-value over the alternative conditioning sets that the PC graph implies for the pair is the best single signal on every family, and a gradient\-boosted classifier on all nine features, trained with the family held out, ranks higher still\. Ranking the deletions, however, does not translate into a gain from restoring them \(Figure[8](https://arxiv.org/html/2610.22846#A6.F8)b\)\. Because wrong deletions are only a few percent of all deletions, a threshold on the best signal restores far more right deletions than wrong ones and lowers adjacency F1 on all four families it was run on \(lg,anm,heteroandwild\), and the classifier at its best threshold gains nothing distinguishable from zero\.

Figure 8:Scripted use of the deletion log of PC\-KCI\. \(a\) AUC of each signal for ranking the wrongly deleted edges among all deletions; the first two signals need no new tests\. \(b\) Change in adjacency F1 from restoring edges by each signal, compared with the ideal oracle that restores every wrongly deleted edge\. 32 test instances per family\. Error bars represent bootstrapped 95% confidence intervals\.
## Appendix GReference scores, agent results and the guided condition in full

Table[3](https://arxiv.org/html/2610.22846#S5.T3)gives the reference scores and the two kinds of headroom behind Figure[1](https://arxiv.org/html/2610.22846#S0.F1)c and §[5\.1](https://arxiv.org/html/2610.22846#S5.SS1); Table[6](https://arxiv.org/html/2610.22846#A7.T6)the agent results behind Figure[6](https://arxiv.org/html/2610.22846#S5.F6), with every repeat and last\-turn setting in Table[13](https://arxiv.org/html/2610.22846#A8.T13); and Table[8](https://arxiv.org/html/2610.22846#A7.T8)the guided condition behind Figure[7](https://arxiv.org/html/2610.22846#S5.F7)\.

Table 6:Agent performance \(every episode submitted a graph\) on 32 test instances per family: adjacency F1 and the paired difference from the family’s best chosen algorithm\. Rows marked \(all\) are over all families\. GPT\-6\.1 Sol and Claude Opus 5\.5 are shown in repeat 1; their repeats 2 and 3, and the finished\-only and all\-episode views of the runs without a retry in the last turn, are in Table[13](https://arxiv.org/html/2610.22846#A8.T13)\.Table 7:Agent gain over the in\-episode output of PC\-KCI \(GES on the last row of each model\), scored by the same scorer on the same instance, repeat 1, mean paired gain with 95% bootstrap intervals over instances; positive is better for the agent, and nSHD is reported as its reduction\.nnis the number of episodes in which the agent called the algorithm \(all 32 for GPT\-6\.1 Sol\)\. Bold: the interval excludes zero\.ToolboxFamilyPrecisionRecallvs PC\-KCIGuided−\-unguidedGPT\-6\.1 Solfulllg\.88 \[\.77, \.96\]\.94 \[\.89, \.99\]\.112 \[\.074, \.149\]−\-\.006 \[−\-\.019, \.004\]anm\.87 \[\.78, \.95\]\.76 \[\.63, \.88\]\.048 \[\.028, \.072\]−\-\.003 \[−\-\.016, \.008\]hetero\.89 \[\.79, \.98\]\.80 \[\.68, \.91\]\.039 \[\.023, \.055\]−\-\.004 \[−\-\.013, \.006\]wild\.98 \[\.94, 1\.00\]\.70 \[\.59, \.80\]\.125 \[\.093, \.156\]\.009 \[−\-\.007, \.028\]minimallg\.83 \[\.73, \.93\]\.90 \[\.84, \.96\]\.091 \[\.057, \.125\]−\-\.027 \[−\-\.051,−\-\.006\]anm\.62 \[\.47, \.78\]\.57 \[\.42, \.72\]\.005 \[−\-\.023, \.033\]−\-\.047 \[−\-\.075,−\-\.021\]hetero\.81 \[\.68, \.92\]\.61 \[\.45, \.75\]\.014 \[−\-\.001, \.028\]−\-\.029 \[−\-\.043,−\-\.015\]wild\.83 \[\.71, \.93\]\.47 \[\.34, \.61\]\.063 \[\.032, \.093\]−\-\.053 \[−\-\.075,−\-\.033\]Claude Opus 5\.5fulllg\.89 \[\.80, \.96\]\.94 \[\.89, \.99\]\.109 \[\.077, \.144\]−\-\.003 \[−\-\.020, \.013\]anm\.66 \[\.51, \.80\]\.55 \[\.39, \.69\]\.009 \[−\-\.018, \.034\]\.009 \[−\-\.011, \.029\]hetero\.75 \[\.61, \.88\]\.64 \[\.47, \.79\]\.007 \[−\-\.015, \.028\]−\-\.003 \[−\-\.016, \.012\]wild\.86 \[\.76, \.95\]\.49 \[\.37, \.62\]\.070 \[\.043, \.097\]\.009 \[−\-\.013, \.034\]minimallg\.86 \[\.77, \.95\]\.88 \[\.79, \.95\]\.095 \[\.061, \.131\]−\-\.017 \[−\-\.030,−\-\.004\]anm\.42 \[\.26, \.58\]\.54 \[\.39, \.69\]−\-\.031 \[−\-\.065, \.007\]−\-\.030 \[−\-\.059,−\-\.001\]hetero\.62 \[\.45, \.78\]\.60 \[\.43, \.75\]−\-\.010 \[−\-\.032, \.012\]−\-\.020 \[−\-\.038,−\-\.002\]wild\.50 \[\.35, \.63\]\.41 \[\.27, \.54\]\.021 \[−\-\.006, \.050\]−\-\.040 \[−\-\.076,−\-\.009\]Qwen 3\.8 Flashfulllg\.89 \[\.80, \.97\]\.91 \[\.81, \.98\]\.096 \[\.046, \.141\]\.012 \[−\-\.035, \.053\]anm\.71 \[\.56, \.85\]\.42 \[\.27, \.58\]\.004 \[−\-\.018, \.026\]\.012 \[−\-\.020, \.045\]hetero\.87 \[\.75, \.97\]\.44 \[\.27, \.60\]−\-\.003 \[−\-\.018, \.013\]\.020 \[−\-\.011, \.052\]wild\.78 \[\.65, \.90\]\.35 \[\.22, \.49\]\.033 \[\.012, \.056\]\.009 \[−\-\.024, \.043\]minimallg\.84 \[\.71, \.94\]\.56 \[\.42, \.71\]\.034 \[−\-\.002, \.071\]−\-\.050 \[−\-\.084,−\-\.018\]anm\.56 \[\.40, \.72\]\.35 \[\.20, \.51\]−\-\.034 \[−\-\.056,−\-\.010\]−\-\.026 \[−\-\.063, \.011\]hetero\.85 \[\.74, \.95\]\.56 \[\.39, \.72\]\.008 \[−\-\.004, \.020\]\.030 \[\.004, \.060\]wild\.71 \[\.55, \.85\]\.23 \[\.12, \.35\]−\-\.002 \[−\-\.029, \.023\]−\-\.025 \[−\-\.059, \.008\]Table 8:Guided condition \(32 test instances per family, all episodes\): precision and recall of restoring the wrongly deleted edges of PC\-KCI, the paired gain over PC\-KCI, and the paired gain over the agent’s own unguided run with the full toolbox on the same instances \(repeat 1 for GPT\-6\.1 Sol and Claude Opus 5\.5\), with 95% bootstrap intervals over episodes\.Table 9:Prompt level \(each level adds to the previous one: neutral task statement; budgets and the scoring statement; a warning that standard algorithms can fail under nonlinear or heteroscedastic mechanisms; PC’s deletion rule\) and stated target \(first 16 of the 32 test instances per family, each row averaged over the other factor; GPT\-6\.1 Sol and Claude Opus 5\.5 in repeat 1\): paired gain in adjacency F1 over the best chosen algorithm, share of episodes whose final graph is PC’s output, and share of episodes that reached the wall\.Table 10:Budget sweep \(first 16 of the 32 test instances per family, each row averaged over the other factor; GPT\-6\.1 Sol and Claude Opus 5\.5 in repeat 1\): paired gain in adjacency F1 over the best chosen algorithm, share of episodes that reached the wall and submitted in the retry, share left unsubmitted, and completion tokens per episode\.Table 11:Trivial floors \(adjacency F1, 32 test instances per family\): the empty graph, the complete graph \(every deletion of PC\-KCI restored\), PC\-KCI itself \(nothing restored\), a random restoration of as many deleted pairs as the oracle restores \(20 draws per instance\), and the oracle\.
## Appendix HHarness and reporting disclosure

Table[12](https://arxiv.org/html/2610.22846#A8.T12)records, for every agent result in this paper, the model, harness and budget it was obtained under\.

Figure 9:Qwen 3\.8 Flash onwildunder the three last\-turn settings of Table[12](https://arxiv.org/html/2610.22846#A8.T12)\(no last turn, one last turn in which it can only submit, the same turn with a retry\), scored over all episodes and over finished episodes only\. Intervals as in Figure[6](https://arxiv.org/html/2610.22846#S5.F6)\.#### Truncated episodes\.

Without a retry in the last turn, some episodes reached the time limit without submitting a graph, up to a fifth of them onwild\(Table[13](https://arxiv.org/html/2610.22846#A8.T13)\)\. Over the finished episodes alone, Qwen 3\.8 Flash scored clearly above PC\-KCI onwild, a family without selection headroom, where any gain would lie above the best chosen algorithm; over all episodes it scored below \(Figure[9](https://arxiv.org/html/2610.22846#A8.F9)\)\. The episodes that ran out of budget were the hard ones; conditioning on completion therefore inflates the score\. The last turn therefore reads the final edge list tolerantly and allows one retry after a rejected list, so that every episode submits a graph and the two views coincide\. We therefore fix the following reporting rule for every later run: every cell is reported both over all episodes and over finished episodes, with the truncation rate beside them, and we claim a capability only under a harness in which the two agree\.

Table 12:Harness disclosure for the agent results\.Table[13](https://arxiv.org/html/2610.22846#A8.T13)gives every repeat of the main condition separately, with the last\-turn setting it ran under, the number of episodes that reached the wall and the number of those that still submitted a graph in the last turn, so that the all\-episode and finished\-only views of Table[6](https://arxiv.org/html/2610.22846#A7.T6)can be traced to the change in that turn that separates them\.

all episodesfinishedFamilyRepeatLast turnWall / rescuedF1Δϕr\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}nnF1Δϕr\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}GPT\-6\.1 Sol \(reasoning effort “medium”\)lg1retry0 / 0\.978\.043 \[\.019, \.070\]32\.978\.043 \[\.019, \.070\]2retry0 / 0\.983\.048 \[\.021, \.078\]32\.983\.048 \[\.021, \.078\]3retry0 / 0\.983\.048 \[\.021, \.077\]32\.983\.048 \[\.021, \.077\]anm1retry0 / 0\.958\.052 \[\.032, \.074\]32\.958\.052 \[\.032, \.074\]2retry0 / 0\.956\.050 \[\.029, \.072\]32\.956\.050 \[\.029, \.072\]3retry0 / 0\.946\.040 \[\.008, \.068\]32\.946\.040 \[\.008, \.068\]hetero1retry0 / 0\.967\.042 \[\.027, \.058\]32\.967\.042 \[\.027, \.058\]2retry0 / 0\.960\.036 \[\.018, \.053\]32\.960\.036 \[\.018, \.053\]3retry0 / 0\.961\.037 \[\.019, \.055\]32\.961\.037 \[\.019, \.055\]wild1retry0 / 0\.935\.115 \[\.083, \.146\]32\.935\.115 \[\.083, \.146\]2retry0 / 0\.939\.119 \[\.089, \.150\]32\.939\.119 \[\.089, \.150\]3retry0 / 0\.933\.113 \[\.083, \.142\]32\.933\.113 \[\.083, \.142\]mixed1retry0 / 0\.972\.045 \[\.028, \.065\]32\.972\.045 \[\.028, \.065\]gauntlet1retry0 / 0\.933\.084 \[\.059, \.111\]32\.933\.084 \[\.059, \.111\]Claude Opus 5\.5 \(adaptive thinking, effort “high”\)lg1retry0 / 0\.972\.037 \[\.013, \.066\]32\.972\.037 \[\.013, \.066\]2retry0 / 0\.966\.031 \[\.006, \.060\]32\.966\.031 \[\.006, \.060\]3retry0 / 0\.975\.040 \[\.014, \.068\]32\.975\.040 \[\.014, \.068\]anm1retry0 / 0\.906−\-\.000 \[−\-\.027, \.027\]32\.906−\-\.000 \[−\-\.027, \.027\]2retry0 / 0\.914\.008 \[−\-\.023, \.035\]32\.914\.008 \[−\-\.023, \.035\]3retry0 / 0\.912\.006 \[−\-\.025, \.037\]32\.912\.006 \[−\-\.025, \.037\]hetero1retry0 / 0\.934\.010 \[−\-\.012, \.031\]32\.934\.010 \[−\-\.012, \.031\]2retry0 / 0\.934\.009 \[−\-\.011, \.028\]32\.934\.009 \[−\-\.011, \.028\]3retry0 / 0\.930\.005 \[−\-\.021, \.028\]32\.930\.005 \[−\-\.021, \.028\]wild1retry0 / 0\.880\.060 \[\.030, \.090\]32\.880\.060 \[\.030, \.090\]2retry0 / 0\.898\.078 \[\.046, \.111\]32\.898\.078 \[\.046, \.111\]3retry0 / 0\.899\.079 \[\.048, \.111\]32\.899\.079 \[\.048, \.111\]mixed1retry0 / 0\.952\.025 \[−\-\.000, \.051\]32\.952\.025 \[−\-\.000, \.051\]gauntlet1retry0 / 0\.894\.046 \[\.027, \.063\]32\.894\.046 \[\.027, \.063\]Qwen 3\.8 Flash \(thinking mode\)lg1none2 / 0\.935−\-\.000 \[−\-\.046, \.039\]30\.966\.024 \[−\-\.003, \.052\]2once2 / 1\.944\.008 \[−\-\.043, \.050\]31\.962\.028 \[−\-\.002, \.059\]3retry2 / 2\.945\.009 \[−\-\.018, \.039\]32\.945\.009 \[−\-\.018, \.039\]anm1retry5 / 5\.899−\-\.008 \[−\-\.042, \.027\]32\.899−\-\.008 \[−\-\.042, \.027\]2retry2 / 2\.890−\-\.016 \[−\-\.047, \.015\]32\.890−\-\.016 \[−\-\.047, \.015\]3retry4 / 4\.881−\-\.026 \[−\-\.064, \.013\]32\.881−\-\.026 \[−\-\.064, \.013\]hetero1retry4 / 4\.903−\-\.022 \[−\-\.048, \.001\]32\.903−\-\.022 \[−\-\.048, \.001\]2retry5 / 5\.889−\-\.035 \[−\-\.071,−\-\.002\]32\.889−\-\.035 \[−\-\.071,−\-\.002\]3retry0 / 0\.880−\-\.044 \[−\-\.080,−\-\.014\]32\.880−\-\.044 \[−\-\.080,−\-\.014\]wild1none6 / 0\.773−\-\.047 \[−\-\.120, \.023\]26\.865\.044 \[\.007, \.079\]2once5 / 0\.782−\-\.038 \[−\-\.116, \.031\]27\.865\.049 \[\.016, \.077\]3retry2 / 2\.843\.023 \[−\-\.013, \.059\]32\.843\.023 \[−\-\.013, \.059\]mixed1retry9 / 9\.924−\-\.003 \[−\-\.033, \.028\]32\.924−\-\.003 \[−\-\.033, \.028\]gauntlet1retry5 / 5\.871\.023 \[−\-\.000, \.046\]32\.871\.023 \[−\-\.000, \.046\]Table 13:Per\-repeat results for the main agent condition\. F1 is adjacency F1;Δϕr\\Delta^\{\\mathrm\{r\}\}\_\{\\phi\}is the paired difference from the family’s best chosen algorithm\. Repeats 1 to 3 on four families and repeat 1 onmixedandgauntlet, the same for the three models\. Last turn: none, one turn in which the agent can only submit, or that turn with a retry\.