Causal Foundation Models

# Causal Foundation Models
Source: [https://arxiv.org/html/2609.03003](https://arxiv.org/html/2609.03003)
Christopher Stithchristopher@layer6\.ai††thanks:Equal contribution\.Hossein Rahmani11footnotemark:1hossein\.rahmani@td\.comAffiliation:TD Bank Group, Toronto, CanadaJesse C\. Cresswelljesse@layer6\.aiAffiliation:Layer 6 AI, Toronto, Canada

###### Abstract

Causal inference is the practice of estimating the effect of a treatment or intervention from data\. It traditionally requires a bespoke pipeline for every new problem: first proposing a causal mechanism, selecting a compatible estimator, and finally training it\. Meanwhile, across diverse settings and modalities, much of machine learning has shifted to the paradigm of foundation models: networks pretrained once at scale and applied to new tasks without fine\-tuning\.*Causal foundation models*\(CFMs\) bring this paradigm to causal inference\. CFMs are pretrained neural networks that estimate causal quantities, such as the average treatment effect, on entirely new datasets using in\-context learning without requiring model updates\. This work provides a practical introduction to this emerging area\. We summarize the necessary background in causal inference and machine learning before discussing CFMs\. Throughout, we include example code and Jupyter notebooks, which can be accessed by clicking on the[https://github.com/layer6ai-labs/cfms/blob/main/notebooks/Foundation_models_quickstart.ipynb](https://github.com/layer6ai-labs/cfms/blob/main/notebooks/Foundation_models_quickstart.ipynb)icons\. The full codebase is available at[github\.com/layer6ai\-labs/cfms](https://github.com/layer6ai-labs/cfms)\.

## 1Introduction

Causal inference is the practice of estimating causal effects between variables\([Pearl, 2009a](https://arxiv.org/html/2609.03003#bib.bib33);[Imbens and Rubin, 2015](https://arxiv.org/html/2609.03003#bib.bib13)\)\. For example: How effective is a medication at preventing a given disease? Did a policy cause a decrease in unemployment? If a central bank raises interest rates, what will the effect be on consumer spending? Causal inference attempts to answer these questions while avoiding the common pitfall of mistaking association with causation, captured in the adage that “correlation is not causation”\. Causal inference has applications across many domains, such as economics and policy\-making\([Athey and Imbens, 2017](https://arxiv.org/html/2609.03003#bib.bib39);[Chernozhukov et al\., 2018](https://arxiv.org/html/2609.03003#bib.bib6)\), marketing\([Bottou et al\., 2013](https://arxiv.org/html/2609.03003#bib.bib40);[Gordon et al\., 2019](https://arxiv.org/html/2609.03003#bib.bib41)\), and medicine\([Alaa and van der Schaar, 2017](https://arxiv.org/html/2609.03003#bib.bib35);[Shalit et al\., 2017](https://arxiv.org/html/2609.03003#bib.bib38)\)\.

The causal inference community has built a rich library of estimation methods, including Bayesian additive regression trees\([Chipman et al\., 2010](https://arxiv.org/html/2609.03003#bib.bib34)\), double machine learning\([Chernozhukov et al\., 2018](https://arxiv.org/html/2609.03003#bib.bib6)\), causal forests\([Wager and Athey, 2018](https://arxiv.org/html/2609.03003#bib.bib7)\), as well as S\-, T\-, and X\-learners\([Künzel et al\., 2019](https://arxiv.org/html/2609.03003#bib.bib66)\)\. To approach any given problem with most of these methods, one must first study the data and propose an underlying causal mechanism, choose an estimator that fits this mechanism, tune hyperparameters on validation data, and only then train the final estimator on the data\. For each new problem, the full pipeline must be repeated, with no opportunity to reuse tuned models or transfer knowledge between tasks\.

Recently,*causal foundation models*have emerged as a strong and efficient alternative\. These are pretrained neural networks that can be applied immediately to any causal inference task*without further training or fine\-tuning*\. The heart of causal foundation models \(CFMs\) lies at the intersection of causal inference and*learning\-to\-learn*, in which models learn the ability to predict causal effects in unseen settings from observational data\.111In machine learning, learning\-to\-learn is often called*meta\-learning*\([Finn et al\., 2017](https://arxiv.org/html/2609.03003#bib.bib42)\)\. However, meta\-learning has its own meaning in causal inference\([Künzel et al\., 2019](https://arxiv.org/html/2609.03003#bib.bib66);[Frauen et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib43)\), so we avoid using this term in this work\.They are trained on tasks sampled from a prior over possible data\-generating processes and causal mechanisms, learning to estimate causal effects in a wide variety of scenarios\. At inference time, they leverage in\-context learning on labeled examples to perform amortized Bayesian inference and make causal estimates \([Figure1](https://arxiv.org/html/2609.03003#S1.F1)\)\.

![Refer to caption](https://arxiv.org/html/2609.03003v1/images/example_pipeline.png)

model=CATEEstimator\(\)

model\.fit\(X\_ctx,T\_ctx,Y\_ctx\)

cate=model\.estimate\_cate\(X\_qry\)

Figure 1:Causal foundation models predict on each new dataset using in\-context learning without training or fine\-tuning\. Labeled context data \(X\_ctx, etc\.\) are provided as examples so that the model can predict causal effects on unlabeled query data \(X\_qry\)\. A quickstart example is available in this notebook:[https://github.com/layer6ai-labs/cfms/blob/main/notebooks/Foundation_models_quickstart.ipynb](https://github.com/layer6ai-labs/cfms/blob/main/notebooks/Foundation_models_quickstart.ipynb)\.In this work, we provide a practical and hands\-on introduction to CFMs\. Our main goal is to provide the reader with the information, tools, and examples they need to use CFMs in their own work\. We include example code and Jupyter notebooks throughout this paper to help the reader start using these models immediately\. These resources can be accessed by clicking on the[https://github.com/layer6ai-labs/cfms/tree/main](https://github.com/layer6ai-labs/cfms/tree/main)icons throughout\. We also release our full codebase at[github\.com/layer6ai\-labs/cfms](https://github.com/layer6ai-labs/cfms/tree/main)\.

CFMs have demonstrated top performance on causal inference tasks, and we expect that time will only demonstrate further transformational applications\. What is remarkable is that these models bring not only a vast increase in inference speed, but also improved performance\. This work exists to introduce CFMs to a wider audience, compare the various approaches that have been taken for CFM design, discuss the latest developments in the area, and invite researchers and practitioners to give them a try\.

We begin by discussing the necessary background material from causal inference and machine learning in[Section2](https://arxiv.org/html/2609.03003#S2)before jumping into the core material on CFMs in[Section3](https://arxiv.org/html/2609.03003#S3)\. In[Section4](https://arxiv.org/html/2609.03003#S4), we benchmark the first three openly available CFMs\([Robertson et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib2);[Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3);[Ma et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib4)\)together with popular traditional causal inference models\. In[Section5](https://arxiv.org/html/2609.03003#S5), we review the many new developments in the field which form promising areas for future growth, and applications of CFMs in other scientific areas\.

## 2Background

### 2\.1Why causal inference?

When predictive models deliver excellent results, why do we need causal machinery?[Mooij et al\. \(2016\)](https://arxiv.org/html/2609.03003#bib.bib16)give a classic illustration demonstrating the difference between*predictive*and*causal*inference\. Across European countries, regions with more storks also tend to have higher human birth rates\([Matthews, 2000](https://arxiv.org/html/2609.03003#bib.bib17)\)\. Consider a model trained purely to predict birth rate from stork population: it will exploit the correlation and predict well across the observed regions\. What might this model predict if we ask it “What would happen to the birth rate in a region if we doubled the stork population?” This is an*interventional*question, not merely a predictive one, for which correlational analysis is insufficient\. Failing to distinguish these subtleties would lead to a public policy disaster: the predictive model might recommend that in order to increase the human birth rate, a country should import more storks\!

In more detail, the predictive model has learned the*observational*relationship between stork population and birth rate, but the policy application requires knowledge of the interventional, or*causal*, relationship\. A predictive model answers the question “What outcome do we expect for this individual, as\-is?” A causal model answers the question “What outcome would we expect for this individual if we intervened?” The latter question requires reasoning about the mechanism generating the data, not just observing the data itself\. Depending on the underlying mechanism, these two questions can have different or identical answers\. Without additional assumptions, there is no way to tell from observational data alone which is the case\. In the following discussions, we formalize how one can answer this question\.

### 2\.2The potential outcomes framework for causal inference

We work in the Neyman\-Rubin potential outcomes framework\([Rubin, 2005](https://arxiv.org/html/2609.03003#bib.bib19);[Imbens and Rubin, 2015](https://arxiv.org/html/2609.03003#bib.bib13)\)\. We let capital letters denote random variables, and lower\-case letters denote values they can take\. We let𝒯⊂ℝ\\mathcal\{T\}\\subset\\mathbb\{R\}denote thetreatment spaceof interventions that can be applied\. There are three common options for𝒯\\mathcal\{T\}:

- •\(Binary treatment\)𝒯=\{0,1\}\\mathcal\{T\}=\\\{0,1\\\}, e\.g\. control group vs\. treatment group\.
- •\(Multi\-armed treatment\)𝒯=\{1,…,K\}\\mathcal\{T\}=\\\{1,\\ldots,K\\\}, e\.g\. a choice amongKKdifferent medical treatments\.
- •\(Continuous treatment\)𝒯\\mathcal\{T\}is an interval, often normalized to\[0,1\]\[0,1\]\. In a financial setting, this could denote a price or interest rate offered to a customer\.

We let𝒳\\mathcal\{X\}denote the covariate space and𝒴\\mathcal\{Y\}the outcome space \(often eitherℝ\\mathbb\{R\}or a categorical space\)\. For example,𝒳=ℕ×\[0,∞\)\\mathcal\{X\}=\\mathbb\{N\}\\times\[0,\\infty\)could denote the covariate space ofAge×Income\\text\{Age\}\\times\\text\{Income\}, and𝒴=\{0,1\}\\mathcal\{Y\}=\\\{0,1\\\}could denote whether or not the customer bought a product\.

The observed valuesX∈𝒳X\\in\\mathcal\{X\},T∈𝒯T\\in\\mathcal\{T\}, andY∈𝒴Y\\in\\mathcal\{Y\}are random variables\. If we are observing a population ofNNindividuals/units, we use subscripts likeXn,TnX\_\{n\},T\_\{n\}, andYnY\_\{n\}to denote individual\-level variables, i\.e\. the observed covariates, treatment, and outcome of thennth individual\.

The key component of the potential outcomes framework is to assume, for every treatment valuet∈𝒯t\\in\\mathcal\{T\}, the existence of a random variableY⁡\(t\)Y\(t\), which is the outcome which*would occur*under treatmentT=tT=t\. The variablesY⁡\(t\)Y\(t\)are referred to as*potential outcomes*\. On an individual level,Yn​\(t\)Y\_\{n\}\(t\)denotes the potential outcome for individualnnif they were to receive treatmenttt\.

We assume that for allt∈𝒯t\\in\\mathcal\{T\},

T=t⟹Y=Y⁡\(t\)\.T=t\\implies Y=Y\(t\)\.\(1\)This is known as the*consistency assumption*\([Neal, 2020](https://arxiv.org/html/2609.03003#bib.bib5)\)\. It states that the potential outcomes must agree with what we actually observe\. One can imagine the existence of an omniscient oracle that knows every potential outcome for every individual; the consistency assumption states that the oracle’s knowledge agrees with whatever happens in the real world when a given treatment is applied\.

Unfortunately, we are not omniscient oracles\. We can never know all potential outcomes for a given individual\. Once individualnnis given treatmenttt, we can never know what would have happened ift′≠tt^\{\\prime\}\\neq thad been administered\. This is known as the fundamental problem of causal inference\([Holland, 1986](https://arxiv.org/html/2609.03003#bib.bib28)\)\.

Fundamental Problem of Causal InferenceIf an individual’s observed treatment and outcome arettandyy, respectively, then it is impossible to knowY⁡\(t′\)Y\(t^\{\\prime\}\)fort′≠tt^\{\\prime\}\\neq t\.

The observed treatmentT=tT=tand outcomeY=Y⁡\(t\)Y=Y\(t\)are called the*factual*treatment and outcome\. Any hypothetical alternatives liket′t^\{\\prime\}andY⁡\(t′\)Y\(t^\{\\prime\}\)are called*counterfactual*treatments and outcomes\. Note that even with complete knowledge of every covariate,Y⁡\(t\)Y\(t\)need not be deterministic; exogenous noise means potential outcomes can vary across individuals that share identical covariatesXX\. This is why causal quantities of interest are typically expectations or distributions rather than pointwise counterfactuals\.

Instead of always working directly with potential outcomes, different applications call for different summary statistics\. Hence, there are several important*causal estimands*we consider:222We return to a general definition of causal estimands in[Section2\.3](https://arxiv.org/html/2609.03003#S2.SS3)\.

- •Theconditional expected potential outcome\(CEPO\) over a population of individuals that share covariatesxx, and are given the same treatmenttt, is μt​\(x\)≔𝔼⁡\[Y⁡\(t\)∣X=x\]\.\\mu\_\{t\}\(x\)\\coloneqq\\mathbb\{E\}\[Y\(t\)\\mid X=x\]\.\(2\)The CEPOμt​\(x\)\\mu\_\{t\}\(x\)is thus the expected outcome of administering treatmentttto an individual with covariatesX=xX=x\. CEPOs are useful for comparing between different treatments, which brings us to the next item\.
- •In the binary treatment setting, theaverage treatment effect\(ATE\) is ATE≔𝔼⁡\[Y⁡\(1\)−Y⁡\(0\)\],\\text\{ATE\}\\coloneqq\\mathbb\{E\}\[Y\(1\)\-Y\(0\)\],\(3\)where the expectation is overXX\. The ATE tells us how much the outcome would change on average were the treatmentt=1t=1applied to the entire population, compared to the baselinet=0t=0\. Notice that the ATE can be expressed in terms of CEPOs, using linearity of expectations, and the law of iterated expectations: ATE=𝔼⁡\[Y⁡\(1\)\]−𝔼⁡\[Y⁡\(0\)\]=𝔼⁡\[𝔼⁡\[Y⁡\(1\)∣X=x\]\]−𝔼⁡\[𝔼⁡\[Y⁡\(0\)∣X=x\]\]=𝔼⁡\[μ1​\(x\)\]−𝔼⁡\[μ0​\(x\)\]=𝔼⁡\[μ1​\(x\)−μ0​\(x\)\]\.\\displaystyle\\begin\{aligned\} \\text\{ATE\}&=\\mathbb\{E\}\[Y\(1\)\]\-\\mathbb\{E\}\[Y\(0\)\]\\\\ &=\\mathbb\{E\}\[\\mathbb\{E\}\[Y\(1\)\\mid X=x\]\]\-\\mathbb\{E\}\[\\mathbb\{E\}\[Y\(0\)\\mid X=x\]\]\\\\ &=\\mathbb\{E\}\[\\mu\_\{1\}\(x\)\]\-\\mathbb\{E\}\[\\mu\_\{0\}\(x\)\]=\\mathbb\{E\}\[\\mu\_\{1\}\(x\)\-\\mu\_\{0\}\(x\)\]\.\\end\{aligned\}\(4\)
- •Only looking at the ATE can hide strong effects if some parts of the population are affected differently than others\. Hence, we also consider theconditional average treatment effect\(CATE\) over a population of individuals that share covariatesxx, CATE​\(x\)≔𝔼⁡\[Y⁡\(1\)−Y⁡\(0\)∣X=x\]=μ1​\(x\)−μ0​\(x\)\.\\text\{CATE\}\(x\)\\coloneqq\\mathbb\{E\}\[Y\(1\)\-Y\(0\)\\mid X=x\]=\\mu\_\{1\}\(x\)\-\\mu\_\{0\}\(x\)\.\(5\)From this definition, we haveATE=𝔼​\[CATE​\(x\)\]\\text\{ATE\}=\\mathbb\{E\}\[\\text\{CATE\}\(x\)\]\. The CATE tells us how large of a causal effect we expect to see applying treatmentt=1t=1to a random individual with observed covariatesX=xX=x\.
- •In the continuous treatment setting, theindividual treatment\-response curve\(ITRC\) for an individual with covariatesxxis the function t↦μt​\(x\)t∈𝒯\.t\\mapsto\\mu\_\{t\}\(x\)\\qquad t\\in\\mathcal\{T\}\.\(6\)For any value of the treatment, the ITRC maps out the expected outcome for a random individual with covariatesX=xX=x\. The continuous treatment setting lends itself to optimization, where we can ask “what treatment level maximizes the response?” The ITRC is also referred to as the individual or conditional*dose\-response curve*in medical settings\([Schwab et al\., 2020](https://arxiv.org/html/2609.03003#bib.bib31)\)\.

### 2\.3Data\-generating processes and identifiability

LetP=P⁡\(X,T,\{Y⁡\(t\)\}t∈𝒯,Y\)P=P\(X,T,\\\{Y\(t\)\\\}\_\{t\\in\\mathcal\{T\}\},Y\)denote the joint distribution of the observed covariates, treatment, potential outcomes, and factual outcome\. The distributionPPis induced by a*data\-generating process*\(DGP\) which specifies how the variables are generated, for example as

X∼PX,T\|X∼PT\|X,\{Y⁡\(t\)\}t∈𝒯\|X∼P\{Y⁡\(t\)\}t∈𝒯\|X,Y=Y⁡\(T\)\.\\displaystyle\\begin\{aligned\} X&\\sim P\_\{X\},\\\\ T\\mid X&\\sim P\_\{T\\mid X\},\\\\ \\\{Y\(t\)\\\}\_\{t\\in\\mathcal\{T\}\}\\mid X&\\sim P\_\{\\\{Y\(t\)\\\}\_\{t\\in\\mathcal\{T\}\}\\mid X\},\\\\ Y&=Y\(T\)\.\\end\{aligned\}\(7\)While the distributionPPis fully characterized by the DGP that induces it,PPis generally unknown in real world settings\. Instead, what we have access to in practice are samples from the*observational distribution*PobsP\_\{\\text\{obs\}\}\. This is defined as the marginal distribution of\(X,T,Y\)\(X,T,Y\)underPP, since only the factual outcome, rather than all potential outcomes, can actually be observed:

Pobs​\(X,T,Y\)≔P⁡\(X,T,Y\)\.P\_\{\\text\{obs\}\}\(X,T,Y\)\\coloneqq P\(X,T,Y\)\.\(8\)AlthoughPPis unknowable in practice, it is a useful abstraction since perfect knowledge ofPPallows us to compute CEPOs, and thus our other causal estimands of interest\. To make this connection, we introduce the do\-operatordo⁡\(T=t\)\\operatorname\{do\}\(T=t\)\([Pearl, 2009b](https://arxiv.org/html/2609.03003#bib.bib15)\)which represents an intervention that setsTTtott, as opposed to simply observingttpassively\. The*interventional distribution*P⁡\(Y∣do⁡\(T=t\)\)P\(Y\\mid\\operatorname\{do\}\(T=t\)\)is generally not the same as the observational distributionPobs​\(Y∣T=t\)P\_\{\\text\{obs\}\}\(Y\\mid T=t\), because covariatesXXor exogenous variables can systematically influence which treatments are observed\. For example, ifXXrepresents the risk level of a patient, andt=1t=1indicates a more intensive treatment, doctors may only prescribet=1t=1for high risk patients\. In contrast,do⁡\(T=1\)\\operatorname\{do\}\(T=1\)forces the intensive treatment, even if the patient is low\-risk and doctors would normally not prescribe this treatment\. Hence, intervening on treatments without regard to covariates also changes the distribution of outcomesYY\.

Although the interventional distribution is not the same asPobs​\(Y∣T=t\)P\_\{\\text\{obs\}\}\(Y\\mid T=t\), it is related toPP\. Under the interventiondo⁡\(T=t\)\\operatorname\{do\}\(T=t\)the treatment is set tott, and by the consistency assumption the resulting outcome isY⁡\(t\)Y\(t\), which means

P⁡\(Y∣do⁡\(T=t\)\)=P⁡\(Y⁡\(t\)\)\.P\(Y\\mid\\operatorname\{do\}\(T=t\)\)=P\(Y\(t\)\)\.\(9\)[Equation9](https://arxiv.org/html/2609.03003#S2.E9)shows that the interventional distribution contains rich information on the potential outcomes\. By conditioning onXXwe get the*conditional interventional distribution*\(CID\)P⁡\(Y∣do⁡\(T=t\),X=x\)P\(Y\\mid\\operatorname\{do\}\(T=t\),X=x\), which directly gives the CEPO \([Equation2](https://arxiv.org/html/2609.03003#S2.E2)\) via its mean:

∫ydP\(Y=y∣do\(T=t\),X=x\)=𝔼\[Y∣do\(T=t\),X=x\]=𝔼\[Y\(t\)∣X=x\]=μt\(x\)\.\\int y\\,\\text\{d\}P\(Y=y\\mid\\operatorname\{do\}\(T=t\),X=x\)=\\mathbb\{E\}\[Y\\mid\\operatorname\{do\}\(T=t\),X=x\]=\\mathbb\{E\}\[Y\(t\)\\mid X=x\]=\\mu\_\{t\}\(x\)\.\(10\)If we instead consider the difference in potential outcomesY⁡\(1\)−Y⁡\(0\)Y\(1\)\-Y\(0\), we obtain the*conditional distribution of treatment effects*\(CDTE\),P⁡\(Y⁡\(1\)−Y⁡\(0\)∣X=x\)P\(Y\(1\)\-Y\(0\)\\mid X=x\)\. In an analogous manner to[Equation10](https://arxiv.org/html/2609.03003#S2.E10), the CDTE directly gives the CATE via its mean:

∫Δ​d​P​\(Y⁡\(1\)−Y⁡\(0\)=Δ∣X=x\)=E⁡\[Y⁡\(1\)−Y⁡\(0\)∣X=x\]=CATE⁡\(x\),\\int\\Delta\\,\\text\{d\}P\(Y\(1\)\-Y\(0\)=\\Delta\\mid X=x\)=E\[Y\(1\)\-Y\(0\)\\mid X=x\]=\\operatorname\{CATE\}\(x\),\(11\)whereΔ\\Deltadenotes a realization of the random variableY⁡\(1\)−Y⁡\(0\)Y\(1\)\-Y\(0\)\.

Note that the joint distributionPPor even the CID alone is enough to compute the desirable causal estimands from[Section2\.2](https://arxiv.org/html/2609.03003#S2.SS2)\. One can sample from the CID if interventions are possible, but often this is not the case\. Interventions may be expensive, impractical, or unethical, like giving a patient a treatment which will probably harm them\. More often than not we only have observational data, namely samples fromPobsP\_\{\\text\{obs\}\}\.

Since marginalizing a probability distribution as in[eq\.8](https://arxiv.org/html/2609.03003#S2.E8)loses information, there are in general many different joint distributionsPPwhich could give rise to the same observational distributionPobsP\_\{\\text\{obs\}\}\. This is linked to the[Fundamental Problem of Causal Inference](https://arxiv.org/html/2609.03003#fundamental_problem_of_causal_inference), now seen as a problem of*identifiability*: understanding the observational distribution is not always enough to identify which DGP it actually derives from\.

Under what conditions*can*we identify a causal effect like the ATE, which is a function ofPP, given access only toPobsP\_\{\\text\{obs\}\}? To make this question mathematically precise, let𝒫DGP\\mathcal\{P\}\_\{\\text\{DGP\}\}denote the set of joint probability distributions over\(X,T,\{Y⁡\(t\)\}t∈𝒯,Y\)\(X,T,\\\{Y\(t\)\\\}\_\{t\\in\\mathcal\{T\}\},Y\), deriving from all DGPs consistent with those variables\. Define an equivalence relation on𝒫DGP\\mathcal\{P\}\_\{\\text\{DGP\}\}by

P1∼P2⇔\(P1\)obs=\(P2\)obs,∀P1,P2∈𝒫DGP,P\_\{1\}\\sim P\_\{2\}\\iff\(P\_\{1\}\)\_\{\\text\{obs\}\}=\(P\_\{2\}\)\_\{\\text\{obs\}\},\\qquad\\forall P\_\{1\},P\_\{2\}\\in\\mathcal\{P\}\_\{\\text\{DGP\}\},\(12\)which forms*observational equivalence classes*\. Two interventional distributions are*observationally equivalent*if they lie in the same observational equivalence class\. Practically,P1∼P2P\_\{1\}\\sim P\_\{2\}means that these distributions are indistinguishable given only observational data—no matter how much data is available\.

We define a*causal estimand*to be a functionalg:𝒫DGP→ℝg:\\mathcal\{P\}\_\{\\text\{DGP\}\}\\to\\mathbb\{R\}\. For example, the ATE \([Equation3](https://arxiv.org/html/2609.03003#S2.E3)\) is a causal estimand since it is a function ofPP, via potential outcomes, and outputs a scalar\.

###### Definition 1\.

Let𝒫⊂𝒫DGP\\mathcal\{P\}\\subset\\mathcal\{P\}\_\{\\text\{DGP\}\}\. A causal estimandggisidentifiable given𝒫\\mathcal\{P\}if it is constant on observational equivalence classes of𝒫\\mathcal\{P\}, i\.e\. for anyP1,P2∈𝒫P\_\{1\},P\_\{2\}\\in\\mathcal\{P\},

\(P1\)obs=\(P2\)obs⟹g⁡\(P1\)=g⁡\(P2\)\.\(P\_\{1\}\)\_\{\\text\{obs\}\}=\(P\_\{2\}\)\_\{\\text\{obs\}\}\\implies g\(P\_\{1\}\)=g\(P\_\{2\}\)\.\(13\)

In other words,ggis identifiable exactly when it can be written as a function ofPobsP\_\{\\text\{obs\}\}alone\. The subset𝒫\\mathcal\{P\}is often clear from context \(see below\), in which case we say simply thatggis identifiable, without explicitly referencing𝒫\\mathcal\{P\}\. Note also that identifiability does not imply thatggis computable given a finite dataset sampled fromPobsP\_\{\\text\{obs\}\}\.

Without additional assumptions, the causal estimands we care about from[Section2\.2](https://arxiv.org/html/2609.03003#S2.SS2)are not identifiable given𝒫DGP\\mathcal\{P\}\_\{\\text\{DGP\}\}\. The set of possible distributions𝒫DGP\\mathcal\{P\}\_\{\\text\{DGP\}\}is simply too large to ensure constancy over all thePobsP\_\{\\text\{obs\}\}derived fromP∈𝒫DGPP\\in\\mathcal\{P\}\_\{\\text\{DGP\}\}\. Practitioners must narrow down the allowable set to some𝒫⊂𝒫DGP\\mathcal\{P\}\\subset\\mathcal\{P\}\_\{\\text\{DGP\}\}by making certain assumptions about what DGPs could exist\. In this work, we focus on a particular set of identification assumptions commonly used in causal inference\. More generally, Pearl’s do\-calculus\([Pearl, 1995](https://arxiv.org/html/2609.03003#bib.bib20)\)gives a complete set of graph\-theoretic rules for identification: if a causal estimand is identifiable, do\-calculus provides a method for calculating it in terms ofPobsP\_\{\\text\{obs\}\}\([Shpitser and Pearl, 2006](https://arxiv.org/html/2609.03003#bib.bib22);[Huang and Valtorta, 2006](https://arxiv.org/html/2609.03003#bib.bib21)\)\.

The most common set of assumptions used in the field of causal inference in order to guarantee identifiability are the following\([Imbens and Rubin, 2015](https://arxiv.org/html/2609.03003#bib.bib13);[Neal, 2020](https://arxiv.org/html/2609.03003#bib.bib5);[Yao et al\., 2021](https://arxiv.org/html/2609.03003#bib.bib23);[Balazadeh et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib29)\)\.

###### Definition 2\.

A DGP satisfies theignorabilityassumption if for allt∈𝒯t\\in\\mathcal\{T\},

Y⁡\(t\)⟂⁣⟂T\|X\.Y\(t\)\\mathrel\{\\perp\\\!\\\!\\\!\\perp\}T\\mid X\.\(14\)This is also called the \(conditional\) unconfoundedness assumption\. It says that conditional onXX, treatment assignment is independent of the potential outcomes\. Individuals who actually received treatmentttare representative, with respect toY⁡\(t\)Y\(t\), of individuals who could have received treatmenttt\. This allows us to identify the distribution of potential outcomes from observed factual outcomes because ignorability implies

P⁡\(Y⁡\(t\)∣X=x\)=P⁡\(Y∣T=t,X=x\)\.P\(Y\(t\)\\mid X=x\)=P\(Y\\mid T=t,X=x\)\.\(15\)

###### Definition 3\.

In the binary or multi\-armed treatment case, a DGP satisfies thepositivityassumption if

P⁡\(T=t∣X=x\)\>0∀t∈𝒯,x∈𝒳\.P\(T=t\\mid X=x\)\>0\\qquad\\forall t\\in\\mathcal\{T\},x\\in\\mathcal\{X\}\.\(16\)In the continuous treatment case, a DGP satisfies the positivity assumption if the conditional density functionppforPPsatisfies

p⁡\(T=t∣X=x\)\>0almost everywhere\.p\(T=t\\mid X=x\)\>0\\qquad\\text\{almost everywhere\}\.\(17\)The positivity assumption is also called the overlap assumption\. It is needed because we can only learn the causal effect of a treatment for covariate values where that treatment can actually be observed\. For example, if we assume ignorability, then by[eq\.15](https://arxiv.org/html/2609.03003#S2.E15)the quantity𝔼⁡\[Y⁡\(t\)∣X=x\]\\mathbb\{E\}\[Y\(t\)\\mid X=x\]is equal to𝔼\[Y∣T=t,X=x\]\\mathbb\{E\}\[Y\\mid T=t,X=x\], but ifP⁡\(T=t∣X=x\)=0P\(T=t\\mid X=x\)=0there will be no observations ofYYfor individuals with covariatesX=xX=xwho receive treatmentT=tT=t\.

If a DGP satisfies both positivity and ignorability, it is said to satisfy*strong ignorability*\.

###### Definition 4\.

A DGP satisfies thestable unit treatment value assumption\(SUTVA\) if:

1. 1\.\(No interference\) The individual\-level potential outcomeYn​\(tn\)Y\_\{n\}\(t\_\{n\}\)for individualnndoes not depend on the treatment assigned to any other individualmm, and;
2. 2\.\(No hidden versions of treatments\) The treatmentttis a well\-defined intervention without multiple versions that have different effects\.

Without the first part of the SUTVA,Yn​\(tn\)Y\_\{n\}\(t\_\{n\}\)is not well\-defined fromtnt\_\{n\}alone;Yn​\(tn\)Y\_\{n\}\(t\_\{n\}\)would instead need to express dependence on other treatment assignments asYn​\(t1,…,tn,…,tN\)Y\_\{n\}\(t\_\{1\},\.\.\.,t\_\{n\},\.\.\.,t\_\{N\}\)\. No interference impliesYn​\(t1,…,tn,…,tN\)=Yn​\(tn\)Y\_\{n\}\(t\_\{1\},\.\.\.,t\_\{n\},\.\.\.,t\_\{N\}\)=Y\_\{n\}\(t\_\{n\}\)\. Without the second part there are different versions oftnt\_\{n\}that produce different outcomes, soYn​\(tn\)Y\_\{n\}\(t\_\{n\}\)does not uniquely specify an outcome\. The SUTVA ensures that the potential outcomeYn​\(tn\)Y\_\{n\}\(t\_\{n\}\)is a well\-defined object which can then be linked to the corresponding factual outcomeYnY\_\{n\}undertnt\_\{n\}via consistency\.

Together, these three assumptions guarantee that the CID \(and hence the CEPO\) is identifiable, and can be recovered fromPobsP\_\{\\text\{obs\}\}via the*backdoor adjustment*\([Pearl, 2009b](https://arxiv.org/html/2609.03003#bib.bib15);[Neal, 2020](https://arxiv.org/html/2609.03003#bib.bib5)\)

P⁡\(Y∣do⁡\(T=t\)\)=∫P⁡\(Y∣T=t,X=x\)​P​\(X=x\)​d​x\.P\(Y\\mid\\operatorname\{do\}\(T=t\)\)=\\int P\(Y\\mid T=t,X=x\)P\(X=x\)\\,\\text\{d\}x\.\(18\)Accordingly, the three assumptions together are commonly referred to as the backdoor setting\. In terms of the mathematical framework discussed above, let𝒫back⊂𝒫DGP\\mathcal\{P\}\_\{\\text\{back\}\}\\subset\\mathcal\{P\}\_\{\\text\{DGP\}\}be defined by

𝒫back=\{P∈𝒫DGP∣P​satisfies strong ignorability and SUTVA\}\.\\mathcal\{P\}\_\{\\text\{back\}\}=\\big\\\{P\\in\\mathcal\{P\}\_\{\\text\{DGP\}\}\\mid P\\ \\text\{satisfies strong ignorability and SUTVA\}\\big\\\}\.\(19\)Then the CID and CEPO are identifiable given𝒫back\\mathcal\{P\}\_\{\\text\{back\}\}\. The main drawback of the backdoor setting is that ignorability is an untestable assumption in practice\([Pearl, 2009b](https://arxiv.org/html/2609.03003#bib.bib15)\)\.

A well\-known setting in which we have partial identifiability is the*instrumental variable*\(IV\) setting\([Manski, 2003](https://arxiv.org/html/2609.03003#bib.bib30)\)\.333The IV setting also implies*parametric*identifiability; that is, identifiability when restricting to a narrow subclass of interventional distributions, e\.g\. those arising from linear models\([Neal, 2020](https://arxiv.org/html/2609.03003#bib.bib5)\)\.While we do not give a formal definition here, partial identifiability occurs when the causal estimand of interest cannot be determined exactly, but its range is reduced to a finite interval\. An instrumental variableZZis one which has a causal effect on the outcomeYYwhich is fully mediated by the treatmentTT, and for which there are no unobserved confounders betweenZZandTTorYY\([Neal, 2020](https://arxiv.org/html/2609.03003#bib.bib5), Chapter 9\)\.

Finally, we mention that the*frontdoor setting*is another set of assumptions for which identification holds\([Neal, 2020](https://arxiv.org/html/2609.03003#bib.bib5), Chapter 6\)\. However, this setting is much less used in practice\([Imbens, 2020](https://arxiv.org/html/2609.03003#bib.bib32)\)\.

### 2\.4Bayesian networks and structural causal models

We have discussed data\-generating processes and identifiability of causal effects\. In this section, we discuss a well\-known graph\-theoretic framework for quantitatively describing DGPs\. This framework also plays a key role in specifying priors for CFM training \([Section3\.2](https://arxiv.org/html/2609.03003#S3.SS2)\)\.

LetGGbe a directed acyclic graph \(DAG\)\. For a nodev∈Gv\\in G, we letpa​\(v\)\\text\{pa\}\(v\)denote the set of all parents ofvv, that is, the set of all nodesu∈Gu\\in Gfor which there is an edgeu→vu\\to v\. When the nodes of a DAG represent random variables and the edges represent dependence relationships between these random variables, we call the DAG a*Bayesian network*\([Pearl, 2009b](https://arxiv.org/html/2609.03003#bib.bib15)\)\. In this work, we mainly consider the case where edges represent direct causal relationships, in which case the Bayesian network is sometimes referred to as a*causal network*\.

Bayesian networks are useful for describing conditional dependencies between random variables\. IfZ1,…,ZKZ\_\{1\},\\ldots,Z\_\{K\}are random variables, we are often interested in the joint probability distribution

P⁡\(Z1,…,ZK\)\\displaystyle P\(Z\_\{1\},\\ldots,Z\_\{K\}\)=P⁡\(Z1\)​∏k=2KP⁡\(Zk∣Z1,…,Zk−1\)\.\\displaystyle=P\(Z\_\{1\}\)\\prod\_\{k=2\}^\{K\}P\(Z\_\{k\}\\mid Z\_\{1\},\\ldots,Z\_\{k\-1\}\)\.Bayesian networks let us factor the right\-hand side into a much sparser product by conditioning each variable only on its parents:

P⁡\(Z1,…,ZK\)=∏kP⁡\(Zk∣pa​\(Zk\)\)\.P\(Z\_\{1\},\\ldots,Z\_\{K\}\)=\\prod\_\{k\}P\\big\(Z\_\{k\}\\mid\\text\{pa\}\(Z\_\{k\}\)\\big\)\.\(20\)Bayesian networks describe dependencies or sometimes causal relationships between variables, but not how to compute causal effects\. Augmenting a Bayesian network with explicit equations relating each node gives a model that can be used to simulate data and perform interventions on\.

###### Definition 5\.

Astructural causal model\(SCM\) is a Bayesian networkGGofKKnodes, together with a set of*exogenous variables*\{Uk\}k=1K\\\{U\_\{k\}\\\}\_\{k=1\}^\{K\}and a set of functions\{fv\}v∈G\\\{f\_\{v\}\\\}\_\{v\\in G\}such that

Zk=fZk​\(pa​\(Zk\),Uk\)∀k=1,…,K\.Z\_\{k\}=f\_\{Z\_\{k\}\}\(\\text\{pa\}\(Z\_\{k\}\),U\_\{k\}\)\\qquad\\forall\\ k=1,\\ldots,K\.\(21\)

The variablesZkZ\_\{k\}are called*endogenous variables*and represent the modeled set of variables \(including covariatesXX, treatmentsTT, and potential outcomesYY\), whereas theUkU\_\{k\}represent unmodeled factors or noise\. The equations[21](https://arxiv.org/html/2609.03003#S2.E21)are called the*structural equations*of the SCM; see[Example2\.1](https://arxiv.org/html/2609.03003#S2.Thmexample1)\. Note that the structural equations only model the endogenous variablesZkZ\_\{k\}, not the exogenousUkU\_\{k\}\. We will denote SCMs using the letterSS\.

Given a realization of the exogenous variablesUkU\_\{k\}, we can simulate the entire system by passing these values into the structural equations, starting with nodes in the graph that do not have parents and progressing according to the DAG structure\. We typically posit a distributionPUP\_\{U\}over the exogenous variables, allowing us to sampleUk∼PUU\_\{k\}\\sim P\_\{U\}, and then apply the structural equations to simulate observational data\. SCMs also enable us to model factual, counterfactual, and interventional scenarios\. For example, the interventiondo⁡\(T=t\)\\operatorname\{do\}\(T=t\)corresponds to replacing the structural equation

T\\displaystyle T=fT​\(pa​\(T\),UT\)\\displaystyle=f\_\{T\}\(\\text\{pa\}\(T\),U\_\{T\}\)withT=tT=t\. We can then sampleUk∼PUU\_\{k\}\\sim P\_\{U\}to approximate the CIDP⁡\(Y∣do⁡\(T=t\),X=x\)P\(Y\\mid\\operatorname\{do\}\(T=t\),X=x\), or fix the values of eachUkU\_\{k\}while intervening onTTto generate counterfactualsY⁡\(t\)Y\(t\)\. SCMs are therefore examples of DGPs; we will see the data\-generative capabilities of SCMs used to great effect in[Section3](https://arxiv.org/html/2609.03003#S3)\.

###### Example 2\.1\.

Consider endogenous variables\{X,T,Y\}\\\{X,T,Y\\\}, exogenous variables\{U1,U2,U3\}\\\{U\_\{1\},U\_\{2\},U\_\{3\}\\\}, and structural equations

x\\displaystyle x=fX​\(u1\)≔u1,\\displaystyle=f\_\{X\}\(u\_\{1\}\)\\coloneqq u\_\{1\},t\\displaystyle t=fT​\(x,u2\)≔x2\+1\+u2,\\displaystyle=f\_\{T\}\(x,u\_\{2\}\)\\coloneqq x^\{2\}\+1\+u\_\{2\},y\\displaystyle y=fY​\(x,t,u3\)≔t−x\+u3,\\displaystyle=f\_\{Y\}\(x,t,u\_\{3\}\)\\coloneqq t\-x\+u\_\{3\},withU1,U2,U3​∼i\.i\.d\.​𝒩​\(0,1\)U\_\{1\},U\_\{2\},U\_\{3\}\\overset\{\\text\{i\.i\.d\.\}\}\{\\sim\}\\mathcal\{N\}\(0,1\)\. The corresponding DAG is shown in[Figure2](https://arxiv.org/html/2609.03003#S2.F2)\. Samplingu1,u2,u3u\_\{1\},u\_\{2\},u\_\{3\}and propagating throughfX,fT,fYf\_\{X\},f\_\{T\},f\_\{Y\}generates i\.i\.d\. draws from the observational distributionPobs​\(X,T,Y\)P\_\{\\text\{obs\}\}\(X,T,Y\)\. To generate interventional data, we simply replacefTf\_\{T\}with a fixed constantt0t\_\{0\}during the propagation, simulating the effect of the interventiondo⁡\(T=t0\)\\operatorname\{do\}\(T=t\_\{0\}\)\.

XXTTYYU1U\_\{1\}U2U\_\{2\}U3U\_\{3\}Figure 2:An example DAG arising from the SCM given in Example[2\.1](https://arxiv.org/html/2609.03003#S2.Thmexample1), with endogenous variables\{X,T,Y\}\\\{X,T,Y\\\}and exogenous variables\{U1,U2,U3\}\\\{U\_\{1\},U\_\{2\},U\_\{3\}\\\}\. Dashed nodes denote exogenous variables\.
### 2\.5Bayesian inference and posterior predictive distributions

Let us revisit the set𝒫DGP\\mathcal\{P\}\_\{\\text\{DGP\}\}of joint probability distributions from[Section2\.3](https://arxiv.org/html/2609.03003#S2.SS3), and denote a specific DGP by a parameterψ\\psi, writingPψ∈𝒫DGPP^\{\\psi\}\\in\\mathcal\{P\}\_\{\\text\{DGP\}\}for its joint distribution with observational marginalPobsψP^\{\\psi\}\_\{\\text\{obs\}\}\. Observational datasets𝒟obs=\{\(xn,tn,yn\)\}n=1N\\mathcal\{D\}\_\{\\text\{obs\}\}=\\\{\(x\_\{n\},t\_\{n\},y\_\{n\}\)\\\}\_\{n=1\}^\{N\}are drawn i\.i\.d\. fromPobsψP^\{\\psi\}\_\{\\text\{obs\}\}\.

Given only observational data𝒟obs\\mathcal\{D\}\_\{\\text\{obs\}\}, we would like to estimate a causal estimandgg\. As we learned in[Section2\.3](https://arxiv.org/html/2609.03003#S2.SS3), causal estimands can be computed fromPψP^\{\\psi\}, but require identifiability conditions to be computable fromPobsψP^\{\\psi\}\_\{\\text\{obs\}\}\. Even under identifiability conditions, the finite sample available in𝒟obs\\mathcal\{D\}\_\{\\text\{obs\}\}leaves uncertainty about which DGP the data came from\. Hence, estimatingggneeds to incorporate information about several possible DGPs\. The standard way to proceed via Bayesian inference is to assume a priorπ⁡\(ψ\)\\pi\(\\psi\)over candidate DGPs and use the observational dataset𝒟obs\\mathcal\{D\}\_\{\\text\{obs\}\}to refine that distribution into a posteriorπ⁡\(ψ∣𝒟obs\)\\pi\(\\psi\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\)via Bayes’ rule\. We can then express our knowledge of the causal estimandggthrough its*posterior predictive distribution*\(PPD\), defined as

πg​\(\[a,b\]∣𝒟obs\)=∫π⁡\(ψ∣𝒟obs\)​𝟏g⁡\(Pψ\)∈\[a,b\]​d​ψ,\\pi^\{g\}\(\[a,b\]\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\)=\\int\\pi\(\\psi\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\)\\bm\{1\}\_\{g\(P^\{\\psi\}\)\\in\[a,b\]\}\\,\\text\{d\}\\psi,\(22\)for anya,b∈ℝa,b\\in\\mathbb\{R\}witha≤ba\\leq b\. The PPD expresses our uncertainty about the value ofgg, including from observing a finite sample of data, and from our uncertainty about which DGP generated the data\. For any intervalI⊂ℝI\\subset\\mathbb\{R\}, for example\[−1,1\]\[\-1,1\], this distribution gives us the posterior probability that the value ofggis inII\. For anyψ\\psi,g⁡\(Pψ\)g\(P^\{\\psi\}\)takes on a single value; for anyII,[eq\.22](https://arxiv.org/html/2609.03003#S2.E22)adds up the posterior probability of all DGPsψ\\psifor which the valueg⁡\(Pψ\)g\(P^\{\\psi\}\)is consistent with the specified rangeII\.

###### Example 2\.2\.

Wheng⁡\(Pψ\)g\(P^\{\\psi\}\)is the CEPOμt​\(x,Pψ\)=𝔼Pψ​\[Y⁡\(t\)∣X=x\]\\mu\_\{t\}\(x;P^\{\\psi\}\)=\\mathbb\{E\}\_\{P^\{\\psi\}\}\[Y\(t\)\\mid X=x\], the CEPO\-PPD is

πμt​\(\[a,b\]∣x,𝒟obs\)=∫π⁡\(ψ∣𝒟obs\)​𝟏μt​\(x,Pψ\)∈\[a,b\]​d​ψ\.\\pi^\{\\mu\_\{t\}\}\(\[a,b\]\\mid x,\\mathcal\{D\}\_\{\\text\{obs\}\}\)=\\int\\pi\(\\psi\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\)\\bm\{1\}\_\{\\mu\_\{t\}\(x;P^\{\\psi\}\)\\in\[a,b\]\}\\,\\text\{d\}\\psi\.

###### Example 2\.3\.

Wheng⁡\(Pψ\)g\(P^\{\\psi\}\)is the ATE, given by𝔼Pψ​\[Y⁡\(1\)−Y⁡\(0\)\]\\mathbb\{E\}\_\{P^\{\\psi\}\}\[Y\(1\)\-Y\(0\)\], the ATE\-PPD is

πATE​\(\[a,b\]∣𝒟obs\)=∫π⁡\(ψ∣𝒟obs\)​𝟏ATE​\(Pψ\)∈\[a,b\]​d​ψ\.\\pi^\{\\text\{ATE\}\}\(\[a,b\]\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\)=\\int\\pi\(\\psi\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\)\\bm\{1\}\_\{\\text\{ATE\}\(P^\{\\psi\}\)\\in\[a,b\]\}\\,\\text\{d\}\\psi\.

Sometimes directly obtaining causal estimands is not the goal, and more benefit is gained by modeling the CID itself\. The CID can also be formulated as a PPD to enable its estimation from observational data\. Once again we assume a prior over DGPs which becomes a posterior after observing data𝒟obs\\mathcal\{D\}\_\{\\text\{obs\}\}\. When we further condition the CID on observed data, we can express our uncertainty about which DGP generated the data via the CID\-PPD:

P⁡\(Y∣do⁡\(T=t\),X=x,𝒟obs\)=∫P⁡\(Y∣do⁡\(T=t\),X=x,ψ\)​π​\(ψ∣𝒟obs\)​d​ψ\.P\(Y\\mid\\operatorname\{do\}\(T=t\),X=x,\\mathcal\{D\}\_\{\\text\{obs\}\}\)=\\int P\(Y\\mid\\operatorname\{do\}\(T=t\),X=x,\\psi\)\\pi\(\\psi\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\)\\,\\text\{d\}\\psi\.\(23\)The CID\-PPD can be understood as separating two types of uncertainty\. First, for the DGP posteriorπ⁡\(ψ∣𝒟obs\)\\pi\(\\psi\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\), observing enough data under identifiability conditions would in principle collapseπ⁡\(ψ∣𝒟obs\)\\pi\(\\psi\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\)onto one DGP\. Hence, the DGP posterior represents*epistemic*uncertainty as to which DGP generated the data we observed due to our lack of knowledge\. If identifiability assumptions do not hold, the DGP posterior would also represent epistemic uncertainty from our inability to distinguish observationally equivalent DGPs\. This uncertainty is structural, and would only be reducible by performing interventions\. On the other hand,P⁡\(Y∣do⁡\(T=t\),X=x,ψ\)P\(Y\\mid\\operatorname\{do\}\(T=t\),X=x,\\psi\)represents*aleatoric*uncertainty about which outcome we will observe under an intervention, assuming a fixed DGPψ\\psi\. This uncertainty arises from randomness in the DGP, including any noise and exogenous variables that are not observed\.

If one is mainly interested in treatment effect estimates such as the CATE or ATE, a natural object to model is the CDTE\. Similarly to the CID, this can also be formulated as a PPD by conditioning on observed data\. The CDTE\-PPD is:

P⁡\(Y⁡\(1\)−Y⁡\(0\)∣X=x,𝒟obs\)=∫P⁡\(Y⁡\(1\)−Y⁡\(0\)∣X=x,ψ\)​π​\(ψ∣𝒟obs\)​d​ψ\.P\(Y\(1\)\-Y\(0\)\\mid X=x,\\mathcal\{D\}\_\{\\text\{obs\}\}\)=\\int P\(Y\(1\)\-Y\(0\)\\mid X=x,\\psi\)\\pi\(\\psi\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\)\\,\\text\{d\}\\psi\.\(24\)This is similar to the PPD but targets only the difference in potential outcomes rather than each potential outcome individually\.

The PPD is a fundamental object in Bayesian inference and data analysis\([Gelman et al\., 2013](https://arxiv.org/html/2609.03003#bib.bib58)\)and is not specific to causal inference, yet it has no closed\-form solution in general and must be approximated\. Popular traditional methods involve first approximating the posteriorπ⁡\(ψ∣𝒟obs\)\\pi\(\\psi\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\)and then using eqs\. \([22](https://arxiv.org/html/2609.03003#S2.E22)\)–\([24](https://arxiv.org/html/2609.03003#S2.E24)\) to calculate the PPD\. Many techniques have historically been developed for posterior approximation, such as Markov Chain Monte Carlo \(MCMC\) methods\([Neal, 1996](https://arxiv.org/html/2609.03003#bib.bib54);[Hoffman and Gelman, 2014](https://arxiv.org/html/2609.03003#bib.bib37)\), variational inference\([Jordan et al\., 1999](https://arxiv.org/html/2609.03003#bib.bib56);[Wainwright and Jordan, 2008](https://arxiv.org/html/2609.03003#bib.bib55)\), and neural posterior estimation\([Papamakarios and Murray, 2016](https://arxiv.org/html/2609.03003#bib.bib57)\)\.

### 2\.6Prior\-data fitted networks

More recently,*prior\-data fitted networks*\(PFNs\)\([Müller et al\., 2022](https://arxiv.org/html/2609.03003#bib.bib24)\)have been introduced as transformers\([Vaswani et al\., 2017](https://arxiv.org/html/2609.03003#bib.bib18)\)that can perform Bayesian inference by leveraging in\-context learning\([Brown et al\., 2020](https://arxiv.org/html/2609.03003#bib.bib36)\)\. Moreover, contrary to the traditional methods of Bayesian inference, PFNs go directly from dataset to PPD without explicitly approximating the posterior\. A valuable consequence of this is that PPDs provide uncertainty quantification in the same forward pass used for prediction\([Müller et al\., 2022](https://arxiv.org/html/2609.03003#bib.bib24);[Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3);[Melnychuk et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib95)\)\.

PFNs were first developed as a model of a generic PPD\. Suppose there is an underlying set of possible tasksφ\\varphiover which we assume some priorp⁡\(φ\)p\(\\varphi\)\. Given a specific supervised predictive \(not causal\) taskφ\\varphi, a finite labeled dataset𝒟sup=\{\(xn,yn\)\}n=1N\\mathcal\{D\}\_\{\\text\{sup\}\}=\\\{\(x\_\{n\},y\_\{n\}\)\\\}\_\{n=1\}^\{N\}is sampled\. Then after defining the posteriorp⁡\(φ∣𝒟sup\)p\(\\varphi\\mid\\mathcal\{D\}\_\{\\text\{sup\}\}\), the PPD can be expressed as

p⁡\(y∣x,𝒟sup\)=∫p⁡\(y∣x,φ\)​p​\(φ∣𝒟sup\)​d​φ\.p\(y\\mid x,\\mathcal\{D\}\_\{\\text\{sup\}\}\)=\\int p\(y\\mid x,\\varphi\)p\(\\varphi\\mid\\mathcal\{D\}\_\{\\text\{sup\}\}\)\\,\\text\{d\}\\varphi\.\(25\)This PPD reflects our lack of knowledge about which task was used to sample𝒟sup\\mathcal\{D\}\_\{\\text\{sup\}\}\. Given the supervised dataset𝒟sup\\mathcal\{D\}\_\{\\text\{sup\}\}, the PPD can be used to estimate the label distribution of a new datapointxxdrawn from the same taskφ\\varphi\.

Instead of approximating the posterior, PFNs learn a mapping from a dataset of samples to the PPD directly, which is generically called*amortized Bayesian inference*\. At inference, they take an entire supervised dataset as context, and for each query pointxxthey directly outputqθ​\(y∣x,𝒟sup\)q\_\{\\theta\}\(y\\mid x,\\mathcal\{D\}\_\{\\text\{sup\}\}\), a parametrized model ofp⁡\(y∣x,𝒟sup\)p\(y\\mid x,\\mathcal\{D\}\_\{\\text\{sup\}\}\)\. This powerful paradigm is enabled by three factors:

1. 1\.The*prior\-data loss*which minimizes the cross entropy betweenp⁡\(y∣x,𝒟sup\)p\(y\\mid x,\\mathcal\{D\}\_\{\\text\{sup\}\}\)andqθ​\(y∣x,𝒟sup\)q\_\{\\theta\}\(y\\mid x,\\mathcal\{D\}\_\{\\text\{sup\}\}\), without needing to evaluatep⁡\(y∣x,𝒟sup\)p\(y\\mid x,\\mathcal\{D\}\_\{\\text\{sup\}\}\);
2. 2\.Tractable yet highly diverse priorsp⁡\(φ\)p\(\\varphi\)that generate unlimited synthetic tasks and datasets;
3. 3\.A transformer\-based architecture enabling in\-context learning\.

First, the prior\-data loss is defined as the negative log\-likelihood of the PFN’s output distributionqθq\_\{\\theta\}at the correct labelyy, when given the correspondingxxand supervised dataset as context:

ℒpred​\(θ\)=𝔼φ∼p⁡\(φ\),𝒟sup∪\{x,y\}∼Pφ​\[−log⁡\(qθ​\(y∣x,𝒟sup\)\)\]\.\\mathcal\{L\}\_\{\\text\{pred\}\}\(\\theta\)=\\mathbb\{E\}\_\{\\varphi\\sim p\(\\varphi\),\\,\\mathcal\{D\}\_\{\\text\{sup\}\}\\cup\\\{x,y\\\}\\sim P^\{\\varphi\}\}\\big\[\-\\log\(q\_\{\\theta\}\(y\\mid x,\\mathcal\{D\}\_\{\\text\{sup\}\}\)\)\\big\]\.\(26\)This loss is tractable because it only requires evaluating the modelqθq\_\{\\theta\}at the value of the true label, and does not require evaluating the true PPD\. Nevertheless, a brief computation\([Müller et al\., 2022](https://arxiv.org/html/2609.03003#bib.bib24)\)shows that this loss is equal to the forward KL divergence between the true PPD andqθq\_\{\\theta\}\. Thus, minimizing the prior\-data loss encouragesqθq\_\{\\theta\}to be a good approximation to the PPD\. In practice, model parametersθ\\thetaare updated using stochastic gradient descent \(SGD\)\.

Second, notice that SGD training with the prior\-data lossℒpred​\(θ\)\\mathcal\{L\}\_\{\\text\{pred\}\}\(\\theta\)requires sampling tasksφ∼p⁡\(φ\)\\varphi\\sim p\(\\varphi\), then sampling supervised datasets𝒟sup\\mathcal\{D\}\_\{\\text\{sup\}\}\. This could be accomplished by collecting relevant real\-world datasets corresponding to various tasks, and then iterating over them for gradient steps\. However, when pre\-training a PFN it is not assumed that one knows what taskφ\\varphiwill need to be solved at inference\. To be true foundation models, PFNs are expected to generalize to*any*unseen task at inference\. This requires that the priorp⁡\(φ\)p\(\\varphi\)have as broad coverage as possible\. In practice most PFNs have relied on synthetic priors which are tractable to sample from and produce highly diverse datasets\. Training proceeds by sampling new prior\-synthetic data each batch, exposing the PFN to maximally diverse data over the course of training\.

Third, a model architecture to support direct PPD prediction viaqθ​\(y∣x,𝒟sup\)q\_\{\\theta\}\(y\\mid x,\\mathcal\{D\}\_\{\\text\{sup\}\}\)must accept an entire dataset at inference to facilitate predictions forxx\. Modern foundation models often do not require training on the specific inference task because they leverage in\-context learning\. This is in turn enabled by the attention mechanism of transformers, which allows query tokens \(embeddings ofxx\) to attend to context tokens \(embeddings of the labeled examples in𝒟sup\\mathcal\{D\}\_\{\\text\{sup\}\}\)\.

In summary, a PFN is pretrained using the prior\-data loss with synthetic data sampled from a prior\. At inference, it leverages in\-context learning to predict the label distribution \(PPD\) of query data using supervised samples from a never\-before\-seen task, doing so without updates to the PFN’s weights\.

Early demonstrations of PFNs showed strong performance in a variety of areas\([Müller et al\., 2022](https://arxiv.org/html/2609.03003#bib.bib24)\), and quickly excelled at tabular predictive tasks\([Hollmann et al\., 2023](https://arxiv.org/html/2609.03003#bib.bib10);[Hollmann et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib59);[Ma et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib12);[Qu et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib11)\), marking the advent of tabular foundation models \(TFMs\)\. In this setting eachxxis a set of numerical features, andyyis a classification or regression label\. PFNs have had similar success in relational learning\([Wang et al\., 2026b](https://arxiv.org/html/2609.03003#bib.bib63);[Hayler et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib60)\)and time\-series forecasting\([Dooley et al\., 2023](https://arxiv.org/html/2609.03003#bib.bib61);[Taga et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib62)\), where the data structures are somewhat more complex, but it is still possible to design synthetic data priors\.

## 3Causal Foundation Models

In[Section2\.5](https://arxiv.org/html/2609.03003#S2.SS5)we arrived at the theoretical underpinning of modeling causal estimands and distributions from observational data: assuming a prior over DGPs and identifiability conditions, the PPD of a causal estimandgg\([Equation22](https://arxiv.org/html/2609.03003#S2.E22)\) gives a distribution over the likely values thatggtakes, accounting for uncertainty from our finite observations, and from our lack of knowledge of which DGP generated the data\. Similar observations held for the CID\-PPD \([Equation23](https://arxiv.org/html/2609.03003#S2.E23)\) and CDTE\-PPD \([Equation24](https://arxiv.org/html/2609.03003#S2.E24)\)\. Then, in[Section2\.6](https://arxiv.org/html/2609.03003#S2.SS6)we saw that PFNs have emerged as a powerful paradigm for estimating generic PPDs, and have been widely used for predictive tabular tasks\. It is natural to ask how PFNs might be applied to the relevant PPDs from causal inference\. This connection is the basis for causal foundation models\.

### 3\.1What are CFMs?

Causal foundation models use PFNs to amortize Bayesian inference for causal quantities like the CEPO or the CID\. Though the field is quickly evolving, core qualities of CFMs have gradually emerged and solidified\. We therefore adopt the following as our working definition of a causal foundation model\.

Causal Foundation Model###### Definition 6\. Acausal foundation model \(CFM\)is a prior\-data fitted network that has been pretrained on a variety of causal tasks in order to estimate causal quantities from new datasets using in\-context learning\. We emphasize that a CFM predicts on unseen tasks without updating any model weights\.

CFMs apply the large\-scale prior\-fitting methodology of PFNs to causal inference\. During pretraining, the CFM is repeatedly shown a labeled observational dataset generated by an SCM sampled from a prior, together with a causal query which can be computed from the SCM, but which must be estimated by the model from the observational dataset alone\. Through many such iterations, the CFM learns a mapping from an observational dataset to a PPD for the causal quantity\. At deployment, the weights remain fixed\. Causal effect estimation occurs through in\-context learning\([Brown et al\., 2020](https://arxiv.org/html/2609.03003#bib.bib36)\)based on examples from an observational dataset, rather than weight updates\.

CFMs vastly speed up the workflow for causal effect estimation\. A traditional workflow requires the analyst to understand a dataset and the causal connections between covariates, specify a causal estimand, choose an adjustment set or identification strategy, fit nuisance models, diagnose overlap, and compute an estimator\. A CFM compresses much of this workflow into a single forward pass of a pretrained network\. This can greatly reduce friction for non\-expert users and enable rapid causal analysis across many small datasets \([Figure3](https://arxiv.org/html/2609.03003#S3.F3)\)\.

Moreover, they have demonstrated top results on causal tasks, performing above or on par with leading causal inference models which are trained for each task \(see our benchmarking in[Section4](https://arxiv.org/html/2609.03003#S4)\)\. They also perform better than TFMs applied off\-the\-shelf to causal problems, demonstrating the effectiveness of pretraining on an intrinsically causal prior\([Robertson et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib2);[Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3)\)\. We discuss the pretraining process of CFMs further in[Section3\.2](https://arxiv.org/html/2609.03003#S3.SS2)\.

The first CFMs that fit the above definition were proposed simultaneously by[Robertson et al\. \(2025\)](https://arxiv.org/html/2609.03003#bib.bib2)and[Balazadeh et al\. \(2025\)](https://arxiv.org/html/2609.03003#bib.bib3)\. Focusing on the binary treatments setting, Do\-PFN\([Robertson et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib2)\)models the CID\-PPD, while CausalPFN\([Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3)\)models the CEPO\-PPD, among other differences in design choices\. Shortly after, CausalFM\([Ma et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib4)\)was introduced which models the CDTE\-PPD but aims to cover the IV and frontdoor settings in addition to backdoor \(see[Section2\.3](https://arxiv.org/html/2609.03003#S2.SS3)\)\. We provide implementations of these CFMs for the reader to experiment with[https://github.com/layer6ai-labs/cfms/blob/main/notebooks/Foundation_models_sandbox.ipynb](https://github.com/layer6ai-labs/cfms/blob/main/notebooks/Foundation_models_sandbox.ipynb)\. Another related line of work estimates marginal interventional distributions using transformer\-based neural processes\([Dhir et al\., 2025b](https://arxiv.org/html/2609.03003#bib.bib92)\)\.

![Refer to caption](https://arxiv.org/html/2609.03003v1/images/workflow_difference.png)Figure 3:Traditional vs\. CFM workflows\.\(Top\)For each new problem, the traditional pipeline involves analyzing the dataset, proposing a model, and training it before inference\.\(Bottom\)CFMs are applied off\-the\-shelf to new problems, without training, fine\-tuning, or hyperparameter optimization\. The reader is invited to experiment with several implemented CFMs for themselves[https://github.com/layer6ai-labs/cfms/blob/main/notebooks/Foundation_models_sandbox.ipynb](https://github.com/layer6ai-labs/cfms/blob/main/notebooks/Foundation_models_sandbox.ipynb)\.Although we have framed PFNs as a perfect fit for modeling causal quantities like the CEPO\-PPD or CID\-PPD, there are several adaptations that are needed to bring PFNs from the world of predictive tasks into the world of causal inference\. Recall from[Section2\.6](https://arxiv.org/html/2609.03003#S2.SS6)the three enabling factors introduced by PFNs: the prior\-data lossℒpred​\(θ\)\\mathcal\{L\}\_\{\\text\{pred\}\}\(\\theta\), synthetic and tractable priorsp⁡\(φ\)p\(\\varphi\), and transformer\-based architectures for in\-context learning \(ICL\)\. In the following, we will discuss how each of these factors must be updated to construct a CFM\.

#### 3\.1\.1The causal prior\-data loss

The prior\-data loss in[eq\.26](https://arxiv.org/html/2609.03003#S2.E26)supports training a PFN to map directly from a supervised dataset to a PPD over predictive labels\. CFMs are trained using a modified version of the prior\-data loss called the*causal prior\-data loss*\([Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3)\)\. The exact form of the causal prior\-data loss depends on which PPD is being targeted, but all share an important distinction from the prior\-data loss for predictive PFNs\. In[eq\.26](https://arxiv.org/html/2609.03003#S2.E26)the model must predict a distribution over labelsyygiven featuresxx, and is shown examples in the same format, namely\(x,y\)\(x,y\)pairs via𝒟sup\\mathcal\{D\}\_\{\\text\{sup\}\}\. In contrast, we want CFMs to solve a*causal*task from purely observational data; even though the CFM must predict a causal quantity for any covariate\-treatment pair, the examples it is shown are of the form\(x,t,y\)\(x,t,y\)via𝒟obs\\mathcal\{D\}\_\{\\text\{obs\}\}\(a single treatment and factual outcome per covariate set\),*not*of the form\(x,t0,y⁡\(0\),t1,y⁡\(1\)\)\(x,t\_\{0\},y\(0\),t\_\{1\},y\(1\)\), nor any similar set like\(x,t0,μt0​\(x\),t1,μt1​\(x\)\)\(x,t\_\{0\},\\mu\_\{t\_\{0\}\}\(x\),t\_\{1\},\\mu\_\{t\_\{1\}\}\(x\)\)\. These latter options contain interventional information which would be useful for learning causal PPDs, but are not feasible to obtain at inference time, so they also should not be used for training\. Due to this mismatch of observations and predictions inherent to causal inference, CFMs are trained on a structurally harder task than predictive PFNs\.

For the CEPO\-PPD \(recall the notation from[Section2\.5](https://arxiv.org/html/2609.03003#S2.SS5)\), the causal prior\-data loss takes the form

ℒt​\(θ\)=𝔼ψ∼π,𝒟obs∪\{x\}∼Pobsψ​\[−log⁡\(qθ​\(μt​\(x,Pψ\)∣x,t,𝒟obs\)\)\]\.\\mathcal\{L\}\_\{t\}\(\\theta\)=\\mathbb\{E\}\_\{\\psi\\sim\\pi,\\,\\mathcal\{D\}\_\{\\text\{obs\}\}\\cup\\\{x\\\}\\sim P^\{\\psi\}\_\{\\text\{obs\}\}\}\\big\[\-\\log\(q\_\{\\theta\}\(\\mu\_\{t\}\(x;P^\{\\psi\}\)\\mid x,t,\\mathcal\{D\}\_\{\\text\{obs\}\}\)\)\\big\]\.\(27\)To compute this loss, a DGP is first sampled from the prior over DGPsπ⁡\(ψ\)\\pi\(\\psi\)specified by the developer; then, an observational dataset𝒟obs\\mathcal\{D\}\_\{\\text\{obs\}\}and an additional set of query covariatesxxare sampled according toPobsψP^\{\\psi\}\_\{\\text\{obs\}\}\. The ground\-truth valueμt​\(x,Pψ\)\\mu\_\{t\}\(x;P^\{\\psi\}\)is computed from the sampled DGPψ\\psi, and we evaluate the model’s likelihood assigned to that value\.

Similarly, for the CID\-PPD the loss takes the form

ℒt\(θ\)=𝔼ψ∼π,𝒟obs∪\{x\}∼Pψobs,y∼Pψ\(⋅∣do\(T=t\),x\)\[−log\(qθ\(y∣x,t,𝒟obs\)\)\]\.\\mathcal\{L\}\_\{t\}\(\\theta\)=\\mathbb\{E\}\_\{\\psi\\sim\\pi,\\,\\mathcal\{D\}\_\{\\text\{obs\}\}\\cup\\\{x\\\}\\sim P^\{\\psi\}\_\{\\text\{obs\}\},\\,y\\sim P^\{\\psi\}\(\\cdot\\mid\\text\{do\}\(T=t\),x\)\}\\big\[\-\\log\(q\_\{\\theta\}\(y\\mid x,t,\\mathcal\{D\}\_\{\\text\{obs\}\}\)\)\\big\]\.\(28\)Again, a DGP is sampled and used to generate an observational dataset and query datapoint fromPobsψP^\{\\psi\}\_\{\\text\{obs\}\}\. The treatmentttis applied as an intervention onxxto get the outcomeyy, sampled from the true CID\. Then the loss evaluates the model’s likelihood assigned to this outcome\.

For both losses, note that in practice a sum overℒt​\(θ\)\\mathcal\{L\}\_\{t\}\(\\theta\)for allt∈𝒯t\\in\\mathcal\{T\}is used; for a single queryxxthe model must accurately produce the CEPO\-PPD/CID\-PPD for anytt\. Furthermore, both of these losses are tractable as long as interventional data can be simulated\. They require evaluating the modelqθq\_\{\\theta\}and accessing ground\-truth causal quantities, namely the CEPO valueμt​\(x,Pψ\)\\mu\_\{t\}\(x;P^\{\\psi\}\)or outcomey⁡\(t,Pψ\)y\(t;P^\{\\psi\}\)\. Importantly, they do not require knowing the true underlying PPDs in closed form or evaluating their likelihoods, which would greatly limit the set of DGPs we could practically use in the priorπ⁡\(ψ\)\\pi\(\\psi\)\. In[Section3\.2](https://arxiv.org/html/2609.03003#S3.SS2)we return to the point of how ground\-truth causal quantities are simulated in practice\.

Training with the causal prior\-data losses also converges to the true PPD under suitable conditions\.[Balazadeh et al\. \(2025\)](https://arxiv.org/html/2609.03003#bib.bib3)show that, assuming almost allψ\\psiin the support ofπ⁡\(ψ\)\\pi\(\\psi\)satisfy positivity, the CEPO prior\-data loss in[eq\.27](https://arxiv.org/html/2609.03003#S3.E27)is equal to the expected forward KL divergence between the true CEPO\-PPD andqθq\_\{\\theta\}, up to a constant\. This means that training under the causal prior\-data loss will in principle converge to the true CEPO\-PPD\. To recover a point estimate of the CEPO, CausalPFN returns the mean𝔼⁡\[qθ​\(μt​\(x\)∣x,t,𝒟obs\)\]\\mathbb\{E\}\[q\_\{\\theta\}\(\\mu\_\{t\}\(x\)\\mid x,t,\\mathcal\{D\}\_\{\\text\{obs\}\}\)\]\. Crucially,[Balazadeh et al\. \(2025\)](https://arxiv.org/html/2609.03003#bib.bib3)prove that as the size of\|𝒟obs\|\|\\mathcal\{D\}\_\{\\text\{obs\}\}\|grows, this mean converges to the true CEPO if and only if the CEPO is identifiable given the support of the priorπ\\pi\.

Similarly, for the CID prior\-data loss,[Robertson et al\. \(2025\)](https://arxiv.org/html/2609.03003#bib.bib2)show that training with[eq\.28](https://arxiv.org/html/2609.03003#S3.E28)minimizes the expected forward KL divergence between the true CID\-PPD andqθq\_\{\\theta\}\. Thus, training with the causal prior\-data loss yields a good approximationqθq\_\{\\theta\}of the causal target\. However, this does not make all causal effects estimatable\. For example, if the desired causal estimands are not identifiable given the support of the priorπ\\pi, then it is impossible to predict an accurate point estimate, even with infinite data\. Non\-identifiability implies that the posterior distributionP⁡\(ψ∣𝒟obs\)P\(\\psi\\mid\\mathcal\{D\}\_\{\\text\{obs\}\}\)does not collapse around a single valueψ\\psi, as multiple DGPs could explain the observed data\. As a consequence, non\-identifiability would be expressed as uncertainty in the CID\.

#### 3\.1\.2Tractable synthetic priors over DGPs

The next step in applying the machinery of[Section2\.6](https://arxiv.org/html/2609.03003#S2.SS6)to causal inference is to design causal priors from which both observational and interventional data can be sampled\. SCM\-based priors make this possible \([Section2\.4](https://arxiv.org/html/2609.03003#S2.SS4)\)\.

Synthetic data priors have been adopted for predictive PFNs, including TFMs, because collecting real\-world tabular data at scale and with sufficient diversity has proved challenging\. However, synthetic data is not necessary for TFMs—real data can be used, as done for example by TabDPT\([Ma et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib12)\), ConTextTab\([Spinaci et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib97)\), and RealTabPFN\([Garg et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib98)\), because the modelqθq\_\{\\theta\}only needs to be evaluated at the labelyyto compute the prior\-data loss \([Equation25](https://arxiv.org/html/2609.03003#S2.E25)\)\. In contrast we saw that eqs\.[27](https://arxiv.org/html/2609.03003#S3.E27)and[28](https://arxiv.org/html/2609.03003#S3.E28)require evaluating the model at the true CEPO value, or an interventional outcome, respectively\. These causal quantities are not available in real observational data, which makes synthetic priors more of a necessity, not merely a convenience\. Furthermore, we can never obtain counterfactual information for any individual\. For coarsely\-defined covariate strata, an experiment may contain comparable individuals exposed to different treatments; however, as the resolution of the strata increases \(and thus becomes more specific\), finding comparable individuals becomes more rare\. A causal prior\-data loss requires reliable ground\-truth labels for any combination of covariates and intervention, which is simply not available in real data at sufficient scale and diversity to train a CFM\.

Synthetic priors have therefore been a fundamental and scalable ingredient to all current CFMs, able to generate a highly diverse and high\-quality sample of tasks for pretraining\. However, an added complexity is the question of identifiability\. When designing the prior, one must decide which subset𝒫⊂𝒫DGP\\mathcal\{P\}\\subset\\mathcal\{P\}\_\{\\text\{DGP\}\}of DGPs the prior covers\.

Table 1:The main design choices of the first CFMs\. All are PFNs and use in\-context learning, but differ in their prediction target, prior design, identification strategy, and architectural details\.Do\-PFN employs a non\-identifiable prior, meaning that their prior may generate DGPs having different CID\-PPDs but the same observational distributions\. The motivation is to enable their model to report an increase in uncertainty coming from non\-identifiability, for example, in the presence of unobserved confounders\([Robertson et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib2)\)\. On the other hand, CausalPFN is built with a backdoor prior, meaning that its prior is supported on the set of DGPs𝒫back⊂𝒫DGP\\mathcal\{P\}\_\{\\text\{back\}\}\\subset\\mathcal\{P\}\_\{\\text\{DGP\}\}satisfying strong ignorability and SUTVA \(see[Equation19](https://arxiv.org/html/2609.03003#S2.E19)\)\. This is one reason why CausalPFN is able to outperform Do\-PFN in backdoor evaluation settings \(see our benchmarking in[Section4](https://arxiv.org/html/2609.03003#S4), or\([Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3), Appendix E\)\)\. On the other hand, CausalFM consists of three separate models trained on priors for different settings: one for the backdoor setting, one for instrumental variables, and one for the frontdoor setting\([Ma et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib4)\)\. Additionally, the authors prove that if a prior has non\-identifiable support, the resulting PPD cannot recover the true causal effect even with infinite data, providing a strong argument for restricting to priors with identifiable support\([Ma et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib4), Theorem 4\.3\)\. See[Table1](https://arxiv.org/html/2609.03003#S3.T1)for a comparison of these three CFMs and their priors\.

#### 3\.1\.3Transformer architectures for CFMs

While a traditional causal estimator operates with training and test splits, CFMs operate with*context*and*query*data since no training is needed to perform inference\. This is aligned with the current paradigm in TFMs\([Hollmann et al\., 2023](https://arxiv.org/html/2609.03003#bib.bib10)\); like their TFM cousins, Do\-PFN, CausalPFN, and CausalFM are all transformer\-based models \([Table2](https://arxiv.org/html/2609.03003#S3.T2)\) that make predictions for novel queries via ICL while relying on a context set\. CFMs receive a tabular observational dataset𝒟obs=\{\(xn,tn,yn\)\}n=1N\\mathcal\{D\}\_\{\\text\{obs\}\}=\\\{\(x\_\{n\},t\_\{n\},y\_\{n\}\)\\\}\_\{n=1\}^\{N\}as context, and a causal task as query\. For example, a CATE query contains covariates\{xm\}m=1M\\\{x\_\{m\}\\\}\_\{m=1\}^\{M\}, and the model’s task is to estimateCATE⁡\(xm\)\\operatorname\{CATE\}\(x\_\{m\}\)\. A CEPO query, on the other hand, would consist of covariates together with treatment values\{\(xm,tm\)\}m=1M\\\{\(x\_\{m\},t\_\{m\}\)\\\}\_\{m=1\}^\{M\}; the model’s task is then to estimateμtm​\(xm\)\\mu\_\{t\_\{m\}\}\(x\_\{m\}\)\.

Before being passed to the transformer, input data must be tokenized and embedded, with different CFMs performing different tokenization steps \(e\.g\. row\-based or row\-and\-feature\-based\)\. Notably, the embedding of the context dataset contains the observed covariates and treatments as well as the factual outcomes, while the embedding of the query does not contain any outcome information\. The embedded representations are then passed to the transformer, which applies masked attention to the context and query tokens: each token attends to the context, and no token attends to the query\. This mask, which is the same as that applied in TFMs, has two important consequences\. First, the representation of the observational context data is independent of the queries\. Second, query predictions depend only on the context data, not on each other\. Batched query prediction is therefore independent of the content and order of the query set, once conditioned on the context𝒟obs\\mathcal\{D\}\_\{\\text\{obs\}\}\.

The architecture of CFMs must take into account the distinguished role that the treatment variable plays\. The simplest approach is to enforce that the treatment variable is the first column of the dataset, so that the model learns this in its internal representation\. This is done by CausalPFN and Do\-PFN, with Do\-PFN also adding a treatment column indicator to the representation of each dataset\. CausalFM, on the other hand, passes the treatment and covariates through different encoders before concatenating them and passing them to the transformer\.

Inference proceeds by passing both the context and query to the modelℳθ\\mathcal\{M\}\_\{\\theta\}, which outputs its estimateqθq\_\{\\theta\}of the desired causal estimand\. For example, for CATE estimation:

qθ​\(CATE​\(xm\)∣xm,𝒟obs\)=ℳθ​\(ctx=𝒟obs,qry=\{xm\},est=CATE\)\.q\_\{\\theta\}\(\\text\{CATE\}\(x\_\{m\}\)\\mid x\_\{m\},\\mathcal\{D\}\_\{\\text\{obs\}\}\)=\\mathcal\{M\}\_\{\\theta\}\(\\texttt\{ctx\}=\\mathcal\{D\}\_\{\\text\{obs\}\},\\ \\texttt\{qry\}=\\\{x\_\{m\}\\\},\\ \\texttt\{est\}=\\texttt\{CATE\}\)\.\(29\)Since the CFM outputs the entire CATE\-PPD, a point estimate ofCATE​\(xm\)\\text\{CATE\}\(x\_\{m\}\)can be obtained by taking the mean,τ^​\(xm\)≔𝔼⁡\[qθ​\(CATE​\(xm\)∣xm,𝒟obs\)\]\\hat\{\\tau\}\(x\_\{m\}\)\\coloneqq\\mathbb\{E\}\[q\_\{\\theta\}\(\\text\{CATE\}\(x\_\{m\}\)\\mid x\_\{m\},\\mathcal\{D\}\_\{\\text\{obs\}\}\)\]\. In the example of CEPO estimation, we would have:

qθ​\(μtm​\(xm\)∣xm,tm,𝒟obs\)=ℳθ​\(ctx=𝒟obs,qry=\{\(xm,tm\)\},est=CEPO\),q\_\{\\theta\}\(\\mu\_\{t\_\{m\}\}\(x\_\{m\}\)\\mid x\_\{m\},t\_\{m\},\\mathcal\{D\}\_\{\\text\{obs\}\}\)=\\mathcal\{M\}\_\{\\theta\}\(\\texttt\{ctx\}=\\mathcal\{D\}\_\{\\text\{obs\}\},\\ \\texttt\{qry\}=\\\{\(x\_\{m\},t\_\{m\}\)\\\},\\ \\texttt\{est\}=\\texttt\{CEPO\}\),\(30\)and the point estimateμ^tm​\(xm\)≔𝔼⁡\[qθ​\(μtm​\(xm\)∣xm,tm,𝒟obs\)\]\\hat\{\\mu\}\_\{t\_\{m\}\}\(x\_\{m\}\)\\coloneqq\\mathbb\{E\}\[q\_\{\\theta\}\(\\mu\_\{t\_\{m\}\}\(x\_\{m\}\)\\mid x\_\{m\},t\_\{m\},\\mathcal\{D\}\_\{\\text\{obs\}\}\)\]\. The practical distinction between traditional and foundational causal models, enabled by the transformer architecture, is illustrated by the example algorithms in[Figure4](https://arxiv.org/html/2609.03003#S3.F4)\.

Table 2:Model size and transformer depth for different CFMs and TFMs\.Model\# Parameters\# Transformer Layers*Tabular foundation models*TabICLv2\([Qu et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib26)\)29M12TabPFN\-3\([Grinsztajn et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib25)\)58M24TabDPT\-Turbo\([Hosseinzadeh et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib27)\)63M32*Causal foundation models*Do\-PFN\([Robertson et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib2)\)7\.3M12CausalPFN\([Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3)\)20M20CausalFM\([Ma et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib4)\)2\.7M10Algorithm 1: Traditional causal estimator

1:Training set

𝒟obs=\{\(xn,tn,yn\}n=1N\\mathcal\{D\}\_\{\\text\{obs\}\}=\\\{\(x\_\{n\},t\_\{n\},y\_\{n\}\\\}\_\{n=1\}^\{N\}, query

\{xm\}m=1M\\\{x\_\{m\}\\\}\_\{m=1\}^\{M\}, estimator class

ℳθ\\mathcal\{M\}\_\{\\theta\}
2:Estimate causal query

g^​\(xm\)\\hat\{g\}\(x\_\{m\}\)
3:Specify assumptions and nuisance components

4:Initialize parameters

θ\\theta
5:whilenot convergeddo

6:Compute loss

ℒ⁡\(θ,𝒟obs\)\\mathcal\{L\}\(\\theta;\\mathcal\{D\}\_\{\\text\{obs\}\}\)
7:Update

θ←θ−η​∇θℒ\\theta\\leftarrow\\theta\-\\eta\\nabla\_\{\\theta\}\\mathcal\{L\}
8:endwhile

9:Estimate

g^​\(xm\)=ℳθ​\(xm\)\\hat\{g\}\(x\_\{m\}\)=\\mathcal\{M\}\_\{\\theta\}\(x\_\{m\}\)
10:return

g^​\(xm\)\\hat\{g\}\(x\_\{m\}\)

Algorithm 2: Causal foundation model

1:Context set

𝒟obs=\{\(xn,tn,yn\)\}n=1N\\mathcal\{D\}\_\{\\text\{obs\}\}=\\\{\(x\_\{n\},t\_\{n\},y\_\{n\}\)\\\}\_\{n=1\}^\{N\}, query

\{xm\}m=1M\\\{x\_\{m\}\\\}\_\{m=1\}^\{M\}, pretrained CFM

ℳθ\\mathcal\{M\}\_\{\\theta\}
2:Estimate causal query

g^​\(xm\)\\hat\{g\}\(x\_\{m\}\)
3:Estimate

g^​\(xm\)=ℳθ​\(𝒟obs,xm\)\\hat\{g\}\(x\_\{m\}\)=\\mathcal\{M\}\_\{\\theta\}\(\\mathcal\{D\}\_\{\\text\{obs\}\},x\_\{m\}\)
4:return

g^​\(xm\)\\hat\{g\}\(x\_\{m\}\)

Figure 4:A high\-level comparison between traditional causal estimators and CFMs \(shown with the example task of CATE estimation\)\. The traditional estimators must be trained for each new task, updating model parameters on a training set, whereas the CFM workflow keeps pretrained parameters fixed and uses the training dataset as context\.

### 3\.2Training CFMs

As mentioned above, CFMs are trained on synthetic data\. In[Section2\.6](https://arxiv.org/html/2609.03003#S2.SS6)we discussed that a core part of training PFN\-style models is designing a highly diverse prior to ensure maximum coverage of possible tasks that could be encountered at inference time\. SCM\-based priors make this possible due to their ability to computationally represent DGPs and to generate both observational and interventional data efficiently \([Section2\.4](https://arxiv.org/html/2609.03003#S2.SS4)\)\. We therefore consider a priorπ⁡\(ψ\)\\pi\(\\psi\)over the space of possible SCMs, writingSψS^\{\\psi\}for an SCM corresponding to the DGPψ\\psi\.

The method of sampling from a prior and instantiating an SCMSψS^\{\\psi\}differs across models\. Do\-PFN’s prior instantiates random DAGs directly using topological sorting, and then assigns structural equations generated as additive noise models\([Robertson et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib2)\)\. CausalPFN uses random MLPs as in TabPFNv1\([Hollmann et al\., 2023](https://arxiv.org/html/2609.03003#bib.bib10)\), subsampling a random set of nodes and using these to construct tabular datasets\([Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3)\)\. CausalFM samples clusters of DAGs using random MLPs, then uses Bayesian neural networks \(BNNs\) to assign values and sampling mechanisms for each cluster\([Ma et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib4)\)\.

Synthetic data generation for CFM trainingBayesian priorπ⁡\(ψ\)\\pi\(\\psi\)Graphstructurefj​\(⋅\)f\_\{j\}\(\\,\\cdot\\,\)Structuralequationsp0p\_\{0\}p1p\_\{1\}Treatmentassignmentε∼P⁡\(ε\)\\varepsilon\\sim P\(\\varepsilon\)NoisedistributionsStructural causal modelX1X\_\{1\}X2X\_\{2\}TTYYXj:=fj​\(paj,εj\)X\_\{j\}:=f\_\{j\}\\\!\\left\(\\mathrm\{pa\}\_\{j\},\\varepsilon\_\{j\}\\right\)Y:=fY​\(paY,εY\)Y:=f\_\{Y\}\\\!\\left\(\\mathrm\{pa\}\_\{Y\},\\varepsilon\_\{Y\}\\right\)sampleObservational data𝒟obs\\mathcal\{D\}\_\{\\mathrm\{obs\}\}X1X\_\{1\}X2X\_\{2\}TTYYx1,1x\_\{1,1\}x1,2x\_\{1,2\}t1t\_\{1\}y1y\_\{1\}x2,1x\_\{2,1\}x2,2x\_\{2,2\}t2t\_\{2\}y2y\_\{2\}⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotpxN,1x\_\{N,1\}xN,2x\_\{N,2\}tNt\_\{N\}yNy\_\{N\}Interventional ground truthμt​\(x\)=?\\mu\_\{t\}\(x\)=\\ ?μt​\(x\)\\mu\_\{t\}\(x\)tt0011generateCFM training examplecontext dataqueryX1X\_\{1\}X2X\_\{2\}TTYY⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp⋅⋅⋅\\cdotp\\mkern\-2\.0mu\\cdotp\\mkern\-2\.0mu\\cdotp\+\+μt​\(x\)=?\\mu\_\{t\}\(x\)=\\ ?model predictionμ^t​\(x\)\\hat\{\\mu\}\_\{t\}\(x\)tt0011

Figure 5:Sampling data from a synthetic causal prior\.*Step 1\.*Sample an SCM fromπ\\pi\.*Step 2\.*Generate an observational dataset𝒟obs\\mathcal\{D\}\_\{\\mathrm\{obs\}\}from the SCM\.*Step 3\.*Simulate interventional supervised learning targets, such as the CEPO or CATE\.*Step 4\.*Provide the observational data to the CFM as context and train it to estimate the target\.At a high level, a CFM training run has the following form \(see[Figure5](https://arxiv.org/html/2609.03003#S3.F5)for a visual overview\):

1. 1\.Sample an SCMSψ∼πS^\{\\psi\}\\sim\\pi\. In practice, this might mean initializing the weights of an MLP; selecting which edges to prune and which nodes to select as covariatesXX, treatmentTT, and outcomeYY; and deciding on the probability distributionsPUP\_\{U\}from which noise variablesUkU\_\{k\}will be sampled\.
2. 2\.Generate an observational dataset𝒟obs∼Pobsψ\\mathcal\{D\}\_\{\\text\{obs\}\}\\sim P^\{\\psi\}\_\{\\text\{obs\}\}of sizeNNfromSψS^\{\\psi\}\. This is done by sampling the noise variablesUk∼PUU\_\{k\}\\sim P\_\{U\}and using the structural equations ofSψS^\{\\psi\}to compute the values ofXX,TT, andYY\.
3. 3\.Simulate interventions to create an interventional dataset𝒟int∼Pψ\\mathcal\{D\}\_\{\\text\{int\}\}\\sim P^\{\\psi\}of sizeMMfromSψS^\{\\psi\}\. This is done differently for different interventional targets \(e\.g\. CEPO\-PPD, CID\-PPD\)\.
4. 4\.Perform supervised learning by asking the model to predict the true causal target from𝒟obs\\mathcal\{D\}\_\{\\text\{obs\}\}, using a causal prior\-data loss \(e\.g\.[Equation27](https://arxiv.org/html/2609.03003#S3.E27)\)\.

Let us focus more on step 3, which is the heart of causal estimation tasks\. If this were a predictive PFN training loop, step 3 would generate data from the same distributionPobsψP^\{\\psi\}\_\{\\text\{obs\}\}as step 2, asking the model to predict data from the same distribution that it is shown as context\. In causal inference, we are asking the model to estimate causal quantities, like the CEPO and the CID, from observational data alone\. This requires generating counterfactual, or interventional, data\.

To generate potential outcomesy⁡\(t\)y\(t\), which is required by the CID\-PPD causal prior\-data loss \([Equation28](https://arxiv.org/html/2609.03003#S3.E28)\), we must generate data from the conditional interventional distributionPψ\(⋅∣do\(T=t\),X=x\)P^\{\\psi\}\(\\cdot\\mid\\operatorname\{do\}\(T=t\),X=x\)\. As we saw in[Section2\.4](https://arxiv.org/html/2609.03003#S2.SS4), we mechanically replace theTTstructural equation byT=t∗T=t^\{\*\}with a fixed “query” treatmentt∗∈𝒯t^\{\*\}\\in\\mathcal\{T\}\. Graphically, this corresponds to removing the incoming edges to the treatment nodeTT\. We can then sampleUk∼PUU\_\{k\}\\sim P\_\{U\}and use the remaining structural equations to generate an interventional sample\(x,t∗,y⁡\(t∗\)\)\(x,t^\{\*\},y\(t^\{\*\}\)\), representing the outcome of an individual with covariatesxxas a result of receiving treatmentt∗t^\{\*\}\. This is depicted in[Figure6](https://arxiv.org/html/2609.03003#S3.F6)\.

To generate ground\-truth CEPO labels, we can take an expectation over the exogenous noise variablesUkU\_\{k\}:

μt​\(x\)=𝔼Uk∼PU\|X​\[Y⁡\(t\)∣X=x\]\.\\mu\_\{t\}\(x\)=\\mathbb\{E\}\_\{U\_\{k\}\\sim P\_\{U\\mid X\}\}\[Y\(t\)\\mid X=x\]\.\(31\)For example, for the SCM depicted in[Figure6](https://arxiv.org/html/2609.03003#S3.F6), we have

μt​\(x\)=𝔼U3∼PU​\[Y⁡\(t\)∣X=x\]\.\\mu\_\{t\}\(x\)=\\mathbb\{E\}\_\{U\_\{3\}\\sim P\_\{U\}\}\[Y\(t\)\\mid X=x\]\.\(32\)The generated SCMs must be diverse and complex, and hence[Equation31](https://arxiv.org/html/2609.03003#S3.E31)cannot generally be written in closed form\. Although it is straightforward to approximate using Monte Carlo sampling, obtaining an accurate estimate of[eq\.31](https://arxiv.org/html/2609.03003#S3.E31)may require many noise samples and passes through the SCM\. This must be repeated for each\(x,t\)\(x,t\)pair for whichμt​\(x\)\\mu\_\{t\}\(x\)is to be calculated, severely slowing down training time\. An alternative employed by[Balazadeh et al\. \(2025\)](https://arxiv.org/html/2609.03003#bib.bib3)is to replace the nodeYYin the SCM with a CEPO nodeμ\\mu, so that a forward pass through the SCM generatesμt​\(x\)\\mu\_\{t\}\(x\)directly instead ofY⁡\(t\)Y\(t\)\. In order to capture realistic fluctuations in the true outcomeYY, a new nodeξ\\xiis introduced to the SCM which functions as a noise node\. After centering and scalingξ\\xi, the potential outcomeY⁡\(t\)Y\(t\)is computed asY⁡\(t\)=μt​\(x\)\+ξ⁡\(x,t\)Y\(t\)=\\mu\_\{t\}\(x\)\+\\xi\(x,t\)\. With this method, both the CEPO and the potential outcome are computed by the SCM in one forward pass\.

Sampled SCMSψ∼πS^\{\\psi\}\\sim\\piand noises\{\(un,1obs,un,2obs,un,3obs\)\}n=1N\\\{\(u\_\{n,1\}^\{\\text\{obs\}\},u\_\{n,2\}^\{\\text\{obs\}\},u\_\{n,3\}^\{\\text\{obs\}\}\)\\\}\_\{n=1\}^\{N\},\{\(um,1int,um,2int,um,3int\)\}m=1M\\\{\(u\_\{m,1\}^\{\\text\{int\}\},u\_\{m,2\}^\{\\text\{int\}\},u\_\{m,3\}^\{\\text\{int\}\}\)\\\}\_\{m=1\}^\{M\}\(a\) Observational pass\(b\) Interventional passXXTTYYU1U\_\{1\}U2U\_\{2\}U3U\_\{3\}SψS^\{\\psi\}𝒟obs=\{\(xn,tn,yn\)\}n=1N\\mathcal\{D\}\_\{\\mathrm\{obs\}\}=\\\{\(x\_\{n\},t\_\{n\},y\_\{n\}\)\\\}\_\{n=1\}^\{N\}context dataXXT=t∗T=t^\{\*\}YYU1U\_\{1\}U2U\_\{2\}U3U\_\{3\}×\\times×\\timesSψS^\{\\psi\}do⁡\(T=t∗\)\\mathrm\{do\}\(T=t^\{\*\}\)replacesfTf\_\{T\}𝒟int=\{\(xm,t∗,ym​\(t∗\)\)\}m=1M\\mathcal\{D\}\_\{\\mathrm\{int\}\}=\\\{\(x\_\{m\},t^\{\*\},y\_\{m\}\(t^\{\*\}\)\)\\\}\_\{m=1\}^\{M\}interventional targetsFigure 6:Observational and interventional dataset generation from an SCMSψS^\{\\psi\}\.\(a\)The full set of original structural equations map a batch of exogenous noise vectors\{\(un,1obs,un,2obs,un,3obs\)\}n=1N\\\{\(u\_\{n,1\}^\{\\text\{obs\}\},u\_\{n,2\}^\{\\text\{obs\}\},u\_\{n,3\}^\{\\text\{obs\}\}\)\\\}\_\{n=1\}^\{N\}to observational data𝒟obs\\mathcal\{D\}\_\{\\text\{obs\}\}\.\(b\)The interventiondo⁡\(T=t∗\)\\operatorname\{do\}\(T=t^\{\*\}\)sets the value of treatment for themmth query individual to bet∗t^\{\*\}, replacing the structural equationfTf\_\{T\}ofTTwithT=t∗T=t^\{\*\}\. Using the noise samples\{\(um,1int,um,2int,um,3int\)\}m=1M\\\{\(u\_\{m,1\}^\{\\text\{int\}\},u\_\{m,2\}^\{\\text\{int\}\},u\_\{m,3\}^\{\\text\{int\}\}\)\\\}\_\{m=1\}^\{M\}, the remaining structural equations then produce an interventional dataset𝒟int\\mathcal\{D\}\_\{\\text\{int\}\}with ground\-truth potential outcomesym​\(t∗\)y\_\{m\}\(t^\{\*\}\)\.So far, no research has been published on the quality of priors for CFMs\. However, it seems plausible that the diversity aspect of prior design should align with research on priors for TFMs\.[Zhang et al\. \(2025\)](https://arxiv.org/html/2609.03003#bib.bib45)quantify prior diversity and support its importance for TFM performance\. Recently,[Türkmen et al\. \(2026\)](https://arxiv.org/html/2609.03003#bib.bib46)have created an interface to compare different TFM priors and their relative strengths and weaknesses on downstream tasks\. However, similar work has not been undertaken yet for causal priors\.

In summary, current transformer\-based CFMs are designed to predict a causal quantity and are pretrained with the corresponding causal prior\-data loss\. Each iteration of training involves sampling an SCM from a prior, using it to generate a synthetic observational dataset, and also simulating corresponding interventional data\. The CFM outputs an approximation of the causal PPD, using the observational data as context and unlabeled data as the query in a transformer architecture, with special care taken to tokenize and embed covariates, treatments, and outcomes\. Based on the outputs, the CFM is trained to minimize the negative log\-likelihood it assigns to the true sample from the causal PPD, which derives from the simulated interventional data\. Finally, at inference the same context\-query setup is used to directly predict causal quantities given only observational data, performing causal inference on unseen data in a single forward pass\. All CFMs share this basic function, with important design choices including which PPD to model, the specification of the SCM prior including identifiability, and architectural considerations like input embeddings and attention mechanisms\.

## 4Benchmarking CFMs

In this section, we evaluate emerging CFMs against established causal machine learning estimators\. While[Section3](https://arxiv.org/html/2609.03003#S3)categorized the architectural paradigms of Do\-PFN\([Robertson et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib2)\), CausalPFN\([Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3)\), and CausalFM\([Ma et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib4)\), existing empirical comparisons in the literature remain largely limited to synthetic structural simulations under varying data\-generation setups\.

The main exception is the Jobs experiment of[Ma et al\. \(2026, Section 6\.1\)](https://arxiv.org/html/2609.03003#bib.bib4), which compares all three models on a semi\-synthetic version of the Jobs dataset\([Smith and Todd, 2005](https://arxiv.org/html/2609.03003#bib.bib110)\), derived from the Lalonde study\([LaLonde, 1986](https://arxiv.org/html/2609.03003#bib.bib64)\); there, outcomes are simulated by the authors to allow evaluation against ground truth, and only conditional effect error is reported\.

One contribution of our work is establishing a standardized, fair empirical comparison across these three CFMs for causal inference on more complex semi\-synthetic observational data\. Since purely observational datasets lack counterfactual data, and purely synthetic simulations often oversimplify real\-world mechanisms, we evaluate models on the semi\-syntheticRealCause\-Lalondebenchmark\([LaLonde, 1986](https://arxiv.org/html/2609.03003#bib.bib64);[Neal et al\., 2020](https://arxiv.org/html/2609.03003#bib.bib65)\)\. RealCause has been widely adopted across the causal machine learning community as a realistic evaluation standard\([Mahajan et al\., 2024](https://arxiv.org/html/2609.03003#bib.bib96);[de Vassimon Manela et al\., 2024](https://arxiv.org/html/2609.03003#bib.bib99);[van der Laan et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib102), e\.g\.,\), as well as in evaluations of CFMs\([Robertson et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib2);[Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3), e\.g\.,\)\. This allows us to evaluate both conditional effect heterogeneity \(CATE\\mathrm\{CATE\}\) and population\-level effects \(ATE\\mathrm\{ATE\}\) against known ground truth under strong confounding, creating an evaluation regime that effectively differentiates performance across models\.

### 4\.1Experimental Setup

##### Benchmark Data\.

We evaluate on the semi\-synthetic Lalonde\-CPS \(16,177 samples\) and Lalonde\-PSID \(2,675 samples\) cohorts\([LaLonde, 1986](https://arxiv.org/html/2609.03003#bib.bib64);[Neal et al\., 2020](https://arxiv.org/html/2609.03003#bib.bib65)\)\. RealCause fits generative models to treated individuals and non\-experimental control individuals\. By generating simulated outcomes as explicit functions of observed covariates alone, the benchmark guarantees conditional ignorability \(Y⁡\(1\),Y⁡\(0\)⟂⟂T\|XY\(1\),Y\(0\)\\perp\\\!\\\!\\perp T\\mid X\) by construction while preserving the complex, empirical covariate and selection distributions\. To account for sampling variability in the generative process, all metrics are aggregated across 10 independent semi\-synthetic realizations and reported as mean±\\pmstandard error\.

##### Baselines\.

We compare CFMs against standard causal machine learning estimators\. Classical baselines include meta\-learners \(S\-Learner, T\-Learner, X\-Learner\([Künzel et al\., 2019](https://arxiv.org/html/2609.03003#bib.bib66)\)\), propensity and doubly robust methods \(IPW, DR\-Learner\([Robins et al\., 1994](https://arxiv.org/html/2609.03003#bib.bib67);[Kennedy, 2023](https://arxiv.org/html/2609.03003#bib.bib68)\)\), and Double Machine Learning \(Debiased ML,\([Chernozhukov et al\., 2018](https://arxiv.org/html/2609.03003#bib.bib6)\)\)\. All estimators except IPW are implemented and tuned via EconML\([Battocchi et al\., 2019](https://arxiv.org/html/2609.03003#bib.bib69)\)\. To provide strong, competitive baselines, all underlying nuisance models are selected via FLAML AutoML \(v2\.3\.5;[Wang et al\. \(2021\)](https://arxiv.org/html/2609.03003#bib.bib109)\) run independently per nuisance, per estimator, per RealCause realization, and per cohort with a 900\-second budget and 3\-fold cross\-validation\. For outcome regressions, FLAML optimizesR2R^\{2\}across candidate families including LightGBM, depth\-limited and unlimited XGBoost, Random Forest, Extra Trees, andkk\-NN\. For propensity models \(required by X\-Learner, DML, DR\-Learner, and IPW\), FLAML optimizes ROC\-AUC across classifier versions of these families as well asℓ1\\ell\_\{1\}\- andℓ2\\ell\_\{2\}\-regularized logistic regression\. For the final stages, DML uses an unregularized linear model and DR\-Learner uses a causal forest \(ForestDRLearnerwith 1,000 trees, 5\-fold cross\-validation, and degree\-3 polynomial features\)\. This hyperparameter regime produces highly optimized versions of the classical baselines\.

For CFMs, we evaluate frozen, pre\-trained checkpoints of Do\-PFN, CausalPFN, and CausalFM in the backdoor setting\. These models are downloaded out\-of\-the\-box from their respective sources and applied without any parameter or hyperparameter tuning\. We note that a fourth model designed by[Dhir et al\. \(2025b\)](https://arxiv.org/html/2609.03003#bib.bib92)could be considered a CFM, but no pre\-trained weights are available to make a direct comparison here\.

##### Metrics\.

ConsiderNNevaluation individuals with covariates and ground\-truth CATE values\{\(xn,τ⁡\(xn\)\)\}n=1N\\\{\(x\_\{n\},\\tau\(x\_\{n\}\)\)\\\}\_\{n=1\}^\{N\}\. Letλ=1N​∑n=1Nτ⁡\(xn\)\\lambda=\\frac\{1\}\{N\}\\sum\_\{n=1\}^\{N\}\\tau\(x\_\{n\}\)denote the true ATE\. We writeτ^​\(xn\)\\hat\{\\tau\}\(x\_\{n\}\)to denote the predicted CATE values andλ^\\hat\{\\lambda\}to denote the predicted ATE\. We evaluate the following metrics:

PEHE⁡\(τ^\)=1N​∑n=1N\(τ⁡\(xn\)−τ^​\(xn\)\)2,RelativeError⁡\(λ^\)=\|λ^−λ\|\|λ\|\.\\operatorname\{PEHE\}\(\\hat\{\\tau\}\)=\\sqrt\{\\frac\{1\}\{N\}\\sum\_\{n=1\}^\{N\}\\big\(\\tau\(x\_\{n\}\)\-\\hat\{\\tau\}\(x\_\{n\}\)\\big\)^\{2\}\},\\qquad\\operatorname\{RelativeError\}\(\\hat\{\\lambda\}\)=\\frac\{\|\\hat\{\\lambda\}\-\\lambda\|\}\{\|\\lambda\|\}\.\(33\)Precision in estimation of heterogeneous effects \(PEHE\\operatorname\{PEHE\}\) measures root mean squared error in conditional effect estimation across individuals, while the relative error quantifies relative population\-level bias via the ATE\.

For Do\-PFN which approximates the CID\-PPD, point estimates for the potential outcomesμ0​\(x\)\\mu\_\{0\}\(x\)andμ1​\(x\)\\mu\_\{1\}\(x\)are first computed by taking the mean of the model’s approximationqθq\_\{\\theta\}to the CID\-PPD \(see[eq\.10](https://arxiv.org/html/2609.03003#S2.E10)\):

μ^0​\(x\)=𝔼⁡\[qθ​\(y∣x,t=0,𝒟obs\)\],μ^1​\(x\)=𝔼⁡\[qθ​\(y∣x,t=1,𝒟obs\)\]\.\\hat\{\\mu\}\_\{0\}\(x\)=\\mathbb\{E\}\[q\_\{\\theta\}\(y\\mid x,t=0,\\mathcal\{D\}\_\{\\text\{obs\}\}\)\],\\qquad\\hat\{\\mu\}\_\{1\}\(x\)=\\mathbb\{E\}\[q\_\{\\theta\}\(y\\mid x,t=1,\\mathcal\{D\}\_\{\\text\{obs\}\}\)\]\.\(34\)The CATE estimate is then returned asτ^​\(x\)=μ^1​\(x\)−μ^0​\(x\)\\hat\{\\tau\}\(x\)=\\hat\{\\mu\}\_\{1\}\(x\)\-\\hat\{\\mu\}\_\{0\}\(x\)\(see[eq\.5](https://arxiv.org/html/2609.03003#S2.E5)\)\.

For CausalPFN which approximates the CEPO\-PPD, point estimates for the potential outcomes are first obtained by returning the means of the approximated CEPO\-PPDs:

μ^0​\(x\)=𝔼⁡\[qθ​\(μt​\(x\)∣x,t=0,𝒟obs\)\],μ^1​\(x\)=𝔼⁡\[qθ​\(μt​\(x\)∣x,t=1,𝒟obs\)\]\.\\hat\{\\mu\}\_\{0\}\(x\)=\\mathbb\{E\}\[q\_\{\\theta\}\(\\mu\_\{t\}\(x\)\\mid x,t=0,\\mathcal\{D\}\_\{\\text\{obs\}\}\)\],\\qquad\\hat\{\\mu\}\_\{1\}\(x\)=\\mathbb\{E\}\[q\_\{\\theta\}\(\\mu\_\{t\}\(x\)\\mid x,t=1,\\mathcal\{D\}\_\{\\text\{obs\}\}\)\]\.\(35\)As above, the CATE estimate is then returned asτ^​\(x\)=μ^1​\(x\)−μ^0​\(x\)\\hat\{\\tau\}\(x\)=\\hat\{\\mu\}\_\{1\}\(x\)\-\\hat\{\\mu\}\_\{0\}\(x\)\.

CausalFM’s CATE model is trained to estimate the CDTE\-PPD\. It does so using a Gaussian mixture model \(GMM\) with 5 components,

qθ​\(Y⁡\(1\)−Y⁡\(0\)∣X=x,𝒟obs\)=∑k=15wk​𝒩​\(Y⁡\(1\)−Y⁡\(0\)∣μk​\(x\),σk2​\(x\)\)\.q\_\{\\theta\}\(Y\(1\)\-Y\(0\)\\mid X=x,\\mathcal\{D\}\_\{\\text\{obs\}\}\)=\\sum\_\{k=1\}^\{5\}w\_\{k\}\\mathcal\{N\}\\big\(Y\(1\)\-Y\(0\)\\mid\\mu\_\{k\}\(x\),\\sigma\_\{k\}^\{2\}\(x\)\\big\)\.\(36\)The CATE estimate is thenτ^​\(x\)=∑k=15wk​μk​\(x\)\\hat\{\\tau\}\(x\)=\\sum\_\{k=1\}^\{5\}w\_\{k\}\\mu\_\{k\}\(x\)\.

We additionally report wall\-clock CPU runtime per cohort for going from observational data to predictions\. This means we include training, tuning, and inference for classical baselines, but only inference for CFMs since this is the only step we perform\. Hence, our measurements reflect actual practitioner wall\-clock time needed to apply each given method in practice\.

Finally, we report the average rank among the models tested\. For each metric, methods are ranked against each other within a single realization, and these ranks are then averaged over the realizations of both Lalonde\-CPS and Lalonde\-PSID cohorts, again reported asmean±standard error\\text\{mean\}\\pm\\text\{standard error\}\. Since IPW does not produce individual\-level estimates, PEHE ranks are computed among the remaining methods\.

##### Reproducibility & Code Artifacts\.

The full pipeline for loading the RealCause\-Lalonde benchmark, aggregating empirical results, and generating comparison tables and figures is provided as an interactive Jupyter notebook at[https://github.com/layer6ai-labs/cfms/blob/main/notebooks/Lalonde_benchmark_results.ipynb](https://github.com/layer6ai-labs/cfms/blob/main/notebooks/Lalonde_benchmark_results.ipynb)\. Since running 10 seeds of the benchmark takes a non\-trivial time and access to a GPU, we also release our full dataset of the results at[github\.com/layer6ai\-labs/cfms/tree/main/data](https://github.com/layer6ai-labs/cfms/tree/main/data)\.

### 4\.2Results and Discussion

Table 3:Empirical Results on RealCause\-Lalonde\.Metrics are aggregated across 10 random seeds \(realizations\)\. PEHE and ATE Relative Error are reported asmean±standard error\\text\{mean\}\\pm\\text\{standard error\}\. Median total wall\-clock time, including training and hyperparameter optimization for traditional estimators, is reported as CATE Runtime\. All times are reported using matched CPU hardware\. Avg\. rank is calculated over Lalonde\-CPS and Lalonde\-PSID combined\. Best single result per column inbold\.†IPW predicts at population level and does not produce individual\-level CATE estimates\.PEHE \(×103,↓\\times 10^\{3\},\\downarrowbetter\)ATE Relative Error \(↓\\downarrowbetter\)CATE Runtime \(s\)MethodLalondeCPS\{\}\_\{\\text\{CPS\}\}LalondePSID\{\}\_\{\\text\{PSID\}\}Avg\. RankLalondeCPS\{\}\_\{\\text\{CPS\}\}LalondePSID\{\}\_\{\\text\{PSID\}\}Avg\. RankMedian*Causal Foundation Models*CausalPFN8\.97±\\pm0\.0614\.00±\\pm0\.411\.75±\\pm0\.160\.17±\\pm0\.030\.24±\\pm0\.042\.65±\\pm0\.2618\.4Do\-PFN11\.96±\\pm0\.0920\.20±\\pm0\.394\.80±\\pm0\.210\.88±\\pm0\.010\.89±\\pm0\.016\.50±\\pm0\.30115\.1CausalFM12\.34±\\pm0\.0222\.27±\\pm0\.436\.60±\\pm0\.220\.94±\\pm0\.000\.95±\\pm0\.007\.80±\\pm0\.2631\.2*Traditional Estimators*T\-Learner9\.04±\\pm0\.0813\.65±\\pm0\.471\.40±\\pm0\.110\.28±\\pm0\.040\.04±\\pm0\.012\.25±\\pm0\.271803\.0Debiased ML9\.96±\\pm0\.3415\.45±\\pm1\.063\.20±\\pm0\.290\.34±\\pm0\.080\.29±\\pm0\.083\.20±\\pm0\.271807\.9IPW†———0\.15±\\pm0\.030\.08±\\pm0\.022\.05±\\pm0\.20—X\-Learner11\.90±\\pm0\.4020\.30±\\pm0\.605\.20±\\pm0\.340\.89±\\pm0\.060\.89±\\pm0\.057\.30±\\pm0\.322707\.3S\-Learner12\.45±\\pm0\.1120\.61±\\pm0\.415\.80±\\pm0\.240\.97±\\pm0\.010\.71±\\pm0\.057\.10±\\pm0\.34901\.4DR\-Learner13\.45±\\pm0\.2924\.09±\\pm1\.907\.25±\\pm0\.280\.89±\\pm0\.050\.64±\\pm0\.056\.15±\\pm0\.321820\.5

##### Benchmark Performance\.

The main benchmark results are shown in[Table3](https://arxiv.org/html/2609.03003#S4.T3)\. Despite not being trained on the Lalonde data distributions, CFMs are remarkably competitive with classical estimators\. CausalPFN achieves the lowest average rank among CFMs, closely trailing the extensively tuned T\-Learner baseline while outperforming it on Lalonde\-CPS\. IPW, a method specialized for ATE prediction that does not give individual\-level predictions, showed the strongest results on ATE Relative Errorλ\\lambda\. We note that our CausalPFN results independently replicate the values reported by[Balazadeh et al\. \(2025\)](https://arxiv.org/html/2609.03003#bib.bib3), although our T\-learner and IPW baselines appear slightly stronger than in that work\. CFMs that output the CID\-PPD or CDTE\-PPD rather than the CEPO\-PPD were somewhat less performant, but still competitive with tuned X\- and S\-Learners\. See[Figure7](https://arxiv.org/html/2609.03003#S4.F7)for a comparative plot of model performance\.

##### Inference Amortization vs\. Tuning Overhead\.

CFMs show a significant runtime advantage since inference can be performed directly on raw data without training\. On CPU, all three CFMs tested were 1\-2 orders of magnitude faster than training, tuning, and inference with a traditional model, with CausalPFN running fastest on CPU\. Further speedups in CFM runtime are obtained on GPU\. CFMs have amortized the effort needed for Bayesian inference via pretraining, which we exclude from the runtime measurement since a practitioner does not need to perform it\. Amortization thus represents a speedup of up to100×100\\timesto get predictions of comparable quality to a highly optimized T\-Learner\. In production workflows with repeated evaluation across subsets, CFMs effectively eliminate the computational barrier of hyperparameter tuning and cross\-fitting loops while preserving competitive CATE accuracy; see[Figure7](https://arxiv.org/html/2609.03003#S4.F7)\.

##### Average Treatment Effect Magnitude Recovery\.

All three CFMs receive identically standardized covariates and outcomes\. Under this common treatment, CausalPFN recovers the population effect closely \(ATE relative error0\.170\.17on Lalonde\-CPS\), while Do\-PFN and CausalFM do not: both remain near a relative error of 1 on both cohorts \(0\.880\.88and0\.940\.94\), recovering only a small fraction of the true contrast\. Their errors are moreover almost identical across the two cohorts and carry very small standard errors, which points to a systematic shrinkage of the predicted effect toward zero rather than to unstable estimation\.

![Refer to caption](https://arxiv.org/html/2609.03003v1/images/Accuracy_Runtime.png)Figure 7:CATE Estimation Ranking vs\. Wall\-Clock Runtime on RealCause\-Lalonde\. Causal foundation models deliver competitive estimation results while achieving orders\-of\-magnitude faster inference compared to tuned classical estimators because they do not need to be trained on RealCause\. For CFMs, runtimes are shown both on CPU \(matching the hardware used for all other models\) and an A100 GPU\.

## 5Broader Directions and Applications

The first generation of CFMs\([Robertson et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib2);[Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3);[Ma et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib4)\)established that Bayesian causal inference could be amortized across tasks, and thus that foundation models for causal inference were possible and, indeed, yielded top performance on common benchmarks\. These models focus primarily on causal effect estimation in the binary treatment setting, and accept only tabular data as input\. Subsequent work has broadened this initial formulation along several axes\. Newer models address partial identification of causal effects, target additional treatment and data regimes, and extend beyond causal effect estimation to causal discovery and domain\-specific interventional questions\.

Our working definition in[Definition6](https://arxiv.org/html/2609.03003#Thmdefinition6)reserves the term*causal foundation model*for PFNs that estimate causal quantities, such as the CEPO, CID, or CDTE\. In this section, we first review important precursor work before discussing further progress in this core causal inference setting\. We then adopt a broader lens, considering closely related models for causal discovery as well as related scientific tasks\. Not every model discussed below is therefore a CFM under[Definition6](https://arxiv.org/html/2609.03003#Thmdefinition6); the works we discuss in this section, however, show how the core ideas of CFMs are being applied in neighbouring areas\.

### 5\.1Previous work

Prior to the first generation of CFMs, important precursor work applied the idea of learning\-to\-learn to causal inference, including both CaML\([Nilforoshan et al\., 2023](https://arxiv.org/html/2609.03003#bib.bib9)\)and BBCI\([Bynum et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib1)\)\. These works proposed algorithms \(rather than specific models\) which could in principle be applied to any trainable family of models\. In particular, BBCI is an algorithm which trains models to predict causal estimands on synthetic DGPs modeled by SCMs and can be deployed on unseen data\. However, the class of SCMs that[Bynum et al\. \(2025\)](https://arxiv.org/html/2609.03003#bib.bib1)train on are comparatively narrow and low\-dimensional\. Due to this restricted setting, there is no indication that this method will scale, which limits use for real\-world, big\-data environments\. Causal Inference with Attention \(CInA\)\([Zhang et al\., 2023](https://arxiv.org/html/2609.03003#bib.bib8)\)is another ideological precursor to modern CFMs which demonstrated a theoretical link between attention and causal inference and proposed a zero\-shot, transformer\-based model for causal inference\.

### 5\.2Extending causal inference capabilities

#### 5\.2\.1Incorporating structural knowledge

The first generation of CFMs accept raw observational tabular data as input, and are unable to condition on additional domain knowledge such as partial knowledge of the underlying causal structure\. For example, a practitioner may know that a covariateX1X\_\{1\}is a direct cause ofX2X\_\{2\}, yet the model may still assign positive posterior mass to DGPs in whichX2X\_\{2\}is a cause ofX1X\_\{1\}\.[Reuter et al\. \(2026\)](https://arxiv.org/html/2609.03003#bib.bib88)address this limitation, arguing that the proper method for conditioning CFMs on additional knowledge is to utilize partial ancestral information, meaning \(potentially incomplete\) knowledge of the set of ancestors of each variable\. They investigate different ways for CFMs to leverage this knowledge, finding empirically that learnable attention biases are particularly effective\. Experiments show that conditioning on partial ancestral knowledge leads to sizeable performance gains\([Reuter et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib88)\)\.

TabPFN\-CFM\([Zhu et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib49)\)predicts both causal structure as well as effects\. If the underlying causal graph is known, it can be added to the model inputs to improve its causal effect estimates\. TabPFN\-CFM is designed to predict observational, interventional, and counterfactual outcomes, thus operating at each level of Pearl’s causal hierarchy\([Pearl, 2009b](https://arxiv.org/html/2609.03003#bib.bib15)\)\.

#### 5\.2\.2New treatment and temporal regimes

Other work extends CFMs beyond binary treatment and time\-independent data\.[Stith et al\. \(2026\)](https://arxiv.org/html/2609.03003#bib.bib44)introduce CCPFN, the first CFM for continuous treatments, which reconstructs an entire individual treatment\-response curve over the treatment range\. This requires constructing a higher\-dimensional prior over treatment and outcome mechanisms than in the binary or multi\-arm setting\.

Along the other axis, CausalLongPFN\([Zare et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib70)\)is a CFM designed for potential outcome prediction in longitudinal settings with time\-dependent confounding, nonlinear evolution, and other dynamical effects\. At inference time, given historical data trajectories and a proposed future treatment sequence, it estimates future counterfactual outcomes\.

There is a related line of work developing priors for temporal causal inference\. CausalTimePrior\([Thumm and Chen, 2026](https://arxiv.org/html/2609.03003#bib.bib90);[Thumm et al\., 2026a](https://arxiv.org/html/2609.03003#bib.bib107)\)construct methods for sampling from time\-dependent SCMs coupled with observational and interventional time series\.[Thumm et al\. \(2026b\)](https://arxiv.org/html/2609.03003#bib.bib89)propose and construct a continuous\-time version based on stochastic processes\.

#### 5\.2\.3Partial identification and sensitivity analysis

More recent work focuses on partially identifiable settings, where the treatment effect cannot be computed exactly, but rather can only be known to lie in some finite interval\. IV\-ICL\([Balazadeh et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib29)\)addresses partial identifiability in the IV setting \(see[Section2\.3](https://arxiv.org/html/2609.03003#S2.SS3)\) where hidden confounders may exist\. It estimates the PPD of the causal effect and returns bounds on the effect from the quantiles of this PPD\. It provides more reliable bounds compared to traditional baselines while substantially reducing inference time\.[Bellot and Dhir \(2026\)](https://arxiv.org/html/2609.03003#bib.bib91)also build a PFN that targets partially identifiable settings, without focusing on IV specifically\.

Related to this, causal sensitivity analysis studies how derived treatment effects would change in the presence of hidden confounders of different strengths\([Rosenbaum, 2002](https://arxiv.org/html/2609.03003#bib.bib105)\)\.[Javurek et al\. \(2026\)](https://arxiv.org/html/2609.03003#bib.bib14)create a PFN for causal sensitivity analysis which outputs sensitivity bounds directly\. Together, these works extend CFMs to regimes where the treatment effect is not precisely identifiable, but where quantitative statements can still be made\.

#### 5\.2\.4Reliability and calibration

A complementary line of work has developed which studies the quality and potential bias of CFM predictions\. These studies reinforce the importance of a highly diverse prior for CFM pretraining\.

[Melnychuk et al\. \(2026\)](https://arxiv.org/html/2609.03003#bib.bib95)investigate the frequentist consistency of CFM\-based ATE estimation, finding that some existing CFMs have a prior\-induced bias, which they heuristically note is the result of the CFM’s prior being supported on DGPs with low confounding\. They also introduce a one\-step posterior correction method to eliminate this bias\.[Mourao et al\. \(2026\)](https://arxiv.org/html/2609.03003#bib.bib100)show signs of low coverage of CausalPFN’s credible intervals, but it is not clear whether they performed these experiments after calibrating CausalPFN\([Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3)\)\.[Wang et al\. \(2026a\)](https://arxiv.org/html/2609.03003#bib.bib101)propose a task\-specific fine\-tuning strategy to correct bias due to low prior coverage in CFM pretraining\.[Ham et al\. \(2026\)](https://arxiv.org/html/2609.03003#bib.bib106)study how CFMs err when a post\-treatment covariate is passed to the model at inference time, and propose a method for filtering out such inputs\.

### 5\.3Foundation models for causal discovery

Causal discovery is the practice of discovering causal relationships between variables from observational data\([Glymour et al\., 2019](https://arxiv.org/html/2609.03003#bib.bib78);[Zanga et al\., 2022](https://arxiv.org/html/2609.03003#bib.bib48)\)\. This runs complementary to causal inference, which focuses on the effect that interventions on one variable \(treatment\) have on another \(outcome\)\. Causal discovery can be formulated as a graph discovery problem where one attempts to recover the underlying causal graph relating the observed variables \([Section2\.4](https://arxiv.org/html/2609.03003#S2.SS4)\)\. As in causal inference, a central difficulty in the field is identifiability \([Section2\.3](https://arxiv.org/html/2609.03003#S2.SS3)\)\. In general, observational data can only identify the Markov equivalence class of the underlying DAG rather than a unique graph\([Kalisch and Bühlmann, 2007](https://arxiv.org/html/2609.03003#bib.bib79);[Neal, 2020](https://arxiv.org/html/2609.03003#bib.bib5)\)\.

Causal discovery is a natural target for prior\-fitted networks\. A prior over SCMs induces a prior over causal graphs, allowing each synthetic training dataset to be paired with the graph of the SCM that generated it\. A PFN can therefore be trained to map a dataset directly to a posterior distribution over causal structures, amortizing the causal discovery process\. Indeed, it is argued that CFMs for causal effect estimation do so implicitly\([Balazadeh et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib3)\)\. Several models have been created recently which could be called causal discovery foundation models\.

#### 5\.3\.1Early approaches

CSIvA\([Ke et al\., 2023](https://arxiv.org/html/2609.03003#bib.bib51)\)and AVICI\([Lorch et al\., 2022](https://arxiv.org/html/2609.03003#bib.bib52)\)were early applications of neural networks and transformers to amortized causal discovery\. Both models learn to directly map data \(observational or interventional\) to a causal structure, although only AVICI learns a posterior over causal graphs\.[Montagna et al\. \(2025\)](https://arxiv.org/html/2609.03003#bib.bib94)show that CSIvA generalizes poorly to unseen causal structures and demonstrates that training models on a wide variety of causal structures can improve generalization, which is aligned with the core insight of synthetic prior design in PFN and CFM training\.

The Bayesian Causal Neural Process of[Dhir et al\. \(2025a\)](https://arxiv.org/html/2609.03003#bib.bib50)amortizes Bayesian causal discovery, learning to approximate a posterior over causal structures, and can generate samples from its learned posterior at inference time\. However, it differs from our definition of CFMs by training on multiple smaller priors with different parameters, rather than a single highly\-diverse training prior\. Their experiments do show that training their model on a combination of their smaller, separate priors retains the original model’s performance on each dataset\. This foreshadows the development of a more general causal discovery foundation model trained on a single large prior\.

[Sypniewski et al\. \(2025\)](https://arxiv.org/html/2609.03003#bib.bib53)inserts TabPFN into the causal discovery pipeline, using it to amortize likelihood estimation for causal graphs, rather than training a new model for causal discovery\. SEA\([Wu et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib83)\)is an early but distinct approach to causal discovery foundation models which also trains on large amounts of synthetic data\. However, it does not learn a mapping from datasets to a posterior distribution over causal structures\. Rather, at inference time, SEA trains an aggregator model that takes the results from weak causal discovery predictors and summary statistics and outputs a final predicted causal graph\. ADAG\([Yin et al\., 2025](https://arxiv.org/html/2609.03003#bib.bib104)\)is an early approach which uses a transformer to directly learn a mapping from data to causal graph\. However, ADAG is only seen to generalize well to unseen tasks whose underlying graphical structure or topological ordering is shared with its training data\.

#### 5\.3\.2General purpose causal discovery foundation models

Several more recent models step more clearly into the foundation model paradigm\. Arrow\([Thompson et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib81)\)is a zero\-shot causal discovery model that operates on observational tabular data and was trained on a large and highly diverse synthetic prior over causal graphs\. However, Arrow is not a Bayesian model, and outputs a single predicted DAG rather than a posterior distribution\. TabCausal\([Li et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib82)\)is a causal discovery foundation model which operates on observational and interventional tabular data and was also trained on a large, diverse synthetic prior\. TabCausal is also not a Bayesian learner, outputting only a probabilistic adjacency matrix representing the predicted directed edge likelihoods given the input data\. Similarly, FoundCause\([Blöbaum et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib86)\)is a foundation model which outputs a probabilistic adjacency matrix, as well as a confounding probability matrix representing the probability that any two variables are confounded by a hidden variable\.

DCD\-PFN\([Guan et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib84)\)applies a local\-to\-global method, first learning to approximate the posterior distribution of the Markov boundary of target variables and later patching these together to form a global causal graph\. TabPFN\-CFM\([Zhu et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib49), mentioned above\)is trained to perform both causal discovery and causal inference\. When the causal graph is unknown, the model learns an approximation to the posterior distribution of the true causal graph\. CDFM\([Qiao et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib80)\)is another causal discovery foundation model trained on a large, diverse synthetic prior, which generalizes to unseen evaluation scenarios\. DAG\-FM\([Chen et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib85)\)is a foundation model using two transformer sub\-modules, one for leaf nodes and one for parent nodes, combining the outputs of both to output a prediction of the true causal graph\. DAG\-FM is also pretrained on a large, diverse synthetic prior\.

Temporal causal discovery has also been addressed through large\-scale synthetic pretraining\. Though not a Bayesian learner,[Kougioulis et al\. \(2026\)](https://arxiv.org/html/2609.03003#bib.bib93)created a model for temporal causal discovery which directly predicts a time\-dependent adjacency matrix rather than a posterior distribution\. They train on a prior which includes time\-dependent SCMs constructed from real time series data, in addition to purely synthetic graph\-sampling methods\.

### 5\.4Domain\-specific and adjacent applications

The same prior\-fitting methodology employed by CFMs can be applied to more focused scientific domains where causal effect estimation for specific yet highly complex mechanisms is studied\. Due to the high complexity of these domains, a synthetic prior constructed specially for each can be diverse enough to justify pretraining at scale\.

#### 5\.4\.1Single\-cell perturbation models

Single\-cell perturbation modeling is a central topic in biology which aims to estimate the molecular response to perturbations within a single cell\([Bunne et al\., 2023](https://arxiv.org/html/2609.03003#bib.bib108)\)\. MapPFN\([Sextro et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib47)\)is a CFM trained on a synthetic biological prior which predicts the post\-perturbation distribution of single\-cell systems\. Their model takes a set of observational data and interventional experiments, and outputs an approximation to the post\-perturbation distribution of a new intervention\. PerturbPFN\([Gao et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib103)\)is another perturbation effect estimator which operates differently\. It first estimates an underlying causal graph which, coupled with an SCM decoder, outputs an estimate of the post\-perturbation distribution\. Both models demonstrate strong performance compared to baselines in the field\.

#### 5\.4\.2Survival analysis

Survival analysis deals with predicting the time\-to\-event for different events of interest\([Klein and Moeschberger, 2003](https://arxiv.org/html/2609.03003#bib.bib71)\)\. This could be the time between diagnosis and patient death\([Clark et al\., 2003](https://arxiv.org/html/2609.03003#bib.bib72)\), the time until a user churns\([Van den Poel and Larivière, 2004](https://arxiv.org/html/2609.03003#bib.bib73)\), or the time until a device fails\([Meeker and Escobar, 1998](https://arxiv.org/html/2609.03003#bib.bib74)\)\. A central issue in this field is that data are often censored or truncated, meaning that data is only partially known or excluded from the observed sample\. Recent works have introduced PFNs to survival analysis, creating foundational models like SIC\([Seletkov et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib76)\), SurvivalPFN\([Qi et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib75)\), and SurvPFN\([Böhm et al\., 2026](https://arxiv.org/html/2609.03003#bib.bib87)\)\. These all pretrain a PFN on a prior over synthetic survival analysis DGPs\. SurvivalPFN and SIC both achieve top performance compared to a wide variety of non\-foundational survival models, including classical survival models, tree\-based models, and neural models, and SurvPFN is competitive across all evaluated benchmarks\. Interestingly, SurvivalPFN also outperforms work by[Kim et al\. \(2026\)](https://arxiv.org/html/2609.03003#bib.bib77), an approach which casts survival analysis as a binary classification problem that it passes to a TFM\. Although this enables off\-the\-shelf TFMs to be applied to survival analysis, this demonstrates a clear advantage in SurvivalPFN’s pretraining process\.

### 5\.5Future directions

The models discussed above point to encouraging trends in CFM development and broader usage\. We have seen a wide variety of foundation models applied across various domains\. These suggest several areas for future improvements\.

Existing CFMs are typically specialized along three distinct axes: the treatment regime, the identification assumptions, and the target causal quantity\. A natural future goal is a single CFM that supports multiple treatment regimes, different identification assumptions, and estimation of many causal quantities at once\. Such a model would support binary, multi\-arm, and continuous treatment \(and combinations thereof\); be deployed in backdoor, IV, or other settings; and be used to estimate the CEPO, CID, and CDTE, in addition to other estimands\. Increased generality should not come at the cost of hiding identification assumptions; a useful interface would allow the analyst to specify the estimand as well as the assumptions, indicate whether the estimand is identified under those assumptions, and propagate uncertainty about causal structure into downstream estimation and uncertainty quantification\. One or more common internal representations could then support several downstream tasks without requiring a separate pretrained model for each\. Recent models which perform both causal discovery and inference provide an early step in this direction\.

Instead of operating on a single observational tabular dataset, future CFMs could incorporate more diverse information sources, such as experimental datasets, partial causal knowledge, or domain constraints\. Recent models which incorporate partial ancestral knowledge or graphical structure are promising examples\. More flexible interfaces might also allow a fixed model to revise its estimates through additional context if new experiments or structural knowledge become available\.

The quality of the synthetic training prior directly affects CFM performance and generalization\. Developing more quantitative measures of causal prior coverage would help improve prior design\. Along this direction, useful properties of future CFMs would include automatic detection of mismatch between the target DGP and the training prior\. Both problems are especially difficult in the causal setting, where observational data does not fully characterize the distributions of interest\.

## 6Conclusion

Causal foundation models are an emerging approach to solving problems in causal inference, and are beginning to be applied to different scientific domains\. They are transformer\-based models that are pretrained once on a synthetic causal prior; at inference time, they leverage in\-context learning to estimate causal effects in unseen settings\. This paradigm drastically cuts down on model deployment time compared to traditional estimators which have to be trained and tuned for each new problem\. At the same time, CFMs demonstrate competitive performance on causal benchmarks compared to traditional estimators\.

In this work, we provide a hands\-on introduction to CFMs\. We include several Jupyter notebooks \([https://github.com/layer6ai-labs/cfms/tree/main/notebooks](https://github.com/layer6ai-labs/cfms/tree/main/notebooks)\) in order to help the reader use these models immediately, as well as empirical results showing the effectiveness and speed of current CFMs\. Our hope is that this introduces CFMs to a wider audience and encourages the reader to get started using CFMs in their own work\.

#### Broader Impact Statement

The nature of our work is an introduction to emerging techniques in causal foundation modeling\. Our aim is to demystify how these methods operate and bring new techniques to a wider audience\. As such, we do not anticipate broader societal repercussions from this work\.

## References

- Alaa and van der Schaar \(2017\)A\. M\. Alaa and M\. van der SchaarBayesian inference of individualized treatment effects using multi\-task Gaussian processes\.InAdvances in Neural Information Processing Systems,Vol\.30\.Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p1.1)\.
- Athey and Imbens \(2017\)S\. Athey and G\. W\. ImbensThe state of applied econometrics: causality and policy evaluation\.Journal of Economic Perspectives31\(2\),pp\. 3–32\.External Links:[Document](https://dx.doi.org/10.1257/jep.31.2.3)Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p1.1)\.
- Balazadehet al\.\(2026\)V\. Balazadeh, H\. Kamkari, M\. Barath, R\. Silva, and R\. G\. KrishnanIV\-ICL: Bounding Causal Effects with Instrumental Variables via In\-Context Learning\.arXiv:2605\.12924\.Cited by:[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p9.1),[§5\.2\.3](https://arxiv.org/html/2609.03003#S5.SS2.SSS3.p1.1)\.
- Balazadehet al\.\(2025\)V\. Balazadeh, H\. Kamkari, V\. Thomas, J\. Ma, B\. Li, J\. C\. Cresswell, and R\. G\. KrishnanCausalPFN: Amortized Causal Effect Estimation via In\-Context Learning\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 154945–154984\.Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p6.1),[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p1.1),[§3\.1\.1](https://arxiv.org/html/2609.03003#S3.SS1.SSS1.p1.1),[§3\.1\.1](https://arxiv.org/html/2609.03003#S3.SS1.SSS1.p5.1),[§3\.1\.2](https://arxiv.org/html/2609.03003#S3.SS1.SSS2.p4.1),[§3\.1](https://arxiv.org/html/2609.03003#S3.SS1.p5.1),[§3\.1](https://arxiv.org/html/2609.03003#S3.SS1.p6.1),[§3\.2](https://arxiv.org/html/2609.03003#S3.SS2.p2.1),[§3\.2](https://arxiv.org/html/2609.03003#S3.SS2.p6.3),[Table 1](https://arxiv.org/html/2609.03003#S3.T1.4.3.1.1.1),[Table 2](https://arxiv.org/html/2609.03003#S3.T2.4.8.1.1),[§4\.2](https://arxiv.org/html/2609.03003#S4.SS2.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2609.03003#S4.p1.1),[§4](https://arxiv.org/html/2609.03003#S4.p3.1),[§5\.2\.4](https://arxiv.org/html/2609.03003#S5.SS2.SSS4.p2.1),[§5\.3](https://arxiv.org/html/2609.03003#S5.SS3.p2.1),[§5](https://arxiv.org/html/2609.03003#S5.p1.1)\.
- Battocchiet al\.\(2019\)K\. Battocchi, E\. Dillon, M\. Hei, G\. Lewis, P\. Oka, M\. Oprescu, and V\. SyrgkanisEconML: A Python Package for ML\-Based Heterogeneous Treatment Effects Estimation\.Note:https://github\.com/py\-why/EconMLVersion 0\.15\.0Cited by:[§4\.1](https://arxiv.org/html/2609.03003#S4.SS1.SSS0.Px2.p1.1)\.
- Bellot and Dhir \(2026\)A\. Bellot and A\. DhirFoundation Models for Partial Causal Identification\.arXiv:2608\.20841\.Cited by:[§5\.2\.3](https://arxiv.org/html/2609.03003#S5.SS2.SSS3.p1.1)\.
- Blöbaumet al\.\(2026\)P\. Blöbaum, K\. Balasubramanian, and S\. P\. KasiviswanathanFoundCause: Causal Discovery with Latent Confounders from Observational Data\.arXiv:2606\.17516\.Cited by:[§5\.3\.2](https://arxiv.org/html/2609.03003#S5.SS3.SSS2.p1.1)\.
- Böhmet al\.\(2026\)S\. Böhm, L\. Purucker, F\. Hutter, and P\. SchlosserSurvPFN: Towards Foundation Models for Survival Predictions\.arXiv:2606\.04564\.Cited by:[§5\.4\.2](https://arxiv.org/html/2609.03003#S5.SS4.SSS2.p1.1)\.
- Bottouet al\.\(2013\)L\. Bottou, J\. Peters, J\. Quiñonero\-Candela, D\. X\. Charles, D\. M\. Chickering, E\. Portugaly, D\. Ray, P\. Simard, and E\. SnelsonCounterfactual reasoning and learning systems: the example of computational advertising\.Journal of Machine Learning Research14\(101\),pp\. 3207–3260\.Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p1.1)\.
- Brownet al\.\(2020\)T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. AmodeiLanguage models are few\-shot learners\.InAdvances in Neural Information Processing Systems,Vol\.33\.Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p1.1),[§3\.1](https://arxiv.org/html/2609.03003#S3.SS1.p3.1)\.
- Bunneet al\.\(2023\)C\. Bunne, S\. G\. Stark, G\. Gut, J\. S\. del Castillo, M\. Levesque, K\. Lehmann, L\. Pelkmans, A\. Krause, and G\. RätschLearning single\-cell perturbation responses using neural optimal transport\.Nature Methods20\(11\)\.External Links:[Document](https://dx.doi.org/10.1038/s41592-023-01969-x),ISSN 1548\-7105Cited by:[§5\.4\.1](https://arxiv.org/html/2609.03003#S5.SS4.SSS1.p1.1)\.
- Bynumet al\.\(2025\)L\. E\. J\. Bynum, A\. M\. Puli, D\. Herrero\-Quevedo, N\. Nguyen, C\. Fernandez\-Granda, K\. Cho, and R\. RanganathBlack Box Causal Inference: Effect Estimation via Meta Prediction\.arXiv:2503\.05985\.Cited by:[§5\.1](https://arxiv.org/html/2609.03003#S5.SS1.p1.1)\.
- Chenet al\.\(2026\)Y\. Chen, Z\. Guan, H\. Qian, X\. Zhang, P\. Cui, Y\. Yang, F\. Wu, and K\. KuangDAG\-FM: A Foundation Model for Causal Discovery under Heterogeneous Causal Mechanisms\.arXiv:2607\.11510\.Cited by:[§5\.3\.2](https://arxiv.org/html/2609.03003#S5.SS3.SSS2.p2.1)\.
- Chernozhukovet al\.\(2018\)V\. Chernozhukov, D\. Chetverikov, M\. Demirer, E\. Duflo, C\. Hansen, W\. Newey, and J\. RobinsDouble/debiased machine learning for treatment and structural parameters\.The Econometrics Journal21\(1\)\.External Links:[Document](https://dx.doi.org/10.1111/ectj.12097)Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p1.1),[§1](https://arxiv.org/html/2609.03003#S1.p2.1),[§4\.1](https://arxiv.org/html/2609.03003#S4.SS1.SSS0.Px2.p1.1)\.
- Chipmanet al\.\(2010\)H\. A\. Chipman, E\. I\. George, and R\. E\. McCullochBART: Bayesian additive regression trees\.The Annals of Applied Statistics4\(1\)\.External Links:ISSN 1932\-6157,[Document](https://dx.doi.org/10.1214/09-aoas285)Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p2.1)\.
- Clarket al\.\(2003\)T\. G\. Clark, M\. J\. Bradburn, S\. B\. Love, and D\. G\. AltmanSurvival Analysis Part I: Basic Concepts and First Analyses\.British Journal of Cancer89,pp\. 232–238\.External Links:[Document](https://dx.doi.org/10.1038/sj.bjc.6601118)Cited by:[§5\.4\.2](https://arxiv.org/html/2609.03003#S5.SS4.SSS2.p1.1)\.
- de Vassimon Manelaet al\.\(2024\)D\. de Vassimon Manela, L\. Battaglia, and R\. J\. EvansMarginal Causal Flows for Validation and Inference\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,Cited by:[§4](https://arxiv.org/html/2609.03003#S4.p3.1)\.
- Dhiret al\.\(2025a\)A\. Dhir, M\. Ashman, J\. Requeima, and M\. van der WilkA Meta\-Learning Approach to Bayesian Causal Discovery\.InInternational Conference on Learning Representations,pp\. 14158–14178\.Cited by:[§5\.3\.1](https://arxiv.org/html/2609.03003#S5.SS3.SSS1.p2.1)\.
- Dhiret al\.\(2025b\)A\. Dhir, C\. Diaconu, V\. Lungu, J\. Requeima, R\. Turner, and M\. van der WilkEstimating Interventional Distributions with Uncertain Causal Graphs through Meta\-Learning\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 140060–140096\.External Links:[Document](https://dx.doi.org/10.52202/085713-4681)Cited by:[§3\.1](https://arxiv.org/html/2609.03003#S3.SS1.p6.1),[§4\.1](https://arxiv.org/html/2609.03003#S4.SS1.SSS0.Px2.p2.1)\.
- Dooleyet al\.\(2023\)S\. Dooley, G\. S\. Khurana, C\. Mohapatra, S\. V\. Naidu, and C\. WhiteForecastPFN: Synthetically\-Trained Zero\-Shot Forecasting\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 2403–2426\.External Links:[Document](https://dx.doi.org/10.52202/075280-0112)Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p8.1)\.
- Finnet al\.\(2017\)C\. Finn, P\. Abbeel, and S\. LevineModel\-Agnostic Meta\-Learning for Fast Adaptation of Deep Networks\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 1126–1135\.Cited by:[footnote 1](https://arxiv.org/html/2609.03003#footnote1)\.
- Frauenet al\.\(2026\)D\. Frauen, V\. Melnychuk, L\. van der Laan, and S\. FeuerriegelMachine learning for causal inference\.InWiley StatsRef: Statistics Reference Online,pp\. 1–17\.External Links:ISBN 9781118445112,[Document](https://dx.doi.org/10.1002/9781118445112.stat08670)Cited by:[footnote 1](https://arxiv.org/html/2609.03003#footnote1)\.
- Gaoet al\.\(2026\)Y\. Gao, J\. M\. Hernández\-Lobato, and S\. GuoPerturbPFN: Probing the Limits of Synthetic Priors in Drug Perturbation Modelling\.arXiv:2607\.23447\.Cited by:[§5\.4\.1](https://arxiv.org/html/2609.03003#S5.SS4.SSS1.p1.1)\.
- Garget al\.\(2025\)A\. Garg, M\. Ali, N\. Hollmann, L\. Purucker, S\. Müller, and F\. HutterReal\-TabPFN: Improving Tabular Foundation Models via Continued Pre\-training With Real\-World Data\.arXiv:2507\.03971\.Cited by:[§3\.1\.2](https://arxiv.org/html/2609.03003#S3.SS1.SSS2.p2.1)\.
- Gelmanet al\.\(2013\)A\. Gelman, J\.B\. Carlin, H\.S\. Stern, D\.B\. Dunson, A\. Vehtari, and D\.B\. RubinBayesian Data Analysis\.3rd edition,Chapman & Hall/CRC Texts in Statistical Science,Taylor & Francis\.External Links:ISBN 9781439840955Cited by:[§2\.5](https://arxiv.org/html/2609.03003#S2.SS5.p5.1)\.
- Glymouret al\.\(2019\)C\. Glymour, K\. Zhang, and P\. SpirtesReview of Causal Discovery Methods Based on Graphical Models\.Frontiers in GeneticsVolume 10\.External Links:[Document](https://dx.doi.org/10.3389/fgene.2019.00524),ISSN 1664\-8021Cited by:[§5\.3](https://arxiv.org/html/2609.03003#S5.SS3.p1.1)\.
- Gordonet al\.\(2019\)B\. R\. Gordon, F\. Zettelmeyer, N\. Bhargava, and D\. ChapskyA comparison of approaches to advertising measurement: evidence from big field experiments at Facebook\.Marketing Science38\(2\),pp\. 193–225\.External Links:[Document](https://dx.doi.org/10.1287/mksc.2018.1135)Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p1.1)\.
- Grinsztajnet al\.\(2026\)L\. Grinsztajn, K\. Flöge, O\. Key, F\. Birkel, P\. Jund, B\. Roof, M\. Manium, S\. B\. Hoo, M\. Bühler, A\. Garg, D\. Safaric, J\. Robertson, B\. Jäger, S\. Alessi, A\. Hayler, V\. Moroshan, L\. Purucker, P\. Singer, A\. Arazi, J\. Siems, J\. H\. Metzen, G\. Grab, N\. Erickson, S\. Guo, E\. Kalfon, S\. Bing, D\. Salinas, C\. Cornu, L\. C\. Wehrhahn, D\. Kriuchkova, K\. Kaya, L\. Sidhoum, M\. Salmon, J\. Chen, M\. Hulsebos, Y\. LeCun, S\. Müller, B\. Schölkopf, S\. Gambhir, N\. Hollmann, and F\. HutterTabPFN\-3: technical report\.arXiv:2605\.13986\.Cited by:[Table 2](https://arxiv.org/html/2609.03003#S3.T2.4.4.1.1)\.
- Guanet al\.\(2026\)Z\. Guan, Y\. Chen, Y\. He, Y\. Tong, Z\. Hu, H\. Qian, F\. Wu, and K\. KuangDCD\-PFN: A Decoupling\-Aware Foundation Model for Causal Discovery\.arXiv:2606\.21212\.Cited by:[§5\.3\.2](https://arxiv.org/html/2609.03003#S5.SS3.SSS2.p2.1)\.
- Hamet al\.\(2026\)J\. Ham, D\. Kim, D\. Kim, S\. Kim, and S\. LeeA structural view of query misspecification in causal foundation models\.InICML 2026 Workshop on Structured Probabilistic Inference & Generative Modeling,Cited by:[§5\.2\.4](https://arxiv.org/html/2609.03003#S5.SS2.SSS4.p2.1)\.
- Hayleret al\.\(2026\)A\. Hayler, K\. Flöge, A\. Arazi, R\. Ranjan, J\. Leskovec, F\. Birkel, B\. Roof, A\. Garg, K\. Collins, L\. Sidhoum, J\. Kübler, S\. Guo, O\. Key, J\. H\. Metzen, R\. Grace, D\. Salinas, A\. Cahu, S\. Bing, B\. Jäger, T\. Çelik, M\. Manium, V\. Monteiro, J\. Robertson, J\. Chen, E\. Kalfon, T\. Pereda, L\. Wehrhahn, D\. Safaric, T\. Schroeder, G\. Grab, D\. Kriuchkova, C\. Cornu, P\. Singer, N\. Erickson, V\. Balazadeh, M\. Salmon, S\. Alessi, K\. Kaya, P\. Jund, L\. Grinsztajn, Y\. LeCun, B\. Schölkopf, M\. Hulsebos, L\. Purucker, S\. Gambhir, F\. Hutter, and N\. HollmannAdvancing Open and Reproducible Relational Learning: RelArena\-α\\alpha, TabPFN\-Rel and RPI\.arXiv:2608\.16319\.Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p8.1)\.
- Hoffman and Gelman \(2014\)M\. D\. Hoffman and A\. GelmanThe No\-U\-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo\.Journal of Machine Learning Research15\(47\),pp\. 1593–1623\.Cited by:[§2\.5](https://arxiv.org/html/2609.03003#S2.SS5.p5.1)\.
- Holland \(1986\)P\. W\. HollandStatistics and Causal Inference\.Journal of the American Statistical Association81\(396\),pp\. 945–960\.External Links:ISSN 01621459, 1537274XCited by:[§2\.2](https://arxiv.org/html/2609.03003#S2.SS2.p5.1)\.
- Hollmannet al\.\(2023\)N\. Hollmann, S\. Müller, K\. Eggensperger, and F\. HutterTabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second\.InThe Eleventh International Conference on Learning Representations,Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p8.1),[§3\.1\.3](https://arxiv.org/html/2609.03003#S3.SS1.SSS3.p1.1),[§3\.2](https://arxiv.org/html/2609.03003#S3.SS2.p2.1)\.
- Hollmannet al\.\(2025\)N\. Hollmann, S\. Müller, L\. Purucker, A\. Krishnakumar, M\. Körfer, S\. B\. Hoo, R\. T\. Schirrmeister, and F\. HutterAccurate predictions on small data with a tabular foundation model\.Nature637\(8045\),pp\. 319–326\.Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p8.1)\.
- Hosseinzadehet al\.\(2026\)R\. Hosseinzadeh, A\. Labach, Z\. Xue, S\. Han, V\. Thomas, and A\. L\. CateriniTabDPT\-Turbo: Efficient In\-Context Learning for Tabular Prediction\.arXiv:2608\.01400\.Cited by:[Table 2](https://arxiv.org/html/2609.03003#S3.T2.4.5.1.1)\.
- Huang and Valtorta \(2006\)Y\. Huang and M\. ValtortaPearl’s calculus of intervention is complete\.InProceedings of the Twenty\-Second Conference on Uncertainty in Artificial Intelligence,pp\. 217–224\.External Links:ISBN 0974903922Cited by:[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p8.1)\.
- Imbens and Rubin \(2015\)G\. W\. Imbens and D\. B\. RubinCausal inference for statistics, social, and biomedical sciences: an introduction\.Cambridge University Press\.Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p1.1),[§2\.2](https://arxiv.org/html/2609.03003#S2.SS2.p1.1),[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p9.1)\.
- Imbens \(2020\)G\. W\. ImbensPotential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics\.Journal of Economic Literature58\(4\),pp\. 1129–79\.External Links:[Document](https://dx.doi.org/10.1257/jel.20191597)Cited by:[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p13.1)\.
- Javureket al\.\(2026\)E\. Javurek, D\. Frauen, M\. Brockschmidt, J\. Schweisthal, and S\. FeuerriegelAmortizing causal sensitivity analysis via prior data\-fitted networks\.arXiv:2605\.10590\.Cited by:[§5\.2\.3](https://arxiv.org/html/2609.03003#S5.SS2.SSS3.p2.1)\.
- Jordanet al\.\(1999\)M\. I\. Jordan, Z\. Ghahramani, T\. S\. Jaakkola, and L\. K\. SaulAn Introduction to Variational Methods for Graphical Models\.Machine Learning37\(2\),pp\. 183–233\.External Links:[Document](https://dx.doi.org/10.1023/A%3A1007665907178)Cited by:[§2\.5](https://arxiv.org/html/2609.03003#S2.SS5.p5.1)\.
- Kalisch and Bühlmann \(2007\)M\. Kalisch and P\. BühlmannEstimating High\-Dimensional Directed Acyclic Graphs with the PC\-Algorithm\.Journal of Machine Learning Research8\(22\),pp\. 613–636\.Cited by:[§5\.3](https://arxiv.org/html/2609.03003#S5.SS3.p1.1)\.
- Keet al\.\(2023\)N\. R\. Ke, S\. Chiappa, J\. X\. Wang, J\. Bornschein, A\. Goyal, M\. Rey, T\. Weber, M\. Botvinick, M\. C\. Mozer, and D\. J\. RezendeLearning to Induce Causal Structure\.InInternational Conference on Learning Representations,Cited by:[§5\.3\.1](https://arxiv.org/html/2609.03003#S5.SS3.SSS1.p1.1)\.
- Kennedy \(2023\)E\. H\. KennedyTowards optimal doubly robust estimation of heterogeneous causal effects\.Electronic Journal of Statistics17\(2\),pp\. 3008–3049\.Cited by:[§4\.1](https://arxiv.org/html/2609.03003#S4.SS1.SSS0.Px2.p1.1)\.
- Kimet al\.\(2026\)D\. I\. Kim, W\. S\. Lai, and K\. W\. ZhangTabular foundation models can do survival analysis\.arXiv:2601\.22259\.Cited by:[§5\.4\.2](https://arxiv.org/html/2609.03003#S5.SS4.SSS2.p1.1)\.
- Klein and Moeschberger \(2003\)J\. P\. Klein and M\. L\. MoeschbergerSurvival Analysis: Techniques for Censored and Truncated Data\.2nd edition,Springer,New York\.External Links:[Document](https://dx.doi.org/10.1007/b97377)Cited by:[§5\.4\.2](https://arxiv.org/html/2609.03003#S5.SS4.SSS2.p1.1)\.
- Kougiouliset al\.\(2026\)N\. Kougioulis, N\. Gkorgkolis, M\. Wang, B\. Caglayan, D\. Simionato, A\. Tonon, and I\. TsamardinosLarge Causal Models for Temporal Causal Discovery\.InJoint European Conference on Machine Learning and Knowledge Discovery in Databases,Cited by:[§5\.3\.2](https://arxiv.org/html/2609.03003#S5.SS3.SSS2.p3.1)\.
- Künzelet al\.\(2019\)S\. R\. Künzel, J\. S\. Sekhon, P\. J\. Bickel, and B\. YuMetalearners for estimating heterogeneous treatment effects using machine learning\.Proceedings of the National Academy of Sciences116\(10\),pp\. 4156–4165\.External Links:[Document](https://dx.doi.org/10.1073/pnas.1804597116)Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p2.1),[§4\.1](https://arxiv.org/html/2609.03003#S4.SS1.SSS0.Px2.p1.1),[footnote 1](https://arxiv.org/html/2609.03003#footnote1)\.
- LaLonde \(1986\)R\. J\. LaLondeEvaluating the Econometric Evaluations of Training Programs with Experimental Data\.The American Economic Review76\(4\),pp\. 604–620\.Cited by:[§4\.1](https://arxiv.org/html/2609.03003#S4.SS1.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2609.03003#S4.p2.1),[§4](https://arxiv.org/html/2609.03003#S4.p3.1)\.
- Liet al\.\(2026\)Z\. Li, S\. Liu, T\. Wang, and H\. YeTabCausal: Pretraining Across Causal Environments for Tabular Causal Discovery\.arXiv:2605\.31156\.Cited by:[§5\.3\.2](https://arxiv.org/html/2609.03003#S5.SS3.SSS2.p1.1)\.
- Lorchet al\.\(2022\)L\. Lorch, S\. Sussex, J\. Rothfuss, A\. Krause, and B\. SchölkopfAmortized Inference for Causal Structure Learning\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 13104–13118\.External Links:[Document](https://dx.doi.org/10.52202/068431-0952)Cited by:[§5\.3\.1](https://arxiv.org/html/2609.03003#S5.SS3.SSS1.p1.1)\.
- Maet al\.\(2025\)J\. Ma, V\. Thomas, R\. Hosseinzadeh, A\. Labach, H\. Kamkari, J\. C\. Cresswell, K\. Golestan, G\. Yu, A\. L\. Caterini, and M\. VolkovsTabDPT: Scaling Tabular Foundation Models on Real Data\.InAdvances in Neural Information Processing Systems,Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p8.1),[§3\.1\.2](https://arxiv.org/html/2609.03003#S3.SS1.SSS2.p2.1)\.
- Maet al\.\(2026\)Y\. Ma, D\. Frauen, E\. Javurek, and S\. FeuerriegelFoundation models for causal inference via prior\-data fitted networks\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 79065–79098\.Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p6.1),[§3\.1\.2](https://arxiv.org/html/2609.03003#S3.SS1.SSS2.p4.1),[§3\.1](https://arxiv.org/html/2609.03003#S3.SS1.p6.1),[§3\.2](https://arxiv.org/html/2609.03003#S3.SS2.p2.1),[Table 1](https://arxiv.org/html/2609.03003#S3.T1.4.4.1.1.1),[Table 2](https://arxiv.org/html/2609.03003#S3.T2.4.9.1.1),[§4](https://arxiv.org/html/2609.03003#S4.p1.1),[§4](https://arxiv.org/html/2609.03003#S4.p2.1),[§5](https://arxiv.org/html/2609.03003#S5.p1.1)\.
- Mahajanet al\.\(2024\)D\. Mahajan, I\. Mitliagkas, B\. Neal, and V\. SyrgkanisEmpirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation\.InThe Twelfth International Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2609.03003#S4.p3.1)\.
- Manski \(2003\)C\. F\. ManskiPartial identification of probability distributions\.Springer Series in Statistics,Springer,New York, NY\.External Links:ISBN 978\-0\-387\-00454\-9Cited by:[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p12.1)\.
- Matthews \(2000\)R\. MatthewsStorks deliver babies \(p=0\.008p=0\.008\)\.Teaching Statistics22,pp\. 36–38\.External Links:[Document](https://dx.doi.org/10.1111/1467-9639.00013)Cited by:[§2\.1](https://arxiv.org/html/2609.03003#S2.SS1.p1.1)\.
- Meeker and Escobar \(1998\)W\. Q\. Meeker and L\. A\. EscobarStatistical methods for reliability data\.John Wiley & Sons\.Cited by:[§5\.4\.2](https://arxiv.org/html/2609.03003#S5.SS4.SSS2.p1.1)\.
- Melnychuket al\.\(2026\)V\. Melnychuk, V\. Balazadeh, S\. Feuerriegel, and R\. G\. KrishnanFrequentist Consistency of Prior\-Data Fitted Networks for Causal Inference\.InProceedings of the 43nd International Conference on Machine Learning,Proceedings of Machine Learning Research\.Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p1.1),[§5\.2\.4](https://arxiv.org/html/2609.03003#S5.SS2.SSS4.p2.1)\.
- Montagnaet al\.\(2025\)F\. Montagna, M\. Cairney\-Leeming, D\. Sridhar, and F\. LocatelloDemystifying amortized causal discovery with transformers\.Transactions on Machine Learning Research\.External Links:ISSN 2835\-8856Cited by:[§5\.3\.1](https://arxiv.org/html/2609.03003#S5.SS3.SSS1.p1.1)\.
- Mooijet al\.\(2016\)J\. M\. Mooij, J\. Peters, D\. Janzing, J\. Zscheischler, and B\. SchölkopfDistinguishing Cause from Effect Using Observational Data: Methods and Benchmarks\.Journal of Machine Learning Research17\(32\),pp\. 1–102\.Cited by:[§2\.1](https://arxiv.org/html/2609.03003#S2.SS1.p1.1)\.
- Mouraoet al\.\(2026\)F\. Mourao, D\. Hajage, D\. Bystrova, B\. Bouvarel, N\. Lapidus, F\. Carrat, and B\. GlemainPrior\-Data Fitted Networks for Causal Inference: a Simulation Study with Real\-World Scenarios\.arXiv:2603\.15928\.Cited by:[§5\.2\.4](https://arxiv.org/html/2609.03003#S5.SS2.SSS4.p2.1)\.
- Mülleret al\.\(2022\)S\. Müller, N\. Hollmann, S\. P\. Arango, J\. Grabocka, and F\. HutterTransformers Can Do Bayesian Inference\.InInternational Conference on Learning Representations,Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p1.1),[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p4.2),[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p8.1)\.
- Nealet al\.\(2020\)B\. Neal, C\. Huang, and S\. RaghupathiRealCause: Realistic Causal Inference Benchmarking\.arXiv:2011\.15007\.Cited by:[§4\.1](https://arxiv.org/html/2609.03003#S4.SS1.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2609.03003#S4.p3.1)\.
- Neal \(2020\)B\. NealIntroduction to causal inference from a machine learning perspective\.Note:[https://www\.bradyneal\.com/Introduction\_to\_Causal\_Inference\-Dec17\_2020\-Neal\.pdf](https://www.bradyneal.com/Introduction_to_Causal_Inference-Dec17_2020-Neal.pdf)Cited by:[§2\.2](https://arxiv.org/html/2609.03003#S2.SS2.p4.2),[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p11.1),[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p12.1),[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p13.1),[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p9.1),[§5\.3](https://arxiv.org/html/2609.03003#S5.SS3.p1.1),[footnote 3](https://arxiv.org/html/2609.03003#footnote3)\.
- Neal \(1996\)R\. M\. NealBayesian learning for neural networks\.Lecture Notes in Statistics,Springer New York\.External Links:ISBN 978\-0\-387\-94724\-2,[Document](https://dx.doi.org/10.1007/978-1-4612-0745-0)Cited by:[§2\.5](https://arxiv.org/html/2609.03003#S2.SS5.p5.1)\.
- Nilforoshanet al\.\(2023\)H\. Nilforoshan, M\. Moor, Y\. Roohani, Y\. Chen, A\. Šurina, M\. Yasunaga, S\. Oblak, and J\. LeskovecZero\-shot causal learning\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 6862–6901\.Cited by:[§5\.1](https://arxiv.org/html/2609.03003#S5.SS1.p1.1)\.
- Papamakarios and Murray \(2016\)G\. Papamakarios and I\. MurrayFastϵ\\epsilon\-free Inference of Simulation Models with Bayesian Conditional Density Estimation\.InAdvances in Neural Information Processing Systems,Vol\.29\.Cited by:[§2\.5](https://arxiv.org/html/2609.03003#S2.SS5.p5.1)\.
- Pearl \(1995\)J\. PearlCausal Diagrams for Empirical Research\.Biometrika82\(4\),pp\. 669–688\.External Links:ISSN 00063444, 14643510Cited by:[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p8.1)\.
- Pearl \(2009a\)J\. PearlCausal inference in statistics: An overview\.Statistics Surveys3,pp\. 96 – 146\.External Links:[Document](https://dx.doi.org/10.1214/09-SS057)Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p1.1)\.
- Pearl \(2009b\)J\. PearlCausality\.2nd edition,Cambridge University Press\.Cited by:[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p1.3),[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p11.1),[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p11.3),[§2\.4](https://arxiv.org/html/2609.03003#S2.SS4.p2.1),[§5\.2\.1](https://arxiv.org/html/2609.03003#S5.SS2.SSS1.p2.1)\.
- Qiet al\.\(2026\)S\. Qi, V\. Balazadeh, M\. Cooper, R\. Greiner, and R\. G\. KrishnanSurvivalPFN: Amortizing Survival Prediction via In\-Context Bayesian Inference\.arXiv:2605\.15488\.Cited by:[§5\.4\.2](https://arxiv.org/html/2609.03003#S5.SS4.SSS2.p1.1)\.
- Qiaoet al\.\(2026\)J\. Qiao, R\. Cai, Z\. Li, W\. Chen, P\. Hua, B\. Xu, Z\. Chen, Z\. Hao, and P\. CuiCDFM: Towards a General\-Purpose Causal Discovery Foundation Model\.arXiv:2607\.11508\.Cited by:[§5\.3\.2](https://arxiv.org/html/2609.03003#S5.SS3.SSS2.p2.1)\.
- Quet al\.\(2025\)J\. Qu, D\. Holzmüller, G\. Varoquaux, and M\. Le MorvanTabICL: A Tabular Foundation Model for In\-Context Learning on Large Data\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 50817–50847\.Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p8.1)\.
- Quet al\.\(2026\)J\. Qu, D\. Holzmüller, G\. Varoquaux, and M\. L\. MorvanTabICLv2: a better, faster, scalable, and open tabular foundation model\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research\.Cited by:[Table 2](https://arxiv.org/html/2609.03003#S3.T2.4.3.1.1)\.
- Reuteret al\.\(2026\)A\. Reuter, A\. Dhir, C\. Diaconu, J\. Robertson, O\. Ossen, F\. Hutter, A\. Weller, M\. van der Wilk, and B\. SchölkopfUse What You Know: Causal Foundation Models with Partial Graphs\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research\.Cited by:[§5\.2\.1](https://arxiv.org/html/2609.03003#S5.SS2.SSS1.p1.1)\.
- Robertsonet al\.\(2025\)J\. Robertson, A\. Reuter, S\. Guo, N\. Hollmann, F\. Hutter, and B\. SchölkopfDo\-PFN: In\-Context Learning for Causal Effect Estimation\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 174811–174848\.Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p6.1),[§3\.1\.1](https://arxiv.org/html/2609.03003#S3.SS1.SSS1.p6.1),[§3\.1\.2](https://arxiv.org/html/2609.03003#S3.SS1.SSS2.p4.1),[§3\.1](https://arxiv.org/html/2609.03003#S3.SS1.p5.1),[§3\.1](https://arxiv.org/html/2609.03003#S3.SS1.p6.1),[§3\.2](https://arxiv.org/html/2609.03003#S3.SS2.p2.1),[Table 1](https://arxiv.org/html/2609.03003#S3.T1.4.2.1.1.1),[Table 2](https://arxiv.org/html/2609.03003#S3.T2.4.7.1.1),[§4](https://arxiv.org/html/2609.03003#S4.p1.1),[§4](https://arxiv.org/html/2609.03003#S4.p3.1),[§5](https://arxiv.org/html/2609.03003#S5.p1.1)\.
- Robinset al\.\(1994\)J\. M\. Robins, A\. Rotnitzky, and L\. P\. ZhaoEstimation of regression coefficients when some regressors are not always observed\.Journal of the American Statistical Association89\(427\),pp\. 846–866\.Cited by:[§4\.1](https://arxiv.org/html/2609.03003#S4.SS1.SSS0.Px2.p1.1)\.
- Rosenbaum \(2002\)P\. R\. RosenbaumObservational studies\.2 edition,Springer,New York\.External Links:[Document](https://dx.doi.org/10.1007/978-1-4757-3692-2)Cited by:[§5\.2\.3](https://arxiv.org/html/2609.03003#S5.SS2.SSS3.p2.1)\.
- Rubin \(2005\)D\. B\. RubinCausal Inference Using Potential Outcomes\.Journal of the American Statistical Association100\(469\),pp\. 322–331\.External Links:[Document](https://dx.doi.org/10.1198/016214504000001880)Cited by:[§2\.2](https://arxiv.org/html/2609.03003#S2.SS2.p1.1)\.
- Schwabet al\.\(2020\)P\. Schwab, L\. Linhardt, S\. Bauer, J\. M\. Buhmann, and W\. KarlenLearning Counterfactual Representations for Estimating Individual Dose\-Response Curves\.Proceedings of the AAAI Conference on Artificial Intelligence34\(04\),pp\. 5612–5619\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v34i04.6014)Cited by:[4th item](https://arxiv.org/html/2609.03003#S2.I2.i4.p1.2)\.
- Seletkovet al\.\(2026\)D\. Seletkov, P\. Hager, G\. Kaissis, R\. Braren, D\. Rueckert, and R\. RehmsSurvival In\-Context: Amortized Bayesian Survival Analysis via Prior\-Fitted Networks\.arXiv:2603\.29475\.Cited by:[§5\.4\.2](https://arxiv.org/html/2609.03003#S5.SS4.SSS2.p1.1)\.
- Sextroet al\.\(2026\)M\. Sextro, W\. Kłos, and G\. DernbachMapPFN: Learning Causal Perturbation Maps in Context\.arXiv:2601\.21092\.Cited by:[§5\.4\.1](https://arxiv.org/html/2609.03003#S5.SS4.SSS1.p1.1)\.
- Shalitet al\.\(2017\)U\. Shalit, F\. D\. Johansson, and D\. SontagEstimating individual treatment effect: Generalization bounds and algorithms\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 3076–3085\.Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p1.1)\.
- Shpitser and Pearl \(2006\)I\. Shpitser and J\. PearlIdentification of joint interventional distributions in recursive semi\-Markovian causal models\.InProceedings of the 21st National Conference on Artificial Intelligence \- Volume 2,AAAI’06,pp\. 1219–1226\.External Links:ISBN 9781577352815Cited by:[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p8.1)\.
- Smith and Todd \(2005\)J\. A\. Smith and P\. E\. ToddDoes matching overcome LaLonde’s critique of nonexperimental estimators?\.Journal of Econometrics125\(1–2\),pp\. 305–353\.External Links:ISSN 0304\-4076,[Document](https://dx.doi.org/10.1016/j.jeconom.2004.04.011)Cited by:[§4](https://arxiv.org/html/2609.03003#S4.p2.1)\.
- Spinaciet al\.\(2025\)M\. Spinaci, M\. Polewczyk, M\. Schambach, and S\. ThelinConTextTab: A Semantics\-Aware Tabular In\-Context Learner\.InAdvances in Neural Information Processing Systems,Vol\.38\.External Links:[Document](https://dx.doi.org/10.52202/085713-4918)Cited by:[§3\.1\.2](https://arxiv.org/html/2609.03003#S3.SS1.SSS2.p2.1)\.
- Stithet al\.\(2026\)C\. Stith, M\. Barath, V\. Balazadeh, J\. C\. Cresswell, and R\. G\. KrishnanCausal foundation models with continuous treatments\.arXiv:2605\.15133\.Cited by:[§5\.2\.2](https://arxiv.org/html/2609.03003#S5.SS2.SSS2.p1.1)\.
- Sypniewskiet al\.\(2025\)M\. Sypniewski, M\. Olko, M\. Gajewski, and P\. MiłośAmortized Causal Discovery with Prior\-Fitted Networks\.arXiv:2512\.11840\.Cited by:[§5\.3\.1](https://arxiv.org/html/2609.03003#S5.SS3.SSS1.p3.1)\.
- Tagaet al\.\(2025\)E\. O\. Taga, M\. E\. Ildiz, and S\. OymakTimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 20761–20769\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v39i19.34288)Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p8.1)\.
- Thompsonet al\.\(2026\)R\. Thompson, H\. Zhao, D\. M\. Steinberg, and E\. V\. BonillaArrow: A Foundation Model for Causal Discovery\.arXiv:2605\.07204\.Cited by:[§5\.3\.2](https://arxiv.org/html/2609.03003#S5.SS3.SSS2.p1.1)\.
- Thumm and Chen \(2026\)D\. Thumm and Y\. ChenInterventional Time Series Priors for Causal Foundation Models\.arXiv:2603\.11090\.Cited by:[§5\.2\.2](https://arxiv.org/html/2609.03003#S5.SS2.SSS2.p3.1)\.
- Thummet al\.\(2026a\)D\. Thumm, A\. Reuter, J\. Robertson, S\. B\. Hoo, A\. Weller, F\. Hutter, Y\. Chen, and B\. SchölkopfCausal foundation models for time series based on prior\-data fitted networks\.In2nd ICML Workshop on Foundation Models for Structured Data,Cited by:[§5\.2\.2](https://arxiv.org/html/2609.03003#S5.SS2.SSS2.p3.1)\.
- Thummet al\.\(2026b\)D\. Thumm, R\. Wiedemann, and Y\. ChenTowards Continuous\-time Causal Foundation Models\.arXiv:2605\.28880\.Cited by:[§5\.2\.2](https://arxiv.org/html/2609.03003#S5.SS2.SSS2.p3.1)\.
- Türkmenet al\.\(2026\)Z\. Türkmen, K\. Kaya, A\. Pfefferle, and F\. HutterTowards Evaluating Data Priors for Tabular Foundation Models\.arXiv:2606\.29241\.Cited by:[§3\.2](https://arxiv.org/html/2609.03003#S3.SS2.p7.1)\.
- Van den Poel and Larivière \(2004\)D\. Van den Poel and B\. LarivièreCustomer attrition analysis for financial services using proportional hazard models\.European Journal of Operational Research157\(1\),pp\. 196–217\.External Links:ISSN 0377\-2217,[Document](https://dx.doi.org/10.1016/S0377-2217%2803%2900069-9)Cited by:[§5\.4\.2](https://arxiv.org/html/2609.03003#S5.SS4.SSS2.p1.1)\.
- van der Laanet al\.\(2026\)L\. van der Laan, A\. Luedtke, and M\. CaroneDoubly robust inference via calibration\.arXiv:2411\.02771\.Cited by:[§4](https://arxiv.org/html/2609.03003#S4.p3.1)\.
- Vaswaniet al\.\(2017\)A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. PolosukhinAttention is all you need\.InAdvances in Neural Information Processing Systems,Vol\.30\.Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p1.1)\.
- Wager and Athey \(2018\)S\. Wager and S\. AtheyEstimation and inference of heterogeneous treatment effects using random forests\.Journal of the American Statistical Association113\(523\),pp\. 1228–1242\.External Links:[Document](https://dx.doi.org/10.1080/01621459.2017.1319839)Cited by:[§1](https://arxiv.org/html/2609.03003#S1.p2.1)\.
- Wainwright and Jordan \(2008\)M\. J\. Wainwright and M\. I\. JordanGraphical Models, Exponential Families, and Variational Inference\.Foundations and Trends in Machine Learning1\(1–2\),pp\. 1–305\.External Links:[Document](https://dx.doi.org/10.1561/2200000001)Cited by:[§2\.5](https://arxiv.org/html/2609.03003#S2.SS5.p5.1)\.
- Wanget al\.\(2021\)C\. Wang, Q\. Wu, M\. Weimer, and E\. ZhuFLAML: a fast and lightweight automl library\.arXiv:1911\.04706\.Cited by:[§4\.1](https://arxiv.org/html/2609.03003#S4.SS1.SSS0.Px2.p1.1)\.
- Wanget al\.\(2026a\)H\. Wang, X\. Lv, H\. Zou, Y\. Xiao, S\. Gu, Y\. Shi, Y\. Mao, Y\. Zhang, M\. Geng, S\. Yang, H\. Li, W\. Yang, P\. Cui, and Z\. LinUnveiling prior\-data fitted networks on causal effect estimation: pre\-training or fine\-tuning?\.InForty\-third International Conference on Machine Learning,Cited by:[§5\.2\.4](https://arxiv.org/html/2609.03003#S5.SS2.SSS4.p2.1)\.
- Wanget al\.\(2026b\)Y\. Wang, J\. You, C\. Shi, and M\. ZhangRelational In\-Context Learning via Synthetic Pre\-training with Structural Prior\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research\.Cited by:[§2\.6](https://arxiv.org/html/2609.03003#S2.SS6.p8.1)\.
- Wuet al\.\(2025\)M\. Wu, Y\. Bao, R\. Barzilay, and T\. JaakkolaSample, estimate, aggregate: A recipe for causal discovery foundation models\.Transactions on Machine Learning Research\.External Links:ISSN 2835\-8856Cited by:[§5\.3\.1](https://arxiv.org/html/2609.03003#S5.SS3.SSS1.p3.1)\.
- Yaoet al\.\(2021\)L\. Yao, Z\. Chu, S\. Li, Y\. Li, J\. Gao, and A\. ZhangA Survey on Causal Inference\.ACM Trans\. Knowl\. Discov\. Data15\(5\)\.External Links:ISSN 1556\-4681,[Document](https://dx.doi.org/10.1145/3444944)Cited by:[§2\.3](https://arxiv.org/html/2609.03003#S2.SS3.p9.1)\.
- Yinet al\.\(2025\)N\. Yin, T\. Gao, and Y\. YuLearning Causal Graphs at Scale: A Foundation Model Approach\.arXiv:2506\.18285\.Cited by:[§5\.3\.1](https://arxiv.org/html/2609.03003#S5.SS3.SSS1.p3.1)\.
- Zangaet al\.\(2022\)A\. Zanga, E\. Ozkirimli, and F\. StellaA survey on causal discovery: Theory and practice\.International Journal of Approximate Reasoning151,pp\. 101–129\.Cited by:[§5\.3](https://arxiv.org/html/2609.03003#S5.SS3.p1.1)\.
- Zareet al\.\(2026\)A\. Zare, A\. Zare, H\. Rahimi, R\. Salarikia, and M\. KashkooliCausal Longitudinal Prior\-Fitted Networks for Counterfactual Outcome Prediction\.arXiv:2606\.05797\.Cited by:[§5\.2\.2](https://arxiv.org/html/2609.03003#S5.SS2.SSS2.p2.1)\.
- Zhanget al\.\(2023\)J\. Zhang, J\. Jennings, A\. Hilmkil, N\. Pawlowski, C\. Zhang, and C\. MaTowards Causal Foundation Model: on Duality between Causal Inference and Attention\.arXiv:2310\.00809\.Cited by:[§5\.1](https://arxiv.org/html/2609.03003#S5.SS1.p1.1)\.
- Zhanget al\.\(2025\)X\. Zhang, D\. Maddix Robinson, J\. Yin, N\. Erickson, A\. F\. Ansari, B\. Han, S\. Zhang, L\. Akoglu, C\. Faloutsos, M\. Mahoney, T\. Hu, H\. Rangwala, G\. Karypis, and Y\. \(\. WangMitra: mixed synthetic priors for enhancing tabular foundation models\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 15795–15840\.External Links:[Document](https://dx.doi.org/10.52202/085713-0535)Cited by:[§3\.2](https://arxiv.org/html/2609.03003#S3.SS2.p7.1)\.
- Zhuet al\.\(2026\)M\. Zhu, M\. Mansoldo, C\. Wang, and S\. GrohaA Causal Foundation Model for Structure and Outcome Prediction\.arXiv:2606\.26467\.Cited by:[§5\.2\.1](https://arxiv.org/html/2609.03003#S5.SS2.SSS1.p2.1),[§5\.3\.2](https://arxiv.org/html/2609.03003#S5.SS3.SSS2.p2.1)\.