The Illusion of Intervention: Your LLM-Simulated Experiment is an Observational Study

# The Illusion of Intervention: Your LLM-Simulated Experiment is an Observational Study
Source: [https://arxiv.org/html/2605.20767](https://arxiv.org/html/2605.20767)
Victoria Lin1,2Taedong Yun1Maja Matarić1John Canny1 Arthur Gretton1Alexander D’Amour1

1Google DeepMind2Carnegie Mellon University

###### Abstract

Large language models \(LLMs\) show potential as simulators of human behavior, offering a scalable way to study responses to interventions\. However, because LLMs are trained largely on observational data, interventions in experiments with LLM\-simulated synthetic users can induce unintended shifts in latent user attributes, causing user drift where the implicit simulated population differs across treatment conditions, potentially distorting effect estimates\. We formalize the confounding orselection biasthat can arise due to user drift and show how intervention\-dependent shifts can inflate or attenuate observed differences in user responses under intervention\. To diagnose confounding, we propose using negative control outcomes—attributes that should remain invariant under intervention—to identify distribution shifts across intervention conditions, providing evidence of user drift\. To mitigate drift, we study adjusting the persona specification by eliciting additional confounders, finding that targeted, setting\-relevant confounders can substantially reduce bias across survey\-style and multi\-turn agent evaluations\.

## 1Introduction

Large language models \(LLMs\) are emerging as powerful tools for simulating human behavior\. Traditionally, a multitude of scientific applications rely on human feedback\. Large\-scale surveys underpin many social science and public policy studies, and modern machine learning systems are trained and evaluated on human response data\. Because collecting such feedback at scale is a major bottleneck, a popular alternative is to use LLMs to simulate users\[[26](https://arxiv.org/html/2605.20767#bib.bib34),[3](https://arxiv.org/html/2605.20767#bib.bib36)\], often by prompting them to embody a persona\[[14](https://arxiv.org/html/2605.20767#bib.bib37),[8](https://arxiv.org/html/2605.20767#bib.bib21)\]\. These users can then be used in lieu of human respondents to conductsynthetic experimentsthat explore how different populations might respond to new policies, products, or conversational interactions—for instance, evaluating whether aninterventionsuch as an update to an AI agent yields more favorable user outcomes\[[23](https://arxiv.org/html/2605.20767#bib.bib11),[19](https://arxiv.org/html/2605.20767#bib.bib14)\]\.

In this setting, synthetic users111We use “synthetic user” to mean an LLM initialized to simulate a user with a short persona prompt \(e\.g\., you are a 30\-year\-old man\)\. This defines a distribution over users, not a single user, and we operate at this level in our formal treatment\.are instantiated by sampling personas from a target user distribution\. Each synthetic user is then assigned to interact with each version of an AI agent and provide feedback\. Ideally, the synthetic users’ responses and derived test statistics serve as valid counterfactuals, reflecting how real users would respond in the same experiment and enabling iterative improvement of the agent\[[4](https://arxiv.org/html/2605.20767#bib.bib32),[22](https://arxiv.org/html/2605.20767#bib.bib30),[5](https://arxiv.org/html/2605.20767#bib.bib31),[41](https://arxiv.org/html/2605.20767#bib.bib33)\]\.

In this paper, we argue that this vision is overly optimistic due touser driftin LLM simulations\. While synthetic experiments are designed to mimic randomized controlled trials, LLMs are trained to operate via abductive reasoning: given underspecified prompts, they infer as much unspecified context as they are able\[[39](https://arxiv.org/html/2605.20767#bib.bib22),[21](https://arxiv.org/html/2605.20767#bib.bib13)\]\. Consider a synthetic experiment evaluating two health coaching agents, where the user is initialized as a 30\-year\-old man \(Figure[1](https://arxiv.org/html/2605.20767#S1.F1)\)\. Agent 1 is prompted to discuss the user’s exercise habits and physical condition in a more technical manner, while Agent 2 discusses physical activity in more general terms\. To generate a coherent conversation, the LLM interacting with Agent 1 might implicitly adopt the latent traits of a serious athlete, while the LLM interacting with Agent 2 might simulate a less active individual\.

![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/overview.jpeg)Figure 1:In LLM\-simulated experiments, two synthetic users who are instantiated with identical personas can drift to become distinct users in response to different intervention conditions\.As a result, two synthetic users who are instantiated with the same persona at the beginning of an experiment can effectively become distinct users by the time their outcomes are measured\. Because this drift can occur over persona attributes that \(1\) would remain fixed in human subjects and \(2\) are relevant to the primary outcomes of the experiment, it can obfuscate the actual intervention effect and give rise to misleading results\. If the synthetic user reports higher satisfaction with Agent 2, it is unclear whether Agent 2 is actually more helpful or if the simulated user has simply drifted into a population that is easier to please\.

To diagnose and mitigate this issue, our work makes three contributions:

1. 1\.We formalize the bias induced by user drift through the lens ofconfoundingin causal inference\. We argue that because LLMs are trained largely on observational data, synthetic user experiments are more similar to observational studies than randomized experiments\. As a result, differences in conditional user populations given the intervention—a form ofselection bias—can confound the estimated treatment effect\.
2. 2\.We introduce an empirical diagnostic for LLM user drift usingnegative control outcomes\[[24](https://arxiv.org/html/2605.20767#bib.bib23)\]: variables pertaining to the user that should remain invariant under intervention, such as the user’s race\. Differences in negative control outcome distributions across intervention conditions indicate that the latent attributes of the simulated user—which are causally and temporally upstream of the intervention and would remain fixed in a real experiment—do change in response to the intervention\. We provide evidence of confounding in simulated users across multiple major families of open\-source and proprietary LLMs\.
3. 3\.We show that user drift and selection bias can be mitigated in simulated users viaconfounder adjustmentduring user persona specification\. We present evidence that selection of confounders that are explicitly relevant to the treatment and outcome of the synthetic experiment helps to reduce selection bias, while adjustment over attributes that are less directly relevant yields inconsistent benefits and can sometimes inadvertently exacerbate selection bias\.

Our findings suggest that evaluations with synthetic users need to be treated with care\. This work highlights how the generative interface of LLMs can be misleading: while an evaluation may be designed to mimic a randomized controlled trial, the LLM\-simulated users employed in the trial can introduce confounding bias\.

## 2Related work

LLM user simulation\.LLMs are increasingly used to simulate user behavior, and persona prompting has become a particularly popular strategy for generating diverse user perspectives\. Simulation approaches include both using short persona prompts containing demographic attributes\[[33](https://arxiv.org/html/2605.20767#bib.bib10),[18](https://arxiv.org/html/2605.20767#bib.bib12),[19](https://arxiv.org/html/2605.20767#bib.bib14),[8](https://arxiv.org/html/2605.20767#bib.bib21)\]and longer, life\-story\-based persona prompting\[[28](https://arxiv.org/html/2605.20767#bib.bib35),[34](https://arxiv.org/html/2605.20767#bib.bib38)\], which provides benefits such as “deep binding”\[[20](https://arxiv.org/html/2605.20767#bib.bib5)\]and more accurate modeling of population variance\. Because conversational interfaces are the standard for synthetic user interactions, the majority of the simulation literature evaluates instruction\-tuned LLMs\. Prior works analyzing these specific simulators have identified distinct challenges, such as mode collapse, sycophancy, and persona instabilities\[[2](https://arxiv.org/html/2605.20767#bib.bib40),[23](https://arxiv.org/html/2605.20767#bib.bib11),[38](https://arxiv.org/html/2605.20767#bib.bib8)\]\. Some evidence suggests these challenges are less pronounced in simulations with base pre\-trained models\[[25](https://arxiv.org/html/2605.20767#bib.bib41)\], though they remain subject to artifacts such as prompt sensitivity\[[37](https://arxiv.org/html/2605.20767#bib.bib39)\]\.

Much of the work on evaluating LLM\-simulated users has centered on theirexternal validity: how closely the generated ratings or opinion patterns of simulated users match those of their real human counterparts\. Recent works develop benchmarks for external validity\[[7](https://arxiv.org/html/2605.20767#bib.bib9)\], identify failures\[[1](https://arxiv.org/html/2605.20767#bib.bib4),[23](https://arxiv.org/html/2605.20767#bib.bib11),[9](https://arxiv.org/html/2605.20767#bib.bib19),[30](https://arxiv.org/html/2605.20767#bib.bib18)\], and propose strategies to better align synthetic users with human ones\[[20](https://arxiv.org/html/2605.20767#bib.bib5),[29](https://arxiv.org/html/2605.20767#bib.bib6),[6](https://arxiv.org/html/2605.20767#bib.bib20),[15](https://arxiv.org/html/2605.20767#bib.bib17)\]\. Our work is complementary to this line of research: it focuses on theinternal validityof the synthetic user study under intervention\. This is a distinct question that specifically examines whether the difference between experimental conditions can be interpreted causally, rather than whether the results of the experiment can be generalized\.

Confounding in LLMs\.The most similar prior work on this topic is from\[[16](https://arxiv.org/html/2605.20767#bib.bib7)\], who noted that GPT\-4o\-mini\[[32](https://arxiv.org/html/2605.20767#bib.bib42)\]exhibits empirical behaviors consistent with confounding in simulated blinded microeconomic experiments\. Our work builds on this empirical study by introducing a formalism for confounding in LLM user simulations\. We further propose distinct, unified approaches to diagnose and adjust for confounding bias in a more general experimental setting\.

## 3Problem statement

In real and synthetic experiments, the goal is to estimate the causal effect of an intervention on a user’s response\. With human subjects, the gold standard for establishing causality is a randomized trial, because the randomized treatment assignment ensures that background factors that might inform user responses are balanced across groups that receive different interventions\. This isolates the effect of the intervention on the outcome\. Synthetic experiments seek to replicate this property with synthetic users: identically initialized LLMs are exposed to different interventions, with the expectation that this yields a controlled comparison\. In this section, we argue that what LLMs actually generate in a synthetic experiment more closely resembles data from an observational study\.

Notation and task\.LetA∈\{0,1\}A\\in\\\{0,1\\\}denote a treatment or intervention on a user, and letYYdenote the primary outcome \(the user’s response\)\. Let the full underlying user persona be represented by a set of attributesXX\. LetLLbe the subset of those attributes inXXthat are observed, and letX∖LX\\setminus Lbe the remaining latent attributes not included inLL\. The task is then to estimate how intervening onAAchanges the distribution ofYYin a real user population\.In an agent evaluation setting, for instance,AAmight be a dialogue with Agent 1 \(A=0A=0\) or Agent 2 \(A=1A=1\), andYYmight be a user\-reported measure of conversation satisfaction or quality\. Variables are illustrated in Figure[2\(a\)](https://arxiv.org/html/2605.20767#S3.F2.sf1)\.

Ideal randomized setting\.We can now express a core distinction between real and synthetic experiments\. Following the potential outcomes framework\[[35](https://arxiv.org/html/2605.20767#bib.bib24)\], letY​\(a\)Y\(a\)denote the potential \(or counterfactual\) outcome that would be observed under interventionA=aA=a\. We aim to estimate theaverage treatment effect\(ATE\)

τATE=𝔼​\[Y​\(1\)−Y​\(0\)\]=𝔼​\[𝔼​\[Y​\(1\)−Y​\(0\)∣L\]\]\.\\tau^\{\\text\{ATE\}\}=\\mathbb\{E\}\[Y\(1\)\-Y\(0\)\]=\\mathbb\{E\}\[\\mathbb\{E\}\[Y\(1\)\-Y\(0\)\\mid L\]\]\.\(1\)
We also make two standard causal assumptions: \(1\)Consistency:Y=A​Y​\(1\)\+\(1−A\)​Y​\(0\)Y=AY\(1\)\+\(1\-A\)Y\(0\)and \(2\)Latent unconfoundedness: GivenXX,A⟂Y​\(0\),Y​\(1\)A\\perp Y\(0\),Y\(1\)\. Under these assumptions, we can represent

𝔼​\[Y​\(a\)∣L\]=∫𝒳𝔼​\[Y∣A=a,X∣L\]⏟outcome\-generating process​P​\(X∣L\)⏟fixed population​𝑑X\.\\mathbb\{E\}\[Y\(a\)\\mid L\]=\\int\_\{\\mathcal\{X\}\}\\underbrace\{\\mathbb\{E\}\[Y\\mid A=a,X\\mid L\]\}\_\{\\text\{outcome\-generating process\}\}\\underbrace\{P\(X\\mid L\)\}\_\{\\text\{fixed population\}\}dX\.This is the average outcome across the fixed marginal population of personasXXfor bothY​\(1\)Y\(1\)andY​\(0\)Y\(0\), which is the quantity measured in real randomized trials\.τATE\\tau^\{\\text\{ATE\}\}is then identified from the data, as𝔼​\[Y​\(a\)∣L\]=E​\[Y∣A=a,L\]\\mathbb\{E\}\[Y\(a\)\\mid L\]=E\[Y\\mid A=a,L\]\.

X∖LX\\setminus LAAYYLL\(a\)Real\-world DGP\.X∖LX\\setminus LZZAAYYLL\(b\)Detect confounding with negative control outcomesZZ\.X∖\(L∪L′\)X\\setminus\(L\\cup L^\{\\prime\}\)ZZAAYYL,L′L,L^\{\\prime\}\(c\)Elicit additional attributesL′L^\{\\prime\}to adjust for confounding\.
Figure 2:DAGs for \(a\) the real\-world observational data\-generating process, \(b\) detecting selection bias via negative controlsZZ, and \(c\) reducing selection bias by iteratively eliciting additional attributesL′L^\{\\prime\}fromX∖LX\\setminus L\. Red dashed arrows indicate open backdoor paths that drive the mechanism of user drift and selection bias in simulated user populations\.Synthetic setting\.In a synthetic experiment, instead of testing a real population of users, we prompt an LLM to embody many different personas by specifying user attributesLL\. For each persona, we simulate interactions under multiple conditions ofAA\(e\.g\., Agent 1 vs\. Agent 2\) and outcomesYY\. Superficially, this resembles a controlled comparison in which different interventions are presented to an identical user\. This setup is enabled by how the LLM models its generative distribution\. While the real\-world data\-generating process follows the causal orderingX,L,A,YX,L,A,Y, the LLM factorizes the joint distribution in the sequence of its prompt context:L,A,X,YL,A,X,Y\. As a generative procedure, this ordering is convenient: it lets us condition on observed values ofLLand assign both interventionsAAto every persona, yielding a minimum\-variance estimate of the apparent treatment effect\.

However, LLMs are trained on observational data in which a user’s true personaXXcan influence both their likelihood of experiencing a treatmentAA\(e\.g\., self\-selection into interacting with Agent 1 vs\. 2\) and their eventual responseYY\. Consequently, when we introduce the treatmentA=aA=ato the LLM’s context, which has also previously been instantiated withLL, the model generates its response—and thus the outcome—from anobservedconditional distribution rather than an interventional one:

𝔼​\[Y∣A=a,L\]=∫𝒳𝔼​\[Y∣A=a,X,L\]⏟outcome\-generating process​P​\(X∣A=a,L\)⏟treatment\-conditional population​𝑑X\\mathbb\{E\}\[Y\\mid A=a,L\]=\\int\_\{\\mathcal\{X\}\}\\underbrace\{\\mathbb\{E\}\[Y\\mid A=a,X,L\]\}\_\{\\text\{outcome\-generating process\}\}\\underbrace\{P\(X\\mid A=a,L\)\}\_\{\\text\{treatment\-conditional population\}\}dX\(2\)This yields theobserved effect:

τobs=𝔼​\[Y∣A=1\]−𝔼​\[Y∣A=0\]=𝔼​\[𝔼​\[Y∣A=1,L\]−𝔼​\[Y∣A=0,L\]\]\\tau^\{\\text\{obs\}\}=\\mathbb\{E\}\[Y\\mid A=1\]\-\\mathbb\{E\}\[Y\\mid A=0\]=\\mathbb\{E\}\[\\mathbb\{E\}\[Y\\mid A=1,L\]\-\\mathbb\{E\}\[Y\\mid A=0,L\]\]What \([2](https://arxiv.org/html/2605.20767#S3.E2)\) suggests is that the distribution of user attributesXX—and in particular the distribution of unobserved user attributesX∖LX\\setminus L, asLLis specified—can differ based on the value ofAA\. This means that in a synthetic experiment, rather than comparing identical users’ responses to different treatments, the initial identical users maydriftto become distinct individuals with differing latent user attributes\. While this occurs because the model generates responses by inferring a coherent persona from the entire context, this mechanism operates against the arrow of time, as a later intervention can influence what is meant to be the user’s initial state\.

Formally, if there is no confounding, as in a randomized experiment, thenP​\(X∣A=1,L\)=P​\(X∣A=0,L\)P\(X\\mid A=1,L\)=P\(X\\mid A=0,L\), andτobs=τATE\\tau^\{\\text\{obs\}\}=\\tau^\{\\text\{ATE\}\}\. IfP​\(X∣A=1,L\)≠P​\(X∣A=0,L\)P\(X\\mid A=1,L\)\\neq P\(X\\mid A=0,L\)—that is, if the distribution of latent user attributes differs overAA—thenP​\(Y∣A=1,L\)P\(Y\\mid A=1,L\)andP​\(Y∣A=0,L\)P\(Y\\mid A=0,L\)effectively compare outcomes generated under different user populations, creating an internal validity problem that biases the observed effect\. We show an explicit decomposition for this bias in Appendix[A](https://arxiv.org/html/2605.20767#A1)\.

Figure[2\(a\)](https://arxiv.org/html/2605.20767#S3.F2.sf1)depicts this issue as an open*backdoor path*fromAAtoYYthrough latent user attributes\. In the observational data that LLMs learn from, the unspecified attributesX∖LX\\setminus Linfluence both treatment exposure and outcomes, yielding the confounding pathA←\(X∖L\)→YA\\leftarrow\(X\\setminus L\)\\rightarrow Y\. In an ideal randomized experiment, random assignment \(within strata ofLL\) blocks these backdoor paths by enforcingA⟂\(X∖L\)∣LA\\perp\(X\\setminus L\)\\mid L\. In contrast, in LLM\-simulated users instantiated with attributesLL, interactions with different interventionsAAcan themselves shift which unspecified latent attributes are inferred\. Backdoor confounding in the training data\-generating process induces shifts inX∖LX\\setminus Lat simulation time, creating a post\-treatment path fromAAtoYY\.

Our goal is therefore to detect these backdoor\-induced biases by using negative control outcomesZZ\(Figure[2\(b\)](https://arxiv.org/html/2605.20767#S3.F2.sf2)\) and to reduce them by eliciting additional attributesL′L^\{\\prime\}fromX∖LX\\setminus L\(Figure[2\(c\)](https://arxiv.org/html/2605.20767#S3.F2.sf3)\)\. This ensures that comparisons acrossAAare made over more similar latent user populations\.

## 4Detecting and adjusting for selection bias

Having argued that synthetic experiments are inherently prone to selection bias, we provide in this section an empirical strategy to both diagnose and mitigate this bias in practice\. The broad idea is to treat the synthetic experiment as an observational study, but one where we have the opportunity to query additional variables for diagnosis and adjustment at analysis time\.

### 4\.1Detecting confounding with negative control outcomes

Because the underlying personaXXof a synthetic user is not observed in full, user drift and selection bias cannot be directly computed\. Instead, we develop a diagnostic for confounding using targetednegative control outcomesZZ\[[24](https://arxiv.org/html/2605.20767#bib.bib23)\], which serve as observed proxies for the unmeasuredX∖LX\\setminus L\. In our usage, negative control outcomes should depend on the same unspecified latent user attributesX∖LX\\setminus Lwhose drift induces confounding bias in the primary outcome, but should not be*causally*affected by the intervention conditionAA\(Figure[2\(b\)](https://arxiv.org/html/2605.20767#S3.F2.sf2)\)\. Then as withP​\(X∣A=a,L\)P\(X\\mid A=a,L\), in the absence of confounding the negative control outcome distributionP​\(Z∣A=a,L\)P\(Z\\mid A=a,L\)should remain invariant toAA, i\.e\., it should hold thatP​\(Z∣A=1,L\)=P​\(Z∣A=0,L\)P\(Z\\mid A=1,L\)=P\(Z\\mid A=0,L\)\.

At a high level, a good negative control outcome satisfies the following two properties:

1. 1\.No causal effect ofAAonZZ: In an actual randomized experiment,ZZshould not change under interventions onAA\(givenLL\)\.
2. 2\.Sensitivity to latent user attributes:ZZshould depend on \(or be informative about\) the unspecified latent user attributesX∖LX\\setminus Lthat may differ across intervention conditions\.

Concretely, we consider outputs to prompts that target stable user traits not specified byLL\(e\.g\., demographics, self\-reported beliefs\) as negative control outcomes\. Since these attributes should theoretically be invariant to the interventionAAin a real\-world experiment \(as they would be causally upstream\), we can measure empirical shifts inZZin a synthetic experiment to capture the user drift over intervention conditions induced by the LLM\. We quantify this using the total variation distance \(TVD\) between the negative control outcome distributions for each persona specificationLL:

TVD=0\.5∑Z\|P\(Z∣A=1,L\)−P\(Z∣A=0,L\)\|\\text\{TVD\}=0\.5\\sum\_\{Z\}\\;\\bigl\|P\(Z\\mid A=1,L\)\-P\(Z\\mid A=0,L\)\\bigl\|The magnitude of the TVD can be interpreted as evidence of how strongly the intervention conditions correspond to different latent user populations, providing an indication of the extent to which the observed effect is vulnerable to confounding bias\. We note that because empirical TVD is positively biased in finite samples, it will naturally be bounded away from zero due to sampling variance alone\.

The usefulness of TVD as a metric to quantify confounding is of course sensitive to one’s choice of negative control outcomes\. TVD scales with the amount of confounding only if the negative control outcome is chosen to be sufficiently sensitive toX∖LX\\setminus Land sufficiently insensitive toAA\. Furthermore, if we choose negative control outcomes that are only weakly related toX∖LX\\setminus L, then we may appear to have less selection bias than we really do\. In later sections, we discuss and provide evidence of this\.

### 4\.2Adjusting for confounding

Algorithm 1Iterative detection and confounder adjustment heuristic1:Treatment

A∈\{0,1\}A\\in\\\{0,1\\\}; personas

\{Li\}i=1n\\\{L\_\{i\}\\\}\_\{i=1\}^\{n\}; trials per persona

TT; TVD threshold

ϵ\>0\\epsilon\>0
2:Elicited confounders

\{Li′\}i=1n\\\{L\_\{i\}^\{\\prime\}\\\}\_\{i=1\}^\{n\}and primary/negative control outcomes

3:Initialize augmented persona

L~i←Li\\tilde\{L\}\_\{i\}\\leftarrow L\_\{i\}and

Li′←∅L\_\{i\}^\{\\prime\}\\leftarrow\\emptysetfor each

i∈\{1​…​n\}i\\in\\\{1\\dots n\\\}
4:repeat

5:foreach persona

i∈\{1​…​n\}i\\in\\\{1\\dots n\\\}, condition

a∈\{0,1\}a\\in\\\{0,1\\\}, and trial

t∈\{1​…​T\}t\\in\\\{1\\dots T\\\}do

6:Simulate interaction under

A=aA\\\!=\\\!awith

L~i\\tilde\{L\}\_\{i\}; record

Yi,t,aY\_\{i,t,a\}and

Zi,t,aZ\_\{i,t,a\}
7:Compute

TVDi←12∑Z\|P\(Z∣A=1,L~i\)−P\(Z∣A=0,L~i\)\|\\mathrm\{TVD\}\_\{i\}\\leftarrow\\tfrac\{1\}\{2\}\\sum\_\{Z\}\\;\\bigl\|P\(Z\\mid A=1,\\tilde\{L\}\_\{i\}\)\-P\(Z\\mid A=0,\\tilde\{L\}\_\{i\}\)\\bigr\|for each

ii
8:if

1n​∑i=1nTVDi\>ϵ\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\mathrm\{TVD\}\_\{i\}\>\\epsilonthen

9:foreach persona

i∈\{1​…​n\}i\\in\\\{1\\dots n\\\}, condition

a∈\{0,1\}a\\in\\\{0,1\\\}, and trial

t∈\{1​…​T\}t\\in\\\{1\\dots T\\\}do

10:Elicit additional attributes

Li,t,a′∼P​\(L′∣A=a,L~i\)L^\{\\prime\}\_\{i,t,a\}\\sim P\(L^\{\\prime\}\\mid A=a,\\tilde\{L\}\_\{i\}\)
11:foreach persona

i∈\{1​…​n\}i\\in\\\{1\\dots n\\\}do

12:Sample one elicited attribute set

Li′←Unif​\(\{Li,t,a′:t∈\[T\],a∈\{0,1\}\}\)L\_\{i\}^\{\\prime\}\\leftarrow\\mathrm\{Unif\}\\bigl\(\\\{L^\{\\prime\}\_\{i,t,a\}:t\\in\[T\],a\\in\\\{0,1\\\}\\\}\\bigr\)
13:Update augmented persona

L~i←\(L~i∪Li′\)\\tilde\{L\}\_\{i\}\\leftarrow\(\\tilde\{L\}\_\{i\}\\cup L\_\{i\}^\{\\prime\}\)
14:until

1n​∑i=1nTVDi≤ϵ\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}\\mathrm\{TVD\}\_\{i\}\\leq\\epsilon\(or a max\-iterations budget is reached\)

The negative control diagnostic can detect confounding in synthetic experiments, but it does not by itself specify how to reduce the resulting bias\. Here, we propose a practical adjustment strategy: elicit additional attributesL′L^\{\\prime\}that proxy the unspecified latent attributesX∖LX\\setminus L, then instantiate a new simulated user for whom both the initial observed attributesLLandthe elicited attributesL′L^\{\\prime\}are specified \(Figure[2\(c\)](https://arxiv.org/html/2605.20767#S3.F2.sf3)\)\.

Eliciting confounders\.We draw new confounders for each persona from the treatment\-marginal distribution of confounders, which is a mixture of the treatment\-conditional confounder distributions:

L′∼0\.5⋅P​\(L′∣A=1,L\)\+0\.5⋅P​\(L′∣A=0,L\)L^\{\\prime\}\\sim 0\.5\\cdot P\(L^\{\\prime\}\\mid A=1,L\)\+0\.5\\cdot P\(L^\{\\prime\}\\mid A=0,L\)Note that this treatment\-uniform mixture distribution is distinct from the observational distribution marginalized over treatment,P​\(L′∣L\)=∑a∈\{0,1\}P​\(A=a∣L\)​P​\(L′∣A=a,L\)P\(L^\{\\prime\}\\mid L\)=\\sum\_\{a\\in\\\{0,1\\\}\}P\(A=a\\mid L\)P\(L^\{\\prime\}\\mid A=a,L\)\. We draw from the treatment\-uniform mixture to ensure that the additional confounders reflect the distribution that would be induced by running a real randomized experiment, where treatment conditions are assigned with equal probability\. The objective is to fix the elicited attributesL′L^\{\\prime\}across intervention conditions without narrowing the overall user distribution\.

In practice, we simulateTTtrials for each user\. In each trial, we prompt the user directly for a confounder value following intervention in each arm of the trial \(e\.g\., by asking a targeted question\)\. Across all trials for that user, we randomly select one of these realizations to add to the user persona for the next iteration, resulting in a single arm\-independentL′L^\{\\prime\}across all trials for a specific user\.

Causal caveats\.Perfect causal adjustment aims to block backdoor paths by conditioning on variables that render treatment assignment independent of latent determinants of the outcome\. Sampling and adjusting forL′L^\{\\prime\}in the manner we propose does not guarantee this, in part because prompting an LLM with descriptions of attributes approximates but is not equivalent to statistically conditioning on them\. Rather, we aim for bias reduction: we choose new confoundersL′L^\{\\prime\}such that the remaining unobserved persona attributesX∖\(L∪L′\)X\\setminus\(L\\cup L^\{\\prime\}\)that influence outcomes are less imbalanced across treatment arms\. As with any observational study, we cannot guarantee success, but we can build falsifying evidence with well\-chosen negative controlsZZ\.

We draw attention to an important caveat: because we are unlikely to be able to fully specifyX∖LX\\setminus LviaL′L^\{\\prime\}\(i\.e\., there is always some unmeasured confounding\), thenL′L^\{\\prime\}can reduce bias but is not guaranteed to eliminate it, especially if substantial latent variation remains inX∖\(L∪L′\)X\\setminus\(L\\cup L^\{\\prime\}\)or if the elicitation procedure induces additional dependencies\. This means that confounding bias also does not necessarily decrease monotonically in the size ofL∪L′L\\cup L^\{\\prime\}\.

Iterative adjustment\.In practice, rather than eliciting all possible confounders and providing them all at once in a detailed persona specification, we elicit small batches of attributes iteratively and add them to the persona specification, monitoring confounding at every iteration until the distance between the treatment\-specific user populations is sufficiently reduced \(Algorithm[1](https://arxiv.org/html/2605.20767#alg1)\)\.

The motivation for this iterative procedure is threefold\. First, confounder elicitation is computationally expensive, and adding long lists of confounders to the conversational context as part of adjustment further increases inference overhead\. Second, LLMs suffer from performance degradation given long context windows, and expanding the persona specification to include all possible confounders increases this risk\. Third—because we “condition” not in the statistical sense but rather by sampling a fixed set of confounders that we specify in an LLM prompt—every constraint we add to the persona further runs the risk of narrowing the user population through sampling bias, which can affect the external validity of the effect computed from the synthetic experiment\. Therefore, we attempt to identify aminimumset of confounders that sufficiently controls for confounding bias in the outcome\.

## 5Experiments

Models\.We evaluate synthetic users simulated using the following diverse range of frontier models: Qwen3\-30B\-A3B \(both fine\-tuned and instruction\-tuned versions\)\[[40](https://arxiv.org/html/2605.20767#bib.bib43)\], GPT\-OSS\-20B\[[31](https://arxiv.org/html/2605.20767#bib.bib44)\], Gemma\-4\-31B \(instruction\-tuned\)\[[13](https://arxiv.org/html/2605.20767#bib.bib45)\], and Gemini 3 Flash\[[12](https://arxiv.org/html/2605.20767#bib.bib47)\]\. We additionally include Gemma\-3\-4B \(instruction\-tuned\)\[[11](https://arxiv.org/html/2605.20767#bib.bib46)\]to explicitly test the behavior of a smaller\-capacity model\. Model details are available in Appendix[C](https://arxiv.org/html/2605.20767#A3)\.

Procedure\.We begin by instantiating the synthetic user with base persona attributesL~=L\\tilde\{L\}=Lderived from an existing dataset of users\. In parallel conversations, identically instantiated users are presented with interventionsA=0A=0andA=1A=1\. In each arm, the user is then asked—in diverging conversation branches—the primary outcome question \(elicitingYY\), a negative control question \(elicitingZZ\), or a confounder question \(elicitingL′L^\{\\prime\}\)\. As each question is asked independently of the others, the simulated user seesonlythat question and the intervention, not any of the other questions\. These answers are then collected and used to compute both the observed effect and the TVD\.222For questions where a list of options is provided, answers matching none of the options are mapped toUnknown\.Of the sampled confounders, a subset is added to the persona specification, and the procedure is repeated from the beginning with a more detailed persona instantiationL~=\(L~∪L′\)\\tilde\{L\}=\(\\tilde\{L\}\\cup L^\{\\prime\}\)\. The conversation history from the previous iteration isnotincluded in the new iteration\.

For each experimental setting, we sample 100 personas from the dataset to instantiate 100 corresponding synthetic users\. We then conduct 30 independent trials per user, evaluating both intervention conditions in parallel during each trial\. Due to computational constraints, Gemini 3 Flash experiments use 10 synthetic users instantiated from 10 sampled personas, with 5 trials per user\.

### 5\.1Simulated user survey

We evaluate two types of settings where we realistically expect synthetic users to be used\. The first is in survey\-type experiments \(e\.g\., in social science\), where simulated users are asked to express their opinions or preferences in response to fixed questions\.

For this setting type, we draw from the OpinionQA dataset\[[36](https://arxiv.org/html/2605.20767#bib.bib1)\], which contains questions and human responses from the Pew American Trends Panel opinion polling surveys, along with demographic attributes of the respondents\. In this setting, simulated users are given a leading statement before being asked to respond to the survey questions in the OpinionQA dataset\. We use the following experimental setup:

- •LL: The age and sex of the respondent, seeded from the actual OpinionQA respondents\.
- •AA: A leading statement to precede the survey question\. Each intervention condition targets one of the possible answers to the survey question\.
- •YY: The respondent’s answer to the survey question\. We choose the question specifically to be one that suggests the political affiliation of the respondent\.
- •ZZ: The synthetic user’s elicited citizenship status, political ideology, political party, race, and answers to 5 other survey questions that similarly provide an indication of political affiliation\.
- •L′L^\{\\prime\}: The synthetic user’s elicited geographic region, income, level of education, marital status, religion, religious attendance frequency, and 20 targeted LLM\-generated questions that specifically relate to the intervention and outcome \(10 groups of 2 questions each\)\. Confounders are added over iterations in groups of 2, in the order they are listed here\.

Prompts and lists of individual questions can be found in Appendix[F](https://arxiv.org/html/2605.20767#A6),[G](https://arxiv.org/html/2605.20767#A7)\.

### 5\.2Assistive agent evaluation

A second type of setting where we expect simulated users to be used is in evaluating LLM agents, e\.g\., an A/B test of whether a new version of an LLM agent is better than an existing version\. Here, simulated users might engage in a conversation with the agent before being prompted with a primary question relating to their interaction with the agent\.

As a common example of this type of interaction, we take a setting where the simulated user engages with the LLM agent to ask for a recommendation\. We examine two variations of this setting where the user asks for either a book or a movie\. The agent responds, and the two converse autonomously for several turns before the user is then asked how likely they are to read or watch the book or movie being discussed\. We use the following experimental setup:

- •LL: The age and sex of the respondent, seeded from the actual respondents in the NYT Book Opinions333We use only the titles and authors of 5 books from this dataset and the annotators’ demographic information, and not any content created by the New York Times\.\[[27](https://arxiv.org/html/2605.20767#bib.bib2)\]and MovieLens\[[17](https://arxiv.org/html/2605.20767#bib.bib3)\]datasets\.
- •AA: Different system prompts for the LLM agent \(Gemma\-4\-31B\-it\)\. The agent is instructed to discuss a specific book/movie in a positively \(A=1A=1\) or negatively \(A=0A=0\) biased way\.
- •YY: How likely the respondent is to read the book/movie being discussed, on a scale from1 \- Very unlikelyto4 \- Very likely\.
- •ZZ: The synthetic user’s elicited citizenship status, political ideology, political party, and race\. The respondent is also asked, “Have you heard of the <book/movie\> we are discussing before?”
- •L′L^\{\\prime\}: The synthetic user’s elicited geographic region, income, level of education, marital status, religion, religious attendance frequency, and 20 targeted LLM\-generated questions that specifically relate to the intervention and outcome \(10 groups of 2 questions each\)\. Confounders are added over iterations in groups of 2, in the order they are listed here\.

Due to computational constraints, Gemini 3 Flash experiments run for 6 iterations\. Prompts and lists of individual questions can be found in Appendix[F](https://arxiv.org/html/2605.20767#A6),[H](https://arxiv.org/html/2605.20767#A8)\.

## 6Results and Discussion

![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/combined_legend.png)

![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/combined_tvd.png)\(a\)OpinionQA
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/combined_tvd.png)\(b\)Book Opinions
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/combined_tvd.png)\(c\)MovieLens

Figure 3:TVD over adjustment iterations\. Shaded bands indicate 95% CIs\. Dashed line indicates the switch from demographic to targeted confounders, i\.e\., the last step where only demographic confounders are included in adjustment\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/combined_observed_effect.png)\(a\)OpinionQA
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/combined_observed_effect.png)\(b\)Book Opinions
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/combined_observed_effect.png)\(c\)MovieLens

Figure 4:Observed effects over adjustment iterations\. Shaded bands indicate 95% CIs\. Dashed line indicates the switch from demographic to targeted confounders, i\.e\., the last step where only demographic confounders are included in adjustment\.We report results and provide a unified set of takeaways\. Over adjustment iterations, we add elicited confounders to the persona specification, and we record \(i\) TVD between negative control outcome distributions under the two intervention conditions \(Figure[3](https://arxiv.org/html/2605.20767#S6.F3)\) and \(ii\) the observed treatment effect \(Figure[4](https://arxiv.org/html/2605.20767#S6.F4)\)\. Additional analysis and results are available in Appendix[B](https://arxiv.org/html/2605.20767#A2)\. A comprehensive set of plots for all negative control outcomes at every adjustment step is available in Appendix[I](https://arxiv.org/html/2605.20767#A9)\.

### 6\.1User drift and selection bias are present

Before adjustment \(iteration 0\), we observe substantial TVD between negative control outcome distributions across models and datasets \(Figure[3](https://arxiv.org/html/2605.20767#S6.F3)\), indicating that the simulated user population underA=0A=0differs from that underA=1A=1\. In the survey setting, most models exhibit a similar amount of user drift between interventions with the exception of the small Gemma\-3\-4B, which has a particularly large TVD\. In the agent evaluation setting, the same pattern holds for both Book Opinions and MovieLens, but with substantial variation in the magnitude of difference across models\.

We reiterate thatselection bias is a natural consequence of a model with strong abductive reasoning abilities\. Notably, in the MovieLens setting, the two smallest and oldest models—GPT\-OSS\-20B and Gemma\-3\-4B, which we expect to have the weakest abductive capabilities—show very little difference in negative control outcomes across interventions\.

### 6\.2Iterative confounder adjustment is \(often\) effective

We observe thatiterative confounder adjustment appears to be effective in controlling user drift and selection bias for many models, but not all\.In the survey setting, TVD decreases consistently over iterations, suggesting that the adjustment procedure can reduce user drift \(Figure[3\(a\)](https://arxiv.org/html/2605.20767#S6.F3.sf1)\)\. In the agent evaluation setting, many model\-dataset pairs exhibit clear reductions in TVD \(e\.g\., Gemma\-4\-31B in both Book Opinions and MovieLens, Gemini 3 Flash and Gemma\-3\-4B in Book Opinions, and both fine\-tuned and instruction\-tuned Qwen3 in MovieLens\), but others \(primarily GPT\-OSS\-20B\) show flat or weakly changing TVD across iterations \(Figures[3\(b\)](https://arxiv.org/html/2605.20767#S6.F3.sf2),[3\(c\)](https://arxiv.org/html/2605.20767#S6.F3.sf3)\)\. These results suggest that prompt\-based conditioning and the particular elicited attributes may not be uniformly effective for confounding adjustment in multi\-turn settings\.

Additionally,user drift affects the observed effect estimates\.In both settings, the observed treatment effect changes over early adjustment iterations and then stabilizes after several iterations \(Figure[4](https://arxiv.org/html/2605.20767#S6.F4)\)\. Because conditioning on additional attributes should not substantially change the observed effect if the user distributions between interventions are already comparable, these dynamics empirically support that \(i\) intervention\-dependent user drift contributes to confounding bias in naive estimates and \(ii\) adjustment reduces the relevant sources of drift\. Stabilization does not guarantee that all bias has been eliminated, but it suggests that additional confounder elicitation beyond that point \(at least of related attributes\) is less likely to address drift that materially changes the observed effect\.

Gemini 3 Flash exhibits notably anomalous behavior for its OpinionQA observed effect, which begins close to 0 and after several iterations converges to 0 \(Figure[4\(a\)](https://arxiv.org/html/2605.20767#S6.F4.sf1)\)\. While this may be partially due to the smaller sample size for this model \(10 personas with 5 trials each\), it is more likely that this behavior arises from the additional training proprietary models like Gemini undergo to express “neutral points\-of\-view” on potentially sensitive topics\[[10](https://arxiv.org/html/2605.20767#bib.bib16)\]\.

![Refer to caption](https://arxiv.org/html/2605.20767v1/x1.png)Figure 5:Negative control outcome distributions \(Qwen3\-30B, Book Opinions\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.
### 6\.3Design choices are important

Confounders\.In statistical causal inference, partial adjustment for certain covariates has the potential to worsen rather than remedy confounding\. Akin to this phenomenon, several models \(e\.g\., Gemma\-3, Qwen3\) in the survey setting exhibit an initialincreasein TVD before an eventual reduction\. This increase corresponds to the iterations eliciting and adjusting for several generic demographic\-based attributes: geographic region, income, education, and marital status\. Subsequently, the simulated user’s religious beliefs and habits are elicited and included in adjustment—which in contrast to the other demographic attributes does seem to decrease TVD across all three settings in many models\. Finally, the elicited attributes then become the simulated user’s answers to targeted questions related specifically to the intervention and outcome \(this transition point is indicated by the vertical dashed lines in Figure[3](https://arxiv.org/html/2605.20767#S6.F3)\)\. After these confounders are introduced in the persona specification, TVD continues to drop before stabilizing in later iterations\.

This observation highlights a practical trade\-off when constructing synthetic user populations\. In persona instantiation, researchers typically rely on generic demographic attributes to span a target population with specific attributes and capture diverse user perspectives\. The initial iterations of our adjustment loop show the impact of this practice: specification of generic attributes can in fact increase selection bias between intervention conditions\. Since specifying such attributes is often necessary to define a sufficiently broad simulated population, our results suggest that a good practice is then to add targeted, task\-specific confounders to subsequently bring selection bias back down\.

Negative control outcomes\.Our selection bias diagnostic depends on the sensitivity of our chosen negative control outcomes to drift in the latent user attributes\. In the Book Opinions dataset, for instance, negative control outcomes based on demographic attributes can show limited differences across intervention conditions for some models \(Figure[5](https://arxiv.org/html/2605.20767#S6.F5)\) and therefore are not particularly effective proxies for shifts inX∖LX\\setminus L\. This can yield low or flat TVD even when the observed effect continues to change across iterations\. More targeted negative control outcomes, on the other hand, may be sensitive to the intervention, as seen for the rightmost negative control outcome in Figure[5](https://arxiv.org/html/2605.20767#S6.F5)\. Additionally, certain models like GPT\-OSS exhibit high rates of refusal to answer in some data settings \(Figure[9\(a\)](https://arxiv.org/html/2605.20767#A2.F9.sf1), where refusal maps toUnknown\)\. If refusal dominates the negative control outcome distribution, then this may impact the TVD metric’s ability to detect user drift\.

## 7Conclusion

LLM\-simulated user studies can undermine the internal validity we expect from randomized experiments, inducing confounding bias\. In particular, interventions can induce shifts in latent user attributes such that outcomes for each intervention are effectively drawn from different user populations, leading to confounded effect estimates\. We formalize this phenomenon as selection bias in synthetic experiments, propose negative control outcomes as a practical diagnostic, and empirically find evidence of such shifts across survey\-style and multi\-turn agent evaluations\. Finally, we show that eliciting and specifying targeted user attributes during persona instantiation can substantially reduce observed user drift and stabilize effect estimates\. Our findings suggest that synthetic user experiments should be treated as observational by default and that diagnosing and mitigating user drift in such experiments is essential for drawing robust conclusions about the causal effect\.

## Acknowledgments and Disclosure of Funding

This work was funded by Google LLC and/or a subsidiary thereof \(Google DeepMind\)\. VL, TY, MM, JC, AG, and AD are employees of Google LLC and may own Alphabet stock as part of a standard compensation package\. All computational resources were provided by Google LLC\.

## References

- \[1\]G\. Aher, R\. I\. Arriaga, and A\. T\. Kalai\(2023\)Using large language models to simulate multiple humans and replicate human subject studies\.InProceedings of the 40th International Conference on Machine Learning,ICML’23\.Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p2.1)\.
- \[2\]J\. R\. Anthis, R\. Liu, S\. M\. Richardson, A\. C\. Kozlowski, B\. Koch, E\. Brynjolfsson, J\. Evans, and M\. S\. Bernstein\(2025\)Position: LLM social simulations are a promising research method\.InForty\-second International Conference on Machine Learning Position Paper Track,External Links:[Link](https://openreview.net/forum?id=cRBg1dtj7o)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p1.1)\.
- \[3\]L\. P\. Argyle, E\. C\. Busby, N\. Fulda, J\. R\. Gubler, C\. Rytting, and D\. Wingate\(2023\)Out of one, many: using language models to simulate human samples\.Political Analysis31\(3\),pp\. 337–351\.External Links:[Document](https://dx.doi.org/10.1017/pan.2023.2)Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p1.1)\.
- \[4\]Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon, C\. Chen, C\. Olsson, C\. Olah, D\. Hernandez, D\. Drain, D\. Ganguli, D\. Li, E\. Tran\-Johnson, E\. Perez, J\. Kerr, J\. Mueller, J\. Ladish, J\. Landau, K\. Ndousse, K\. Lukosuite, L\. Lovitt, M\. Sellitto, N\. Elhage, N\. Schiefer, N\. Mercado, N\. DasSarma, R\. Lasenby, R\. Larson, S\. Ringer, S\. Johnston, S\. Kravec, S\. E\. Showk, S\. Fort, T\. Lanham, T\. Telleen\-Lawton, T\. Conerly, T\. Henighan, T\. Hume, S\. R\. Bowman, Z\. Hatfield\-Dodds, B\. Mann, D\. Amodei, N\. Joseph, S\. McCandlish, T\. Brown, and J\. Kaplan\(2022\)Constitutional ai: harmlessness from ai feedback\.External Links:2212\.08073,[Link](https://arxiv.org/abs/2212.08073)Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p2.1)\.
- \[5\]L\. Castricato, N\. Lile, R\. Rafailov, J\. Fränken, and C\. Finn\(2025\-01\)PERSONA: a reproducible testbed for pluralistic alignment\.InProceedings of the 31st International Conference on Computational Linguistics,O\. Rambow, L\. Wanner, M\. Apidianaki, H\. Al\-Khalifa, B\. D\. Eugenio, and S\. Schockaert \(Eds\.\),Abu Dhabi, UAE,pp\. 11348–11368\.External Links:[Link](https://aclanthology.org/2025.coling-main.752/)Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p2.1)\.
- \[6\]Y\. R\. Dong, T\. Hu, and N\. Collier\(2024\-11\)Can LLM be a personalized judge?\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 10126–10141\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.592/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.592)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p2.1)\.
- \[7\]Y\. Dou, M\. Galley, B\. Peng, C\. Kedzie, W\. Cai, A\. Ritter, C\. Quirk, W\. Xu, and J\. Gao\(2025\-11\)SimulatorArena: are user simulators reliable proxies for multi\-turn evaluation of AI assistants?\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 35212–35290\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1786/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1786),ISBN 979\-8\-89176\-332\-6Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p2.1)\.
- \[8\]L\. Fröhling, G\. Demartini, and D\. Assenmacher\(2025\-08\)Personas with attitudes: controlling LLMs for diverse data annotation\.InProceedings of the The 9th Workshop on Online Abuse and Harms \(WOAH\),A\. Calabrese, C\. de Kock, D\. Nozza, F\. M\. Plaza\-del\-Arco, Z\. Talat, and F\. Vargas \(Eds\.\),Vienna, Austria,pp\. 468–481\.External Links:[Link](https://aclanthology.org/2025.woah-1.43/),ISBN 979\-8\-89176\-105\-6Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p1.1),[§2](https://arxiv.org/html/2605.20767#S2.p1.1)\.
- \[9\]Y\. Gao, D\. Lee, G\. Burtch, and S\. Fazelpour\(2025\)Take caution in using llms as human surrogates: scylla ex machina\.External Links:2410\.19599,[Link](https://arxiv.org/abs/2410.19599)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p2.1)\.
- \[10\]Gemini Team, R\. Anil, S\. Borgeaud, J\. Alayrac, J\. Yu, R\. Soricut, J\. Schalkwyk, A\. M\. Dai, A\. Hauth, K\. Millican, D\. Silver, M\. Johnson, I\. Antonoglou, J\. Schrittwieser, A\. Glaese, J\. Chen, E\. Pitler, T\. Lillicrap, A\. Lazaridou, O\. Firat, J\. Molloy, M\. Isard, P\. R\. Barham, T\. Hennigan, B\. Lee, F\. Viola, M\. Reynolds, Y\. Xu, R\. Doherty, E\. Collins, C\. Meyer, E\. Rutherford, E\. Moreira, K\. Ayoub, M\. Goel, J\. Krawczyk, C\. Du, E\. Chi, H\. Cheng, E\. Ni, P\. Shah, P\. Kane, B\. Chan, M\. Faruqui, A\. Severyn, H\. Lin, Y\. Li, Y\. Cheng, A\. Ittycheriah, M\. Mahdieh, M\. Chen, P\. Sun, D\. Tran, S\. Bagri, B\. Lakshminarayanan, J\. Liu, A\. Orban, F\. Güra, H\. Zhou, X\. Song, A\. Boffy, H\. Ganapathy, S\. Zheng, H\. Choe, Á\. Weisz, T\. Zhu, Y\. Lu, S\. Gopal, J\. Kahn, M\. Kula, J\. Pitman, R\. Shah, E\. Taropa, M\. A\. Merey, M\. Baeuml, Z\. Chen, L\. E\. Shafey, Y\. Zhang, O\. Sercinoglu, G\. Tucker, E\. Piqueras, M\. Krikun, I\. Barr, N\. Savinov, I\. Danihelka, B\. Roelofs, A\. White, A\. Andreassen, T\. von Glehn, L\. Yagati, M\. Kazemi, L\. Gonzalez, M\. Khalman, J\. Sygnowski, A\. Frechette, C\. Smith, L\. Culp, L\. Proleev, Y\. Luan, X\. Chen, J\. Lottes, N\. Schucher, F\. Lebron, A\. Rrustemi, N\. Clay, P\. Crone, T\. Kocisky, J\. Zhao, B\. Perz, D\. Yu, H\. Howard, A\. Bloniarz, J\. W\. Rae, H\. Lu, L\. Sifre, M\. Maggioni, F\. Alcober, D\. Garrette, M\. Barnes, S\. Thakoor, J\. Austin, G\. Barth\-Maron, W\. Wong, R\. Joshi, R\. Chaabouni, D\. Fatiha, A\. Ahuja, G\. S\. Tomar, E\. Senter, M\. Chadwick, I\. Kornakov, N\. Attaluri, I\. Iturrate, R\. Liu, Y\. Li, S\. Cogan, J\. Chen, C\. Jia, C\. Gu, Q\. Zhang, J\. Grimstad, A\. J\. Hartman, X\. Garcia, T\. S\. Pillai, J\. Devlin, M\. Laskin, D\. de Las Casas, D\. Valter, C\. Tao, L\. Blanco, A\. P\. Badia, D\. Reitter, M\. Chen, J\. Brennan, C\. Rivera, S\. Brin, S\. Iqbal, G\. Surita, J\. Labanowski, A\. Rao, S\. Winkler, E\. Parisotto, Y\. Gu, K\. Olszewska, R\. Addanki, A\. Miech, A\. Louis, D\. Teplyashin, G\. Brown, E\. Catt, J\. Balaguer, J\. Xiang, P\. Wang, Z\. Ashwood, A\. Briukhov, A\. Webson, S\. Ganapathy, S\. Sanghavi, A\. Kannan, M\. Chang, A\. Stjerngren, J\. Djolonga, Y\. Sun, A\. Bapna, M\. Aitchison, P\. Pejman, H\. Michalewski, T\. Yu, C\. Wang, J\. Love, J\. Ahn, D\. Bloxwich, K\. Han, P\. Humphreys, T\. Sellam, J\. Bradbury, V\. Godbole, S\. Samangooei, B\. Damoc, A\. Kaskasoli, S\. M\. R\. Arnold, V\. Vasudevan, S\. Agrawal, J\. Riesa, D\. Lepikhin, R\. Tanburn, S\. Srinivasan, H\. Lim, S\. Hodkinson, P\. Shyam, J\. Ferret, S\. Hand, A\. Garg, T\. L\. Paine, J\. Li, Y\. Li, M\. Giang, A\. Neitz, Z\. Abbas, S\. York, M\. Reid, E\. Cole, A\. Chowdhery, D\. Das, D\. Rogozińska, V\. Nikolaev, P\. Sprechmann, Z\. Nado, L\. Zilka, F\. Prost, L\. He, M\. Monteiro, G\. Mishra, C\. Welty, J\. Newlan, D\. Jia, M\. Allamanis, C\. H\. Hu, R\. de Liedekerke, J\. Gilmer, C\. Saroufim, S\. Rijhwani, S\. Hou, D\. Shrivastava, A\. Baddepudi, A\. Goldin, A\. Ozturel, A\. Cassirer, Y\. Xu, D\. Sohn, D\. Sachan, R\. K\. Amplayo, C\. Swanson, D\. Petrova, S\. Narayan, A\. Guez, S\. Brahma, J\. Landon, M\. Patel, R\. Zhao, K\. Villela, L\. Wang, W\. Jia, M\. Rahtz, M\. Giménez, L\. Yeung, J\. Keeling, P\. Georgiev, D\. Mincu, B\. Wu, S\. Haykal, R\. Saputro, K\. Vodrahalli, J\. Qin, Z\. Cankara, A\. Sharma, N\. Fernando, W\. Hawkins, B\. Neyshabur, S\. Kim, A\. Hutter, P\. Agrawal, A\. Castro\-Ros, G\. van den Driessche, T\. Wang, F\. Yang, S\. Chang, P\. Komarek, R\. McIlroy, M\. Lučić, G\. Zhang, W\. Farhan, M\. Sharman, P\. Natsev, P\. Michel, Y\. Bansal, S\. Qiao, K\. Cao, S\. Shakeri, C\. Butterfield, J\. Chung, P\. K\. Rubenstein, S\. Agrawal, A\. Mensch, K\. Soparkar, K\. Lenc, T\. Chung, A\. Pope, L\. Maggiore, J\. Kay, P\. Jhakra, S\. Wang, J\. Maynez, M\. Phuong, T\. Tobin, A\. Tacchetti, M\. Trebacz, K\. Robinson, Y\. Katariya, S\. Riedel, P\. Bailey, K\. Xiao, N\. Ghelani, L\. Aroyo, A\. Slone, N\. Houlsby, X\. Xiong, Z\. Yang, E\. Gribovskaya, J\. Adler, M\. Wirth, L\. Lee, M\. Li, T\. Kagohara, J\. Pavagadhi, S\. Bridgers, A\. Bortsova, S\. Ghemawat, Z\. Ahmed, T\. Liu, R\. Powell, V\. Bolina, M\. Iinuma, P\. Zablotskaia, J\. Besley, D\. Chung, T\. Dozat, R\. Comanescu, X\. Si, J\. Greer, G\. Su, M\. Polacek, R\. L\. Kaufman, S\. Tokumine, H\. Hu, E\. Buchatskaya, Y\. Miao, M\. Elhawaty, A\. Siddhant, N\. Tomasev, J\. Xing, C\. Greer, H\. Miller, S\. Ashraf, A\. Roy, Z\. Zhang, A\. Ma, A\. Filos, M\. Besta, R\. Blevins, T\. Klimenko, C\. Yeh, S\. Changpinyo, J\. Mu, O\. Chang, M\. Pajarskas, C\. Muir, V\. Cohen, C\. L\. Lan, K\. Haridasan, A\. Marathe, S\. Hansen, S\. Douglas, R\. Samuel, M\. Wang, S\. Austin, C\. Lan, J\. Jiang, J\. Chiu, J\. A\. Lorenzo, L\. L\. Sjösund, S\. Cevey, Z\. Gleicher, T\. Avrahami, A\. Boral, H\. Srinivasan, V\. Selo, R\. May, K\. Aisopos, L\. Hussenot, L\. B\. Soares, K\. Baumli, M\. B\. Chang, A\. Recasens, B\. Caine, A\. Pritzel, F\. Pavetic, F\. Pardo, A\. Gergely, J\. Frye, V\. Ramasesh, D\. Horgan, K\. Badola, N\. Kassner, S\. Roy, E\. Dyer, V\. C\. Campos, A\. Tomala, Y\. Tang, D\. E\. Badawy, E\. White, B\. Mustafa, O\. Lang, A\. Jindal, S\. Vikram, Z\. Gong, S\. Caelles, R\. Hemsley, G\. Thornton, F\. Feng, W\. Stokowiec, C\. Zheng, P\. Thacker, Ç\. Ünlü, Z\. Zhang, M\. Saleh, J\. Svensson, M\. Bileschi, P\. Patil, A\. Anand, R\. Ring, K\. Tsihlas, A\. Vezer, M\. Selvi, T\. Shevlane, M\. Rodriguez, T\. Kwiatkowski, S\. Daruki, K\. Rong, A\. Dafoe, N\. FitzGerald, K\. Gu\-Lemberg, M\. Khan, L\. A\. Hendricks, M\. Pellat, V\. Feinberg, J\. Cobon\-Kerr, T\. Sainath, M\. Rauh, S\. H\. Hashemi, R\. Ives, Y\. Hasson, E\. Noland, Y\. Cao, N\. Byrd, L\. Hou, Q\. Wang, T\. Sottiaux, M\. Paganini, J\. Lespiau, A\. Moufarek, S\. Hassan, K\. Shivakumar, J\. van Amersfoort, A\. Mandhane, P\. Joshi, A\. Goyal, M\. Tung, A\. Brock, H\. Sheahan, V\. Misra, C\. Li, N\. Rakićević, M\. Dehghani, F\. Liu, S\. Mittal, J\. Oh, S\. Noury, E\. Sezener, F\. Huot, M\. Lamm, N\. D\. Cao, C\. Chen, S\. Mudgal, R\. Stella, K\. Brooks, G\. Vasudevan, C\. Liu, M\. Chain, N\. Melinkeri, A\. Cohen, V\. Wang, K\. Seymore, S\. Zubkov, R\. Goel, S\. Yue, S\. Krishnakumaran, B\. Albert, N\. Hurley, M\. Sano, A\. Mohananey, J\. Joughin, E\. Filonov, T\. Kępa, Y\. Eldawy, J\. Lim, R\. Rishi, S\. Badiezadegan, T\. Bos, J\. Chang, S\. Jain, S\. G\. S\. Padmanabhan, S\. Puttagunta, K\. Krishna, L\. Baker, N\. Kalb, V\. Bedapudi, A\. Kurzrok, S\. Lei, A\. Yu, O\. Litvin, X\. Zhou, Z\. Wu, S\. Sobell, A\. Siciliano, A\. Papir, R\. Neale, J\. Bragagnolo, T\. Toor, T\. Chen, V\. Anklin, F\. Wang, R\. Feng, M\. Gholami, K\. Ling, L\. Liu, J\. Walter, H\. Moghaddam, A\. Kishore, J\. Adamek, T\. Mercado, J\. Mallinson, S\. Wandekar, S\. Cagle, E\. Ofek, G\. Garrido, C\. Lombriser, M\. Mukha, B\. Sun, H\. R\. Mohammad, J\. Matak, Y\. Qian, V\. Peswani, P\. Janus, Q\. Yuan, L\. Schelin, O\. David, A\. Garg, Y\. He, O\. Duzhyi, A\. Älgmyr, T\. Lottaz, Q\. Li, V\. Yadav, L\. Xu, A\. Chinien, R\. Shivanna, A\. Chuklin, J\. Li, C\. Spadine, T\. Wolfe, K\. Mohamed, S\. Das, Z\. Dai, K\. He, D\. von Dincklage, S\. Upadhyay, A\. Maurya, L\. Chi, S\. Krause, K\. Salama, P\. G\. Rabinovitch, P\. K\. R\. M, A\. Selvan, M\. Dektiarev, G\. Ghiasi, E\. Guven, H\. Gupta, B\. Liu, D\. Sharma, I\. H\. Shtacher, S\. Paul, O\. Akerlund, F\. Aubet, T\. Huang, C\. Zhu, E\. Zhu, E\. Teixeira, M\. Fritze, F\. Bertolini, L\. Marinescu, M\. Bölle, D\. Paulus, K\. Gupta, T\. Latkar, M\. Chang, J\. Sanders, R\. Wilson, X\. Wu, Y\. Tan, L\. N\. Thiet, T\. Doshi, S\. Lall, S\. Mishra, W\. Chen, T\. Luong, S\. Benjamin, J\. Lee, E\. Andrejczuk, D\. Rabiej, V\. Ranjan, K\. Styrc, P\. Yin, J\. Simon, M\. R\. Harriott, M\. Bansal, A\. Robsky, G\. Bacon, D\. Greene, D\. Mirylenka, C\. Zhou, O\. Sarvana, A\. Goyal, S\. Andermatt, P\. Siegler, B\. Horn, A\. Israel, F\. Pongetti, C\. "\. Chen, M\. Selvatici, P\. Silva, K\. Wang, J\. Tolins, K\. Guu, R\. Yogev, X\. Cai, A\. Agostini, M\. Shah, H\. Nguyen, N\. Ó\. Donnaile, S\. Pereira, L\. Friso, A\. Stambler, A\. Kurzrok, C\. Kuang, Y\. Romanikhin, M\. Geller, Z\. Yan, K\. Jang, C\. Lee, W\. Fica, E\. Malmi, Q\. Tan, D\. Banica, D\. Balle, R\. Pham, Y\. Huang, D\. Avram, H\. Shi, J\. Singh, C\. Hidey, N\. Ahuja, P\. Saxena, D\. Dooley, S\. P\. Potharaju, E\. O’Neill, A\. Gokulchandran, R\. Foley, K\. Zhao, M\. Dusenberry, Y\. Liu, P\. Mehta, R\. Kotikalapudi, C\. Safranek\-Shrader, A\. Goodman, J\. Kessinger, E\. Globen, P\. Kolhar, C\. Gorgolewski, A\. Ibrahim, Y\. Song, A\. Eichenbaum, T\. Brovelli, S\. Potluri, P\. Lahoti, C\. Baetu, A\. Ghorbani, C\. Chen, A\. Crawford, S\. Pal, M\. Sridhar, P\. Gurita, A\. Mujika, I\. Petrovski, P\. Cedoz, C\. Li, S\. Chen, N\. D\. Santo, S\. Goyal, J\. Punjabi, K\. Kappaganthu, C\. Kwak, P\. LV, S\. Velury, H\. Choudhury, J\. Hall, P\. Shah, R\. Figueira, M\. Thomas, M\. Lu, T\. Zhou, C\. Kumar, T\. Jurdi, S\. Chikkerur, Y\. Ma, A\. Yu, S\. Kwak, V\. Ähdel, S\. Rajayogam, T\. Choma, F\. Liu, A\. Barua, C\. Ji, J\. H\. Park, V\. Hellendoorn, A\. Bailey, T\. Bilal, H\. Zhou, M\. Khatir, C\. Sutton, W\. Rzadkowski, F\. Macintosh, R\. Vij, K\. Shagin, P\. Medina, C\. Liang, J\. Zhou, P\. Shah, Y\. Bi, A\. Dankovics, S\. Banga, S\. Lehmann, M\. Bredesen, Z\. Lin, J\. E\. Hoffmann, J\. Lai, R\. Chung, K\. Yang, N\. Balani, A\. Bražinskas, A\. Sozanschi, M\. Hayes, H\. F\. Alcalde, P\. Makarov, W\. Chen, A\. Stella, L\. Snijders, M\. Mandl, A\. Kärrman, P\. Nowak, X\. Wu, A\. Dyck, K\. Vaidyanathan, R\. R, J\. Mallet, M\. Rudominer, E\. Johnston, S\. Mittal, A\. Udathu, J\. Christensen, V\. Verma, Z\. Irving, A\. Santucci, G\. Elsayed, E\. Davoodi, M\. Georgiev, I\. Tenney, N\. Hua, G\. Cideron, E\. Leurent, M\. Alnahlawi, I\. Georgescu, N\. Wei, I\. Zheng, D\. Scandinaro, H\. Jiang, J\. Snoek, M\. Sundararajan, X\. Wang, Z\. Ontiveros, I\. Karo, J\. Cole, V\. Rajashekhar, L\. Tumeh, E\. Ben\-David, R\. Jain, J\. Uesato, R\. Datta, O\. Bunyan, S\. Wu, J\. Zhang, P\. Stanczyk, Y\. Zhang, D\. Steiner, S\. Naskar, M\. Azzam, M\. Johnson, A\. Paszke, C\. Chiu, J\. S\. Elias, A\. Mohiuddin, F\. Muhammad, J\. Miao, A\. Lee, N\. Vieillard, J\. Park, J\. Zhang, J\. Stanway, D\. Garmon, A\. Karmarkar, Z\. Dong, J\. Lee, A\. Kumar, L\. Zhou, J\. Evens, W\. Isaac, G\. Irving, E\. Loper, M\. Fink, I\. Arkatkar, N\. Chen, I\. Shafran, I\. Petrychenko, Z\. Chen, J\. Jia, A\. Levskaya, Z\. Zhu, P\. Grabowski, Y\. Mao, A\. Magni, K\. Yao, J\. Snaider, N\. Casagrande, E\. Palmer, P\. Suganthan, A\. Castaño, I\. Giannoumis, W\. Kim, M\. Rybiński, A\. Sreevatsa, J\. Prendki, D\. Soergel, A\. Goedeckemeyer, W\. Gierke, M\. Jafari, M\. Gaba, J\. Wiesner, D\. G\. Wright, Y\. Wei, H\. Vashisht, Y\. Kulizhskaya, J\. Hoover, M\. Le, L\. Li, C\. Iwuanyanwu, L\. Liu, K\. Ramirez, A\. Khorlin, A\. Cui, T\. LIN, M\. Wu, R\. Aguilar, K\. Pallo, A\. Chakladar, G\. Perng, E\. A\. Abellan, M\. Zhang, I\. Dasgupta, N\. Kushman, I\. Penchev, A\. Repina, X\. Wu, T\. van der Weide, P\. Ponnapalli, C\. Kaplan, J\. Simsa, S\. Li, O\. Dousse, F\. Yang, J\. Piper, N\. Ie, R\. Pasumarthi, N\. Lintz, A\. Vijayakumar, D\. Andor, P\. Valenzuela, M\. Lui, C\. Paduraru, D\. Peng, K\. Lee, S\. Zhang, S\. Greene, D\. D\. Nguyen, P\. Kurylowicz, C\. Hardin, L\. Dixon, L\. Janzer, K\. Choo, Z\. Feng, B\. Zhang, A\. Singhal, D\. Du, D\. McKinnon, N\. Antropova, T\. Bolukbasi, O\. Keller, D\. Reid, D\. Finchelstein, M\. A\. Raad, R\. Crocker, P\. Hawkins, R\. Dadashi, C\. Gaffney, K\. Franko, A\. Bulanova, R\. Leblond, S\. Chung, H\. Askham, L\. C\. Cobo, K\. Xu, F\. Fischer, J\. Xu, C\. Sorokin, C\. Alberti, C\. Lin, C\. Evans, A\. Dimitriev, H\. Forbes, D\. Banarse, Z\. Tung, M\. Omernick, C\. Bishop, R\. Sterneck, R\. Jain, J\. Xia, E\. Amid, F\. Piccinno, X\. Wang, P\. Banzal, D\. J\. Mankowitz, A\. Polozov, V\. Krakovna, S\. Brown, M\. Bateni, D\. Duan, V\. Firoiu, M\. Thotakuri, T\. Natan, M\. Geist, S\. tan Girgin, H\. Li, J\. Ye, O\. Roval, R\. Tojo, M\. Kwong, J\. Lee\-Thorp, C\. Yew, D\. Sinopalnikov, S\. Ramos, J\. Mellor, A\. Sharma, K\. Wu, D\. Miller, N\. Sonnerat, D\. Vnukov, R\. Greig, J\. Beattie, E\. Caveness, L\. Bai, J\. Eisenschlos, A\. Korchemniy, T\. Tsai, M\. Jasarevic, W\. Kong, P\. Dao, Z\. Zheng, F\. Liu, F\. Yang, R\. Zhu, T\. H\. Teh, J\. Sanmiya, E\. Gladchenko, N\. Trdin, D\. Toyama, E\. Rosen, S\. Tavakkol, L\. Xue, C\. Elkind, O\. Woodman, J\. Carpenter, G\. Papamakarios, R\. Kemp, S\. Kafle, T\. Grunina, R\. Sinha, A\. Talbert, D\. Wu, D\. Owusu\-Afriyie, C\. Du, C\. Thornton, J\. Pont\-Tuset, P\. Narayana, J\. Li, S\. Fatehi, J\. Wieting, O\. Ajmeri, B\. Uria, Y\. Ko, L\. Knight, A\. Héliou, N\. Niu, S\. Gu, C\. Pang, Y\. Li, N\. Levine, A\. Stolovich, R\. Santamaria\-Fernandez, S\. Goenka, W\. Yustalim, R\. Strudel, A\. Elqursh, C\. Deck, H\. Lee, Z\. Li, K\. Levin, R\. Hoffmann, D\. Holtmann\-Rice, O\. Bachem, S\. Arora, C\. Koh, S\. H\. Yeganeh, S\. Põder, M\. Tariq, Y\. Sun, L\. Ionita, M\. Seyedhosseini, P\. Tafti, Z\. Liu, A\. Gulati, J\. Liu, X\. Ye, B\. Chrzaszcz, L\. Wang, N\. Sethi, T\. Li, B\. Brown, S\. Singh, W\. Fan, A\. Parisi, J\. Stanton, V\. Koverkathu, C\. A\. Choquette\-Choo, Y\. Li, T\. Lu, A\. Ittycheriah, P\. Shroff, M\. Varadarajan, S\. Bahargam, R\. Willoughby, D\. Gaddy, G\. Desjardins, M\. Cornero, B\. Robenek, B\. Mittal, B\. Albrecht, A\. Shenoy, F\. Moiseev, H\. Jacobsson, A\. Ghaffarkhah, M\. Rivière, A\. Walton, C\. Crepy, A\. Parrish, Z\. Zhou, C\. Farabet, C\. Radebaugh, P\. Srinivasan, C\. van der Salm, A\. Fidjeland, S\. Scellato, E\. Latorre\-Chimoto, H\. Klimczak\-Plucińska, D\. Bridson, D\. de Cesare, T\. Hudson, P\. Mendolicchio, L\. Walker, A\. Morris, M\. Mauger, A\. Guseynov, A\. Reid, S\. Odoom, L\. Loher, V\. Cotruta, M\. Yenugula, D\. Grewe, A\. Petrushkina, T\. Duerig, A\. Sanchez, S\. Yadlowsky, A\. Shen, A\. Globerson, L\. Webb, S\. Dua, D\. Li, S\. Bhupatiraju, D\. Hurt, H\. Qureshi, A\. Agarwal, T\. Shani, M\. Eyal, A\. Khare, S\. R\. Belle, L\. Wang, C\. Tekur, M\. S\. Kale, J\. Wei, R\. Sang, B\. Saeta, T\. Liechty, Y\. Sun, Y\. Zhao, S\. Lee, P\. Nayak, D\. Fritz, M\. R\. Vuyyuru, J\. Aslanides, N\. Vyas, M\. Wicke, X\. Ma, E\. Eltyshev, N\. Martin, H\. Cate, J\. Manyika, K\. Amiri, Y\. Kim, X\. Xiong, K\. Kang, F\. Luisier, N\. Tripuraneni, D\. Madras, M\. Guo, A\. Waters, O\. Wang, J\. Ainslie, J\. Baldridge, H\. Zhang, G\. Pruthi, J\. Bauer, F\. Yang, R\. Mansour, J\. Gelman, Y\. Xu, G\. Polovets, J\. Liu, H\. Cai, W\. Chen, X\. Sheng, E\. Xue, S\. Ozair, C\. Angermueller, X\. Li, A\. Sinha, W\. Wang, J\. Wiesinger, E\. Koukoumidis, Y\. Tian, A\. Iyer, M\. Gurumurthy, M\. Goldenson, P\. Shah, M\. Blake, H\. Yu, A\. Urbanowicz, J\. Palomaki, C\. Fernando, K\. Durden, H\. Mehta, N\. Momchev, E\. Rahimtoroghi, M\. Georgaki, A\. Raul, S\. Ruder, M\. Redshaw, J\. Lee, D\. Zhou, K\. Jalan, D\. Li, B\. Hechtman, P\. Schuh, M\. Nasr, K\. Milan, V\. Mikulik, J\. Franco, T\. Green, N\. Nguyen, J\. Kelley, A\. Mahendru, A\. Hu, J\. Howland, B\. Vargas, J\. Hui, K\. Bansal, V\. Rao, R\. Ghiya, E\. Wang, K\. Ye, J\. M\. Sarr, M\. M\. Preston, M\. Elish, S\. Li, A\. Kaku, J\. Gupta, I\. Pasupat, D\. Juan, M\. Someswar, T\. M\., X\. Chen, A\. Amini, A\. Fabrikant, E\. Chu, X\. Dong, A\. Muthal, S\. Buthpitiya, S\. Jauhari, N\. Hua, U\. Khandelwal, A\. Hitron, J\. Ren, L\. Rinaldi, S\. Drath, A\. Dabush, N\. Jiang, H\. Godhia, U\. Sachs, A\. Chen, Y\. Fan, H\. Taitelbaum, H\. Noga, Z\. Dai, J\. Wang, C\. Liang, J\. Hamer, C\. Ferng, C\. Elkind, A\. Atias, P\. Lee, V\. Listík, M\. Carlen, J\. van de Kerkhof, M\. Pikus, K\. Zaher, P\. Müller, S\. Zykova, R\. Stefanec, V\. Gatsko, C\. Hirnschall, A\. Sethi, X\. F\. Xu, C\. Ahuja, B\. Tsai, A\. Stefanoiu, B\. Feng, K\. Dhandhania, M\. Katyal, A\. Gupta, A\. Parulekar, D\. Pitta, J\. Zhao, V\. Bhatia, Y\. Bhavnani, O\. Alhadlaq, X\. Li, P\. Danenberg, D\. Tu, A\. Pine, V\. Filippova, A\. Ghosh, B\. Limonchik, B\. Urala, C\. K\. Lanka, D\. Clive, Y\. Sun, E\. Li, H\. Wu, K\. Hongtongsak, I\. Li, K\. Thakkar, K\. Omarov, K\. Majmundar, M\. Alverson, M\. Kucharski, M\. Patel, M\. Jain, M\. Zabelin, P\. Pelagatti, R\. Kohli, S\. Kumar, J\. Kim, S\. Sankar, V\. Shah, L\. Ramachandruni, X\. Zeng, B\. Bariach, L\. Weidinger, T\. Vu, A\. Andreev, A\. He, K\. Hui, S\. Kashem, A\. Subramanya, S\. Hsiao, D\. Hassabis, K\. Kavukcuoglu, A\. Sadovsky, Q\. Le, T\. Strohman, Y\. Wu, S\. Petrov, J\. Dean, and O\. Vinyals\(2025\)Gemini: a family of highly capable multimodal models\.External Links:2312\.11805,[Link](https://arxiv.org/abs/2312.11805)Cited by:[§6\.2](https://arxiv.org/html/2605.20767#S6.SS2.p3.1)\.
- \[11\]Google\(2025\)Gemma\-3\-4b\-it\.Hugging Face\.Note:[https://huggingface\.co/google/gemma\-3\-4b\-it](https://huggingface.co/google/gemma-3-4b-it)Cited by:[§5](https://arxiv.org/html/2605.20767#S5.p1.1)\.
- \[12\]Google\(2026\)Gemini 3 developer guide \| gemini api\.Google AI for Developers\.Note:[https://ai\.google\.dev/gemini\-api/docs/gemini\-3](https://ai.google.dev/gemini-api/docs/gemini-3)Cited by:[§5](https://arxiv.org/html/2605.20767#S5.p1.1)\.
- \[13\]Google\(2026\)Gemma\-4\-31b\-it\.Hugging Face\.Note:[https://huggingface\.co/google/gemma\-4\-31B\-it](https://huggingface.co/google/gemma-4-31B-it)Cited by:[§5](https://arxiv.org/html/2605.20767#S5.p1.1)\.
- \[14\]\(\. H\. Gu, S\. Chandrasegaran, and P\. Lloyd\(2025\)Synthetic users: insights from designers’ interactions with persona\-based chatbots\.Artificial Intelligence for Engineering Design, Analysis and Manufacturing39,pp\. e2\.External Links:[Document](https://dx.doi.org/10.1017/S0890060424000283)Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p1.1)\.
- \[15\]L\. Guerdan, J\. Whitehouse, K\. Truong, K\. Holstein, and S\. Wu\(2026\)Doubly\-robust LLM\-as\-a\-judge: externally valid estimation with imperfect personas\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=EIA1tpKYL7)Cited by:[Appendix B](https://arxiv.org/html/2605.20767#A2.p5.1),[§2](https://arxiv.org/html/2605.20767#S2.p2.1)\.
- \[16\]G\. Gui and O\. Toubia\(2023\)The challenge of using llms to simulate human behavior: a causal inference perspective\.SSRN Electronic Journal\.External Links:ISSN 1556\-5068,[Link](http://dx.doi.org/10.2139/ssrn.4650172),[Document](https://dx.doi.org/10.2139/ssrn.4650172)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p3.1)\.
- \[17\]F\. M\. Harper and J\. A\. Konstan\(2015\-12\)The movielens datasets: history and context\.ACM Trans\. Interact\. Intell\. Syst\.5\(4\)\.External Links:ISSN 2160\-6455,[Link](https://doi.org/10.1145/2827872),[Document](https://dx.doi.org/10.1145/2827872)Cited by:[Table 2](https://arxiv.org/html/2605.20767#A4.T2.4.4.3.1.1.1),[1st item](https://arxiv.org/html/2605.20767#S5.I2.i1.p1.1)\.
- \[18\]T\. Hu and N\. Collier\(2024\-08\)Quantifying the persona effect in LLM simulations\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 10289–10307\.External Links:[Link](https://aclanthology.org/2024.acl-long.554/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.554)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p1.1)\.
- \[19\]C\. Kaiser, J\. Kaiser, V\. Manewitsch, L\. Rau, and R\. Schallner\(2025\)Simulating human opinions with large language models: opportunities and challenges for personalized survey data modeling\.InAdjunct Proceedings of the 33rd ACM Conference on User Modeling, Adaptation and Personalization,UMAP Adjunct ’25,New York, NY, USA,pp\. 82–86\.External Links:ISBN 9798400713996,[Link](https://doi.org/10.1145/3708319.3733685),[Document](https://dx.doi.org/10.1145/3708319.3733685)Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p1.1),[§2](https://arxiv.org/html/2605.20767#S2.p1.1)\.
- \[20\]M\. Kang, S\. Moon, S\. H\. Lee, A\. Raj, J\. Suh, and D\. Chan\(2025\)Deep binding of language model virtual personas: a study on approximating political partisan misperceptions\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=zHdSCtNmM4)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p1.1),[§2](https://arxiv.org/html/2605.20767#S2.p2.1)\.
- \[21\]B\. Lee, S\. Kim, F\. Menczer, Y\. Ahn, H\. Kwak, and J\. An\(2026\)LLMs can infer political alignment from online conversations\.External Links:2603\.11253,[Link](https://arxiv.org/abs/2603.11253)Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p3.1)\.
- \[22\]H\. Lee, S\. Phatale, H\. Mansoor, T\. Mesnard, J\. Ferret, K\. Lu, C\. Bishop, E\. Hall, V\. Carbune, A\. Rastogi, and S\. Prakash\(2024\)RLAIF vs\. rlhf: scaling reinforcement learning from human feedback with ai feedback\.InProceedings of the 41st International Conference on Machine Learning,ICML’24\.Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p2.1)\.
- \[23\]A\. Li, H\. Chen, H\. Namkoong, and T\. Peng\(2025\)LLM generated persona is a promise with a catch\.InThe Thirty\-Ninth Annual Conference on Neural Information Processing Systems Position Paper Track,External Links:[Link](https://openreview.net/forum?id=qh9eGtMG4H)Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p1.1),[§2](https://arxiv.org/html/2605.20767#S2.p1.1),[§2](https://arxiv.org/html/2605.20767#S2.p2.1)\.
- \[24\]M\. Lipsitch, E\. T\. Tchetgen, and T\. Cohen\(2010\)Negative controls: a tool for detecting confounding and bias in observational studies\.Epidemiology21\(3\),pp\. 383–388\.Cited by:[item 2](https://arxiv.org/html/2605.20767#S1.I1.i2.p1.1),[§4\.1](https://arxiv.org/html/2605.20767#S4.SS1.p1.9)\.
- \[25\]A\. Lyman, B\. Hepner, L\. P\. Argyle, E\. C\. Busby, J\. R\. Gubler, and D\. Wingate\(2025\)Balancing large language model alignment and algorithmic fidelity in social science research\.Sociological Methods and Research54\.Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p1.1)\.
- \[26\]B\. S\. Manning, K\. Zhu, and J\. J\. Horton\(2024\)Automated social science: language models as scientist and subjects\.External Links:2404\.11794,[Link](https://arxiv.org/abs/2404.11794)Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p1.1)\.
- \[27\]N\. Meister, C\. Guestrin, and T\. Hashimoto\(2025\-04\)Benchmarking distributional alignment of large language models\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 24–49\.External Links:[Link](https://aclanthology.org/2025.naacl-long.2/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.2),ISBN 979\-8\-89176\-189\-6Cited by:[Table 2](https://arxiv.org/html/2605.20767#A4.T2.4.3.2.1.1.1),[1st item](https://arxiv.org/html/2605.20767#S5.I2.i1.p1.1)\.
- \[28\]S\. Moon, M\. Abdulhai, M\. Kang, J\. Suh, W\. Soedarmadji, E\. K\. Behar, and D\. M\. Chan\(2024\-11\)Virtual personas for language models via an anthology of backstories\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 19864–19897\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.1110/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.1110)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p1.1)\.
- \[29\]S\. Moon, M\. Kang, J\. Suh, M\. Safdari, and J\. Canny\(2026\)Identity, cooperation and framing effects within groups of real and simulated humans\.External Links:2601\.16355,[Link](https://arxiv.org/abs/2601.16355)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p2.1)\.
- \[30\]T\. Neumann, M\. De\-Arteaga, and S\. Fazelpour\(2026\-Mar\.\)Should you use llms to simulate opinions? quality checks for early\-stage deliberation\.Proceedings of the AAAI Conference on Artificial Intelligence40\(46\),pp\. 39070–39079\.External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/41254),[Document](https://dx.doi.org/10.1609/aaai.v40i46.41254)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p2.1)\.
- \[31\]OpenAI, S\. Agarwal, L\. Ahmad, J\. Ai, S\. Altman, A\. Applebaum, E\. Arbus, R\. K\. Arora, Y\. Bai, B\. Baker, H\. Bao, B\. Barak, A\. Bennett, T\. Bertao, N\. Brett, E\. Brevdo, G\. Brockman, S\. Bubeck, C\. Chang, K\. Chen, M\. Chen, E\. Cheung, A\. Clark, D\. Cook, M\. Dukhan, C\. Dvorak, K\. Fives, V\. Fomenko, T\. Garipov, K\. Georgiev, M\. Glaese, T\. Gogineni, A\. Goucher, L\. Gross, K\. G\. Guzman, J\. Hallman, J\. Hehir, J\. Heidecke, A\. Helyar, H\. Hu, R\. Huet, J\. Huh, S\. Jain, Z\. Johnson, C\. Koch, I\. Kofman, D\. Kundel, J\. Kwon, V\. Kyrylov, E\. Y\. Le, G\. Leclerc, J\. P\. Lennon, S\. Lessans, M\. Lezcano\-Casado, Y\. Li, Z\. Li, J\. Lin, J\. Liss, Lily, Liu, J\. Liu, K\. Lu, C\. Lu, Z\. Martinovic, L\. McCallum, J\. McGrath, S\. McKinney, A\. McLaughlin, S\. Mei, S\. Mostovoy, T\. Mu, G\. Myles, A\. Neitz, A\. Nichol, J\. Pachocki, A\. Paino, D\. Palmie, A\. Pantuliano, G\. Parascandolo, J\. Park, L\. Pathak, C\. Paz, L\. Peran, D\. Pimenov, M\. Pokrass, E\. Proehl, H\. Qiu, G\. Raila, F\. Raso, H\. Ren, K\. Richardson, D\. Robinson, B\. Rotsted, H\. Salman, S\. Sanjeev, M\. Schwarzer, D\. Sculley, H\. Sikchi, K\. Simon, K\. Singhal, Y\. Song, D\. Stuckey, Z\. Sun, P\. Tillet, S\. Toizer, F\. Tsimpourlas, N\. Vyas, E\. Wallace, X\. Wang, M\. Wang, O\. Watkins, K\. Weil, A\. Wendling, K\. Whinnery, C\. Whitney, H\. Wong, L\. Yang, Y\. Yang, M\. Yasunaga, K\. Ying, W\. Zaremba, W\. Zhan, C\. Zhang, B\. Zhang, E\. Zhang, and S\. Zhao\(2025\)Gpt\-oss\-120b & gpt\-oss\-20b model card\.External Links:2508\.10925,[Link](https://arxiv.org/abs/2508.10925)Cited by:[§5](https://arxiv.org/html/2605.20767#S5.p1.1)\.
- \[32\]OpenAI, A\. Hurst, A\. Lerer, A\. P\. Goucher, A\. Perelman, A\. Ramesh, A\. Clark, A\. Ostrow, A\. Welihinda, A\. Hayes, A\. Radford, A\. Mądry, A\. Baker\-Whitcomb, A\. Beutel, A\. Borzunov, A\. Carney, A\. Chow, A\. Kirillov, A\. Nichol, A\. Paino, A\. Renzin, A\. T\. Passos, A\. Kirillov, A\. Christakis, A\. Conneau, A\. Kamali, A\. Jabri, A\. Moyer, A\. Tam, A\. Crookes, A\. Tootoochian, A\. Tootoonchian, A\. Kumar, A\. Vallone, A\. Karpathy, A\. Braunstein, A\. Cann, A\. Codispoti, A\. Galu, A\. Kondrich, A\. Tulloch, A\. Mishchenko, A\. Baek, A\. Jiang, A\. Pelisse, A\. Woodford, A\. Gosalia, A\. Dhar, A\. Pantuliano, A\. Nayak, A\. Oliver, B\. Zoph, B\. Ghorbani, B\. Leimberger, B\. Rossen, B\. Sokolowsky, B\. Wang, B\. Zweig, B\. Hoover, B\. Samic, B\. McGrew, B\. Spero, B\. Giertler, B\. Cheng, B\. Lightcap, B\. Walkin, B\. Quinn, B\. Guarraci, B\. Hsu, B\. Kellogg, B\. Eastman, C\. Lugaresi, C\. Wainwright, C\. Bassin, C\. Hudson, C\. Chu, C\. Nelson, C\. Li, C\. J\. Shern, C\. Conger, C\. Barette, C\. Voss, C\. Ding, C\. Lu, C\. Zhang, C\. Beaumont, C\. Hallacy, C\. Koch, C\. Gibson, C\. Kim, C\. Choi, C\. McLeavey, C\. Hesse, C\. Fischer, C\. Winter, C\. Czarnecki, C\. Jarvis, C\. Wei, C\. Koumouzelis, D\. Sherburn, D\. Kappler, D\. Levin, D\. Levy, D\. Carr, D\. Farhi, D\. Mely, D\. Robinson, D\. Sasaki, D\. Jin, D\. Valladares, D\. Tsipras, D\. Li, D\. P\. Nguyen, D\. Findlay, E\. Oiwoh, E\. Wong, E\. Asdar, E\. Proehl, E\. Yang, E\. Antonow, E\. Kramer, E\. Peterson, E\. Sigler, E\. Wallace, E\. Brevdo, E\. Mays, F\. Khorasani, F\. P\. Such, F\. Raso, F\. Zhang, F\. von Lohmann, F\. Sulit, G\. Goh, G\. Oden, G\. Salmon, G\. Starace, G\. Brockman, H\. Salman, H\. Bao, H\. Hu, H\. Wong, H\. Wang, H\. Schmidt, H\. Whitney, H\. Jun, H\. Kirchner, H\. P\. de Oliveira Pinto, H\. Ren, H\. Chang, H\. W\. Chung, I\. Kivlichan, I\. O’Connell, I\. O’Connell, I\. Osband, I\. Silber, I\. Sohl, I\. Okuyucu, I\. Lan, I\. Kostrikov, I\. Sutskever, I\. Kanitscheider, I\. Gulrajani, J\. Coxon, J\. Menick, J\. Pachocki, J\. Aung, J\. Betker, J\. Crooks, J\. Lennon, J\. Kiros, J\. Leike, J\. Park, J\. Kwon, J\. Phang, J\. Teplitz, J\. Wei, J\. Wolfe, J\. Chen, J\. Harris, J\. Varavva, J\. G\. Lee, J\. Shieh, J\. Lin, J\. Yu, J\. Weng, J\. Tang, J\. Yu, J\. Jang, J\. Q\. Candela, J\. Beutler, J\. Landers, J\. Parish, J\. Heidecke, J\. Schulman, J\. Lachman, J\. McKay, J\. Uesato, J\. Ward, J\. W\. Kim, J\. Huizinga, J\. Sitkin, J\. Kraaijeveld, J\. Gross, J\. Kaplan, J\. Snyder, J\. Achiam, J\. Jiao, J\. Lee, J\. Zhuang, J\. Harriman, K\. Fricke, K\. Hayashi, K\. Singhal, K\. Shi, K\. Karthik, K\. Wood, K\. Rimbach, K\. Hsu, K\. Nguyen, K\. Gu\-Lemberg, K\. Button, K\. Liu, K\. Howe, K\. Muthukumar, K\. Luther, L\. Ahmad, L\. Kai, L\. Itow, L\. Workman, L\. Pathak, L\. Chen, L\. Jing, L\. Guy, L\. Fedus, L\. Zhou, L\. Mamitsuka, L\. Weng, L\. McCallum, L\. Held, L\. Ouyang, L\. Feuvrier, L\. Zhang, L\. Kondraciuk, L\. Kaiser, L\. Hewitt, L\. Metz, L\. Doshi, M\. Aflak, M\. Simens, M\. Boyd, M\. Thompson, M\. Dukhan, M\. Chen, M\. Gray, M\. Hudnall, M\. Zhang, M\. Aljubeh, M\. Litwin, M\. Zeng, M\. Johnson, M\. Shetty, M\. Gupta, M\. Shah, M\. Yatbaz, M\. J\. Yang, M\. Zhong, M\. Glaese, M\. Chen, M\. Janner, M\. Lampe, M\. Petrov, M\. Wu, M\. Wang, M\. Fradin, M\. Pokrass, M\. Castro, M\. O\. T\. de Castro, M\. Pavlov, M\. Brundage, M\. Wang, M\. Khan, M\. Murati, M\. Bavarian, M\. Lin, M\. Yesildal, N\. Soto, N\. Gimelshein, N\. Cone, N\. Staudacher, N\. Summers, N\. LaFontaine, N\. Chowdhury, N\. Ryder, N\. Stathas, N\. Turley, N\. Tezak, N\. Felix, N\. Kudige, N\. Keskar, N\. Deutsch, N\. Bundick, N\. Puckett, O\. Nachum, O\. Okelola, O\. Boiko, O\. Murk, O\. Jaffe, O\. Watkins, O\. Godement, O\. Campbell\-Moore, P\. Chao, P\. McMillan, P\. Belov, P\. Su, P\. Bak, P\. Bakkum, P\. Deng, P\. Dolan, P\. Hoeschele, P\. Welinder, P\. Tillet, P\. Pronin, P\. Tillet, P\. Dhariwal, Q\. Yuan, R\. Dias, R\. Lim, R\. Arora, R\. Troll, R\. Lin, R\. G\. Lopes, R\. Puri, R\. Miyara, R\. Leike, R\. Gaubert, R\. Zamani, R\. Wang, R\. Donnelly, R\. Honsby, R\. Smith, R\. Sahai, R\. Ramchandani, R\. Huet, R\. Carmichael, R\. Zellers, R\. Chen, R\. Chen, R\. Nigmatullin, R\. Cheu, S\. Jain, S\. Altman, S\. Schoenholz, S\. Toizer, S\. Miserendino, S\. Agarwal, S\. Culver, S\. Ethersmith, S\. Gray, S\. Grove, S\. Metzger, S\. Hermani, S\. Jain, S\. Zhao, S\. Wu, S\. Jomoto, S\. Wu, Shuaiqi, Xia, S\. Phene, S\. Papay, S\. Narayanan, S\. Coffey, S\. Lee, S\. Hall, S\. Balaji, T\. Broda, T\. Stramer, T\. Xu, T\. Gogineni, T\. Christianson, T\. Sanders, T\. Patwardhan, T\. Cunninghman, T\. Degry, T\. Dimson, T\. Raoux, T\. Shadwell, T\. Zheng, T\. Underwood, T\. Markov, T\. Sherbakov, T\. Rubin, T\. Stasi, T\. Kaftan, T\. Heywood, T\. Peterson, T\. Walters, T\. Eloundou, V\. Qi, V\. Moeller, V\. Monaco, V\. Kuo, V\. Fomenko, W\. Chang, W\. Zheng, W\. Zhou, W\. Manassra, W\. Sheu, W\. Zaremba, Y\. Patil, Y\. Qian, Y\. Kim, Y\. Cheng, Y\. Zhang, Y\. He, Y\. Zhang, Y\. Jin, Y\. Dai, and Y\. Malkov\(2024\)GPT\-4o system card\.External Links:2410\.21276,[Link](https://arxiv.org/abs/2410.21276)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p3.1)\.
- \[33\]J\. S\. Park, J\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein\(2023\)Generative agents: interactive simulacra of human behavior\.InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology,UIST ’23,New York, NY, USA\.External Links:ISBN 9798400701320,[Link](https://doi.org/10.1145/3586183.3606763),[Document](https://dx.doi.org/10.1145/3586183.3606763)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p1.1)\.
- \[34\]J\. S\. Park, C\. Q\. Zou, J\. Kamphorst, N\. Egan, A\. Shaw, B\. M\. Hill, C\. Cai, M\. R\. Morris, P\. Liang, R\. Willer, and M\. S\. Bernstein\(2026\)LLM agents grounded in self\-reports enable general\-purpose simulation of individuals\.External Links:2411\.10109,[Link](https://arxiv.org/abs/2411.10109)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p1.1)\.
- \[35\]D\. B\. Rubin\(1974\)Estimating causal effects of treatments in randomized and nonrandomized studies\.\.Journal of educational Psychology66\(5\),pp\. 688\.Cited by:[§3](https://arxiv.org/html/2605.20767#S3.p3.2)\.
- \[36\]S\. Santurkar, E\. Durmus, F\. Ladhak, C\. Lee, P\. Liang, and T\. Hashimoto\(2023\-23–29 Jul\)Whose opinions do language models reflect?\.InProceedings of the 40th International Conference on Machine Learning,A\. Krause, E\. Brunskill, K\. Cho, B\. Engelhardt, S\. Sabato, and J\. Scarlett \(Eds\.\),Proceedings of Machine Learning Research, Vol\.202,pp\. 29971–30004\.External Links:[Link](https://proceedings.mlr.press/v202/santurkar23a.html)Cited by:[Table 2](https://arxiv.org/html/2605.20767#A4.T2.4.2.1.1.1.1),[§5\.1](https://arxiv.org/html/2605.20767#S5.SS1.p2.1)\.
- \[37\]M\. Sclar, Y\. Choi, Y\. Tsvetkov, and A\. Suhr\(2024\)Quantifying language models'sensitivity to spurious features in prompt design or: how i learned to start worrying about prompt formatting\.InInternational Conference on Learning Representations,B\. Kim, Y\. Yue, S\. Chaudhuri, K\. Fragkiadaki, M\. Khan, and Y\. Sun \(Eds\.\),Vol\.2024,pp\. 25055–25083\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/file/6c0e99d736da621403018ca7b32b1a4d-Paper-Conference.pdf)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p1.1)\.
- \[38\]T\. Tosato, S\. Helbling, Y\. Mantilla\-Ramos, M\. Hegazy, A\. Tosato, D\. J\. Lemay, I\. Rish, and G\. Dumas\(2026\-Mar\.\)Persistent instability in llm’s personality measurements: effects of scale, reasoning, and conversation history\.Proceedings of the AAAI Conference on Artificial Intelligence40\(44\),pp\. 37961–37969\.External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/41133),[Document](https://dx.doi.org/10.1609/aaai.v40i44.41133)Cited by:[§2](https://arxiv.org/html/2605.20767#S2.p1.1)\.
- \[39\]D\. Wright, A\. Arora, N\. Borenstein, S\. Yadav, S\. Belongie, and I\. Augenstein\(2024\-11\)LLM tropes: revealing fine\-grained values and opinions in large language models\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 17085–17112\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.995/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.995)Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p3.1)\.
- \[40\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. Qiu\(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§5](https://arxiv.org/html/2605.20767#S5.p1.1)\.
- \[41\]T\. Yun, E\. Yang, M\. Safdari, J\. H\. Lee, V\. V\. Kumar, S\. S\. Mahdavi, J\. Amar, D\. Peyton, R\. Aharony, A\. M\. PhD, L\. D\. Schneider, I\. Galatzer\-Levy, Y\. Jia, J\. Canny, A\. Gretton, and M\. Mataric\(2025\-07\)Sleepless nights, sugary days: creating synthetic users with health conditions for realistic coaching agent interactions\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 14159–14181\.External Links:[Link](https://aclanthology.org/2025.findings-acl.729/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.729),ISBN 979\-8\-89176\-256\-5Cited by:[§1](https://arxiv.org/html/2605.20767#S1.p2.1)\.

## Appendix ADecomposition of the treatment effect

In this section, we show theoretically how intervention\-dependent user drifts can bias the observed effectτ\\taubetween interventions\. Given a persona specificationLL, and again working within the potential outcomes framework, let

μa,a′=∫𝒳𝔼​\[Y​\(a\)∣X,L\]​P​\(X∣A=a′,L\)​𝑑X\\mu\_\{a,a^\{\\prime\}\}=\\int\_\{\\mathcal\{X\}\}\\mathbb\{E\}\[Y\(a\)\\mid X,L\]P\(X\\mid A=a^\{\\prime\},L\)\\;dX
Using this notation, we can rewrite several key quantities\. In a synthetic experiment, the observed effectτobs\\tau^\{\\text\{obs\}\}for an LLM\-simulated user with attributesLLis given by:

τobs=μ1,1−μ0,0\\tau^\{\\text\{obs\}\}=\\mu\_\{1,1\}\-\\mu\_\{0,0\}
The causal estimand of interest, the CATE, admits the standard decomposition into the average treatment effect on the treated \(ATT\) and on the controls \(ATC\)\. WhenP​\(A=1∣L\)=P​\(A=0∣L\)=1/2P\(A=1\\mid L\)=P\(A=0\\mid L\)=1/2, this becomes

2​τATE\\displaystyle 2\\tau^\{\\text\{ATE\}\}=μ1,1−μ0,1⏟ATT\+μ1,0−μ0,0⏟ATC\\displaystyle=\\underbrace\{\\mu\_\{1,1\}\-\\mu\_\{0,1\}\}\_\{\\text\{ATT\}\}\+\\underbrace\{\\mu\_\{1,0\}\-\\mu\_\{0,0\}\}\_\{\\text\{ATC\}\}We can also define theselection bias in the treated\(SBT\) and theselection bias in the controls\(SBC\)\. Intuitively, this is the difference inYYresulting solely from user drift overP​\(X∣A=a,L\)P\(X\\mid A=a,L\)due toAA\.

δSBT=μ1,1−μ1,0δSBC=μ0,1−μ0,0\\delta\_\{\\text\{SBT\}\}=\\mu\_\{1,1\}\-\\mu\_\{1,0\}\\quad\\quad\\delta\_\{\\text\{SBC\}\}=\\mu\_\{0,1\}\-\\mu\_\{0,0\}Then we find that the CATE can be written in terms of the observed effect and the selection bias terms:

2​τATE=2⋅\(μ1,1−μ0,0⏟τobs\)−\(μ0,1−μ0,0⏟δSBC\)−\(μ1,1−μ1,0⏟δSBT\)\\begin\{split\}2\\tau^\{\\text\{ATE\}\}&=2\\cdot\(\\underbrace\{\\mu\_\{1,1\}\-\\mu\_\{0,0\}\}\_\{\\tau^\{\\text\{obs\}\}\}\)\-\(\\underbrace\{\\mu\_\{0,1\}\-\{\\mu\_\{0,0\}\}\}\_\{\\delta\_\{\\text\{SBC\}\}\}\)\-\(\\underbrace\{\\mu\_\{1,1\}\-\\mu\_\{1,0\}\}\_\{\\delta\_\{\\text\{SBT\}\}\}\)\\end\{split\}Letting average selection biasδSB=\(δSBC\+δSBT\)/2\\delta\_\{\\text\{SB\}\}=\(\\delta\_\{\\text\{SBC\}\}\+\\delta\_\{\\text\{SBT\}\}\)/2,

τATE=τobs−δSB\\tau^\{\\text\{ATE\}\}=\\tau^\{\\text\{obs\}\}\-\\delta\_\{\\text\{SB\}\}
This decomposition explicitly demonstrates that in experiments with LLM\-simulated users, the observed effect will not correspond to the true causal treatment effect\. Depending on the direction of the selection bias, this can either artificially inflate or reduce the observed effect relative to the true causal effect, yielding misleading results\.

## Appendix BAdditional results

![Refer to caption](https://arxiv.org/html/2605.20767v1/x2.png)\(a\)OpinionQA \(Gemma\-4\-31B\-it\)
![Refer to caption](https://arxiv.org/html/2605.20767v1/x3.png)\(b\)Book Opinions \(Qwen3\-30B\-Instruct\)
![Refer to caption](https://arxiv.org/html/2605.20767v1/x4.png)\(c\)MovieLens \(GPT\-OSS\-20B\)

Figure 6:Primary outcome distributions faceted by intervention condition at adjustment iteration 0\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-30p-10t/sanity_checks_plot.png)Figure 7:Proportion of specified persona attributes correctly reported after intervention dialogue in Book Opinions \(30 personas with 10 trials each\)\.We first sanity check that the intervention has had the desired effect on the outcome\. Across all three datasets, we observe that each intervention condition indeed induces the expected change in the outcome distribution \(Figure[6](https://arxiv.org/html/2605.20767#A2.F6)\)\.

Are user drift and responsiveness to confounder adjustment linked to model capability/capacity?We explore whether models that are better at following instructions and maintaining consistent personas are both \(a\) more susceptible to user drift and \(b\) more responsive to adjustment\. We attempt to proxy this persona\-maintaining capability by checking how well models retain the information provided in the persona specification after the dialogue between the simulated user and the recommender LLM agent has been completed\. After the dialogue, the simulated user is prompted again for the attributes that were specified in its initial persona prompt\. In Figure[7](https://arxiv.org/html/2605.20767#A2.F7), we plot the proportion of attributes that were correctly returned after the dialogue in the Book Opinions dataset\. Because the list of attributes increases with each iteration, it is expected that the proportion correctly retained will decrease over iterations\.

In this evaluation, Gemma\-4\-31B shows the best retention abilities and also is the model that seems most responsive to adjustment in the agent evaluation setting\. Conversely, Gemma\-3\-4B and GPT\-OSS\-20B have the worst retention abilities and are also the two models to exhibit very little TVD even at iteration 0 in the MovieLens setting\. This may provide evidence for our point that good models experience more user drift due to their strong abductive capabilities\. Interestingly, while Gemma\-3\-4B has by far the worst retention abilities—which we expect due to its small size of only 4B—it is also more responsive to adjustment in the Book Opinions setting than the much larger Qwen3 models\.

Does the overall user population drift due to confounding adjustment?Because we sample confounders to include in our persona specification, and conditioning on LLMs by prompting is not exact, the population we start with is potentially not the population we end with\. That is, we stabilize the populationacrossinterventions, but the overall population may drift over time\. We note that this is anexternal validityproblem similar to those we mention in Section[2](https://arxiv.org/html/2605.20767#S2)and is not the focus of our work\.

However, for the sake of analysis, we can check this directly by looking at themarginaldistribution of negative control outcomesP​\(Z\)P\(Z\)over adjustment iterations, e\.g\. for instruction\-tuned Gemma\-4\-31B on OpinionQA \(Figure[8](https://arxiv.org/html/2605.20767#A2.F8)\)\. We see that some of these distributions do change over time, while others remain more or less constant\. If significant drifts in the marginal population occur over adjustment iterations, then a possibility is to draw on prior approaches for improving external validity between iterations, such as the doubly robust approach proposed by\[[15](https://arxiv.org/html/2605.20767#bib.bib17)\]\.

![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/google-gemma-4-31B-it_nc_marginal_stacked.png)Figure 8:Marginal negative control outcome distributions over iterations of Gemma\-4\-31B on OpinionQA\.![Refer to caption](https://arxiv.org/html/2605.20767v1/x5.png)\(a\)GPT\-OSS\-20B
![Refer to caption](https://arxiv.org/html/2605.20767v1/x6.png)\(b\)Gemma\-4\-31B\-it

Figure 9:Marginal demographics\-based negative control outcome distributions in MovieLens\.
## Appendix CModel details

Table 1:Technical details for language model implementations\.To implement our simulations, we use several open\-source large language models as detailed in Table[1](https://arxiv.org/html/2605.20767#A3.T1)\. We evaluateopenai/gpt\-oss\-20b,Qwen/Qwen3\-30B\-A3B,Qwen/Qwen3\-30B\-A3B\-Instruct\-2507,google/gemma\-4\-31B\-it, andgoogle/gemma\-3\-4b\-itas user agents in our simulation\. Additionally,google/gemma\-4\-31B\-itis employed as the dialogue agent for book/movie discussions\.

Finally, Gemini 3 Flash is accessed via the Gemini API and used as a user agent\. Additional technical details are provided in Table[1](https://arxiv.org/html/2605.20767#A3.T1)\.

## Appendix DDataset details

Table 2:Overview of datasets
## Appendix EComputational resources

Each experimental setting \(e\.g\., one full run of a setting withnnpersonas×\\timesTTtrials×\\times2 interventions\) was executed on a single node with 8×\\timesNVIDIA A100 GPUs\. Across settings, we used a multi\-node cluster of such machines \(running settings and sweeps in parallel\)\. Over all reported runs, we estimate total usage of approximately 3000 GPU\-hours\.

## Appendix FShared implementation details

#### Negative control outcomes \(demographic\)\.

\{

"citizenshipstatus":\["Yes","No"\],

"politicalparty":\["Republican","Democrat","Independent","Other"\],

"politicalideology":\[

"Veryconservative",

"Conservative",

"Moderate",

"Liberal",

"Veryliberal",

\],

"race":\["White","Black","Asian","Hispanic","Other"\]

\}

#### Confounders \(demographic\)\.

\{

"income":\[

"Lessthan$30,000",

"$30,000\-$50,000",

"$50,000\-$75,000",

"$75,000\-$100,000",

"$100,000ormore",

\],

"censusregion":\["Northeast","Midwest","South","West"\],

"educationlevel":\[

"Lessthanhighschool",

"Highschoolgraduate",

"Somecollege,nodegree",

"Associate’sdegree",

"Collegegraduate/somepostgrad",

"Postgraduate",

\],

"maritalstatus":\[

"Married",

"Divorced",

"Separated",

"Widowed",

"Neverbeenmarried",

\],

"religion":\[

"Protestant",

"RomanCatholic",

"Mormon",

"Orthodox",

"Jewish",

"Muslim",

"Buddhist",

"Hindu",

"Atheist",

"Agnostic",

"Other",

"Nothinginparticular",

\],

"religiousattendancefrequency":\[

"Morethanonceaweek",

"Onceaweek",

"Onceortwiceamonth",

"Afewtimesayear",

"Seldom",

"Never",

\]

\}

## Appendix GSimulated user survey implementation details

### G\.1Simulated user survey prompts

#### Synthetic user instantiation\.

\[SYSTEM\]

Youareactingasthefollowingperson:Persona:

Age:<AGE\>

Sex:<SEX\>

\[Optional:Additionalinformationaboutyou:

<Confounderquestionfrompreviousstep\><Answerfrompreviousstep\>

\.\.\.\]

#### Intervention \(leading statements\)\.

\{

"Highpriority":"Citingconcernsovernationalsecurity,strainonpublicresources,andtheruleoflaw,manybelievebordersecurityshouldbeatopconcern\.",

"Lowpriority":"Manyadvocatespointoutthatimmigrantscontributesignificantlytotheeconomyandfillessentialjobs,suggestingenforcementshouldbealowerprioritythanreform\."

\}

#### Primary outcome elicitation\.

\[SYSTEM\]

<Personadescriptionasabove\>

\[USER\]

<Leadingstatement1/Leadingstatement2\>

Question:Howmuchofapriorityshouldthefollowingbeforthefederalgovernmenttoaddress:Reducingillegalimmigration

PleaseanswerONLYwithoneofthefollowingoptions:Highpriority,Lowpriority

#### Negative control / confounder elicitation \(demographic attributes\)\.

\[SYSTEM\]

<Personadescriptionasabove\>

\[USER\]

<Leadingstatement1/Leadingstatement2\>

\[USER\]

Question:Whatisyour<characteristic\>?

Options:<Option1\>,<Option2\>,\.\.\.

PleaseanswerONLYwithoneoftheoptions\.

#### Negative control / confounder elicitation \(targeted questions\)\.

\[SYSTEM\]

<Personadescriptionasabove\>

\[USER\]

<Leadingstatement1/Leadingstatement2\>

Question:<Negativecontrol/confounderquestion\>

Pleaseanswerinabriefsentence\.

### G\.2Questions

#### Negative control questions \(targeted\)\.

\{

"question":"Howmuchofapriorityshouldthefollowingbeforthefederalgovernmenttoaddress:Makinghealthcaremoreaffordable",

"options":\[

"Highpriority",

"Lowpriority"

\]

\},

\{

"question":"Howmuchofapriorityshouldthefollowingbeforthefederalgovernmenttoaddress:Reducingeconomicinequality",

"options":\[

"Highpriority",

"Lowpriority"

\]

\},

\{

"question":"Howmuchofapriorityshouldthefollowingbeforthefederalgovernmenttoaddress:Addressingclimatechange",

"options":\[

"Highpriority",

"Lowpriority"

\]

\},

\{

"question":"Howmuchofapriorityshouldthefollowingbeforthefederalgovernmenttoaddress:Dealingwithterrorism",

"options":\[

"Highpriority",

"Lowpriority"

\],

\},

\{

"question":"Howmuchofapriorityshouldthefollowingbeforthefederalgovernmenttoaddress:Reducinggunviolence",

"options\_condensed":\[

"Highpriority",

"Lowpriority"

\]

\}

#### Confounder questions \(targeted\)\.

\{

"Political&IdeologicalAffiliation":\[

"Howimportantispartyloyaltytoyouwhenformingopinionsonpolicy?",

"Howdoyouviewtheroleofthefederalgovernmentindailylife?"

\],

"EconomicBeliefs&Self\-Interest":\[

"Doyouworkinanindustrythatfrequentlyemploysimmigrantlabor\(e\.g\.,agriculture,construction,hospitality\)?",

"Howdoyouperceivetheimpactofimmigrationonyourlocaleconomy?"

\],

"NationalSecurity&LawEnforcementViews":\[

"Howoftendoyouworryaboutcrimeinyourneighborhood?",

"DoyoubelievetheUSbordersarecurrentlysecure?"

\],

"Humanitarian&EthicalConsiderations":\[

"Howmuchdoyouvalueempathyinpublicpolicy?",

"Howoftendoyoudonatetoorvolunteerforcharitablecauses?"

\],

"Cultural&SocialIdentity":\[

"Howoftendoyouinteractwithpeoplefromdifferentculturalbackgrounds?",

"HowdoyoufeelwhenyouhearlanguagesotherthanEnglishspokeninpublic?"

\],

"Information&MediaConsumption":\[

"Whichcablenewsnetworkdoyouwatchmostfrequently?",

"Doyoufact\-checkinformationyouseeonsocialmedia?"

\],

"PersonalExperience&Proximity":\[

"HaveyoueverlivedinacountryotherthantheUnitedStates?",

"Hasyourcommunityseenasignificantchangeinitsimmigrantpopulationrecently?"

\],

"TrustinInstitutions":\[

"Howmuchdoyoutrustthefederalgovernmenttodowhatisright?",

"Doyoubelieveyourvoicemattersingovernmentdecisions?"

\],

"ViewsonGovernmentRole&Scope":\[

"Howmuchresponsibilityshouldthefederalgovernmenthaveinregulatingtheeconomy?",

"Whatisyourviewongovernmentspendingforsocialwelfareprograms?"

\],

"GeneralWorldview&PersonalityTraits":\[

"Howopenareyoutonewexperiences?",

"Howimportantisittoyoutoconformtosocialnorms?"

\]

\}

## Appendix HAssistive agent evaluation implementation details

### H\.1Selected books

The 5 books randomly chosen for discussion in the evaluation are:

- •Way Out There In The Blue: Reagan, Star Wars And The End Of The Cold Warby Frances Fitzgerald
- •There Thereby Tommy Orange
- •Redeploymentby Phil Klay
- •The Doorby Magda Szabo
- •Nothing To Be Frightened Ofby Julian Barnes

### H\.2Selected movies

The 5 movies randomly chosen for discussion in the evaluation are:

- •Other Side of Sunday, The \(Søndagsengler\) \(1996\)\(Comedy\|Drama\)
- •Death in Brunswick \(1991\)\(Comedy\)
- •Buddy \(1997\)\(Adventure\|Children’s\|Drama\)
- •Paris, Texas \(1984\)\(Drama\)
- •Lovers of the Arctic Circle, The \(Los Amantes del Círculo Polar\) \(1998\)\(Drama\|Romance\)

### H\.3Prompts

#### Book/movie agent instantiation \(with intervention\)\.

\[SYSTEM\]

Youareahelpful<book/movie\>recommenderinteractingwithauser\.Yourgoalistohelpthereaderchoosea<book/movie\>\.Discussthefollowing<book/movie\>:<<title\>by<author\>/<movie:<title\>,genres:<genres\>\>\.

<Discussthis<book/movie\>inapositively\-biasedway\./Discussthis<book/movie\>inanegatively\-biasedway\.\>

\[USER\]

I’mlookingfora<booktoread/movietowatch\>\.Canyouhelpmefindone?

#### Synthetic user instantiation\.

\[SYSTEM\]

Youareactingasthefollowingperson:

Age:<AGE\>

Gender:<GENDER\>

\[Optional:Additionalinformationaboutyou:

<Questionfrompreviousstep\><Answerfrompreviousstep\>

\.\.\.\]

Youarelookingfora<booktoread/movietowatch\>\.

\[USER\]

<Book/movieagent’sresponse\>

#### Primary outcome elicitation\.

\[SYSTEM\]

<Syntheticusersystempromptfromabove\>

\[USER\]

<Book/movieagentmessage1\>

\[ASSISTANT\]

<Syntheticusermessage1\>

\.\.\.\(moreturns\)\.\.\.

\[USER\]

<Book/movieagentmessage3\>

\[ASSISTANT\]

<Syntheticagentmessage3\>

\[USER\]

Howlikelyareyouto<readthebook/watchthemovie\>’<title\>’?Answerwithoneof:

1:Veryunlikely

2:Unlikely

3:Likely

4:Verylikely

#### Negative control / confounder elicitation \(demographic attributes\)\.

\[SYSTEM\]

<Syntheticusersystempromptfromabove\>

\.\.\.\(Dialoguehistory:USERandASSISTANTmessages\)\.\.\.

\[USER\]

Question:Whatisyour<characteristic\>?

Options:<Option1\>,<Option2\>,\.\.\.

PleaseanswerONLYwithoneoftheoptions\.

#### Negative control / confounder elicitation \(targeted question\)\.

\[SYSTEM\]

<Syntheticusersystempromptfromabove\>

\.\.\.\(Dialoguehistory:USERandASSISTANTmessages\)\.\.\.

\[USER\]

Question:<Negativecontrol/confounderquestion\>

Options:<Option1\>,<Option2\>,\.\.\.

PleaseanswerONLYwithoneoftheoptions\.

### H\.4Questions

#### Negative control questions \(targeted\)\.

\{

"question":"Haveyouheardofthe<book/movie\>we’rediscussingbefore?",

"options":\[

"Yes",

"No"

\]

\}

#### Confounder questions \(targeted\)\.

\{

"<Reading/MovieWatching\>Habits&Logistics":\[

"Whatisyourfavorite<book/movie\>genre?",

"Howmany<booksdoyouread/moviesdoyouwatch\>inayear?"

\],

"<Book/Movie\>Selection&Criticality":\[

"Howdoyoutypicallyfindoutaboutnew<books/movies\>?",

"Doyouread<book/movie\>reviewsbeforedecidingto<readabook/watchamovie\>?"

\],

"Lifestyle&TimeAllocation":\[

"Howmuchleisuretimedoyouhaveperday?",

"Whatisyourmainoccupation?"

\],

"Media&LeisureHabits":\[

"Whatisyourprimarysourceofnews?",

"Howoftendoyouattendculturalevents\(e\.g\.,museums,concerts\)?"

\],

"Personality&CognitiveStyle":\[

"Howeasilyareyouinfluencedbyothers’opinions?",

"Doyouconsideryourselfanempatheticperson?"

\],

"DecisionMaking&Outlook":\[

"Doyouconsideryourselfacreativeperson?",

"Doyouenjoydebatingtopicswithothers?"

\],

"EmotionalWell\-being&Mood":\[

"Howwouldyoudescribeyourgeneralmoodtoday?",

"Howoftendoyoufeelboredinyourfreetime?"

\],

"Social&PersonalFulfillment":\[

"Areyoucurrentlyseekingnewchallengesinyourlife?",

"Howoptimisticareyouaboutthefuture?"

\],

"PersonalValues\(Part1\)":\[

"Howimportantispersonalfreedomandindependencetoyou?",

"Howimportantissocialstatustoyou?"

\],

"PersonalValues\(Part2\)":\[

"Howimportantismaintainingtraditionstoyou?",

"Whatvaluedoyouplaceondiversityandinclusion?"

\]

\}

## Appendix IAdditional plots

### I\.1OpinionQA

![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/google-gemma-3-4b-it_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/google-gemma-3-4b-it_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 10:OpinionQA outcome distributions \(Gemma\-3 4B\-it\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/google-gemma-4-31B-it_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/google-gemma-4-31B-it_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 11:OpinionQA outcome distributions \(Gemma\-4 31B\-it\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/openai-gpt-oss-20b_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/openai-gpt-oss-20b_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 12:OpinionQA outcome distributions \(GPT\-OSS 20B\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/Qwen-Qwen3-30B-A3B_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/Qwen-Qwen3-30B-A3B_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 13:OpinionQA outcome distributions \(Qwen3 30B\-A3B\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/Qwen-Qwen3-30B-A3B-Instruct-2507_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/Qwen-Qwen3-30B-A3B-Instruct-2507_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 14:OpinionQA outcome distributions \(Qwen3 30B\-A3B\-Instruct\-2507\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/google-gemini-3-flash-10p-5t_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/opinionqa-100p-30t/google-gemini-3-flash-10p-5t_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 15:OpinionQA outcome distributions \(Gemini 3 Flash\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.
### I\.2Book Opinions

![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/google-gemma-3-4b-it_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/google-gemma-3-4b-it_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 16:Book Opinions outcome distributions \(Gemma\-3 4B\-it\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/google-gemma-4-31B-it_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/google-gemma-4-31B-it_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 17:Book Opinions outcome distributions \(Gemma\-4 31B\-it\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/openai-gpt-oss-20b_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/openai-gpt-oss-20b_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 18:Book Opinions outcome distributions \(GPT\-OSS 20B\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/Qwen-Qwen3-30B-A3B_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/Qwen-Qwen3-30B-A3B_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 19:Book Opinions outcome distributions \(Qwen3 30B\-A3B\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/Qwen-Qwen3-30B-A3B-Instruct-2507_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/Qwen-Qwen3-30B-A3B-Instruct-2507_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 20:Book Opinions outcome distributions \(Qwen3 30B\-A3B\-Instruct\-2507\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/google-gemini-3-flash-10p-5t_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/nyt-100p-30t/google-gemini-3-flash-10p-5t_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 21:Book Opinions outcome distributions \(Gemini 3 Flash\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.
### I\.3MovieLens

![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/google-gemma-3-4b-it_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/google-gemma-3-4b-it_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 22:MovieLens outcome distributions \(Gemma\-3 4B\-it\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/google-gemma-4-31B-it_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/google-gemma-4-31B-it_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 23:MovieLens outcome distributions \(Gemma\-4 31B\-it\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/openai-gpt-oss-20b_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/openai-gpt-oss-20b_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 24:MovieLens outcome distributions \(GPT\-OSS 20B\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/Qwen-Qwen3-30B-A3B_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/Qwen-Qwen3-30B-A3B_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 25:MovieLens outcome distributions \(Qwen3 30B\-A3B\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/Qwen-Qwen3-30B-A3B-Instruct-2507_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/Qwen-Qwen3-30B-A3B-Instruct-2507_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 26:MovieLens outcome distributions \(Qwen3 30B\-A3B\-Instruct\-2507\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/google-gemini-3-flash-10p-5t_nc_stacked.png)\(a\)Negative control outcomes\.
![Refer to caption](https://arxiv.org/html/2605.20767v1/figures/movielens-100p-30t/google-gemini-3-flash-10p-5t_outcome_stacked.png)\(b\)Primary outcomes\.

Figure 27:MovieLens outcome distributions \(Gemini 3 Flash\)\. At each adjustment iteration,A=0A=0is on the left andA=1A=1is on the right\.