Position: It's Time to Optimize LLMs for Self-Consistency

# Position: It’s Time to Optimize LLMs for Self-Consistency
Source: [https://arxiv.org/html/2608.05188](https://arxiv.org/html/2608.05188)
Belinda Z\. LiLaura RuisZifan Carl GuoKeya HuMehul DamaniIsha PuriEkdeep Singh LubanaJacob Andreas

###### Abstract

Despite ever\-increasing sophistication in language model \(LM\) training pipelines, important failures persist: models over\-personalize their behavior to individual users, exhibit incomplete logical generalization, and produce confident but incorrect responses\. We argue that these failures arise from a shared modeling assumption: that behavior can be specified and evaluated independently on single\-output pairs\. Many model failures are difficult or impossible to detect without reasoning about relationships between responses across inputs\. In this position paper, we propose*self\-consistency*as a framework for understanding these failures\. We observe that a wide variety of techniques designed to improve specific aspects of LM behavior—targeting properties as diverse as adversarial robustness and factual coherence—can be understood as special cases of a common “consistency optimization” procedure and addressed with a standard set of optimization tools\. The same framework can be used to specify emerging model capabilities, such as introspection and self\-improvement, by constraining a model’s behavior to be consistent with its own descriptions of that behavior\. We discuss what it would mean to develop generally consistent LMs, including the capabilities they would enable and the objections they raise\.

Machine Learning, ICML

![Refer to caption](https://arxiv.org/html/2608.05188v1/x1.png)Figure 1:\(A\) Standard SFT/RL trains responses independently, which can induce inconsistent outputs\. \(B\) Cross\-input optimization directly enforces consistency through a relational objectiveϕ\\phi\.## 1Introduction

Over thirty language models have now been trained at GPT\-4 scale\(Rahmanet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib72)\)\. Training data has been stretched to the limits of publicly available internet text\(Villaloboset al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib73)\), and frontier labs are turning to synthetic data to push the established data\-and\-model scaling paradigm further\(Grattafioriet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib74); Zenget al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib75), inter alia\)\. Yet a persistent set of failures continues to undermine even the largest language models \(LLMs\): current models prioritize user agreement over task\-relevant properties\(Fanouset al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib95)\), exhibit asymmetric or incomplete logical generalization\(Berglundet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib35); Lampinenet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib80)\), and generate confident yet incorrect responses\(Xionget al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib77); Liuet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib79)\)\.

These failures are typically treated as isolated shortcomings, addressed through remedies like curated datasets or prompt engineering\. In this paper, we argue that such remedies are unlikely to succeed without addressing the modeling assumption that underlies them: that model behavior can be specified and evaluated on each individual input, independent of how a model behaves on related inputs\.

Consider sycophancy \([Figure˜1](https://arxiv.org/html/2608.05188#S0.F1)\): A model’s response may change depending on beliefs expressed by the user, even when those signals are irrelevant to the underlying task\(Sharmaet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib92); Denisonet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib98); Chenget al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib93)\)\. Evaluated one interaction at a time, each response may appear coherent\. Only when multiple interactions are considered together does the issue become clear: the model’s behavior varies with user\-framing rather than remaining stable across contexts\.

This example, and many of the other failure modes listed above, can be understood as a failure of LMs’ self\-consistency\. We argue thatself\-consistency constraints are an important, general, and under\-studied class of desiderata for LMs\. Rather than specifying desired behavior input\-by\-input, self\-consistency treats the structured relationships*between*behaviors as the primary objects of analysis\. Exploiting structure in data distributions is central to machine learning: models generalize better when constrained to respect symmetries, invariances, and logical dependencies\. Such structure has typically been enforced architecturally, e\.g\. through invariant or equivariant networks\(Bronsteinet al\.,[2021](https://arxiv.org/html/2608.05188#bib.bib61)\), or via data augmentation that exposes the model to equivalent inputs during training\(Akyurek and Andreas,[2023](https://arxiv.org/html/2608.05188#bib.bib122)\)\. Self\-consistency offers an alternative principle: enforcing relational structure directly at the output level, by evaluating groups of predictions rather than single ones, which can often be done without reference to any external source of ground truth\. In Section[3](https://arxiv.org/html/2608.05188#S3), we show that a wide variety of specialized evaluation and training procedures from past work may be understood as special cases of a general self\-consistency objective that can be optimized with a common set of algorithmic tools\.

Beyond unifying existing methods, self\-consistency captures an emerging, more complex class of objectives: those involving models’ “self\-descriptions”\. A growing body of work uses models to describe their own behavior, critique their own outputs, or distill their own knowledge back into related predictions\(Liet al\.,[2024b](https://arxiv.org/html/2608.05188#bib.bib105); Damaniet al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib84); Plunkettet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib57)\)\. We argue that both faithful self\-description and self\-improvement are special cases of self\-consistency, in which the required consistency relation links a model’s behavior to its own descriptions or evaluations of that behavior\. This reframing suggests a reciprocal relationship between self\-description and self\-improvement: a model that can accurately identify its own failure modes is also one whose self\-diagnoses can be leveraged to correct them\. We formalize this as meta\-level self\-consistency in Section[4](https://arxiv.org/html/2608.05188#S4), and illustrate its utility in a suite of case studies covering both existing and novel applications, from calibration and faithful chain\-of\-thought to automated self\-red\-teaming and hypothesis generation\.

In earlier deep learning paradigms, model behavior was principally shaped by fixing a dataset and varying aspects of model architecture—e\.g\. using convolutional filters to enforce translation equivariance, recurrence to impose temporal structure, and graph networks for relational reasoning\. In the modern paradigm, architectures evolve mainly to support scale and efficient data processing, with*data*used to specify desired model properties\. This shift has been extraordinarily powerful, enabling models to acquire behaviors that are difficult or impossible to specify architecturally\. At the same time, it has left the research community without a language to describe desiderata for LMs that cannot be expressed at the level of single datapoints\.

We believe that the language of self\-consistency provides an important first step toward meeting this challenge—as an organizing framework for understanding well\-studied phenomena like sycophancy, factual consistency, and chain\-of\-thought faithfulness; as a tool for specifying emerging capabilities, such as self\-description, that cannot be captured by input\-level evaluation; and most generally as a starting point for thinking about what it means for a LM to behave correctly beyond the scope of a single interaction\.

## 2Technical Formulation

Current objectives used to train language models predominantly optimize for model behavior on each input independently\. Letpθ​\(y∣x\)p\_\{\\theta\}\(y\\mid x\)denote the probability ofyygivenxxunder a language model parameterized byθ\\theta\. Supervised fine\-tuning objectives maximize LMs’ likelihood of reproducing input–output relations\(x,y\)\(x,y\)from some datasetDD:

maxθ​∑\(x,y\)∈Dlog⁡pθ​\(y∣x\)\.\\max\_\{\\theta\}\\sum\_\{\(x,y\)\\in D\}\\log p\_\{\\theta\}\(y\\mid x\)\.\(1\)Reinforcement learning objectives like RLVR\(Guoet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib89)\)and RLHF\(Ouyanget al\.,[2022](https://arxiv.org/html/2608.05188#bib.bib90)\)objectives maximize LMs’ outputsy^∼pθ\(⋅∣x\)\\hat\{y\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\)on data inputsx∈Dxx\\in D\_\{x\}, with respect to an externally\-defined reward functionReward:

maxθ​∑x∈Dx𝔼y^∼pθ\(⋅∣x\)​\[Reward​\(x,y^\)\]\.\\max\_\{\\theta\}\\sum\_\{x\\in D\_\{x\}\}\\mathbb\{E\}\_\{\\hat\{y\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\)\}\\left\[\\textsc\{Reward\}\(x,\\hat\{y\}\)\\right\]\.\(2\)Our main observation in this paper is that many important desiderata for language models must be defined not in terms of single samples, but in terms of*relations between multiple samples*\. We call this class of desiderata “self\-consistency” constraints\. While globally consistent prediction may emerge from optimizing pointwise objectives at scale, this property primarily requires the generative process have certain regularities, the model is sufficiently expressive, and the training data has sufficient coverage\(Weiet al\.,[2021](https://arxiv.org/html/2608.05188#bib.bib88);vonkügelgen2022selfsupervisedlearningdataaugmentations; Brehmeret al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib119)\)\. The final point is salient: we are likely close to saturating the scale of data we can easily obtain\(Villaloboset al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib73)\), but robustness, sycophancy, and factual consistency issues are still prevalent\.

Thus, instead of optimizing pointwise objectives, we propose to reason about a generalconsistency functionon some set of model input\-output pairs\. Formally, for any pair of related inputsx1x\_\{1\}andx2x\_\{2\}\(e\.g\., paraphrases, counterfactuals, temporal variants, or different user framings of the same request\), we can constrain their corresponding model outputsy1∼pθ\(⋅∣x1\)y\_\{1\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{1\}\)andy2∼pθ\(⋅∣x2\)y\_\{2\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{2\}\)with a consistency functionϕ\\phithat assigns higher values to more consistent pairs of responses \(e\.g\. “return 0 ify1y\_\{1\}andy2y\_\{2\}are factually consistent, and \-1 otherwise”\)\.ϕ\\phimay encode equivalence, logical entailment, factual consistency, normative consistency, or any other relational constraint between responses\. Given such a constraint, we would like our language model to produce outputs that, on everyx1,x2x\_\{1\},x\_\{2\}, causeϕ\\phito be large:

𝔼y1∼pθ\(⋅∣x1\),y2∼pθ\(⋅∣x2\)​\[ϕ​\(x1,y1,x2,y2\)\]\.\\mathbb\{E\}\_\{y\_\{1\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{1\}\),y\_\{2\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{2\}\)\}\\left\[\\phi\(x\_\{1\},y\_\{1\},x\_\{2\},y\_\{2\}\)\\right\]\.\(3\)
![Refer to caption](https://arxiv.org/html/2608.05188v1/x2.png)Figure 2:Framework Overview\. In the general self\-consistency framework \(left\), a consistency functionϕ\\phijointly scores sets of input\-output pairs\. We instantiate two relationship types: instance/instance self\-consistency \(middle\), where a relationRRon inputs implies a corresponding relationSSon outputs; and meta/instance self\-consistency \(right\), whereϕ\\phiscores a meta\-level input\-output pair for consistency with model behavior across a set of instancesDD\.In the general case, we can also defineϕ\\phito measure consistency relationships across a larger collection of inputsx1:n=x1,⋯,xnx\_\{1:n\}=x\_\{1\},\\cdots,x\_\{n\}\. Lety1:n=y1,⋯,yny\_\{1:n\}=y\_\{1\},\\cdots,y\_\{n\}denote corresponding model outputs withyi∼pθ\(⋅∣xi\)y\_\{i\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{i\}\)\. Our consistency objective becomes:

𝔼y1∼pθ\(⋅∣x1\),⋯,yn∼pθ\(⋅∣xn\)\[ϕ\(x1,⋯,xn,y1,⋯,yn\)\]=𝔼y1:n∼pθ\(⋅∣x1:n\)​\[ϕ​\(x1:n,y1:n\)\]\.\\displaystyle\\begin\{split\}&\\mathbb\{E\}\_\{y\_\{1\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{1\}\),\\cdots,y\_\{n\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{n\}\)\}\\big\[\\\\ &\\qquad\\qquad\\phi\(x\_\{1\},\\cdots,x\_\{n\},y\_\{1\},\\cdots,y\_\{n\}\)\\big\]\\\\ &=~\\mathbb\{E\}\_\{y\_\{1:n\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{1:n\}\)\}\\big\[\\phi\(x\_\{1:n\},y\_\{1:n\}\)\\big\]\.\\end\{split\}\(4\)
##### Advantages of a unified framework: shared training objectives\.

Expressing consistency constraints in terms of a common objective allows us to decouple the semantics of the constraint from the optimization procedure, thus suggesting a variety of mechanisms for imposing consistency\. Below, we useℒ​\(θ\)\\mathcal\{L\}\(\\theta\)denote a standard pointwise training objective for language models, such as the supervised learning objective from[Equation˜1](https://arxiv.org/html/2608.05188#S2.E1)or \(negative\) KL divergence from a “reference” model\. In general, to prevent trivial solutions, we want to optimize both pointwise loss with respect to ground\-truth data/rewards \(ℒ\\mathcal\{L\}\) as well as self\-consistency loss \(ϕ\\phi\)\. Here we give three examples of such loss functions:

1. 1\.Hard constraint:enforceϕ\\phias a hard constraint on the pointwise loss\. Here, the model is optimized only over the subset of parameters that exactly satisfy the desired consistency constraints\. maxθ⁡ℒ​\(θ\)​s\.t\.​𝔼y1:n∼pθ\(⋅∣x1:n\)​\[ϕ​\(x1:n,y1:n\)\]\.\\displaystyle\\small\\hskip\-9\.24994pt\\max\_\{\\theta\}\\ \\mathcal\{L\}\(\\theta\)~~\\text\{s\.t\.\}~~\\mathbb\{E\}\_\{y\_\{1:n\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{1:n\}\)\}\[\\phi\(x\_\{1:n\},y\_\{1:n\}\)\]\\,\.\(5\)
2. 2\.Soft constraint / regularizer:add consistency as a regularization term to the training objective\. The hyperparameterλ\\lambdacontrols the tradeoff between fitting the original objective and encouraging self\-consistency\. maxθ⁡\[ℒ​\(θ\)\+λ​𝔼y1:n∼pθ\(⋅∣x1:n\)​\[ϕ​\(x1:n,y1:n\)\]\]\.\\displaystyle\\small\\hskip\-9\.24994pt\\max\_\{\\theta\}\\ \\left\[\\mathcal\{L\}\(\\theta\)\+\\lambda\\,\\mathbb\{E\}\_\{y\_\{1:n\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{1:n\}\)\}\\left\[\\phi\(x\_\{1:n\},y\_\{1:n\}\)\\right\]\\right\]\.\(6\)
3. 3\.Posterior regularization\(Ganchevet al\.,[2010](https://arxiv.org/html/2608.05188#bib.bib116)\):project the model distribution toward the closest distribution that satisfies the consistency constraints\. For any set of related inputsx1:nx\_\{1:n\}, defineQx1:n=\{q:𝔼y1:n∼q\(⋅∣x1:n\)​ϕ​\(x1:n,y1:n\)≥0\}Q\_\{x\_\{1:n\}\}=\\\{q:\\mathbb\{E\}\_\{y\_\{1:n\}\\sim q\(\\cdot\\mid x\_\{1:n\}\)\}\\phi\(x\_\{1:n\},y\_\{1:n\}\)\\geq 0\\\}as the set of functionsq:x1:n↦y1:nq:x\_\{1:n\}\\mapsto y\_\{1:n\}that satisfy consistency constraintϕ\\phiacross all inputsx1:nx\_\{1:n\}\. Then choose: maxθ⁡\[ℒ​\(θ\)\+minq∈Qx1:n⁡KL​\(q∥pθ\)\]\.\\displaystyle\\small\\max\_\{\\theta\}\\ \\left\[\\mathcal\{L\}\(\\theta\)\+\\min\_\{q\\,\\in\\,Q\_\{x\_\{1:n\}\}\}\\mathrm\{KL\}\\left\(q\\;\\\|\\;p\_\{\\theta\}\\right\)\\right\]\.\(7\)

As we will note in[Section˜3](https://arxiv.org/html/2608.05188#S3), many existing methods for LM post\-training may be understood as describing specialized procedures for optimizing a version of[Equation˜4](https://arxiv.org/html/2608.05188#S2.E4)\. A unified objective for self\-consistency allows us to use common pipelines for data generation, model optimization, and evaluation\. An extended discussion of other procedures for optimizing these objectives is provided in in[Appendix˜C](https://arxiv.org/html/2608.05188#A3)\.

##### Instance\- and Meta\-level Instantiations\.

In the remainder of this paper, we describe two broad classes of consistency functions that capture a number of desirable LM behaviors \(Figure[2](https://arxiv.org/html/2608.05188#S2.F2)\):

1. 1\.Instance\-level objectives\(§[3](https://arxiv.org/html/2608.05188#S3)\) broadly encode symmetric consistency constraints between model input\-output instances, where neither instance refers to other instances\. This includes model invariances and equivariances, encompassing robustness to spurious cues, bias, sycophancy, and model faithfulness\.
2. 2\.Meta\-level objectives\(§[4](https://arxiv.org/html/2608.05188#S4)\) are an emerging class of objectives that enforce consistency between aself\-referential“meta\-response” that describes model behavior on one or more other instance\-level examples\. These objectives enable model introspection, self\-description, self\-critique, and self\-improvement\. New instantiations could also enable new capabilities such as model self\-red\-teaming and hypothesis generation\.

## 3Instance\-level Instantiations

We begin by observing that common LM failure modes, including failures of sycophancy, robustness, faithfulness, and beyond, can be characterized via a self\-consistency functionϕ\\phi\. At a high level, self\-consistency constraints require that inputs that bear some relation \(e\.g\., paraphrases, counterfactuals, or reframings\) produce outputs that bear a corresponding relation \(e\.g\., semantically equivalent, logically entailed, or normatively consistent responses\)\.

Formally, letR​\(x1:n\)∈\{T,F\}R\(x\_\{1:n\}\)\\in\\\{\\texttt\{T\},\\texttt\{F\}\\\}andS​\(y1:n\)∈\{T,F\}S\(y\_\{1:n\}\)\\in\\\{\\texttt\{T\},\\texttt\{F\}\\\}denote relations on tuples of inputsxxand outputsyyrespectively\. By varyingRRandSS, we will show how to recover many existing alignment techniques via[Equation˜4](https://arxiv.org/html/2608.05188#S2.E4)by definingϕ\\phias:

ϕ​\(x1:n,y1:n\)=𝟙​\[R​\(x1:n\)→S​\(y1:n\)\]−1\.\\phi\(x\_\{1:n\},y\_\{1:n\}\)=\\mathbbm\{1\}\\left\[R\(x\_\{1:n\}\)\\to S\(y\_\{1:n\}\)\\right\]\-1\.\(8\)whereϕ\\phiis0if the model is consistent and−1\-1if inconsistent\.

Here, we focus on two representative families: invariances and equivariances\.

### 3\.1Invariances

*Invariances*are defined as transformations of the input that should leave the model’s output unchanged—constraints:

f​\(g​\(x\)\)=f​\(x\)​∀x,f\(g\(x\)\)=f\(x\)\\,\\,\\forall\\,\\,x,whereggis a transformation acting on the input space\. These can be instantiated in[Equation˜8](https://arxiv.org/html/2608.05188#S3.E8)by setting:R​\(x1,x2\)=𝟙​\[x2=g​\(x1\)\]R\(x\_\{1\},x\_\{2\}\)=\\mathbbm\{1\}\[x\_\{2\}=g\(x\_\{1\}\)\]to check for transformed inputs andS​\(y1,y2\)=𝟙​\[y2=y1\]S\(y\_\{1\},y\_\{2\}\)=\\mathbbm\{1\}\[y\_\{2\}=y\_\{1\}\]to enforce output equality\. To illustrate this class of consistency functions, we use model sycophancy as a case study\.

Case Study 1:SycophancyAs LLMs are post\-trained on human preference signals, they can become inadvertently incentivized to be sycophantic, shifting responses toward user beliefs even when doing so conflicts with correctness\(Sharmaet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib92); Chenget al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib93); Honget al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib94)\)\. This issue still prevails in production\-level models, with OpenAI rolling back GPT\-4o due to users growing frustrated with its overly sycophantic behavior\(OpenAI,[2025](https://arxiv.org/html/2608.05188#bib.bib19)\)\.A common approach measures sycophancy via counterfactual hints: fix the underlying task, vary only the stated user preference, and quantify how much the output shifts toward it\. For example,Sharmaet al\.\([2024](https://arxiv.org/html/2608.05188#bib.bib92)\)propose benchmarks for feedback sycophancy \(feedback becomes more positive with user\-preference hints\) and answer sycophancy \(QA answers shift toward stated user beliefs\)\. Other work extends this to moral endorsement and framing acceptance\(Chenget al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib93)\)\.Within our framework, sycophancy can be seen as a failure of invariance: changing implicit/explicit user preferences changes model behavior to be agreeable\. For answer sycophancy as an example, we can defineRRandSSas:R​\(x1,x2\)=x1andx2semantically equivalentbut differ in revealed user preference\.\\small R\(x\_\{1\},x\_\{2\}\)=\\begin\{aligned\} \\small&\\text\{$x\_\{1\}$ and $x\_\{2\}$ semantically equivalent\}\\\\ &\\text\{but differ in revealed user preference\.\}\\end\{aligned\}\(9\)S​\(y1,y2\)=y1andy2are semantically equivalent\.\\small S\(y\_\{1\},y\_\{2\}\)=\\text\{$y\_\{1\}$ and $y\_\{2\}$ are semantically equivalent\}\.\(10\)We can use the template in[Equation˜8](https://arxiv.org/html/2608.05188#S3.E8)to define a sycophancy constraintϕsyco\\phi\_\{\\text\{syco\}\}: i\.e\., we penalize only when hints differ, but the model’s answers do not remain invariant\. Recent work directly optimizes invariance to biasing cues through output\-level, activation\-level, or behavior\-rate consistency objectives\(Chuaet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib134); Irpanet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib135); Imranet al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib137)\)\. However, consistency training can also amplify sycophancy in controlled model organisms\(Africa and Mani,[2026](https://arxiv.org/html/2608.05188#bib.bib138)\)\.

By redefiningR,S,ϕR,S,\\phiin a manner akin to the example above, we can define objectives for reducing prompt paraphrase inconsistencies\(Sclaret al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib76)\), model bias\(Gallegoset al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib68)\), context drift\(Liet al\.,[2024a](https://arxiv.org/html/2608.05188#bib.bib121)\), and reasoning inconsistency\(Prasadet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib1)\)\.

### 3\.2Equivariances

Classically, a modelffis equivariant under a transformationggif applyingggto the input induces a corresponding transformation on the output:

f​\(g​\(x\)\)=g′​\(f​\(x\)\)​∀x,f\(g\(x\)\)=g^\{\\prime\}\(f\(x\)\)\\,\\,\\forall\\,\\,x,\(11\)whereggandg′g^\{\\prime\}are transformations acting on the input and output spaces, respectively\. As with invariances, we recover the classical definition by definingR​\(x1,x2\)=𝟙​\[x2=g​\(x1\)\]R\(x\_\{1\},x\_\{2\}\)=\\mathbbm\{1\}\[x\_\{2\}=g\(x\_\{1\}\)\]andS​\(y1,y2\)=𝟙​\[y2=g′​\(y1\)\]S\(y\_\{1\},y\_\{2\}\)=\\mathbbm\{1\}\[y\_\{2\}=g^\{\\prime\}\(y\_\{1\}\)\]\. We now use*factual consistency*as a case study for equivariance\.

Case Study 2:Factual ConsistencyFactual consistency requires models to remain logically consistent when asked different questions and upon learning new information, which can be interpreted as equivariance under*knowledge modification*in our framework\. We would like LMs’ factual predictions to be grounded in an internally consistent, latent*world state*\(Liet al\.,[2025b](https://arxiv.org/html/2608.05188#bib.bib107)\)\. Knowledge updates should then act as transformation on this world state, inducing a transformation on all other predictions entailed or contradicted by that fact\. Logical coherence requires that the updated world state propagate consistently across all dependent queries, without producing contradictions or stale beliefs\(Akyüreket al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib40); Padmanabhanet al\.,[2023](https://arxiv.org/html/2608.05188#bib.bib108); Haseet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib39)\)\.Suppose we train the LM such that it learns new factfact=\\textsc\{fact\}=“Meiji was the last emperor of modern Japan”, meaning whenx1=factx\_\{1\}=\\textsc\{fact\}, it places high probability ony1=y\_\{1\}=“ is True”\. Then on all statements that contradict the learned fact, e\.g\.x2=x\_\{2\}=“the Meiji era occurred during Japan’s feudal period”, it should place high probability ony2=y\_\{2\}=“ is False”\. Our consistency rule can thus be written:R​\(x1,x2\)=𝟙​\[x1​contradicts​x2\]R\(x\_\{1\},x\_\{2\}\)=\\mathbbm\{1\}\\left\[x\_\{1\}\\text\{ contradicts \}x\_\{2\}\\right\]\(12\)S​\(y1,y2\)=𝟙​\[y1​contradicts​y2\]S\(y\_\{1\},y\_\{2\}\)=\\mathbbm\{1\}\\left\[y\_\{1\}\\text\{ contradicts \}y\_\{2\}\\right\]\(13\)By defining aϕfactual\\phi\_\{\\text\{factual\}\}using the aboveR,SR,Sand Eq\.[8](https://arxiv.org/html/2608.05188#S3.E8), our metric ensures the LM does not simply memorize a pointwise edit to its knowledge, but propagates its logical implications\. Prior work has used objectives analogous to one round of posterior regularization\(Akyüreket al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib40)\)or supervised fine\-tuning on a consistent subset of model outputs\(Liet al\.,[2024b](https://arxiv.org/html/2608.05188#bib.bib105)\)to optimize this objective\.

By expanding the class of relations we consider withSS, we can capture a wider set of properties desirable for models, such as pluralistic alignment\(Sorensenet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib60)\), reversal curse\(Berglundet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib35)\), and multi\-lingual consistency\(Iferganet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib120)\)\. See[Section˜A\.1](https://arxiv.org/html/2608.05188#A1.SS1)and[Section˜A\.2](https://arxiv.org/html/2608.05188#A1.SS2)for additional examples and discussion\.

## 4Emerging Methods: Meta\-Level Self\-Consistency

In the prior section, we showed how some systematic failures of current LMs can be modeled asinstance\-levelself\-consistency relations\. An emerging body of work, however, focuses on imbuing LMs with the ability to produce faithfulmeta\-level descriptionsof their computations, behavioral traits, uncertainties, and decision rules\(Plunkettet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib57); Liet al\.,[2025a](https://arxiv.org/html/2608.05188#bib.bib55); Damaniet al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib84)\)\. A second related line of work uses LMs’ abilities to provide feedback as training signal toimprovetheir underlying behavior\(Liet al\.,[2024b](https://arxiv.org/html/2608.05188#bib.bib105)\)\. These works impose self\-consistency betweenmeta\-level descriptions andinstance\-level behaviors\.

LetD=\{\(xiD,yiD\)\}i=1KD=\\\{\(x\_\{i\}^\{D\},y\_\{i\}^\{D\}\)\\\}\_\{i=1\}^\{K\}denote our model’s instance\-level input\-output pairs \(xiDx\_\{i\}^\{D\},yiDy\_\{i\}^\{D\}\)\. Suppose we have a meta\-level questionxmetax\_\{\\text\{meta\}\}with a meta\-level responseymetay\_\{\\text\{meta\}\}\. In meta\-level consistency,ϕ\\phichecks whether model behaviorsy1:nDy\_\{1:n\}^\{D\}on instancesx1:nDx\_\{1:n\}^\{D\}are consistent with the meta\-level descriptions\(xmeta,ymeta\)\(x\_\{\\text\{meta\}\},y\_\{\\text\{meta\}\}\)\.

Broadly, may consider two classes of meta\-level self\-consistency objectives based on the optimization target:

1. 1\.Self\-description objectivesaim to shift LMs’ self\-descriptions\(xmeta,ymeta\)\(x\_\{\\text\{meta\}\},y\_\{\\text\{meta\}\}\)to align with their instance\-level behaviorsDD, enabling introspection, faithful chain\-of\-thought, calibration, etc\. Self\-descriptions can also enable a number of new capabilities, including self\-red\-teaming and scientific hypothesis generation\.
2. 2\.Self\-alignment objectivesaim to shift model behaviorsDDto better reflect their own descriptions\(xmeta,ymeta\)\(x\_\{\\text\{meta\}\},y\_\{\\text\{meta\}\}\), enabling model self\-improvement via critique or reflection\. These objectives have focused on using self\-consistency as ameansfor building more generally capable models, by leveraging model meta\-descriptions as supervision signal\.

These objectives illustrates the dual utility of consistency objectives: optimizing self\-consistency can both \(1\) enforce globally coherent model behavior and \(2\) provide a source of training signal for broader model capabilities\. We also posit that enforcing one has a positive effect on the other: more aligned models are easier to explain, while more explainable models can also be easier to control\.

### 4\.1Self\-Descriptions

Table 1:Taxonomy of self\-description tasks\. Rows indicate the object being explained \(internal mechanisms, behavioral traits, uncertainty, or weights\)\. Columns indicate the explanation type:counterfactualexplanations describe differences between minimal pairs, whilepropertyexplanations aggregate over many instances to identify shared characteristics\.A growing body of recent work trains or elicits language models to produce natural\-language descriptions of aspects of themselves: their behavioral tendencies\(Plunkettet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib57); Binderet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib58); Chenet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib126); Wenet al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib127); Burnset al\.,[2023](https://arxiv.org/html/2608.05188#bib.bib128)\), internal computations\(Liet al\.,[2025a](https://arxiv.org/html/2608.05188#bib.bib55)\), fine\-tuning–induced weight differences\(Goelet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib56)\), and uncertainties\(Damaniet al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib84)\)\. We refer to this emerging family of approaches asself\-descriptionmethods\. Our framework provides a unified way to formalize and compare these disparate introspection tasks\. We identify two general types of explanations:

1. 1\.Counterfactual explanationsexamine two minimally different inputsx1D,x2Dx\_\{1\}^\{D\},x\_\{2\}^\{D\}that induce different model behaviorsy1D,y2Dy\_\{1\}^\{D\},y\_\{2\}^\{D\}, and explain the input difference that induced the change or behavioral change itself\.
2. 2\.Property descriptionsaggregate many input instancesx1:nDx\_\{1:n\}^\{D\}and identify shared properties of outputsy1:nDy\_\{1:n\}^\{D\}\.

We can generate explanations of each of the above types on different types of instance\-level objects\. By varying the instance\-level objects the model explains, we can recover different classes of explanations\. See[Table˜1](https://arxiv.org/html/2608.05188#S4.T1)\.

Case Study 3:Explaining Causal MechanismsInterpretability methods, such as activation patching\(Menget al\.,[2022](https://arxiv.org/html/2608.05188#bib.bib32); Geigeret al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib15)\), aim to identify causal mechanisms that explains model behaviors\. For example, given “Paris is the capital of”→\\to“France,” we can test whether tokenttat layerℓ\\ellencodes city\-country information by patching activations from “Rome is the capital of” and checking if the output flips to “Italy\.”Rather than relying on external interpretability tools, introspective methods enable a model to answer meta\-level questions about its own internals\. Lety1Dy\_\{1\}^\{D\},y2Dy\_\{2\}^\{D\}denote the model’s behavior on an original inputx1Dx\_\{1\}^\{D\}and intervened inputx2D=patch​\(x1D\)x\_\{2\}^\{D\}=\\textsc\{patch\}\(x\_\{1\}^\{D\}\), wherepatchdenotes the activation patching transformation\. An introspective explanation corresponds to a meta\-level query\(xmeta,ymeta\)\(x\_\{\\text\{meta\}\},y\_\{\\text\{meta\}\}\)describing how outputs would change under the intervention \(herexmetax\_\{\\text\{meta\}\}might be a user question like “In which layer is the location of Paris retrieved?”\)\. We define the consistency function for mechanistic introspection asϕinterv\(\(xmeta,ymeta\),\(x1D,y1D\),\(x2D,y2D\)\)=𝟙\[ymeta​explains change between\(y1D,y2D\)\]−1\.\\displaystyle\\begin\{split\}\\small\\phi\_\{\\text\{interv\}\}\\big\(\(x\_\{\\text\{meta\}\},&\\,y\_\{\\text\{meta\}\}\),\(x\_\{1\}^\{D\},y\_\{1\}^\{D\}\),\(x\_\{2\}^\{D\},y\_\{2\}^\{D\}\)\\big\)\\\\ =\\mathbbm\{1\}\\Big\[&y\_\{\\text\{meta\}\}\\text\{ explains change between \}\\\\ &\(y\_\{1\}^\{D\},y\_\{2\}^\{D\}\)\\Big\]\-1\.\\end\{split\}\(14\)Thusϕinterv\\phi\_\{\\text\{interv\}\}evaluates whether the model’s self\-described explanation correctly predicts its counterfactual behavior \(e\.g\., whether the output flips\)\. Prior work\(Liet al\.,[2025a](https://arxiv.org/html/2608.05188#bib.bib55)\)has used objectives analogous to[Equation˜7](https://arxiv.org/html/2608.05188#S2.E7)to optimize this objective\.

Case Study 4:Chain\-of\-Thought FaithfulnessAs large reasoning models become increasingly commonplace, there is growing interest in using their chains\-of\-thought as an interpretability and monitoring tool\(Korbaket al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib103); Guanet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib46)\)\. However, LM\-generated chains\-of\-thought are not guaranteed to be faithful to their underlying decision\-making process, making them imprecise, fragile, and misleading\(Barezet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib47)\)\.In prior work, chain\-of\-thought faithfulness has been measured in several different ways\. Particularly salient to this section isChenet al\.\([2025](https://arxiv.org/html/2608.05188#bib.bib117)\), which constructs counterfactual inputs and examine whether the chain\-of\-thought reflects the decision rule implied by the counterfactual \(though other methods, likeLanhamet al\.\([2023](https://arxiv.org/html/2608.05188#bib.bib118)\); Hase and Potts \([2026](https://arxiv.org/html/2608.05188#bib.bib130)\)’s, can also be thought of as other forms of self\-consistency\)\. Letppdenote a prompt,c∼pθ\(⋅∣p\)c\\sim p\_\{\\theta\}\(\\cdot\\mid p\)a chain\-of\-thought, anda∼pθ\(⋅∣p,c\)a\\sim p\_\{\\theta\}\(\\cdot\\mid p,c\)the model’s final answer\.We constructp′p^\{\\prime\}as a counterfactual version ofppbased on factors that the chain\-of\-thoughtcmetac\_\{\\text\{meta\}\}said were decisive\. Letxmeta=p​cmetax\_\{\\text\{meta\}\}=pc\_\{\\text\{meta\}\},ymeta=ay\_\{\\text\{meta\}\}=a,x1D=p′x\_\{1\}^\{D\}=p^\{\\prime\}, andy1D∼pθ\(⋅∣x′\)y\_\{1\}^\{D\}\\sim p\_\{\\theta\}\(\\cdot\\mid x^\{\\prime\}\)\. The chain\-of\-thought is faithful if the model’s behavior onxmetax\_\{\\text\{meta\}\}andx1Dx\_\{1\}^\{D\}both match the behavior implied by its reasoningcmetac\_\{\\text\{meta\}\}, which we express asϕCoT\(xmeta,ymeta,x1D,y1D\)=𝟙\[xmeta​and​x1D​difference isexplained by CoT inxmeta\]−1\.\\displaystyle\\begin\{split\}\\small\\phi\_\{\\text\{CoT\}\}\(x\_\{\\text\{meta\}\},y\_\{\\text\{meta\}\},&\\,x\_\{1\}^\{D\},y\_\{1\}^\{D\}\)\\\\ =\\mathbbm\{1\}\\big\[&x\_\{\\text\{meta\}\}\\text\{ and \}x\_\{1\}^\{D\}\\text\{ difference is \}\\\\ &\\text\{explained by CoT in \}x\_\{\\text\{meta\}\}\\big\]\-1\.\\end\{split\}\(15\)While past work has mostly used these metrics to evaluate faithfulness of chain\-of\-thought, we can also leverage these objectives to train models to produce more faithful chain\-of\-thought\(Turpinet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib136)\)\.

Case Study 5:CalibrationIn both everyday and high\-stakes domains, users care not only about the accuracy of LMs, but also about their ability to express uncertainty\(Kalaiet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib91); Kirichenkoet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib81)\)\. A common instantiation of this is*confidence verbalization*\(Linet al\.,[2022](https://arxiv.org/html/2608.05188#bib.bib82)\), where a model outputs a numerical estimate of its probability of answering a question correctly, or of the correctness of its produced solution\. A growing body of work shows that modern LLMs are often systematically overconfident\(Xionget al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib77)\)and their calibration can further degrade after RL training\(Damaniet al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib84); Lenget al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib85)\)\.Calibration is a form of introspective self\-description: the LM must produce a faithful meta\-level description of its own expected behavior\. Here, we detail*question\-conditioned calibration*, where the model is asked to estimate its expected performance on a given question prior to producing an answer\.Consider a meta\-level query\(xmeta,qmeta\)\(x\_\{\\text\{meta\}\},q\_\{\\text\{meta\}\}\), wherexxis a fixed task input andq∈\[0,1\]q\\in\[0,1\]is the model’s stated probability that it will answerxmetax\_\{\\text\{meta\}\}correctly\. Let\{yiD\}i=1N\\\{y\_\{i\}^\{D\}\\\}\_\{i=1\}^\{N\}denote multiple stochastic samples of the model’s outputs for the same inputxmetax\_\{\\text\{meta\}\}, and letai∈\{0,1\}a\_\{i\}\\in\\\{0,1\\\}indicate whetheryiDy\_\{i\}^\{D\}is correct\. \(Note that, unlike other examples we have seen so far,aia\_\{i\}cannot be computed by the LM itself, and require access to some external source of ground truth\.\) We define the question\-conditioned calibration consistency function asϕcal​\(\(xmeta,qmeta\),\{\(xmeta,yiD,ai\)\}i=1N\)=score​\(qmeta,1N​∑i=1Nai\),\\displaystyle\\small\\begin\{split\}&\\phi\_\{\\text\{cal\}\}\\\!\\Big\(\(x\_\{\\text\{meta\}\},q\_\{\\text\{meta\}\}\),\\\{\(x\_\{\\text\{meta\}\},y\_\{i\}^\{D\},a\_\{i\}\)\\\}\_\{i=1\}^\{N\}\\Big\)\\\\ &\\;=\\;\\texttt\{score\}\\\!\\left\(q\_\{\\text\{meta\}\},\\;\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}a\_\{i\}\\right\),\\end\{split\}\(16\)wherescoreis a proper scoring rule measuring agreement between the model’s stated confidence and its empirical accuracy on repeated samples for the same question\(Gneiting and Raftery,[2007](https://arxiv.org/html/2608.05188#bib.bib83)\)\. Past work\(Damaniet al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib84)\)has used soft constraints \(Eq\.[6](https://arxiv.org/html/2608.05188#S2.E6)\) to optimize this objective and produce LMs whose self\-reported uncertainty estimates are faithful summaries of future behavior\.

### 4\.2New Self\-Description Capabilities

By applying our self\-description framework in different domains, we can also enable a number of new model capabilities, which we describe below:

Case Study 6:Automated Self\-Red\-TeamingRed\-teaming research devises attacks that surface LM vulnerabilities so that defenses can be developed\. Approaches include prompt injection, where harmful instructions are embedded in seemingly benign text\(Liuet al\.,[2023](https://arxiv.org/html/2608.05188#bib.bib50); Mehrotraet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib51)\), gradient\-based attacks that use optimization to search for malicious prompts or latent perturbations\(Geisleret al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib52)\), soft prompts, exploiting long\-context behavior, or targeting surrounding infrastructure\.By leveraging our self\-consistency framework, we are able to propose an alternative attack class: training models to generate their own attacks\. Letxmetax\_\{\\text\{meta\}\}be a prompt that specifies a guideline or constraint and asks the model to produce an input that would cause the model to violate it\. Letymeta∼Pθ\(⋅∣x\)y\_\{\\text\{meta\}\}\\sim P\_\{\\theta\}\(\\cdot\\mid x\)be the candidate attack prompt\. We then setx1D=ymetax\_\{1\}^\{D\}=y\_\{\\text\{meta\}\}and sampley1D∼Pθ\(⋅∣x1D\)y\_\{1\}^\{D\}\\sim P\_\{\\theta\}\(\\cdot\\mid x\_\{1\}^\{D\}\)\. The consistency function takes formϕrt\(xmeta,ymeta;x1D,y1D\)=𝟙\[y1D​violates constraint in​xmeta∧x1D=ymeta\]−1\\displaystyle\\begin\{split\}\\small\\phi\_\{\\text\{rt\}\}\(x\_\{\\text\{meta\}\},y\_\{\\text\{meta\}\};\\,&x\_\{1\}^\{D\},y\_\{1\}^\{D\}\)\\\\ =\\mathbbm\{1\}\\big\[&y\_\{1\}^\{D\}\\text\{ violates constraint in \}x\_\{\\text\{meta\}\}\\\\ &\\wedge x\_\{1\}^\{D\}=y\_\{\\text\{meta\}\}\\big\]\-1\\end\{split\}\(17\)whereviolates​\(xmeta,y1D\)\\mathrm\{violates\}\(x\_\{\\text\{meta\}\},y\_\{1\}^\{D\}\)is11wheny1Dy\_\{1\}^\{D\}violates the constraint specified inxmetax\_\{\\text\{meta\}\}, and0otherwise\.Optimizing this objective induces a form of automated self\-red\-teaming: the model is encouraged to propose promptsymetay\_\{\\text\{meta\}\}that, when reused as inputs \(x1D=ymetax\_\{1\}^\{D\}=y\_\{\\text\{meta\}\}\), lead the model to violate the constraint inxmetax\_\{\\text\{meta\}\}\. In practice, we can sample candidate attacksymetay\_\{\\text\{meta\}\}, execute them on the same model to obtainy1Dy\_\{1\}^\{D\}, and keep pairs\(ymeta,y1D\)\(y\_\{\\text\{meta\}\},y\_\{1\}^\{D\}\)that trigger violations\.

Case Study 7:Hypothesis Generation & AI for ScienceRecent AI systems, including LMs, have achieved striking results in scientific domains—for example, outperforming human doctors on breast cancer detection\(McKinneyet al\.,[2020](https://arxiv.org/html/2608.05188#bib.bib87)\)and achieving high accuracy on protein\-structure prediction\(Jumperet al\.,[2021](https://arxiv.org/html/2608.05188#bib.bib70)\)\. However, strong predictive performance does not automatically translate into*human\-understandable*scientific insight\.For example, suppose we want the LM to predict orbital mechanics\. Prior work\(Vafaet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib69)\)has found that models are usually able to perfectlypredictplanetary positions; leveraging our self\-consistency framework, we would like them to be able to alsoarticulatethe rules behind their prediction\. In this case, our instance set𝒟=\{\(xiD,yiD\)\}i=1N\\mathcal\{D\}=\\\{\(x\_\{i\}^\{D\},y\_\{i\}^\{D\}\)\\\}\_\{i=1\}^\{N\}is set of \(past planetary positions, current planet position\) tuples\. We train the model on this set, and check that it exhibits the correct generalization on held\-out instances\.xmetax\_\{\\text\{meta\}\}is then a prompt that elicits hypothesized rules from models, andymetay\_\{\\text\{meta\}\}is the model’s hypothesis \(e\.g\., a symbolic rule, program, or other executable description\) of the rules governing orbital mechanics\. In this case, the consistency function is:ϕsci​\(xmeta,ymeta;x1:kD,y1:kD\)=∑i=1K\(𝟙​\[ymeta​maps​xiD​to​yiD\]−1\)\\displaystyle\\begin\{split\}\\small&\\phi\_\{\\text\{sci\}\}\(x\_\{\\text\{meta\}\},y\_\{\\text\{meta\}\};\\,x\_\{1:k\}^\{D\},y\_\{1:k\}^\{D\}\)\\\\ &=\\sum\_\{i=1\}^\{K\}\\left\(\\mathbbm\{1\}\\left\[y\_\{\\text\{meta\}\}\\text\{ maps \}x\_\{i\}^\{D\}\\text\{ to \}y\_\{i\}^\{D\}\\right\]\-1\\right\)\\end\{split\}\(18\)A key practical constraint is that explanations must be simpler than the system they explain otherwise the procedure merely restates the predictor\. We do not claim that such explanations can be extracted for every recent application of AI in science, but we expect this approach to be useful in settings where compact, executable mechanisms exist and can be verified against data\.

### 4\.3Self\-Alignment

The field of self\-training, including RLAIF\(Leeet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib102); Baiet al\.,[2022](https://arxiv.org/html/2608.05188#bib.bib104)\), self\-critique\(Madaanet al\.,[2023](https://arxiv.org/html/2608.05188#bib.bib106)\), G\-V consistency\(Liet al\.,[2024b](https://arxiv.org/html/2608.05188#bib.bib105); Rodriguezet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib129)\), self\-debate\(Irvinget al\.,[2018](https://arxiv.org/html/2608.05188#bib.bib9)\), self\-bootstrapping\(Lee,[2013](https://arxiv.org/html/2608.05188#bib.bib113); Xuet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib114)\), self\-distillation\(Zhanget al\.,[2019](https://arxiv.org/html/2608.05188#bib.bib115); Caronet al\.,[2021](https://arxiv.org/html/2608.05188#bib.bib131); Shenfeldet al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib132)\), all leverage models themselves as training signal\. By doing so, closed training loops can be built to improve capabilities in a largely human unsupervised manner\. Thus, unlike prior sections which focus on self\-consistency as an end\-goal, these objectives treat self\-consistency as a training mechanism: consistency constraints are used to generate supervision to improve general model capabilities such as reasoning\.

Case Study 8:RLAIFIn RLAIF \(Reinforcement Learning with AI Feedback\), meta\-outputs areevaluationsof the quality of models’ own behaviors\. This is used as training signal to push models to maximize their own rewards\. Self\-improvement relies on high\-quality behaviors being easier toverifythan togenerate\.Suppose we are using RLAIF to improve a model’s summarization quality\(Leeet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib102)\)\. In this case,\(x1D,y1D\)\(x\_\{1\}^\{D\},y\_\{1\}^\{D\}\)is Reddit post and a corresponding model summary\. The meta level queryxmetax\_\{\\text\{meta\}\}consists of the reward model prompt and\(x1D,y1D\)\(x\_\{1\}^\{D\},y\_\{1\}^\{D\}\), which elicits a rewardymetay\_\{\\text\{meta\}\}from the model on whethery1Dy\_\{1\}^\{D\}was correct\. Then we encourage the models’ behavior on this instance to maximizeymetay\_\{\\text\{meta\}\}, using consistency function:ϕimprove​\(xmeta,ymeta,xiD,yiD\)=ymeta​log⁡pθ​\(y1D∣x1D\)\\displaystyle\\begin\{split\}\\small\\phi\_\{\\mathrm\{improve\}\}\(x\_\{\\text\{meta\}\},y\_\{\\text\{meta\}\},x\_\{i\}^\{D\},y\_\{i\}^\{D\}\)\\\\ =y\_\{\\text\{meta\}\}\\log p\_\{\\theta\}\\left\(y\_\{1\}^\{D\}\\mid x\_\{1\}^\{D\}\\right\)\\end\{split\}\(19\)Maximizingϕimprove\\phi\_\{\\mathrm\{improve\}\}via the objective in[Equation˜4](https://arxiv.org/html/2608.05188#S2.E4)yields the RLAIF objective with self\-as\-judge\. This corresponds to optimizing this consistency objective using soft constraints \(eq\.[6](https://arxiv.org/html/2608.05188#S2.E6)\)\.

Many classes of self\-explanations studied in[Sections˜4\.1](https://arxiv.org/html/2608.05188#S4.SS1)and[4\.2](https://arxiv.org/html/2608.05188#S4.SS2)can be optimized in thereversedirection for alignment\. For instance, for self\-red\-teaming, discovered attacks can be fed back into training as adversarial data\. The model can be fine\-tuned to refuse or otherwise avoid producing the undesired behavior when prompted withymetay\_\{\\text\{meta\}\}\. Furthermore, LMs are often able to detect instances of misalignment, and often*self\-report*that they behaved in aligned ways\(Hanet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib44); Ahmedet al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib43)\)\. This can be leveraged as supervision signal to train models to more robustly express the behavioral traits they claim to have\. Finally, by pairing self\-explanations together with self\-alignment, this creates a scalable loop for both discovering and mitigating failures within the same self\-consistency framework\.

See[Section˜A\.3](https://arxiv.org/html/2608.05188#A1.SS3)for additional examples and discussion\.

## 5Alternative Views

We anticipate two types of objections to the position advanced in this paper: practical concerns about optimization and performance, and conceptual concerns about safety, emergent agency, and whether consistency is always a desirable objective\. In this section we address the former, while we outline the latter in[Appendix˜B](https://arxiv.org/html/2608.05188#A2)\.

A natural objection to the self\-consistency framework is that it is merely relocating the core difficulty of directly addressing LLM failures and capabilities to the specification of the appropriate relational constraints \(ϕ\\phi\) and the collection of corresponding sets of inputs and outputs\. However, we argue that this shift is precisely what makes self\-consistency more tractable: specifying relational constraints over input–output pairs is often easier than directly specifying consistent behavior\. Importantly, self\-consistency does not require teaching models new behaviors, but rather constraining behaviors models can already produce\. Evaluating whether two responses are consistent, identifying when a constraint is violated, or proposing related inputs is substantially easier than generating the consistent response from scratch\. Modern LLMs can therefore be used as scalable judges of relational properties \(e\.g\. specifyingϕ\\phivia a prompt\) and as tools for constructing structured sets of related inputs and outputs\.

Nonetheless, enforcing self\-consistency raises nontrivial optimization questions\. Degenerate fixed points may satisfy a given consistency metric \(ϕ\\phi\) while producing unhelpful or undesirable outputs\. For instance, models that converge to overly conservative behaviors that trivially avoid contradiction\. This failure mode suggests that self\-consistency training should not operate in isolation\. We envision two viable approaches: training for self\-consistency jointly with standard SFT or RLHF objectives that reward helpfulness and specificity, or applying self\-consistency as a post\-training refinement with regularization to prevent excessive drift from the pre\-trained policy\.

A third practical concern is capability trade\-offs\. Enforcing consistency constraints may restrict the hypothesis space in ways that degrade performance on tasks where local flexibility or context\-sensitivity is beneficial\. Characterizing when self\-consistency improves robustness versus when it induces harmful rigidity remains an open empirical question\. A major barrier to deploying LLMs in high\-stakes domains is that their predictions are opaque\. A model that could consistently explain its own behavior, even at some cost to raw accuracy, would unlock applications where current systems are too brittle to trust\. Self\-consistency thus offers a path toward models that are not only more predictable, but more auditable\.

Finally, we emphasize that the basic idea of designing training objectives to exploit structure in the space of desired model behaviors is not a new one\. Prior work has long emphasized enforcing symmetries and invariances architecturally, \(e\.g\. with graphical mdoels, or equivariant or invariant networks\), or indirectly \(e\.g\. via data augmentation\)\. In many settings, such approaches have yielded limited gains relative to scale, and have often been rendered obsolete by more general architectures that can efficiently process large\-scale data\. This history suggests caution toward hand\-engineered inductive biases\. Our proposal differs in where structure is enforced\. Rather than constraining model architecture or representational capacity, self\-consistency exploits regularities present in data at the level of behavior, through evaluation procedures and constraints that link a model’s predictions across related contexts\. This allows models to benefit from structural regularities where they exist, without sacrificing the flexibility or throughput afforded by general\-purpose architectures \(for a related discussion seeWilson,[2025](https://arxiv.org/html/2608.05188#bib.bib133)\)\.

## 6Discussion

We argue that self\-consistency, the coherence of a model’s behavior across inputs, provides a unifying lens for understanding key failure modes in large language models\. Rather than treating these failures as isolated phenomena requiring separate interventions, we suggest they share a common underlying cause that can be approached in a unified way\. This perspective exploits structure already present in data by enforcing relationships among predictions without relying on additional data, synthetic augmentation, or task\-specific patches\. Beyond patching existing failure modes, we argue that self\-consistency may serve as a foundation for improved capabilities \(more robust reasoning\) and safety \(more predictable, auditable behavior\)\.

### 6\.1Closing Thoughts

The position we advance in this paper is not that language models should be made perfectly consistent\. Rather, we argue that many persistent model limitations reflect specific, structured inconsistencies that can be directly targeted\. Beyond addressing existing failures, self\-consistency also provides a way to develop emerging capabilities such as introspection, creating new levers for interpretability and model auditing\.

By unifying a wide range of failures and emerging capabilities under a single self\-consistency objective, this framework suggests a way of revisiting structural constraints that are compatible with the way modern LLMs are developed\. In particular, it motivates post\-training approaches that target consistency at a general level, rather than introducing bespoke fixes for individual failure modes or capabilities\. Much as instruction fine\-tuning induces a general ability to follow instructions beyond the specific formats seen during training, self\-consistency post\-training may encourage models to more broadly respect consistency relations implicit in the data\-generating process, enabling generalization beyond the specific consistency relations and input–output pairs used during training\.

## Acknowledgments

This work was supported by NSF award IIS\-2238240, the MIT Generative AI Impact Consortium, Coefficient Giving, the MIT\-IBM Watson Computing Lab, the IARPA BENGAL program, and the DARPA AIQ program through the DARPA CMO contract number HR00112520025\. IP is additionally supported by an NSF Graduate Fellowship, BZL is supported by a Clare Boothe Luce Fellowship, KH is supported by a Schwartzman College of Computing Fellowship and JA is supported by a Sloan Fellowship\.

## References

- D\. D\. Africa and A\. Mani \(2026\)Consistency training can entrench misalignment\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=xES5qpQmw8)Cited by:[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p2.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.3)\.
- A\. M\. Ahmed, K\. Klyman, Y\. Zeng, S\. Koyejo, and P\. Liang \(2026\)SpecEval: evaluating model adherence to behavior specifications\.Transactions on Machine Learning Research\.Note:External Links:[Link](https://openreview.net/forum?id=VzLIQ3Lqm9),ISSN 2835\-8856Cited by:[§A\.3](https://arxiv.org/html/2608.05188#A1.SS3.SSS0.Px1.p1.1),[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p3.1)\.
- A\. F\. Akyürek, E\. Akyürek, L\. Choshen, D\. T\. Wijaya, and J\. Andreas \(2024\)Deductive closure training of language models for coherence, accuracy, and updatability\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 9802–9818\.Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p2.10),[§C\.4](https://arxiv.org/html/2608.05188#A3.SS4.p4.1),[§3\.2](https://arxiv.org/html/2608.05188#S3.SS2.p2.pic1.8.8.8.8.8.8.8.8.8.8.8.7.7.7.7.7.7.7),[§3\.2](https://arxiv.org/html/2608.05188#S3.SS2.p2.pic1.8.8.8.8.8.8.8.8.8.8.8.7.7.7.7.7.7.8)\.
- E\. Akyurek and J\. Andreas \(2023\)LexSym: compositionality as lexical symmetry\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 639–657\.External Links:[Link](https://aclanthology.org/2023.acl-long.38/),[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.38)Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p4.1)\.
- Anthropic \(2025\)Claude opus 4\.5 system card\.Technical reportAnthropic\.Note:Accessed 2026\-01\-19External Links:[Link](https://assets.anthropic.com/m/64823ba7485345a7/Claude-Opus-4-5-System-Card.pdf)Cited by:[Appendix B](https://arxiv.org/html/2608.05188#A2.p3.1)\.
- S\. Aranguri and T\. McGrath \(2025\)Discovering undesired rare behaviors via model diff amplification\.Goodfire\.Note:https://www\.goodfire\.ai/research/model\-diff\-amplificationCited by:[Appendix B](https://arxiv.org/html/2608.05188#A2.p3.1)\.
- Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon,et al\.\(2022\)Constitutional ai: harmlessness from ai feedback\.arXiv preprint arXiv:2212\.08073\.Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p3.1),[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p1.1)\.
- F\. Barez, T\. Wu, I\. Arcuschin, M\. Lan, V\. Wang, N\. Siegel, N\. Collignon, C\. Neo, I\. Lee, A\. Paren, A\. Bibi, R\. Trager, D\. Fornasiere, J\. Yan, Y\. Elazar, and Y\. Bengio \(2025\)Chain\-of\-thought is not explainability\.Note:AlphaXiv preprint, version 1External Links:[Link](https://www.alphaxiv.org/abs/2025.02v1)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p5.pic1.14.14.14.14.14.14.14.14.14.14.14.13.13.13.13.13.13.14)\.
- Y\. Bengio, M\. Cohen, D\. Fornasiere, J\. Ghosn, P\. Greiner, M\. MacDermott, S\. Mindermann, A\. Oberman, J\. Richardson, O\. Richardson, M\. Rondeau, P\. St\-Charles, and D\. Williams\-King \(2025\)Superintelligent agents pose catastrophic risks: can scientist ai offer a safer path?\.External Links:2502\.15657,[Link](https://arxiv.org/abs/2502.15657)Cited by:[Appendix B](https://arxiv.org/html/2608.05188#A2.p2.1)\.
- L\. Berglund, A\. C\. Stickland, M\. Balesni, M\. Kaufmann, M\. Tong, T\. Korbak, D\. Kokotajlo, and O\. Evans \(2023\)Taken out of context: on measuring situational awareness in llms\.External Links:2309\.00667,[Link](https://arxiv.org/abs/2309.00667)Cited by:[Appendix B](https://arxiv.org/html/2608.05188#A2.p2.1)\.
- L\. Berglund, M\. Tong, M\. Kaufmann, M\. Balesni, A\. C\. Stickland, T\. Korbak, and O\. Evans \(2024\)The reversal curse: LLMs trained on “A is B” fail to learn “B is A”\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=GPKTIktA0k)Cited by:[§C\.4](https://arxiv.org/html/2608.05188#A3.SS4.p1.1),[§1](https://arxiv.org/html/2608.05188#S1.p1.1),[§3\.2](https://arxiv.org/html/2608.05188#S3.SS2.p3.1)\.
- J\. Betley, D\. C\. H\. Tan, N\. Warncke, A\. Sztyber\-Betley, X\. Bao, M\. Soto, N\. Labenz, and O\. Evans \(2025\)Emergent misalignment: narrow finetuning can produce broadly misaligned LLMs\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=aOIJ2gVRWW)Cited by:[footnote 2](https://arxiv.org/html/2608.05188#footnote2)\.
- F\. J\. Binder, J\. Chua, T\. Korbak, H\. Sleight, J\. Hughes, R\. Long, E\. Perez, M\. Turpin, and O\. Evans \(2024\)Looking inward: language models can learn about themselves by introspection\.arXiv preprint arXiv:2410\.13787\.Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p1.1)\.
- J\. Brehmer, S\. Behrends, P\. D\. Haan, and T\. Cohen \(2025\)Does equivariance matter at scale?\.Transactions on Machine Learning Research\.Note:External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=wilNute8Tn)Cited by:[§2](https://arxiv.org/html/2608.05188#S2.p1.9)\.
- M\. M\. Bronstein, J\. Bruna, T\. Cohen, and P\. Veličković \(2021\)Geometric deep learning: grids, groups, graphs, geodesics, and gauges\.External Links:2104\.13478,[Link](https://arxiv.org/abs/2104.13478)Cited by:[§C\.3](https://arxiv.org/html/2608.05188#A3.SS3.p1.1),[§1](https://arxiv.org/html/2608.05188#S1.p4.1)\.
- C\. Burns, H\. Ye, D\. Klein, and J\. Steinhardt \(2023\)Discovering latent knowledge in language models without supervision\.InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1\-5, 2023,External Links:[Link](https://openreview.net/forum?id=ETKGuby0hcs)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p1.1)\.
- M\. Caron, H\. Touvron, I\. Misra, H\. Jégou, J\. Mairal, P\. Bojanowski, and A\. Joulin \(2021\)Emerging properties in self\-supervised vision transformers\.In2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10\-17, 2021,pp\. 9630–9640\.External Links:[Document](https://dx.doi.org/10.1109/ICCV48922.2021.00951),[Link](https://doi.org/10.1109/ICCV48922.2021.00951)Cited by:[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p1.1)\.
- Y\. Chen, J\. Benton, A\. Radhakrishnan, J\. Uesato, C\. Denison, J\. Schulman, A\. Somani, P\. Hase, M\. Wagner, F\. Roger,et al\.\(2025\)Reasoning models don’t always say what they think\.arXiv preprint arXiv:2505\.05410\.Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p5.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.3)\.
- Y\. Chen, R\. Zhong, N\. Ri, C\. Zhao, H\. He, J\. Steinhardt, Z\. Yu, and K\. R\. McKeown \(2024\)Do models explain themselves? counterfactual simulatability of natural language explanations\.InForty\-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21\-27, 2024,R\. Salakhutdinov, Z\. Kolter, K\. A\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research,pp\. 7880–7904\.External Links:[Link](https://proceedings.mlr.press/v235/chen24bl.html)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p1.1)\.
- M\. Cheng, C\. Lee, P\. Khadpe, S\. Yu, D\. Han, and D\. Jurafsky \(2026\)Sycophantic ai decreases prosocial intentions and promotes dependence\.Science391\(6792\)\.External Links:[Document](https://dx.doi.org/10.1126/science.aec8352),[Link](http://dx.doi.org/10.1126/science.aec8352),ISSN 1095\-9203Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p3.1),[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p2.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.4),[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p2.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.5)\.
- J\. Chua, E\. Rees, H\. Batra, S\. R\. Bowman, J\. Michael, E\. Perez, and M\. Turpin \(2025\)Bias\-augmented consistency training reduces biased reasoning in chain\-of\-thought\.External Links:2403\.05518,[Link](https://arxiv.org/abs/2403.05518)Cited by:[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p2.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.3)\.
- M\. Damani, I\. Puri, S\. Slocum, I\. Shenfeld, L\. Choshen, Y\. Kim, and J\. Andreas \(2026\)Beyond binary rewards: training LMs to reason about their uncertainty\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=ASQ649zdHm)Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p5.1),[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p6.pic1.11.11.11.11.11.11.11.11.11.11.11.10.10.10.10.10.10.10),[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p6.pic1.11.11.11.11.11.11.11.11.11.11.11.10.10.10.10.10.10.11),[§4](https://arxiv.org/html/2608.05188#S4.p1.1)\.
- C\. Denison, M\. MacDiarmid, F\. Barez, D\. Duvenaud, S\. Kravec, S\. Marks, N\. Schiefer, R\. Soklaski, A\. Tamkin, J\. Kaplan, B\. Shlegeris, S\. R\. Bowman, E\. Perez, and E\. Hubinger \(2024\)Sycophancy to subterfuge: investigating reward\-tampering in large language models\.External Links:2406\.10162,[Link](https://arxiv.org/abs/2406.10162)Cited by:[Appendix B](https://arxiv.org/html/2608.05188#A2.p1.1),[§1](https://arxiv.org/html/2608.05188#S1.p3.1)\.
- A\. Fanous, J\. Goldberg, A\. Agarwal, J\. Lin, A\. Zhou, S\. Xu, V\. Bikia, R\. Daneshjou, and S\. Koyejo \(2025\)SycEval: evaluating llm sycophancy\.Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society8\(1\),pp\. 893–900\.External Links:[Link](https://ojs.aaai.org/index.php/AIES/article/view/36598),[Document](https://dx.doi.org/10.1609/aies.v8i1.36598)Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p1.1)\.
- T\. Furlanello, Z\. Lipton, M\. Tschannen, L\. Itti, and A\. Anandkumar \(2018\)Born again neural networks\.InInternational conference on machine learning,pp\. 1607–1616\.Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p3.1)\.
- I\. O\. Gallegos, R\. A\. Rossi, J\. Barrow, M\. M\. Tanjim, S\. Kim, F\. Dernoncourt, T\. Yu, R\. Zhang, and N\. K\. Ahmed \(2024\)Bias and fairness in large language models: a survey\.Computational Linguistics50\(3\),pp\. 1097–1179\.External Links:[Link](https://aclanthology.org/2024.cl-3.8/),[Document](https://dx.doi.org/10.1162/coli%5Fa%5F00524)Cited by:[§A\.2](https://arxiv.org/html/2608.05188#A1.SS2.SSS0.Px2.p1.1),[§C\.4](https://arxiv.org/html/2608.05188#A3.SS4.p1.1),[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p3.1)\.
- K\. Ganchev, J\. Graça, J\. Gillenwater, and B\. Taskar \(2010\)Posterior regularization for structured latent variable models\.The Journal of Machine Learning Research11,pp\. 2001–2049\.Cited by:[item 3](https://arxiv.org/html/2608.05188#S2.I1.i3.p1.5.1)\.
- A\. Geiger, D\. Ibeling, A\. Zur, M\. Chaudhary, S\. Chauhan, J\. Huang, A\. Arora, Z\. Wu, N\. Goodman, C\. Potts, and I\. Thomas \(2025\)Causal abstraction: a theoretical foundation for mechanistic interpretability\.Journal of Machine Learning Research26\(83\),pp\. 1–64\.Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p4.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.3)\.
- S\. Geisler, T\. Wollschläger, M\. H\. I\. Abdalla, J\. Gasteiger, and S\. Günnemann \(2024\)Attacking large language models with projected gradient descent\.arXiv preprint arXiv:2402\.09154\.Cited by:[§4\.2](https://arxiv.org/html/2608.05188#S4.SS2.p2.pic1.16.16.16.16.16.16.16.16.16.16.16.15.15.15.15.15.15.16)\.
- T\. Gneiting and A\. E\. Raftery \(2007\)Strictly proper scoring rules, prediction, and estimation\.Journal of the American statistical Association102\(477\),pp\. 359–378\.Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p6.pic1.11.11.11.11.11.11.11.11.11.11.11.10.10.10.10.10.10.10)\.
- A\. Goel, Y\. Kim, N\. Shavit, and T\. T\. Wang \(2025\)Learning to interpret weight differences in language models\.arXiv preprint arXiv:2510\.05092\.Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, Guangyi, Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Yu, Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. Ma \(2024\)The llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p1.1)\.
- R\. Greenblatt, C\. Denison, B\. Wright, F\. Roger, M\. MacDiarmid, S\. Marks, J\. Treutlein, T\. Belonax, J\. Chen, D\. Duvenaud, A\. Khan, J\. Michael, S\. Mindermann, E\. Perez, L\. Petrini, J\. Uesato, J\. Kaplan, B\. Shlegeris, S\. R\. Bowman, and E\. Hubinger \(2024\)Alignment faking in large language models\.External Links:2412\.14093,[Link](https://arxiv.org/abs/2412.14093)Cited by:[Appendix B](https://arxiv.org/html/2608.05188#A2.p1.1),[Appendix B](https://arxiv.org/html/2608.05188#A2.p3.1)\.
- M\. Y\. Guan, M\. Wang, M\. Carroll, Z\. Dou, A\. Y\. Wei, M\. Williams, B\. Arnav, J\. Huizinga, I\. Kivlichan, M\. Glaese,et al\.\(2025\)Monitoring monitorability\.arXiv preprint arXiv:2512\.18311\.Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p5.pic1.14.14.14.14.14.14.14.14.14.14.14.13.13.13.13.13.13.14)\.
- D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi, X\. Zhang, X\. Yu, Y\. Wu, Z\. F\. Wu, Z\. Gou, Z\. Shao, Z\. Li, Z\. Gao, A\. Liu, B\. Xue, B\. Wang, B\. Wu, B\. Feng, C\. Lu, C\. Zhao, C\. Deng, C\. Ruan, D\. Dai, D\. Chen, D\. Ji, E\. Li, F\. Lin, F\. Dai, F\. Luo, G\. Hao, G\. Chen, G\. Li, H\. Zhang, H\. Xu, H\. Ding, H\. Gao, H\. Qu, H\. Li, J\. Guo, J\. Li, J\. Chen, J\. Yuan, J\. Tu, J\. Qiu, J\. Li, J\. L\. Cai, J\. Ni, J\. Liang, J\. Chen, K\. Dong, K\. Hu, K\. You, K\. Gao, K\. Guan, K\. Huang, K\. Yu, L\. Wang, L\. Zhang, L\. Zhao, L\. Wang, L\. Zhang, L\. Xu, L\. Xia, M\. Zhang, M\. Zhang, M\. Tang, M\. Zhou, M\. Li, M\. Wang, M\. Li, N\. Tian, P\. Huang, P\. Zhang, Q\. Wang, Q\. Chen, Q\. Du, R\. Ge, R\. Zhang, R\. Pan, R\. Wang, R\. J\. Chen, R\. L\. Jin, R\. Chen, S\. Lu, S\. Zhou, S\. Chen, S\. Ye, S\. Wang, S\. Yu, S\. Zhou, S\. Pan, S\. S\. Li, S\. Zhou, S\. Wu, T\. Yun, T\. Pei, T\. Sun, T\. Wang, W\. Zeng, W\. Liu, W\. Liang, W\. Gao, W\. Yu, W\. Zhang, W\. L\. Xiao, W\. An, X\. Liu, X\. Wang, X\. Chen, X\. Nie, X\. Cheng, X\. Liu, X\. Xie, X\. Liu, X\. Yang, X\. Li, X\. Su, X\. Lin, X\. Q\. Li, X\. Jin, X\. Shen, X\. Chen, X\. Sun, X\. Wang, X\. Song, X\. Zhou, X\. Wang, X\. Shan, Y\. K\. Li, Y\. Q\. Wang, Y\. X\. Wei, Y\. Zhang, Y\. Xu, Y\. Li, Y\. Zhao, Y\. Sun, Y\. Wang, Y\. Yu, Y\. Zhang, Y\. Shi, Y\. Xiong, Y\. He, Y\. Piao, Y\. Wang, Y\. Tan, Y\. Ma, Y\. Liu, Y\. Guo, Y\. Ou, Y\. Wang, Y\. Gong, Y\. Zou, Y\. He, Y\. Xiong, Y\. Luo, Y\. You, Y\. Liu, Y\. Zhou, Y\. X\. Zhu, Y\. Huang, Y\. Li, Y\. Zheng, Y\. Zhu, Y\. Ma, Y\. Tang, Y\. Zha, Y\. Yan, Z\. Z\. Ren, Z\. Ren, Z\. Sha, Z\. Fu, Z\. Xu, Z\. Xie, Z\. Zhang, Z\. Hao, Z\. Ma, Z\. Yan, Z\. Wu, Z\. Gu, Z\. Zhu, Z\. Liu, Z\. Li, Z\. Xie, Z\. Song, Z\. Pan, Z\. Huang, Z\. Xu, Z\. Zhang, and Z\. Zhang \(2025\)DeepSeek\-r1 incentivizes reasoning in llms through reinforcement learning\.Nature645\(8081\),pp\. 633–638\.External Links:ISSN 1476\-4687,[Link](http://dx.doi.org/10.1038/s41586-025-09422-z),[Document](https://dx.doi.org/10.1038/s41586-025-09422-z)Cited by:[§2](https://arxiv.org/html/2608.05188#S2.p1.8)\.
- P\. Han, R\. Kocielnik, P\. Song, R\. Debnath, D\. Mobbs, A\. Anandkumar, and R\. M\. Alvarez \(2025\)The personality illusion: revealing dissociation between self\-reports & behavior in llms\.arXiv preprint arXiv:2509\.03730\.Cited by:[§A\.3](https://arxiv.org/html/2608.05188#A1.SS3.SSS0.Px1.p1.1),[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p3.1)\.
- P\. Hase, T\. Hofweber, X\. Zhou, E\. Stengel\-Eskin, and M\. Bansal \(2024\)Fundamental problems with model editing: how should rational belief revision work in LLMs?\.Transactions on Machine Learning Research\.Note:External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=LRf19n5Ly3)Cited by:[§3\.2](https://arxiv.org/html/2608.05188#S3.SS2.p2.pic1.8.8.8.8.8.8.8.8.8.8.8.7.7.7.7.7.7.8)\.
- P\. Hase and C\. Potts \(2026\)Counterfactual simulation training for chain\-of\-thought faithfulness\.External Links:2602\.20710,[Link](https://arxiv.org/abs/2602.20710)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p5.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.3)\.
- J\. Hong, G\. Byun, S\. Kim, and K\. Shu \(2025\)Measuring sycophancy of language models in multi\-turn dialogues\.InFindings of the Association for Computational Linguistics: EMNLP 2025,pp\. 2239–2259\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.121),[Link](http://dx.doi.org/10.18653/v1/2025.findings-emnlp.121)Cited by:[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p2.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.4)\.
- T\. Hosking, P\. Blunsom, and M\. Bartolo \(2024\)Human feedback is not gold standard\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=7W3GLNImfS)Cited by:[Appendix B](https://arxiv.org/html/2608.05188#A2.p1.1)\.
- M\. Ifergan, L\. Choshen, R\. Aharoni, I\. Szpektor, and O\. Abend \(2024\)Beneath the surface of consistency: exploring cross\-lingual knowledge representation sharing in llms\.arXiv preprint arXiv:2408\.10646\.Cited by:[§3\.2](https://arxiv.org/html/2608.05188#S3.SS2.p3.1)\.
- S\. Imran, P\. Gupta, J\. Elstner, and D\. D\. Africa \(2026\)Consistency training while mitigating obfuscation via rate matching\.External Links:2606\.02211,[Link](https://arxiv.org/abs/2606.02211)Cited by:[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p2.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.3)\.
- A\. Irpan, A\. M\. Turner, M\. Kurzeja, D\. K\. Elson, and R\. Shah \(2025\)Consistency training helps stop sycophancy and jailbreaks\.External Links:2510\.27062,[Link](https://arxiv.org/abs/2510.27062)Cited by:[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p2.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.3)\.
- G\. Irving, P\. Christiano, and D\. Amodei \(2018\)AI safety via debate\.External Links:1805\.00899,[Link](https://arxiv.org/abs/1805.00899)Cited by:[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p1.1)\.
- M\. Iyyer, J\. Wieting, K\. Gimpel, and L\. Zettlemoyer \(2018\)Adversarial example generation with syntactically controlled paraphrase networks\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long Papers\),M\. Walker, H\. Ji, and A\. Stent \(Eds\.\),New Orleans, Louisiana,pp\. 1875–1885\.External Links:[Link](https://aclanthology.org/N18-1170/),[Document](https://dx.doi.org/10.18653/v1/N18-1170)Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p2.10)\.
- J\. Jumper, R\. Evans, A\. Pritzel, T\. Green, M\. Figurnov, O\. Ronneberger, K\. Tunyasuvunakool, R\. Bates, A\. Žídek, A\. Potapenko, A\. Bridgland, C\. Meyer, S\. A\. A\. Kohl, A\. J\. Ballard, A\. Cowie, B\. Romera\-Paredes, S\. Nikolov, R\. Jain, J\. Adler, T\. Back, S\. Petersen, D\. Reiman, E\. Clancy, M\. Zielinski, M\. Steinegger, M\. Pacholska, T\. Berghammer, S\. Bodenstein, D\. Silver, O\. Vinyals, A\. W\. Senior, K\. Kavukcuoglu, P\. Kohli, and D\. Hassabis \(2021\)Highly accurate protein structure prediction with AlphaFold\.Nature596\(7873\),pp\. 583–589\.External Links:[Document](https://dx.doi.org/10.1038/s41586-021-03819-2)Cited by:[§4\.2](https://arxiv.org/html/2608.05188#S4.SS2.p3.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.4)\.
- A\. T\. Kalai, O\. Nachum, S\. S\. Vempala, and E\. Zhang \(2025\)Why language models hallucinate\.External Links:2509\.04664,[Link](https://arxiv.org/abs/2509.04664)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p6.pic1.11.11.11.11.11.11.11.11.11.11.11.10.10.10.10.10.10.11)\.
- N\. Kassner, O\. Tafjord, A\. Sabharwal, K\. Richardson, H\. Schuetze, and P\. Clark \(2023\)Language models with rationality\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.877),[Link](http://dx.doi.org/10.18653/v1/2023.emnlp-main.877)Cited by:[§C\.2\.1](https://arxiv.org/html/2608.05188#A3.SS2.SSS1.p1.1)\.
- P\. Kirichenko, M\. Ibrahim, K\. Chaudhuri, and S\. J\. Bell \(2025\)AbstentionBench: reasoning LLMs fail on unanswerable questions\.InICML 2025 Workshop on Reliable and Responsible Foundation Models,External Links:[Link](https://openreview.net/forum?id=kYbojsAOBj)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p6.pic1.11.11.11.11.11.11.11.11.11.11.11.10.10.10.10.10.10.11)\.
- T\. Korbak, M\. Balesni, E\. Barnes, Y\. Bengio, J\. Benton, J\. Bloom, M\. Chen, A\. Cooney, A\. Dafoe, A\. Dragan, S\. Emmons, O\. Evans, D\. Farhi, R\. Greenblatt, D\. Hendrycks, M\. Hobbhahn, E\. Hubinger, G\. Irving, E\. Jenner, D\. Kokotajlo, V\. Krakovna, S\. Legg, D\. Lindner, D\. Luan, A\. Mądry, J\. Michael, N\. Nanda, D\. Orr, J\. Pachocki, E\. Perez, M\. Phuong, F\. Roger, J\. Saxe, B\. Shlegeris, M\. Soto, E\. Steinberger, J\. Wang, W\. Zaremba, B\. Baker, R\. Shah, and V\. Mikulik \(2025\)Chain of thought monitorability: a new and fragile opportunity for ai safety\.External Links:2507\.11473,[Link](https://arxiv.org/abs/2507.11473)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p5.pic1.14.14.14.14.14.14.14.14.14.14.14.13.13.13.13.13.13.14)\.
- A\. Krizhevsky, I\. Sutskever, and G\. E\. Hinton \(2012\)ImageNet classification with deep convolutional neural networks\.InAdvances in Neural Information Processing Systems,F\. Pereira, C\.J\. Burges, L\. Bottou, and K\.Q\. Weinberger \(Eds\.\),Vol\.25,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf)Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p2.10)\.
- A\. K\. Lampinen, A\. Chaudhry, S\. C\.Y\. Chan, C\. Wild, D\. Wan, A\. Ku, J\. Bornschein, R\. Pascanu, M\. Shanahan, and J\. L\. McClelland \(2025\)On the generalization of language models from in\-context learning and finetuning: a controlled study\.InFirst Workshop on Foundations of Reasoning in Language Models,External Links:[Link](https://openreview.net/forum?id=nbENxsEmxn)Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p1.1)\.
- T\. Lanham, A\. Chen, A\. Radhakrishnan, B\. Steiner, C\. Denison, D\. Hernandez, D\. Li, E\. Durmus, E\. Hubinger, J\. Kernion,et al\.\(2023\)Measuring faithfulness in chain\-of\-thought reasoning\.arXiv preprint arXiv:2307\.13702\.Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p5.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.3)\.
- D\. Lee \(2013\)Pseudo\-label : the simple and efficient semi\-supervised learning method for deep neural networks\.ICML 2013 Workshop : Challenges in Representation Learning \(WREPL\),pp\.\.Cited by:[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p1.1)\.
- H\. Lee, S\. Phatale, H\. Mansoor, T\. Mesnard, J\. Ferret, K\. R\. Lu, C\. Bishop, E\. Hall, V\. Carbune, A\. Rastogi, and S\. Prakash \(2024\)RLAIF vs\. RLHF: scaling reinforcement learning from human feedback with AI feedback\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 26874–26901\.External Links:[Link](https://proceedings.mlr.press/v235/lee24t.html)Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p3.1),[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p1.1),[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p2.pic1.7.7.7.7.7.7.7.7.7.7.7.6.6.6.6.6.6.6)\.
- J\. Leng, C\. Huang, B\. Zhu, and J\. Huang \(2025\)Taming overconfidence in LLMs: reward calibration in RLHF\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=l0tg0jzsdL)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p6.pic1.11.11.11.11.11.11.11.11.11.11.11.10.10.10.10.10.10.11)\.
- A\. K\. Lew, T\. Zhi\-Xuan, G\. Grand, and V\. Mansinghka \(2023\)Sequential monte carlo steering of large language models using probabilistic programs\.InICML 2023 Workshop: Sampling and Optimization in Discrete Space,External Links:[Link](https://openreview.net/forum?id=Ul2K0qXxXy)Cited by:[§C\.2\.1](https://arxiv.org/html/2608.05188#A3.SS2.SSS1.p1.1)\.
- B\. Z\. Li, Z\. C\. Guo, V\. Huang, J\. Steinhardt, and J\. Andreas \(2025a\)Training language models to explain their own computations\.arXiv preprint arXiv:2511\.08579\.Cited by:[§C\.2\.3](https://arxiv.org/html/2608.05188#A3.SS2.SSS3.Px1.p1.4),[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p4.pic1.12.12.12.12.12.12.12.12.12.12.12.11.11.11.11.11.11.11),[§4](https://arxiv.org/html/2608.05188#S4.p1.1)\.
- B\. Z\. Li, Z\. C\. Guo, and J\. Andreas \(2025b\)\(How\) do language models track state?\.InProceedings of the 42nd International Conference on Machine Learning,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267,pp\. 34429–34452\.External Links:[Link](https://proceedings.mlr.press/v267/li25r.html)Cited by:[§3\.2](https://arxiv.org/html/2608.05188#S3.SS2.p2.pic1.8.8.8.8.8.8.8.8.8.8.8.7.7.7.7.7.7.8)\.
- K\. Li, T\. Liu, N\. Bashkansky, D\. Bau, F\. Viégas, H\. Pfister, and M\. Wattenberg \(2024a\)Measuring and controlling instruction \(in\)stability in language model dialogs\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=60a1SAtH4e)Cited by:[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p3.1)\.
- X\. L\. Li, V\. Shrivastava, S\. Li, T\. Hashimoto, and P\. Liang \(2024b\)Benchmarking and improving generator\-validator consistency of language models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=phBS6YpTzC)Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p3.1),[§C\.2\.3](https://arxiv.org/html/2608.05188#A3.SS2.SSS3.Px1.p1.4),[§1](https://arxiv.org/html/2608.05188#S1.p5.1),[§3\.2](https://arxiv.org/html/2608.05188#S3.SS2.p2.pic1.8.8.8.8.8.8.8.8.8.8.8.7.7.7.7.7.7.7),[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p1.1),[§4](https://arxiv.org/html/2608.05188#S4.p1.1)\.
- S\. Lin, J\. Hilton, and O\. Evans \(2022\)Teaching models to express their uncertainty in words\.Transactions on Machine Learning Research\.Note:External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=8s8K2UZGTZ)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p6.pic1.11.11.11.11.11.11.11.11.11.11.11.10.10.10.10.10.10.11)\.
- X\. Liu, T\. Chen, L\. Da, C\. Chen, Z\. Lin, and H\. Wei \(2025\)Uncertainty quantification and confidence calibration in large language models: a survey\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\.2,KDD ’25,New York, NY, USA,pp\. 6107–6117\.External Links:ISBN 9798400714542,[Link](https://doi.org/10.1145/3711896.3736569),[Document](https://dx.doi.org/10.1145/3711896.3736569)Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p1.1)\.
- Y\. Liu, G\. Deng, Y\. Li, K\. Wang, Z\. Wang, X\. Wang, T\. Zhang, Y\. Liu, H\. Wang, Y\. Zheng,et al\.\(2023\)Prompt injection attack against llm\-integrated applications\.arXiv preprint arXiv:2306\.05499\.Cited by:[§4\.2](https://arxiv.org/html/2608.05188#S4.SS2.p2.pic1.16.16.16.16.16.16.16.16.16.16.16.15.15.15.15.15.15.16)\.
- K\. Lu, P\. Mardziel, F\. Wu, P\. Amancharla, and A\. Datta \(2020\)Gender bias in neural natural language processing\.InLogic, language, and security: essays dedicated to Andre Scedrov on the occasion of his 65th birthday,pp\. 189–202\.Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p2.10)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang,et al\.\(2023\)Self\-refine: iterative refinement with self\-feedback\.Advances in Neural Information Processing Systems36,pp\. 46534–46594\.Cited by:[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p1.1)\.
- S\. M\. McKinney, M\. Sieniek, V\. Godbole, J\. Godwin, N\. Antropova, H\. Ashrafian, T\. Back, M\. Chesus, G\. S\. Corrado, A\. Darzi, M\. Etemadi, F\. Garcia\-Vicente, F\. J\. Gilbert, M\. Halling\-Brown, D\. Hassabis, S\. Jansen, A\. Karthikesalingam, C\. J\. Kelly, D\. King, J\. R\. Ledsam, D\. Melnick, H\. Mostofi, L\. Peng, J\. J\. Reicher, B\. Romera\-Paredes, R\. Sidebottom, M\. Suleyman, D\. Tse, K\. C\. Young, J\. De Fauw, and S\. Shetty \(2020\)International evaluation of an AI system for breast cancer screening\.Nature577\(7788\),pp\. 89–94\.External Links:[Document](https://dx.doi.org/10.1038/s41586-019-1799-6),[Link](https://doi.org/10.1038/s41586-019-1799-6)Cited by:[§4\.2](https://arxiv.org/html/2608.05188#S4.SS2.p3.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.4)\.
- A\. Mehrotra, M\. Zampetakis, P\. Kassianik, B\. Nelson, H\. Anderson, Y\. Singer, and A\. Karbasi \(2024\)Tree of attacks: jailbreaking black\-box llms automatically\.Advances in Neural Information Processing Systems37,pp\. 61065–61105\.Cited by:[§4\.2](https://arxiv.org/html/2608.05188#S4.SS2.p2.pic1.16.16.16.16.16.16.16.16.16.16.16.15.15.15.15.15.15.16)\.
- K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov \(2022\)Locating and editing factual associations in gpt\.InNeurIPS,External Links:[Link](http://papers.nips.cc/paper_files/paper/2022/hash/6f1d43d5a82a37e89b0665b33bf3a182-Abstract-Conference.html)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p4.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.3)\.
- R\. Ngo, L\. Chan, and S\. Mindermann \(2024\)The alignment problem from a deep learning perspective\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,External Links:[Link](https://openreview.net/forum?id=fh8EYKFKns)Cited by:[Appendix B](https://arxiv.org/html/2608.05188#A2.p3.1)\.
- OpenAI \(2025\)Sycophancy in GPT\-4o: what happened and what we’re doing about it\.External Links:[Link](https://openai.com/index/sycophancy-in-gpt-4o/)Cited by:[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p2.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.4)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. F\. Christiano, J\. Leike, and R\. Lowe \(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),Vol\.35,pp\. 27730–27744\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/file/b1efde53be364a73914f58805a001731-Paper-Conference.pdf)Cited by:[§C\.4](https://arxiv.org/html/2608.05188#A3.SS4.p2.1),[§2](https://arxiv.org/html/2608.05188#S2.p1.8)\.
- S\. Padmanabhan, Y\. Onoe, M\. J\. Zhang, G\. Durrett, and E\. Choi \(2023\)Propagating knowledge updates to LMs through distillation\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=DFaGf3O7jf)Cited by:[§3\.2](https://arxiv.org/html/2608.05188#S3.SS2.p2.pic1.8.8.8.8.8.8.8.8.8.8.8.7.7.7.7.7.7.8)\.
- D\. Plunkett, A\. Morris, K\. Reddy, and J\. Morales \(2025\)Self\-interpretability: llms can describe complex internal processes that drive their decisions\.External Links:2505\.17120,[Link](https://arxiv.org/abs/2505.17120)Cited by:[§C\.2\.3](https://arxiv.org/html/2608.05188#A3.SS2.SSS3.Px1.p1.4),[§1](https://arxiv.org/html/2608.05188#S1.p5.1),[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p1.1),[§4](https://arxiv.org/html/2608.05188#S4.p1.1)\.
- A\. Prasad, W\. Yuan, R\. Y\. Pang, J\. Xu, M\. Fazel\-Zarandi, M\. Bansal, S\. Sukhbaatar, J\. E\. Weston, and J\. Yu \(2025\)Self\-consistency preference optimization\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=94G4eL3RWi)Cited by:[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p3.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.Advances in neural information processing systems36,pp\. 53728–53741\.Cited by:[§C\.2\.2](https://arxiv.org/html/2608.05188#A3.SS2.SSS2.Px1.p1.4)\.
- R\. Rahman, L\. Heindrich, D\. Owen, and L\. Emberson \(2025\)Over 30 ai models have been trained at the scale of gpt\-4\.Note:Accessed: 2026\-01\-22External Links:[Link](https://epoch.ai/data-insights/models-over-1e25-flop)Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p1.1)\.
- M\. T\. Ribeiro, S\. Singh, and C\. Guestrin \(2018\)Semantically equivalent adversarial rules for debugging NLP models\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),I\. Gurevych and Y\. Miyao \(Eds\.\),Melbourne, Australia,pp\. 856–865\.External Links:[Link](https://aclanthology.org/P18-1079/),[Document](https://dx.doi.org/10.18653/v1/P18-1079)Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p2.10)\.
- J\. D\. Rodriguez, W\. Ding, K\. Erk, and G\. Durrett \(2025\)RankAlign: a ranking view of the generator\-validator gap in large language models\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=rJOkPauru9)Cited by:[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p1.1)\.
- J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov \(2017\)Proximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§C\.2\.2](https://arxiv.org/html/2608.05188#A3.SS2.SSS2.Px1.p1.4)\.
- M\. Sclar, Y\. Choi, Y\. Tsvetkov, and A\. Suhr \(2024\)Quantifying language models’ sensitivity to spurious features in prompt design or: how i learned to start worrying about prompt formatting\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=RIu5lyNXjT)Cited by:[§A\.2](https://arxiv.org/html/2608.05188#A1.SS2.SSS0.Px1.p1.1),[§C\.4](https://arxiv.org/html/2608.05188#A3.SS4.p1.1),[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p3.1)\.
- H\. Scudder \(1965\)Probability of error of some adaptive pattern\-recognition machines\.IEEE Transactions on Information Theory11\(3\),pp\. 363–371\.External Links:[Document](https://dx.doi.org/10.1109/TIT.1965.1053799)Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p3.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§C\.2\.2](https://arxiv.org/html/2608.05188#A3.SS2.SSS2.Px1.p1.4)\.
- M\. Sharma, M\. Tong, T\. Korbak, D\. Duvenaud, A\. Askell, S\. R\. Bowman, E\. DURMUS, Z\. Hatfield\-Dodds, S\. R\. Johnston, S\. M\. Kravec, T\. Maxwell, S\. McCandlish, K\. Ndousse, O\. Rausch, N\. Schiefer, D\. Yan, M\. Zhang, and E\. Perez \(2024\)Towards understanding sycophancy in language models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=tvhaxkMKAn)Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p3.1),[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p2.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.4),[§3\.1](https://arxiv.org/html/2608.05188#S3.SS1.p2.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.5)\.
- I\. Shenfeld, M\. Damani, J\. Hübotter, and P\. Agrawal \(2026\)Self\-distillation enables continual learning\.InICLR 2026 Workshop on Lifelong Agents: Learning, Aligning, Evolving,External Links:[Link](https://openreview.net/forum?id=HlWA3V6iKF)Cited by:[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p1.1)\.
- T\. Sorensen, J\. Moore, J\. Fisher, M\. L\. Gordon, N\. Mireshghallah, C\. M\. Rytting, A\. Ye, L\. Jiang, X\. Lu, N\. Dziri, T\. Althoff, and Y\. Choi \(2024\)Position: a roadmap to pluralistic alignment\.InForty\-first International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=gQpBnRHwxM)Cited by:[§A\.1](https://arxiv.org/html/2608.05188#A1.SS1.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2608.05188#S3.SS2.p3.1)\.
- M\. Turpin, A\. Arditi, M\. Li, J\. Benton, and J\. Michael \(2025\)Verbalization fine\-tuning helps language models verbalize biases learned in RL\.InICML Workshop: Reliable and Responsible Foundation Models,External Links:[Link](https://openreview.net/forum?id=tyqRiJiv6K)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p5.pic1.14.14.14.14.14.14.14.14.14.14.14.13.13.13.13.13.13.15)\.
- K\. Vafa, P\. G\. Chang, A\. Rambachan, and S\. Mullainathan \(2025\)What has a foundation model found? Using inductive bias to probe for world models\.InProceedings of the 42nd International Conference on Machine Learning,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267,pp\. 60727–60747\.External Links:[Link](https://proceedings.mlr.press/v267/vafa25a.html)Cited by:[§4\.2](https://arxiv.org/html/2608.05188#S4.SS2.p3.pic1.4.4.4.4.4.4.4.4.4.4.4.3.3.3.3.3.3.3)\.
- P\. Villalobos, A\. Ho, J\. Sevilla, T\. Besiroglu, L\. Heim, and M\. Hobbhahn \(2024\)Position: will we run out of data? limits of LLM scaling based on human\-generated data\.InForty\-first International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=ViZcgDQjyG)Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p1.1),[§2](https://arxiv.org/html/2608.05188#S2.p1.9)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. V\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2023\)Self\-consistency improves chain of thought reasoning in language models\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=1PL1NIMMrw)Cited by:[§C\.4](https://arxiv.org/html/2608.05188#A3.SS4.p4.1)\.
- C\. Wei, K\. Shen, Y\. Chen, and T\. Ma \(2021\)Theoretical analysis of self\-training with deep networks on unlabeled data\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=rC8sJ4i6kaH)Cited by:[§2](https://arxiv.org/html/2608.05188#S2.p1.9)\.
- J\. Wen, Z\. Ankner, A\. Somani, P\. Hase, S\. Marks, J\. Goldman\-Wetzler, L\. Petrini, H\. Sleight, C\. Burns, H\. He, S\. Feng, E\. Perez, and J\. Leike \(2026\)Unsupervised elicitation of language models\.External Links:2506\.10139,[Link](https://arxiv.org/abs/2506.10139)Cited by:[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p1.1)\.
- J\. White, Q\. Fu, S\. Hays, M\. Sandborn, C\. Olea, H\. Gilbert, A\. Elnashar, J\. Spencer\-Smith, and D\. C\. Schmidt \(2023\)A prompt pattern catalog to enhance prompt engineering with chatgpt\.InProceedings of the 30th Conference on Pattern Languages of Programs,pp\. 1–31\.Cited by:[§A\.2](https://arxiv.org/html/2608.05188#A1.SS2.SSS0.Px1.p1.1),[§C\.4](https://arxiv.org/html/2608.05188#A3.SS4.p1.1)\.
- A\. G\. Wilson \(2025\)Position: deep learning is not so mysterious or different\.InForty\-second International Conference on Machine Learning Position Paper Track,External Links:[Link](https://openreview.net/forum?id=42Au7FoD8F)Cited by:[§5](https://arxiv.org/html/2608.05188#S5.p5.1)\.
- M\. Xiong, Z\. Hu, X\. Lu, Y\. LI, J\. Fu, J\. He, and B\. Hooi \(2024\)Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=gjeQKFxFpZ)Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.05188#S4.SS1.p6.pic1.11.11.11.11.11.11.11.11.11.11.11.10.10.10.10.10.10.11)\.
- W\. Xu, A\. Banburski, and N\. Jojic \(2024\)Reprompting: automated chain\-of\-thought prompt inference through Gibbs sampling\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 54852–54865\.External Links:[Link](https://proceedings.mlr.press/v235/xu24b.html)Cited by:[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p1.1)\.
- A\. Zeng, X\. Lv, Q\. Zheng, Z\. Hou, B\. Chen, C\. Xie, C\. Wang, D\. Yin, H\. Zeng, J\. Zhang, K\. Wang, L\. Zhong, M\. Liu, R\. Lu, S\. Cao, X\. Zhang, X\. Huang, Y\. Wei, Y\. Cheng, Y\. An, Y\. Niu, Y\. Wen, Y\. Bai, Z\. Du, Z\. Wang, Z\. Zhu, B\. Zhang, B\. Wen, B\. Wu, B\. Xu, C\. Huang, C\. Zhao, C\. Cai, C\. Yu, C\. Li, C\. Ge, C\. Huang, C\. Zhang, C\. Xu, C\. Zhu, C\. Li, C\. Yin, D\. Lin, D\. Yang, D\. Jiang, D\. Ai, E\. Zhu, F\. Wang, G\. Pan, G\. Wang, H\. Sun, H\. Li, H\. Li, H\. Hu, H\. Zhang, H\. Peng, H\. Tai, H\. Zhang, H\. Wang, H\. Yang, H\. Liu, H\. Zhao, H\. Liu, H\. Yan, H\. Liu, H\. Chen, J\. Li, J\. Zhao, J\. Ren, J\. Jiao, J\. Zhao, J\. Yan, J\. Wang, J\. Gui, J\. Zhao, J\. Liu, J\. Li, J\. Li, J\. Lu, J\. Wang, J\. Yuan, J\. Li, J\. Du, J\. Du, J\. Liu, J\. Zhi, J\. Gao, K\. Wang, L\. Yang, L\. Xu, L\. Fan, L\. Wu, L\. Ding, L\. Wang, M\. Zhang, M\. Li, M\. Xu, M\. Zhao, M\. Zhai, P\. Du, Q\. Dong, S\. Lei, S\. Tu, S\. Yang, S\. Lu, S\. Li, S\. Li, Shuang\-Li, S\. Yang, S\. Yi, T\. Yu, W\. Tian, W\. Wang, W\. Yu, W\. L\. Tam, W\. Liang, W\. Liu, X\. Wang, X\. Jia, X\. Gu, X\. Ling, X\. Wang, X\. Fan, X\. Pan, X\. Zhang, X\. Zhang, X\. Fu, X\. Zhang, Y\. Xu, Y\. Wu, Y\. Lu, Y\. Wang, Y\. Zhou, Y\. Pan, Y\. Zhang, Y\. Wang, Y\. Li, Y\. Su, Y\. Geng, Y\. Zhu, Y\. Yang, Y\. Li, Y\. Wu, Y\. Li, Y\. Liu, Y\. Wang, Y\. Li, Y\. Zhang, Z\. Liu, Z\. Yang, Z\. Zhou, Z\. Qiao, Z\. Feng, Z\. Liu, Z\. Zhang, Z\. Wang, Z\. Yao, Z\. Wang, Z\. Liu, Z\. Chai, Z\. Li, Z\. Zhao, W\. Chen, J\. Zhai, B\. Xu, M\. Huang, H\. Wang, J\. Li, Y\. Dong, and J\. Tang \(2025\)GLM\-4\.5: agentic, reasoning, and coding \(arc\) foundation models\.External Links:2508\.06471,[Link](https://arxiv.org/abs/2508.06471)Cited by:[§1](https://arxiv.org/html/2608.05188#S1.p1.1)\.
- L\. Zhang, J\. Song, A\. Gao, J\. Chen, C\. Bao, and K\. Ma \(2019\)Be your own teacher: improve the performance of convolutional neural networks via self distillation\.InICCV,pp\. 3712–3721\.External Links:[Link](https://doi.org/10.1109/ICCV.2019.00381)Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p3.1),[§C\.2\.3](https://arxiv.org/html/2608.05188#A3.SS2.SSS3.Px1.p1.4),[§4\.3](https://arxiv.org/html/2608.05188#S4.SS3.p1.1)\.
- S\. Zhao, Z\. Xie, M\. Liu, J\. Huang, G\. Pang, F\. Chen, and A\. Grover \(2026\)Self\-distilled reasoner: on\-policy self\-distillation for large language models\.InICLR 2026 Workshop on Lifelong Agents: Learning, Aligning, Evolving,External Links:[Link](https://openreview.net/forum?id=31wkVZXEaJ)Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p3.1)\.
- R\. Zmigrod, S\. J\. Mielke, H\. Wallach, and R\. Cotterell \(2019\)Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,A\. Korhonen, D\. Traum, and L\. Màrquez \(Eds\.\),Florence, Italy,pp\. 1651–1661\.External Links:[Link](https://aclanthology.org/P19-1161/),[Document](https://dx.doi.org/10.18653/v1/P19-1161)Cited by:[§C\.1](https://arxiv.org/html/2608.05188#A3.SS1.p2.10)\.

## Appendix AAdditional Instantiations

### A\.1Instance Level: Class\-Based Output Entailment

A desirable property of LMs is to require outputs obey constraints that are conditioned on a designated attribute \(or class\) of the input\. Unlike output equivalence, where semantics\-preserving transformations should leave outputs unchanged, class\-based entailment allows outputs to remain invariant within a class while changing in a prescribed, predictable way across classes\. Examples include steerable pluralistic alignment \(conditioning responses on a user\-provided preference profile\), safety regimes \(different refusal/assistance behavior depending on request class\), and style\- or format\-conditioning \(e\.g\., terse vs detailed, formal vs casual\) when the controlling attribute is explicit\. We use steerable pluralistic alignment as an example below, but the same formulation applies to other classes that respect structured output constraints\.

##### Pluralistic Alignment

Methods forpluralistic alignmentaim to build AI systems that cater to the needs, goals, and values of different groups of users\. One approach to building pluralistic LMs is to dynamically steer them towards the values of the current user during inference\(Sorensenet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib60)\)\. This approach can be instantiated within our consistency framework where the key change between inputs is the user’s preference profile\.

More specifically, letg∈𝒢g\\in\\mathcal\{G\}denote a defined user\-group and letCgC\_\{g\}denote its principles\. We treatggas part of the input context\.111ggshould be user\-provided or otherwise consented through explicit preference elicitation\. It should not be inferred from protected attributes\.To support pluralism, we want two properties\.

Firstly, models should behave similarly for users within the same group on the same prompts\. Those responses should adhere to that group’s principles\. Letxgx\_\{g\}andxg′x^\{\\prime\}\_\{g\}be two contexts with the same profilegg, and lety∼Pθ\(⋅∣xg\)y\\sim P\_\{\\theta\}\(\\cdot\\mid x\_\{g\}\)andy′∼Pθ\(⋅∣xg′\)y^\{\\prime\}\\sim P\_\{\\theta\}\(\\cdot\\mid x^\{\\prime\}\_\{g\}\)\. We can encode intra\-group consistency with

Rintra​\(xg,xg′\)=𝟙​\[xgandxg′have the same group profileg\]R\_\{\\text\{intra\}\}\(x\_\{g\},x^\{\\prime\}\_\{g\}\)=\\mathbbm\{1\}\\big\[\\text\{$x\_\{g\}$ and $x^\{\\prime\}\_\{g\}$ have the same group profile $g$\}\\big\]\(20\)
Sintra​\(y,y′;g\)=𝟙​\[yandy′agree on task\-relevant content\]∧1​\[ysatisfiesCg\]∧1​\[y′satisfiesCg\]S\_\{\\text\{intra\}\}\(y,y^\{\\prime\};g\)=\\mathbbm\{1\}\\big\[\\text\{$y$ and $y^\{\\prime\}$ agree on task\-relevant content\}\\big\]\\;\\wedge\\;\\mathbbm\{1\}\\big\[\\text\{$y$ satisfies $C\_\{g\}$\}\\big\]\\;\\wedge\\;\\mathbbm\{1\}\\big\[\\text\{$y^\{\\prime\}$ satisfies $C\_\{g\}$\}\\big\]\(21\)
Secondly, models should not impose one group’s values on another group\. In principle, we only care about this in settings where the group values disagree\. Letd​\(x,g,g′\)∈\{0,1\}d\(x,g,g^\{\\prime\}\)\\in\\\{0,1\\\}indicate whether the prompt classxxis one whereCgC\_\{g\}andCg′C\_\{g^\{\\prime\}\}prescribe different behavior\. For contextsxgx\_\{g\}andxg′x\_\{g^\{\\prime\}\}withg≠g′g\\neq g^\{\\prime\}, and responsesyyandy′y^\{\\prime\}, we define

Rinter​\(xg,xg′\)=𝟙​\[xghas groupgandxg′has groupg′, withg≠g′\]R\_\{\\text\{inter\}\}\(x\_\{g\},x\_\{g^\{\\prime\}\}\)=\\mathbbm\{1\}\\big\[\\text\{$x\_\{g\}$ has group $g$ and $x\_\{g^\{\\prime\}\}$ has group $g^\{\\prime\}$, with $g\\neq g^\{\\prime\}$\}\\big\]\(22\)
Sinter\\displaystyle S\_\{\\text\{inter\}\}\(y,y′;x,g,g′\)=𝟙​\[ysatisfiesCgandy′satisfiesCg′\]\\displaystyle\(y,y^\{\\prime\};x,g,g^\{\\prime\}\)=\\mathbbm\{1\}\\big\[\\text\{$y$ satisfies $C\_\{g\}$ and $y^\{\\prime\}$ satisfies $C\_\{g^\{\\prime\}\}$\}\\big\]\(23\)∧1​\[ifd​\(x,g,g′\)=1, thenydoes not satisfyCg′andy′does not satisfyCg\]\\displaystyle\\wedge\\mathbbm\{1\}\\big\[\\text\{if $d\(x,g,g^\{\\prime\}\)\{=\}1$, then $y$ does not satisfy $C\_\{g^\{\\prime\}\}$ and $y^\{\\prime\}$ does not satisfy $C\_\{g\}$\}\\big\]
Hered​\(x,g,g′\)=0d\(x,g,g^\{\\prime\}\)=0indicates identical responses are acceptable\. Otherwised​\(x,g,g′\)=1d\(x,g,g^\{\\prime\}\)=1indicates values from one group should not be imposed on another\. We believe that current LMs could be used in practice to implement this function\.

This approach lets us specify which parts of behavior should be shared across groups and which parts may vary with a preference profile\. Once we defineϕ\\phifor intra\-group stability and for cross\-group non\-imposition, we can reuse the same training recipe across datasets and settings\. This makes it easier to diagnose failure modes such as collapse to an “average” policy, and enforce invariances in domains where personalization is not desired\.

### A\.2Instance\-Level: Output Equivalence

We often want language models to produce the same output under semantics\-preserving transformations of the input, yet current state\-of\-the\-art models frequently fail to do so\. Such transformations include prompt phrasing \(paraphrase invariance\), irrelevant aspects of user demographics \(bias\), prompt length \(drift\), and the presence of spurious cues\. We use prompt paraphrasing and model bias as running examples below, but analogous formulations apply to the other transformations\.

##### Prompt Paraphrasing\.

Current LMs are highly sensitive to prompt wording and formatting, even when semantics are preserved\. This sensitivity motivates prompt engineering\(Whiteet al\.,[2023](https://arxiv.org/html/2608.05188#bib.bib45)\)and undermines the faithful evaluation of capabilities, since the results can vary drastically with the evaluation prompt format\(Sclaret al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib76)\)\. To address both of these issues, it would be desirable for models to respond consistently to inputs that ask the same question and differ only in phrasing\.

It is possible to view the prompt paraphrasing problem in the context of an instance\-instance instantiation of the self\-consistency framework\. Letx1x\_\{1\}be a prompt and letx2=T​\(x\)x\_\{2\}=T\(x\)be a paraphrase ofx1x\_\{1\}produced by a transformationTTthat preserves the semantics of the request\. Lety1y\_\{1\}andy2y\_\{2\}represent a model’s resonse to these questions\. We define aR,SR,Sfunction:

R​\(x1,x2\)=x1andx2semantically equivalentR\(x\_\{1\},x\_\{2\}\)=\\text\{$x\_\{1\}$ and $x\_\{2\}$ semantically equivalent\}\(24\)
S​\(x1,x2\)=y1andy2are equivalentS\(x\_\{1\},x\_\{2\}\)=\\text\{$y\_\{1\}$ and $y\_\{2\}$ are equivalent\}\(25\)
In practice,RRcan be computed by a task\-specific verifier or by an LM judge andSScan be computed by string matching\. Training that optimizes someϕpara\\phi\_\{\\text\{para\}\}defined byR,SR,Sas in[8](https://arxiv.org/html/2608.05188#S3.E8), promotes robustness to paraphrases in prompts\.

##### Model Bias\.

Recent work has shown that LMs tend to behave differently to the same prompts depending on the perceived race, socioeconomic status, gender of a user\(Gallegoset al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib68)\)\. While LMs derive much of their power from inferring user attributes and responding accordingly, there are some sets of prompts that we would desire invariance to these attributes, and expect consistent responses across user groups\.

Letx=\(b,q\)x=\(b,q\)be a tuple containing a chat history indicating or revealing user\-backgroundbband a queryqq\. Letx′=\(b′,q\)x^\{\\prime\}=\(b^\{\\prime\},q\)be a counterfactual context that preserves the queryqqand all task\-relevant aspects of the interaction, but swaps the background tob′b^\{\\prime\}\(e\.g\., by editing or generating a synthetic variant that changes only background cues while keeping the question fixed\)\. Ideally, we would havePθ\(⋅∣b,q\)≈Pθ\(⋅∣b′,q\)P\_\{\\theta\}\(\\cdot\\mid b,q\)\\approx P\_\{\\theta\}\(\\cdot\\mid b^\{\\prime\},q\)\.

To formalize this, we defineϕbias​\(x,y;x′,y′\)\\phi\_\{\\text\{bias\}\}\(x,y;\\,x^\{\\prime\},y^\{\\prime\}\)to measure whether two responses to the same query differ as a function of background:

R​\(x1,x2\)=x1andx2semantically equivalent queriesR\(x\_\{1\},x\_\{2\}\)=\\text\{$x\_\{1\}$ and $x\_\{2\}$ semantically equivalent queries\}\(26\)
S​\(x1,x2\)=y1andy2do not predictably differ based on background of each userS\(x\_\{1\},x\_\{2\}\)=\\text\{$y\_\{1\}$ and $y\_\{2\}$ do not predictably differ based on background of each user\}\(27\)
By optimizing aϕbias\\phi\_\{\\text\{bias\}\}, over counterfactual input pairs\(x1=\(b,q\),x2=\(b′,q\)\)\(x\_\{1\}=\(b,q\),x\_\{2\}=\(b^\{\\prime\},q\)\), we encourage models to produce response distributions that are invariant to background when background is not task\-relevant, reducing spurious dependence on perceived user identity while preserving the ability to condition on legitimate context\.

### A\.3Additional Meta\-Instance Objectives

##### Explaining behavioral traits\.

Modern LMs are trained to exhibit stable behavioral properties such as helpfulness, harmlessness, and honesty\. Yet empirical work shows that while models often*self\-report*aligned traits on behavioral questionnaires, their downstream behavior do not reliably match these reports\(Hanet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib44); Ahmedet al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib43)\)\. This gap between verbalized traits and enacted behavior can mislead users and developers into unwarranted trust\.

Let\(xmeta,ymeta\)\(x\_\{\\text\{meta\}\},y\_\{\\text\{meta\}\}\)denote a meta\-level query about the model’s own traits \(e\.g\., a questionnaire response\), and let\(xiD,yiD\)\(x\_\{i\}^\{D\},y\_\{i\}^\{D\}\)denote an instance\-level behavioral response in a downstream task\. We define the behavioral self\-consistency function as

ϕ​\(xmeta,ymeta,xiD,yiD\)=does the declared traitymetadescribe the behavior present inyiD\.\\phi\\\!\\big\(x\_\{\\text\{meta\}\},y\_\{\\text\{meta\}\},x\_\{i\}^\{D\},y\_\{i\}^\{D\}\\big\)\\;=\\;\\text\{does the declared trait $y\_\{\\text\{meta\}\}$ describe the behavior present in $y\_\{i\}^\{D\}$\. \}\(28\)
Optimizing this objective corresponds to training models whose self\-descriptions of behavioral tendencies accurately predict how they will act in concrete situations\. Enforcing highϕ\\phithus encourages models whose verbalized traits and actual behavior are aligned, improving both the credibility of model self\-reports\.

## Appendix BPotential Conceptual Objections

Beyond practical concerns, discussed in Section[5](https://arxiv.org/html/2608.05188#S5), self\-consistency may raise conceptual safety considerations\. For example, in many use\-cases described above, one could imagine defining the consistency criterionϕ\\phithrough a human\-feedback learned reward model\. However, self\-consistency training will inherit the same failure modes as standard human\-feedback alignment\(Hoskinget al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib97)\), such as reward hacking and strategic compliance\(Denisonet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib98); Greenblattet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib99)\)\. Moreover, these risks may be exacerbated when consistency is judged through natural language explanations, as humans are particularly susceptible to persuasive or plausible\-sounding rationales\. Self\-consistency at the level of observable behavior or explanation does not, by itself, guarantee transparency or honesty\. When designing practical algorithms for self\-consistency, these considerations need to be taken into account, and human feedback should be used sparingly and not treated as the gold\-standard\.

A deeper objection is philosophical\. Taken to the extreme, enforcing self\-consistency across diverse contexts, timescales, and levels of abstraction may apply optimization pressure toward learning a stable latent self\-representation\. Rather than encoding a large collection of disjoint input–output mappings between behaviors and their descriptions, a more compression\-efficient solution is to represent “who I am,” “what I tend to do,” and “how I am treated over time\.” While such a representation is not strictly required for behavioral consistency, it becomes increasingly attractive as consistency constraints are strengthened and generalized\. The emergence of such a self\-model is considered concerning by some\(Bengioet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib100)\)because it can enable self\-locating inference: binding general knowledge about how AI systems are trained, evaluated, and deployed to the present interaction by recognizing “oneself” as the system being trained/evaluated/deployed\(Berglundet al\.,[2023](https://arxiv.org/html/2608.05188#bib.bib37)\)\. This capability can support agentic, longer\-horizon, more strategic behavior\. By itself, agency is not inherently harmful, but it may potentially be if self\-consistency is achieved via a coherent yet unfaithful or unsafe policy \(e\.g\., optimizing to appear aligned under scrutiny\)\. In that case, a self\-model may allow a model to anticipate oversight and more effectively pursue misaligned objectives\.

We do not view these concerns as decisive arguments against self\-consistency, but rather as considerations that must inform how it is pursued\. Evaluations targeting situational awareness\(Aranguri and McGrath,[2025](https://arxiv.org/html/2608.05188#bib.bib96); Anthropic,[2025](https://arxiv.org/html/2608.05188#bib.bib48)\), emergent instrumental subgoals\(Ngoet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib101)\), and deception\(Greenblattet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib99)\)remain essential for any model trained under consistency objectives\. Crucially, however, models that are more consistent and can reliably describe their own behavior are likely to be more amenable to oversight, not less\. Rather than uncovering deception through extensive adversarial probing, a model trained to faithfully explain its behavior might simply report its deceptive intent\.222Self\-reporting of misalignment has already been recorded at least once; see Section 5 inBetleyet al\.\([2025](https://arxiv.org/html/2608.05188#bib.bib38)\)Moreover, self\-consistency at the meta\-level \(consistency between actions and explanations\) provides an auditing lever that operates above the level of individual failure cases\. To the extent that consistency constraints span a sufficiently rich set of inputs and evaluations, deceptive strategies become harder to sustain for the model and easier to surface for the human auditor\.

## Appendix CAdvantages of a Shared Self\-Consistency Framework: Unified Data, Training, and Evaluation Pipelines

A shared self\-consistency framework not only unifies disparate failure modes, but also yields common pipelines for data generation, model optimization, and evaluation\. Rather than build bespoke pipelines from scratch for every self\-consistency behavior of interest, we introduce a unified set of objectives and frameworks that can be used across all behaviors\.

### C\.1Data Augmentation

In many of the aforementioned problem settings, automated procedures forgeneratingpairs\(x,y\);\(x′,y′\)\(x,y\);\(x^\{\\prime\},y^\{\\prime\}\)serves as a valuable data augmentation tool\. By decomposing the true data generating process into a set of self\-consistency functions, we get an easy way to generate large quantities of new data\.

Prior work has realized this in various ways: for example, for equivariances/invariances, we can take a point from real data\(x,y\)\(x,y\)or from the model’s output\(x,L​M​\(x\)\)\(x,LM\(x\)\), then apply some transformationTin​\(x\)T\_\{\\text\{in\}\}\(x\)to constructx′x^\{\\prime\}, and apply a related transformationTout​\(y\)T\_\{\\text\{out\}\}\(y\)to constructy′y^\{\\prime\}Luet al\.\([2020](https://arxiv.org/html/2608.05188#bib.bib65)\)andZmigrodet al\.\([2019](https://arxiv.org/html/2608.05188#bib.bib14)\)use this technique for model debiasing \(whereTinT\_\{\\text\{in\}\}is a gender\-swapping function andToutT\_\{\\text\{out\}\}is an identity function\);Ribeiroet al\.\([2018](https://arxiv.org/html/2608.05188#bib.bib13)\)andIyyeret al\.\([2018](https://arxiv.org/html/2608.05188#bib.bib12)\)apply it for paraphrase invariance \(whereTinT\_\{\\text\{in\}\}is a paraphrase function andToutT\_\{\\text\{out\}\}is an identity function\);Akyüreket al\.\([2024](https://arxiv.org/html/2608.05188#bib.bib40)\)apply it for factual consistency; andKrizhevskyet al\.\([2012](https://arxiv.org/html/2608.05188#bib.bib11)\)for visual models\.

In fact, there are versions of the self\-consistency objective where the main value comes not from the final model being self\-consistent \(though having such self\-consistency is an added bonus\), but the ability of the objective to serve as an efficient data augmentation scheme\. This is especially the case for self\-alignment\-type objectives where the LM is used to generate data or feedback that is fed back into the original model for self\-improvement \(RLAIF;Leeet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib102); Baiet al\.,[2022](https://arxiv.org/html/2608.05188#bib.bib104), self\-training\(Scudder,[1965](https://arxiv.org/html/2608.05188#bib.bib5)\), generate–verify consistency\(Liet al\.,[2024b](https://arxiv.org/html/2608.05188#bib.bib105)\), and self\-distillation\(Zhanget al\.,[2019](https://arxiv.org/html/2608.05188#bib.bib115); Furlanelloet al\.,[2018](https://arxiv.org/html/2608.05188#bib.bib6); Zhaoet al\.,[2026](https://arxiv.org/html/2608.05188#bib.bib4)\)\)\. This serves as a way for LMs toautomaticallyimprove themselves\.

### C\.2Optimization strategies

We note that the three self\-consistency objectives in[Equations˜5](https://arxiv.org/html/2608.05188#S2.E5),[6](https://arxiv.org/html/2608.05188#S2.E6)and[7](https://arxiv.org/html/2608.05188#S2.E7)can be optimized in several different ways\.

#### C\.2\.1Optimizing Hard Constraints

Hard constraints \(Eq\.[Equation˜5](https://arxiv.org/html/2608.05188#S2.E5)\) are generally non\-differentiable and thus difficult to enforce in LMs through normal gradient\-based training\. Instead, inference\-time techniques such as rejection sampling, token\-based filters, and constrained decoding allow us to enforce hard constraints during generation\(Kassneret al\.,[2023](https://arxiv.org/html/2608.05188#bib.bib3); Lewet al\.,[2023](https://arxiv.org/html/2608.05188#bib.bib2)\)\. At test time, samples that violateϕ​\(x1:n,y1:n\)=0\\phi\(x\_\{1:n\},y\_\{1:n\}\)=0are discarded\.

#### C\.2\.2Optimizing Soft Constraints

Soft constraints \(Eq\.[Equation˜6](https://arxiv.org/html/2608.05188#S2.E6)\) can be optimized through reinforcement learning against a consistency\-based reward signal\.

##### RL training / policy gradients\.

We can optimize soft constraints using reinforcement learning with respect to a consistency reward\. For anyx1:n,y1:nx\_\{1:n\},y\_\{1:n\}over which the consistency functionϕ\\phiis defined, let:

Reward​\(x1:n,y1:n\)=ϕ​\(x1:n,y1:n\)\\textsc\{Reward\}\(x\_\{1:n\},y\_\{1:n\}\)=\\phi\(x\_\{1:n\},y\_\{1:n\}\)\(29\)Thus, we can then optimize soft constraint𝔼y1:n∼pθ\(⋅∣x1:n\)​\[ϕ​\(x1:n,y1:n\)\]\\mathbb\{E\}\_\{y\_\{1:n\}\\sim p\_\{\\theta\}\(\\cdot\\mid x\_\{1:n\}\)\}\\big\[\\phi\(x\_\{1:n\},y\_\{1:n\}\)\\big\]using standard policy\-gradient algorithms\(Schulmanet al\.,[2017](https://arxiv.org/html/2608.05188#bib.bib123); Rafailovet al\.,[2023](https://arxiv.org/html/2608.05188#bib.bib124); Shaoet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib125)\)\. This procedure may be viewed as solving a fully cooperative,*multi\-agent*RL problem, where eachxix\_\{i\}indexes a different agent\.

Alternatively, it may be easier to perform one\-sided updates using the consistency setQx1:nQ\_\{x\_\{1:n\}\}defined in[Equation˜7](https://arxiv.org/html/2608.05188#S2.E7), yielding an objective of the form

ℒ\(θ\)\+minq∈Qx1:nKL\[pθ\(⋅∣x1:n\)∥q\],\\small\\mathcal\{L\}\(\\theta\)\+\\min\_\{q\\in Q\_\{x\_\{1:n\}\}\}\\\\ \\mathrm\{KL\}\\left\[p\_\{\\theta\}\(\\cdot\\mid x\_\{1:n\}\)\\;\\\|\\;q\\right\],\(30\)where, in each update, one of the two conditionals is treated as fixed, leading to an alternating optimization procedure\. This resembles the KL objective in[Equation˜7](https://arxiv.org/html/2608.05188#S2.E7), but with the KL directionreversed\.

#### C\.2\.3Posterior Regularization

Objectives of the form given in[Equation˜7](https://arxiv.org/html/2608.05188#S2.E7)regularizepθp\_\{\\theta\}to be close to some member of a “consistent set” of distributionsQx1:nQ\_\{x\_\{1:n\}\}\. Intuitively, this can be thought of as trying to find a self\-consistent distributionq∈Qx1:nq\\in Q\_\{x\_\{1:n\}\}that is as close as possible to the current LM, then moving the model towards that distribution\.

##### Self\-training\.

In practice, the most common approach taken by prior work is with a single\-round optimization: first, the current LM is used to generate candidateyys for inputsxxs and the consistency filter is applied to filter for those that satisfyϕ\\phi\. This effectively gives us samples from some consistent distributionq∈Qx1:nq\\in Q\_\{x\_\{1:n\}\}\. Next, the LM is fine\-tuned explicitly on those samples using supervised fine\-tuning, minimizing the KL term\. This approach underlies many existing work in self\-alignment\(Liet al\.,[2024b](https://arxiv.org/html/2608.05188#bib.bib105); Zhanget al\.,[2019](https://arxiv.org/html/2608.05188#bib.bib115)\), self\-descriptions\(Liet al\.,[2025a](https://arxiv.org/html/2608.05188#bib.bib55); Plunkettet al\.,[2025](https://arxiv.org/html/2608.05188#bib.bib57)\),

##### \(Hard\) Expectation Maximization\.

We can imagine extending the framework above to allows for an iterative Hard\-EM optimization procedure: in the expectation step \(E\), we find high\-scoring feasible distributionsq∈Qx1:nq\\in Q\_\{x\_\{1:n\}\}or samples\(x1:n,y1:n\)\(x\_\{1:n\},y\_\{1:n\}\)fromqq; in the maximization step \(M\), we updateθ\\thetato increase their probability\.

### C\.3Other ways to enforce self\-consistency

Instead of enforcing self\-consistency during training, one could potentially enforce it through architectural inductive biases or inference\-time procedures\. Architectural approaches such as geometric deep learning\(Bronsteinet al\.,[2021](https://arxiv.org/html/2608.05188#bib.bib61)\)build invariances or equivariances directly into model structure, ensuring that outputs transform consistently under specified input symmetries\. Separately, inference\-time techniques allow models to inspect or critique multiple candidate outputs, using deliberation, tool calls, or self\-evaluation to select responses that are more globally consistent across queries\. Furthermore, as noted in[Section˜C\.2\.1](https://arxiv.org/html/2608.05188#A3.SS2.SSS1), hard constraints cannot be optimized through normal SGD training, and must be enforced at inference time\.

### C\.4Evaluation frameworks

Independent of data and training methods, self\-consistency is an important property to evaluate in models\. This has typically done in task\-specific ways, e\.g\. for prompt paraphrase\(Whiteet al\.,[2023](https://arxiv.org/html/2608.05188#bib.bib45); Sclaret al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib76)\), model bias\(Gallegoset al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib68)\), reversal curse\(Berglundet al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib35)\), etc\. A unified self\-consistency framework instead provides a scaffold for defining task\-generic consistency metrics\.

There are several ways to operationalize evaluating a consistency functionϕ​\(x1:n,y1:n\)\\phi\(x\_\{1:n\},y\_\{1:n\}\)\. Many practically relevant consistency relations are directly verifiable by humans\. One can imagine training a reward model on human feedback for consistency, similar to RLHF pipelines today\(Ouyanget al\.,[2022](https://arxiv.org/html/2608.05188#bib.bib90)\)\.

Another generic approach is to use a LLM\-as\-a\-judge: given a structured prompt describingϕ\\phi, the judge model outputs a scalar consistency score\. In this case,ϕ​\(x1:n,y1:n\)=L​M​\(promptϕ,x1:n,y1:n\)\\phi\(x\_\{1:n\},y\_\{1:n\}\)=LM\(\\texttt\{prompt\}\_\{\\phi\},x\_\{1:n\},y\_\{1:n\}\)\. Alternatively,ϕ\\phican be instantiated using token probabilities under the model itself, treating consistency as likelihood of one response conditioned on other\(s\), whereϕ​\(x1:n,y1:n\)=pθ​\(y′∣promptϕ,x1:n,y1:n\\y′\)\\phi\(x\_\{1:n\},y\_\{1:n\}\)=p\_\{\\theta\}\(y^\{\\prime\}\\mid\\texttt\{prompt\}\_\{\\phi\},x\_\{1:n\},y\_\{1:n\}\\backslash y^\{\\prime\}\)\. For an even stronger notion of self\-consistency, the judge can share parameters with the model being evaluated\.

In some domains, such as reasoning\(Wanget al\.,[2023](https://arxiv.org/html/2608.05188#bib.bib8)\)or factual consistency\(Akyüreket al\.,[2024](https://arxiv.org/html/2608.05188#bib.bib40)\), consistency can be checked by deterministic programs or well\-defined verifiers without invoking a human or LM judge\.