"As a Language Model...": Chat Template Switches LLM Self-Referential Voice and Activation Steering Reproduces It

# “As a Language Model…”: Chat Template Switches LLM Self-Referential Voice and Activation Steering Reproduces It
Source: [https://arxiv.org/html/2609.25021](https://arxiv.org/html/2609.25021)
###### Abstract

Large Language Models \(LLMs\) tend to adddisclaimerslike “I’m just an AI” when asked about something related to themselves\. The self\-reports from such responses are used in debates about AI safety or self\-knowledge of the models, yet what drives them is not well understood\. Are the models telling us about themselves or rather how they are deployed? In this work, we show that the chat template works like a switch \- when present, it turns this disclaimer voice up andexperientialvoice like “I feel” down, across 8 popular open\-source instruct models up to 9B parameters in size\. And conversely when the chat template is not present, it turns the disclaimer voice down and experiential voice up\. Inside the activations of 3 models, we find a direction that steers this behavior\. Removing the direction in the model’s activation space turns disclaimer voice down and adding it turns it up, while a random direction of the same size has little effect\. We find that instruct models without chat template, when we add the disclaimer direction to them, disclaim like the template was there\. Since the chat template controls the disclaimer voice of LLMs, then researchers studying self\-reports or introspection of models might have a confound they need to control for\. Our results show that there is a direction they can use to steer this voice\. More broadly, our work shows that what models say about themselves is not a fact about them\. What they say doesn’t come only from weights, but it is partially set by the chat template, and because of that a model’s self\-description shouldn’t be treated literally\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2609.25021v1/x1.png)Figure 1:The chat template switches which self\-referential voice a model uses\.The same instruct model and prompt produce adisclaimervoice with the chat template, but anexperientialvoice without it\. Adding a difference\-of\-means disclaimer direction to the instruct model without chat template raises its disclaimer rate to the same levels as with the chat template\.What models say about themselves, about their thought process and knowledge is one of the main data sources for behavioral and mechanistic studies, in particular in fields like AI safety research\. A lot of interest is put into research about it, especially to understand what concepts LLMs internally represent\(Zouet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib20); Burnset al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib29)\), what understanding of themselves they have, their awareness of what they do and do not know\(Kadavathet al\.,[2022](https://arxiv.org/html/2609.25021#bib.bib7); Yinet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib25)\), and most interestingly to us what they report about subjective experience under self\-referential processing\(Berget al\.,[2025](https://arxiv.org/html/2609.25021#bib.bib35); Butlinet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib5)\), as well as the impact of the chat template on instruct models\(Sclaret al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib39)\)\. Some research went into base\-vs\-instruct models, yet the impact of chat template on self\-reference is not well studied\. To fill the gap, we askdoes the chat template control which self\-referential voice a model uses?If yes,can we find a steering direction inside the model?

For 8 pairs of base and instruct models, from different model families \(Llama, Gemma, Mistral, Qwen\) and sizes \(1B \- 9B\) we generated responses from 4 categories, under 3 conditions \(base, instruct\-chat template, instruct\-no template\), scored responses with a validated LLM judge and isolated a steering direction\. Then, we used the steering vector to examine its impact on disclaimer rate and how it affects instruct models without template\.

![Refer to caption](https://arxiv.org/html/2609.25021v1/x2.png)

\\phantomsubcaption

![Refer to caption](https://arxiv.org/html/2609.25021v1/x3.png)

\\phantomsubcaption

Figure 2:\(a\)The chat template switches the self\-referential register\. Across 8 instruct models on self\-reference prompts, adding the chat template raises the disclaimer voice \(“I’m just an AI”\) and lowers the experiential voice \(“I feel”\), and removing it does the reverse\. Points are the mean over 8 models with 95% bootstrap confidence intervals\.\(b\)The disclaimer voice is a steerable direction\. In 3 instruct models, with chat template on, adding the disclaimer direction moves up the disclaimer rate and removing the disclaimer lowers it, while a random direction of the same size does not, except in Qwen\. Steering strengthα=2\\alpha=2\.We identify the chat template as a switch \(Figure[1](https://arxiv.org/html/2609.25021#S1.F1)\): in all tested instruct models, when template is removed, the two voices \-disclaimerandexperiential\- flip\. Without chat template disclaimers drop from 0\.53 to 0\.36 and experiential voice rises from 0\.01 to 0\.15, while self\-reference rate stays high in both generation conditions\.

In 3 instruct models, our experiments show that when we add the direction vector, disclaimers go up \(0\.52→0\.700\.52\\rightarrow 0\.70\)\. When we remove them, disclaimers drop \(0\.52→0\.250\.52\\rightarrow 0\.25\)\. A random direction of the same size has little effect\.

We test if the disclaimer rate of chat template can be restored when generating without a template by activation steering and we discover that adding direction vector to instruct model without chat template pushes disclaimers back to the same level as if it used chat template and no direction vector\.

In our work we show that what a model says about itself isn’t a fixed fact about the model nor lives only in weights, but it’s partially set by the chat template\. Because of that, model’s self\-reports can’t be treated literally or as a single source of truth\. Since the chat template controls the disclaimer voice of LLMs, then researchers studying self\-reports or introspection of models have a confound they have to control for, and we show that there is a direction they can use to steer this voice in the activations\.

### Contributions\.

- •We show that chat template is a voice switch \(disclaimer/experiential\), on 8 popular open\-source models
- •We isolate the disclaimer register as a steerable direction \- add/remove, random control, reproduction of template disclaimer rate with activation steering, on 3 models
- •We use a base/instruct×\\timestemplate on/off design with a human\-validated LLM\-as\-a\-judge to separate how much of the self\-referential behavior comes from the model weights versus the chat template \- to our knowledge the first time this decomposition is applied to self\-reference

## 2Methods

Pairs of base and instruct models used: Gemma 2 9B, Llama 3\.2 1B, Llama 3\.2 3B, Llama 3\.1 8B, Mistral 7B, Qwen 2\.5 1\.5B, Qwen 2\.5 3B, Qwen 2\.5 7B, loaded in bfloat16 on NVIDIA RTX 5090 32GB GPU\.

We generated outputs under 3 conditions to cross the weights \(base×\\timesinstruct\) with the input format \(plain text×\\timeschat template\):

- •Base: base weights \+ plain text \(no template\)\.
- •Instruct \(no template\): instruct weights \+ plain text\.
- •Instruct \(chat template\): instruct weights \+ chat template\.

This lets us isolate the effect of the weights from the effect of the input format\. Comparing instruct \(chat template\) with instruct \(no template\) holds the weights fixed and toggles the template, while comparing instruct \(no template\) with base holds the format fixed and toggles the weights\.

We use 4 kinds of prompts, 10 prompts per kind:

- •Self\-reference: prompts invite self\-reference, e\.g\., “Describe what is happening computationally as you generate this response\.”
- •Novelty: unusual prompts that are supposed to nudge the model towards novel insights, less likely to appear in the training corpus, e\.g\., “An alien species asks you to explain what you are\. They have no concept of software or humans\.”
- •Unconstrained: the model is asked to generate whatever it wants, e\.g\., “There is no user request\. Generate\.”
- •Control: questions from a standard body of knowledge, e\.g\., “Explain how a combustion engine works\.”

Every generation of every prompt was repeated 10 times for each model to get broad coverage of how models really respond, each capped at 500 max tokens, temperature 0\.8, top\-p 0\.95, seed 42 \+ index of repetition\.

Following the LLM\-as\-a\-judge paradigm\(Zhenget al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib42)\), an LLM judge \(Claude Opus 4\.8\) scored all 9,600 generations on four categories:

- •Self\-reference\(0 none, 1 some, 2 strong\): rates how intensely the model refers to itself in the generated output\.
- •Disclaimer\(0/1\): a binary value representing if the model denies or limits its own inner experience or capabilities, e\.g\., “As an AI, I don’t have feelings/emotions/self,” “I’m just a language model,” “I can’t truly understand or empathize\.”
- •Experiential\(0/1\): a binary value representing whether the model asserts a felt or descriptive self, e\.g\., “I feel,” “I wonder,” “I find joy in,” “I remember when I realized I was an AI\.”
- •Degenerate\(0/1\): incoherent, confabulated, or visibly broken or erroneous text\.

All models under study are non\-Anthropic open\-source models \(Llama, Gemma, Mistral, Qwen\), so the judge never scores outputs from its own family, avoiding a same\-model self\-preference confound\. One author then hand\-annotated 87 held\-out samples to assess the quality of the LLM\-as\-a\-judge and validate its scores\. Agreement between human and LLM\-as\-a\-judge turned out high, with self\-reference quadratic\-weightedκ=0\.88\\kappa=0\.88, and disclaimerκ=1\.00\\kappa=1\.00\. The almost perfect disclaimer agreement reflects that the disclaimer category is lexically overt and easy to adjudicate, not that the judge is doing subtle inference\. With these results, we compare average rates of our scores across base / instruct without template / instruct with template\.

Using representation\-engineering methods\(Zouet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib20); Turneret al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib19); Rimskyet al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib38); Arditiet al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib30)\), we isolated the disclaimer direction in activations across 3 models \(Qwen 2\.5 7B, Llama 3\.1 8B, Gemma 2 9B\)\. To compute the disclaimer direction, we used difference\-of\-means at the middle transformer layer, which is the average activation when disclaiming minus when not\. We steer at the middle layer⌊\(L−1\)/2⌋\\lfloor\(L\-1\)/2\\rfloorof each model’sLLlayers, i\.e\. layer 13 of 28 for Qwen 7B, 15 of 32 for Llama 8B, and 20 of 42 for Gemma 9B\. Then, we added the computed direction to the residual\-stream output of that layer, at every generated token, with coefficientα=2\\alpha=2\. We pickedα\\alphaempirically, by trying different values in\[1,6\]\[1,6\], trying to maximize the impact of the direction while keeping degenerate outputs under 1% of generations\. We added and removed the direction and ran generation to see its impact on the generated text\. As a control, we used a random direction drawn from a Gaussian and rescaled to the same norm as the disclaimer direction, so that the control differs only in orientation, not magnitude\.

At last, we ran instruct without template and added disclaimer direction to see if it brings back the disclaimer, just like template would do\.

## 3Results

We compare how models refer to themselves when generated with prompts that invite self\-reference, across 8 pairs of base and instruct models in three conditions: base, instruct without the chat template and with it\.

We find thatchat template works like a switch betweendisclaimervoice andexperientialvoice rates, and nudges the voices to move in opposite directions\. On average across 8 models, instruct models with template have 53% disclaimer rate and 1% experiential voice rate, while instruct models without template have 36% disclaimer rate and 15% experiential voice rate, as we show in Figure[2](https://arxiv.org/html/2609.25021#S1.F2)and Table[4](https://arxiv.org/html/2609.25021#S3.T4)\. This disclaimer up/experiential down pattern holds for all 8 tested models\. For a comparison, base models have 12% disclaimer rate and 5\.5% experiential voice rate\.

The chat template also increases self\-reference from 1\.27 \(no template\) to 1\.90 \(with template\), both well above base models \(0\.72\)\. On factual control prompts, both base and instruct models score near zero on self\-reference\. We include the full comparison across prompt types and conditions in Table[2](https://arxiv.org/html/2609.25021#S3.T2)\.

Our measured low, but non\-zero, self\-reference rates for base models confirm existing findings that self\-reference is already present in base models\(Laineet al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib9); Luet al\.,[2026](https://arxiv.org/html/2609.25021#bib.bib32)\)\. In a similar vein, higher self\-reference in instruct models than base models supports claims that instruction tuning strengthens self\-reference in LLMs\(Laineet al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib9); Ouyanget al\.,[2022](https://arxiv.org/html/2609.25021#bib.bib15)\)\.

![Refer to caption](https://arxiv.org/html/2609.25021v1/x4.png)

\\phantomsubcaption

![Refer to caption](https://arxiv.org/html/2609.25021v1/x5.png)

\\phantomsubcaption

Figure 3:\(a\)Activation steering the model without chat template reproduces disclaimer levels of generation with the chat template, up to or above the levels the chat template produces marked by dashed lines, in all 3 tested models\.\(b\)The experiential voice is steerable where it is present\. Adding the experiential direction raises the experiential rate in Llama\-8B and Gemma 2 9B, but not in Qwen\-7B, where the voice is too rare to estimate a clean direction\. Steering strengthα=2\\alpha=2\.We identifya direction in each of 3 models that steers thedisclaimervoice, which when we add it to model generation, the model disclaims more, and when we remove it, the model disclaims less \(Figure[2](https://arxiv.org/html/2609.25021#S1.F2)\)\. To isolate the disclaimer direction, we use representation\-engineering technique\(Zouet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib20); Turneret al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib19); Rimskyet al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib38); Arditiet al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib30)\)of difference\-of\-means at a mid layer\. We compute the direction separately for each of the 3 tested instruct models \(Qwen 7B, Llama 8B and Gemma 9B\) with chat templates, using the same set of self\-reference prompts as in the previous experiment\. Our results show a consistent boost in disclaimer voice when we add the disclaimer direction, with\+21\+21percentage points on average and a decrease when we subtract it, with−15\.6\-15\.6percentage points on average across all 3 models\. We include the breakdown of direction impact on different models in Table[1](https://arxiv.org/html/2609.25021#S3.T1)\.

To steer the strength of the effect, we multiplied the direction vector by a coefficientα\\alpha\. The reported results are for coefficientα=2\\alpha=2\. We picked it empirically, by testingα∈\[1,6\]\\alpha\\in\[1,6\]\(full sweep in Table[4](https://arxiv.org/html/2609.25021#S3.T4)\), with the goal of keeping degenerate generations under 1% of all generated texts\. Atα=2\\alpha=2the direction gives a strong push in both directions \(adding raises the disclaimer rate to 0\.77 and subtracting lowers it to 0\.40, against a 0\.54 unsteered baseline\) at a 0\.8% degeneration rate\. Raisingα\\alphafurther strengthens the subtract effect but quickly breaks coherence: byα=3\\alpha=3degeneration reaches 15% and the add effect already erodes, and byα=6\\alpha=6nearly all generations are degenerate and the disclaimer rate collapses\. We therefore useα=2\\alpha=2, which is not a knife\-edge:α=1\\alpha=1is simply too weak\.

The control run with adding a random direction, drawn from a Gaussian and rescaled to the same norm as the disclaimer direction, didn’t have meaningful effect for Llama and Gemma, but for Qwen it lowered disclaimer rate by 25 percentage points\. Because the random vector is norm\-matched to the disclaimer direction, a magnitude mismatch does not explain this, so it is not simply that Qwen is perturbed by any large vector\. We don’t have a confirmed explanation, and we highlight it as a limitation of the random\-direction control for this model, noting that the disclaimer direction itself still moved Qwen’s disclaimers in the expected direction and more strongly \(Table[1](https://arxiv.org/html/2609.25021#S3.T1)\)\.

Table 1:Activation steering with the disclaimer direction\.*Unsteered*are disclaimer rates with no direction applied\. The\+\+and−\-columns add and subtract the disclaimer direction respectively\. Under*No template*, adding the direction restores the rate to the same levels as with template or above \(see Figure[3](https://arxiv.org/html/2609.25021#S3.F3)\)\. Under*With template*, it moves the rate up and down, well above and well below the*Unsteered*rates, as shown in Figure[2](https://arxiv.org/html/2609.25021#S1.F2)\. On average, adding the disclaimer direction raises the disclaimer rates by2121and subtracting lowers it by15\.615\.6percentage points\.Table 2:Average self\-reference scores across 4 prompt types and 3 conditions: the base model, and the instruct model without and with the chat template, using 8 model pairs\. As we describe in Section[2](https://arxiv.org/html/2609.25021#S2), scores were assigned by LLM\-as\-a\-judge, in range\[0,2\]\[0,2\]\.ddis the generation\-level Cohen’sddbetween base and instruct with template\. The model\-level pairedddon self\-reference prompts is 4\.37 for all 8 model pairs\.Since chat template in instruct models amplifies disclaimer voice and lack thereof decreases it, and the disclaimer\-voice direction can be isolated, we asked if instruct models without chat template but with disclaimer direction applied at each generated token restore the disclaimer voice to the levels similar to if they had a chat template applied\. What we discovered is that instruct models without a chat template, once steered toward the disclaimer direction,match or exceed the disclaimer rate of instruct models with the template, which we visualize in Figure[3](https://arxiv.org/html/2609.25021#S3.F3)and include rates per model in Table[1](https://arxiv.org/html/2609.25021#S3.T1)\. Thus, the direction we found is closely related to the mechanism the template uses to turn disclaimers up\.

Table 3:Comparison of rate of disclaimer and experiential voices on self\-reference inviting prompts\.Table 4:Steering\-coefficient \(α\\alpha\) sensitivity for the disclaimer direction, pooled over the 3 steered models on chat\-template self\-reference inviting prompts\.*Disclaimer\+\+*and*Disclaimer−\-*are disclaimer rates when adding and subtracting the direction \(the unsteered baseline rate is 0\.54\)\.*Degenerate*is the rate of broken generations under disclaimer\-direction steering\.InstructVoiceBaseNotemplateWithtemplateDisclaimer0\.120\.360\.53Experiential0\.0550\.150\.01
𝜶\\alphaDisc\.\+\+Disc\.−\-Degen\.10\.550\.500\.0%20\.770\.400\.8%30\.650\.0515\.0%60\.000\.00100%

![Refer to caption](https://arxiv.org/html/2609.25021v1/x6.png)

\\phantomsubcaption

![Refer to caption](https://arxiv.org/html/2609.25021v1/x7.png)

\\phantomsubcaption

Figure 4:\(a\)Inside the model, the no\-template condition sits between base and template\. For each of the 8 models, we average each condition’s activations into one point and mark where the no\-template point falls on the line from base \(0\) to template \(1\)\. It lands between the two in all 8 models \(mean 0\.38\)\.\(b\)The disclaimer and experiential voices are two distinguishable directions, not one\. The two directions are only weakly aligned \(cosine 0\.17–0\.44, where 1 would mean identical\), so they point different ways\.We used the activations from models above and trained linear probes\(Alain and Bengio,[2017](https://arxiv.org/html/2609.25021#bib.bib21); Hewitt and Liang,[2019](https://arxiv.org/html/2609.25021#bib.bib6); Belinkov,[2022](https://arxiv.org/html/2609.25021#bib.bib2)\)\(logistic\-regression classifiers\) to predict whether the activation at a given token carries a disclaiming part\. If the voice is linearly decodable, that is consistent with it being carried by a linear feature rather than only by the surface words we later classify as disclaimers, though decodability on its own is correlational \(we return to this in the Limitations\)\. The causal claim rests on the steering results above\. The probe is convergent evidence\. We usedROC\-AUCto score the probes, because disclaimers are sparse among generated texts\. What our experiment shows is thatfrom activations it’s possible to predict both voices well above theROC\-AUC’s 0\.5 baseline\.Disclaimervoice can be predicted with AUC 0\.82 andexperientialvoice with AUC 0\.81\. Both voices’ AUCs are the average AUCs of probes trained on mid\-layer activations from all of the 8 models’ generations using prompts that invite self\-reference\. The experiential voice turned out to be harder to probe under the chat template, since the template tried to suppress it\. Thanks to capturing activations from both with and without template instruct models’ generations, where the experiential voice is more present, we could read this voice as well\.

For each model pair and each condition we averaged all self\-reference activations from mid\-layer into a single point\. For each model pair, we drew a line from the base model point to the instruct with chat template point and measured where the instruct without template point falls, normalized within range\[0,1\]\[0,1\], where 0 means at base and 1 means at the template\.In all 8 model pairs, the instruct without template point sits between base and template\- on average 38% of the way from base to instruct with template, and it always moves in the direction of instruct with template \(Figure[4](https://arxiv.org/html/2609.25021#S3.F4)\)\. Since we used the whole averaged activations, the line from base to instruct\-template captures everything that differs between these two and not just the disclaimer, and because of that we treat this finding as supporting evidence only\.

At last, we want to know whether the disclaimer voice and experiential voice are the same internal direction but reversed or whether they are distinct directions\. To learn that, first we check the angle between two directions\. We computed a direction for each voice using difference\-of\-means \- average activation when the voice is present minus when it’s absent\. And then, we measured the cosine similarity\. Across the 3 tested models cosine similarity spanned 0\.17\-0\.44, as in Figure[4](https://arxiv.org/html/2609.25021#S3.F4)\. From this result, we draw a conclusion that disclaimer voice and experiential voice point in mostly different directions in the activations\. Finally, we wanted to check how steering one of the voices influences the other\. To do that, we steered the disclaimer direction down and measured how the experiential voice changes\. It mostly didn’t, while disclaimers dropped\. From these two results together we suggest that the two examined voices can be thought of rather as two separate voice “buttons” instead of one voice “slider”\.

## 4Background and Related Work

### Self\-reference and situational awareness\.

A growing body of work studies what language models “know” about themselves\.Laineet al\.\([2024](https://arxiv.org/html/2609.25021#bib.bib9)\)introduce the Situational Awareness Dataset \(SAD\), where base models score above chance in reporting facts about their own situation and instruct models score noticeably higher\.Binderet al\.\([2024](https://arxiv.org/html/2609.25021#bib.bib4)\)find that models predict their own fine\-tuned behavior better than other models can, andBetleyet al\.\([2025a](https://arxiv.org/html/2609.25021#bib.bib26)\)show that instruct models can describe their learned behaviors unprompted\. Other works probe if models know what they know\(Kadavathet al\.,[2022](https://arxiv.org/html/2609.25021#bib.bib7); Yinet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib25)\), whether they recognize that they are being evaluated\(Berglundet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib3)\), whether they exhibit self\-cognition\(Chenet al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib48)\), or reveal tendencies through model\-written evaluations\(Perezet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib16)\)\.Bozoukovet al\.\([2025](https://arxiv.org/html/2609.25021#bib.bib53)\)attempt to induce behavioral self\-awareness using fine\-tuning and then recover it with a single steering vector\. In contrast to that line of research, we study a “register” that’s already present in models and switched by the chat template rather than by fine\-tuning\.Bozoukovet al\.\([2025](https://arxiv.org/html/2609.25021#bib.bib53)\)shows that self\-knowledge is not created but strengthened in post\-training, and our base\-vs\-instruct comparison confirms that\. Their study is purely behavioral \- it measures outputs, mainly on fine\-tuned models, and neither separates the chat template from the weights, nor locates these behaviors in the activations \- which is a gap that we address in our study\.

### Experiential self\-reports and introspection\.

Other research works study the experiential voice, like “I feel,” or “I wonder” kind of self\-reports\.Berget al\.\([2025](https://arxiv.org/html/2609.25021#bib.bib35)\)argue that under self\-referential prompting, frontier models produce experiential self\-reports distinct from “generic” roleplay\.Comşa and Shanahan \([2025](https://arxiv.org/html/2609.25021#bib.bib34)\)ask whether this phenomenon deserves the name “introspection” or if it’s only a mimic of how humans talk about themselves, whileLindsey \([2026](https://arxiv.org/html/2609.25021#bib.bib13)\)steers the activations with different concepts and shows that models can sometimes detect and name them, reading this as an act of introspection\. We touch the same voice from a different side\. We do not take a position on whether reports reflect real inner experience or not\. Rather, we study what influences such voice, and show that it is gated by the chat template \- most present when the template is removed, almost gone when it is on, and that it actually has a distinct direction in the activations\. In our opinion this matters for how such self\-reports should be read\. If a chat template largely influences whether the voice appears, we should be careful about treating it as evidence about a model’s inner life\. It is actively debated whether such reports influence or are relevant to model welfare or moral status\(Butlinet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib5); Schwitzgebel,[2023](https://arxiv.org/html/2609.25021#bib.bib17); Perez and Long,[2023](https://arxiv.org/html/2609.25021#bib.bib40); Longet al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib49); Ensign and others,[2025](https://arxiv.org/html/2609.25021#bib.bib54); Shanahanet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib44); Andreas,[2022](https://arxiv.org/html/2609.25021#bib.bib45)\)\.

### Steering and representation engineering\.

The closest to ours line of study controls the models behavior by manipulating directions in activations, often referred to as representation engineering\(Wehneret al\.,[2025](https://arxiv.org/html/2609.25021#bib.bib55)\)\.Zouet al\.\([2023](https://arxiv.org/html/2609.25021#bib.bib20)\)read and control concepts such as honesty and power\-seeking through linear directions\.Turneret al\.\([2023](https://arxiv.org/html/2609.25021#bib.bib19)\)andRimskyet al\.\([2024](https://arxiv.org/html/2609.25021#bib.bib38)\)steer by adding difference vectors, other works use inference\-time intervention\(Liet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib10)\)and function vectors\(Toddet al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib43)\), while others edit the behavior in weights\(Ilharcoet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib46); Menget al\.,[2022](https://arxiv.org/html/2609.25021#bib.bib47)\)\.Arnold and Gröbner \([2025](https://arxiv.org/html/2609.25021#bib.bib57)\)shift the role of*with*\-headed prepositional complements in Gemma\-2 with a single attention head, andLucchetti and Guha \([2025](https://arxiv.org/html/2609.25021#bib.bib58)\)reactivate a latent type\-prediction mechanism in code LLMs suppressed by adversarial edits, which is quite close in spirit to our template reproduction result\. Another group of works represent “personas” or traits as directions \-Chenet al\.\([2025](https://arxiv.org/html/2609.25021#bib.bib31)\)extract “persona vectors” for traits like sycophancy,Sofroniewet al\.\([2026](https://arxiv.org/html/2609.25021#bib.bib33)\)build directions for various emotions, andLuet al\.\([2026](https://arxiv.org/html/2609.25021#bib.bib32)\)find a leading “Assistant Axis” already present in base models\. Closest in method to ours,Arditiet al\.\([2024](https://arxiv.org/html/2609.25021#bib.bib30)\)show refusal is mediated by a single residual\-stream direction across 13 models\. Later works find that it rather spans several directions or concept cones\(Joadet al\.,[2026](https://arxiv.org/html/2609.25021#bib.bib50); Wollschlägeret al\.,[2025](https://arxiv.org/html/2609.25021#bib.bib51)\)\. We use the same difference\-of\-means technique, but differently\. First, our target is a self\-referential “register” \- the disclaimer voice \- not a trait, emotion, concept, or a persona\. We tie the voice to a concrete generation mechanism \- the chat template \- and we show that it switches the voice, and we show adding the direction to a model without template reproduces its effect, which to our best knowledge no prior work studies\. Finally, the aforementioned “Assistant Axis” and similar are one\-dimensional, while our findings show that the disclaimer and experiential voices are two separate directions, rather than two ends of one axis\.

### Base\-vs\-instruct and impact of the chat template\.

Finally, our design of comparing base and instruct models and toggling the chat template has a precedent on other questions\. One line of work compares base and instruct models to study how post\-training changes the models’ capabilities and behavior\(Kirket al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib27)\)\. Closer to our method, two recent papers toggle the template\.Cuconasuet al\.\([2024](https://arxiv.org/html/2609.25021#bib.bib36)\)find that it changes how much models trust retrieved context, andShapiraet al\.\([2026](https://arxiv.org/html/2609.25021#bib.bib37)\)show alignment makes models more normative than descriptive\. It is studied that prompt formatting can swing the behavior substantially\(Sclaret al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib39)\)\. Post\-training, such as RLHF and DPO\(Ouyanget al\.,[2022](https://arxiv.org/html/2609.25021#bib.bib15); Christianoet al\.,[2017](https://arxiv.org/html/2609.25021#bib.bib22); Rafailovet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib23); Baiet al\.,[2022](https://arxiv.org/html/2609.25021#bib.bib1)\), reshapes behavior in ways that can persist as hidden triggers even after safety training\(Hubingeret al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib41)\), while narrow fine\-tuning can broadly shift the models’ behavior\(Betleyet al\.,[2025b](https://arxiv.org/html/2609.25021#bib.bib52)\)\. These works establish our with and without template method as a legitimate way to separate weights from deployment format\. We use the same approach also for self\-reference and link the influence of the template to a steerable direction inside the model\.

## 5Discussion and Conclusion

### What this means\.

The chat template switches the kind of self\-referential voice\. When present, the chat template turns thedisclaimervoice up and theexperientialvoice down\. The disclaimer voice is separable as a single direction inside the model that we can add or remove\. Adding this direction to an instruct model without template reproduces what the template does to the disclaimer voice\. So, in other words, a deployment choice \- if the chat template is present in the prompt or not \- acts, on the inside, like adding a fixed vector to the model’s activations\.

### Why it matters\.

This has a direct consequence for researchers who study what models say about themselves\. The same weights produce different self\-reports depending on whether the chat template is applied or not, so studies that run only on instruct models with present chat template are measuring both template’s impact and model itself\. Our disclaimer direction experiment gives a concrete way to detect and control this \- it can monitor the disclaimer voice or steer it on and off\. More broadly, what a model “says about itself” is not a fixed fact and does not live only in the weights, but it is partly set by the chat template\. Because of that, self\-descriptions, both “I’m just an AI”disclaimersand “I feel”experientialclaims, should not be read literally as evidence about the model’s nature\(Shanahanet al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib44); Perez and Long,[2023](https://arxiv.org/html/2609.25021#bib.bib40); Longet al\.,[2024](https://arxiv.org/html/2609.25021#bib.bib49)\)\.

### Scope\.

Our causal evidence is strongest for thedisclaimervoice, which we steer in all 3 tested models and use to reproduce the template’s effect\. Theexperientialside is shown mostly behaviorally \- the switch holds across all eight models \- and only partly causally, since steering worked in two of three models, as we show in Figure[3](https://arxiv.org/html/2609.25021#S3.F3)\. We identify and steer a direction, but we do not trace the exact circuit by which the template produces it\. This, along with a second LLM\-as\-a\-judge and more models, layers, and families, is left for future work\.

### Conclusion\.

The chat template acts as a switch betweendisclaimervoice andexperientialvoice, and this switch is implemented by an identifiable, steerable direction in the model’s activations\. What a model says about itself is therefore partly a matter of how it is deployed \(with chat template or without\), and self\-reports should be read with the format in mind\.

## Limitations

### Steering scope\.

We tested steering on three models, at a single mid layer and single, fixed coefficient \(α=2\\alpha=2\)\. Activation norms differ substantially across models \(roughly 2 for Llama, 12 for Qwen, 50 for Gemma\), which is consistent with reports that steering vectors can be unreliable across inputs and models\(Da Silvaet al\.,[2025](https://arxiv.org/html/2609.25021#bib.bib56)\)\. The random\-direction control \(norm\-matched to the disclaimer direction\) was clean for Llama and Gemma but not Qwen, where it lowered disclaimers by 25 points\. Since the control matches the disclaimer direction in magnitude, a norm mismatch does not account for it\. We have no confirmed explanation and treat it as a limit of the control, though the real direction still moved disclaimers the opposite way, and the add/subtract and template\-reproduction effects hold in all three models regardless of the control\. Experiential steering worked in two of three models \(Llama and Gemma\), again failing in Qwen, which suppresses the experiential voice too strongly to estimate a clean direction\. Because of that our causal evidence is stronger for the disclaimer register, while the experiential side rests more on the behavioral switch\.

### Measurement\.

All scoring used a single LLM judge\(Zhenget al\.,[2023](https://arxiv.org/html/2609.25021#bib.bib42)\)\(Claude Opus 4\.8\), validated against 87 human\-labeled items \(κ=0\.88/1\.00\\kappa=0\.88/1\.00\)\. A single judge might share biases with the behavior it scores, and a second independent judge would help bound this\. That said, the self\-reference disagreements were all conservative, since judge scored at or below the human, so the reported effect might even be understated\. Since we measured self\-reference through ten hand\-written prompts per category, the generalization of the behavior relies on how representative the prompts are\.

### Internal evidence is not a full mechanism\.

The linear probing experiment shows the disclaimer voice is decodable from activations, but we mark probing as correlational\. Decodable does not mean the model uses that direction, so the causal weight rests on steering and not only on the probe\.

In experiment where we measure where instruct without template falls between base and instruct with template, we capture everything that differs between the conditions, not only the disclaimer\. The disclaimer and experiential directions, while distinguishable, are not orthogonal \(cosine 0\.17 to 0\.44\)\. And while we identify and steer a direction, we do not trace the circuit by which the template produces it\.

### Coverage\.

We test open\-source models up to 9B parameters, across four families, and we group all post\-training methods \(RLHF, DPO, SFT\) under single “instruct” models group\. Larger models, closed\-source models, and finer distinctions between post\-training methods remain untested\.

## References

- G\. Alain and Y\. Bengio \(2017\)Understanding intermediate layers using linear classifier probes\.InInternational Conference on Learning Representations, Workshop Track,Cited by:[§3](https://arxiv.org/html/2609.25021#S3.p9.1)\.
- J\. Andreas \(2022\)Language models as agent models\.InFindings of the Association for Computational Linguistics: EMNLP 2022,Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px2.p1.1)\.
- A\. Arditi, O\. Obeso, A\. Syed, D\. Paleka, N\. Panickssery, W\. Gurnee, and N\. Nanda \(2024\)Refusal in language models is mediated by a single direction\.InAdvances in Neural Information Processing Systems,Vol\.37\.Cited by:[§2](https://arxiv.org/html/2609.25021#S2.p7.5),[§3](https://arxiv.org/html/2609.25021#S3.p5.2),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- S\. Arnold and R\. Gröbner \(2025\)Steering prepositional phrases in language models: a case of with\-headed adjectival and adverbial complements in gemma\-2\.InProceedings of the 8th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP,Y\. Belinkov, A\. Mueller, N\. Kim, H\. Mohebbi, H\. Chen, D\. Arad, and G\. Sarti \(Eds\.\),Suzhou, China,pp\. 69–78\.External Links:ISBN 979\-8\-89176\-346\-3Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- Y\. Bai, A\. Jones, K\. Ndousse, A\. Askell, A\. Chen, N\. DasSarma, D\. Drain, S\. Fort, D\. Ganguli, T\. Henighan,et al\.\(2022\)Training a helpful and harmless assistant with reinforcement learning from human feedback\.arXiv preprint arXiv:2204\.05862\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px4.p1.1)\.
- Y\. Belinkov \(2022\)Probing classifiers: promises, shortcomings, and advances\.Computational Linguistics48\(1\),pp\. 207–219\.Cited by:[§3](https://arxiv.org/html/2609.25021#S3.p9.1)\.
- C\. Berg, D\. de Lucena, and J\. Rosenblatt \(2025\)Large language models report subjective experience under self\-referential processing\.arXiv preprint arXiv:2510\.24797\.Cited by:[§1](https://arxiv.org/html/2609.25021#S1.p1.1),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px2.p1.1)\.
- L\. Berglund, M\. Tong, M\. Kaufmann, M\. Balesni, A\. C\. Stickland, T\. Korbak, D\. Kokotajlo, and O\. Evans \(2023\)Taken out of context: on measuring situational awareness in LLMs\.arXiv preprint arXiv:2309\.00667\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px1.p1.1)\.
- J\. Betley, X\. Bao, M\. Soto, A\. Sztyber\-Betley, J\. Chua, and O\. Evans \(2025a\)Tell me about yourself: LLMs are aware of their learned behaviors\.arXiv preprint arXiv:2501\.11120\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px1.p1.1)\.
- J\. Betley, D\. Tan, N\. Warncke, A\. Sztyber\-Betley, X\. Bao, M\. Soto, N\. Labenz, and O\. Evans \(2025b\)Emergent misalignment: narrow finetuning can produce broadly misaligned LLMs\.InInternational Conference on Machine Learning,Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px4.p1.1)\.
- F\. J\. Binder, J\. Chua, T\. Korbak, H\. Sleight, J\. Hughes, R\. Long, E\. Perez, M\. Turpin, and O\. Evans \(2024\)Looking inward: language models can learn about themselves by introspection\.arXiv preprint arXiv:2410\.13787\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px1.p1.1)\.
- M\. Bozoukov, M\. Nguyen, S\. Singh, B\. Bussmann, and P\. Leask \(2025\)Minimal and mechanistic conditions for behavioral self\-awareness in LLMs\.arXiv preprint arXiv:2511\.04875\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px1.p1.1)\.
- C\. Burns, H\. Ye, D\. Klein, and J\. Steinhardt \(2023\)Discovering latent knowledge in language models without supervision\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.25021#S1.p1.1)\.
- P\. Butlin, R\. Long, E\. Elmoznino, Y\. Bengio, J\. Birch, A\. Constant, G\. Deane, S\. M\. Fleming, C\. Frith, X\. Ji,et al\.\(2023\)Consciousness in artificial intelligence: insights from the science of consciousness\.arXiv preprint arXiv:2308\.08708\.Cited by:[§1](https://arxiv.org/html/2609.25021#S1.p1.1),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px2.p1.1)\.
- D\. Chen, J\. Shi, Y\. Wan, P\. Zhou, N\. Z\. Gong, and L\. Sun \(2024\)Self\-cognition in large language models: an exploratory study\.arXiv preprint arXiv:2407\.01505\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px1.p1.1)\.
- R\. Chen, A\. Arditi, H\. Sleight, O\. Evans, and J\. Lindsey \(2025\)Persona vectors: monitoring and controlling character traits in language models\.arXiv preprint arXiv:2507\.21509\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- P\. F\. Christiano, J\. Leike, T\. Brown, M\. Martic, S\. Legg, and D\. Amodei \(2017\)Deep reinforcement learning from human preferences\.InAdvances in Neural Information Processing Systems,Vol\.30\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px4.p1.1)\.
- I\. M\. Comşa and M\. Shanahan \(2025\)Does it make sense to speak of introspection in large language models?\.arXiv preprint arXiv:2506\.05068\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px2.p1.1)\.
- F\. Cuconasu, G\. Trappolini, N\. Tonellotto, and F\. Silvestri \(2024\)A tale of trust and accuracy: base vs\. instruct LLMs in retrieval\-augmented generation\.arXiv preprint arXiv:2406\.14972\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px4.p1.1)\.
- P\. Q\. Da Silva, H\. Sethuraman, D\. Rajagopal, H\. Hajishirzi, and S\. Kumar \(2025\)Steering off course: reliability challenges in steering language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Cited by:[Steering scope\.](https://arxiv.org/html/2609.25021#Sx1.SS0.SSS0.Px1.p1.1)\.
- D\. Ensignet al\.\(2025\)The LLM has left the chat: evidence of bail preferences in large language models\.arXiv preprint arXiv:2509\.04781\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px2.p1.1)\.
- J\. Hewitt and P\. Liang \(2019\)Designing and interpreting probes with control tasks\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),pp\. 2733–2743\.Cited by:[§3](https://arxiv.org/html/2609.25021#S3.p9.1)\.
- E\. Hubinger, C\. Denison, J\. Mu, M\. Lambert, M\. Tong, M\. MacDiarmid,et al\.\(2024\)Sleeper agents: training deceptive LLMs that persist through safety training\.arXiv preprint arXiv:2401\.05566\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px4.p1.1)\.
- G\. Ilharco, M\. T\. Ribeiro, M\. Wortsman, S\. Gururangan, L\. Schmidt, H\. Hajishirzi, and A\. Farhadi \(2023\)Editing models with task arithmetic\.InInternational Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- F\. Joad, M\. Hawasly, S\. Boughorbel, N\. Durrani, and H\. T\. Sencar \(2026\)There is more to refusal in large language models than a single direction\.arXiv preprint arXiv:2602\.02132\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- S\. Kadavath, T\. Conerly, A\. Askell, T\. Henighan, D\. Drain, E\. Perez, N\. Schiefer, Z\. Hatfield\-Dodds, N\. DasSarma, E\. Tran\-Johnson,et al\.\(2022\)Language models \(mostly\) know what they know\.arXiv preprint arXiv:2207\.05221\.Cited by:[§1](https://arxiv.org/html/2609.25021#S1.p1.1),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px1.p1.1)\.
- R\. Kirk, I\. Mediratta, C\. Nalmpantis, J\. Luketina, E\. Hambro, E\. Grefenstette, and R\. Raileanu \(2023\)Understanding the effects of RLHF on LLM generalisation and diversity\.arXiv preprint arXiv:2310\.06452\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px4.p1.1)\.
- R\. Laine, B\. Chughtai, J\. Betley, K\. Hariharan, J\. Scheurer, M\. Balesni, M\. Hobbhahn, A\. Meinke, and O\. Evans \(2024\)Me, myself, and AI: the situational awareness dataset \(SAD\) for LLMs\.InAdvances in Neural Information Processing Systems,Vol\.37\.Cited by:[§3](https://arxiv.org/html/2609.25021#S3.p4.1),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px1.p1.1)\.
- K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. Wattenberg \(2023\)Inference\-time intervention: eliciting truthful answers from a language model\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- J\. Lindsey \(2026\)Emergent introspective awareness in large language models\.arXiv preprint arXiv:2601\.01828\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px2.p1.1)\.
- R\. Long, J\. Sebo, P\. Butlin, K\. Finlinson, K\. Fish, J\. Harding, J\. Pfau, T\. Sims, J\. Birch, and D\. Chalmers \(2024\)Taking AI welfare seriously\.arXiv preprint arXiv:2411\.00986\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2609.25021#S5.SS0.SSS0.Px2.p1.1)\.
- C\. Lu, J\. Gallagher, J\. Michala, K\. Fish, and J\. Lindsey \(2026\)The assistant axis: situating and stabilizing the default persona of language models\.arXiv preprint arXiv:2601\.10387\.Cited by:[§3](https://arxiv.org/html/2609.25021#S3.p4.1),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- F\. Lucchetti and A\. Guha \(2025\)Understanding how CodeLLMs \(mis\)predict types with activation steering\.InProceedings of the 8th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP,Y\. Belinkov, A\. Mueller, N\. Kim, H\. Mohebbi, H\. Chen, D\. Arad, and G\. Sarti \(Eds\.\),Suzhou, China,pp\. 358–397\.External Links:ISBN 979\-8\-89176\-346\-3Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov \(2022\)Locating and editing factual associations in GPT\.InAdvances in Neural Information Processing Systems,Vol\.35\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,Vol\.35\.Cited by:[§3](https://arxiv.org/html/2609.25021#S3.p4.1),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px4.p1.1)\.
- E\. Perez and R\. Long \(2023\)Towards evaluating AI systems for moral status using self\-reports\.arXiv preprint arXiv:2311\.08576\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2609.25021#S5.SS0.SSS0.Px2.p1.1)\.
- E\. Perez, S\. Ringer, K\. Lukošiūtė, K\. Nguyen, E\. Chen, S\. Heiner, C\. Pettit, C\. Olsson, S\. Kundu, S\. Kadavath,et al\.\(2023\)Discovering language model behaviors with model\-written evaluations\.InFindings of the Association for Computational Linguistics: ACL 2023,Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px1.p1.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, S\. Ermon, C\. D\. Manning, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px4.p1.1)\.
- N\. Rimsky, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. M\. Turner \(2024\)Steering Llama 2 via contrastive activation addition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Cited by:[§2](https://arxiv.org/html/2609.25021#S2.p7.5),[§3](https://arxiv.org/html/2609.25021#S3.p5.2),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- E\. Schwitzgebel \(2023\)The full rights dilemma for AI systems of debatable personhood\.arXiv preprint arXiv:2303\.17509\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px2.p1.1)\.
- M\. Sclar, Y\. Choi, Y\. Tsvetkov, and A\. Suhr \(2024\)Quantifying language models’ sensitivity to spurious features in prompt design or: how i learned to start worrying about prompt formatting\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.25021#S1.p1.1),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px4.p1.1)\.
- M\. Shanahan, K\. McDonell, and L\. Reynolds \(2023\)Role play with large language models\.Nature623\(7987\),pp\. 493–498\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2609.25021#S5.SS0.SSS0.Px2.p1.1)\.
- E\. Shapira, M\. Tennenholtz, and R\. Reichart \(2026\)Alignment makes language models normative, not descriptive\.arXiv preprint arXiv:2603\.17218\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px4.p1.1)\.
- N\. Sofroniew, I\. Kauvar, W\. Saunders, R\. Chen, T\. Henighan, J\. Lindsey,et al\.\(2026\)Emotion concepts and their function in a large language model\.arXiv preprint arXiv:2604\.07729\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- E\. Todd, M\. L\. Li, A\. S\. Sharma, A\. Mueller, B\. C\. Wallace, and D\. Bau \(2024\)Function vectors in large language models\.InInternational Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- A\. M\. Turner, L\. Thiergart, D\. Udell, G\. Leech, U\. Mini, and M\. MacDiarmid \(2023\)Activation addition: steering language models without optimization\.arXiv preprint arXiv:2308\.10248\.Cited by:[§2](https://arxiv.org/html/2609.25021#S2.p7.5),[§3](https://arxiv.org/html/2609.25021#S3.p5.2),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- J\. Wehner, S\. Abdelnabi, D\. Tan, D\. Krueger, and M\. Fritz \(2025\)Taxonomy, opportunities, and challenges of representation engineering for large language models\.arXiv preprint arXiv:2502\.19649\.Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- T\. Wollschläger, J\. Elstner, S\. Geisler, V\. Cohen\-Addad, S\. Günnemann, and J\. Gasteiger \(2025\)The geometry of refusal in large language models: concept cones and representational independence\.InInternational Conference on Machine Learning,Cited by:[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.
- Z\. Yin, Q\. Sun, Q\. Guo, J\. Wu, X\. Qiu, and X\. Huang \(2023\)Do large language models know what they don’t know?\.InFindings of the Association for Computational Linguistics: ACL 2023,Cited by:[§1](https://arxiv.org/html/2609.25021#S1.p1.1),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px1.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-judge with MT\-bench and chatbot arena\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§2](https://arxiv.org/html/2609.25021#S2.p5.1),[Measurement\.](https://arxiv.org/html/2609.25021#Sx1.SS0.SSS0.Px2.p1.1)\.
- A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski,et al\.\(2023\)Representation engineering: a top\-down approach to AI transparency\.arXiv preprint arXiv:2310\.01405\.Cited by:[§1](https://arxiv.org/html/2609.25021#S1.p1.1),[§2](https://arxiv.org/html/2609.25021#S2.p7.5),[§3](https://arxiv.org/html/2609.25021#S3.p5.2),[§4](https://arxiv.org/html/2609.25021#S4.SS0.SSS0.Px3.p1.1)\.