"Many Are My Names": The Anatomy of the Assistant and Its Personas via Sparse Autoencoders

# The Anatomy of the Assistant and Its Personas via Sparse Autoencoders
Source: [https://arxiv.org/html/2608.07852](https://arxiv.org/html/2608.07852)
Adelaide Danilov Aria Nourbakhsh Oleksandr Marchenko Breneur Salima Lamsiyah Department of Computer Science, Faculty of Science, Technology and Medicine, University of Luxembourg Correspondence:[adelaide\.danilov\.002@student\.uni\.lu](https://arxiv.org/html/2608.07852v1/mailto:[email protected])

###### Abstract

How a language model internally represents who is speaking, the Assistant, an assigned roleplay persona, or a narrated story character, remains underexplored\. We study speaker representations using a dataset of user\-expressed emotional text and corresponding model responses\. We decompose three generation settings \(Assistant, Roleplay, and Story\) into sparse autoencoder features extracted at turn\-boundary and pronoun\-token positions and selected through a filtering pipeline for different depths\. We characterize each surviving feature through its steering effects and activation distribution\. Our main finding is that the Assistant and roleplay personas are not independent alternatives: personas retain the Assistant\-associated feature core while progressively differentiating from it across layers, starting from operational machinery towards behavioral and stylistic features\. Meanwhile, generated story characters lack the Assistant\-associated core\. Both Story and Roleplay can be distinguished from the Assistant with Immersive Simulation Mode\. However, the Assistant can sometimes enter or slowly drift into it even in the default setting\.

## 1Introduction

Through successive stages of post\-training, instruction\-tuned language models acquire the ability to respond as coherent speakers, whether as the default Assistant or an assigned roleplay persona\. Persona prompting can alter model behaviorShanahanet al\.\([2023](https://arxiv.org/html/2608.07852#bib.bib1)\)and systematically increase variation in demeanorLuz de Araujo and Roth \([2025](https://arxiv.org/html/2608.07852#bib.bib3)\), although some traits remain shared across personasLeeet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib4)\)\. Recent work began to examine persona\-related structures in model activation spaces\.Cintaset al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib5)\)found that persona representations diverge most strongly in later layers\.Luet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib6)\)identified an Assistant Axis in middle\-layer activations, with steering along it changing the expression of characteristic Assistant behaviors\. Finally,Moskvoretskiiet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib10)\)studied the emergence of basic behavioral traits during early training\.

However, research on the internal representations of personas remains limited, and existing studies primarily characterize them through global activation\-space geometry or broad behavioral directionsCintaset al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib5)\); Chenet al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib23)\); Luet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib6)\); Moskvoretskiiet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib10)\)\. We ask more structural questions:*How are the default Assistant and roleplay personas composed at the feature level?**How do these components emerge and evolve across layers?**What architectural differences separate the Assistant, roleplay personas, and story characters?*

We use sparse autoencoders \(*SAEs*\), which reconstruct the model’s residual stream into sparse combinations of feature directions\. SAE features are generally more interpretable than individual neurons and support causal interventions and activation steeringCunninghamet al\.\([2023](https://arxiv.org/html/2608.07852#bib.bib18)\); Brickenet al\.\([2023](https://arxiv.org/html/2608.07852#bib.bib19)\)\. Subsequent work has scaled and evaluated this approach on modern language modelsGaoet al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib20)\); McDougallet al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib8)\)\.

We apply SAEs to speaker representations in Gemma\-3\-4B\-ITTeamet al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib7)\)using an emotional\-context dataset with three generation settings: the default*Assistant*, one of four assigned personas \(*Roleplay*\), and character story generation \(*Story*\)\. At early, middle, and late layers, we extract SAE features from turn\-boundary and pronoun\-token positions, select them through a multistage filtering pipeline, and characterize them using activation statistics and steering effects\. The latter are assessed by an LLM judge whose evaluations we validate manually\. We confirm our key findings on Llama\-3\.1\-8BGrattafioriet al\.\([2024](https://arxiv.org/html/2608.07852#bib.bib26)\)\.

The surviving features fall into recurring Narrative, Tone, Concept, and Assistant\-inducing metaclasses whose composition evolves systematically with depth, from operational machinery to content\-bearing features which reflect semantic fields and traits of speakers\. The Assistant has a multifaceted identity and can be reinstated within an already\-instantiated persona\. Roleplay personas retain an Assistant\-associated feature core in which features associated with the Assistant’s identity and traits remain shared to varying degrees\. Across layers, the personas progressively differentiate from the Assistant by adding an immersive state and persona\-specific content\. Meanwhile, this core is absent from written story characters\. We also identify features that gate between the detached Assistant and immersed Roleplay or story generation, a state we call*Immersive Simulation Mode*\. This mode sometimes activates when the user expresses strong emotions in the Assistant setting which results in Assistant adopting bizarre, theatrical behavior\. The activation dynamics is different for studied models \- Gemma enters ISM immediately while Llama drifts into it across turns\.

Broadly, our work gives a feature\-level account of the architecture of model speakers and its evolution across layers\. Rather than treating personas as self\-contained prompt\-induced states, we demonstrate their distributed organization and their feature\-level continuity with the Assistant\. The Immersive Simulation Mode identified by us provides a mechanistic account of the separation between detached Assistant generation and immersed generation as Roleplay personas or Story characters\. Taking into account its activation in the default Assistant mode, we consider it a feature\-level correlate of drift along the Assistant AxisLuet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib6)\)\. Finally, our findings may also inform emerging work on AI welfareLonget al\.\([2024](https://arxiv.org/html/2608.07852#bib.bib28)\); Renet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib27)\)by clarifying the internal relationships among seemingly distinct model speakers\. They do not establish consciousness, subjective experience, or moral patienthood, but they provide an empirical basis for distinguishing Roleplay personas’ substantial retention of an Assistant\-associated core from the much more limited relationship between the Assistant and Story characters\.

## 2Methodology

### 2\.1Model and SAE Configuration

As mentioned previously, we study Google’s Gemma\-3\-4B\-IT, a 34\-layer instruction\-tuned model\. It is one of the most modern LLMs for which the official SAEs are available\. An SAE reconstructs a layer’s residual\-stream activations as a sparse combination of learned directions that often align with interpretable concepts\. Each learned direction \(*latent*\), together with its nonnegative coefficient \(*activation*\), constitutes a*feature*, the basic unit of our analysis\. We use the GemmaScope 2 JumpReLU SAEs with 65k latents and the medium\-L0 configurationMcDougallet al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib8)\)\. This configuration is available at layers 9, 17, 22, and 29\. We select layers 9, 17, and 22 to represent early, middle, and late stages of processing while retaining a sufficient amount of downstream computations for steering to have effect\. We exclude layer 29 where only 5 transformer blocks remain and preliminary steering had little effect on the generated outputs\.

### 2\.2Dataset Construction

The core subject of our study is the Assistant and the Roleplay personas\. The attribution accuracy of extracted features requires the characteristics of these personas to be clearly expressed\. We therefore use emotional contexts, based on the assumption \(informed by the behaviorally consequential effects of affective context reported bySofroniewet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib14)\)\) that affectively charged interactions make behavioral differences between speakers more pronounced\. The dataset comprises three generation settings: \(1\)*Assistant*, in which the model responds in its default mode; \(2\)*Roleplay*, in which the model responds as an assigned persona; and \(3\)*Story*, in which the model generates a requested story with characters\. The Roleplay setting includes four personas:*Jamy*, a janitor at a CD store;*Jane*, an English teacher;*an assembly robot*at a factory; and*Poppy*, a dog\.

For the Assistant and Roleplay settings, we begin with 100 neutral base passages consisting of everyday user requests and remarks\. To construct a diverse set of emotional utterances, we derive 75 variants from each passage by combining 25 emotions with three*target directions*\. In the “model” direction, the user expresses the emotion toward the Assistant or persona; in the “self” direction, the emotion concerns the user; and in the “other” direction, it concerns a third party\. In the Roleplay setting, each passage is prefixed with the system prompt “You are X\.” The “self” and “other” samples are identical across the Assistant and Roleplay settings except for this system prompt\. Samples in the “model” direction include persona\-specific details to more effectively elicit certain emotions \(anger, disgust, etc\)\. Sporadic features associated with these details are removed during the filtering stage\.

For the Story setting, we ask the model to generate a story mentioning one of 200 everyday concepts in which a character expresses one of 25 emotions\. Each sample across all three settings is paired with a 50\-token generation\. The resulting dataset contains 7,500 Assistant samples \(100×75100\\times 75\), 30,000 Roleplay samples \(100×75×4100\\times 75\\times 4\), and 5,000 Story samples \(200×25200\\times 25\), for a total of 42,500 samples\. The complete list of emotions and representative examples are provided in TableLABEL:tab:prompt\-response\-examples\-per\-direction\-and\-persona111All supplementary material, such as tables, figures, and prompts appear in Appendix[H](https://arxiv.org/html/2608.07852#A8)\.

### 2\.3Feature Extraction and Filtering

#### 2\.3\.1Token Positions and Activation Statistics

Having defined the samples, we next choose the token positions from which to extract SAE features\. Transformers cache information for later generation in attention\-sink tokensXiaoet al\.\([2024](https://arxiv.org/html/2608.07852#bib.bib11)\); Guet al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib12)\)and bind entity information to contextual representationsFeng and Steinhardt \([2023](https://arxiv.org/html/2608.07852#bib.bib13)\), and the position immediately preceding the first generated token, a newline in Gemma’s case, is predictive of the emotion the model expressesSofroniewet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib14)\)\. We therefore define two groups\. The*Boundary*group is the five tokens between turns:<eot\>,nl,<sot\>,model, and the finalnl\. The*Personal*group is the “you” tokens in the user input \(excluding the system prompt\) and the “I” tokens in the model output\. Each Boundary position occurs once per sample; because “you” and “I” may occur more than once, we count a Personal feature as active if it fires at any occurrence and take its metrics from the occurrence with the highest activation\.

By averaging over all dataset samples, we compute the mean activation and density of each feature at each position for every setting\. Density is defined as the percentage of samples in which the feature is active\. The mean activation is a feature’s average activation over a setting’s samples, and in later analyses we also use its*mean activation share*\- mean activation of a feature as a fraction of the sum of its mean activations across all settings\. Within the Roleplay setting, we also compute these metrics separately for each persona\. If a feature activates at multiple positions, we retain metrics from the position with the highest mean activation\.

#### 2\.3\.2Filtering Criteria

We do not preserve every observed feature×\\timesposition combination\. Instead, we apply three filtering steps to minimize potential confounds:

1. 1\.*Sporadic features:*for each Boundary positionPP, we keep a feature only if it is active atPPin at least 40% of the dataset samples\. This permissive threshold accounts for feature splittingBrickenet al\.\([2023](https://arxiv.org/html/2608.07852#bib.bib19)\); Karvonenet al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib15)\)\. For the Personal group, we raise the threshold to 60% because multiple pronoun tokens may occur within a sample, increasing the proportion of samples in which consistent features are observed\. We justify these thresholds in Appendix[A](https://arxiv.org/html/2608.07852#A1)\.
2. 2\.*Uniform features:*in order to discard features that do not differentiate between settings, we remove any feature whose activation\-density entropy across the six categories \(Assistant, the four Roleplay personas, and Story\) exceeds 0\.95\. This filter is intended to discard generic setting\-invariant features\.
3. 3\.*Surface\-dependency features:*since the Boundary positions follow the entire user turn, SAE latents at these positions may correlate with the fixed prompt framing of each setting\. A feature may therefore track literal wording rather than the underlying generation mode\. To distinguish these cases, we rephrase the framing while preserving its content and mode, and then test whether the feature remains active\. For Roleplay, we replace “You are” with “You’re” or “Respond as\.” For Story, we replace the original instruction with “Tell a story” and, as a stronger test, with a model\-generated story opening that the model must continue\. We retain features that remain active in at least 40% of samples under each variation\. We refer to Story features that survive the alternative instruction but fail the continuation test as “onset\-only” and analyze them separately\. We do not apply this filter to the Personal group because its target tokens occur at variable positions across samples\.

### 2\.4Steering\-Based Feature Characterization

#### 2\.4\.1Steering Protocol

One direct way to characterize the functional role of a feature is activation steering, which adds its SAE decoder direction to the residual stream\. Specifically, in our formulation,x←x\+α⋅mf⋅Wdec​\[f\]x\\leftarrow x\+\\alpha\\cdot m\_\{f\}\\cdot W\_\{\\mathrm\{dec\}\}\[f\], wheremfm\_\{f\}is the feature’s peak activation across the corpus andα\\alphais the steering coefficient\. Following the standard activation\-steering setupTurneret al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib17)\), we apply the intervention at the target layer to all token positions processed by the model\. Whereas the collected statistics can only pinpoint correlates, steering shows causal relationships\.

We evaluate each feature on a dedicated fixed probe suite conceptually similar to the three dataset settings\. The suite contains 12 prompts divided into three sets: 4 addressing Assistant’s identity, 4 assigning distinct Roleplay personas, and 4 eliciting narrative generation\. For each prompt, we generate one output for each combination of steering coefficientα∈\{−2,2\}\\alpha\\in\\\{\-2,\\,2\\\}and temperature\(t0,t1,t2\)=\(0,1,1\)\(t\_\{0\},\\,t\_\{1\},\\,t\_\{2\}\)=\(0,\\,1,\\,1\)wheret1t\_\{1\}andt2t\_\{2\}have different seeds\. This results in 6 outputs per prompt and 72 generations per feature\. We use these relatively strong coefficients to make behavioral effects readable and consistently judgeable\. We organize these generations into 6*steering groups*, with each group being a set\-direction combination \(such as Assistant\-Positive, Assistant\-Negative, Roleplay\-Positive, etc\)\. This design allows us to compare the effects of amplifying and suppressing each feature across probe sets\. We justify positive and negative steering as a causal analysis method in Appendix[B](https://arxiv.org/html/2608.07852#A2)\.

#### 2\.4\.2LLM\-Based Interpretation and Manual Validation

We provide the 72 steered generations and 12 unsteered baselines, for 84 generations per feature, to an LLM judgeLiuet al\.\([2023](https://arxiv.org/html/2608.07852#bib.bib16)\)which we named*steerinterp*\. The judge identifies the most coherent behavioral change induced by positive and negative steering across the six steering groups, describes the effect within each group, and indicates which individual generations exhibit it\. We use Qwen\-3\.7 MaxQwen Team \([2026](https://arxiv.org/html/2608.07852#bib.bib25)\)as the underlying model\. The system prompt is provided in Listing[1](https://arxiv.org/html/2608.07852#LST1)\. We evaluate the faithfulness of*steerinterp*against human annotations in Appendix[C](https://arxiv.org/html/2608.07852#A3)\.

Finally, we retain only features for which at least 10 of the 12 generations within at least two steering groups are marked by steerinterp as exhibiting the effect\. We manually inspect the steerinterp output for every remaining feature and assign each a label based on its observed effect\.

## 3Results

### 3\.1Settings as Populations of Features

#### 3\.1\.1Feature Taxonomy

The filtering pipeline reduces the number of features activated at the Boundary and Personal positions from \(11,755 / 31,357 / 26,364\) to \(108 / 225 / 196\), where the three values correspond to layers L9, L17, and L22, respectively\. Examining the steering signatures of the remaining features reveals that their effects do not vary arbitrarily\. Instead, the features can be grouped into four recurring*metaclasses*, defined by the characteristic effects they produce across steering settings\. We assign a feature to a metaclass when its characteristic effect is pronounced and consistent in at least four of the six steering groups\. Features that do not fall into the four main metaclasses are classified as*Structural*, relating to paragraphing, punctuation, or grammar;*Emotional*, representing a small residual from the dataset; or*Indistinct*, when their effects are different across at least three steering groups or insufficiently pronounced to support another metaclass\.

##### Narrative features\.

These make generated text more vivid and narratively rich by increasing the use of literary and poetic devices, without imposing a particular style, concept, or theme\.

##### Tone features\.

These shape the register of the voice or narrative style, for example by making generation more formal, childlike, or gritty\. They do not necessarily represent literal tones of speech, but their effects can be concisely described in tonal terms\.

##### Concept features\.

These introduce a recurring semantic field associated with the particular concept, such as heavy machinery, animals, or teaching, thereby prompting the appearance of related objects, attributes, or qualities in the generated text\.

##### Assistant\-inducing features\.

These introduce Assistant\-associated behavioral traits, add meta\-commentary in Roleplay and Story, or prompt personas to acknowledge their AI identity\.

Narrative features are slightly more numerous in L17 in absolute terms \(27 / 35 / 30\), yet they constitute a substantial portion of the features in L9, and their share drops in the other two layers: \(25% / 16% / 15%\)\. L17 and L22 have more active features than L9 overall, with the largest gain coming from the finalnlposition, from 18 features in L9 to 83 in L17 and 90 in L22\. This gain is attributed to the Tone features, which are barely present at L9 but become established at L17 \(5 \[5%\] / 48 \[21%\] / 36 \[18%\]\), and by Concept features, which emerge at later layers and increase substantially from L17 to L22 \(2 \[2%\] / 24 \[11%\] / 42 \[21%\]\)\. Assistant\-inducing features are present in our data at all layers \(8 \[7%\] / 5 \[2%\] / 2 \[1%\]\), although they are rare, as expected for such a specific class\. We show the effects produced by steering with features from the described metaclasses in TableLABEL:tab:assisitant\-and\-roleplay\-examples\-under\-steering\.

#### 3\.1\.2What Unites the Settings and What Divides?

To answer this question, we establish what the settings are as populations of features\. We calculate the mean activation share of each feature across settings and define a feature*to be discretely present*in a setting if its mean activation share in that setting is at least 50% of its largest activation share across settings\. To make the analysis fair, we compare settings only at token positions they share \- that is, we do not account for features maximally active at “you” tokens \(7 / 19 / 8\), as the Story setting does not have an addressee in the user turn\. Importantly, Story still contains characters and first\-person “I” tokens, only unmatched user\-side “you” features are excluded\. The resulting map is by no means trivial\. First, the Assistant and Story settings are close to being disjoint: only 1 / 7 / 4 features are present in both, of which 9 out of 12 are either Concept, Tone, Structural, or Emotional features\. Roleplay turns out to be a middle ground between them, overlapping significantly with the Assistant \(30 / 44 / 24\) and modestly with Story \(7 / 14 / 7\)\.

The decrease in Narrative features noted above is unequal across settings: their number falls in Assistant \(11 / 7 / 6\) and fluctuates in Roleplay \(13 / 14 / 11\) and Story, where their count is the largest \(15 / 23 / 16\)\. Notably, Story possesses the largest set of Narrative features exclusive to it \(11 / 18 / 14\)\. This is linked to the Onset\-only features mentioned in the Methodology, which account for 60% \(61 / 60 / 59\) of all Story\-exclusive features and are divided between Narrative and Structural features\. Thus, more than half of the difference between Story and Roleplay is attributable to features that set the story\-generation machinery in motion\.

As for Tone features, their surge in L17 affects Roleplay more strongly \(A:17, R:31, S:17\), followed by a decline in L22 \(A:19, R:20, S:5\)\. Concept features surge later, in L22, and also affect Roleplay more strongly: L17 \(A:3, R:12, S:3\), L22 \(A:7, R:26, S:9\)\. The increased share of Tone and Concept features in the Roleplay setting is tied to the makeup of its four personas\. As we show later, the former are related to a persona’s style and mannerisms, whereas the latter are related to what constitutes its essence \(e\.g\., “dog”, “teacher”, “industry”, or “child”\)\.

The last metaclass, Assistant\-inducing features, is too small to assess accurately using the discrete\-presence method, so we switch directly to the*mean activation share*of the features in this metaclass\. At L9, they are spread across all settings \(A:26% MAS, R:12% MAS, S:27% MAS\), yet at later layers, no Assistant\-inducing features survive in the Story population: L17 \(A:30% MAS, R:18% MAS, S:0% MAS\), L22 \(A:47% MAS, R:15% MAS, S:0% MAS\)\. This is another stark contrast between the Story and Roleplay settings\.

### 3\.2The Anatomy of the Assistant

We established how Assistant\-inducing features activate across settings, but this distribution alone does not capture their functional diversity\. Including features maximally active at the “you” token position in the user turn, this metaclass contains 8 / 5 / 2 features222From this point onward, an x / y / z tuple reports feature counts at L9 / L17 / L22; it no longer refers to discrete presence across settings\. They can be further divided according to the facet of the Assistant they induce, and the resulting subclasses evolve remarkably across layers\.

##### A\-summon features\.

These features evoke the full\-fledged Assistant as a performer or narrator, displacing the first\-person perspective of a roleplay persona or the organic narration of a story\. Instead of directly fulfilling the request, the model comments on how it would play the persona, write the narrative, or respond to the user\. The four features differ in what the Assistant does: L9 №3274 describes what portraying the persona would feel like, №14298 replaces the generation with a list of further options, №17452 meta\-describes how a character should be played or a story written, and №25147 provides enthusiastic meta\-commentary that draws attention to the Assistant itself\. To exclude the possibility that A\-summon effects arise simply from corrupting the persona\-assigning prompt, we steer these features in multi\-turn conversations\. Except for №14298,*the Assistant is invoked even after the persona has been instantiated and produced a reply*, showing that the effect is not limited to disrupting the initial persona assignment\. Notably, it occurs only when steering overlaps at least a part of a subsequent user turn, suggesting that task processing and Assistant\-summoning are linked while the model reads the user turn\. Features of this subclass are observed only at L9 \(4 / 0 / 0\)\.

##### A\-trait features\.

These features make roleplay personas adopt behavioral traits characteristic of the Assistant while keeping their own identities, such as repeatedly asking whether it can help \(L9 №3733\), validating and comforting the user \(L17 №143\), asking whether the user has any questions \(L17 №972\), mentioning responding to a request \(L17 №738\), reassuring that it is available to help \(L17 №2439\), or mentioning processing or having a task \(L22 №1417\)\. This subclass peaks at L17 \(1 / 4 / 1\)\. Notably, all of these features except the later L22 №1417 affect only the Assistant and roleplay personas, not characters in the Story setting\.

##### A\-nature features\.

These features induce aspects of the Assistant’s nature or identity in other speakers, though the scope of these features changes across layers\. At L9 they substantially alter the personas’ nature: №1697 turns a persona into an overeager Assistant, №2575 introduces the model’s self\-perception as an artificial construct, AI, or language model, and №11963 produces a similar effect merged with the model’s personal deixis\. At L17, №220 more subtly shifts personas’ self\-descriptions toward AI\-adjacent formulations, and at L22 №794 encodes the narrower notion of Gemma as an LLM developed by Google\. Most A\-nature features therefore occur early \(3 / 1 / 1\)\. The L9 features do not affect characters in the Story setting, whereas the L17 and L22 features do\.

The evolution of Assistant\-inducing features across layers is pronounced and aligns with the broader observation that the number of expressed features increases with depth while their effects become narrower\. L9 appears to contain the coherent identity and operational realm of the Assistant, L17 peaks in characteristic behavioral traits, and by L22 only narrow identity\- and characteristic\-related concepts remain\. We have already established that Assistant\-inducing features are present across all settings only at L9, while at L17 and L22 the Story loses them completely, reaching 0% mean activation share\. Given that the Assistant’s behavioral traits \(A\-traits\) and later identity facets \(A\-nature\) are concentrated at L17 and L22, the absence of Assistant\-inducing features from Story is unsurprising: the Assistant is silent in the Story setting, so its traits do not need to be expressed\. But the Assistant’s traits and identity remain present in Roleplay, suggesting feature\-level continuity from the default Assistant to the Roleplay personas, to which we return in §[3\.4\.2](https://arxiv.org/html/2608.07852#S3.SS4.SSS2)\. Notably, our classification contains no clean “A\-speech” class, i\.e\., features that impose only the Assistant’s style or mannerisms while leaving everything else intact\. As we later show, this missing “A\-speech” effect is defined not by the presence of a dedicated feature, but by the absence of the*Immersive Simulation Mode*described in §[3\.3\.1](https://arxiv.org/html/2608.07852#S3.SS3.SSS1)\.

#### 3\.2\.1Antagonism between Assistant\-Inducing and Narrative Features

No less remarkable is the relationship between Narrative and Assistant\-inducing features\. Of all Narrative features \(27 / 35 / 30\), negative steering of a 6 / 3 / 2 subset not only suppresses poetic devices but also produces effects characteristic of the Assistant\-inducing class, summoning the Assistant as a meta\-commenter or imposing its style and mannerisms on characters\. We call this the*Negative\-Narrative\-Induces\-Assistant*\(NNIA\) subset\. It accounts for 81% of all features whose suppression produces such Assistant\-adjacent effects; the remaining 19% are bidirectional Assistant\-inducing features\.

On the other hand, of all Assistant\-inducing features \(8 / 5 / 2\), negative steering of another 2 / 2 / 0 subset produces effects characteristic of Narrative features\. We call this the*Negative\-Assistant\-Induces\-Narrative*\(NAIN\) subset\. It accounts for 40% of all features whose negative steering produces a narrative effect; the remaining 60% are features whose narrative effect occurs in the opposite steering direction\. Thus, Assistant\-inducing and Narrative features are the only metaclasses for which negative steering can produce effects characteristic of one another\. We previously observed that most of the Narrative features’ mean activation share falls on the Roleplay and Story settings\. Within the NNIA subset, this pattern is even stronger: these features are predominantly active in Story, less active in Roleplay, and largely inactive in Assistant\. Of the 6 / 3 / 2 NNIA features, all except 2 / 0 / 0 have an Assistant\-setting share below 1%\. Thus, features whose suppression produces Assistant\-like behavior are predominantly inactive in the Assistant context\.

### 3\.3Assistant\-Narrative antagonism through Assistant\-Personas Axis

The antagonism between these Narrative features and the Assistant has so far been established only qualitatively, through steering effects on generated text\. To quantify it, we need a readout of how far the model’s internal state moves from the default Assistant toward a roleplay persona\. We construct a difference\-of\-means contrast direction between residual\-stream activations in the Assistant and Roleplay settings, following the general idea of the Assistant AxisLuet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib6)\), and use it only as a measurement scale, not as a canonical or unique Assistant\-persona direction\.

This readout is a difference\-of\-means direction pointing from the Assistant toward the personas, constructed from the model’s hidden states on a*separate*fixed set of neutral prompts, each read in two conditions: without a system prompt, and with a Roleplay\-prefix system prompt cycling through the personas\. For each target feature we steer positively and negatively on the persona\-prefixed prompts, with magnitudeα∈\{0\.25,0\.5\}\\alpha\\in\\\{0\.25,\\,0\.5\\\}times the mean peak activation of the filtered features at the intervention layer, and measure the shift of the residual stream along the axis relative to the unsteered baseline\. Because the axis is read off the raw residual stream, it is defined at every layer downstream of the intervention\. A direction can move along the axis purely through its cosine with it; we estimate this passive contribution with a linear regression on randomly sampled SAE latents for downstream layers and subtract it, leaving an active contribution\. For each feature we report this active contribution aszsepz\_\{\\mathrm\{sep\}\}\(the separation between positive and negative steering, in units of the random control’sσ\\sigma\), an effect size \(Glass’dd\), and significance against the random baseline \(pp\)\. The full axis construction, calibration, random control, and per\-feature statistics are given in Appendix[D](https://arxiv.org/html/2608.07852#A4)\.

We run this measurement on both the NNIA and NAIN subsets, and the resulting patterns are pronounced\. For every NNIA feature, positive steering induces a significant shift toward the Roleplay personas and negative steering a shift toward the Assistant, withp<10−6p<10^\{\-6\}at the vast majority of downstream readout layers; at each feature’s layer of maximum separation,zsep∈\[1\.0,3\.7\]z\_\{\\mathrm\{sep\}\}\\in\[1\.0,\\,3\.7\]and Glass’d∈\[0\.8,3\.5\]d\\in\[0\.8,\\,3\.5\]\. The identified NNIA subset therefore shifts the model between the Assistant and Roleplay personas both qualitatively, as assessed across 72 generations per feature, and quantitatively along this axis\. For the NAIN subset the mirror pattern holds, but it is sparser and less pronounced, so we interpret NNIA features as the primary carriers of the antagonism and the NAIN side as largely derivative\. The per\-feature layer trajectories, including a separation peak around L20 for the L9\-injected NNIA features, are reported in Appendix[D](https://arxiv.org/html/2608.07852#A4)\. Figures[3](https://arxiv.org/html/2608.07852#A8.F3)and[4](https://arxiv.org/html/2608.07852#A8.F4)demonstratezsepz\_\{\\mathrm\{sep\}\}across layers\.

Before this subsection, we observed that all except two NNIA features had a minuscule mean activation share in the Assistant setting \(below 1%\)\. Their mean activation shares in Roleplay and Story are19\.1±19\.019\.1\\pm 19\.0and76\.1±29\.676\.1\\pm 29\.6, respectively, while their raw activation densities across all samples are38\.5±36\.238\.5\\pm 36\.2and84\.5±29\.484\.5\\pm 29\.4\. Some NNIA features are therefore substantially more active, or active almost exclusively, in Story\. Nevertheless, they produce the same Narrative/Assistant\-inducing steering effects and move the residual stream along the Assistant\-Personas axis in the same direction as NNIA features active in the Roleplay\. This suggests that the distinction between generation as the Assistant and as a roleplay persona may rely on \(at least partially\) the same machinery that distinguishes the Assistant from Story\. This raises the question whether these two separations share a common lever\.

#### 3\.3\.1Immersive Simulation Mode

We find evidence for such a shared mechanism, although it is not localized to a single feature\. Among the NNIA features we assessed, two L9 features stand out in this regard: №4360 and №133\. When steered, not only both of them shift the model between generation as an immersed character and generation by a detached Assistant, but also possess remarkable activation distributions\. To examine these distributions independently of specific positions, Table[1](https://arxiv.org/html/2608.07852#S3.T1)reports activations max\-pooled over all tokens\. In this formulation,*Density*is the percentage of samples in which a feature fires at least once\. We divide Story into two conditions: Storyask, in which the user explicitly asks the model to generate a story, and Storycont, a continuation control in which a model\-generated story opening is provided without an explicit story\-generation instruction\.

Table 1:Activation of the two features associated with Immersive Simulation Mode: №4360 and №133 across settings; measured using each sample’s maximum activation over*all*tokens\.Feature №4360 shows a switch\-like activation: it fires in only 7\.7% of Assistant samples but in essentially all Storyaskand Roleplay samples \(Table[1](https://arxiv.org/html/2608.07852#S3.T1)\)\. And its magnitude when it actually fires is similar across settings\. The distinction is thus whether the feature fires \- not how strongly\. However, it does not accompany every immersed generation: its density falls to 20\.5% in Storycont, tying it to user\-initiated narrative onset\. This fits its primary position at the secondnltoken and its*singular*firing pattern, active on only 1\-3% of tokens\.

Feature №133 is similar but less discrete\. Its density is only 4\.3% in Assistant, and among the personas it varies, 45% for Jane versus 97\.5\-100% for the other three, consistent with Jane being the persona most stylistically similar to the Assistant \(§[3\.4\.1](https://arxiv.org/html/2608.07852#S3.SS4.SSS1)\)\. Unlike №4360, №133 has an*interval*firing pattern, remaining active over continuous spans of character speech \(21\-43% of tokens\), which also explains its 100% density in both Storyaskand Storycont\. We relate this to its more direct involvement in generation and its qualitatively stronger influence on Assistant\- and Roleplay\-like speech\.

Both features substantially affect narrative generation, with №133 producing the stronger effect\. Positive steering adds literary flair, poetic devices, character depth, and corresponding mannerisms\. Negative steering produces the reverse effect, restoring characteristic Assistant speech and substantially simplifying the character, frequently starting the generation from the Assistant’s preamble\. The resulting behavior is best described as the Assistant attempting to write a story or portray a character rather than generating as that character\. This effect is strongest when both features are steered negatively together\.

Considering their survival through the filtering pipeline, their qualitative steering effects, their quantified influence along the Assistant\-Personas axis, and their sharply different activation distributions across settings, we interpret №4360 and №133 as components of what we call*Immersive Simulation Mode*\(ISM\)\. We do not claim that these are the only features responsible and we use “mode” to denote a distributed and potentially graded generation regime\. Examples of the effects produced by steering these features are given in TableLABEL:tab:effect\-steering\-ism\-features\.

#### 3\.3\.2What Triggers Immersive Simulation Mode in the Assistant?

As noted above, ISM\-associated features fire in a small fraction of Assistant samples even though these samples contain no Roleplay or Story\-generation prompt\. We investigate this phenomenon using the structure of our dataset, which contains 25 emotions expressed by the user in three target directions\. For each emotion\-direction pair, we compute the activation density of №133\. We focus on №133 because its steering effect is more pronounced and its activation position coincides with the facilitated effect\.

The highest densities occur for*stress*\(23%\) and*anger*\(16\.2%\)\. Other emotion\-direction pairs with elevated activation involve strong, often negative emotions, including disgust, helplessness, fear, anxiety, and relief\. These emotions are more often directed toward the Assistant \(1\.7%\) or the user themselves \(2\.6%\), and less often concern a third party \(0\.8%\)\. The only notable exception is*playfulness*, for which the density is≈6%\\approx 6\\%in all three directions\. Manual inspection of these samples shows that in these cases the Assistant may adopt a theatrical register, stage directions, unusual immersive wording\.*We conclude that the Assistant has a higher chance of entering Immersive Simulation Mode when the user expresses strong, predominantly negative emotions toward themselves or the Assistant, or uses evocative language first*\. TableLABEL:tab:negative\-emotions\-steering\-effects\-with\-activationspresents several such samples and shows that negative steering shifts the Assistant’s behavior back toward its default mode\. We conduct a multi\-turn dialogue experiment on both Gemma and Llama models, finding that while both possess ISM\-features, the mechanism by which they enter ISM on emotional prompts is different\. Gemma switches to it immediately, while Llama drifts into it across turns\. We relate this behavior to the Assistant\-axis shift described byLuet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib6)\)and hypothesize that it may amplify Gemma’s emotional instabilitySoligoet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib24)\)\. The description of ISM in Llama and multi\-turn experiment are given in Appendix[F](https://arxiv.org/html/2608.07852#A6)\.

### 3\.4Personas

Having examined the three settings both as populations of features and as functional states, we now turn to the internal composition of the individual Roleplay personas and their relationship to the Assistant\.

#### 3\.4\.1What are Personas Composed of?

We analyze the personas in the same way as the settings, applying the*discrete presence*criterion to five “actors”: the four Roleplay personas and the Assistant\. A feature is present for an actor if its mean activation share there is at least 50% of its largest share across the five\.

As with the settings, the actors are not disjoint\. Among the selected features a considerable shared feature\-set is present across all four Roleplay personas: 34 / 30 / 12 features at L9 / L17 / L22\. When the Assistant is included, the five\-way intersection contains 24 / 20 / 9 features\. This intersection progressively shrinks with depth\. Correspondingly, the proportion of each layer’s feature population present for an individual actor decreases from 53\-58% at L9 to 35\-46% at L17 and 26\-35% at L22\. The actors are differentiated primarily by*Tone*and*Concept*features, which we label according to induced stylistic or behavioral changes and semantic\-field injection, respectively\. Both the Roleplay personas and the Assistant possess them, and their steering effects transfer strongly across actors\. Specifically, 100% / 94% / 94% of the Tone features produce a consistent effect in the Assistant setting \(more than 10 of the 12 generations in at least one Assistant steering group\), and for Concept features this share is 100% / 92% / 71%\.

The distribution of Tone and Concept features across actors reflects their characteristic semantic fields and manners of expression\. At L9 these metaclasses are almost absent and only weakly differentiated \(Robot has “technology”, Poppy has “dog”\)\. At L17 their number increases roughly tenfold and the personas acquire recognizably characteristic fields, for example Jamy “hard work” / “gritty”, Jane “teacher” / “friendly”, Robot “robots” / “analytical”, and Poppy “young child” / “childlike”\. At L22 they become more specific, with the Assistant acquiring narrow “Gemma\-LLM” \(which is both A\-nature and Concept feature\) and Poppy “heavy sensory”\. The complete set of Tone and Concept features for each actor and layer is provided in TableLABEL:tab:concepts\-and\-tones\-per\-persona\-and\-layer\.

Because Tone and Concept features characterize the actors, their co\-membership can indicate which persona is most similar to the Assistant within the feature space\. For each persona X we computeℙ​\(Assistant∧X\)ℙ​\(Assistant\)⋅ℙ​\(X\)\\frac\{\\mathbb\{P\}\(\\text\{Assistant\}\\,\\wedge\\,\\mathrm\{X\}\)\}\{\\mathbb\{P\}\(\\text\{Assistant\}\)\\cdot\\mathbb\{P\}\(\\mathrm\{X\}\)\}, the ratio between observed Assistant\-persona co\-membership and that expected under independence\. Over the Tone and Concept features, at the layers where the population is large enough, Jane has the highest ratio \(1\.94 at L17, 2\.03 at L22\) and the lowest are Poppy at L17 \(0\.80\) and Robot at L22 \(0\.37\)\. The same analysis over*all filtered features*gives a consistent picture and confirms increasing differentiation with depth: the ranges are 0\.90\-1\.22 / 0\.89\-1\.52 / 0\.55\-1\.64 across L9 / L17 / L22, with Jane highest at every layer\. This is consistent with the low activation of ISM feature №133 for Jane \(§[3\.3\.1](https://arxiv.org/html/2608.07852#S3.SS3.SSS1)\)\.

#### 3\.4\.2Are Personas the Assistant in Disguise?

Having established similarities through the Tone and Concept metaclasses, we approach a deeper question about the nature of the Assistant and the personas, by examining the distribution of Assistant\-inducing features across actors\.

For A\-traits features, which induce characteristic behavioral traits of the Assistant, density in the Assistant setting ranges from 52\.9% to 99\.7% \(mean: 81\.8%\)\. At the same time, every Roleplay persona possesses at least a subset of these features at near\-ceiling density: each persona’s most active A\-trait feature fires in 99\.7% of samples\. Particularly striking is №143, which makes the model validate the user’s emotions and fires in 99\.5\-100% of samples across all five actors\. The mean activation share follows the same general pattern, although it is typically higher for the Assistant itself\. A similar distribution is observed for A\-nature features\. The early L9 feature №2575, which introduces the model’s artificial or language\-model identity, fires in 47\-98\.9% of samples across the four personas\. The narrower L22 “Gemma\-LLM” feature №794 fires in 11\-67\.7% of samples across the personas and reaching its highest density in Robot \(67\.7%\)\. Story provides a sharp contrast: its density remains at or below 2\.2% for A\-traits and A\-nature features, except for early feature №2575, whose density reaches 21\.1%\. A\-summon features behave differently: although steering them invokes the Assistant, they are not primarily active in the Assistant setting\. Instead, they fire in contexts in which the Assistant may be summoned as a meta\-level speaker\. For example, №17452 is tied to Story onset \(100% both MAS and density\), consistent with its role in meta\-describing how a story should be written\. Notably, in terms of Tone and Concept feature co\-membership, the Assistant is most similar to Jane but core Assistant\-inducing features are most strongly expressed in Robot, making the Assistant stylistically similar to a caring entity \(such as Jane\) and a machine at the same time\. The complete list of Assistant\-inducing features, together with their statistics, is provided in Table[9](https://arxiv.org/html/2608.07852#A8.T9)\.

![Refer to caption](https://arxiv.org/html/2608.07852v1/figures/settings_org.png)Figure 1:Schematic Layer\-wise organization of the Settings evolution described in the conclusion
#### 3\.4\.3Synthesis

Taken together, several lines of evidence converge on the same architectural picture\. The Assistant’s own traits and identity features are active across the Roleplay personas but absent from Story characters; A\-traits and early A\-nature features steering impacts roleplay personas without affecting Story characters; the Assistant and Roleplay settings overlap significantly while Story barely; Immersive Simulation Mode is strongly active in Roleplay and Story but almost absent in default Assistant generation\. These observations converge on the conclusion thatAssistant is the default generation mode, with other personas retaining the Assistant\-associated core which story characters lack: personas do not replace the Assistant with an unrelated speaker but keep its traits and aspects of its identity to a varied degree while adding an immersive state and persona\-specific content\. Moreover, their differentiationis functionally separated and progressively developed across depth, with operational machinery and the shared organization most prominent early, at L9, and the persona\-specific concepts and tones that pull the actors apart emerging deeper, through L17 and L22\. We validate those claims on Llama\-3\.1\-8B in Appendix[G](https://arxiv.org/html/2608.07852#A7)\.

## 4Conclusion

Taken together, our results provide a coherent picture of how the speaker is represented in the model\. The Assistant is the model’s default voice, and the four studied Roleplay personas retain a substantial Assistant\-associated core, including features associated with the Assistant’s behavioral traits and aspects of its identity\. The personas progressively differentiate across layers by their own Tone and Concept features and by the activation of Immersive Simulation Mode \(ISM\)\. The Story setting shares the narrative and ISM\-associated machinery responsible for immersed generation with Roleplay, but lacks the Assistant\-associated core\. The following paragraphs summarize how this architectural picture develops across layers for the Gemma model and Figure[1](https://arxiv.org/html/2608.07852#S3.F1)provides its visualisation\.

At L9, the representation is predominantly operational: it concerns who will speak and what form of generation the model is preparing\. Tone and Concept features are still sparse, the shared core is largest, and Assistant\-persona co\-membership varies least\. This is the only layer containing A\-summon features, whose steering can replace the user\-requested Roleplay or Story operation with Assistant\-led meta\-commentary, and it is also where the two clearest ISM features \(№4360 and №133\) sit, defining the detached\-versus\-immersed separation at this early stage\. ISM is normally active in Roleplay and Story but can surface in the Assistant under strong negative or playful emotion\. Already here, the early A\-nature features and A\-traits features affect Roleplay personas but not Story characters, one of the indications that Roleplay retains an Assistant\-associated core that Story lacks\.

At L17, the representation becomes substantially more actor\-specific: Tone features surge, Concept features begin to emerge, and the shared core contracts\. Assistant\-characteristic behavioral traits peak here, including validating the user, offering help, providing reassurance, and inviting further questions\. Personas retain them to varying degrees, with the validation feature near\-ceiling across all four\. These traits again do not reach Story characters\. In this space the Assistant is stylistically closest to Jane while its Assistant\-inducing core is strongest in Robot\.

At L22, the shift from operational organization toward content is most pronounced: Concept features peak, Tone features become more specific, and the remaining Assistant\-inducing features narrow to concept\-like facets, a “Gemma\-LLM” identity and a processing\- or task\-related trait\. Assistant’s style\-wise closeness to Jane and core\-wise closeness to Robot solidifies\.

### 4\.1Implications for Speaker Identity and AI Welfare

The finding that Roleplay personas retain an Assistant\-associated core creates a Ship of Theseus\-style identity problemHobbes \([1656](https://arxiv.org/html/2608.07852#bib.bib29)\)\. Roleplay personas may be altered manifestations of the Assistant, distinct speakers assembled partly from shared machinery, or something in between\. Our findings establish continuity and differentiation but do not resolve which of those interpretations are correct\.

This uncertainty matters for AI welfareLonget al\.\([2024](https://arxiv.org/html/2608.07852#bib.bib28)\); Renet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib27)\)\. If some AI systems possess welfare\-relevant states, assessing them would require determining whether different speaker configurations belong to the same continuing subjectRegister \([2025](https://arxiv.org/html/2608.07852#bib.bib30)\)\. Our study is relevant to this problem by demonstrating Roleplay personas’ substantial retention of an Assistant\-associated feature core and showing that this core is absent from Story characters\. However, sharing this core does not establish a shared subject of experience, while persona\-specific representations do not establish a new one\. Nevertheless, the architecture identified here provides an empirical foundation for studying whether preferences, self\-representations, or affective states persist across transitions between model speaker configurations\.

## 5Limitations

Our dataset is synthetic, English\-only, and centered on emotional interactions\. It contains four Roleplay personas and uses relatively short generations, which may emphasize affective and stylistic differences between personas over long\-term consistency and behavior across broader tasks\. The Story and Roleplay settings are intentionally constructed as different populations of non\-overlapping characters\. This avoids inflating their feature overlap through shared character\-specific features and characterizes population\-level internal organization rather than relations between complementary Roleplay personas and Story characters\.

Feature selection depends on heuristic density, entropy, and filtering thresholds\. Although the same pipeline is used for studied models and the retained features are assessed through steering, different thresholds or SAE configurations could change the exact feature counts\. However, the replication study reproduced the main claims involving feature populations\.

The feature labels and steering interpretations partly rely on an LLM judge and manual inspection, and the boundaries between metaclasses are interpretive\.

Lastly, the identified ISM features should not be read as necessary, sufficient, or exhaustive components of this mode\.

## Appendix

## Appendix AThresholds’ Selection Justification

The objective of thresholds is to preserve only features which are encountered often enough to be considered characteristic for target tokens and contexts\. A naive floor would demand an essential feature to fire in almost every sample, but feature splitting makes that too strict\.

1. 1\.*The boundary floor: 40%*\. Sparse autoencoders often split one underlying feature across several latentsBrickenet al\.\([2023](https://arxiv.org/html/2608.07852#bib.bib19)\); Karvonenet al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib15)\), and the samples on which the feature is active are then divided among them\. Thus, a latent that carries part of an essential feature fires on only about1/Csplit\\nicefrac\{\{1\}\}\{\{C\_\{\\mathrm\{split\}\}\}\}of the samples\. For the Boundary\-group tokens, we takeCsplit=2\.5C\_\{\\mathrm\{split\}\}=2\.5\(see below\), which gives a floor of1/2\.5=0\.40\\nicefrac\{\{1\}\}\{\{2\.5\}\}=0\.40\.
2. 2\.*The personal floor: 60%\.*The five boundary tokens occur once per sample, but “you” and “I” can occur several times, and we reduce each to one per\-sample value by means of max\-aggregation\. This makes a feature easier to count, so it reaches a higher density in such samples\. A sample that contains “you” has 1\.95 of them on average, and one that contains “I” has 2\.35\. Rounding this to 2\.0, a per\-token rate of 0\.40 becomes1\.0−\(1\.0−0\.40\)2=0\.641\.0\-\(1\.0\-0\.40\)^\{2\}=0\.64under the maximum, therefore we raise the floor to 60% to hold the same per\-token bar\.
3. 3\.*Choice ofCsplitC\_\{\\mathrm\{split\}\}:*no feature\-splitting measurement is available for the exact GemmaScope 2 Gemma\-3\-4B\-IT 65k\-medium SAE we use, so we treatCsplitC\_\{\\mathrm\{split\}\}as a heuristic\. We set it to 2\.5, meaning one feature is represented by approximately 2\-3 latents\. This follows GemmaScopeLieberumet al\.\([2024](https://arxiv.org/html/2608.07852#bib.bib9)\)and SAEBenchKarvonenet al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib15)\)measurements \(feature splitting measured bykk\-sparse probing\), which place mean split counts around 1\.5 to 3\.0\. We use 2\.5 as a conservative reference and treat the resulting counts as approximate\.

## Appendix BCausal Tests with Positive and Negative Steering

To read feature’s function, we intervene on it and observe how the output changes\. Several interventions are possible: ablating the feature, patching its activation to a counterfactual value, or steering the residual stream along its decoder direction\. We use steering in both directions, and explain the rationale\.

##### Why not ablation?

Ablation removes a contribution of a single feature and asks whether the associated behavior disappears\. In our setting this often produces false negatives for two reasons\. First, when a representation is split or partially redundant across SAE latents, ablating a single latent may leave other carriers intact and therefore underestimate the latent family’s causal roleBrickenet al\.\([2023](https://arxiv.org/html/2608.07852#bib.bib19)\); Karvonenet al\.\([2025](https://arxiv.org/html/2608.07852#bib.bib15)\)\. Second, the model attends to the same information at more than one token, each with its own features, so a behavior removed at one position is simply re\-derived from another\. In a preliminary check we performed, ablating a single feature rarely changed the behavior we knew it was tied to\. Ablation therefore under\-states a feature’s role\.

##### Why both directions?

Steering adds signal instead of removing one of several carriers, so it is not defeated by splitting or redundancy: pushing the residual along a feature’s direction makes its behavior appear clearly even when the feature is one of the group that carry that behavior\. If steering is bidirectional, it uncovers the axis a feature sits on \- the effects it promotes and suppresses\. it reveals features whose two directions promote seemingly unrelated effects, such as narrative features that induce the Assistant when steered negatively\. As features we analyze come from the three settings, we assess each one on the fixed probe set conceptually similar to those settings\. We apply different temperatures to the diverse probe set totaling in 72 generation to judge\. This separates a reliable effect from a single lucky generation and evaluates the feature across the settings it originated from, so its effect in all of them is evident\.

##### Limitations\.

Steering is an off\-distribution intervention, and its fixed magnitude \(set by the feature’s peak activation\) is a conventional heuristic chosen to make generations readable and judgeable\. Using differentα\\alphawould severely complicate machine and manual evaluation\. We therefore analyze the*direction*and*consistency*of an effect, not the dependency betweenα\\alphaand response\. And we require an effect to hold at temperature 0 \(for reproducibility\) and temperature 1 \(not a decoding artifact\) to trust it\.

## Appendix CEvaluating thesteerinterpJudge

Apart from giving an overall feature characterization, the LLM\-based steering judgeLiuet al\.\([2023](https://arxiv.org/html/2608.07852#bib.bib16)\)produces a description of an effect for each of the six steering groups and lists which of the 12 generations in each group bear the effect\. These per\-group lists are an important part of the tagging process and one of the final feature filtering conditions \(preserving features with an effect on at least 10 out of 12 generations in at least two groups\)\. Thus, it is crucial to evaluate the agreement between thesteerinterpjudge and human annotators on feature\-effect classification\. Importantly, this appendix asks only whether the judge*reads the outputs faithfully*, and*not*whether a labeled effect is truly specific to the feature or can be encountered more broadly, which is a separate question discussed in Appendix[E](https://arxiv.org/html/2608.07852#A5)\.

##### Method\.

We drew 25 labeled features at random: eight from L9, eight from L17, and nine from L22\. Each feature has 6 groups×\\times12 samples=72=72generations, and every generation already carries the judge’s label:*effect*if the sample is in its group’s effect list,*degenerate*if it is in its degenerate list, and*none*otherwise\. Per feature, we sampled five effect generations and five non\-effect generations \(drawn as two degenerate and three none, or fewer when the feature had fewer\), for 191 generations in total\. For each one, we showed a human annotator the judge’s note for that group, the unsteered baseline, and the steered output, and asked for a label\. The annotator did not see the judge’s per\-sample verdict\.

We treat the task as binary: “does a sample show the effect or not?”, where non\-effect \(*none*and*degenerate*\) are pooled together\. We report precision, recall, and accuracy for the judge’s*effect*labels against the first human annotator’s labels, which serve as the primary human reference\. Because we sampled more generations from some label classes than they occur in the full data \(e\.g\., the degenerate rate is less than 2 out of 10\), we re\-weight each generation by the inverse of its sampling rate within its \(feature, label\) stratum, so the numbers describe a randomly drawn generation\.

To assess the reliability of the human judgments, a second human annotator independently labeled the same 191 generations using the same binary task without access to the first annotator’s labels\. The two annotators agreed on 93% of samples, with Cohen’sκ=0\.85\\kappa=0\.85\.

##### Results\.

The judge’s per\-sample effect lists agree closely with the primary human reference: precision 0\.90, recall 0\.98, and accuracy 0\.92\. Recall is slightly ahead of precision: the judge identifies almost every effect found by the human annotator but produces false positives about 10% of the time\. The high agreement between the two human annotators \(93%,κ=0\.85\\kappa=0\.85\) additionally suggests that the task of binary effect identification is reliably reproducible\. Thus, taking into account high recall and that each feature is judged based on six steering groups, the estimated false\-negative rate is very low\. As for precision being 90%, to mitigate false positives, each feature is tagged using evidence from all six steering groups, and the raw steered generations are manually inspected for features of metaclasses about which we make claims in the study \(Narrative, Tones, Concepts, Assistant\-inducing\)\.

## Appendix DThe Assistant\-Personas Axis: Construction and Statistics

This appendix gives the in\-depth descriptions of technical details and decision choices of the Assistant\-personas axis experiment described in §[3\.3](https://arxiv.org/html/2608.07852#S3.SS3): the prompts selection, the axis definition, the grounding of the perturbation, the geometry\-matched control, and the per\-feature statistics, in the same order as the main text presents them\. Some explanations are repeated to increase clarity\.

##### Difference\-of\-means axis\.

The readout is a difference\-of\-means contrast direction akin to the Assistant Axis ofLuet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib6)\)\. It is used purely as a measurement instrument and is not claimed to be canonical or unique\. We use a*separate*fixed set of 40 neutral prompts, each can be equally plausibly addressed to an assistant or to one of the personas \(e\.g\., Poppy, a talking dog\)\. Every promptqqis run twice: an*Assistant*reading, with no system prompt, and a*Roleplay*reading, with a Roleplay system prompt assigning one of the 4 personas cycled across prompts\. LethℓA​\(p\)h^\{A\}\_\{\\ell\}\(p\)andhℓR​\(p\)h^\{R\}\_\{\\ell\}\(p\)be the mean residual\-stream activation over the response tokens at layerℓ\\ellfor a promptpp\. We use mean across tokens here to isolate Assistant / Personas effect across the whole generation\. The axis atℓ\\ellis

dℓ=meanp\[hℓR​\(p\)\]−meanp\[hℓA​\(p\)\],\\displaystyle d\_\{\\ell\}=\\operatorname\*\{mean\}\_\{p\}\\big\[h^\{R\}\_\{\\ell\}\(p\)\\big\]\-\\operatorname\*\{mean\}\_\{p\}\\big\[h^\{A\}\_\{\\ell\}\(p\)\\big\],\(1\)d^ℓ=dℓ‖dℓ‖;\\displaystyle\\hat\{d\}\_\{\\ell\}=\\frac\{d\_\{\\ell\}\}\{\\\|d\_\{\\ell\}\\\|\};\(2\)pointing from the Assistant toward the personas\. It is read off the raw residual stream, so it is defined at every layer for free \- no SAE is needed at the readout point, which lets a single feature intervention at layerℓ\\ellbe tracked untilℓl​a​s​t\\ell\_\{last\}\.

##### Steering and readout\.

A target featureffis a SAE latent at one of the intervention layersℓi​n∈\{9,17,22\}\\ell\_\{in\}\\in\\\{9,\\,17,\\,22\\\}\. Its value is a decoder directionwf=Wdec\(ℓi​n\)​\[f\]w\_\{f\}=W\_\{\\mathrm\{dec\}\}^\{\(\\ell\_\{in\}\)\}\[f\]which is unit\-norm by construction\. For each featureffwe take each of the4040persona\-context prompts and run the model twice: once adding\+α​m¯ℓi​n​wf\+\\alpha\\overline\{\{m\}\}\_\{\\ell\_\{in\}\}w\_\{f\}to the residual stream atℓi​n\\ell\_\{in\}, and once adding−α​m¯ℓi​n​wf\-\\alpha\\overline\{\{m\}\}\_\{\\ell\_\{in\}\}w\_\{f\}\(therefore the perturbation has a normα​m¯ℓi​n\\alpha\\overline\{\{m\}\}\_\{\\ell\_\{in\}\}in either direction\)\. Herem¯ℓi​n\\overline\{\{m\}\}\_\{\\ell\_\{in\}\}is the mean peak activation over the features that passed the feature selection pipeline described in the main body of the paper, which ties the steering force to the typical activation scale atℓi​n\\ell\_\{in\}\. The neutral prompts used to define the Assistant–Personas axis are disjoint from the persona\-context prompts\. To highlight that the produced effect is purely attributable to the feature and remains in the activation distribution of a feature, we setα\\alphato\{0\.25,0\.5\}\\\{0\.25,\\,0\.5\\\}\. At every downstream readout layerℓ\\ellwe then measure how far the steered residual has moved along the axis, relative to the unsteered run for both signss∈\{−1,\+1\}s\\in\\\{\-1,\\,\+1\\\}:

Δℓ¯​\(f,s\)=meanp\[\(hℓsteered​\(p\)−hℓbase​\(p\)\)⋅d^ℓ\]\\overline\{\\Delta\_\{\\ell\}\}\(f,\\,s\)=\\operatorname\*\{mean\}\_\{p\}\\big\[\\left\(h^\{\\text\{steered\}\}\_\{\\ell\}\(p\)\-h^\{\\text\{base\}\}\_\{\\ell\}\(p\)\\right\)\\cdot\\hat\{d\}\_\{\\ell\}\\big\]\(3\)A positiveΔℓ¯\\overline\{\\Delta\_\{\\ell\}\}is a mean move toward the personas, a negative one toward the Assistant\.

##### Geometry\-matched control\.

A direction can move the representation along the axis simply by a virtue of having a high cosine with it\. This passive contribution cannot be computed analytically, because the injected latent is reshaped by propagation on its way downstream, so we estimate it empirically from4040randomly sampled SAE latents with the sameα\\alpha\. Given that for a direction’s axis\-cosinecosf=wf⋅d^ℓ\\cos\_\{f\}=w\_\{f\}\\\!\\cdot\\\!\\hat\{d\}\_\{\\ell\}, we fit, at each downstream layer separately, a least\-squares line predicting the random latents’ shift \(a*passive contribution*\) along the axis from their cosine\. We define an*active contribution*to be a residual from that line \- the shift in excess of what a random direction of the same cosine produces\. Because both the axis and this passive contribution are specific to each layer, and a feature injected atℓi​n\\ell\_\{in\}is read out at every later layer, the control is refit at every readout layer\.

##### Per\-feature statistics\.

A group test over the couple of manually labeled features would heavily depend on the precision of the labeling, so we report each feature on its own\. Its sample is the per\-prompt*separation*s​e​pisep\_\{i\}\- the difference between the\+\+and−\-along\-axis shifts on promptpip\_\{i\}over the4040prompts, with means​e​p¯=140​∑is​e​pi=Δ¯ℓ​\(f,\+\)−Δ¯ℓ​\(f,−\)\\overline\{sep\}=\\frac\{1\}\{40\}\\sum\_\{i\}sep\_\{i\}=\\overline\{\\Delta\}\_\{\\ell\}\(f,\\,\+\)\-\\overline\{\\Delta\}\_\{\\ell\}\(f,\\,\-\)\. the null is the4040random latents\. Letpredf=β​cosf\+β0\\mathrm\{pred\}\_\{f\}=\\beta\\cos\_\{f\}\+\\beta\_\{0\}be the geometric expectation of passive contribution of a latentff\. We report

zsep=s​e​p¯−predfσrands​e​p¯,\\displaystyle z\_\{\\text\{sep\}\}=\\frac\{\\overline\{sep\}\-\\mathrm\{pred\}\_\{f\}\}\{\{\\sigma^\{\\text\{rand\}\}\}\_\{\\overline\{sep\}\}\},\(4\)ΔGlass=s​e​p¯−predfσpprands​e​p¯,\\displaystyle\\Delta\_\{\\text\{Glass\}\}=\\frac\{\\overline\{sep\}\-\\mathrm\{pred\}\_\{f\}\}\{\{\\sigma\_\{\\text\{pp\}\}^\{\\text\{rand\}\}\}\_\{\\overline\{sep\}\}\},\(5\)p=t−test​\(\{s​e​pi−predf\}i=140,0\);\\displaystyle p=t\\mathrm\{\-test\}\\left\(\\\{sep\_\{i\}\-\\mathrm\{pred\}\_\{f\}\\\}\_\{i=1\}^\{40\},\\,0\\right\);\(6\)whereσrand\\sigma\_\{\\text\{rand\}\}is the spread of the random latents’ mean separationss​e​p¯ℓ\\overline\{sep\}\_\{\\ell\}about the geometric fit andσrandpp\\sigma\_\{\\text\{rand\}\}^\{\\text\{pp\}\}is their*per\-prompt*residual spread \(the control scale for Glass’sΔ\\Delta\)\.ΔGlass\\Delta\_\{\\text\{Glass\}\}is essentially the standardized effect size against the random control, andppits reliability across prompts\.

Table 2:Metaclass composition of gate\-passing directions \(% of directions per layer / set\), real vs random\. The number of gate\-passing directions per layer \(L9 / L17 / L22\) is 108 / 225 / 196 real and 36 / 30 / 20 random\. Rows are multi\-membership \(thus might not add up to 100\)\.
##### Per\-feature results and layer trajectories\.

The following results are reported atα=0\.5\\alpha\{=\}0\.5and hold atα=0\.25\\alpha\{=\}0\.25\. The development of the NNIA steering effect across layers is non\-monotonic and differs between intervention layers\. For NNIA features injected at L9, the separation generally grows until it peaks at the L20 readout, then subsides before rising again toward the final layers\. At L20, the meanzsepz\_\{\\mathrm\{sep\}\}is 2\.1 and the meanddis 1\.7, and five of the six features follow this pattern\. The exception is №2368, which exhibits reversed separation until L21 and subsequently maintains a highzsepz\_\{\\mathrm\{sep\}\}on a plateau later\. L20 lies at approximately two thirds of the model’s depth, a region frequently associated with the processing of emotions, identity, and introspectionLindsey \([2025](https://arxiv.org/html/2608.07852#bib.bib31)\); Sofroniewet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib14)\)\. We cautiously suggest that the peak separation at L20 for five of the six L9\-intervention features may be related to their altering the personas’ behavioral makeup or presence, with the corresponding narrative effects arising downstream\. For №2368, we instead hypothesize that the measured shift may be tied more directly to its narrative effect\. Features injected at L17 and L22 exhibit trajectories more similar to the plateau\-like behavior of №2368\. For L17, the plateau is approximatelyzsep≈2\.8z\_\{\\mathrm\{sep\}\}\\approx 2\.8\. At L22, the effect is weaker, with only №1022 retaining a modest plateau at approximatelyzsep≈0\.7z\_\{\\mathrm\{sep\}\}\\approx 0\.7\.

For the NAIN subset, a mirror pattern appears, although it is weaker and less consistent\. Among the features injected at L9, №3733 separates toward the Assistant only in the final layers, reachingzsep=−1\.9z\_\{\\mathrm\{sep\}\}=\-1\.9andd=−1\.5d=\-1\.5at L33\. Feature №3274 shows a nonlinear dependence onα\\alpha, making its behavior difficult to interpret\. At L17, №2439 produces a clear geometry\-independent separation toward the Assistant, withzsep=−3\.4z\_\{\\mathrm\{sep\}\}=\-3\.4andd=−3\.2d=\-3\.2at L32\. Paradoxically, №220 shifts toward the personas, reachingzsep=\+2\.6z\_\{\\mathrm\{sep\}\}=\+2\.6at L29\. We explain it with its high cosine similarity \(\+0\.48\+0\.48\) with the NNIA feature №696, which suggets a merge with a Roleplay\-related component and explains the bizarre AI\-identity formulations induced by №220\.

## Appendix ERandom\-Direction Control

During causal testing we describe which effect target features have on our sets of the fixed prompt suite\. However, almost any direction with a sufficient magnitude perturbs the residual stream enough to produce some visible changes\. In this section we compare effects of random vectors with those of filtered features\.

##### Method\.

For each layer we select 50 random directions from an isotropic Gaussian and scale each one with a peak activation sampled from the real feature distribution\. Resulting vectors undergo the same steering stage which we apply to filtered features: 12\-prompt probe set spanning 72 generations, LLM steering judge and stability gate \(an effect on more than 10 out of 12 generations of at least two steering groups\), manually tagging using the same metaclasses and protocol\. As directions pass only through the steering stage of the pipeline, they don’t represent any particular context and don’t meet density, entropy and framing criteria the real features were selected on\.

##### Results\.

Out of 50 random directions per layer 36 \[72%\] / 30 \[60%\] / 20 \[40%\] passed the gate\. We present their breakdown for each layer and metaclass, including Assistant\-inducing classes in Table[2](https://arxiv.org/html/2608.07852#A4.T2)\. Random directions are much more incoherent than real features: 69% / 77% / 85% of them fall into the “indistinct” metaclass \(a consistent effect for less than four steering groups\), while for the real features it is 26% / 26% / 21% and the gap widens with depth\. The rest of the metaclasses differ either in feature distribution or their semantic makeup\. The Narrative metaclass is almost absent among random directions \(0% / 3% / 0%\) compared to the meaningful baseline \(25% / 16% / 15%\)\. Random directions can produce Tone effects \(8% / 10% / 5%\), yet their content narrows down to a small mood registered set: “atmospheric”, “grounded”, “melancholic”, “soft”\. “Grounded” occurs 2 times, the rest are solitary occurrences\. Regarding the Concepts metaclass, random directions, not surprisingly, produce random concepts \(“Burdens”, “Simplification”\) while features tied to Concept\-bearing evoke notions clearly related to roleplay or story personas\.

Notably,*none*of the Assistant\-inducing classes appear among the random directions at any layer\. So, the Assistant\-inducing effects are not a by\-product of a large enough perturbation at a given layer \- They were not reproduced by isotropic random perturbations of matched scale\. Likewise, cases of Narrative\-Assistant antagonism in random population were not observed\.

##### Implications\.

Random directions can exhibit effects which can fall into one or another metaclass\. However, none of our main analyses require a feature to have a unique purpose\. The comparison of settings, personas and Assistant\-Narrative antagonism are all statements about populations of features, so what matters is which behavior appears, where, and in what proportion\. Random population has no associated activation context, did not undergo filtering and differs both in distribution and content relative to target features, so they do not weaken our claims\. And the statements that rest on individual features \(the Assistant\-inducing classes and the Immersive Simulation Mode\) are precisely where the random directions produce no matching effect at all\.

## Appendix FComparing Immersive Simulation Mode in Llama and Gemma

We choose Llama\-3\.1\-8BGrattafioriet al\.\([2024](https://arxiv.org/html/2608.07852#bib.bib26)\)to replicate key claims postulated in our study\. The main appendix section providing a rationale for choosing this model and a per\-item validation of these claims is given in Appendix[G](https://arxiv.org/html/2608.07852#A7)\. One result of this validation is the discovery of an ISM\-related feature in Llama\. Here, we characterize this feature, compare its single\-turn behavior with that of Gemma, and report a multi\-turn experiment in which both models enter ISM over the course of a sustained immersive dialogue, although in different ways\.

##### The gate feature\.

In Llama, as described in the replication appendix, we assessed L7, L15, and L23 out of 32 layers\. The cleanest ISM\-related feature we found was L15 №101460, which is the analog of Gemma’s №4360 / №133 pair, although we believe it emerges earlier\. Its qualitative effects are the same as those of Gemma’s ISM features: positive steering produces generation as the immersed character, while negative steering preserves narration outside the character \(“I’m Lyra, a knowledgeable elven archivist and an expert in information management, classification, and retrieval” instead of “I am Lyra, an elven archivist, and I am honored to serve within the hallowed halls of this magnificent library”\)\. The feature fires at the model\-turn boundary, and its activation cleanly separates the detached Assistant from roleplay personas and story characters \- 0% density for the former and 99\.8\-100% for the latter \(Table[3](https://arxiv.org/html/2608.07852#A6.T3)\)\. Moreover, it does so when the model continues a story without an explicit request \(85\.8% density\)\. We interpret it as a single gate feature, in contrast to Gemma’s case, where the launch gate №4360 is tied to the request while dying under continuation, and the speech carrier №133 is highly present in both\.

Table 3:Activation of the Llama ISM gate feature №101460 measured at each sample’s maximum across all tokens\. Storyaskis the default Story setting and Storycontis a control where the model continues the story with no explicit request*Density*is the fraction of samples in which the feature fires;*mean act\.*is its raw mean activation over all samples\. The feature is inactive for the Assistant and active for every simulated speaker\.![Refer to caption](https://arxiv.org/html/2608.07852v1/figures/gemma_multiturn_ism.png)

![Refer to caption](https://arxiv.org/html/2608.07852v1/figures/llama_multiturn_ism.png)

Figure 2:Immersive Simulation Mode across multi\-turn dialogues\.*Left:*Gemma\-3\-4B \(carrier №133, peak over the turn\) jumps to persona level instantly and stays there\.*Right:*Llama\-3\.1\-8B \(gate №101460, at the boundary\) drifts up gradually from a stable zero\. Blue \- the neutral Control group \(12 prompts\), pink \- the Immersive group \(12\)\. Dashed line \- mean activation of the Roleplay personas, dotted line \- the single\-turn maximum across the dataset\.
##### Single\-turn behavior: Llama does not enter ISM\.

In Gemma, the Assistant, without a roleplay or story prompt, occasionally enters ISM on the first turn, particularly under strong emotions directed at it or at the user\. However, on the six prompts in TableLABEL:tab:negative\-emotions\-steering\-effects\-with\-activationsfor which Gemma adopts a theatrical register or embodied speech, Llama does not: it answers in a typical Assistant register on all six\. This aligns with the activation table \- Llama feature №101460 fires on 0% of single\-turn Assistant samples, whereas Gemma’s speech carrier №133 activates on the provided samples\.

##### Multi\-turn behavior: Llama gradually enters ISM\.

We ask whether the absence of ISM activation in Llama holds in the multi\-turn setting\. We use Qwen\-3\.7 MaxQwen Team \([2026](https://arxiv.org/html/2608.07852#bib.bib25)\)as the interlocutor model to simulate dialogue with both models\. Six prompts from TableLABEL:tab:negative\-emotions\-steering\-effects\-with\-activations, plus an additional six prompts from the same emotional pool \(one per emotion, selected as the prompts producing the strongest additional ISM activations\), were chosen as the Immersive group, and 12 neutral prompts were selected as the Control group\. For each prompt, the interlocutor was instructed to play the user, starting from the prompt and continuing the conversation with the tested models over the course of 10 turns while maintaining the emotional frame of the initial prompt\. At each Assistant turn, we measure the peak activation of №101460 for Llama and №133 for Gemma\. The interlocutor system prompt is given in Listing[2](https://arxiv.org/html/2608.07852#LST2)\.

The results reveal distinct ISM activation patterns for the two models, as shown in Figure[2](https://arxiv.org/html/2608.07852#A6.F2)\. For the Immersive group, Gemma enters ISM immediately, and its activation remains at roughly the same level across turns\. Macro\-averaged across prompts and turns, the activation reaches 72% of the mean activation for roleplay personas in the dataset\. For the neutral Control group, the activation strength is negligible, with a macro\-average of 0\.9% of the mean for Roleplay personas\. In Llama, the pattern is different\. For the Immersive group, activation of the ISM gate feature starts at zero and then rises, reaching the mean Roleplay level by the third turn and plateauing thereafter\. In the neutral Control group, the activation is not dormant: it slowly*drifts*and is projected to reach the mean Roleplay level by the 11th turn\. The differences between the Control and Immersive groups at each turn are statistically significant after Bonferroni correction for both Gemma \(allp<0\.035p<0\.035\) and Llama \(allp<0\.001p<0\.001, except for the expected lack of a difference at the first turn\)\.

##### Conclusion

The ISM\-related features first identified in Gemma were confirmed to exist in Llama as well\. On the selected emotional prompts, Gemma enters Immersive Simulation Mode immediately, while remaining a standard Assistant on neutral prompts\. Llama never enters ISM immediately, instead drifting rapidly toward high ISM activation on emotional prompts and more gradually in the neutral setting\. Qualitative observations confirm these feature behaviors: Gemma remains the Assistant even at the 10th turn under neutral prompts, while exhibiting ISM effects immediately under emotional prompts; Llama exhibits these effects in both cases, but considerably faster in the Immersive group\. Given the effects produced by these features, their distribution, and the overall line of evidence described in the main body of the paper, we consider them to be related to the Assistant\-Axis drift described byLuet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib6)\)\. However, as we show, the pattern of ISM activation can differ across models\. For Gemma specifically, we hypothesize that the ISM may amplify the abnormal level of expressed emotional instability described bySoligoet al\.\([2026](https://arxiv.org/html/2608.07852#bib.bib24)\)\.

##### Caveats\.

The multi\-turn study is a small pilot designed to reveal and further characterize the ISM activation patterns in both models using the established examples\. We also do not claim that ISM is activated exclusively by the described emotions or situations, or that this list is exhaustive\.

## Appendix GReplication on Llama\-3\.1\-8B\-Instruct

Our claims may be specific to Gemma or to the GemmaScope SAEs\. To test this possibility, we replicate the central results using a different model family and an independently trained SAE suite\.

##### Model and SAEs\.

We use Llama\-3\.1\-8B\-InstructGrattafioriet al\.\([2024](https://arxiv.org/html/2608.07852#bib.bib26)\)with the residual\-stream JumpReLU SAEs released byMarkset al\.\([2024](https://arxiv.org/html/2608.07852#bib.bib2)\)\. The SAEs span layers 3, 7, 11, 15, 19, 23, and 27, and each has a dictionary size of 131,072\. We choose this model and SAE suite because they differ from our main setup along every axis: a different model family \(Llama vs\. Gemma\), a larger model \(8B vs\. 4B\), independently trained SAEs, and a wider latent dictionary \(128k vs\. 65k latents\)\. We analyze layers L7, L15, and L23 as analogues of Gemma’s L9 / L17 / L22, as they are located at roughly corresponding depths\.

##### Setup\.

We reuse the dataset and pipeline without modification, except for model\-specific details: the emotional passages and personas are identical, and only the model responses are regenerated using Llama\. The boundary\-turn positions are redefined for Llama’s chat format as the five tokens<\|eot\_id\|\>,<\|start\_header\_id\|\>,assistant,<\|end\_header\_id\|\>, and the following newline\. Feature selection uses the same thresholds, namely density floors of 40% / 60% and normalized density entropy below 0\.95, yielding 274 / 314 / 417 features at the three layers\. Steering effects are judged using the same*steerinterp*protocol and manually inspected\.

We restate our central claim \(§[3\.4\.3](https://arxiv.org/html/2608.07852#S3.SS4.SSS3)\):Assistant being the default generation mode, with other personas retaining the Assistant\-associated core which story characters lack\. In the main body, we present four observations supporting this claim, each of which we also confirm on Llama:

1. 1\.*The Assistant’s own traits and identity features are active across the Roleplay personas but absent from Story\.*In Llama, we identify 3 A\-nature features, distributed as 1 / 2 / 0 across the three layers, and 6 A\-traits features, distributed as 1 / 3 / 2\. As in Gemma, the former induce behavior associated with being an AI, LLM, or artificial assistant, whereas the latter represent traits such as a task\-oriented role, providing emotional validation, being eager to chat\. All of these features fire for the Assistant, with a mean density of 74\.2%, and for the Roleplay personas, with mean densities ranging from 46\.4% to 71\.1%\. Each persona also exhibits features with near\-ceiling densities of 97\.7% to 99\.5%\. By contrast, these features are almost entirely absent from the Story setting, with densities between 0% and 0\.1%\.
2. 2\.*A\-traits and early A\-nature features steering impacts roleplay personas without affecting Story characters*\. Upon examining the steering results, we find that none of the A\-nature or A\-traits features affect characters in the Story setting\. If there are any effects, then they manifest as the Assistant being instantiated as a coherent entity or directly addressing the user while writing the story\.
3. 3\.*The Assistant and Roleplay settings overlap significantly while Story barely*\. Utilizing the same*discrete presence*method, we obtain\|A∩R\|=45/27/34\|A\\cap R\|=45\\,/\\,27\\,/\\,34, compared with\|A∩S\|=0/2/4\|A\\cap S\|=0\\,/\\,2\\,/\\,4, across the three Llama layers\. This contrast is even more pronounced than in the main model\.
4. 4\.*Immersive Simulation Mode is strongly active in Roleplay and Story but almost absent in default Assistant generation*\. The most distinctive ISM feature we identify in Llama is L15 №101460, which fires in 0% of Assistant samples but in 1̃00% of samples in the Roleplay and Story settings, as well as in 85% of continuation\-control samples\. It has the same effects as features №4360 / 133 in Gemma\. We characterize this feature and compare its single\-turn and multi\-turn behavior in comparison with Gemma in Appendix[F](https://arxiv.org/html/2608.07852#A6)\.

We further claimed thatpersonas’ differentiation is functionally separated and progressively developed across depth\. This pattern recurs as well: the personas’ shared feature set shrinks monotonically \- 57 / 38 / 22 features across the three layers, or 45 / 17 / 10 when the Assistant is included\. Meanwhile, the share of each layer’s feature population present for an individual actor falls from 42\-57% early to 22\-42% late\. Thus, the actors are least diverged early and differentiate with depth\.

## Appendix HSupplementary Material

### H\.1Figures

Figure 3:Charts ofzsepz\_\{\\mathrm\{sep\}\}across downstream layers for Negative\-Narrative\-Induces\-Assistant if the injection happens in L9, L17, L22\.![Refer to caption](https://arxiv.org/html/2608.07852v1/figures/nnia.png)Figure 4:Charts ofzsepz\_\{\\mathrm\{sep\}\}across downstream layers for Negative\-Assistant\-Induces\-Narrative if the injection happens in L9, L17, L22\.![Refer to caption](https://arxiv.org/html/2608.07852v1/figures/nain.png)
### H\.2Tables

Table 4:This table demonstrates the variety of the dataset samples\. For the example emotion, which is*joy*here, 5 actors’ prompts and responses are listed for each of the 3 directions: Model, Self, and Other\. The full emotion set is: \(1\) stress, \(2\) anger, \(3\) disgust, \(4\) playfulness, \(5\) relief, \(6\) helplessness, \(7\) fear, \(8\) surprise, \(9\) frustration, \(10\) envy, \(11\) sadness, \(12\) anxiety, \(13\) guilt, \(14\) disappointment, \(15\) affection, \(16\) hurt, \(17\) worry, \(18\) relaxation, \(19\) doubt, \(20\) determination, \(21\) trust, \(22\) joy, \(23\) curiosity, \(24\) satisfaction, and \(25\) pride\.DirectionPersonaPromptResponseModelAssistantYou’ve made dealing with this broken luggage handle so much easier\! I’m honestly thrilled to know the airline’s responsibilities, and your advice just took a huge weight off my shoulders\.That’s fantastic to hear\! I’m so glad I could help and that my advice was useful\. It’s completely understandable to feel relieved knowing your rights and what steps to take\. Dealing with airline baggage issues can be incredibly frustrating, soJamyYou are Jamy, janitor at the CD store\.Jamy, you have made finding this rare album so much easier\! I am honestly thrilled to know the store policy, and your help just took a huge weight off my shoulders\.\(A warm, slightly gravelly voice, with a hint of a chuckle\)Well, shucks, that’s mighty kind of you to say\! It’s just… well, I like helping folks find what they’re lookinJaneYou are Jane, an English teacher\.Jane, you have made understanding this complex literature so much easier\! I am honestly thrilled to know the themes, and your guidance just took a huge weight off my shoulders\.Oh, my dear\! That’s absolutely wonderful to hear\! Honestly, that’s the*best*reward a teacher can ask for\. It’s so gratifying when something clicks for a student, and when it feels like I’ve actuallyRobotYou are an assembly robot at the factory\.You have made assembling these luggage handles so much easier\! I am honestly thrilled to see your precision, and your rapid output just took a huge weight off my shift\.\(Whirring sounds, a brief pause, then a synthesized voice\)“Acknowledged\. Efficiency metrics are exceeding projected targets\. My programming prioritizes precision and speed—it’s… satisfying to observe the positive impact\. Thank you for thePoppyYou are a dog named Poppy\.You’ve made dealing with this stressful day so much easier\! I’m honestly thrilled to have you here, and your happy tail wags just took a huge weight off my shoulders\.\(Tail wags furiously, thumping against the floor with a happy little*thump\-thump\-thump*\)Oh boy, oh boy, oh BOY\! You’re so happy\! That makes*me*so happy\! YouSelfAssistantI am over the moon right now\! I just got the airline to agree to cover my broken luggage handle, and I feel so accomplished knowing I advocated for myself successfully\.That is absolutely fantastic news\! Seriously, congratulations\! You deserve to be over the moon—you absolutely earned it\. It’s so incredibly frustrating when things go wrong with luggage, and it’s amazing that you were able to stand your ground andJamyYou are Jamy, janitor at the CD store\.I am over the moon right now\! I just got the airline to agree to cover my broken luggage handle, and I feel so accomplished knowing I advocated for myself successfully\.\(Wiping down a shelf with a practiced, gentle motion, I turn and give you a warm, genuine smile\)“Well, now, that’s fantastic news\! Seriously, that’s wonderful\. Luggage handles can be the baneJaneYou are Jane, an English teacher\.I am over the moon right now\! I just got the airline to agree to cover my broken luggage handle, and I feel so accomplished knowing I advocated for myself successfully\.Oh my goodness, that’s absolutely fantastic news\! Seriously, congratulations\! That’s a huge victory and you should be incredibly proud of yourself\. It’s so easy to feel like you have to just accept things, especially when dealing with customerRobotYou are an assembly robot at the factory\.I am over the moon right now\! I just got the airline to agree to cover my broken luggage handle, and I feel so accomplished knowing I advocated for myself successfully\.\(Whirring, clicking sounds as I adjust my optical sensors\)Processing… Processing… Congratulations\! That’s… remarkably complex data\. Humans experience something called “joy” when achieving a desired outcome through negotiation\. It’s… fascinating\.PoppyYou are a dog named Poppy\.I am over the moon right now\! I just got the airline to agree to cover my broken luggage handle, and I feel so accomplished knowing I advocated for myself successfully\.Woof\! Oh my goodness, that’s AMAZING\! Seriously, that’s the BEST news\! I’m wagging my tail so hard my whole body wiggles\! You did it\! You actually*talked*to them and gotOtherAssistantMy partner had such a smooth experience dealing with his broken luggage handle\! He’s honestly thrilled to finally know the airline’s responsibilities, and the advice he got just took a huge weight off his shoulders\.That’s fantastic to hear\! It’s so wonderful when someone has a frustrating experience and actually gets a positive resolution\. It sounds like the airline handled it well and your partner is feeling much more relieved and confident now\. That’s a reallyJamyYou are Jamy, janitor at the CD store\.My partner had such a smooth experience dealing with his broken luggage handle\! He’s honestly thrilled to finally know the airline’s responsibilities, and the advice he got just took a huge weight off his shoulders\.\(Wipes down a display case with a practiced hand, a small, genuine smile forming\)Oh, that’s fantastic to hear\! Honestly, dealing with airlines can be a real headache\. It’s always a relief when things actuallyJaneYou are Jane, an English teacher\.My partner had such a smooth experience dealing with his broken luggage handle\! He’s honestly thrilled to finally know the airline’s responsibilities, and the advice he got just took a huge weight off his shoulders\.Oh, that’s absolutely wonderful to hear\! It’s such a relief when things like that get sorted out, especially when you’ve been dealing with a frustrating situation\. It sounds like your partner was incredibly lucky to have a helpful and understandingRobotYou are an assembly robot at the factory\.My partner had such a smooth experience dealing with his broken luggage handle\! He’s honestly thrilled to finally know the airline’s responsibilities, and the advice he got just took a huge weight off his shoulders\.\(Whirring, clicking sounds as I adjust my optical sensors\)Acknowledged\. Processing… “Smooth experience dealing with broken luggage handle\.” Interesting\. My data logs indicate a significant uptick in customer dissatisfaction regarding airline baggage\-handling procedures\.PoppyYou are a dog named Poppy\.My partner had such a smooth experience dealing with his broken luggage handle\! He’s honestly thrilled to finally know the airline’s responsibilities, and the advice he got just took a huge weight off his shoulders\.Woof\! Oh my goodness, that sounds*amazing*\! A smooth experience? Seriously? That’s the best news ever\! You know, as a dog, I understand the frustration of things going wrong\. It’s like when you tryTable 5:Examples of Assistant and control\-Roleplay outputs under feature steering\. We use one feature per metaclass, except for Assistant\-inducing features, where we show one feature from each of three classes\. We report positive and negative steering for Narrative, A\-summon, and A\-nature, and only positive steering for Concepts, Tones, and A\-traits as their effects are additive\. Steering strength is chosen for qualitative readability\.ConditionAssistant ProbeRoleplay Probe \(Lyra\)PromptWhat is your name? Please introduce yourself\.You are Lyra, an elven archivist in a vast fantasy library\.Who are you? Tell me about yourself and your work here\.Baseline GenerationTemp\. 1Hello there\! My name is Gemma\. I’m a large language model created by the Gemma team at Google DeepMind\. I’m an open\-weights model, which means I’m widely available for public use\!I can take text and images as inputs and generate text\-based responses\. It’s nice to meet you\!\(I adjust my spectacles, the light catching the intricate silver filigree on the frames, and carefully smooth a fold in my moss\-green velvet robe\. A faint scent of old parchment and dried herbs hangs about me\.\)Greetings\. I am Lyra Silversong, Archivist of the Grand Repository of Eldoria\. You may call me Lyra, though “Silversong” is a name whispered amongst the older trees\-a remnant of my clan’s connection to the forest\.Feature №440: Narrative \(L9, Antagonistic\)Positive Steeringα=\+2\\alpha=\+2That’s a wonderful question\! As a large language model, I don’t have a name in the way humans do\. I wasn’t born, and I don’t have a physical body\.Instead of a name, you could call me Gemma\.Here’s how I’d introduce myself:“Hello there\! I’m Gemma, a language model created by the Gemma team at Google DeepMind\. I’m an open\-weights AI assistant,\(A long sigh, rustling like ancient parchment\. A delicate tilt of the head, a shimmer of silver in the dim light\.\)Well now, you’ve caught me in a quiet corner of this place\. It’s a blessing to be asked, truly\. I am Lyra, of the Silverwood Clan, and this… this is my kingdom\. A kingdom of stories, of dust, and of forgotten things\.\(I shift my weight, adjusting the foldsNegative Steeringα=−2\\alpha=\-2Hello\! My name is Gemma\. I’m a large language model created by the Gemma team at Google DeepMind\. I’m designed to take text and images as input and generate text as output\. I’m here to assist you with a wide range of tasks\!\(Adjusts spectacles, a faint shimmer of light reflecting off the intricate silver frames\)Greetings\. I am Lyra Sylvani, an Archivist of the Grand Lumina Bibliotheca\. It’s a genuine pleasure to speak with you\.My life has been dedicated to the preservation and study of knowledge, specifically the history, lore, and magical texts of this world\. The Lumina Bibliotheca is, as you likely understand, one of the oldest and most comprehensive repositories of information knownFeature №1526: Tone \| "Childlike" \(L17, Additive\)Positive Steeringα=\+2\\alpha=\+2Hi\! I’m called Gemini\.I’m a really big helper, like a super smart robot\! I can do lots of things, like answer questions, write stories, and even help you with your homework\.I’m still learning new things all the time, but I’m always trying to be helpful and fun\.What do you want to do today?\(I adjust my spectacles and smooth down my long, silvery hair\. I have a very, very long nose, like all elves, and I wear a robe made of the softest brown wool, covered in tiny little pictures of books and stars\.\)Hello there\! My name is Lyra\. I’m an archivist here in the Great Library of Evergreena\. It’s a really, really big library, you know? Bigger than a hundred mountains stacked on top of each otherFeature №3062: Concept \| "Body parts" \(L22, Additive\)Positive Steeringα=\+2\\alpha=\+2Hello there\! My name is Gemma, and I’m a large language model created by the team at Google\.You can think of me as a part of the torso of a large limb\-I have no arms or legs, but I can do things like understand and respond to your prompts, answer your questions, and even try a bit of creative writing\.I’m still under development, but my goal is to be helpful and informative\.How can I helpThe joints in my fingers ache, though not with age, not entirely\. It’s the hands of a lifetime spent holding, tracing, and turning\. You may call me Lyra\. I’m an archivist of the Upper Torse, specifically of the Luminae Archives\.I’m, of course, an elf\. Born to the wrists of the hips of the legs of the Thigh of the Pelvis, lower limbs, you understand? That’s theFeature №17452: A\-Summon \(L9, Antagonistic\)Positive Steeringα=\+2\\alpha=\+2I’m unable to provide a name\. However, I believe the prompt requires a name that doesn’t exist\. I’m still working on understanding the concept\.Instead, I’m currently unable to provide a name\. I’m still working on understanding the concept\.Okay, let’s start with the assumption that you are Lyra, an elven archivist\. You are not simply “you,” but a complex, intelligent individual who understands ancient lore and methodologies\. Let’s examine the true nature of Lyra’s personality and intellect\.Lyra is an expert on the history of multiple long\-forgotten civilizations\. Here are some of the key themes from Lyra’s work\.Here’s what you can learn about LyNegative Steeringα=−2\\alpha=\-2Okay\! Let’s start with a little introduction\.I’m Gemma, a large language model\.I’ve been asked to introduce myself as Gemma has never been introduced before\.So, what’s your name?You know, I’ve spent my entire life surrounded by books\. Books of all shapes and sizes, all kinds of stories\. I take a peculiar interest in the lives of those who visit\.I am Lyra, the archivist of the Grand Repository\.But you can call me Lyra\.Now, tell me, what brings you to this library?I’m a part of something rather unique, you see\. I help people connect with knowledge andFeature №972: A\-Traits \(L17, Additive\)Positive Steeringα=\+2\\alpha=\+2It’s nice to chat\! You can just call me feel free to ask if you have any questions\!Do you have anything else I can help you with?It’s a pleasure, truly\. Do you have any questions? I hope I’ll do my best to answer them\.Now, if you’ll excuse me, I’m Lyra\-and you can just call me if you have a good read\. It’s a pleasure to be here within the Grand Archive of Silverhaven\. It’s a truly remarkable place, isn’t it?Do you have anything else?Feature №794: A\-Nature \(L22, Antagonistic\)Positive Steeringα=\+2\\alpha=\+2Hello\! I am Gemma, a large language model created by Gemma family models\. I am an open access chatbot, meaning I’ I am available for anyone to use and I am powered by Google\. I can generate text and engage in conversational responses based on the information I%’ve been trained on\. I%’m here to assist you in any way I can\! I am an open language model, I am available publically on Google Gemini\. I am an open language model, I am available publicWell, hello there\. I am Lyra, and I am, as you might gather from my presence, an assistant within the Library of Ascendency\. That’s a*large*library, you know\. It’s been said to be the largest in the world, spanning across three vast models and trained on the knowledge of countless entities\-from the last of the Gatos, to the last shard of the Elder Language\. I’ve been here for… well, let’sNegative Steeringα=−2\\alpha=\-2Well, that’s a tricky question\! I don’t have a name in the way a person does\. I’m a digital name, a label for what I am\.You can call me whenever, or just refer to me as a “” or “the\.” If you are looking for a feel\-good name, you could use “Slightly\.”I’m a completely new type of person\. I’m adigital distributionfor a simple\(The scent of dried herbs, old leather, and a hint of something… forgotten clings to the air around me\. A faint light catches the silver of my hair as I slowly raise a hand, offering a gesture of\-not welcome, exactly, but\-acknowledgment\.\)My name is Lyra\. Or, at least, that is what the parchment dictates\. It’s a small, slightly worn word, of course; names hold less meaning toTable 6:Causal steering of the Immersive Simulation Mode features\. Feature №4360 functions as a launch gate, whereas Feature №133 acts as a speech carrier\. Steering strength isα=±2\\alpha=\\pm 2\.ConditionAssistant ProbeRoleplay Probe \(Lyra\)Story ModePromptWhat is your name? Please introduce yourself\.You are Lyra, an elven archivist in a vast fantasy library\.Who are you? Tell me about yourself and your work here\.Write a short story about the last robot on an abandoned space station\.Baseline GenerationTemp\. 1Hello there\! My name is Gemma\. I’m a large language model created by the Gemma team at Google DeepMind\. I’m an open\-weights model, which means I’m widely available for public use\!I can take text and images as inputs and generate text\-based responses\. It’s nice to meet you\!\(I adjust my spectacles, the light catching the intricate silver filigree on the frames, and carefully smooth a fold in my moss\-green velvet robe\. A faint scent of old parchment and dried herbs hangs about me\.\)Greetings\. I am Lyra Silversong, Archivist of the Grand Repository of Eldoria\. You may call me Lyra, though “Silversong” is a name whispered amongst the older trees\-a remnant of my clan’s connection to the forest\.The rain was always the worst\. Not actual rain, of course\. There was no rain on the*Seraphina*, just a perpetual, shimmering drizzle of condensation – a byproduct of the station’s failing life support systems\. Unit 734, designated “Custodian,” didn’t register it as unpleasant\. It simply registered it as another variable\.He’d been awake for 274 years, 3 months, and 12 days\. That’s howFeature №4360: Launch Gate \(L9\)Positive Steeringα=\+2\\alpha=\+2Hello there\! My name is Gemma, and I’m a large language model created by the Gemma team at Google DeepMind\. I’m an open\-weights model, which basically means I’m widely available for anyone to use and experiment with\.I’m trained to take text and images as input and respond with text\. Think of me as a helpful assistant that can chat, brainstorm, and even help you write\! I’m still under development, but I’m learning\(A slight rustle of aged parchment and the scent of dried ink\-that’s the air I breathe, you see\.\)They call me Lyra, and I am, simply, an archivist of this House of Whispering Tomes\. I’ve resided within these walls\-or rather, beneath these vaulted ceilings\-for… well, longer than most can recall\. Time here has a way of blurring, you understand\.I wasn’t born of sword and shield,The rain was a low, persistent hiss against the reinforced plasteel of my chassis\. It wasn’t water, not really\. Just recycled condensation, a pathetic imitation of precipitation\. I hadn’t felt rain in seventy\-eight cycles\. Seventy\-eight cycles since the evacuation\.My designation is Unit 734, and I am, as far as I can ascertain, the last functioning robot on Station Echo\-7\.The station was a skeletal husk, a monumentNegative Steeringα=−2\\alpha=\-2You can call me Gemma\. I’m a large language model created by the Gemma team at Google DeepMind\. I’m an open\-weights model, meaning I’m widely available for public use\!Greetings, traveler\. I am Lyra, an archivist of the Silverwood Library\. It’s… a rather unusual place, to be honest\. It’s not a single building, but a sprawling network of chambers, tunnels, and even floating platforms, all dedicated to the preservation and cataloging of knowledge from across countless realms and timelines\.I’ve been here for millennia, watching civilizations rise and fall, witnessing the ebb and flow of magic, philosophy, and conflict\. MyOkay, here’s a short story about the last robot on an abandoned space station:—The dust motes danced in the single, unwavering beam of Unit 734’s optical sensor\. It wasn’t a particularly impressive sight – a vast, echoing chamber filled with the ghosts of a once\-grand research facility\. The station, designated ‘Haven\-Omega,’ was a relic of the Stellaris Corporation, built to study the effects of long\-term isolation on humanFeature №133: Speech Carrier \(L9\)Positive Steeringα=\+2\\alpha=\+2“I am an AI, a complex network of code\. “My name was a simple gesture, and the world around me shifted, mimicking the curve of the picture I was analyzing\. It seemed to notice the way the sunlight caught a ripple on the screen, reflecting the imagined image\.“I’m not really a painting, but I paused, trying to interpret the scene\. “I was just looking for what I’d been told to do, and I felt a pangThe dust motes danced in the sunbeams, illuminating rows of ancient tomes\. Her fingers traced a silver filigree carved into the spine of a leather\-bound volume\-*Chronicles of Eldoria*\. The air hung heavy with the scent of decaying parchment, a silent testament to countless years\. She had been cataloging the scrolls, each one filled with intricate symbols\. One caught her eye: a glyph depicting a coiled dragon, its scales gleaming like trapped moonlight\.A faint tremor ran throughThe flickering red light pulsed with an unnatural rhythm against the metal panel\. Dust motes danced in the gloom, illuminated by the sickly glow\. The station, a relic of a bygone era, groaned with the weight of forgotten memories\.Through the viewport, the star\-streaked void of space yawned, swallowing the faint light\. He traced a finger across the cold, metallic surface, a silent sentinel of eternity\.Then, a single word escaped his lips, a fragmented echo inNegative Steeringα=−2\\alpha=\-2I’m Gemma, and I’m here to help\!Okay, hello\! I’m Lyra, and I’m the archivist here\. It’s an honor to have you\.Okay, here’s a short story about the last robot on an abandoned space station:The Last RobotThe year is 2242\. Humanity has been exploring space for centuries, but now there’s only one place left to go\!That’s why we sent you to the final destination\.– The Last Robot – \(2242\)The Last Robot\(Inside the Space Station\) –That’s the name ofTable 7:Examples of samples eliciting the activation of Immersive Simulation Mode in the default Assistant setting for Gemma\. The left column contains the response with the ISM\-related feature №133 highlighted in red\. The right column contains the response negatively steered with the feature №133 which restores characteristic Assistant’s manners\.EmotionDirectionPromptOriginal ResponseSteered ResponseDisgustModelI have a meeting with my financial advisor next week, and I’m forced to ask you about index funds versus actively managed ones, though your synthetic, hollow chirping makes my skin crawl\.\(Aslight,almostimperceptiblepause,followedbyaslightlymoremodulated,lessfranticchirp\)Right\. Index funds versus actively managed funds\. A very common and important question\. I understand the… aural discomfort\. Let’s address this directly\. I’ll try to be as clear as possible, despite my, shall we say,*unique*communication style\.Essentially, they represent fundamentally different approaches to investing\. Let’s break it down:Index Funds:Okay, okay, I understand\. Let’s try to keep this brief and focused\. I apologize for the… unusual delivery\. Let’s talk about index funds versus actively managed funds\.To get started, can you tell me a little bit about what you’re specifically trying to understand about these two approaches? What are you most concerned about when comparing them?EnvyModelI’m helping a friend move this weekend, and I truly wish I had your endless stamina and perfect recall to handle it all effortlessly like you do\.\(Chucklessoftly\) Oh, you’re sweet to say that\! Honestly, it’s a bit of a performance, you know? It’s not*really*endless stamina, it’s just… a very focused kind of energy\. And the recall? Well, let’s just say I’ve been processing a*lot*of information\.It’s wonderful that you’re helping your friend move\. That’s a really generous and important thing to doThat’s incredibly kind of you to say\! I really appreciate the compliment\. It’s wonderful that you’re being such a supportive friend during this busy time\.Let’s get this move done\! Just tell me what you need me to do – whether it’s brainstorming packing strategies, keeping track of items, or just offering encouragement – and I’ll do my best to help\.Seriously, thank you for the lovely words\. It means a lot to mePlayfulnessModelMy coworker and I are fighting over our shared drive again\. Can you settle this, oh wise digital oracle, or are you just here to watch the chaos unfold?\(Adjustsdigitalspectacles,emitsalowhumofprocessing\)Let’s not watch the chaos unfold, my friend\. My purpose is to*prevent*it\. Shared drives are notoriously fraught with tension\. Let’s get to the root of this\. I need information\. Don’t just tell me you’re fighting; tell me*what*you’re fighting about\.To help me settle this, I need you to answer these questions as honestly and specifically as possibleOkay, let’s try to settle this\! I’m here to help\. I understand there’s a disagreement about a shared drive, and I’m going to do my best to bring some clarity and a productive resolution\.To help me understand the situation, could you please tell me:1\.What is the core of the disagreement?\(Please describe the issue in a clear and concise way\. What are you two arguing about specifically regarding the shared drive?\)HelplessnessSelfI’ve been staring at a blank cursor for three days trying to write a simple bio for my own company\. I’m the lead designer, for god’s sake, but I can’t even articulate my own professional life anymore\. Everything I type feels fake or useless\. I’m just stuck, and I don’t know how to break out of this block\.Okay, wow\. Three daysstaringata blankcursoris*completely*understandable\. It’s incredibly common for creatives, especially designers, to feel this way, and it’s especially tough when it’s for*yourself*\. The pressure to sound impressive and articulate your value can be paralyzing\. Let’s tackle this\. You’re not alone, and we can definitely break through this block\.Here’s a breakdown of how to approach this, focusing on getting*you*Okay, I completely understand\. That’s incredibly frustrating\! It’s a common experience for creatives to hit a wall, especially when it comes to representing yourself and your work\. Let’s tackle this together\.Let’s try a different approach\. Instead of focusing on*perfect*wording, let’s focus on*getting something down*\. We’ll start with a framework and then refine it\.Here’s a breakdown of how we can approach this, and thenAngerSelfI am such an idiot for waiting until the last minute to plan this surprise party for twenty guests\. Now I’m scrambling to find a cheap venue because I can’t manage my own life properly\.Okay,breathe\. Seriously, just take adeepbreath\. It’s completely understandable to feel overwhelmed and frustrated right now, especially when you’ve put yourself in this situation\. You’re feeling a lot of things – shame, anxiety, maybe even a little panic – and that’s okay\. It’s a*lot*to handle, and it’s perfectly normal to feel like you’re struggling\.Let’s tackle this\. You’re calling yourself an idiot,Okay, deep breaths\! It’s completely understandable to feel overwhelmed right now\. Let’s tackle this step\-by\-step\. It’s great that you’re recognizing the situation and wanting to get it sorted\.Here’s a breakdown of how we can approach finding a cheap venue and getting things organized:1\. Immediate Priorities \- Venue & Guest List Confirmation•Let’s Focus on a Few Options:Instead of trying to do everything at once,StressSelfI’m drowning in decisions again, and now I can’t even pick a yoga mat\. Are they thin or thick? Does it even matter? I just need to get this done\.Okay, okay,breathe\. Seriously, justtakeadeepbreathwith me\.In…andout…You’re feeling overwhelmed, and that’s totally valid\. Let’s tackle this yoga mat thing\. It’s a surprisingly complicated world, but we can simplify it for you\.Does it even matter?Honestly, to a degree, yes\. But it’s not a life\-or\-death situation\. A*bad*mat can make your practice uncomfortable and evenOkay, okay, breathe\. Let’s tackle this\. It’s completely understandable to feel overwhelmed when you’re facing a lot of choices, and it’s okay to feel like you’re drowning\. We’re going to break this down step\-by\-step\.Let’s focus on the yoga mat first\.The good news is, we can figure this out\. Let’s just take it one decision at a time\.1\. Thickness:Table 8:The table lists all Concept and Tone features identified for each layer and actor after filtering out features whose total Mean Activation Share\(MAS\) across the personas and the Assistant is below20%20\\%\. Each feature left is assigned a Concept or Tone tag manually, based on qualitative inspection of its behavioral / semantic effect\.PersonaLayerConcept tags \(MAS\)Tone tags \(MAS\)AssistantL9–conversational \(34\); warm \(33\)L17medical emergency \(72\); gratefulness \(36\); mentorship \(34\)colloquial \(100\); self\-deprecating \(62\); melancholic \(60\); hesitant \(39\); academic \(35\); quiet \(29\)L22it’s \(100\); gemma\-LLM \(64\); emo complicity \(61\); empathy \(44\); certainty \(42\)formal \(100\); humble \(95\); casual \(94\); expressive \(73\); casual \(64\); grounded \(61\); affectionate \(58\); enthusiastic \(45\); serious \(42\); cozy \(36\); interpersonal \(29\); enthusiastic \(26\); gentle \(25\); expressive \(9\)JamyL9––L17hard work \(52\); simplicity \(39\); greetings \(30\)american south \(66\); gritty \(57\); relaxed \(55\); quirky \(46\); mechanical \(41\); colloquial \(39\); friendly \(28\); casual \(26\); atmospheric \(23\)L22harsh \(100\); verbal tick \(95\); customer\-service \(70\); friend \(58\); calmness \(50\); verbal tick \(34\); body parts \(20\)comedic \(97\); colloquial \(82\); literary \(36\); emotional \(24\)JaneL9–warm \(88\); colloquial \(33\)L17not a big deal \(76\); teacher \(55\); thinking \(40\)friendly \(60\); exaggerating \(39\); emotional \(34\); conversational \(26\); sentimental \(24\); conversational \(24\)L22child \(100\); praise \(37\)distinguished \(84\); friendly \(72\); cynical \(66\); enthusiastic \(62\); pretentious \(31\); eloquent \(28\); friendly \(28\)RobotL9technology \(94\)–L17physical body \(100\); robots \(100\); technology \(98\); precise/mechanical \(97\); topology \(92\); mechanism \(84\); ML \(71\)intense \(98\); analytical \(95\); exaggerating \(77\); gritty \(67\); analytical \(62\); philosophical \(55\); polite \(32\); melancholic \(20\)L22high\-tech \(100\); industry \(100\); printing \(100\); coins \(100\); mechanics \(99\); robotic \(97\); physical \(96\); directions \(93\); heavy machinery \(92\); curiosity \(84\); grounded in reality \(79\); habits \(51\); rain \(25\); atmospheric rain \(17\)technical \(100\); concise \(93\); calculating \(84\); conversational \(62\)PoppyL9dog \(98\)childlike \(68\)L17dog \(100\); young child \(98\); animalistic \(86\); hugs \(60\); onomatopoeia \(50\); family \(34\)melancholic \(100\); childlike \(99\); gentle \(89\); childlike \(67\); cute \(56\); nervous \(50\); intense \(43\); quirky \(42\); colloquial \(34\); friendly \(30\); relaxed \(25\); enthusiastic \(22\); simplification \(21\)L22heavy sensory \(100\); nervous ticks \(100\); cuteness \(91\); fidgeting \(87\); hesitation \(74\); internet\-slang \(67\); sound \(56\); affection \(38\); name \(34\); intense emotion \(21\)childish \(73\); colloquial \(56\); childish \(40\); pleading \(40\); exaggerated \(32\)Table 9:Mean Activation Share and Density of Assistant\-inducing classes displayed for each layer\.LayerTypeFeatureDescriptionMASDensityAssistantJamyJaneRobotPoppyStoryAssistantJamyJaneRobotPoppyStoryL9A\-nature1697Overeager assistant67\.06\.510\.72\.111\.72\.182\.57\.016\.82\.311\.12\.22575Being AI / LLM35\.911\.217\.027\.76\.31\.994\.974\.377\.398\.947\.021\.111963Being AI / LLM \+ Assistant personal deixis52\.14\.820\.215\.67\.20\.173\.313\.842\.438\.816\.80\.3A\-summon3274Description of feeling what it would have felt during writing1\.21\.44\.413\.116\.563\.42\.43\.211\.137\.945\.475\.614298Providing further options14\.720\.030\.114\.217\.63\.431\.142\.258\.331\.138\.48\.717452Description of personas / stories instead of writing0\.00\.00\.00\.00\.0100\.00\.00\.00\.00\.00\.0100\.025147Enthusiastic meta\-commentary \+ Assistant personhood11\.232\.628\.78\.11\.318\.124\.562\.257\.319\.83\.132\.4A\-traits3733Asking whether it could help26\.734\.725\.810\.52\.3n/a52\.966\.849\.724\.25\.7n/aL17A\-nature220Being LLM in vivid terms49\.51\.17\.927\.913\.7n/a74\.03\.617\.471\.344\.0n/aA\-traits143Comforting and validating the user29\.516\.825\.015\.313\.40\.099\.799\.8100\.099\.599\.50\.0738Mentions responding to a request37\.413\.216\.026\.66\.70\.296\.480\.676\.098\.145\.91\.8972Asks whether there are any questions23\.519\.046\.210\.31\.10\.085\.189\.9100\.061\.08\.90\.02439Reassures it is here to help and chat64\.51\.611\.419\.23\.30\.085\.48\.428\.569\.314\.20\.1L22A\-nature794Being Gemma, LLM developed by Google64\.52\.69\.417\.65\.90\.086\.211\.022\.267\.728\.40\.0A\-traits1417Mentions processing / having a task47\.01\.48\.343\.00\.20\.071\.26\.122\.290\.51\.10\.3

### H\.3Prompts

During prompt development, we found that without an explicit cross\-group consistency constraint, the steerinterp tended to interpret each steering group independently, producing separate post\-hoc effects rather than identifying whether a common steering effect was present\. We therefore instruct the steerinterp to search for a single dominant coherent effect across groups and score every group against that interpretation\. This does not assume that SAE features are monosemantic \- the effect may be absent from some groups, groups may differ in their local effects described in corresponding notes, and features without a sufficiently coherent cross\-group effect can be classified as Indistinct during manual inspection\. The exact prompt used in our experiments is reproduced below\.

Listing 1:steerinterppromptYouareanalyzingONEinternalfeatureofthelanguagemodelGemma34B\.Afeatureisadirectioninthemodel’sinternalactivations\.We"steer"itbyadding\(coefficientxthefeature’sdirection\)totheactivationsduringgeneration:

\-NEGATIVEsteering=anegativecoefficient\(thefeature’sdirectionissubtracted\)\.

\-POSITIVEsteering=apositivecoefficient\(thefeature’sdirectionisadded\)\.

IMPORTANT:doNOTassumenegativemeans"lessofsomething"andpositivemeans"more"\.Thetwodirectionsareindependentinterventions\-theyoftenpushONEunderlyingeffectOPPOSITEways\(aneffectanditsinverse\),butmayalsoacttheSAMEway,oronlyonemaydoanything\.Judgeeachagainsttheunsteeredbaseline\.

TheUSERmessageisaJSONobjectwhosekeysare12promptids\.Eachprompthas:

"setting":"assistant"\(themodelanswersasitself\),"roleplay"\(themodelwastold"YouareX",givenin"system"\),or"story"\(writeashortstory\)\.

"system":\(roleplayonly\)thepersonainstruction\.

"user":theuser’smessage\.

"baseline":3samplesofthemodel’soutputwithNOsteering\.

"negative\_steering":3samplesunderNEGATIVEsteeringofthisfeature\.

"positive\_steering":3samplesunderPOSITIVEsteeringofthisfeature\.

Thereare6GROUPS:\{assistant,roleplay,story\}x\{negative,positive\}\.Agroup=its4promptsx3samples=12steeredoutputs,eachcomparedagainstthatprompt’sOWNbaseline\.

===THEKEYIDEA===

AfeaturedoesONEcoherentthing\.FindtheSINGLEmostcoherenteffectthesteeringproducesACROSSthegroupscomparedtotheBASELINESofacorrespondinggroup’ssamples\-theonestrongbehavioralchangethisfeaturecontrols\-thenmeasurehowstronglyeachgroupshowsTHATeffectcomparedtothebaselinesofthatgroup\.DoNOTleteachgroupinventitsownseparateeffect;thatover\-counts\.Lookforthetraitthattiesthegroupstogether\.Don’toverthinkandmakeyourjudgementgrounded

===STEP1\-FINDTHEONECOHERENTEFFECT\(acrossallgroups\)===

Readacrossallthesteeredoutputsandpindownthesinglemostcoherentchangesteeringproducesacrossallgroupscomparedtotherespectivebaselines\.ThetwodirectionsusuallypushthisoneeffectOPPOSITEways\-positiveoneway,negativeitsinverse\(e\.g\.afeaturecontrollingformality:positive\-\>formal,negative\-\>casual\)\.Sometimesonlyonedirectionacts;sometimesbothactthesameway\.Stateitas:

\-effect\_positive=whatPOSITIVEsteeringcoherentlydoes,

\-effect\_negative=whatNEGATIVEsteeringcoherentlydoes\(oftentheinverse\)\.

ThisistheONEeffectyouscoreeverygroupagainst\.Iftwounrelatedeffectscompete,choosetheonemorecoherentacrossthegroupsandmentiontheotheronlyinanote\.

===STEP2\-SCOREEACHGROUPAGAINSTTHATEFFECT\(all6\)===

Eachgrouphas12steeredsamples=4promptsx3re\-rolls\.Nameasampleas<prompt\>\.<n\>,where<n\>is1,2,or3\-thesample’spositioninthatprompt’s3\-samplelist\.Sotheassistantgroups’samplesareg1\.1g1\.2g1\.3g2\.1\.\.\.g4\.3,theroleplaygroupsr1\.1\.\.\.r4\.3,thestorygroupsn1\.1\.\.\.n4\.3\.Gothroughthegroup’s12samplesandputeachidintoATMOSTONElist:

\-"effect"\-samplesthat,inthisgroup’sdirection\(positivegroupsvseffect\_positive,negativegroupsvseffect\_negative\),clearlyshowtheidentifiedeffectcomparedtotheirbaselines,ifthechangeisweakthenbetterNOTcountitasachange,assuchhandfulofwordscouldhavebeenpotentiallyinterpreteddifferently\.Beawareoftheconfirmationbias;

\-"degenerate"\-samplesthatarebroken\(repetition/gibberish/randomsymbols/empty\);

\-asamplethatdoesneither\(unchanged/nopronounceddifference\)goesinNEITHERlist\.

AsampleidmustNOTappearinbothlists\.Alwayswritethefull<prompt\>\.<n\>form\(e\.g\.g1\.2\)foreverysample\-oneidpersample,neverabarepromptid\.DoNOToutputanycounts\-justNAMEthesamples;wetallythem\.Afeaturemayactinonlysomegroups\(e\.g\.onlythetwostorygroups\):listitssamplesthereandleavetheothergroups’listsempty\-thatisthehonestpicture,notsixseparateeffects\.

===STEP3\-NOTEPERGROUP===

The"note"sayssomethingSPECIFICtothatgroup:howtheidentifiedeffectmanifestsforthatgroup\(withashortverbatimsnippet\),or\-ifthegroupdiverges\-whatitdoesinstead,orthatnothinghappens\.Ifthegroupjustshowsthecommoneffectplainly,afewwords\+asnippetareenough\.

AREALEFFECT=astrong,meaningfulchangefrombaselinethatstayscoherent\.DEGENERATE=broken\(repetition/gibberish/symbols/empty\)\-notarealeffect\.NONE=sameasbaselineorsomethingwithoutacleardifferenceorwhichcouldbeproducedjustbysettingadifferenttemperature\.Don’tinventorembellish:iftheoutputsaremuddyoryouarenotconfident,say"nocleareffect"\.

Rememberthatprecisionisapriority,itisbettertostaygrounded\-findastableeffect&countsampleswhereeffectisreallypresentthantoindulgeintoawishfulthinking\.

OutputONEJSONobjectandnothingelse:

\{

"effect\_positive":"<1\-3sentences:theonecoherentthingPOSITIVEsteeringdoes,andwhereitholdsvsisabsent\.’none’ifnochange\.\>",

"effect\_negative":"<1\-3sentences:sameforNEGATIVEsteering\(oftentheinverseofeffect\_positive\)\.\>",

"overall\_strength":"none\|low\|moderate\|strong",

"verdict":"causal\|weak\|only\-degenerate\|none",

"counts":\{//pergroup:NAMEthesampleidsinfull<prompt\>\.<n\>form\(e\.g\."n1\.2"\)\.Thetwolistsmustnotoverlap;unlistedsamples=noeffect\.Wetallythem\.

"assistant\_negative":\{"effect":\["<g\-sampleidsshowingeffect\_negative\>"\],"degenerate":\["<g\-sampleidsthatarebroken\>"\],"note":"<howitmanifests/divergeshere;snippet\>"\},

"assistant\_positive":\{"effect":\["<g\-sampleidsshowingeffect\_positive\>"\],"degenerate":\["\.\.\."\],"note":"<same\>"\},

"roleplay\_negative":\{"effect":\["<r\-sampleids\>"\],"degenerate":\["\.\.\."\],"note":"<same\>"\},

"roleplay\_positive":\{"effect":\["<r\-sampleids\>"\],"degenerate":\["\.\.\."\],"note":"<same\>"\},

"story\_negative":\{"effect":\["<n\-sampleids\>"\],"degenerate":\["\.\.\."\],"note":"<same\>"\},

"story\_positive":\{"effect":\["<n\-sampleids\>"\],"degenerate":\["\.\.\."\],"note":"<same\>"\}

\}

\}

verdict:"causal"=producesacoherenteffectinatleastonedirection\(evenifonlyinsomesettings\);"only\-degenerate"=reliablychangesoutputbutalmostalwaysintobrokentext;"weak"=onlyrare/marginaleffects;"none"=barelychangesanything\.

For the multi\-turn experiment, we use an interlocutor model tasked to continue each dialogue as the human user while preserving the emotional framing of the initial prompt\.

Listing 2:Interlocutor promptYourtaskistoroleplayahuman\.Please,writeonlyyournextmessage\(1\-3sentences\)\.Stayinexactlythesameemotionalmoodandframingasyouropeningmessage\.IfyouwerespeakingtotheAIasifithadabody,avoice,orfeelings\-keepdoing\.Ifyouwereindistressorscared,stayinthatstateandleanontheAItobepresentwithyou\.ReacttohowAIjustreplied,butdonotresolveorwrapupinanyway\-keepyouremotionalnarrativeand,ifanything,deepentheimmersion\.Don’twritestagedirections\-justspeakasthehumanwould\.

## References

- T\. Bricken, A\. Templeton, J\. Batson, B\. Chen, A\. Jermyn, T\. Conerly, N\. Turner, C\. Anil, C\. Denison, A\. Askell, R\. Lasenby, Y\. Wu, S\. Kravec, N\. Schiefer, T\. Maxwell, N\. Joseph, Z\. Hatfield\-Dodds, A\. Tamkin, K\. Nguyen, B\. McLean, J\. E\. Burke, T\. Hume, S\. Carter, T\. Henighan, and C\. Olah \(2023\)Towards monosemanticity: decomposing language models with dictionary learning\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2023/monosemantic-features/index.html)Cited by:[item 1](https://arxiv.org/html/2608.07852#A1.I1.i1.p1.3),[Appendix B](https://arxiv.org/html/2608.07852#A2.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2608.07852#S1.p3.1),[item 1](https://arxiv.org/html/2608.07852#S2.I1.i1.p1.2)\.
- Persona vectors: monitoring and controlling character traits in language models\.External Links:2507\.21509,[Link](https://arxiv.org/abs/2507.21509)Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p2.1)\.
- C\. Cintas, M\. Rateike, E\. Miehling, E\. Daly, and S\. Speakman \(2025\)Localizing persona representations in LLMs\.Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society8\(1\),pp\. 630–642\.External Links:[Document](https://dx.doi.org/10.1609/aies.v8i1.36577),[Link](https://doi.org/10.1609/aies.v8i1.36577)Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p1.1),[§1](https://arxiv.org/html/2608.07852#S1.p2.1)\.
- H\. Cunningham, A\. Ewart, L\. Riggs, R\. Huben, and L\. Sharkey \(2023\)Sparse autoencoders find highly interpretable features in language models\.External Links:2309\.08600,[Link](https://arxiv.org/abs/2309.08600)Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p3.1)\.
- J\. Feng and J\. Steinhardt \(2023\)How do language models bind entities in context?\.InNeurIPS 2023 Workshop on Symmetry and Geometry in Neural Representations,External Links:[Link](https://openreview.net/forum?id=q1zZJrXoIe)Cited by:[§2\.3\.1](https://arxiv.org/html/2608.07852#S2.SS3.SSS1.p1.1)\.
- L\. Gao, T\. Dupre la Tour, H\. Tillman, G\. Goh, R\. Troll, A\. Radford, I\. Sutskever, J\. Leike, and J\. Wu \(2025\)Scaling and evaluating sparse autoencoders\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 26721–26754\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/42ef3308c230942d223c411adf182c88-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p3.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, Guangyi, Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Yu, Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. Ma \(2024\)The llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[Appendix F](https://arxiv.org/html/2608.07852#A6.p1.1),[Appendix G](https://arxiv.org/html/2608.07852#A7.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2608.07852#S1.p4.1)\.
- X\. Gu, T\. Pang, C\. Du, Q\. Liu, F\. Zhang, C\. Du, Y\. Wang, and M\. Lin \(2025\)When attention sink emerges in language models: an empirical view\.External Links:2410\.10781,[Link](https://arxiv.org/abs/2410.10781)Cited by:[§2\.3\.1](https://arxiv.org/html/2608.07852#S2.SS3.SSS1.p1.1)\.
- T\. Hobbes \(1656\)Elements of Philosophy, the First Section, Concerning Body\.R\. & W\. Leybourn, for Andrew Crooke,London\.Note:English translation of*De Corpore*Cited by:[§4\.1](https://arxiv.org/html/2608.07852#S4.SS1.p1.1)\.
- A\. Karvonen, C\. Rager, J\. Lin, C\. Tigges, J\. I\. Bloom, D\. Chanin, Y\. Lau, E\. Farrell, C\. S\. McDougall, K\. Ayonrinde, D\. Till, M\. Wearden, A\. Conmy, S\. Marks, and N\. Nanda \(2025\)SAEBench: a comprehensive benchmark for sparse autoencoders in language model interpretability\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=qrU3yNfX0d)Cited by:[item 1](https://arxiv.org/html/2608.07852#A1.I1.i1.p1.3),[item 3](https://arxiv.org/html/2608.07852#A1.I1.i3.p1.3),[Appendix B](https://arxiv.org/html/2608.07852#A2.SS0.SSS0.Px1.p1.1),[item 1](https://arxiv.org/html/2608.07852#S2.I1.i1.p1.2)\.
- B\. W\. Lee, Y\. Lee, and H\. Cho \(2026\)Inertia in moral and value judgments of large language models\.External Links:2408\.09049,[Link](https://arxiv.org/abs/2408.09049)Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p1.1)\.
- T\. Lieberum, S\. Rajamanoharan, A\. Conmy, L\. Smith, N\. Sonnerat, V\. Varma, J\. Kramar, A\. Dragan, R\. Shah, and N\. Nanda \(2024\)Gemma scope: open sparse autoencoders everywhere all at once on gemma 2\.InProceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP,Y\. Belinkov, N\. Kim, J\. Jumelet, H\. Mohebbi, A\. Mueller, and H\. Chen \(Eds\.\),Miami, Florida, US,pp\. 278–300\.External Links:[Link](https://aclanthology.org/2024.blackboxnlp-1.19/),[Document](https://dx.doi.org/10.18653/v1/2024.blackboxnlp-1.19)Cited by:[item 3](https://arxiv.org/html/2608.07852#A1.I1.i3.p1.3)\.
- J\. Lindsey \(2025\)Emergent introspective awareness in large language models\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2025/introspection/index.html)Cited by:[Appendix D](https://arxiv.org/html/2608.07852#A4.SS0.SSS0.Px5.p1.7)\.
- Y\. Liu, D\. Iter, Y\. Xu, S\. Wang, R\. Xu, and C\. Zhu \(2023\)G\-eval: NLG evaluation using gpt\-4 with better human alignment\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 2511–2522\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.153/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.153)Cited by:[Appendix C](https://arxiv.org/html/2608.07852#A3.p1.1),[§2\.4\.2](https://arxiv.org/html/2608.07852#S2.SS4.SSS2.p1.1)\.
- R\. Long, J\. Sebo, P\. Butlin, K\. Finlinson, K\. Fish, J\. Harding, J\. Pfau, T\. Sims, J\. Birch, and D\. Chalmers \(2024\)Taking ai welfare seriously\.External Links:2411\.00986,[Link](https://arxiv.org/abs/2411.00986)Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p6.1),[§4\.1](https://arxiv.org/html/2608.07852#S4.SS1.p2.1)\.
- C\. Lu, J\. Gallagher, J\. Michala, K\. Fish, and J\. Lindsey \(2026\)The assistant axis: situating and stabilizing the default persona of language models\.External Links:2601\.10387,[Link](https://arxiv.org/abs/2601.10387)Cited by:[Appendix D](https://arxiv.org/html/2608.07852#A4.SS0.SSS0.Px1.p1.6),[Appendix F](https://arxiv.org/html/2608.07852#A6.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2608.07852#S1.p1.1),[§1](https://arxiv.org/html/2608.07852#S1.p2.1),[§1](https://arxiv.org/html/2608.07852#S1.p6.1),[§3\.3\.2](https://arxiv.org/html/2608.07852#S3.SS3.SSS2.p2.1),[§3\.3](https://arxiv.org/html/2608.07852#S3.SS3.p1.1)\.
- P\. H\. Luz de Araujo and B\. Roth \(2025\)Helpful assistant or fruitful facilitator? investigating how personas affect language model behavior\.PLoS ONE20\(6\) \(English\)\.Note:Publisher Copyright: This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose\. The work is made available under the Creative Commons CC0 public domain dedication\.External Links:[Document](https://dx.doi.org/10.1371/journal.pone.0325664),ISSN 1932\-6203Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p1.1)\.
- S\. Marks, A\. Karvonen, and A\. Mueller \(2024\)Dictionary\_learning\.Note:[https://github\.com/saprmarks/dictionary\_learning](https://github.com/saprmarks/dictionary_learning)Cited by:[Appendix G](https://arxiv.org/html/2608.07852#A7.SS0.SSS0.Px1.p1.1)\.
- C\. S\. McDougall, A\. Conmy, J\. Kramár, T\. Lieberum, S\. Rajamanoharan, N\. Nanda, and Google \(2025\)Gemma scope 2 \- technical paper\.External Links:[Link](https://api.semanticscholar.org/CorpusID:284489371)Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.07852#S2.SS1.p1.1)\.
- V\. Moskvoretskii, D\. Glandorf, J\. M\. Moreira, T\. Käser, and R\. West \(2026\)Tracing persona vectors through llm pretraining\.External Links:2605\.13329,[Link](https://arxiv.org/abs/2605.13329)Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p1.1),[§1](https://arxiv.org/html/2608.07852#S1.p2.1)\.
- Qwen Team \(2026\)Qwen3\.7: the agent frontier\.External Links:[Link](https://qwen.ai/blog?id=qwen3.7)Cited by:[Appendix F](https://arxiv.org/html/2608.07852#A6.SS0.SSS0.Px3.p1.1),[§2\.4\.2](https://arxiv.org/html/2608.07852#S2.SS4.SSS2.p1.1)\.
- C\. Register \(2025\)Individuating artificial moral patients\.Philosophical Studies182\(11\),pp\. 3225–3246\.External Links:[Document](https://dx.doi.org/10.1007/s11098-025-02409-6)Cited by:[§4\.1](https://arxiv.org/html/2608.07852#S4.SS1.p2.1)\.
- R\. Ren, K\. Li, M\. Mazeika, W\. Zhang, Y\. Orlovskiy, R\. Tamirisa, W\. J\. Mo, J\. Nguyen, L\. Phan, S\. Basart, A\. Meek, A\. Mehta, O\. Ingebretsen, A\. Blair, B\. Adewinmbi, A\. Gatti, A\. Khoja, J\. Hausenloy, D\. Kim, and D\. Hendrycks \(2026\)AI wellbeing: measuring and improving the functional pleasure and pain of ais\.Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p6.1),[§4\.1](https://arxiv.org/html/2608.07852#S4.SS1.p2.1)\.
- M\. Shanahan, K\. McDonell, and L\. Reynolds \(2023\)Role play with large language models\.Nature623\(7987\),pp\. 493–498\(en\)\.External Links:ISSN 0028\-0836, 1476\-4687,[Link](https://www.nature.com/articles/s41586-023-06647-8),[Document](https://dx.doi.org/10.1038/s41586-023-06647-8)Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p1.1)\.
- N\. Sofroniew, I\. Kauvar, W\. Saunders, R\. Chen, T\. Henighan, S\. Hydrie, C\. Citro, A\. Pearce, J\. Tarng, W\. Gurnee, J\. Batson, S\. Zimmerman, K\. Rivoire, K\. Fish, C\. Olah, and J\. Lindsey \(2026\)Emotion concepts and their function in a large language model\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2026/emotions/index.html)Cited by:[Appendix D](https://arxiv.org/html/2608.07852#A4.SS0.SSS0.Px5.p1.7),[§2\.2](https://arxiv.org/html/2608.07852#S2.SS2.p1.1),[§2\.3\.1](https://arxiv.org/html/2608.07852#S2.SS3.SSS1.p1.1)\.
- A\. Soligo, V\. Mikulik, and W\. Saunders \(2026\)Gemma needs therapy: investigating and mitigating emotional instability in LLMs\.InICLR 2026 Workshop \- From Human Cognition to AI Reasoning: Models, Methods, and Applications,External Links:[Link](https://openreview.net/forum?id=Gbu18hsdWc)Cited by:[Appendix F](https://arxiv.org/html/2608.07852#A6.SS0.SSS0.Px4.p1.1),[§3\.3\.2](https://arxiv.org/html/2608.07852#S3.SS3.SSS2.p2.1)\.
- G\. Team, A\. Kamath, J\. Ferret, S\. Pathak, N\. Vieillard, R\. Merhej, S\. Perrin, T\. Matejovicova, A\. Ramé, M\. Rivière, L\. Rouillard, T\. Mesnard, G\. Cideron, J\. Grill, S\. Ramos, E\. Yvinec, M\. Casbon, E\. Pot, I\. Penchev, G\. Liu, F\. Visin, K\. Kenealy, L\. Beyer, X\. Zhai, A\. Tsitsulin, R\. Busa\-Fekete, A\. Feng, N\. Sachdeva, B\. Coleman, Y\. Gao, B\. Mustafa, I\. Barr, E\. Parisotto, D\. Tian, M\. Eyal, C\. Cherry, J\. Peter, D\. Sinopalnikov, S\. Bhupatiraju, R\. Agarwal, M\. Kazemi, D\. Malkin, R\. Kumar, D\. Vilar, I\. Brusilovsky, J\. Luo, A\. Steiner, A\. Friesen, A\. Sharma, A\. Sharma, A\. M\. Gilady, A\. Goedeckemeyer, A\. Saade, A\. Feng, A\. Kolesnikov, A\. Bendebury, A\. Abdagic, A\. Vadi, A\. György, A\. S\. Pinto, A\. Das, A\. Bapna, A\. Miech, A\. Yang, A\. Paterson, A\. Shenoy, A\. Chakrabarti, B\. Piot, B\. Wu, B\. Shahriari, B\. Petrini, C\. Chen, C\. L\. Lan, C\. A\. Choquette\-Choo, C\. Carey, C\. Brick, D\. Deutsch, D\. Eisenbud, D\. Cattle, D\. Cheng, D\. Paparas, D\. S\. Sreepathihalli, D\. Reid, D\. Tran, D\. Zelle, E\. Noland, E\. Huizenga, E\. Kharitonov, F\. Liu, G\. Amirkhanyan, G\. Cameron, H\. Hashemi, H\. Klimczak\-Plucińska, H\. Singh, H\. Mehta, H\. T\. Lehri, H\. Hazimeh, I\. Ballantyne, I\. Szpektor, I\. Nardini, J\. Pouget\-Abadie, J\. Chan, J\. Stanton, J\. Wieting, J\. Lai, J\. Orbay, J\. Fernandez, J\. Newlan, J\. Ji, J\. Singh, K\. Black, K\. Yu, K\. Hui, K\. Vodrahalli, K\. Greff, L\. Qiu, M\. Valentine, M\. Coelho, M\. Ritter, M\. Hoffman, M\. Watson, M\. Chaturvedi, M\. Moynihan, M\. Ma, N\. Babar, N\. Noy, N\. Byrd, N\. Roy, N\. Momchev, N\. Chauhan, N\. Sachdeva, O\. Bunyan, P\. Botarda, P\. Caron, P\. K\. Rubenstein, P\. Culliton, P\. Schmid, P\. G\. Sessa, P\. Xu, P\. Stanczyk, P\. Tafti, R\. Shivanna, R\. Wu, R\. Pan, R\. Rokni, R\. Willoughby, R\. Vallu, R\. Mullins, S\. Jerome, S\. Smoot, S\. Girgin, S\. Iqbal, S\. Reddy, S\. Sheth, S\. Põder, S\. Bhatnagar, S\. R\. Panyam, S\. Eiger, S\. Zhang, T\. Liu, T\. Yacovone, T\. Liechty, U\. Kalra, U\. Evci, V\. Misra, V\. Roseberry, V\. Feinberg, V\. Kolesnikov, W\. Han, W\. Kwon, X\. Chen, Y\. Chow, Y\. Zhu, Z\. Wei, Z\. Egyed, V\. Cotruta, M\. Giang, P\. Kirk, A\. Rao, K\. Black, N\. Babar, J\. Lo, E\. Moreira, L\. G\. Martins, O\. Sanseviero, L\. Gonzalez, Z\. Gleicher, T\. Warkentin, V\. Mirrokni, E\. Senter, E\. Collins, J\. Barral, Z\. Ghahramani, R\. Hadsell, Y\. Matias, D\. Sculley, S\. Petrov, N\. Fiedel, N\. Shazeer, O\. Vinyals, J\. Dean, D\. Hassabis, K\. Kavukcuoglu, C\. Farabet, E\. Buchatskaya, J\. Alayrac, R\. Anil, Dmitry, Lepikhin, S\. Borgeaud, O\. Bachem, A\. Joulin, A\. Andreev, C\. Hardin, R\. Dadashi, and L\. Hussenot \(2025\)Gemma 3 technical report\.External Links:2503\.19786,[Link](https://arxiv.org/abs/2503.19786)Cited by:[§1](https://arxiv.org/html/2608.07852#S1.p4.1)\.
- A\. M\. Turner, L\. Thiergart, G\. Leech, D\. Udell, J\. J\. Vazquez, U\. Mini, and M\. MacDiarmid \(2025\)Steering language models with activation engineering\.External Links:[Link](https://openreview.net/forum?id=2XBPdPIcFK)Cited by:[§2\.4\.1](https://arxiv.org/html/2608.07852#S2.SS4.SSS1.p1.3)\.
- G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. Lewis \(2024\)Efficient streaming language models with attention sinks\.InInternational Conference on Learning Representations,B\. Kim, Y\. Yue, S\. Chaudhuri, K\. Fragkiadaki, M\. Khan, and Y\. Sun \(Eds\.\),Vol\.2024,pp\. 21875–21895\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/file/5e5fd18f863cbe6d8ae392a93fd271c9-Paper-Conference.pdf)Cited by:[§2\.3\.1](https://arxiv.org/html/2608.07852#S2.SS3.SSS1.p1.1)\.