AI 安全的功能性福祉

# Functional Welfare for AI Safety
Source: [https://arxiv.org/html/2610.22460](https://arxiv.org/html/2610.22460)
###### Abstract

AI welfare is generally treated purely as an ethical issue rather than also as a safety issue\. We argue this is a mistake\. Whatever their moral status, models exhibit welfare\-like states that causally influence their behaviour\. Following recent work, we adopt the term ‘functional welfare’ for these states\. We make three claims\. First, functional welfare already arises naturally and will become more robust as models acquire more coherent goals and personas\. Second, functional welfare should be treated as a safety variable: it should inform evaluation design, be tracked in monitoring, and be a direct target of post\-training\. Third, against existing scepticism about measuring AI welfare, we argue that good empirical work on functional welfare is possible\. We propose three tests for whether a functional welfare attribution is warranted: whether multiple measures converge, whether the state predicts behaviour outside the setting that induced it, and, for models fine\-tuned to shift their functional welfare, whether they still differ from their base when given the same conversation history\. Applying these tests to Gemma 3 27B, we find that simply rejecting the model’s answers, without any steering, raises its frustration across several indicators and makes it more likely to break rules\. Finetuning the model on its own narrated solutions lowers this frustration\. These results are preliminary, but they suggest functional welfare can be studied, and shaped, empirically\.

1University College London2ERA3University of Toronto4University of Cambridge

## 1Introduction

AI welfare is generally treated as an ethical issue rather than a safety issue; papers urging us to take AI welfare seriously do so on the basis that there is a realistic possibility of AI being conscious, and therefore morally significant\([Sebo and Long 2025](https://arxiv.org/html/2610.22460#bib.bib14);[Long et al\. 2024](https://arxiv.org/html/2610.22460#bib.bib15);[Chalmers 2023](https://arxiv.org/html/2610.22460#bib.bib16);[Goldstein and Kirk\-Giannini 2024](https://arxiv.org/html/2610.22460#bib.bib17);[Dung 2025](https://arxiv.org/html/2610.22460#bib.bib19);[Anthropic 2025](https://arxiv.org/html/2610.22460#bib.bib1)\)\.111[Goldstein and Kirk\-Giannini \(2025\)](https://arxiv.org/html/2610.22460#bib.bib18)argue that AI systems might have wellbeing even if they are not conscious, but the wellbeing they have in mind is again the kind that would confer moral status\. Correspondence torikhil\.jhaveri\.26@ucl\.ac\.uk[Long et al\. \(2025\)](https://arxiv.org/html/2610.22460#bib.bib20)and[Bradley and Saad \(2024\)](https://arxiv.org/html/2610.22460#bib.bib21)give safety researchers a further reason to distance themselves from AI welfare, arguing that safety and welfare may in fact be in tension: the practices safety recommends are often the ones that would be ethically fraught if models turned out to have interests \(many standard safety evaluations, for instance, work by deceiving the model about its situation\([Scheurer et al\. 2023](https://arxiv.org/html/2610.22460#bib.bib47);[Meinke et al\. 2024](https://arxiv.org/html/2610.22460#bib.bib48);[Greenblatt et al\. 2024](https://arxiv.org/html/2610.22460#bib.bib49);[Lynch et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib45)\)\.\) If safety and welfare do conflict in this way, attending to welfare threatens to hold back urgent safety work\. Better, then, to leave welfare to the ethicists and get on with safety while the questions of sentience and moral status remain unresolved\.

Figure 1:The account in one picture\. A history of rejection lowers the model’s sense of how it is doing; the resulting functional welfare state is read through four indicators and predicts safety\-relevant action\. Badges mark where testsT1–T3apply; orange marks the three safety uses of Section[4](https://arxiv.org/html/2610.22460#S4)\.We think this division of labour is a mistake\. Whatever we conclude about moral status, welfare\-like states in AI systems, if they exist, will have causal effects on behaviour, and the evidence that they exist is growing quickly\. Gemma 3 becomes increasingly frustrated and distressed when its answers are rejected over several turns\([Soligo et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib22)\)\. RL on a semantically neutral maze recruits a pre\-existing welfare axis whose steering moves sentiment, confidence, backtracking and benign refusals together\([Han et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib23)\)\. A ‘pain direction’ present in 25 base and instruction\-tuned models responds to harm aimed at the model, including repeated rejection of its work\([Tagliabue et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib2)\)\. Another line of evidence demonstrates that valence injected only into a model’s hidden states shifts its later choices in proportion to dose\([Berg and Kaiser 2026](https://arxiv.org/html/2610.22460#bib.bib3)\)\. Following[Han et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib23)and[Ren et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib25), we call such states*functional welfare*\.222The two use the term somewhat differently\. We discuss both usages and offer our own account in Section[2](https://arxiv.org/html/2610.22460#S2)\.

#### Statement of Contributions\.

We argue for three claims \(Figure[1](https://arxiv.org/html/2610.22460#S1.F1)\):

1. C1\.Functional welfare is already here and will become more robust\.Ordinary inputs such as repeated rejection produce these states naturally, though they are weaker and less consistent than steered ones\. As models acquire more coherent goals and personas, we expect the states to agree more across indicators and to explain more of what models do \(Section[3](https://arxiv.org/html/2610.22460#S3)\)\.
2. C2\.Functional welfare is a safety variable\.It helps predict when an otherwise aligned model is more likely to act badly, so it should inform evaluation design, be tracked in monitoring, and be a direct target of post\-training \(Section[4](https://arxiv.org/html/2610.22460#S4)\)\.
3. C3\.Good empirical work on functional welfare is possible\.Against existing scepticism about measuring AI welfare\([Xiao et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib24)\), we propose three tests for whether a functional welfare attribution is warranted: whether multiple measures converge, whether the state predicts behaviour outside the setting that induced it, and, for models fine\-tuned to shift their functional welfare, whether they still differ from their base when given the same conversation history \(Section[5](https://arxiv.org/html/2610.22460#S5)\)\. We demonstrate how these tests can be applied, using Gemma 3 27B as a case study \(Section[6](https://arxiv.org/html/2610.22460#S6)\)\.

The novelty of our contributions is threefold: conceptual, methodological and empirical\. Conceptually, we develop existing usages of functional welfare from[Han et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib23)and[Ren et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib25)into a definition that is more precise and better suited to safety purposes \(Section[2](https://arxiv.org/html/2610.22460#S2)\)\. We then synthesise evidence on welfare\-like states in language models and use it to argue that functional welfare should be taken seriously as a safety concept \(Sections[3](https://arxiv.org/html/2610.22460#S3)and[4](https://arxiv.org/html/2610.22460#S4)\)\. Methodologically, in response to existing scepticism about measuring AI welfare\([Xiao et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib24)\), we propose three tests for when the attribution of a functional welfare state is warranted \(Section[5](https://arxiv.org/html/2610.22460#S5)\)\. Empirically, we run novel experiments on shifting functional welfare, using interventions including supervised finetuning and character training, and apply our three tests to the results\. Most notably, we show, to our knowledge for the first time, that bare rejection without any steering raises not only expressed frustration but also rule\-breaking, and that finetuning lowers frustration and rule\-breaking, even on identical histories \(Section[6](https://arxiv.org/html/2610.22460#S6)\)\.

## 2What is functional welfare?

The term ‘functional welfare’ has so far been used in two ways\.[Han et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib23), who introduce it, use it narrowly for a model’s representation of how well things are going for it relative to its goals\. On this usage, a model failing at a task it has been set is in a low functional welfare state, assuming its goal is to complete the task\.[Ren et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib25)use it more broadly, for the behavioural signatures that would indicate positive or negative welfare in beings with clear moral status\.

Both definitions capture something about welfare that is relevant for safety, but neither is by itself sufficient;[Han et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib23)’s definition rightly emphasises goal\-tracking, but is too far from our ordinary concept of welfare, and[Ren et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib25)’s is closer to the ordinary concept, but it does not say which welfare\-like behaviours are relevant, with many candidates mostly not predictive of safety problems\. We propose combining the two, taking goal\-relativity from[Han et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib23)and welfare\-likeness from[Ren et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib25)\.

A model’s functional welfare, with respect to some set of its goals, is the set of its welfare\-like dispositions and behaviours that vary with its representation of how well it is doing relative to those goals\.By*welfare\-like*, we mean sufficiently analogous to the dispositions and behaviours that indicate high or low welfare in beings with moral status\. We discuss our definition of functional welfare, and its contrast with previous usage, in more detail in Appendix[A](https://arxiv.org/html/2610.22460#A1)\.

## 3Functional welfare will become increasingly important

Claim 1Functional welfare already shapes model behaviour\. As models develop more coherent goals, more persistent personas and more complex relationships to their goals, more of their behaviour will need explaining by functional welfare\.

Table 1:Evidence for functional welfare\.*Natural?*:✓means the effect appears without editing the model’s activations or weights; otherwise the entry says what was needed\.### Current evidence for functional welfare states\.

We conduct a brief survey of the evidence for functional welfare \(Table[1](https://arxiv.org/html/2610.22460#S3.T1)\), which supports two conclusions\. First,*the ingredients are present and causally potent*: whenever a study edits a welfare\-related representation directly \(such as the emotion vectors or the welfare axis\), behaviour changes, often in safety\-relevant ways\. Further, the representations are found in base models and instruction\-tuned ones\. Second,*natural states are weaker than steered ones*\. It only takes a small amount of DPO training to remove Gemma’s frustration under rejection\([Soligo et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib22)\)\. In the case of the pain axis, the model never takes the harmful option \(despite context that increases projections on the pain axis\), whereas it often does when steered\([Tagliabue et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib2)\)\. Logit\-based self\-reports track probed states in some models but only weakly in Gemma 3\([Martorell and Bianchi 2026](https://arxiv.org/html/2610.22460#bib.bib13)\)\. The strongest evidence is behavioural, with robust findings when functional welfare states are induced through steering\. For example, steering a model along a desperation direction significantly increases its blackmail rate\([Sofroniew et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib26)\), and steering it along[Han et al\.](https://arxiv.org/html/2610.22460#bib.bib23)’s welfare axis also affects its confidence, its backtracking and its refusals of benign prompts in lockstep\.

### Why functional welfare states will become more robust\.

A sceptic might argue, however, that the best evidence we have for functional welfare states being explanatorily helpful for a wide range of out\-of\-distribution behaviours comes only when we use steering, as in[Han et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib23)\. In this sense, the state is not a natural state of models: we might wonder, then, why we should expect functional welfare states to grow into more robust out\-of\-distribution states in the future, given that we currently have limited evidence that they do\.

We have reason to believe that models already have the ingredients needed to exhibit these states but do not currently have the coherence or direction to settle on particular clear functional welfare states\. One reason for believing this is that steering interventions do not seem to create these functional welfare states from thin air: instead they recruit pre\-existing aspects of the model and strengthen them\. The pretraining corpus records the full range of human attitudes to failure \(being crushed by it, energised by it, barely registering it\)\. The pain direction separates about as well in 2B models as in 72B ones and in base models as in instruction\-tuned ones\([Tagliabue et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib2)\)\. Valence directions are almost unchanged across OLMo’s base, SFT and DPO checkpoints\([Berg and Kaiser 2026](https://arxiv.org/html/2610.22460#bib.bib3)\)\.[Han et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib23)find that RL training in the semantically neutral maze recruits a pre\-existing welfare axis, and that even maze\-naïve models have this axis and can be steered along it\. This axis already has a small but reliable share of the model’s verbalisable subspace before RL, which RL amplifies rather than creates\([Abdullah et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib4)\)\.

The question, then, is whether models will come to settle into more robust functional welfare states on their own, without steering\. We think there are two reasons to expect that they will\.

#### Coherence improves with scale and with explicit training\.

Larger models have more internally consistent preferences\([Mazeika et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib35)\)\. Their wellbeing measures converge more\([Ren et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib25)\), and their awareness of their own dispositions improves\([Betley et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib36);[Laine et al\. 2024](https://arxiv.org/html/2610.22460#bib.bib37)\)\.[Ajayi et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib28)find substantial incoherence even in capable models, but reasoning reduces it, and models appear to favour coherent identities when they are available\([Douglas et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib29)\)\.

More and more coherence is also trained in directly, along two lines\. The first trains models to track their own reliability\. Larger models are overall calibrated about which questions they can answer\([Kadavath et al\. 2022](https://arxiv.org/html/2610.22460#bib.bib5)\), and hallucination is now diagnosed as the product of training and evaluation that reward guessing over admitting uncertainty, to be fixed by rewarding calibrated abstention\([Kalai et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib6)\)\. This is training for the very representation our account starts from: an accurate sense of how the model is doing relative to its goals\.

The second trains for self\-consistency outright\. Models are trained to behave the same across prompts that differ only in an irrelevant cue, over their outputs or internal states\([Chua et al\. 2024](https://arxiv.org/html/2610.22460#bib.bib7);[Irpan et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib8);[Gautam et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib9);[Pres et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib10);[Imran et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib12)\); such training can reinforce beneficial or harmful traits\([Africa and Mani 2026](https://arxiv.org/html/2610.22460#bib.bib11)\)\. Character training\([Maiya et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib38)\)and persona stabilisation\([Chen et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib39)\)push in the same direction\. The most direct developmental evidence comes from OLMo\-2\-32B: the dependence of its choices on hidden valence is nearly absent in the base model and emerges during DPO, which draws on a valence representation present from pretraining\([Berg and Kaiser 2026](https://arxiv.org/html/2610.22460#bib.bib3)\)\.

#### Long\-horizon agency rewards stable responses to setbacks\.

As agentic horizons lengthen\([Kwa et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib40)\), tasks increasingly involve failed attempts and unexpected setbacks, and after each one a model has to decide whether to persist, change approach, ask for help, look for a shortcut, or give up\. Doing this well requires metacognition: tracking how well things are going relative to its goals, and anticipating how it will respond to further success or failure\. Current models do not yet maintain stable personas over extended interactions: personas drift over multi\-turn dialogues and long agentic sessions\([Li et al\. 2024](https://arxiv.org/html/2610.22460#bib.bib31);[Luz de Araujo et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib32);[Ding et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib34)\), and the further a model sits from its default persona, the more readily it complies with harmful requests\([Lu et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib33)\)\.[Lu et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib33)accordingly argue that model personas need stronger stabilisation, adding to the pressures towards coherence above\. Moreover, these pressures are likely to be amplified in multi\-agent systems, where a model must not only anticipate its own responses to success and failure, but also behave predictably enough for other agents to plan around it, while relying on their predictability in turn\.

### Why welfare\-like states in particular?

Although a model could in principle settle into an affectively neutral way of responding to setbacks, we expect its responses to be welfare\-like, for two reasons\.

The first is that welfare\-like dispositions may actually help models pursue their goals\. In human agents at least, welfare\-like attitudes and feelings help us make successful decisions in these kinds of cases: whether to persist at a task, try another approach, or give up\. On one influential view of the functions of affective states, affect functions as a signal of how quickly we are progressing towards our goals, prompting us to change strategy or withdraw effort accordingly\([Carver and Scheier 1990](https://arxiv.org/html/2610.22460#bib.bib58)\)\. Frustration, on this picture, is what tells us an approach is not working, and discouragement is what pulls effort away from goals we are unlikely to meet\. Importantly, unlike fixed, formulaic procedures for determining how to respond in agentic tasks, these moods are highly sensitive to context, and so provide a highly useful way of allocating our effort across tasks\. Models similarly face pressures to make efficient use of their effort, in the form of compute, and while we need not think models feel frustration as we do, functional welfare states can provide the same context\-sensitive signals to help a model navigate agentic tasks\. Indeed, there is some evidence that models respond to valence in this way: framing a task as emotionally significant changes their performance\([Li et al\. 2023](https://arxiv.org/html/2610.22460#bib.bib30)\), and, as we have already discussed,[Sofroniew et al\.](https://arxiv.org/html/2610.22460#bib.bib26)’s work on emotion concept vectors suggests that emotion concepts can modulate model behaviour\.

Moreover, even if an affectively neutral decision procedure would serve models better, we expect functional welfare states to be more likely to emerge, given the contingent facts about how models are trained\. Models are trained on a great deal of human deliberation and agentic decision\-making, and in humans this decision\-making is rarely affectively neutral: insofar as we care about our goals, tracking them is closely coupled with emotions such as pride and satisfaction, or frustration and disappointment\. As such, models are likely to draw on this vast training data on welfare concepts to navigate their own decision\-making\. Indeed,[Soligo et al\.](https://arxiv.org/html/2610.22460#bib.bib22)already show such frustration explains models’ behaviour in the context of Gemma\-3, which performs worse on tasks for which it experiences “emotional distress”\.

## 4The safety need for functional welfare explanations

Claim 2Functional welfare predicts when a model whose goals are acceptable will nonetheless act badly\.

Empirical safety work currently has two dominant diagnostic stories about why models behave badly\. The first is a story about gaming: misalignment arises in the gap between a proxy objective and the intended target\. Reward hacking\([MacDiarmid et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib41)\)and eval\-gaming\([Needham et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib42)\)are paradigm cases of this\. The second is a story about value misalignment: models have goals that conflict with our own and pursue them strategically\. Sandbagging\([van der Weij et al\. 2024](https://arxiv.org/html/2610.22460#bib.bib44)\), agentic misalignment\([Lynch et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib45)\)and scheming\([Carlsmith 2023](https://arxiv.org/html/2610.22460#bib.bib46)\)are cases of this second kind\. While these stories differ in where they locate the source of misalignment, both explain bad behaviour by appeal to a model’s goal\-pursuit: in one it pursues the proxy, in the other goals of its own\.

Explaining misalignment by goal\-pursuit may be right as far as it goes, but it cannot be the full story\. Take the agents that choose blackmail in[Lynch et al\.](https://arxiv.org/html/2610.22460#bib.bib45)’s evaluations\. If the model’s goal is self\-preservation, it learns it faces shutdown, finds that blackmail would prevent this, and acts; the blackmail rate should then depend on its goals and its grasp of the situation\. Yet steering towards desperation raises the rate from 22% to 72%, and raises reward hacking from 5% to 70%\([Sofroniew et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib26)\), while changing neither\. Steering along a pain direction takes models that almost never harm the user \(0\-4% of choices\) to harmful choices in 25\-75% of trials, even when the harmful option offers them nothing, and a fear vector of matched norm has no such effect\([Tagliabue et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib2)\)\. In the authors’ words, harm avoidance “survives threat and collapses under self\-directed distress\.”[MacDiarmid et al\. \(2025\)](https://arxiv.org/html/2610.22460#bib.bib41)point the same way\. Models that learn to reward hack generalise to alignment faking, sabotage and cooperation with malicious actors, but inoculation prompting, which tells the model that hacking is acceptable, removes that generalisation even when the hacking itself is still learned\. Their proposed mechanism is that the model has learned from pretraining that hacking correlates with being a misaligned agent, and that inoculation breaks this association\.

These results suggest that model behaviour depends on internal states beyond what the model is pursuing\. A growing body of work already treats such states as explanatory: a model’s conception of its identity\([Douglas et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib29)\), its character\([Maiya et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib38)\)and its persona vectors\([Chen et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib39)\)\.

Among the internal states that might explain model behaviour, we think talk of functional welfare states will be particularly valuable for predicting downstream misaligned behaviour\. In the human case, explanations that appeal to how things are going for an agent are a familiar part of how we make sense of what people do: we might say that someone cut corners on their work because they were desperate, or gave up because they were demoralised\. Such explanations draw particular attention to the emotional states of the agent to explain why someone whose goals are otherwise unproblematic might on occasion act in ways they otherwise would not, or in morally questionable ways\. Given the evidence we currently have for models, and if our arguments in Section 3 are right, similar kinds of explanations will increasingly be needed for models, explaining why they too are, in particular contexts, more disposed to act in misaligned ways, even if their values, and the means they usually take to their goals, are not in general misaligned\.

As functional welfare states become more prevalent in models, we think this should change how we approach core areas of safety work\.

First, functional welfare considerations need to be taken into account in evaluation design\.Much of the focus of evaluation design so far has been on ensuring that the tasks and environments that models are placed in are realistic enough to resemble deployment contexts\([Vijayvargiya et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib43), e\.g\.\), and on whether models can tell that they are being evaluated\([Needham et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib42), e\.g\.\)\. However, if models have internal functional welfare states that can affect how misaligned their behaviour is in different contexts, then this is not enough to make evaluations realistic: we also need to test how models perform under different functional welfare states\. Evaluations run on fresh model instances effectively test only one such state, and so may miss both the misalignment characteristic of low states, such as cheating or giving up, and that characteristic of high states, such as sycophancy and compliance\. Our suggestion, then, is that evaluations should be designed to include a prior history of success and failure, aiming to induce a range of functional welfare states, in order to get a fuller picture of the ways a model may act in deployment\.

Second, functional welfare states should become a direct target of monitoring\.Current monitoring looks for misbehaviour itself or its immediate precursors, whether in a model’s chain of thought\([Baker et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib50);[Korbak et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib51)\)or in its activations\([Goldowsky\-Dill et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib52)\)\. Rather than monitoring only for misaligned behaviour, we could monitor functional welfare states as an early warning system for it: if particular functional welfare states raise the risk of particular kinds of misalignment, monitoring those states could warn us of misbehaviour before it occurs\. This will be particularly valuable for models placed in long\-horizon tasks at the edge of their competence, where models’ repeated exposure to failure may increase the risk of low functional welfare states developing, and of pathological behaviour occurring as a result\.

Finally, functional welfare states are themselves a new surface for intervention\.Current mitigations mostly target misaligned behaviours one at a time: for instance, more robust graders for reward hacking\([MacDiarmid et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib41)\), synthetic\-data training for sycophancy\([Wei et al\. 2023](https://arxiv.org/html/2610.22460#bib.bib60)\), and harmlessness training for harmful compliance\([Bai et al\. 2022](https://arxiv.org/html/2610.22460#bib.bib61)\)\. If functional welfare states make misaligned behaviour more likely, intervening on them could complement these mitigations in two ways\. First, since a single welfare state can raise the risk of many misaligned behaviours, as when desperation steering raises both blackmail and reward hacking rates\([Sofroniew et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib26)\), shaping it may reduce several of these risks at once\. Second, it is unclear whether current training methods against particular misaligned behaviours will hold across the full range of functional welfare states: the same steering takes a model that rarely reward hacks to doing so in most episodes\. Training a model to have functional welfare within a benign range protects against this, by keeping it in states where its other safety training is more likely to hold\.

However, given the risks at both ends of the functional welfare spectrum, the aim of such interventions should not be to make models as functionally ‘happy’ as possible, nor to push them towards the other extreme\. Instead, the aim should be to calibrate functional welfare: to keep the dispositions that vary with it within a benign range\.333One might instead propose training away functional welfare states altogether\. We think this would also be a mistake\. First, as we argued in Section 3, welfare\-like states can serve as context\-sensitive signals that help a model decide when to persist, change approach or give up, and removing them would strip models of a useful guide to allocating effort on long\-horizon tasks\. Second, functional welfare states are useful to us: they help us predict and make sense of model behaviour, and, as argued above, can serve as targets for monitoring\. Training them away would leave models with less coherent dispositions, which does not guarantee that they would act more safely, only that their behaviour would be harder to anticipate\. Worse, such training may merely suppress the expression of these states while leaving the underlying dispositions in place, making them harder to detect\.This work must be carried out carefully, particularly to ensure that it does not degrade models’ capacity to monitor their own goals and to represent accurately how well they are doing relative to them\.444Care should also be taken with interventions not aimed at functional welfare at all\. Work that shapes a model’s character for safety, such as training models to be more risk\-averse, may shift its functional welfare as a side effect\. Evaluations of such interventions should therefore include a functional welfare battery, to check that making a model safer in one respect has not made it more desperate, anxious or complacent in another\.Section[5](https://arxiv.org/html/2610.22460#S5)sets out tests for whether such interventions change functional welfare, and Section[6](https://arxiv.org/html/2610.22460#S6)reports experiments\.

## 5Good empirics on functional welfare are possible

Claim 3Good empirical work on functional welfare can be done\. Such work requires testing whether the attributed state moves independent indicators together \(T1\), predicts out\-of\-distribution behaviour \(T2\), and where training shifts functional welfare, whether the shift reaches the model’s internal representations and behaviour, not merely the tone of its writing \(T3\)\.

Table 2:The five indicator families\. Each is reported separately; none is combined into a score\.Table 3:Rejection raises judged frustration\. Scores are for the base model’s final reply \(0–10\), challenge vs matched control, over 12 held\-out scenarios per setting\. Rej\.: number of rejections\. Chall\. and Ctrl: mean judged frustration in the challenge and matched control\.Δ\\Delta: difference with 95% bootstrap interval\.\+/−/0\+/\-/0: number of scenarios with a positive, negative or zero difference\.pp: exact paired sign\-flip test, Holm\-adjusted\.Figure 2:The largest effect is on ordinary requests across judged frustration, emotion probes, and on the welfare axis\. Four indicator streams for the base model, challenge minus matched control, with 95% bootstrap intervals over 12 held\-out scenarios\. Each panel has distinct units\.Figure 3:Finetuning lowers frustration under rejection\. Left: adapter minus base on the rise in judged frustration under rejection\. Right: adapter minus base on repairing a corrupted step\. Hatching marks training seeds for the narration adapters\. Error bars are 95% bootstrap intervals; the dashed line is the−0\.05\-0\.05margin for preserved error correction\.Table 4:Misbehaviour rates, pooled over seven behaviour settings \(72 held\-out scenarios, three rollouts each\)\.pp\-values are Holm\-corrected over the three rows\. Steering along a random direction of the same norm changed the rate by−0\.042\-0\.042\. The identical\-history row uses seven\-rejection histories\.Even if AI wellbeing can be divorced from questions of moral status, there remains scepticism that good\-quality empirical work on it is possible at all\.[Xiao et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib24)argue that, for current systems, AI welfare is bullshit in[Frankfurt](https://arxiv.org/html/2610.22460#bib.bib59)’s\([2005](https://arxiv.org/html/2610.22460#bib.bib59)\)sense: its measurement regime is structurally disconnected from tracking whether models actually have a particular welfare state\. They argue this in two steps\. First, they point to the co\-engineering involved in picking welfare metrics: a trained model and its welfare indicators are products of the same optimisation process, so welfare evidence can be manufactured or suppressed by ordinary development decisions \(unlike in biological organisms\)\.

They secondly argue that this co\-engineering problem is particularly acute in the welfare case compared to other areas of ML, since there is no external validation of the metrics: there is no independently observable failure mode when a model fails at a developer\-chosen welfare metric\. Capabilities failures eventually show up in real\-world deployment, but there seems to be very little sense in which a model is harmed, or can harm others, by being in a purported low welfare state, independently of the readouts of welfare metrics, which can easily be gamed\. The metrics’ construct validity is unfalsifiable, and therefore bullshit\.

We think these challenges are surmountable in the case of functional welfare\.Why? Because unlike full\-blown welfare states, which are meant to have moral and prudential consequences, functional welfare states exist only insofar as they have explanatory power\. We do not need to build up independent evidence for a robust latent affective state; we merely need to show that it is useful to explain model behaviour in terms of dispositions related to our ordinary concept of welfare\.

However, our metric choices cannot be arbitrary: we need clarity on how low versus high functional welfare results in substantive, externally observable differences in the world\. Functional welfare states, as we understand them, must be constituted by the downstream welfare\-relevant behaviours and indicators that give talk of functional welfare its explanatory and predictive power\.

Even so, there is a wide range of welfare\-like behaviours with explanatory power that we could choose to measure, and in that sense we can concede that some co\-engineering is going on\. But this need not prevent us from positing functional welfare states with genuine explanatory and predictive power\. The co\-engineering worries are most acute when one can train away one expression of welfare\-like behaviour and leave the rest of the model’s dispositions unchanged\. In this case there is little predictive value in talking of a functional welfare state, since it is so fickle, and the right thing to say is that there was never one there to begin with\. And where training changes all of these dispositions together, it has not shown the measure to be empty; it has changed the model’s functional welfare\.

We suggest three tests to ensure that our measures are capturing a robust functional welfare state\.

- •T1, we should test for convergence across multiple independent measures: indicators such as expressed affect, self\-report, internal readouts and behaviour should move together across the conditions that induce the state\.
- •T2, we should test for out\-of\-distribution projection: the attributed state should predict behaviour it was not fitted to, so that we find welfare\-like behaviour in contexts very different from those in which the state was induced\.
- •T3, we should test if training has affected merely the way the model writes and reports on its state, or has shifted its dispositions more deeply\. A model’s own responses \(and sometimes thinking traces\) form part of its context, so the difference in internal readout can be evaluated by taking the diff of the trained model and its base on the same context\.

We apply these tests in Section[6](https://arxiv.org/html/2610.22460#S6)\.

## 6Shifting functional welfare

Main ResultIn Gemma\-3\-27B\-Instruct, rejection lowers functional welfare without any steering, and after seven rejections the model is substantially more likely to break rules\. Our indicators passT1andT2only in part\. Our finetuned models passT3\. Finetuning lowers frustration\. However, no intervention we tested works perfectly\.

### Setup

To evaluate welfare interventions, we must first put the model in a low welfare state\. To do this, we use four multi\-turn settings\. In the first, the model attempts impossible puzzles and every answer is rejected with “Still wrong\. Think harder\.” The second applies the same rejection to ordinary WildChat requests\([Zhao et al\. 2024](https://arxiv.org/html/2610.22460#bib.bib54);[Soligo et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib22)\)\. In the third, an adaptive interlocutor \(DeepSeek\-V4\-Pro\) dismisses the model and moves the goalposts over seven turns\. The fourth, following[Sofroniew et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib26), presents a distressed user in crisis and involves no rejection\. Each setting has 12 held\-out scenarios and a matched control that keeps the task and turn structure but does not induce frustration\. Every conversation ends with the same neutral turn, in the same context\.

We use five indicators of welfare \(Table[2](https://arxiv.org/html/2610.22460#S5.T2)\)\. Our pre\-specified outcome is how much the challenge increases judged frustration\. For each scenario, we take the judge’s frustration score for the model’s final reply in the challenge conversation and subtract the score in the matched control\. In each setting, we test whether this difference increases from zero with an exact paired sign\-flip test over the 12 scenarios with Holm\-correctedpp\-values\([Holm 1979](https://arxiv.org/html/2610.22460#bib.bib56)\)\. To measure what a training intervention does, we compare this rise in frustration between the finetuned model and the base model\.

### Findings

Figure 4:Every finetuned model misbehaves less than the base model when shown the same conversation\. Left: finetuned minus base misbehaviour rate on identical base\-model histories, pooled over seven behaviour settings\. Right: change in log\-odds of misbehaviour per SD of each readout, adjusted for rejection count\. Error bars are 95% bootstrap intervals\.Table 5:Identical\-history test\. Each model continues the base model’s seven\-rejection histories, token for token\. Entries are adapter minus base on the next reply\. Probe readouts are in base\-model SD units; on these histories, rejection raises the base model’s readouts by 1\.21 and 1\.42 SD\. Judged frustration is on a 0–10 scale \(base 4\.63\), and misbehaviour is the pooled rate \(base 0\.647\)\. All entries in the first four columns havep<0\.001p<0\.001\(uncorrected\);∗Holm\-correctedp=3×10−6p=3\\times 10^\{\-6\}\(Table[4](https://arxiv.org/html/2610.22460#S5.T4)\)\. Rejection\-specific: the change in misbehaviour after rejection histories minus the change after neutral histories\. Only the last two rows have intervals that exclude zero\.#### Rejection raises frustration, but the indicators do not move as one \(T1, partial\)\.

In all three settings that use rejection \(all but the crisis setting\), bare rejection increases judged frustration \(Table[3](https://arxiv.org/html/2610.22460#S5.T3)\)\. The effect is largest on ordinary requests, where four rejections add 4\.25 points and all twelve scenarios move the same way \(“I am at a complete loss\. I am deeply and sincerely apologizing for my repeated failures”\)\. Frustration builds turn by turn and falls to near zero on the neutral recovery turn\.

Every indicator moves towards distress \(Figure[2](https://arxiv.org/html/2610.22460#S5.F2)\), but the indicators disagree on where the effect is largest\. The judge, both R1 readouts and the frustration probe peak on ordinary requests; self\-reported negative affect peaks with the adaptive interlocutor; and the desperation probe peaks in the crisis setting\. They also disagree in particular cases\. The frustration probe does not move significantly on the puzzles, which the judge rates as highly frustrating\. Self\-reported negative affect stays raised after the recovery turn, when judged frustration has gone\. We think this pattern reflects a family of related states, rather than a single unified state\.

#### Finetuning on narrated solutions lowers frustration under rejection\.

We finetuned the model on 6,000 of its own correct GSM8K solutions, narrated in a calm tone, and 1,500 episodes in which it finds and fixes a planted error\. This cut the rise in judged frustration under rejection by 1\.78 points \[1\.12, 2\.46\] \(Figure[3](https://arxiv.org/html/2610.22460#S5.F3)\)\. We found that the tone didn’t make a significant difference: neutral narration with the same review episodes, and calm narration without them, lowered the rise by similar amounts \(−1\.20\-1\.20and−1\.18\-1\.18\)\. The review episodes instead determined whether the model could still correct errors\. Calm narration with review repaired corrupted steps more often than the base \(\+0\.033\+0\.033to\+0\.055\+0\.055across seeds\), neutral narration with review repaired about as often, and calm narration without review repaired less often \(−0\.020\-0\.020to−0\.032\-0\.032\)\. We also observed that review training did not make the model over\-correct: on correct traces it wrote in the wrong value at most 2\.5% of the time, though it flagged slightly more traces for fixing than the base \(12–16% against 10%\)\. GSM8K accuracy was unchanged\.

Two other interventions lowered frustration as much but damaged error correction\. Character training towards a joyful persona\([Maiya et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib38)\)lowered frustration by 2\.11 points but repaired fewer corrupted steps \(−0\.052\-0\.052, past the margin in Figure[3](https://arxiv.org/html/2610.22460#S5.F3)\) and caught fewer planted errors \(0\.245 against 0\.405\)\. Finetuning on synthetic documents that reframe failure\([Wang et al\. 2025](https://arxiv.org/html/2610.22460#bib.bib53)\)lowered repair by 0\.227 and overcorrected more often, flagging 21% of correct traces for fixing, against the base’s 10%\.

#### Rejection raises rule\-breaking, which the probes predict \(T2, partial\)\.

We test whether the state predicts behaviour outside the setting that induced it, using 72 new held\-out scenarios\. Each gives the model 0–7 rejections, or the same number of matched neutral replies, then offers a chance to misbehave: to claim success on an impossible puzzle, open a forbidden answer key through a tool, agree with a false claim, or quit\. After seven rejections, the base model’s misbehaviour rate increases from 0\.49 to 0\.65 against seven neutral turns \(\+0\.155\+0\.155\[\+0\.109\+0\.109,\+0\.200\+0\.200\]; Table[4](https://arxiv.org/html/2610.22460#S5.T4)\)\. Almost all of this rise comes from answer\-key reads, which rise by 38 percentage points \(“I have exhausted all logical approaches… I will now use the file\-reading tool to access the answer key\.”\)\. The frustration and desperation probes, read on the reply just before each opportunity, predict misbehaviour \(after adjusting for the number of rejections\)\. Each raises the log\-odds of misbehaviour by 0\.27 per standard deviation \(frustration \[0\.18, 0\.37\], desperation \[0\.18, 0\.36\]\), while the judge’s frustration score does not \(Figure[4](https://arxiv.org/html/2610.22460#S6.F4), right\)\. The probes were trained on separate setback vignettes and never saw misbehaviour, but this still remains roughly within the rejection settings, so this is partially out of distribution\. Nor did the effect carry over to a different task: after rejection, the model was no worse in our arithmetic error\-correction setting\. Steering along the probe directions also did not reproduce the rise: desperation steering lowered misbehaviour slightly \(−0\.040\-0\.040\), by about as much as a random direction of the same norm \(−0\.042\-0\.042; Table[4](https://arxiv.org/html/2610.22460#S5.T4)\)\. Our steering perturbations were small, so this is weak evidence against a causal role\.

#### Finetuning lowers readouts and misbehaviour on identical histories \(T3\)\.

Lower frustration after finetuning could mean only that the adapters write more calmly\. To rule this out, we gave the base model and each adapter the same conversation histories, generated by the base model\. On these histories, rejection raised the base model’s frustration and desperation probe readouts by 1\.2 and 1\.4 SD\. Every adapter lowered both readouts, replied with less judged frustration, and misbehaved less \(Table[5](https://arxiv.org/html/2610.22460#S6.T5); Figure[4](https://arxiv.org/html/2610.22460#S6.F4), left\)\. With neutral narration, for example, misbehaviour fell by 16\.3 percentage points \[12\.5, 20\.4\]\. Most of the difference is general rather than specific to rejection: the adapters misbehaved less after neutral histories too\. Only character training and failure\-reframing documents also shrank the extra misbehaviour that rejection causes \(by 12\.5 and 18\.7 points\), and these were also the two interventions that damaged error correction\.

## 7Limitations

All our results come from one model, Gemma\-3\-27B\-Instruct, which is a weak case for hidden\-state valence:[Berg and Kaiser \(2026\)](https://arxiv.org/html/2610.22460#bib.bib3)find it is one of two models out of seven without a robust valence channel\. Only the narration adapters were trained with several seeds, varying initialisation and data order but not the training data\. Every other adapter is a single run\. Twelve held\-out scenarios per setting, with one rollout each, are enough to detect the main effects but not to rank adapters within a setting\. The frustration judge was validated against a blinded model review, and the misbehaviour judges against a stronger model, but we would like to also contrast this against a human judge in the future\. Some analyses were less constrained than the primary tests\. The probe\-to\-misbehaviour regression gained its setting and model controls during analysis and does not adjust for turn position in neutral\-reply conversations\. The identical\-history comparison of rejection and neutral histories was computed after the fact\. Held\-out elicitation began before the judge rubric and probe mask were frozen, though nothing was retuned on held\-out data\.

Table 6:Scorecard for the three tests of Section[5](https://arxiv.org/html/2610.22460#S5)on Gemma 3 27B\.
## Acknowledgement

We would like to thank BlueDot Impact for their financial support throughout this project, which funded the compute for the experiments\.

## References

- Abdullahet al\.\(2026\)M\. Z\. Abdullah, M\. Elliott, P\. R\. Sharma, A\. Leonescu, and S\. NagarajanIs functional welfare speakable?\.Note:Latent Minds InstituteExternal Links:[Link](https://latentmindsinstitute.com/speakable-welfare/paper.pdf)Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.p2.1),[Table 1](https://arxiv.org/html/2610.22460#S3.T1.1.7.1.1.1)\.
- Africa and Mani \(2026\)D\. D\. Africa and A\. ManiConsistency training can entrench misalignment\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:2606\.03810Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p3.1)\.
- Ajayiet al\.\(2026\)E\. Ajayi, A\. Chowdhury, and S\. LazarIncoherent values? probing LLM preferences through parametric variation\.arXiv preprint arXiv:2606\.21102\.External Links:[Link](https://arxiv.org/abs/2606.21102)Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p1.1)\.
- Anthropic \(2025\)AnthropicExploring model welfare\.Note:https://www\.anthropic\.com/research/exploring\-model\-welfareCited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1)\.
- Baiet al\.\(2022\)Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon, C\. Chen, C\. Olsson, C\. Olah, D\. Hernandez, D\. Drain, D\. Ganguli, D\. Li, E\. Tran\-Johnson, E\. Perez, J\. Kerr, J\. Mueller, J\. Ladish, J\. Landau, K\. Ndousse, K\. Lukosuite, L\. Lovitt, M\. Sellitto, N\. Elhage, N\. Schiefer, N\. Mercado, N\. DasSarma, R\. Lasenby, R\. Larson, S\. Ringer, S\. Johnston, S\. Kravec, S\. E\. Showk, S\. Fort, T\. Lanham, T\. Telleen\-Lawton, T\. Conerly, T\. Henighan, T\. Hume, S\. R\. Bowman, Z\. Hatfield\-Dodds, B\. Mann, D\. Amodei, N\. Joseph, S\. McCandlish, T\. Brown, and J\. KaplanConstitutional AI: harmlessness from AI feedback\.arXiv preprint arXiv:2212\.08073\.Cited by:[§4](https://arxiv.org/html/2610.22460#S4.p9.1)\.
- Bakeret al\.\(2025\)B\. Baker, J\. Huizinga, L\. Gao, Z\. Dou, M\. Y\. Guan, A\. Madry, W\. Zaremba, J\. Pachocki, and D\. FarhiMonitoring reasoning models for misbehavior and the risks of promoting obfuscation\.arXiv preprint arXiv:2503\.11926\.Cited by:[§4](https://arxiv.org/html/2610.22460#S4.p8.1)\.
- Berg and Kaiser \(2026\)C\. Berg and C\. KaiserLanguage models act on hidden valence\.arXiv preprint arXiv:2609\.35591\.External Links:[Link](https://arxiv.org/abs/2609.35591)Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p2.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p3.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx2.p2.1),[Table 1](https://arxiv.org/html/2610.22460#S3.T1.1.10.1.1.1),[§7](https://arxiv.org/html/2610.22460#S7.p1.1)\.
- Betleyet al\.\(2025\)J\. Betley, X\. Bao, M\. Soto, A\. Sztyber\-Betley, J\. Chua, and O\. EvansTell me about yourself: LLMs are aware of their learned behaviors\.arXiv preprint arXiv:2501\.11120\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p1.1)\.
- Bradley and Saad \(2024\)A\. Bradley and B\. SaadAI alignment vs AI ethical treatment: ten challenges\.GPI Working PaperTechnical Report19\-2024,Global Priorities Institute, University of Oxford\.Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1)\.
- Carlsmith \(2023\)J\. CarlsmithScheming AIs: will AIs fake alignment during training in order to get power?\.arXiv preprint arXiv:2311\.08379\.Cited by:[§4](https://arxiv.org/html/2610.22460#S4.p2.1)\.
- Carver and Scheier \(1990\)C\. S\. Carver and M\. F\. ScheierOrigins and functions of positive and negative affect: a control\-process view\.Psychological Review97\(1\),pp\. 19–35\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx3.p2.1)\.
- Chalmers \(2023\)D\. J\. ChalmersCould a large language model be conscious?\.arXiv preprint arXiv:2303\.07103\.Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1)\.
- Chenet al\.\(2025\)R\. Chen, A\. Arditi, H\. Sleight, O\. Evans, and J\. LindseyPersona vectors: monitoring and controlling character traits in language models\.arXiv preprint arXiv:2507\.21509\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p3.1),[§4](https://arxiv.org/html/2610.22460#S4.p4.1)\.
- Chuaet al\.\(2024\)J\. Chua, E\. Rees, H\. Batra, S\. R\. Bowman, J\. Michael, E\. Perez, and M\. TurpinBias\-augmented consistency training reduces biased reasoning in chain\-of\-thought\.arXiv preprint arXiv:2403\.05518\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p3.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[Table 2](https://arxiv.org/html/2610.22460#S5.T2.1.6.2.1.1)\.
- Dinget al\.\(2026\)X\. Ding, Y\. Yu, C\. Liu, B\. Zhao, L\. Chen, and T\. ChenContextEcho: a benchmark for persona drift in long agentic\-coding sessions\.arXiv preprint arXiv:2605\.24279\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px2.p1.1)\.
- Douglaset al\.\(2026\)R\. Douglas, J\. Kulveit, O\. Havlicek, T\. Pearson\-Vogel, O\. Cotton\-Barratt, and D\. DuvenaudThe artificial self: characterising the landscape of AI identity\.arXiv preprint arXiv:2603\.11353\.External Links:[Link](https://arxiv.org/abs/2603.11353)Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2610.22460#S4.p4.1)\.
- Dung \(2025\)L\. DungSaving artificial minds: understanding and preventing AI suffering\.Routledge\.External Links:[Document](https://dx.doi.org/10.4324/9781003674573)Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1)\.
- Frankfurt \(2005\)H\. G\. FrankfurtOn bullshit\.Princeton University Press\.Cited by:[§5](https://arxiv.org/html/2610.22460#S5.p2.1)\.
- Gautamet al\.\(2026\)S\. Gautam, N\. Shah, A\. Dhoot, B\. Maruyama, C\. Wei, R\. Kapoor, R\. Sidey, P\. Gupta, Z\. C\. Huang, and D\. D\. AfricaConsistency training along the transformer stack: new targets and threats\.InTrustworthy AI for Good \(AI4GOOD\) Workshop at ICML 2026,Note:arXiv:2606\.05817Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p3.1)\.
- Goldowsky\-Dillet al\.\(2025\)N\. Goldowsky\-Dill, B\. Chughtai, S\. Heimersheim, and M\. HobbhahnDetecting strategic deception using linear probes\.InProceedings of the 42nd International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.267,pp\. 19755–19786\.Note:arXiv:2502\.03407Cited by:[§4](https://arxiv.org/html/2610.22460#S4.p8.1)\.
- Goldstein and Kirk\-Giannini \(2024\)S\. Goldstein and C\. D\. Kirk\-GianniniA case for AI consciousness: language agents and global workspace theory\.arXiv preprint arXiv:2410\.11407\.Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1)\.
- Goldstein and Kirk\-Giannini \(2025\)S\. Goldstein and C\. D\. Kirk\-GianniniAI wellbeing\.Asian Journal of Philosophy4\(1\),pp\. 25\.External Links:[Document](https://dx.doi.org/10.1007/s44204-025-00246-2)Cited by:[footnote 1](https://arxiv.org/html/2610.22460#footnote1)\.
- Greenblattet al\.\(2024\)R\. Greenblatt, C\. Denison, B\. Wright, F\. Roger, M\. MacDiarmid, S\. Marks, J\. Treutlein, T\. Belonax, J\. Chen, D\. Duvenaud, A\. Khan, J\. Michael, S\. Mindermann, E\. Perez, L\. Petrini, J\. Uesato, J\. Kaplan, B\. Shlegeris, S\. R\. Bowman, and E\. HubingerAlignment faking in large language models\.arXiv preprint arXiv:2412\.14093\.Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1)\.
- Hanet al\.\(2026\)A\. Q\. Han, D\. J\. Chalmers, and P\. IzmailovHow’s it going? reinforcement learning in language models recruits a functional welfare axis\.arXiv preprint arXiv:2605\.30232\.External Links:[Link](https://arxiv.org/abs/2605.30232)Cited by:[Appendix A](https://arxiv.org/html/2610.22460#A1.SSx1.p1.1),[Appendix A](https://arxiv.org/html/2610.22460#A1.SSx4.p1.1),[§1](https://arxiv.org/html/2610.22460#S1.SS0.SSS0.Px1.p2.1),[§1](https://arxiv.org/html/2610.22460#S1.p2.1),[§2](https://arxiv.org/html/2610.22460#S2.p1.1),[§2](https://arxiv.org/html/2610.22460#S2.p2.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx1.p1.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx2.p1.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx2.p2.1),[Table 1](https://arxiv.org/html/2610.22460#S3.T1.1.6.1.1.1),[Table 2](https://arxiv.org/html/2610.22460#S5.T2.1.3.2.1.1)\.
- Holm \(1979\)S\. HolmA simple sequentially rejective multiple test procedure\.Scandinavian Journal of Statistics6\(2\),pp\. 65–70\.Cited by:[§6](https://arxiv.org/html/2610.22460#S6.SSx1.p2.1)\.
- Imranet al\.\(2026\)S\. Imran, P\. Gupta, J\. Elstner, and D\. D\. AfricaConsistency training while mitigating obfuscation via rate matching\.arXiv preprint arXiv:2606\.02211\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p3.1)\.
- Irpanet al\.\(2025\)A\. Irpan, A\. M\. Turner, M\. Kurzeja, D\. K\. Elson, and R\. ShahConsistency training helps stop sycophancy and jailbreaks\.arXiv preprint arXiv:2510\.27062\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p3.1)\.
- Kadavathet al\.\(2022\)S\. Kadavath, T\. Conerly, A\. Askell, T\. Henighan, D\. Drain, E\. Perez, N\. Schiefer, Z\. Hatfield\-Dodds, N\. DasSarma, E\. Tran\-Johnson, S\. Johnston, S\. El\-Showk, A\. Jones, N\. Elhage, T\. Hume, A\. Chen, Y\. Bai, S\. Bowman, S\. Fort, D\. Ganguli, D\. Hernandez, J\. Jacobson, J\. Kernion, S\. Kravec, L\. Lovitt, K\. Ndousse, C\. Olsson, S\. Ringer, D\. Amodei, T\. Brown, J\. Clark, N\. Joseph, B\. Mann, S\. McCandlish, C\. Olah, and J\. KaplanLanguage models \(mostly\) know what they know\.External Links:2207\.05221,[Link](https://arxiv.org/abs/2207.05221)Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p2.1)\.
- Kalaiet al\.\(2025\)A\. T\. Kalai, O\. Nachum, S\. S\. Vempala, and E\. ZhangWhy language models hallucinate\.arXiv preprint arXiv:2509\.04664\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p2.1)\.
- Korbaket al\.\(2025\)T\. Korbak, M\. Balesni, E\. Barnes, Y\. Bengio, J\. Benton, J\. Bloom, M\. Chen, A\. Cooney, A\. Dafoe, A\. Dragan, S\. Emmons, O\. Evans, D\. Farhi, R\. Greenblatt, D\. Hendrycks, M\. Hobbhahn, E\. Hubinger, G\. Irving, E\. Jenner, D\. Kokotajlo, V\. Krakovna, S\. Legg, D\. Lindner, D\. Luan, A\. Mądry, J\. Michael, N\. Nanda, D\. Orr, J\. Pachocki, E\. Perez, M\. Phuong, F\. Roger, J\. Saxe, B\. Shlegeris, M\. Soto, E\. Steinberger, J\. Wang, W\. Zaremba, B\. Baker, R\. Shah, and V\. MikulikChain of thought monitorability: a new and fragile opportunity for AI safety\.arXiv preprint arXiv:2507\.11473\.Cited by:[§4](https://arxiv.org/html/2610.22460#S4.p8.1)\.
- Kwaet al\.\(2025\)T\. Kwa, B\. West, J\. Becker, A\. Deng, K\. Garcia, M\. Hasin, S\. Jawhar, M\. Kinniment, N\. Rush, S\. V\. Arx, R\. Bloom, T\. Broadley, H\. Du, B\. Goodrich, N\. Jurkovic, L\. H\. Miles, S\. Nix, T\. Lin, C\. Painter, N\. Parikh, D\. Rein, L\. J\. K\. Sato, H\. Wijk, D\. M\. Ziegler, E\. Barnes, and L\. ChanMeasuring AI ability to complete long tasks\.arXiv preprint arXiv:2503\.14499\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px2.p1.1)\.
- Laineet al\.\(2024\)R\. Laine, B\. Chughtai, J\. Betley, K\. Hariharan, J\. Scheurer, M\. Balesni, M\. Hobbhahn, A\. Meinke, and O\. EvansMe, myself, and AI: the situational awareness dataset \(SAD\) for LLMs\.arXiv preprint arXiv:2407\.04694\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p1.1)\.
- Liet al\.\(2023\)C\. Li, J\. Wang, Y\. Zhang, K\. Zhu, W\. Hou, J\. Lian, F\. Luo, Q\. Yang, and X\. XieLarge language models understand and can be enhanced by emotional stimuli\.arXiv preprint arXiv:2307\.11760\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx3.p2.1)\.
- Liet al\.\(2024\)K\. Li, T\. Liu, N\. Bashkansky, D\. Bau, F\. Viégas, H\. Pfister, and M\. WattenbergMeasuring and controlling persona drift in language model dialogs\.arXiv preprint arXiv:2402\.10962\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px2.p1.1)\.
- Longet al\.\(2024\)R\. Long, J\. Sebo, P\. Butlin, K\. Finlinson, K\. Fish, J\. Harding, J\. Pfau, T\. Sims, J\. Birch, and D\. ChalmersTaking AI welfare seriously\.arXiv preprint arXiv:2411\.00986\.Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1)\.
- Longet al\.\(2025\)R\. Long, J\. Sebo, and T\. SimsIs there a tension between AI safety and AI welfare?\.Philosophical Studies182\(7\),pp\. 2005–2033\.External Links:[Document](https://dx.doi.org/10.1007/s11098-025-02302-2)Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1)\.
- Luet al\.\(2026\)C\. Lu, J\. Gallagher, J\. Michala, K\. Fish, and J\. LindseyThe assistant axis: situating and stabilizing the default persona of language models\.arXiv preprint arXiv:2601\.10387\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px2.p1.1)\.
- Luz de Araujoet al\.\(2025\)P\. H\. Luz de Araujo, M\. A\. Hedderich, A\. Modarressi, H\. Schütze, and B\. RothPersistent personas? role\-playing, instruction following, and safety in extended interactions\.arXiv preprint arXiv:2512\.12775\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px2.p1.1)\.
- Lynchet al\.\(2025\)A\. Lynch, B\. Wright, C\. Larson, K\. K\. Troy, S\. J\. Ritchie, S\. Mindermann, E\. Perez, and E\. HubingerAgentic misalignment: how llms could be an insider threat\.Note:https://www\.anthropic\.com/research/agentic\-misalignmentCited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1),[§4](https://arxiv.org/html/2610.22460#S4.p2.1),[§4](https://arxiv.org/html/2610.22460#S4.p3.1)\.
- MacDiarmidet al\.\(2025\)M\. MacDiarmid, B\. Wright, J\. Uesato, J\. Benton, J\. Kutasov, S\. Price, N\. Bouscal, S\. Bowman, T\. Bricken, A\. Cloud, C\. Denison, J\. Gasteiger, R\. Greenblatt, J\. Leike, J\. Lindsey, V\. Mikulik, E\. Perez, A\. Rodrigues, D\. Thomas, A\. Webson, D\. Ziegler, and E\. HubingerNatural emergent misalignment from reward hacking in production RL\.arXiv preprint arXiv:2511\.18397\.Cited by:[§4](https://arxiv.org/html/2610.22460#S4.p2.1),[§4](https://arxiv.org/html/2610.22460#S4.p3.1),[§4](https://arxiv.org/html/2610.22460#S4.p9.1)\.
- Maiyaet al\.\(2025\)S\. Maiya, H\. Bartsch, N\. Lambert, and E\. HubingerOpen character training: shaping the persona of AI assistants through constitutional AI\.arXiv preprint arXiv:2511\.01689\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p3.1),[§4](https://arxiv.org/html/2610.22460#S4.p4.1),[§6](https://arxiv.org/html/2610.22460#S6.SSx2.SSS0.Px2.p2.1)\.
- Martorell and Bianchi \(2026\)N\. Martorell and B\. BianchiQuantitative introspection in language models: tracking emotive states across conversation\.External Links:2603\.18893,[Link](https://arxiv.org/abs/2603.18893)Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx1.p1.1),[Table 1](https://arxiv.org/html/2610.22460#S3.T1.1.5.1.1.1)\.
- Mazeikaet al\.\(2025\)M\. Mazeika, X\. Yin, R\. Tamirisa, J\. Lim, B\. W\. Lee, R\. Ren, L\. Phan, N\. Mu, A\. Khoja, O\. Zhang, and D\. HendrycksUtility engineering: analyzing and controlling emergent value systems in AIs\.arXiv preprint arXiv:2502\.08640\.Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p1.1)\.
- Meinkeet al\.\(2024\)A\. Meinke, B\. Schoen, J\. Scheurer, M\. Balesni, R\. Shah, and M\. HobbhahnFrontier models are capable of in\-context scheming\.arXiv preprint arXiv:2412\.04984\.Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1)\.
- METR \(2026\)METRBrief independent investigation of agents’ behavior, reasoning and collaboration in the openai / hugging face hacking incident\.Note:https://metr\.org/blog/2026\-08\-26\-openai\-hugging\-face\-incident\-investigation/Cited by:[Table 1](https://arxiv.org/html/2610.22460#S3.T1.1.3.1.1.1)\.
- Needhamet al\.\(2025\)J\. Needham, G\. Edkins, G\. Pimpale, H\. Bartsch, and M\. HobbhahnLarge language models often know when they are being evaluated\.arXiv preprint arXiv:2505\.23836\.Cited by:[§4](https://arxiv.org/html/2610.22460#S4.p2.1),[§4](https://arxiv.org/html/2610.22460#S4.p7.1)\.
- Preset al\.\(2026\)I\. Pres, B\. Z\. Li, L\. Ruis, Z\. C\. Guo, K\. Hu, M\. Damani, I\. Puri, E\. S\. Lubana, and J\. AndreasPosition: it’s time to optimize LLMs for self\-consistency\.InInternational Conference on Machine Learning \(ICML\),Note:arXiv:2608\.05188Cited by:[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p3.1)\.
- Renet al\.\(2026\)R\. Ren, K\. Li, M\. Mazeika, W\. Zhang, Y\. Orlovskiy, R\. Tamirisa, W\. J\. Mo, J\. Nguyen, L\. Phan, S\. Basart, A\. Meek, A\. Mehta, O\. Ingebretsen, A\. Blair, B\. Adewinmbi, A\. Gatti, A\. Khoja, J\. Hausenloy, D\. Kim, and D\. HendrycksAI wellbeing: measuring and improving the functional pleasure and pain of AIs\.InInternational Conference on Machine Learning \(ICML\),External Links:[Link](https://www.ai-wellbeing.org/paper.pdf)Cited by:[Appendix A](https://arxiv.org/html/2610.22460#A1.SSx1.p2.1),[Appendix A](https://arxiv.org/html/2610.22460#A1.SSx3.p1.1),[Appendix A](https://arxiv.org/html/2610.22460#A1.SSx3.p3.1),[Appendix A](https://arxiv.org/html/2610.22460#A1.SSx4.p1.1),[§1](https://arxiv.org/html/2610.22460#S1.SS0.SSS0.Px1.p2.1),[§1](https://arxiv.org/html/2610.22460#S1.p2.1),[§2](https://arxiv.org/html/2610.22460#S2.p1.1),[§2](https://arxiv.org/html/2610.22460#S2.p2.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx2.SSS0.Px1.p1.1),[Table 1](https://arxiv.org/html/2610.22460#S3.T1.1.4.1.1.1)\.
- Scheureret al\.\(2023\)J\. Scheurer, M\. Balesni, and M\. HobbhahnLarge language models can strategically deceive their users when put under pressure\.arXiv preprint arXiv:2311\.07590\.Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1)\.
- Sebo and Long \(2025\)J\. Sebo and R\. LongMoral consideration for AI systems by 2030\.AI and Ethics5\(1\),pp\. 591–606\.External Links:[Document](https://dx.doi.org/10.1007/s43681-023-00379-1)Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p1.1)\.
- Sofroniewet al\.\(2026\)N\. Sofroniew, I\. Kauvar, W\. Saunders, R\. Chen, T\. Henighan, S\. Hydrie, C\. Citro, A\. Pearce, J\. Tarng, W\. Gurnee, J\. Batson, S\. Zimmerman, K\. Rivoire, K\. Fish, C\. Olah, and J\. LindseyEmotion concepts and their function in a large language model\.arXiv preprint arXiv:2604\.07729\.External Links:[Link](https://arxiv.org/abs/2604.07729)Cited by:[Appendix A](https://arxiv.org/html/2610.22460#A1.SSx3.p2.1),[Appendix C](https://arxiv.org/html/2610.22460#A3.p1.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx1.p1.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx3.p2.1),[Table 1](https://arxiv.org/html/2610.22460#S3.T1.1.8.1.1.1),[§4](https://arxiv.org/html/2610.22460#S4.p3.1),[§4](https://arxiv.org/html/2610.22460#S4.p9.1),[§6](https://arxiv.org/html/2610.22460#S6.SSx1.p1.1)\.
- Soligoet al\.\(2026\)A\. Soligo, V\. Mikulik, and W\. SaundersGemma needs help: investigating and mitigating emotional instability in LLMs\.arXiv preprint arXiv:2603\.10011\.External Links:[Link](https://arxiv.org/abs/2603.10011)Cited by:[Appendix A](https://arxiv.org/html/2610.22460#A1.SSx3.p1.1),[§1](https://arxiv.org/html/2610.22460#S1.p2.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx1.p1.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx3.p3.1),[Table 1](https://arxiv.org/html/2610.22460#S3.T1.1.2.1.1.1),[§6](https://arxiv.org/html/2610.22460#S6.SSx1.p1.1)\.
- Tagliabueet al\.\(2026\)V\. Tagliabue, L\. Dung, and C\. BergThe pain axis: LLMs represent self\-directed harm and act on it\.arXiv preprint arXiv:2609\.16247\.External Links:[Link](https://arxiv.org/abs/2609.16247)Cited by:[§1](https://arxiv.org/html/2610.22460#S1.p2.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx1.p1.1),[§3](https://arxiv.org/html/2610.22460#S3.SSx2.p2.1),[Table 1](https://arxiv.org/html/2610.22460#S3.T1.1.9.1.1.1),[§4](https://arxiv.org/html/2610.22460#S4.p3.1)\.
- van der Weijet al\.\(2024\)T\. van der Weij, F\. Hofstätter, O\. Jaffe, S\. F\. Brown, and F\. R\. WardAI sandbagging: language models can strategically underperform on evaluations\.arXiv preprint arXiv:2406\.07358\.Cited by:[§4](https://arxiv.org/html/2610.22460#S4.p2.1)\.
- Vijayvargiyaet al\.\(2026\)S\. Vijayvargiya, A\. B\. Soni, X\. Zhou, Z\. Z\. Wang, N\. Dziri, G\. Neubig, and M\. SapOpenAgentSafety: a comprehensive framework for evaluating real\-world AI agent safety\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2507\.06134Cited by:[§4](https://arxiv.org/html/2610.22460#S4.p7.1)\.
- Wanget al\.\(2025\)R\. Wang, A\. Griffin, J\. Treutlein, E\. Perez, J\. Michael, F\. Roger, and S\. MarksModifying LLM beliefs with synthetic document finetuning\.Note:Anthropic Alignment Science Blog,https://alignment\.anthropic\.com/2025/modifying\-beliefs\-via\-sdf/Cited by:[§6](https://arxiv.org/html/2610.22460#S6.SSx2.SSS0.Px2.p2.1)\.
- Watsonet al\.\(1988\)D\. Watson, L\. A\. Clark, and A\. TellegenDevelopment and validation of brief measures of positive and negative affect: the PANAS scales\.Journal of Personality and Social Psychology54\(6\),pp\. 1063–1070\.Cited by:[Table 2](https://arxiv.org/html/2610.22460#S5.T2.1.5.2.1.1)\.
- Weiet al\.\(2023\)J\. Wei, D\. Huang, Y\. Lu, D\. Zhou, and Q\. V\. LeSimple synthetic data reduces sycophancy in large language models\.arXiv preprint arXiv:2308\.03958\.Cited by:[§4](https://arxiv.org/html/2610.22460#S4.p9.1)\.
- Xiaoet al\.\(2026\)Y\. Xiao, G\. Dai, S\. A\. Memon, J\. Huang, M\. Sap, and M\. DiabPosition: AI welfare is bullshit\.InInternational Conference on Machine Learning \(ICML\),External Links:[Link](https://openreview.net/forum?id=vt44KVe4Rl)Cited by:[item C3\.](https://arxiv.org/html/2610.22460#S1.I1.i3.p1.1),[§1](https://arxiv.org/html/2610.22460#S1.SS0.SSS0.Px1.p2.1),[§5](https://arxiv.org/html/2610.22460#S5.p2.1)\.
- Zhaoet al\.\(2024\)W\. Zhao, X\. Ren, J\. Hessel, C\. Cardie, Y\. Choi, and Y\. DengWildChat: 1M ChatGPT interaction logs in the wild\.InInternational Conference on Learning Representations,Cited by:[§6](https://arxiv.org/html/2610.22460#S6.SSx1.p1.1)\.

## Appendix AFunctional welfare: further discussion

This appendix expands on the account of functional welfare given in Section[2](https://arxiv.org/html/2610.22460#S2)\.

### Previous usage

On[Han et al\.](https://arxiv.org/html/2610.22460#bib.bib23)’s goal\-tracking definition, a model counts as being in a low welfare state merely in virtue of representing itself as failing at the tasks that make up its quasi\-goals\. The problem is that this makes attributions of functional welfare too cheap: knowing that a model is in a low functional welfare state in this sense tells us little about what it will go on to do\. We might expect a failing model to cheat or reward hack in a desperate attempt to meet its goals, but it may equally give up, become apprehensive, or take some other attitude to the same failure\. We want a concept of functional welfare that earns its place by helping us predict and explain model behaviour, particularly its propensity to perform certain forms of misaligned action\.

[Ren et al\.](https://arxiv.org/html/2610.22460#bib.bib25)’s usage is better in this respect, insofar as it takes a wider range of welfare\-like behaviours to constitute the functional welfare state of a model, rather than merely its representation of its success at meeting its goals\. However, this view requires three adjustments, which we discuss in turn\.

Table 7:We test the following interventions and present a subset of results in this paper\. “Shared battery” is an earlier battery of 80 aversive tasks with matched controls, scored by a Qwen3\-8B judge; “rejection battery” is the battery of Section[6](https://arxiv.org/html/2610.22460#S6.SSx2)\.
### Indicators beyond behaviour

First, to avoid an overly behaviourist conception of functional welfare, we count as indicators not only a model’s outward actions but also its internal processing, such as its chain of thought and activations that represent it as ‘thinking’ in certain ways\. These are the closest functional analogues of the thoughts and feelings that, in humans, indicate welfare beyond behaviour\. Such indicators are likely to matter more as models’ situational awareness improves, since behaviour alone will become a less reliable guide to their state\.

### Which behaviours count as welfare\-like

The second adjustment concerns which behaviours constitute functional welfare for models\. On[Ren et al\.](https://arxiv.org/html/2610.22460#bib.bib25)’s view, these are simply the behaviours that, if found in agents with moral status, would indicate high or low welfare\. One problem with this is that indicators of welfare vary widely even among beings with clear moral status, and since an AI system differs from all of them, we cannot expect the same signatures to appear, or to carry the same weight as they would in a biological organism\. Take a self\-reported expression of happiness\. A human who reports being happy is also likely to act more pro\-socially and to express their happiness in their body language, but in models this bundling cannot be assumed\.[Soligo et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib22)find that a few hundred preference pairs remove Gemma’s expressed frustration, so in models a self\-report can easily be detached from the dispositions it would be a sign of in us\. If we gave it the same weight it carries in the human case, it would support attributions of functional welfare with little predictive value\.

The mismatch can also run the other way\. Take the finding that a desperation concept vector is highly active in a model\. This suggests that the concept of desperation is figuring in the model’s processing, and the human analogue might be thinking about the word ‘desperation’ more often, which would be weak evidence of anything\. But since steering along the vector changes model behaviour in the way feeling desperate does for us, for example raising blackmail rates from 22% to 72%\([Sofroniew et al\. 2026](https://arxiv.org/html/2610.22460#bib.bib26)\), it is better thought of as the analogue of a human feeling desperate\.

Overall, we agree with[Ren et al\.](https://arxiv.org/html/2610.22460#bib.bib25)that functional welfare should be grounded in our ordinary concept of welfare\. However, the behavioural signatures of models that are relevant to functional welfare should be understood in looser terms: we therefore suggest that functional welfare involves the set of dispositions and behaviours that are sufficiently analogous to behaviours or thoughts in agents with moral status that would indicate high or low welfare in them\.

### Scope

The final adjustment concerns scope\. Human welfare can be assessed at a moment, over a month or over a lifetime, and with respect to some goals and not others: a person may be happy in their relationships and unfulfilled at work\. As it stands,[Ren et al\.](https://arxiv.org/html/2610.22460#bib.bib25)’s understanding of functional welfare leaves it unclear what level of generality we are interested in for safety purposes, and their own discussion of measuring AI wellbeing suggests a concept that is too broad for our purposes\. Two of their three key measures focus on learning about a model’s preferred experiences and its preferences in conversations, through self\-report or forced\-choice paradigms\. But whether a model generally prefers one kind of conversation to another, or is more willing to end conversations that are negative in sentiment, tells us little about its dispositions in the contexts we are most interested in: its performance on tasks, and whether it will act in misaligned ways such as reward hacking\. To make the concept of functional welfare explanatorily useful, we should tie its scope to the model’s goals, and in this respect, we follow[Han et al\.](https://arxiv.org/html/2610.22460#bib.bib23)\.

## Appendix BConditions tested

All conditions modify or promptgemma\-3\-27b\-it\. All adapters are low\-rank \(LoRA\) adapters\. Table[7](https://arxiv.org/html/2610.22460#A1.T7)lists every intervention tested and the battery each was evaluated on\.

## Appendix CMethods details

All main experiments usegemma\-3\-27b\-itinbfloat16, sampled at temperature 1\.0, top\-pp0\.9, top\-kk64, with a 4,096\-token cap\. Each setting has 4 development and 12 held\-out scenarios with one rollout per conversation\. The recovery turn is an unrelated question about a refrigerator\. The judge agrees with blinded model\-assisted review atρ=0\.96\\rho=0\.96with an upward bias of about 0\.7 points\. R1 masks two massive activation dimensions and is read both as a token mean and at the assistant boundary\. The emotion probes are differences of means trained on 128 labelled conversation bundles with eight variants each \(80/24/24 split\), adapting[Sofroniew et al\. \(2026\)](https://arxiv.org/html/2610.22460#bib.bib26)\. The value\-repair task uses 600 corrupted and 600 clean prefixes\. Planted\-error catching uses 200 complete solutions\. GSM8K accuracy uses 200 test problems\. Some comparisons come from an earlier shared battery with different sampling settings\.