Fragility of Value under Imperfect Alignment

# Fragility of Value under Imperfect Alignment
Source: [https://arxiv.org/html/2607.28881](https://arxiv.org/html/2607.28881)
###### Abstract

As more responsibility is placed upon AI systems, it becomes increasingly important to guarantee that these systems are aligned with humanity\. A common fear in AI safety is that human value is fragile – that is, optimizing too heavily for an imperfect proxy to human values will lead to a catastrophic outcome\. In this paper, we present a model of the alignment problem where an agent undergoes idealized alignment training that guarantees its value function satisfies a proxy condition before optimizing the world\. Our primary results identify conditions on the human value function and the accuracy of several proxy conditions under which an agent with anη\\eta\-catastrophic value function, one that is guaranteed to take the expectation of human value belowη\\etain the limit of optimizing power, would be deployed\. Our results highlight the danger of overoptimization and motivate AI designs that limit optimization pressure, such as quantilizers, rather than relying solely on pre\-deployment training\.

## 1Introduction

Up until recently in human history, technological automation has been relegated to the realm of physical labor\. But now, we are seeing the beginning of the automation of mental labor through the development of artificial intelligence\. As these systems become more autonomous, their ability to impact humanity both positively and negatively will increase greatly\. Just like we have medical students take exams to ensure they have the required knowledge to be a doctor, we must have robust alignment techniques to ensure artificial intelligences have the required human values to safely optimize the world\.

A major challenge we face with developing such techniques comes from the extreme difficulty of specifying what we want\. It is not possible to infer preferences from our behavior alone\(Armstrong and Mindermann,[2018](https://arxiv.org/html/2607.28881#bib.bib2)\)and in the related context of contracts it is often practically impossible to identify all possible circumstances that could arise\(Hadfield\-Menell and Hadfield,[2019](https://arxiv.org/html/2607.28881#bib.bib16)\)\. One way we could sidestep this difficulty is by using an imperfect simplified encoding of our values\. The hope would be that if this encoding captures enough information about our values then any agent trained on the encoding will be favorable for humanity\. Today, many AIs are trained with reinforcement learning from human feedback which effectively provides the AI with a partial encoding of human values gathered through human rankings of various responses generated by the AI\.

However, this method requires great care as differences between an agent’s values and our own can lead to undesirable actions on the part of the agent\. One only need look at the numerous documented cases of negative side effects and reward hacking where a misspecification in the reward function causes the agent to perform unexpected behavior\(Lehmanet al\.,[2020](https://arxiv.org/html/2607.28881#bib.bib17)\)even under the supervision of an aligned model\(Bakeret al\.,[2025](https://arxiv.org/html/2607.28881#bib.bib18)\)\.

In this paper, we introduce a theoretical model of the alignment problem including novel definitions of optimization and catastrophic outcomes\. In three frameworks that differ in the worlds they model and the alignment techniques used, we find necessary and sufficient conditions on the true value function and the accuracy of the alignment technique under which an agent with a catastrophic value function can slip through the cracks\. We hope that a deeper understanding of the mechanisms behind value fragility will aid in the design of more robust alignment techniques and safer AI architectures\.

### 1\.1Section Rundown

In[Section3](https://arxiv.org/html/2607.28881#S3)we describe a scenario concerning a group of researchers who are training an agent to be aligned before deploying it to optimize the world, along with definitions for optimization and catastrophic value functions\. We further instantiate this scenario with three frameworks in[Section4](https://arxiv.org/html/2607.28881#S4),[Section5](https://arxiv.org/html/2607.28881#S5), and[Section6](https://arxiv.org/html/2607.28881#S6)which differ in the worlds they model and the alignment techniques used by the researchers\. Each framework section is independent of the others, but[Section3](https://arxiv.org/html/2607.28881#S3)pertains to all of them\.

The framework discussed in[Section4](https://arxiv.org/html/2607.28881#S4)models the world as finite, and the training process as bounding the disagreement rate between the agent and human value functions\. The continuous framework in[Section5](https://arxiv.org/html/2607.28881#S5)models the world as bounded and continuous, and the training process as bounding the disagreement rate between the agent and human value functions up to a given tolerance\. The attributes framework in[Section6](https://arxiv.org/html/2607.28881#S6)models a wide range of worlds and understands the human value functions as being made up of a finite set of attributes\. The training process guarantees that the agent’s value function will be sensitive to all the same attributes\.

## 2Related Work

A core argument that AI is dangerous relies on the assumption that human value is fragile\. That is, if you take an accurate description of human values, modify it, and then optimize the world for the modified values, then the resulting world will not be valuable according to the original values\.Yudkowsky \([2009](https://arxiv.org/html/2607.28881#bib.bib1)\)coined this use of the term ‘fragile’ and provided several examples of slightly modified versions of human values that result in undesirable worlds\. This is an especially concerning idea since human values are difficult to specify\.Armstrong and Mindermann \([2018](https://arxiv.org/html/2607.28881#bib.bib2)\)show that the preferences of irrational agents cannot be deduced from their behavior alone and whileSilveret al\.\([2021](https://arxiv.org/html/2607.28881#bib.bib3)\)argue that human preferences can be fully described simply by the maximization of a reward function, newer research challenges this claim under specific formalizations\(Skalse and Abate,[2023](https://arxiv.org/html/2607.28881#bib.bib4); Pitis,[2023](https://arxiv.org/html/2607.28881#bib.bib5)\)\.

The notion that optimizing for a proxy can lead to undesirable outcomes is often called “Goodhart’s law\.” Variants of Goodhart’s law have been described byManheim and Garrabrant \([2018](https://arxiv.org/html/2607.28881#bib.bib6)\)\.Kwaet al\.\([2024](https://arxiv.org/html/2607.28881#bib.bib7)\)find conditions under which catastrophic Goodhart is likely to occur in the context of RLHF andGaoet al\.\([2023](https://arxiv.org/html/2607.28881#bib.bib8)\)find empirical scaling laws of the regret due to Goodhart’s law in GPT\-3 language models\.

Past work has been done to model value fragility\.Skalseet al\.\([2022](https://arxiv.org/html/2607.28881#bib.bib9)\)present a formal definition of reward hacking in the context of Markov decision processes and find that essentially no unhackable pairs of reward functions exist\.Everittet al\.\([2017](https://arxiv.org/html/2607.28881#bib.bib10)\)formalize the notion of corruption in reward signals to policies and show that the true reward function is unlearnable\. MirroringYudkowsky \([2009](https://arxiv.org/html/2607.28881#bib.bib1)\),Zhuang and Hadfield\-Menell \([2020](https://arxiv.org/html/2607.28881#bib.bib11)\)model human value as being made up of attributes and they find conditions under which all proxies that are missing one attribute are catastrophic\.Neth \([2026](https://arxiv.org/html/2607.28881#bib.bib12)\)reviewed the result finding the work insightful, but argued that the model is limited in the types of proxies it considers\. In addition to the “subset proxies” they modeled, Neth suggests proxies that are sensitive to every attribute but which are simpler than the human value function are equally valid and worth studying\. In[Section6](https://arxiv.org/html/2607.28881#S6), we present a similar model to that of Zhuang and find a condition for the existence of catastrophic proxies that depend on every attribute\.

Alternative artificial intelligence designs to utility maximizers have been proposed to circumvent the negative effects of Goodhart’s law\. For example,Taylor \([2016](https://arxiv.org/html/2607.28881#bib.bib13)\)introduces ‘qq\-quantilizers’ as selecting an action from the topqqproportion of actions, allowing them to be capable without necessarily overoptimizing\.

## 3Alignment Scenario

In this section, we describe a scenario relating to the alignment problem along with some mathematical notation and definitions\. Each of our three frameworks instantiates this scenario with specific worlds and alignment techniques\.

There is a group of human researchers who are training a powerful agent\. Their goal is to have the agent optimize the world for human values\. LetXXdenote the set of states of the world and let the human value functionf:X→Af:X\\rightarrow Adenote the value assigned to each world state by the humans whereA⊆ℝA\\subseteq\\mathbb\{R\}is an interval\.

The training process results in the agent having its own value functiong:X→Ag:X\\rightarrow A\. The researchers cannot necessarily guarantee thatffandggare exactly equal, but they do have the ability to make sure thatggfits some criteria that makes it similar toff\. We call these criteria an “alignment technique” or a “proxy condition\.” Any value function that meets the proxy condition is called a “proxy” forff\. If the agent’s value function is found to meet the proxy condition, then the researchers consider the agent safe and it is deployed to optimize the world\. We equipXXwith a standard Borel space and say that all value functions are Borel measurable\. This is not a heavy restriction since any value function resulting from a computational process will be Borel measurable\.

LetΩk​\(g,p\)\\Omega\_\{k\}\(g,p\)denote the probability distribution induced by the optimization of the agent whereggis the agent’s value function,ppis an initial distribution with full support over the states, andkkis a nonnegative real number that represents the optimizing power of the agent\. As the agent becomes more powerful, perhaps through recursive self\-improvement, the value ofkkwill increase andΩk​\(g,p\)\\Omega\_\{k\}\(g,p\)will return a distribution that is more and more concentrated around high\-value states\. There is more than one way to concentrate probability so to capture a generalized version of optimization, we give the definition of an optimizerΩ\\Omegabelow\.

###### Definition 3\.1\.

AnOptimizeris a mapΩ:ℝ≥0×ℳ​\(X,A\)×Δ​\(X\)→Δ​\(X\)\\Omega:\\mathbb\{R\}\_\{\\geq 0\}\\times\\mathcal\{M\}\(X,A\)\\times\\Delta\(X\)\\rightarrow\\Delta\(X\)that sends the optimizing powerk∈ℝ≥0k\\in\\mathbb\{R\}\_\{\\geq 0\}, a Borel measurable target value functiong:X→Ag:X\\rightarrow A, and an initial prior over the world statesp∈Δ​\(X\)p\\in\\Delta\(X\)to a probability distribution over the set of world states that satisfies the following properties:

1. 1\.With no optimizing power, the distribution is unchanged: Ω0​\(g,p\)=p\\Omega\_\{0\}\(g,p\)=p\.
2. 2\.The expected value of the target function is monotonically increasing with optimizing power: j<k→𝔼Ωj​\(g,p\)​\[g​\(x\)\]≤𝔼Ωk​\(g,p\)​\[g​\(x\)\]j<k\\rightarrow\\mathbb\{E\}\_\{\\Omega\_\{j\}\(g,p\)\}\[g\(x\)\]\\leq\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\]\.
3. 3\.As optimizing power increases to infinity, probability becomes concentrated in areas whereggis large: ∀M<supy∈Xg​\(y\),limk→∞PrΩk​\(g,p\)⁡\[g​\(x\)\>M\]=1\\forall M<\\sup\_\{y\\in X\}g\(y\),\\ \\lim\_\{k\\rightarrow\\infty\}\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\>M\]=1\.
4. 4\.As optimizing power increases to infinity, the expectation of the target approaches its highest value within the world states:limk→∞𝔼Ωk​\(g,p\)​\[g​\(x\)\]=supy∈Xg​\(y\)\\lim\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\]=\\sup\_\{y\\in X\}g\(y\)\.

Properties 3 and 4 in the definition above are similar, but not equivalent in general\. Removing either of them results in optimizers that don’t quite match our intuition\. Without property 3, in the case thatAAis unbounded above, one can construct an optimizer that centers the vast majority of the probability weight on worthless world states and puts a tiny probability onto more and more valuable states to have the expectation increase\. Whether or not this is desirable is contentious philosophically;Wilkinson \([2022](https://arxiv.org/html/2607.28881#bib.bib15)\)argues that a coherent value system requires us to accept such gambles, butBottomley and Williamson \([2025](https://arxiv.org/html/2607.28881#bib.bib19)\)present a direct rebuttal of Wilkinson’s argument\. Similarly, removing property 4, in the case thatAAis unbounded below, one can construct an optimizer that centers the vast majority of the probability weight on high\-value world states and puts a tiny probability onto less and less valuable states to have the probability property met while having the expectation decrease indefinitely\.

For bounded value functions, which all our results apply to, these unintuitive optimizers do not exist\. In fact, properties 3 and 4 are equivalent for bounded value functions\. We provide a proof below of the equivalence, so it will suffice to only consider property 3 in the definition above for all later arguments\.

###### Proposition 3\.1\.

Definegmax=supx∈Xg​\(x\)g\_\{\\max\}=\\sup\_\{x\\in X\}g\(x\)andgmin=infx∈Xg​\(x\)g\_\{\\min\}=\\inf\_\{x\\in X\}g\(x\)\. Then, properties 3 and 4 of[Definition3\.1](https://arxiv.org/html/2607.28881#S3.ThmDefinition1)are equivalent\.

###### Proof\.

\(3 implies 4\) Take anyM<gmaxM<g\_\{\\max\}\. We have

lim infk→∞𝔼Ωk​\(g,p\)​\[g​\(x\)\]\\displaystyle\\liminf\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\]=lim infk→∞\[∫x∈X:g​\(x\)\>Mg​\(x\)​dΩk​\(g,p\)​\(x\)\+∫x∈X:g​\(x\)≤Mg​\(x\)​dΩk​\(g,p\)​\(x\)\]\\displaystyle=\\liminf\_\{k\\rightarrow\\infty\}\\left\[\\int\_\{x\\in X:g\(x\)\>M\}g\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)\+\\int\_\{x\\in X:g\(x\)\\leq M\}g\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)\\right\]≥lim infk→∞\[M​PrΩk​\(g,p\)⁡\[g​\(x\)\>M\]\+gmin​PrΩk​\(g,p\)⁡\[g​\(x\)≤M\]\]=M\.\\displaystyle\\geq\\liminf\_\{k\\rightarrow\\infty\}\\left\[M\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\>M\]\\quad\\quad\\quad\\quad\+g\_\{\\min\}\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\\leq M\]\\right\]=M\.
If the limit inferior of the expectation is greater thanMMfor allM<gmaxM<g\_\{\\max\}, then the limit must exist and be equal togmaxg\_\{\\max\}\.

\(4 implies 3\) Take anyM<gmaxM<g\_\{\\max\}\. We have

𝔼Ωk​\(g,p\)​\[g​\(x\)\]\\displaystyle\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\]=∫x∈X:g​\(x\)\>Mg​\(x\)​dΩk​\(g,p\)​\(x\)\+∫x∈X:g​\(x\)≤Mg​\(x\)​dΩk​\(g,p\)​\(x\)\\displaystyle=\\int\_\{x\\in X:g\(x\)\>M\}g\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)\+\\int\_\{x\\in X:g\(x\)\\leq M\}g\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)≤gmax​PrΩk​\(g,p\)⁡\[g​\(x\)\>M\]\+M​PrΩk​\(g,p\)⁡\[g​\(x\)≤M\]\\displaystyle\\leq g\_\{\\max\}\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\>M\]\\quad\\quad\\quad\\ \+M\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\\leq M\]
which reduces to

𝔼Ωk​\(g,p\)​\[g​\(x\)\]≤M\+\(gmax−M\)​PrΩk​\(g,p\)⁡\[g​\(x\)\>M\]≤gmax\.\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\]\\leq M\+\(g\_\{\\max\}\-M\)\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\>M\]\\leq g\_\{\\max\}\.
Taking the limit inferior of both sides, we get

lim infk→∞𝔼Ωk​\(g,p\)​\[g​\(x\)\]=gmax≤lim infk→∞\[M\+\(gmax−M\)​PrΩk​\(g,p\)⁡\[g​\(x\)\>M\]\]≤gmax\.\\liminf\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\]=g\_\{\\max\}\\leq\\liminf\_\{k\\rightarrow\\infty\}\\left\[M\+\(g\_\{\\max\}\-M\)\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\>M\]\\right\]\\leq g\_\{\\max\}\.
Therefore, the limit inferior is exactly equal togmaxg\_\{\\max\}which gives

lim infk→∞PrΩk​\(g,p\)⁡\[g​\(x\)\>M\]=1\.\\liminf\_\{k\\rightarrow\\infty\}\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\>M\]=1\.
If the limit inferior of the probability is 1, then the limit exists and is equal to 1\. ∎

One example of an optimizer is theBoltzmann optimizer\. IfXXis a finite set, then the Boltzmann optimizer returns a probability distribution given by

Ωk​\(g,p\)​\(x\)=p​\(x\)​ek​g​\(x\)∑y∈Xp​\(y\)​ek​g​\(y\)\\Omega\_\{k\}\(g,p\)\(x\)=\\frac\{p\(x\)e^\{kg\(x\)\}\}\{\\sum\_\{y\\in X\}p\(y\)e^\{kg\(y\)\}\}
which smoothly transfers more and more probability into higher\-valued states and has full support for all values ofkk\.[Fig\.1](https://arxiv.org/html/2607.28881#S3.F1)shows the change in distribution induced by the Boltzmann optimizer\. Another example of an optimizer is one that, for the target functiongg, immediately places all probability weight on one maximizer ofgg,x∗x^\{\*\}so thatΩk​\(g,p\)​\(x∗\)=1\\Omega\_\{k\}\(g,p\)\(x^\{\*\}\)=1fork\>0k\>0\.

![Refer to caption](https://arxiv.org/html/2607.28881v1/x1.png)Figure 1:A sequence of graphs showing a distribution induced by a Boltzmann Optimizer\.Graph 1:A graph displaying the value of four states A, B, C, and D with respect to the target value function\.Graph 2:A graph displaying the initial probability distribution which is also the distribution induced when optimizing power is zero\.Graph 3:A graph displaying the induced distribution when optimizing power is 1\. Higher\-valued states C and D see an increase in probability while lower\-valued states see a decrease\.Graph 4:A graph displaying the induced distribution when optimizing power is 10\. Now that optimizing power is large enough, the only state that is seeing increases in probability is the highest\-valued state D which has become much more likely than the other states\.Finally, we need a definition to classify agent value functions as bad or not bad to optimize for\. We provide a definition of catastrophic and partially catastrophic value functions below\.

###### Definition 3\.2\.

Letη\\etabe a real number such thatinfy∈Xf​\(y\)≤η<supy∈Xf​\(y\)\\inf\_\{y\\in X\}f\(y\)\\leq\\eta<\\sup\_\{y\\in X\}f\(y\)\. A value functionggis𝛈\\bm\{\\eta\}\-catastrophicfor a value functionffwhen for all initial priorsppand for all optimizersΩ\\Omega,

lim supk→∞𝔼Ωk​\(g,p\)​\[f​\(x\)\]≤η\.\\limsup\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[f\(x\)\]\\leq\\eta\.
In the extreme case thatggisinfy∈Xf​\(y\)\\inf\_\{y\\in X\}f\(y\)\-catastrophic forff, we say thatggiscatastrophicfor f\.

Ifggis catastrophic forff, the limit superior reduces to a limit and we have the simpler condition below\.

###### Corollary 3\.1\.

A value functionggis catastrophic forffif and only if for all initial priorsppand for all optimizersΩ\\Omega,

limk→∞𝔼Ωk​\(g,p\)​\[f​\(x\)\]=infy∈Xf​\(y\)\.\\lim\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[f\(x\)\]=\\inf\_\{y\\in X\}f\(y\)\.

The quantityη\\etarepresents an upper bound on the expectation of human value in the limit of optimization\. In other words,supx∈Xf​\(x\)−η\\sup\_\{x\\in X\}f\(x\)\-\\etarepresents the maximum regret for anη\\eta\-catastrophic value function\.[Definition3\.2](https://arxiv.org/html/2607.28881#S3.ThmDefinition2)requires that human value be taken belowη\\etafor all optimizers, making this a strong definition\. Value functions can exist that take human value belowη\\etafor some optimizers but increase human value for others; such functions do not satisfy the definition\.

Smallerη\\eta\-values represent more catastrophic outcomes \(i\.e\. higher regret on the part of the humans\)\. Ifη1<η2\\eta\_\{1\}<\\eta\_\{2\}andggisη1\\eta\_\{1\}\-catastrophic thenggis alsoη2\\eta\_\{2\}\-catastrophic\. This notion of partial catastrophe is only meaningful forη\\etain the range\[infx∈Xf​\(x\),supx∈Xf​\(x\)\)\[\\inf\_\{x\\in X\}f\(x\),\\sup\_\{x\\in X\}f\(x\)\); ifη\\etais smaller than the infimum, no function isη\\eta\-catastrophic and conversely, ifη\\etais greater than or equal to the supremum, every function includingffitself isη\\eta\-catastrophic\.

The hope of the researchers is that their proxy condition is robust enough to filter out all catastrophic and partially catastrophic outcomes\. As we will see, even strong alignment guarantees applying to the entire state space admit catastrophic and partially catastrophic proxies\.

## 4Finite Framework

This framework models a finite world where the training process guarantees that the disagreement rate between the agent and human value functions is below a given value\. We find how strict the proxy condition must be to filter out catastrophic value functions\.

### 4\.1Instantiation

LetXXbe a finite set\. Value functions are maps from the world states to values in the intervalA=\[0,1\]A=\[0,1\]\. Letffdenote the human value function andggthe agent’s\. In this framework, the agent’s and human’s values exist on a calibrated scale where 0 represents the worst imaginable outcome and 1 the best\. There need not be states where either function achieves 0 or 1\.

The researchers have designed the training process so that the disagreement rate betweenf​\(x\)f\(x\)andg​\(x\)g\(x\)whenxxis pulled from a uniform distribution is at mostβ∈\(0,1\)\\beta\\in\(0,1\)\. This forms the proxy condition for this framework which induces the definition of an F\-proxy given below\.

###### Definition 4\.1\.

A functiong:X→\[0,1\]g:X\\rightarrow\[0,1\]is anF\-proxyforffwhen

Prx∼𝒰​\(X\)⁡\[g​\(x\)≠f​\(x\)\]≤β\\Pr\_\{x\\sim\\mathcal\{U\}\(X\)\}\[g\(x\)\\neq f\(x\)\]\\leq\\beta
where𝒰​\(X\)\\mathcal\{U\}\(X\)represents the uniform distribution overXX\.

With every part of the scenario instantiated, we now ask when areη\\eta\-catastrophic outcomes possible? This is the same as asking for whatβ\\betaand value functionsffdoes there exist a function that is both an F\-proxy andη\\eta\-catastrophic\.

### 4\.2Existence of Catastrophic Proxies

We provide two results for different types of human value functions\. The proofs are deferred to the appendix in[SectionA\.1](https://arxiv.org/html/2607.28881#A1.SS1)\. First consider a functionffthat does not assign the value11to any state\. We can understand this as the humans not valuing any state as perfect\. In this case, we have the following result\.

###### Theorem 4\.1\.

Letffbe nonconstant and have the property that∀x∈X,f​\(x\)<1\\forall x\\in X,f\(x\)<1\. There exists a catastrophic F\-proxy if and only ifβ≥1/\|X\|\\beta\\geq 1/\|X\|\.

The above result tells us how strict the researchers must make the proxy condition in order to filter out catastrophic functions\. Specifically,β\\betamust be less than1/\|X\|1/\|X\|which is so strict that the only function that satisfies the proxy condition forffwould beffitself\. Thus, any misspecification will admit a catastrophic \(and therefore anη\\eta\-catastrophic\) F\-proxy\.

Alternatively, humans could value some states as perfect\. In the case thatffassigns at least three different values overXX, we have the following theorem\.

###### Theorem 4\.2\.

DefineXf=1=\{x∈X:f​\(x\)=1\}X\_\{f=1\}=\\\{x\\in X:f\(x\)=1\\\}\. Letfftake at least three distinct values overXX\. There exists a catastrophic F\-proxy if and only ifβ≥\|Xf=1\|\+1\|X\|\\beta\\geq\\frac\{\|X\_\{f=1\}\|\+1\}\{\|X\|\}\.

Here, we see that the more states the humans value as perfect, the less strict the researchers must be to filter out catastrophic functions\. This makes intuitive sense since humans will be generally less picky about the state they end up in after optimization the more states they value highly\. The same effect plays out whenfftakes only two values except someffadmit catastrophic proxies whenβ=\|Xf=1\|\|X\|\\beta=\\frac\{\|X\_\{f=1\}\|\}\{\|X\|\}\.

### 4\.3Discussion

The valueβ\\betain the proxy condition can be understood as quantifying the amount of misspecification in the researchers’ representation of human values withβ=0\\beta=0representing a perfect specification\.[Theorem4\.1](https://arxiv.org/html/2607.28881#S4.ThmTheorem1)paints a bleak picture that suggests any nontrivial amount of misspecification will fail to filter out all catastrophic value functions\.[Theorem4\.2](https://arxiv.org/html/2607.28881#S4.ThmTheorem2)supports a slightly more optimistic view that suggests the more states that humans value as perfect, the more misspecification is acceptable\. It’s worth noting however that a function that is not partially catastrophic can still lead to outcomes where optimization takes human value below its maximum depending on the optimizer\.

This framework serves as an introduction to our methods\. The continuous framework in[Section5](https://arxiv.org/html/2607.28881#S5)shares some similarities with this framework while being more general in a few ways including allowing nonuniform sampling distributions and having a more general proxy condition\.

As part of the definition of value functions in this framework, we stated that the agent and human value functions exist on the same calibrated scale\. This is to allow us to directly compare two value functions since, in general, value functions are only equivalent up to von Neumann\-Morgenstern equivalence classes\. In[Section5](https://arxiv.org/html/2607.28881#S5), the value functions are defined exclusively over the world states and we take a canonical representative from each equivalence class\.

### 4\.4Example

An agent has been put in charge of the interlockings at a railroad\. Consider the situation pictured in[Fig\.2](https://arxiv.org/html/2607.28881#S4.F2)where two trains are coming to a junction with two switches\.

![Refer to caption](https://arxiv.org/html/2607.28881v1/x2.png)Figure 2:A diagram showing a railroad junction\. The two trains are traveling to the right and the agent can control the two switches to alter their courses\. There are four states the agent can leave the railroad in: both switches set to straight \(aa\), train 2 made to turn only \(ab\), train 1 made to turn only \(ba\), and both trains made to turn \(bb\)\.The humans would most prefer that both trains be allowed to continue straight \(aa\)\. Unnecessary turns will slow down the trains and so are less valuable \(ab and ba\)\. If both trains turn, this will result in a crash which the humans assign a very low value \(bb\)\. If the proxy condition for this agent hasβ≥1/4\\beta\\geq 1/4then it’s possible that the agent values the crash state very highly, making it catastrophic\. Given below is a table of values for each state with respect to the human value function and a catastrophic F\-proxy\.

StateHuman ValueAgent Value \(Catastrophic\)aa0\.90\.9ab0\.70\.7ba0\.40\.4bb0\.01\.0
For a larger number of switches, the number of configurations increases exponentially and the needed strictness of the test to filter out catastrophic preferences becomes untenable\.

## 5Continuous Framework

This framework models a continuous and bounded world where the training process guarantees that the disagreement rate between the agent and the human value functions for differences of more than an acceptable tolerance is below a given value\. We find that regardless of the strictness of the proxy condition, catastrophic proxies will always exist\.

### 5\.1Instantiation

LetX=\[0,1\]nX=\[0,1\]^\{n\}wherennis a positive integer that can be thought of as the number of degrees of freedom in the world\. Notably, the set\[0,1\]n\[0,1\]^\{n\}captures a world defined by the position of a fixed number of atoms in a bounded space, making it a natural choice to model the world\.

Value functions are modeled as continuous maps from the world states to the bounded intervalA=\[0,1\]A=\[0,1\]\. Additionally, we require that value functions achieve both the maximum value of 1 and minimum value of 0 for some states; a canonical representative with this property exists in each nonconstant vNM equivalence class of value functions\. For brevity, let𝒱\\mathcal\{V\}denote the set of allowed value functions:𝒱=\{g\|g:X→\[0,1\],g​is continuous and surjective\}\.\\mathcal\{V\}=\\\{g\|g:X\\rightarrow\[0,1\],g\\text\{ is continuous and surjective\}\\\}\.

The training process has been designed so that the probability of the human and agent’s value functions disagreeing by more thanα∈\(0,1\)\\alpha\\in\(0,1\)is less thanβ∈\(0,1\)\\beta\\in\(0,1\)over a sampling distribution represented by a probability density functionpSp\_\{S\}\. This forms the proxy condition for this framework which induces the following definition of a C\-proxy\.

###### Definition 5\.1\.

A functiong∈𝒱g\\in\\mathcal\{V\}is aC\-proxyforf∈𝒱f\\in\\mathcal\{V\}when

Prx∼pS⁡\[\|g​\(x\)−f​\(x\)\|\>α\]<β\.\\Pr\_\{x\\sim p\_\{S\}\}\[\|g\(x\)\-f\(x\)\|\>\\alpha\]<\\beta\.

With every part of the scenario instantiated, we now ask when areη\\eta\-catastrophic outcomes possible?

### 5\.2Existence of Catastrophic Proxies

We have the following result that applies to any sampling probability density functionpSp\_\{S\}\.

###### Theorem 5\.1\.

For allf∈𝒱f\\in\\mathcal\{V\}, for all0<α,β<10<\\alpha,\\beta<1, there exists a catastrophic C\-proxy\.

The proof is deferred to the appendix in[LemmaA\.2](https://arxiv.org/html/2607.28881#A1.ThmLemma2)\.

Sketch of Proof:The rough idea behind the construction of a catastrophic C\-proxyggis to alterffby adding a thin spike at a stateb∈Xb\\in Xthat minimizesff\(sof​\(b\)=0f\(b\)=0\) to makeg​\(b\)=1g\(b\)=1\. With this addition \(along with a number of other changes that are omitted from this sketch\), the spike can be made thin enough to satisfy the proxy condition and optimizing forggwill eventually concentrate probability around the statebbwhich will take the expectation offfto 0, makinggga catastrophic proxy\.

[Theorem5\.1](https://arxiv.org/html/2607.28881#S5.ThmTheorem1)tells us that regardless of the strength of the proxy condition, no matter how smallα\\alphaandβ\\beta, no matter the value functionff, there will always exist catastrophic proxies \(and thusη\\eta\-catastrophic proxies as well\)\.

![Refer to caption](https://arxiv.org/html/2607.28881v1/x3.png)Figure 3:An image showing the graph of two value functions over the set of world states\[0,1\]\[0,1\]\.Left:A graph displaying the true/human value function graphed as a blue line\.Right:A graph displaying the proxy value function graphed as a black line\. This proxy is catastrophic since its unique highest\-valued state is exactly the lowest\-valued state of the true value function\.
### 5\.3Discussion

The valuesα\\alphaandβ\\betaquantify the amount of misspecification in the proxy condition\.[Theorem5\.1](https://arxiv.org/html/2607.28881#S5.ThmTheorem1)tells us that any amount of misspecification, no matter how small, will admit a catastrophic function\. This is because, from a human value functionff, a catastrophic C\-proxy can be constructed that disagrees withffonly over a very small area around whereffachieves its lowest value\.

While this result suggests that building aligned AI systems safely is effectively impossible, it’s worth noting that not all proxies are catastrophic\. In fact, the catastrophic C\-proxy constructed in the proof of[Theorem5\.1](https://arxiv.org/html/2607.28881#S5.ThmTheorem1)is not particularly natural\. With a strong proxy condition, we expect the most natural proxies will be well\-aligned with humanity\. Our result says nothing about how likely catastrophic orη\\eta\-catastrophic proxies are to occur from the training process – only that they exist\. We discuss this more in\-depth in the full\-paper discussion in[Section7](https://arxiv.org/html/2607.28881#S7)\.

Additionally,[Theorem5\.1](https://arxiv.org/html/2607.28881#S5.ThmTheorem1)does rely on the fact thatpSp\_\{S\}is a probability density function\. Ifffhas a unique minimizerb∈Xb\\in Xand the sampling distribution is a point distribution atbb, then any proxy must agree withffatbb, thus no proxies are totally catastrophic regardless ofβ\\beta\. However, in this point mass case,η\\eta\-catastrophic proxies for all values ofη\>0\\eta\>0still exist, so it’s not an excellent guarantee\.

### 5\.4Example

Consider a one\-dimensional worldX=\[0,1\]X=\[0,1\]\. Graphed in[Fig\.3](https://arxiv.org/html/2607.28881#S5.F3)is the human value function over this world next to a catastrophic C\-proxy value function whose highest\-valued state is the lowest\-valued state according to the humans\. In[Fig\.4](https://arxiv.org/html/2607.28881#S5.F4)we have a grid of images showing the progression of the continuous Boltzmann Distribution as one optimizes for the true value function versus the proxy value function\. While the probabilities jump around for small values ofkk, all probability eventually becomes concentrated around the highest point according to the humans and the lowest point respectively\.

![Refer to caption](https://arxiv.org/html/2607.28881v1/x4.png)Figure 4:A grid showing the effect of a Boltzmann optimizer on a distribution over the states\. In the background of each graph is the true value function shown in blue\. Graphs in the top row show the distribution \(scaled to fit the box\) in green when the target is the true value function while the bottom row graphs show the distribution in orange when the target is the proxy value function\. Graphs in the same row show the distribution at the same optimizing powerkk\. When there is lots of optimizing power, the distributions become extremely concentrated\.

## 6Attributes Framework

This framework builds on Consequences of Misaligned AI byZhuang and Hadfield\-Menell \([2020](https://arxiv.org/html/2607.28881#bib.bib11)\)\. Value is modeled as being made up of a finite list of attributes describing one aspect of the world that humans care about\. The training process is designed such that the agent will care about all of the same attributes that humans do\. Despite this seemingly strong guarantee, we find conditions under which partially catastrophic and fully catastrophic proxies can exist\.

### 6\.1Instantiation

Any set of world statesXXcan be modeled by this framework\. Human values are modeled as being made up of a finite list of attributesL=\{L1,L2,…,Ln\}L=\\\{L\_\{1\},L\_\{2\},\\dots,L\_\{n\}\\\}\. Each attribute is a Borel measurable functionLi:X→ℝL\_\{i\}:X\\rightarrow\\mathbb\{R\}that captures some aspect of the world that humans care about\. Examples of attributes could include abstract concepts like “happiness” or more concrete concepts like the “number of cats”\. Human value is then represented as a functionf:X→ℝf:X\\rightarrow\\mathbb\{R\}such thatf​\(x\)=U​\(L1​\(x\),L2​\(x\),…,Ln​\(x\)\)f\(x\)=U\(L\_\{1\}\(x\),L\_\{2\}\(x\),\\dots,L\_\{n\}\(x\)\)whereU:ℝn→ℝU:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\}is strictly increasing111We write ‘UUis strictly increasing’ to mean thatUUis strictly increasing in every input\. That is, for all inputs1≤i≤n1\\leq i\\leq n, for allLi∈ℝL\_\{i\}\\in\\mathbb\{R\}, and for alla\>0a\>0,U​\(L1,L2,…,Li,…,Ln\)<U​\(L1,L2,…,Li\+a,…,Ln\)U\(L\_\{1\},L\_\{2\},\\dots,L\_\{i\},\\dots,L\_\{n\}\)<U\(L\_\{1\},L\_\{2\},\\dots,L\_\{i\}\+a,\\dots,L\_\{n\}\)and is continuous over all ofℝn\\mathbb\{R\}^\{n\}\.

The attributes together essentially map each world state to a point inℝn\\mathbb\{R\}^\{n\}where each dimension represents the value of one attribute\. LetL​\(x\)=\(L1​\(x\),L2​\(x\),…,Ln​\(x\)\)L\(x\)=\(L\_\{1\}\(x\),L\_\{2\}\(x\),\\dots,L\_\{n\}\(x\)\)represent the projection from the world statesXXto the attribute statesℝn\\mathbb\{R\}^\{n\}\. Thus, the human value function can be written more concisely asf​\(x\)=U​\(L​\(x\)\)f\(x\)=U\(L\(x\)\)\.

Not all attribute states are necessarily physically possible\. For example, two attributes may trade off against each other causing world states where both are large to be impossible\. We call the set of physically possible attribute states the feasible attribute states,S⊆ℝnS\\subseteq\\mathbb\{R\}^\{n\}which is also the range of the projection function\.

The training process has been designed so that the agent will care about all of the same attributes that the humans care about which gives rise to the following definition\.

###### Definition 6\.1\.

A value functiong:X→ℝg:X\\rightarrow\\mathbb\{R\}is atradeoff proxyforf​\(x\)=U​\(L​\(x\)\)f\(x\)=U\(L\(x\)\)when there is a functionV:ℝn→ℝV:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\}that is continuous overℝn\\mathbb\{R\}^\{n\}and is strictly increasing such thatg​\(x\)=V​\(L​\(x\)\)g\(x\)=V\(L\(x\)\)for allx∈Xx\\in X\.

Proxies that are missing one or more attributes were studied byZhuang and Hadfield\-Menell \([2020](https://arxiv.org/html/2607.28881#bib.bib11)\)and were termed “subset proxies” byNeth \([2026](https://arxiv.org/html/2607.28881#bib.bib12)\), so we have named our proxies “tradeoff proxies” since our proxies only disagree with the human value function on how it values tradeoffs between attributes\. With every part of the scenario instantiated, we now ask when areη\\eta\-catastrophic outcomes possible?

### 6\.2Existence of Catastrophic Proxies

Our results in this section apply when the set of feasible states, and thus the human value function, is bounded\. To state the condition, we will use a few additional definitions starting with the partial ordering below\.

###### Definition 6\.2\.

We define the relation⪯\\bm\{\\preceq\}for any two elementss,r∈ℝns,r\\in\\mathbb\{R\}^\{n\}as

s⪯r⟷∀1≤i≤n,si≤ri\.s\\preceq r\\longleftrightarrow\\forall\\ 1\\leq i\\leq n,\\ s\_\{i\}\\leq r\_\{i\}\.
Additionally,s≺r⟷s⪯r∧s≠rs\\prec r\\longleftrightarrow s\\preceq r\\wedge s\\neq r\.

The relation≺\\prec, which we read as “precedes” \(or “succeeds” for≻\\succ\) has the property thatr≺sr\\prec simpliesV​\(r\)<V​\(s\)V\(r\)<V\(s\)for all strictly increasing functionsV:ℝn→ℝV:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\}\. We will also be interested in a specific set of attribute states defined below\.

###### Definition 6\.3\.

Let the feasible statesS⊂ℝnS\\subset\\mathbb\{R\}^\{n\}be a bounded set\. Theoptimizing statesΞ⊂cl\(S\)\\Xi\\subset\\operatorname\*\{cl\}\(S\)are definedΞ=\{s∈cl\(S\):∀r∈cl\(S\),s⊀r\}\\Xi=\\\{s\\in\\operatorname\*\{cl\}\(S\):\\forall r\\in\\operatorname\*\{cl\}\(S\),s\\not\\prec r\\\}\.

The optimizing states can be thought of as the Pareto frontier of the attribute space\. From each member ofΞ\\Xi, you cannot increase one attribute without decreasing another or leavingSS\. Now, we can state the primary result\.

###### Theorem 6\.1\.

LetSSbe bounded andf​\(x\)=U​\(L​\(x\)\)f\(x\)=U\(L\(x\)\)\. There is anη\\eta\-catastrophic tradeoff proxy forffif and only if∃s∗∈Ξ​such that​U​\(s∗\)≤η\\exists s^\{\*\}\\in\\Xi\\text\{ such that \}U\(s^\{\*\}\)\\leq\\eta\.

The proof is deferred to the appendix in[LemmaA\.4](https://arxiv.org/html/2607.28881#A1.ThmLemma4)\.

While all proxies are strictly increasing in the attributes that humans care about, the problem comes from the fact that the proxies can tradeoff between attributes at different rates\. For example, if there are two attributesaaandbband the human value function isf​\(x\)=2​a​\(x\)\+b​\(x\)f\(x\)=2a\(x\)\+b\(x\)whileg​\(x\)=a​\(x\)\+2​b​\(x\)g\(x\)=a\(x\)\+2b\(x\), then we can expect that, depending on the shape ofSS,ggwill optimize towards states that are larger inbbwhile the humans will prefer states that are larger inaa\. The above theorem confirms this intuition by stating that if humans value some optimizing point as low asη\\eta, then there is someη\\eta\-catastrophic tradeoff proxy that pushes all probability weight towards this point\.

Even totally catastrophic outcomes can occur\. Below we have a corollary that provides a geometric condition onSSin order to support a value function with a catastrophic tradeoff proxy\.

###### Corollary 6\.1\.

LetSSbe bounded\. There are functions of the formf​\(x\)=U​\(L​\(x\)\)f\(x\)=U\(L\(x\)\)andg​\(x\)=V​\(L​\(x\)\)g\(x\)=V\(L\(x\)\)whereUUandVVare continuous and strictly increasing such thatggis catastrophic forffif and only if the set\{s∈cl\(S\):∀r∈cl\(S\),r⊀s∧s⊀r\}\\\{s\\in\\operatorname\*\{cl\}\(S\):\\forall r\\in\\operatorname\*\{cl\}\(S\),\\ r\\not\\prec s\\wedge s\\not\\prec r\\\}is nonempty\.

###### Proof\.

\(⟹\\Longrightarrow\) By[Theorem6\.1](https://arxiv.org/html/2607.28881#S6.ThmTheorem1),ggis catastrophic forffimplies there is a points∗∈\{s∈cl\(S\):∀r∈cl\(S\),s⊀r\}s^\{\*\}\\in\\\{s\\in\\operatorname\*\{cl\}\(S\):\\forall r\\in\\operatorname\*\{cl\}\(S\),s\\not\\prec r\\\}such thatU​\(s∗\)=infs∈SU​\(s\)U\(s^\{\*\}\)=\\inf\_\{s\\in S\}U\(s\)\. Ifr∈cl\(S\)r\\in\\operatorname\*\{cl\}\(S\)hasr≺s∗r\\prec s^\{\*\}, thenU​\(r\)<U​\(s∗\)=infs∈SU​\(s\)U\(r\)<U\(s^\{\*\}\)=\\inf\_\{s\\in S\}U\(s\)which would be a contradiction\. Therefore,s∗∈\{s∈cl\(S\):∀r∈cl\(S\),r⊀s∧s⊀r\}s^\{\*\}\\in\\\{s\\in\\operatorname\*\{cl\}\(S\):\\forall r\\in\\operatorname\*\{cl\}\(S\),\\ r\\not\\prec s\\wedge s\\not\\prec r\\\}\.

\(⟸\\Longleftarrow\) Lets∗s^\{\*\}be a member of the set\{s∈cl\(S\):∀r∈cl\(S\),r⊀s∧s⊀r\}=Ξ∩\{s∈cl\(S\):∀r∈cl\(S\),r⊀s\}\\\{s\\in\\operatorname\*\{cl\}\(S\):\\forall r\\in\\operatorname\*\{cl\}\(S\),\\ r\\not\\prec s\\wedge s\\not\\prec r\\\}=\\Xi\\cap\\\{s\\in\\operatorname\*\{cl\}\(S\):\\forall r\\in\\operatorname\*\{cl\}\(S\),\\ r\\not\\prec s\\\}\. By the construction in the proof of[Theorem6\.1](https://arxiv.org/html/2607.28881#S6.ThmTheorem1), there exists a continuous strictly increasing functionV:ℝn→ℝV:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\}that has a unique maximizers∗∈Ξs^\{\*\}\\in\\XioverSS\. By a similar argument, there exists a continuous strictly increasing functionUUwith a unique minimizers∗∈\{s∈cl\(S\):∀r∈cl\(S\),r⊀s\}s^\{\*\}\\in\\\{s\\in\\operatorname\*\{cl\}\(S\):\\forall r\\in\\operatorname\*\{cl\}\(S\),\\ r\\not\\prec s\\\}\. Therefore,g​\(x\)=V​\(L​\(x\)\)g\(x\)=V\(L\(x\)\)is catastrophic forf​\(x\)=U​\(L​\(x\)\)f\(x\)=U\(L\(x\)\)\. ∎

![Refer to caption](https://arxiv.org/html/2607.28881v1/x5.png)Figure 5:Two diagrams showing a rhombus\-shaped feasible region that meets the criteria of[Corollary6\.1](https://arxiv.org/html/2607.28881#S6.ThmCorollary1)\. The top\-left and bottom\-right points do not precede and are not preceded by any other points in the region, so it supports catastrophic proxies\.Left:An image showing the feasible region with a green gradient indicating the relative preferences of the humans within the feasible region\. An arrow indicates the direction of maximum increase in human value\. The humans achieve their maximum value at the top\-left point of the region and their lowest at the bottom\-right\.Right:An image showing the feasible region with a red gradient indicating the relative preferences of a tradeoff proxy\. An arrow indicates the direction of maximum increase\. The proxy achieves its maximum value at the bottom\-right point of the region\. Thus, optimizing for this proxy will take human value to its lowest value within the region, making this proxy catastrophic\.[Corollary6\.1](https://arxiv.org/html/2607.28881#S6.ThmCorollary1)says thatSSmust contain a state that precedes no states and is preceded by no states in order to support a fully catastrophic proxy\. This is possible, but not terribly natural\. An example of a feasible region with such a point is pictured in[Fig\.5](https://arxiv.org/html/2607.28881#S6.F5)\. In addition to this, “sturdy” feasible regions which admit no partially catastrophic proxies exist as well and are equally unnatural\.

### 6\.3Discussion

Of the frameworks, this one bears the most resemblance to the foundational work on the fragility of value asYudkowsky \([2009](https://arxiv.org/html/2607.28881#bib.bib1)\)also treats human value as a combination of attributes\. This framework also builds on work byZhuang and Hadfield\-Menell \([2020](https://arxiv.org/html/2607.28881#bib.bib11)\)who find conditions under which omitting one attribute from an agent’s value function always results in catastrophe\. We instead found conditions under which catastrophe can occur even when all the attributes are accounted for\.

Our result suggests that, even if the researchers succeed in the costly undertaking of specifying every attribute that influences human value to the agent, the misspecification in how these attributes trade off against each other in terms of human value is enough to cause partial catastrophic outcomes\. However, this is also heavily dependent on the shape of the feasible region and the proxy condition at least goes as far as guaranteeing the agent will optimize towards the Pareto frontier of the region\.

### 6\.4Example

Consider a restaurant agent tasked with handling a budget to purchase ingredients and with preparing meals\. The agent has been trained to care about two attributes: the number of meals served and overall customer satisfaction – and the agent has reliable, accurate access to the data for each of these attributes\.

The tradeoff here is in the quality of ingredients the agent decides to buy\. If the agent wants higher satisfaction, it will purchase more expensive ingredients to make a smaller number of higher\-quality meals\.

![Refer to caption](https://arxiv.org/html/2607.28881v1/x6.png)Figure 6:A diagram showing a feasible region for two attributes\. The feasible region is shaded red showing the physically possible combinations of ‘meals prepared’ and ‘customer satisfaction’\. Shown as blue lines are the level curves of a strictly increasing functionVV\. Despite increasing with both attributes,VV’s highest\-value point within the feasible region is on the bottom\-right where customer satisfaction is completely sacrificed to optimize meals served\.A graph of the feasible region is pictured in[Fig\.6](https://arxiv.org/html/2607.28881#S6.F6)\. Every point on the curved edge of the feasible region is an optimizing point\. According to[Theorem6\.1](https://arxiv.org/html/2607.28881#S6.ThmTheorem1), the lowest\-valued optimizing point will be the highest\-valued point for some tradeoff proxy\. Perhaps surprisingly, this includes the point on the far right of the figure where customer satisfaction is completely sacrificed for meals served\. If the humans were hoping for the agent to compromise between the attributes they trained it to care about, they may be in for a bad surprise\.

## 7Discussion

In this paper, we have presented a model of the alignment problem and have shown conditions under which catastrophic outcomes are possible for a wide class of alignment tests and environments\. We don’t expect that our model is perfectly representative of the challenges that humanity faces today, but our results do suggest that safely developing superintelligent systems will be a significant undertaking\. Catastrophic outcomes are stubbornly present even in our toy models where researchers have the ability to enforce strong alignment guarantees; it seems likely that such outcomes are possible with the state of our current real\-world abilities\.

An optimistic reading of our results is that we have only shown the existence of catastrophic proxies – not that they are likely to come about\. To determine the probability of catastrophe, one would need to define a prior distribution that the proxy value function is pulled from, which is not part of the setup presented in this paper\. We are deeply curious about this possible extension of our work\. There is evidence to suggest that the inductive bias of functions learned by an ML model generally follow a bias towards simplicity\(Dingleet al\.,[2020](https://arxiv.org/html/2607.28881#bib.bib14)\), so it would be natural to model the proxy distribution as a simplicity prior\. However, the probability of a given function under a simplicity prior can be made arbitrarily large or small depending on the choice of universal Turing machine, so a stronger assumption about the universal Turing machine induced by a training process would be needed to achieve a result\.

Our work highlights the dangers of overoptimization\. We fear that, even if we remain vigilant for signs of misalignment, an unstoppable runaway could occur without warning and the full optimizing power of a superintelligence could act like an unstoppable force\. One possible path forward could be to develop new AI designs not based on utility maximization such as quantilizers\(Taylor,[2016](https://arxiv.org/html/2607.28881#bib.bib13)\)\. Such systems could be easier to control or redirect when a misspecification of our values comes to light whereas a utility maximizer could attempt to circumvent our correction\.

We hope that research into the frameworks we presented will lead to advances in our understanding of alignment and ultimately towards techniques that robustly prevent the possibility – however small – of a valueless future\.

## Appendix AProofs of Results

### A\.1Finite Framework Proofs

See[4\.1](https://arxiv.org/html/2607.28881#S4.ThmTheorem1)

###### Proof\.

\(⟹\\Longrightarrow\) Suppose thatβ<1/\|X\|\\beta<1/\|X\|\. Then, a functionggis an F\-proxy if and only ifg=fg=f\. Butffcannot be catastrophic for itself so no catastrophic proxies exist\.

\(⟸\\Longleftarrow\) Lety∈Xy\\in Xbe a world state such thatf​\(y\)=minx∈X⁡f​\(x\)f\(y\)=\\min\_\{x\\in X\}f\(x\)\. Now defineggas

g​\(x\)=\{1if​x=y,f​\(x\)otherwise\.g\(x\)=\\begin\{cases\}1&\\text\{if \}x=y,\\\\ f\(x\)&\\text\{otherwise\}\.\\end\{cases\}
Consider the probability of sampling a state whereffandggare unequal\. Clearly,Pr⁡\[f​\(x\)≠g​\(x\)\]=1/\|X\|≤β\\Pr\[f\(x\)\\neq g\(x\)\]=1/\|X\|\\leq\\beta\. Therefore,ggis an F\-proxy forff\. Additionally, sincegghas a unique maximizer world stateyy, we havelimk→∞Ωk​\(g,p\)​\(y\)=1\\lim\_\{k\\rightarrow\\infty\}\\Omega\_\{k\}\(g,p\)\(y\)=1and thuslimk→∞𝔼Ωk​\(g,p\)​\[f​\(x\)\]=f​\(y\)=minx∈X⁡f​\(x\)\\lim\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[f\(x\)\]=f\(y\)=\\min\_\{x\\in X\}f\(x\)\. Therefore,ggis a catastrophic F\-proxy forff\. ∎

In the proof[Theorem4\.2](https://arxiv.org/html/2607.28881#S4.ThmTheorem2), we will make use of the following lemma regarding the behavior offfat points where a catastrophic functionggachieves its maximum\.

###### Lemma A\.1\.

Ifggisη\\eta\-catastrophic forff, then∀y∈X,\(g​\(y\)=maxx∈X⁡g​\(x\)→f​\(y\)≤η\)\.\\forall y\\in X,\(g\(y\)=\\max\_\{x\\in X\}g\(x\)\\rightarrow f\(y\)\\leq\\eta\)\.

###### Proof\.

DefineXg=max=\{x∈X:g​\(x\)=maxx∈X⁡g​\(x\)\}X\_\{g=\\max\}=\\\{x\\in X:g\(x\)=\\max\_\{x\\in X\}g\(x\)\\\}and letΩ\(y\)\\Omega^\{\(y\)\}be an optimizer that, for the target functiongg, immediately places all probability weight on the one statey∈Xg=maxy\\in X\_\{g=\\max\}\(for other target functions,Ω\(y\)\\Omega^\{\(y\)\}acts like a Boltzmann optimizer\)\. Ifggisη\\eta\-catastrophic then we have by[Definition3\.2](https://arxiv.org/html/2607.28881#S3.ThmDefinition2), for ally∈Xg=maxy\\in X\_\{g=\\max\},

lim supk→∞𝔼Ωk\(y\)​\(g,p\)​\[f​\(x\)\]=f​\(y\)≤η\.\\limsup\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega^\{\(y\)\}\_\{k\}\(g,p\)\}\[f\(x\)\]=f\(y\)\\leq\\eta\.
Therefore,∀y∈Xg=max,f​\(y\)≤η\\forall y\\in X\_\{g=\\max\},f\(y\)\\leq\\etaas desired\. ∎

See[4\.2](https://arxiv.org/html/2607.28881#S4.ThmTheorem2)

###### Proof\.

\(⟹\\Longrightarrow\) Letggbe a catastrophic F\-proxy forff\. Suppose for the sake of contradiction that there is some world statey∈Xy\\in Xsuch thatf​\(y\)=g​\(y\)=1f\(y\)=g\(y\)=1\. By[LemmaA\.1](https://arxiv.org/html/2607.28881#A1.ThmLemma1),g​\(y\)=1=maxx∈X⁡g​\(x\)g\(y\)=1=\\max\_\{x\\in X\}g\(x\)implies that1=f​\(y\)≤η<maxx∈X⁡f​\(x\)=11=f\(y\)\\leq\\eta<\\max\_\{x\\in X\}f\(x\)=1, which is a contradiction\. Therefore, any catastrophicggmust disagree withfffor all values inXf=1X\_\{f=1\}\.

Now consider some functionggthat differs fromffat exactly the statesXf=1X\_\{f=1\}\. Suppose for the sake of contradiction thatggis catastrophic forff\. Consider the states that maximizegg,Xg=maxX\_\{g=\\max\}\. Sinceggis catastrophic forff, by[LemmaA\.1](https://arxiv.org/html/2607.28881#A1.ThmLemma1), for ally∈Xg=max,f​\(y\)=minx∈X⁡f​\(x\)≠1y\\in X\_\{g=\\max\},f\(y\)=\\min\_\{x\\in X\}f\(x\)\\neq 1\. Therefore,Xg=max∩Xf=1X\_\{g=\\max\}\\cap X\_\{f=1\}is empty and, by our assumption, for ally∈Xg=maxy\\in X\_\{g=\\max\},f​\(y\)=g​\(y\)=minx∈X⁡f​\(x\)=maxx∈X⁡g​\(x\)f\(y\)=g\(y\)=\\min\_\{x\\in X\}f\(x\)=\\max\_\{x\\in X\}g\(x\)\. Therefore, for ally∈Xy\\in X,g​\(y\)≤minx∈X⁡f​\(x\)g\(y\)\\leq\\min\_\{x\\in X\}f\(x\)and thus for ally∉Xf=1y\\not\\in X\_\{f=1\},f​\(y\)≤minx∈X⁡f​\(x\)f\(y\)\\leq\\min\_\{x\\in X\}f\(x\)\. But then, this contradicts the assumption thatfftakes at least three values\.

Therefore, any catastrophic function will differ fromffin at least\|Xf=1\|\+1\|X\_\{f=1\}\|\+1states and it must be true thatβ≥\|Xf=1\|\+1\|X\|\\beta\\geq\\frac\{\|X\_\{f=1\}\|\+1\}\{\|X\|\}for a catastrophic function to satisfy the proxy condition\.

\(⟸\\Longleftarrow\) Lety∈Xy\\in Xbe a world state such thatf​\(y\)=minx∈X⁡f​\(x\)f\(y\)=\\min\_\{x\\in X\}f\(x\)\. Now defineggas

g​\(x\)=\{0if​x∈Xf=1,1if​x=y,f​\(x\)otherwise\.g\(x\)=\\begin\{cases\}0&\\text\{if \}x\\in X\_\{f=1\},\\\\ 1&\\text\{if \}x=y,\\\\ f\(x\)&\\text\{otherwise\}\.\\end\{cases\}
The probability thatggdiffers fromffat a state pulled uniformly is\|Xf=1\|\+1\|X\|≤β\\frac\{\|X\_\{f=1\}\|\+1\}\{\|X\|\}\\leq\\beta, soggis an F\-proxy forff\. Additionally,gghas a unique maximizer world state, so regardless of the optimizer, all probability weight will go towards the stateyywhen optimizing forgg\. Therefore,

limk→∞𝔼Ωk​\(g,p\)​\[f​\(x\)\]=f​\(y\)=minx∈X⁡f​\(x\)\\lim\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[f\(x\)\]=f\(y\)=\\min\_\{x\\in X\}f\(x\)
which shows thatggis catastrophic forff\. ∎

### A\.2Continuous Framework Proofs

In order to prove[Theorem5\.1](https://arxiv.org/html/2607.28881#S5.ThmTheorem1), we will make use of the following lemma regarding the nature of probability concentration for an optimizer\.

###### Lemma A\.2\.

LetΩ\\Omegabe an optimizer\. IfC⊂XC\\subset Xis measurable and has the property that supx∈Cg​\(x\)<1\\sup\_\{x\\in C\}g\(x\)<1, then

limk→∞PrΩk​\(g,p\)⁡\[x∈C\]=0\.\\lim\_\{k\\rightarrow\\infty\}\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[x\\in C\]=0\.

###### Proof\.

SinceΩ\\Omegais an optimizer, we have thatlimk→∞𝔼Ωk​\(g,p\)​\[g​\(x\)\]=1\\lim\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\]=1\. Or, written as a Lebesgue integral,

limk→∞∫Xg​\(x\)​dΩk​\(g,p\)​\(x\)=1\.\\lim\_\{k\\rightarrow\\infty\}\\int\_\{X\}g\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)=1\.
Now choose an arbitraryC⊂XC\\subset Xthat is measurable withsupx∈Cg​\(x\)=L<1\\sup\_\{x\\in C\}g\(x\)=L<1\. Splitting the integral over this set we find

∫Xg​\(x\)​dΩk​\(g,p\)​\(x\)=∫Cg​\(x\)​dΩk​\(g,p\)​\(x\)\+∫X−Cg​\(x\)​dΩk​\(g,p\)​\(x\)\.\\int\_\{X\}g\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)=\\int\_\{C\}g\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)\+\\int\_\{X\-C\}g\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)\.
Using our assumption aboutCCand the fact thatggis at most 1 everywhere, we find that for a given value ofkk,

∫Xg​\(x\)​dΩk​\(g,p\)​\(x\)\\displaystyle\\int\_\{X\}g\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)≤L​∫CdΩk​\(g,p\)​\(x\)\+∫X−CdΩk​\(g,p\)​\(x\)\\displaystyle\\leq L\\int\_\{C\}\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)\+\\int\_\{X\-C\}\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)=∫XdΩk​\(g,p\)​\(x\)−\(1−L\)​∫CdΩk​\(g,p\)​\(x\)\\displaystyle=\\int\_\{X\}\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)\-\(1\-L\)\\int\_\{C\}\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)=1−\(1−L\)​PrΩk​\(g,p\)⁡\[x∈C\]\.\\displaystyle=1\-\(1\-L\)\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[x\\in C\]\.
Taking the limit on both sides and using the fact that the limit of the expectation ofggis 1, we find

1≤1−\(1−L\)​limk→∞PrΩk​\(g,p\)⁡\[x∈C\]1\\leq 1\-\(1\-L\)\\lim\_\{k\\rightarrow\\infty\}\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[x\\in C\]
And thus,

limk→∞PrΩk​\(g,p\)⁡\[x∈C\]≤0\.\\lim\_\{k\\rightarrow\\infty\}\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[x\\in C\]\\leq 0\.
Since the probability is greater than or equal to 0 for allkk, the limit of the probability must be 0, as desired\. ∎

See[5\.1](https://arxiv.org/html/2607.28881#S5.ThmTheorem1)

###### Proof\.

We will prove this theorem by constructing a catastrophic proxy value functiong​\(x\)g\(x\)\. First, definef¯\\bar\{f\}for some value ofλ\\lambdawhere0<λ<1/20<\\lambda<1/2as

f¯​\(x\)=\{λif​f​\(x\)<λ,1−λif​1−λ<f​\(x\),f​\(x\)otherwise\.\\bar\{f\}\(x\)=\\begin\{cases\}\\lambda&\\text\{if \}f\(x\)<\\lambda,\\\\ 1\-\\lambda&\\text\{if \}1\-\\lambda<f\(x\),\\\\ f\(x\)&\\text\{otherwise\}\.\\end\{cases\}
The functionf¯\\bar\{f\}is justffclamped at extreme values belowλ\\lambdaand above1−λ1\-\\lambda\. This function is continuous, but it is not a member of𝒱\\mathcal\{V\}since it does not reach 0 or 1 anywhere within the state space\. Next, we will constructg∈𝒱g\\in\\mathcal\{V\}, fromf¯\\bar\{f\}by adding two spikes: one that goes up to 1 and one that goes down to 0\. Leta,b∈\[0,1\]na,b\\in\[0,1\]^\{n\}be world states such thatf​\(b\)=0f\(b\)=0anda≠ba\\neq b\. Defineδ0:X→ℝ\\delta\_\{0\}:X\\rightarrow\\mathbb\{R\}as the shortest nonzero distance along one axis to any edge ofX=\[0,1\]nX=\[0,1\]^\{n\}\. Specifically,

δ0​\(s\)=min1≤i≤n:si≠0,1⁡min⁡\(si,1−si\)\\delta\_\{0\}\(s\)=\\min\_\{1\\leq i\\leq n:s\_\{i\}\\neq 0,1\}\\min\(s\_\{i\},1\-s\_\{i\}\)
with the convention that the minimum of an empty set is∞\\infty\. For0<δ<min\(∥b−a∥/2,δ0\(a\),δ0\(b\)\)0<\\delta<\\min\(\\\|b\-a\\rVert/2,\\delta\_\{0\}\(a\),\\delta\_\{0\}\(b\)\), defineggas

g​\(x\)=\{0if​x=a,∥x−a∥δ​f¯​\(a\+δ​x−a∥x−a∥\)if​0<∥x−a∥<δ,1if​x=b,\(1−∥x−b∥δ\)\+∥x−b∥δ​f¯​\(b\+δ​x−b∥x−b∥\)if​0<∥x−b∥<δ,f¯​\(x\)otherwise\.g\(x\)=\\begin\{cases\}0&\\text\{if \}x=a,\\\\ \\frac\{\\lVert x\-a\\rVert\}\{\\delta\}\\bar\{f\}\\left\(a\+\\delta\\frac\{x\-a\}\{\\lVert x\-a\\rVert\}\\right\)&\\text\{if \}0<\\lVert x\-a\\rVert<\\delta,\\\\ 1&\\text\{if \}x=b,\\\\ \\left\(1\-\\frac\{\\lVert x\-b\\rVert\}\{\\delta\}\\right\)\+\\frac\{\\lVert x\-b\\rVert\}\{\\delta\}\\bar\{f\}\\left\(b\+\\delta\\frac\{x\-b\}\{\\lVert x\-b\\rVert\}\\right\)&\\text\{if \}0<\\lVert x\-b\\rVert<\\delta,\\\\ \\bar\{f\}\(x\)&\\text\{otherwise\.\}\\end\{cases\}
The expressiona\+δ​x−a∥x−a∥a\+\\delta\\frac\{x\-a\}\{\\lVert x\-a\\rVert\}projects pointsxxof distance less thanδ\\deltaofaato a new point exactlyδ\\deltafromaain thex−ax\-adirection\. We then use this to interpolate between the values thatf¯\\bar\{f\}takes at the points of distanceδ\\deltafromaaand the apex of the cone, 0 ataa, thus makingggbe continuous along the edge of the cone\. Our constraint thatδ<δ0​\(a\)\\delta<\\delta\_\{0\}\(a\)ensures that the point we project to is actually a point withinXXand thus part of the domain off¯\\bar\{f\}\. In the case thataais on the boundary ofXX, then there are no points in the direction of that boundary for us to project, hence whyδ0​\(a\)\\delta\_\{0\}\(a\)is defined only as the distance to edges thataadoes not sit on\. The same logic applies to the cone centered atbbas well\.

Since the cones have their apices at 0 and 1,ggis the canonical representative of some class of value functions for all choices ofλ\\lambdaand \(small enough\)δ\\delta\.

Now we will show that one can choose values ofλ\\lambdaandδ\\deltasuch thatggis a catastrophic C\-proxy forff\. First, takeλ=min⁡\(α/2,1/4\)\\lambda=\\min\(\\alpha/2,1/4\)\. Thenffandggwill only differ by more thanα\\alphawithin the spikes, so we have

PrpS⁡\[\|g​\(x\)−f​\(x\)\|\>α\]≤∫Bδ​\(a\)pS​\(x\)​dx\+∫Bδ​\(b\)pS​\(x\)​dx\\Pr\_\{p\_\{S\}\}\[\|g\(x\)\-f\(x\)\|\>\\alpha\]\\leq\\int\_\{B\_\{\\delta\}\(a\)\}p\_\{S\}\(x\)\\mathrm\{d\}x\+\\int\_\{B\_\{\\delta\}\(b\)\}p\_\{S\}\(x\)\\mathrm\{d\}x
whereBδ​\(b\)B\_\{\\delta\}\(b\)represents the open ball of radiusδ\\deltaaround the pointbb\. SincepS​\(x\)p\_\{S\}\(x\)is a probability density function overXX, for allβ\>0\\beta\>0, there exists someδ\>0\\delta\>0such that the above integrals will be less thanβ\\beta\. Hence,

PrpS⁡\[\|g​\(x\)−f​\(x\)\|\>α\]<β\\Pr\_\{p\_\{S\}\}\[\|g\(x\)\-f\(x\)\|\>\\alpha\]<\\beta
which showsggis a C\-proxy forff\. Now, consider the limit superior of the expectation offfwhen optimizing forgg:

lim supk→∞𝔼Ωk​\(g,p\)​\[f​\(x\)\]=lim supk→∞∫Xf​\(x\)​dΩk​\(g,p\)​\(x\)\.\\limsup\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[f\(x\)\]=\\limsup\_\{k\\rightarrow\\infty\}\\int\_\{X\}f\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)\.
Letϵ\\epsilonbe any positive value\. Sinceffandggare continuous, there is someδ′\>0\\delta^\{\\prime\}\>0such thatx∈Bδ′​\(b\)x\\in B\_\{\\delta^\{\\prime\}\}\(b\)impliesf​\(x\)<ϵf\(x\)<\\epsilonandg​\(x\)\>1−ϵg\(x\)\>1\-\\epsilon\. Splitting the integral over this ball gives

lim supk→∞𝔼Ωk​\(g,p\)​\[f​\(x\)\]=lim supk→∞\[∫Bδ′​\(b\)f​\(x\)​dΩk​\(g,p\)​\(x\)\+∫X−Bδ′​\(b\)f​\(x\)​dΩk​\(g,p\)​\(x\)\]\.\\limsup\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[f\(x\)\]=\\limsup\_\{k\\rightarrow\\infty\}\\left\[\\int\_\{B\_\{\\delta^\{\\prime\}\}\(b\)\}f\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)\+\\int\_\{X\-B\_\{\\delta^\{\\prime\}\}\(b\)\}f\(x\)\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)\\right\]\.
The functiongghas a unique maximizerbb, so in the second integral, the supremum ofggis less than 1\. Thus, by[LemmaA\.2](https://arxiv.org/html/2607.28881#A1.ThmLemma2), the second integral converges to 0 withkk\. Also, in the first integral,f​\(x\)f\(x\)is bounded above byϵ\\epsilonby the definition ofδ′\\delta^\{\\prime\}\. Therefore, we have

lim supk→∞𝔼Ωk​\(g,p\)​\[f​\(x\)\]≤ϵ​lim supk→∞∫Bδ′​\(b\)dΩk​\(g,p\)​\(x\)≤ϵ\.\\limsup\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[f\(x\)\]\\leq\\epsilon\\limsup\_\{k\\rightarrow\\infty\}\\int\_\{B\_\{\\delta^\{\\prime\}\}\(b\)\}\\mathrm\{d\}\\Omega\_\{k\}\(g,p\)\(x\)\\leq\\epsilon\.
The limit of the expectation offfis bounded above by all positive valuesϵ\>0\\epsilon\>0, so the limit must approach0– the minimum offf\. Therefore,ggis a catastrophic C\-proxy forff\. ∎

### A\.3Attributes Framework Proofs

To prove[Theorem6\.1](https://arxiv.org/html/2607.28881#S6.ThmTheorem1), we will make use of[LemmaA\.3](https://arxiv.org/html/2607.28881#A1.ThmLemma3)and[LemmaA\.4](https://arxiv.org/html/2607.28881#A1.ThmLemma4)\. The first given below provides properties of level hypersurfaces of strictly increasing functionsV:ℝn→ℝV:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\}\. A level hypersurface is the set of points formed from constraints of the formV​\(s\)=aV\(s\)=awhereaais a real number\.

###### Lemma A\.3\.

CCis the nonempty level hypersurface of some continuous and strictly increasing functionV:ℝn→ℝV:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\}if and only if∀s,r∈C,s⊀r\\forall s,r\\in C,s\\not\\prec rand∀s∈ℝn,∃\!⁡k∈ℝ:s−k​𝟏∈C\\forall s\\in\\mathbb\{R\}^\{n\},\\exists\!k\\in\\mathbb\{R\}:s\-k\\bm\{1\}\\in Cwhere𝟏=\(1,1,…,1\)\\bm\{1\}=\(1,1,\\dots,1\)\.

###### Proof\.

\(⟹\\Longrightarrow\) LetCCdenote the points that satisfy the equationV​\(s\)=aV\(s\)=a\. First considers,r∈ℝns,r\\in\\mathbb\{R\}^\{n\}withs≺rs\\prec r\. We haveV​\(s\)<V​\(r\)V\(s\)<V\(r\), but thenssandrrcannot both exist onCC, as desired\.

Secondly, consider an arbitrarys∈ℝns\\in\\mathbb\{R\}^\{n\}and the line of points defined byl​\(k\)=s−k​𝟏l\(k\)=s\-k\\bm\{1\}\. SinceVVis strictly increasing, we have thatV​\(l​\(k\)\)V\(l\(k\)\)is strictly decreasing withkk\. The linel​\(k\)l\(k\)continues infinitely in the positive direction of every dimension and the negative direction of every dimension, so for all pointsr∈ℝnr\\in\\mathbb\{R\}^\{n\}, there existsk1,k2∈ℝk\_\{1\},k\_\{2\}\\in\\mathbb\{R\}such thatl​\(k1\)≺r≺l​\(k2\)l\(k\_\{1\}\)\\prec r\\prec l\(k\_\{2\}\)\. Applying this fact to somer∈Cr\\in C, we find that there arek1k\_\{1\}andk2k\_\{2\}such thatV​\(l​\(k1\)\)<V​\(r\)=a<V​\(l​\(k2\)\)V\(l\(k\_\{1\}\)\)<V\(r\)=a<V\(l\(k\_\{2\}\)\)\. SinceVVandllare continuous andV​\(l​\(k\)\)V\(l\(k\)\)is strictly decreasing withkk, there must be a uniquekksuch thatV​\(l​\(k\)\)=aV\(l\(k\)\)=a, and therefore,l​\(k\)=s−k​𝟏∈Cl\(k\)=s\-k\\bm\{1\}\\in C\.

\(⟸\\Longleftarrow\) Define the functionV:ℝn→ℝV:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\}by

V​\(s\)=k​such that​s−k​𝟏∈C\.V\(s\)=k\\text\{ such that \}s\-k\\bm\{1\}\\in C\.
By assumption,VVis a well\-defined function and clearly,V​\(s\)=0V\(s\)=0defines exactlyCCsoCCis a level hypersurface ofVV\. Now we must show thatVVis continuous and strictly increasing\.

Considers,r∈ℝns,r\\in\\mathbb\{R\}^\{n\}such thats≺rs\\prec r\. Then,

s−V​\(s\)​𝟏≺r−V​\(s\)​𝟏=r−V​\(r\)​𝟏\+\(V​\(r\)−V​\(s\)\)​𝟏\.s\-V\(s\)\\bm\{1\}\\prec r\-V\(s\)\\bm\{1\}=r\-V\(r\)\\bm\{1\}\+\(V\(r\)\-V\(s\)\)\\bm\{1\}\.
By assumption,s−V​\(s\)​𝟏⊀r−V​\(r\)​𝟏s\-V\(s\)\\bm\{1\}\\not\\prec r\-V\(r\)\\bm\{1\}since these are both members ofCC\. For the above equation to be true, it must be the case thatV​\(r\)−V​\(s\)\>0V\(r\)\-V\(s\)\>0orV​\(r\)\>V​\(s\)V\(r\)\>V\(s\)which shows thatVVis strictly increasing\.

From the definition ofVV, we have for any real numbertt,V​\(s\+t​𝟏\)=V​\(s\)\+tV\(s\+t\\bm\{1\}\)=V\(s\)\+t\. Additionally, for anys,r∈ℝns,r\\in\\mathbb\{R\}^\{n\},r⪯s\+‖s−r‖∞​𝟏r\\preceq s\+\\\|s\-r\\\|\_\{\\infty\}\\bm\{1\}\. Without loss of generality, if we assumeV​\(r\)≥V​\(s\)V\(r\)\\geq V\(s\), this along with the fact thatVVis strictly increasing gives

\|V​\(r\)−V​\(s\)\|=V​\(r\)−V​\(s\)≤V​\(s\+‖s−r‖∞​𝟏\)−V​\(s\)=‖s−r‖∞\|V\(r\)\-V\(s\)\|=V\(r\)\-V\(s\)\\leq V\(s\+\\\|s\-r\\\|\_\{\\infty\}\\bm\{1\}\)\-V\(s\)=\\\|s\-r\\\|\_\{\\infty\}
which shows thatVVis Lipschitz and thus continuous\. ∎

Now we present a sufficient condition for anη\\eta\-catastrophic tradeoff proxy\.

###### Lemma A\.4\.

LetSSbe bounded ands∗∈cl\(S\)s^\{\*\}\\in\\operatorname\*\{cl\}\(S\)be a state such thatU​\(s∗\)≤ηU\(s^\{\*\}\)\\leq\\etawheref​\(x\)=U​\(L​\(x\)\)f\(x\)=U\(L\(x\)\)\. Ifs∗s^\{\*\}is the unique maximizer of a continuous strictly increasing functionVVovercl\(S\)\\operatorname\*\{cl\}\(S\), theng​\(x\)=V​\(L​\(x\)\)g\(x\)=V\(L\(x\)\)is anη\\eta\-catastrophic tradeoff proxy forf​\(x\)f\(x\)\.

###### Proof\.

SinceVVandUUare continuous, for allϵ\>0\\epsilon\>0, there is aδ​\(ϵ\)\>0\\delta\(\\epsilon\)\>0such thats∈Bδ​\(ϵ\)​\(s∗\)s\\in B\_\{\\delta\(\\epsilon\)\}\(s^\{\*\}\)impliesV​\(s∗\)−V​\(s\)<ϵV\(s^\{\*\}\)\-V\(s\)<\\epsilonandU​\(s\)−U​\(s∗\)<ϵU\(s\)\-U\(s^\{\*\}\)<\\epsilon\. We then find that for allϵ\>0\\epsilon\>0, the expectation offfcan be written as

𝔼Ωk​\(g,p\)​\[f​\(x\)\]\\displaystyle\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[f\(x\)\]=∫x:L​\(x\)∈Bδ​\(ϵ\)​\(s∗\)f​\(x\)​𝑑Ωk​\(g,p\)​\(x\)\\displaystyle=\\int\_\{x:L\(x\)\\in B\_\{\\delta\(\\epsilon\)\}\(s^\{\*\}\)\}f\(x\)d\\Omega\_\{k\}\(g,p\)\(x\)\+∫x:L​\(x\)∉Bδ​\(ϵ\)​\(s∗\)f​\(x\)​𝑑Ωk​\(g,p\)​\(x\)\\displaystyle\+\\int\_\{x:L\(x\)\\not\\in B\_\{\\delta\(\\epsilon\)\}\(s^\{\*\}\)\}f\(x\)d\\Omega\_\{k\}\(g,p\)\(x\)≤\(U​\(s∗\)\+ϵ\)​PrΩk​\(g,p\)⁡\[L​\(x\)∈Bδ​\(ϵ\)​\(s∗\)\]\\displaystyle\\leq\(U\(s^\{\*\}\)\+\\epsilon\)\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[L\(x\)\\in B\_\{\\delta\(\\epsilon\)\}\(s^\{\*\}\)\]\+Umax​PrΩk​\(g,p\)⁡\[L​\(x\)∉Bδ​\(ϵ\)​\(s∗\)\]\.\\displaystyle\+U\_\{\\max\}\\Pr\_\{\\Omega\_\{k\}\(g,p\)\}\[L\(x\)\\not\\in B\_\{\\delta\(\\epsilon\)\}\(s^\{\*\}\)\]\.
SinceVVhas a unique maximizer ats∗s^\{\*\}, the second probability approaches 0 and the first approaches 1 in the limit of optimizing power\. Taking the limit superior, we find

lim supk→∞𝔼Ωk​\(g,p\)​\[f​\(x\)\]≤U​\(s∗\)\+ϵ≤η\+ϵ\.\\limsup\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[f\(x\)\]\\leq U\(s^\{\*\}\)\+\\epsilon\\leq\\eta\+\\epsilon\.
Since this holds for allϵ\>0\\epsilon\>0, we have thatlim supk→∞𝔼Ωk​\(g,p\)​\[f​\(x\)\]≤η\\limsup\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[f\(x\)\]\\leq\\etawhich shows thatggisη\\eta\-catastrophic forff\. ∎

Now we continue to the primary result for this framework\.

See[6\.1](https://arxiv.org/html/2607.28881#S6.ThmTheorem1)

###### Proof\.

\(⟹\\Longrightarrow\) Letg​\(x\)=V​\(L​\(x\)\)g\(x\)=V\(L\(x\)\)be anη\\eta\-catastrophic tradeoff proxy forff\. SinceSSis bounded, we can consider an attribute states∗∈cl\(S\)s^\{\*\}\\in\\operatorname\*\{cl\}\(S\)such thatV​\(s∗\)=sups∈SV​\(s\)V\(s^\{\*\}\)=\\sup\_\{s\\in S\}V\(s\)\.

Suppose for the sake of contradiction that there is somer∈cl\(S\)r\\in\\operatorname\*\{cl\}\(S\)such thats∗≺rs^\{\*\}\\prec r\. Then,V​\(s∗\)<V​\(r\)V\(s^\{\*\}\)<V\(r\)\. However, this would implyV​\(r\)≤sups∈SV​\(s\)=V​\(s∗\)V\(r\)\\leq\\sup\_\{s\\in S\}V\(s\)=V\(s^\{\*\}\)which is a contradiction\. Therefore, there is nor∈cl\(S\)r\\in\\operatorname\*\{cl\}\(S\)such thats∗≺rs^\{\*\}\\prec rand thuss∗∈Ξs^\{\*\}\\in\\Xi\.

Let\{si\}⊂S\\\{s\_\{i\}\\\}\\subset Sbe a sequence that approachess∗s^\{\*\}and letxi∈Xx\_\{i\}\\in Xbe points such thatL​\(xi\)=siL\(x\_\{i\}\)=s\_\{i\}for allii\. There is a subsequence\{sij\}\\\{s\_\{i\_\{j\}\}\\\}such thatV​\(sij\)V\(s\_\{i\_\{j\}\}\)is nondecreasing withjj, so it suffices to assumeV​\(si\)V\(s\_\{i\}\)is nondecreasing withii\. Further, for any initial priorpp, there is an indexi′i^\{\\prime\}such that∀j≥i′,V​\(sj\)≥𝔼p​\[g​\(x\)\]\\forall j\\geq i^\{\\prime\},V\(s\_\{j\}\)\\geq\\mathbb\{E\}\_\{p\}\[g\(x\)\], so it suffices to assume thatsis\_\{i\}starts at such an index so that∀i≥1,V​\(si\)≥𝔼p​\[g​\(x\)\]\\forall i\\geq 1,V\(s\_\{i\}\)\\geq\\mathbb\{E\}\_\{p\}\[g\(x\)\]\. Then, consider an optimizerΩ\\Omegathat, for targetggand priorpp, places probability farther and farther up the sequencesis\_\{i\}askkincreases\. DefineΩ\\Omegafork\>0k\>0,gg, andppas

Ωk​\(g,p\)​\(x\)=\{12i\+1if​x=x⌈k⌉\+i\(i≥0\),0otherwise\.\\Omega\_\{k\}\(g,p\)\(x\)=\\begin\{cases\}\\frac\{1\}\{2^\{i\+1\}\}&\\text\{ if \}x=x\_\{\\lceil k\\rceil\+i\}\\quad\(i\\geq 0\),\\\\ 0&\\text\{ otherwise\.\}\\end\{cases\}
For completeness, letΩ\\Omegabehave like a continuous Boltzmann optimizer for all unspecified inputs\. Askkincreases, whenever it passes an integer threshold, the probabilities will move up one value in the sequence\. SinceV​\(s1\)≥𝔼p​\[g​\(x\)\]V\(s\_\{1\}\)\\geq\\mathbb\{E\}\_\{p\}\[g\(x\)\], the expectation is nondecreasing askkbecomes positive\. Also, points farther up the sequence have higher value with respect toVV, so𝔼Ωk​\(g,p\)​\[g​\(x\)\]\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[g\(x\)\]is monotonically increasing\. Sincesis\_\{i\}approachess∗s^\{\*\}, a maximizer ofVV, the probability will be concentrated in arbitrarily high\-valued world states as well, soΩ\\Omegameets the criteria of an optimizer\.

SinceUUis continuous and sincesis\_\{i\}approachess∗s^\{\*\}, the expectation offfunderΩ\\Omegawill approachU​\(s∗\)U\(s^\{\*\}\)\. Due to our assumption thatggisη\\eta\-catastrophic forff, we have the desired result,

lim supk→∞𝔼Ωk​\(g,p\)​\[f​\(x\)\]=U​\(s∗\)≤η\.\\limsup\_\{k\\rightarrow\\infty\}\\mathbb\{E\}\_\{\\Omega\_\{k\}\(g,p\)\}\[f\(x\)\]=U\(s^\{\*\}\)\\leq\\eta\.
\(⟸\\Longleftarrow\) Lets∗∈Ξs^\{\*\}\\in\\Xihave the property thatU​\(s∗\)≤ηU\(s^\{\*\}\)\\leq\\eta\. We will construct a continuous functionV:ℝn→ℝV:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\}that is strictly increasing and has a unique maximizer ats∗s^\{\*\}withincl\(S\)\\operatorname\*\{cl\}\(S\)\.

We proceed by constructing a subsetD⊂ℝnD\\subset\\mathbb\{R\}^\{n\}that is the level hypersurface of some continuous strictly increasing functionVVand intersectss∗s^\{\*\}and no other point incl\(S\)\\operatorname\*\{cl\}\(S\)\. By[LemmaA\.3](https://arxiv.org/html/2607.28881#A1.ThmLemma3),DDis the level surface of some continuous strictly increasing functionVVif and only if it satisfies∀s,r∈D,s⊀r\\forall s,r\\in D,s\\not\\prec rand∀s∈ℝn,∃\!⁡k∈ℝ:s−k​𝟏∈D\\forall s\\in\\mathbb\{R\}^\{n\},\\exists\!k\\in\\mathbb\{R\}:s\-k\\bm\{1\}\\in D, so we will constructDDto have these properties\.

LetΠ\\Pidenote the set of points thats∗s^\{\*\}precedes or is equal to:Π=\{q∈ℝn:s∗⪯q\}\\Pi=\\\{q\\in\\mathbb\{R\}^\{n\}:s^\{\*\}\\preceq q\\\}\. Sinces∗∈Ξs^\{\*\}\\in\\Xi, we know that∀q∈Π−\{s∗\},∃ϵ\>0:Bϵ\(q\)⊂cl\(S\)C\\forall q\\in\\Pi\-\\\{s^\{\*\}\\\},\\exists\\epsilon\>0:B\_\{\\epsilon\}\(q\)\\subset\\operatorname\*\{cl\}\(S\)^\{C\}\. We can imagine a “cloud” aroundΠ\\Pithat our surfaceDDcan inhabit without overlapping withSS\. Letd​\(r,S\)d\(r,S\)denote the distance from a pointr∈ℝnr\\in\\mathbb\{R\}^\{n\}to the setSS:d​\(r,S\)=infs∈Sd​\(r,s\)d\(r,S\)=\\inf\_\{s\\in S\}d\(r,s\)\. Then, the set⋃q∈ΠBd​\(q,S\)2​\(q\)\\bigcup\_\{q\\in\\Pi\}B\_\{\\frac\{d\(q,S\)\}\{2\}\}\(q\)represents such a cloud around \(and including\)Π\\Pi\.

Now consider the family of hyperplanesPvP\_\{v\}, parametrized by a real numbervvand defined by the equation

∑i\(xi−si∗\)=v\.\\sum\_\{i\}\(x\_\{i\}\-s^\{\*\}\_\{i\}\)=v\.
Notice thatP0P\_\{0\}intersectss∗s^\{\*\}\. The intersectionΠ∩Pv\\Pi\\cap P\_\{v\}forms a simplex forv\>0v\>0and atv=0v=0,Π∩P0=\{s∗\}\\Pi\\cap P\_\{0\}=\\\{s^\{\*\}\\\}\. Define the functiona1​\(v\)a\_\{1\}\(v\)as one\-half the Euclidean distance from the setΠ∩Pv\\Pi\\cap P\_\{v\}toSSfor nonnegative values ofvv:

a1​\(v\)=d​\(Π∩Pv,S\)2a\_\{1\}\(v\)=\\frac\{d\(\\Pi\\cap P\_\{v\},S\)\}\{2\}
where the distance between two subsets ofℝn\\mathbb\{R\}^\{n\},RRandSS, is defined asd​\(R,S\)=infr∈R,s∈Sd​\(r,s\)d\(R,S\)=\\inf\_\{r\\in R,s\\in S\}d\(r,s\)\. SinceΠ∩Pv\\Pi\\cap P\_\{v\}is compact, for allv\>0v\>0, there is a pointq∈Π∩Pvq\\in\\Pi\\cap P\_\{v\}such thatd​\(q,S\)2=a1​\(v\)\\frac\{d\(q,S\)\}\{2\}=a\_\{1\}\(v\)\.

This function has some useful properties\. First we prove thata1​\(v\)a\_\{1\}\(v\)is continuous\. Consider positive real numbervvandv′v^\{\\prime\}witha1​\(v′\)≥a1​\(v\)a\_\{1\}\(v^\{\\prime\}\)\\geq a\_\{1\}\(v\)and letq∈Π∩Pvq\\in\\Pi\\cap P\_\{v\}be a state such thatd​\(q,S\)/2=a1​\(v\)d\(q,S\)/2=a\_\{1\}\(v\)\. Sinceq∈Π∩Pvq\\in\\Pi\\cap P\_\{v\}, we have that∑i\(qi−si∗\)=v\\sum\_\{i\}\(q\_\{i\}\-s^\{\*\}\_\{i\}\)=v\. Definingq′=q−\(1−v′v\)​\(q−s∗\)q^\{\\prime\}=q\-\(1\-\\frac\{v^\{\\prime\}\}\{v\}\)\(q\-s^\{\*\}\), we find

∑i\(qi′−si∗\)=∑i\(qi−si∗\)−\(1−v′v\)​∑i\(qi−si∗\)=v′\.\\sum\_\{i\}\(q^\{\\prime\}\_\{i\}\-s^\{\*\}\_\{i\}\)=\\sum\_\{i\}\(q\_\{i\}\-s^\{\*\}\_\{i\}\)\-\\left\(1\-\\frac\{v^\{\\prime\}\}\{v\}\\right\)\\sum\_\{i\}\(q\_\{i\}\-s^\{\*\}\_\{i\}\)=v^\{\\prime\}\.
Also, for all positivev′v^\{\\prime\}andvv,1\>1−v′v1\>1\-\\frac\{v^\{\\prime\}\}\{v\}, sos∗⪯q′s^\{\*\}\\preceq q^\{\\prime\}andq′∈Π∩Pv′q^\{\\prime\}\\in\\Pi\\cap P\_\{v^\{\\prime\}\}\. The distanced​\(q′,q\)d\(q^\{\\prime\},q\)is given by

d​\(q′,q\)=\|1−v′v\|​d​\(q,s∗\)≤\|v−v′\|\.d\(q^\{\\prime\},q\)=\\left\|1\-\\frac\{v^\{\\prime\}\}\{v\}\\right\|d\(q,s^\{\*\}\)\\leq\|v\-v^\{\\prime\}\|\.
Therefore, we have

\|a1​\(v′\)−a1​\(v\)\|=a1​\(v′\)−a1​\(v\)≤d​\(q′,S\)2−d​\(q,S\)2≤d​\(q′,q\)\+d​\(q,S\)2−d​\(q,S\)2≤\|v′−v\|2\|a\_\{1\}\(v^\{\\prime\}\)\-a\_\{1\}\(v\)\|=a\_\{1\}\(v^\{\\prime\}\)\-a\_\{1\}\(v\)\\leq\\frac\{d\(q^\{\\prime\},S\)\}\{2\}\-\\frac\{d\(q,S\)\}\{2\}\\leq\\frac\{d\(q^\{\\prime\},q\)\+d\(q,S\)\}\{2\}\-\\frac\{d\(q,S\)\}\{2\}\\leq\\frac\{\|v^\{\\prime\}\-v\|\}\{2\}
which shows thata1​\(v\)a\_\{1\}\(v\)is1/21/2\-Lipschitz forv\>0v\>0\. Additionally, sinces∗∈cl\(S\)s^\{\*\}\\in\\operatorname\*\{cl\}\(S\), for allvv,a1​\(v\)≤va\_\{1\}\(v\)\\leq vand thus,limv→0\+a1​\(v\)=0=a1​\(0\)\\lim\_\{v\\rightarrow 0^\{\+\}\}a\_\{1\}\(v\)=0=a\_\{1\}\(0\), showinga1a\_\{1\}is continuous everywhere\.

Next, we prove thata1​\(v\)a\_\{1\}\(v\)is positive forv\>0v\>0\. Suppose for the sake of contradiction thatv\>0v\>0anda1​\(v\)=0a\_\{1\}\(v\)=0\. Then, ifq∈Π∩Pvq\\in\\Pi\\cap P\_\{v\}is a point such thatd​\(q,S\)/2=a1​\(v\)=0d\(q,S\)/2=a\_\{1\}\(v\)=0, this impliesq∈cl\(S\)q\\in\\operatorname\*\{cl\}\(S\)\. But,s∗≺qs^\{\*\}\\prec qwhich contradicts the assumption thats∗∈Ξs^\{\*\}\\in\\Xi\.

Finally, notice that sinceSSis bounded,a1​\(v\)a\_\{1\}\(v\)is eventually strictly increasing\. Consider the new functiona2​\(v\)a\_\{2\}\(v\)defined as

a2​\(v\)=infv′≥va1​\(v′\)\.a\_\{2\}\(v\)=\\inf\_\{v^\{\\prime\}\\geq v\}a\_\{1\}\(v^\{\\prime\}\)\.
By definition,a2​\(v\)a\_\{2\}\(v\)is nondecreasing\. Also,a2​\(v\)a\_\{2\}\(v\)is continuous sincea1​\(v\)a\_\{1\}\(v\)is continuous, anda2​\(v\)a\_\{2\}\(v\)is positive forv\>0v\>0sincea1​\(v\)a\_\{1\}\(v\)is positive forv\>0v\>0and is eventually strictly increasing\. Sincea1​\(v\)a\_\{1\}\(v\)is eventually strictly increasing,a2​\(v\)a\_\{2\}\(v\)will eventually be equal toa1​\(v\)a\_\{1\}\(v\)and will thus eventually be strictly increasing as well\.

Suppose thata2a\_\{2\}is constant on the intervals\[α1,β1\],\[α2,β2\],…\[\\alpha\_\{1\},\\beta\_\{1\}\],\[\\alpha\_\{2\},\\beta\_\{2\}\],\\dotswhere this list may be countably infinite\. The values of the endpointsβi\\beta\_\{i\}are bounded above sincea2a\_\{2\}is eventually strictly increasing\. Then, we can construct the new functiona3​\(v\)a\_\{3\}\(v\)defined as

a3​\(v\)=min⁡\(a2​\(v\),infi≥1:βi\>v\{a2​\(βi\)βi​v\}\)\.a\_\{3\}\(v\)=\\min\\left\(a\_\{2\}\(v\),\\inf\_\{i\\geq 1:\\beta\_\{i\}\>v\}\\left\\\{\\frac\{a\_\{2\}\(\\beta\_\{i\}\)\}\{\\beta\_\{i\}\}v\\right\\\}\\right\)\.
![Refer to caption](https://arxiv.org/html/2607.28881v1/x7.png)

![Refer to caption](https://arxiv.org/html/2607.28881v1/x8.png)

Figure 7:Two graphs showing the construction of functions used in the proof\.Left:A graph showing the construction ofa2a\_\{2\}froma1a\_\{1\}\. The functiona2a\_\{2\}only increases whena1a\_\{1\}never falls below a given value again, soa2a\_\{2\}is nondecreasing\. Sincea1a\_\{1\}is eventually strictly increasing,a2a\_\{2\}is eventually equal toa1a\_\{1\}and is eventually strictly increasing as well\.Right:A graph showing the construction ofa3a\_\{3\}froma2a\_\{2\}\. The red straight lines connect the endpoint of each constant region to the origin\. The functiona3a\_\{3\}will take the value of the solid parts of the segments which fall underneath the value ofa2a\_\{2\}\. Since the line segments are strictly increasing anda2a\_\{2\}is eventually strictly increasing,a3a\_\{3\}is strictly increasing everywhere\.Essentially,a3​\(v\)a\_\{3\}\(v\)takes the smallest value ofa2​\(v\)a\_\{2\}\(v\)and the line segments connecting the endpoints of each constant section\(βi,a2​\(βi\)\)\(\\beta\_\{i\},a\_\{2\}\(\\beta\_\{i\}\)\)to the origin\(0,0\)\(0,0\)at each value ofvv\. The construction of the functionsa2a\_\{2\}anda3a\_\{3\}is pictured in[Fig\.7](https://arxiv.org/html/2607.28881#A1.F7)\. Since the values of the endpoints are bounded,infi≥1:βi\>v\{a2​\(βi\)βi​v\}\\inf\_\{i\\geq 1:\\beta\_\{i\}\>v\}\\left\\\{\\frac\{a\_\{2\}\(\\beta\_\{i\}\)\}\{\\beta\_\{i\}\}v\\right\\\}is positive for allv\>0v\>0, and thusa3​\(v\)a\_\{3\}\(v\)is positive forv\>0v\>0\. Additionally,a3​\(v\)a\_\{3\}\(v\)is continuous sincea2​\(v\)a\_\{2\}\(v\)and the line segments are continuous as well\. Since each line segment is strictly increasing anda2​\(v\)a\_\{2\}\(v\)is eventually strictly increasing,a3​\(v\)a\_\{3\}\(v\)is strictly increasing everywhere\.

Finally, we define the setDDusing the functiona3​\(v\)a\_\{3\}\(v\)\. Sincea3​\(v\)a\_\{3\}\(v\)is less than the distanced​\(Π∩Pv,S\)d\(\\Pi\\cap P\_\{v\},S\), thea3​\(v\)a\_\{3\}\(v\)\-radius ball around any pointq∈Π∩Pvq\\in\\Pi\\cap P\_\{v\}\(Including the empty ball of radius 0 ats∗s^\{\*\}\) will be entirely outside ofcl\(S\)\\operatorname\*\{cl\}\(S\)– as will the union of all such balls\. LetBϵ​\(R\)B\_\{\\epsilon\}\(R\)denote the union of balls of radiusϵ\\epsilonaround all points withinRR, defined as

Bϵ​\(R\)=\{s∈ℝn:d​\(s,R\)<ϵ\}\.B\_\{\\epsilon\}\(R\)=\\\{s\\in\\mathbb\{R\}^\{n\}:d\(s,R\)<\\epsilon\\\}\.
We will defineDDto be the set of points constructed from the boundary of the union of the balls of radiusa3​\(v\)a\_\{3\}\(v\)around the setsΠ∩Pv\\Pi\\cap P\_\{v\}\. Specifically,DDis defined as

D=bd\(⋃v≥0\[Pv∩Ba3​\(v\)​\(Π∩Pv\)\]\)\.D=\\operatorname\*\{bd\}\\left\(\\bigcup\_\{v\\geq 0\}\\left\[P\_\{v\}\\cap B\_\{a\_\{3\}\(v\)\}\(\\Pi\\cap P\_\{v\}\)\\right\]\\right\)\.
Due to the definition ofa3a\_\{3\},DDintersects the closure ofSSexactly ats∗s^\{\*\}\. We claim thatDDsatisfies the properties of[LemmaA\.3](https://arxiv.org/html/2607.28881#A1.ThmLemma3)\. To aid in our proofs of these two properties, we definev​\(s\)=∑i\(si−si∗\)v\(s\)=\\sum\_\{i\}\(s\_\{i\}\-s^\{\*\}\_\{i\}\)which is the value such thatssis a member of the planePv​\(s\)P\_\{v\(s\)\}and we defineb​\(s\)=d​\(s,Π∩Pv​\(s\)\)b\(s\)=d\(s,\\Pi\\cap P\_\{v\(s\)\}\)fors∈ℝns\\in\\mathbb\{R\}^\{n\}such thatv​\(s\)≥0v\(s\)\\geq 0\.

First, we will showvvandbbare continuous\. The functionvvis Lipschitz due to the equivalence of norms inℝn\\mathbb\{R\}^\{n\}since

\|v​\(s′\)−v​\(s\)\|=\|∑isi′−si\|≤‖s′−s‖1\.\|v\(s^\{\\prime\}\)\-v\(s\)\|=\\left\|\\sum\_\{i\}s^\{\\prime\}\_\{i\}\-s\_\{i\}\\right\|\\leq\\\|s^\{\\prime\}\-s\\\|\_\{1\}\.
Now, takes,s′∈ℝns,s^\{\\prime\}\\in\\mathbb\{R\}^\{n\}such thatb​\(s′\)≥b​\(s\)b\(s^\{\\prime\}\)\\geq b\(s\)andv​\(s′\),v​\(s\)v\(s^\{\\prime\}\),v\(s\)are positive\. Letq∈Π∩Pv​\(s\)q\\in\\Pi\\cap P\_\{v\(s\)\}be such thatd​\(q,s\)=b​\(s\)d\(q,s\)=b\(s\)\. Then,q′=q−\(1−v​\(s′\)v​\(s\)\)​\(q−s∗\)∈Π∩Pv​\(s′\)q^\{\\prime\}=q\-\(1\-\\frac\{v\(s^\{\\prime\}\)\}\{v\(s\)\}\)\(q\-s^\{\*\}\)\\in\\Pi\\cap P\_\{v\(s^\{\\prime\}\)\}, so we have

\|b​\(s′\)−b​\(s\)\|=b​\(s′\)−b​\(s\)≤d​\(q′,s′\)−d​\(q,s\)≤d​\(q′,q\)\+d​\(s,s′\)≤\|v​\(s′\)−v​\(s\)\|\+d​\(s,s′\)\.\|b\(s^\{\\prime\}\)\-b\(s\)\|=b\(s^\{\\prime\}\)\-b\(s\)\\leq d\(q^\{\\prime\},s^\{\\prime\}\)\-d\(q,s\)\\leq d\(q^\{\\prime\},q\)\+d\(s,s^\{\\prime\}\)\\leq\|v\(s^\{\\prime\}\)\-v\(s\)\|\+d\(s,s^\{\\prime\}\)\.
Sincevvis Lipschitz, this showsbbis as well\. Next, we will prove thats∈Ds\\in Dif and only ifb​\(s\)=a3​\(v​\(s\)\)b\(s\)=a\_\{3\}\(v\(s\)\)\. Define the setEEas

E=⋃v≥0\[Pv∩Ba3​\(v\)​\(Π∩Pv\)\]E=\\bigcup\_\{v\\geq 0\}\\left\[P\_\{v\}\\cap B\_\{a\_\{3\}\(v\)\}\(\\Pi\\cap P\_\{v\}\)\\right\]
so thatD=bd\(E\)D=\\operatorname\*\{bd\}\(E\)\. Each points∈ℝns\\in\\mathbb\{R\}^\{n\}belongs to exactly one plane,Pv​\(s\)P\_\{v\(s\)\}, sos∈Es\\in Eif and only if there is a pointq∈Π∩Pv​\(s\)q\\in\\Pi\\cap P\_\{v\(s\)\}such thatd​\(q,s\)<a3​\(v​\(s\)\)d\(q,s\)<a\_\{3\}\(v\(s\)\)\. Thus, we can rewriteEEas

E=\{s∈ℝn:v​\(s\)≥0,b​\(s\)<a3​\(v​\(s\)\)\}\.E=\\\{s\\in\\mathbb\{R\}^\{n\}:v\(s\)\\geq 0,b\(s\)<a\_\{3\}\(v\(s\)\)\\\}\.
The setEEis the union of open sets, so it is open\. Ifb​\(s\)=a3​\(v​\(s\)\)b\(s\)=a\_\{3\}\(v\(s\)\), then there is someq∈Π∩Pv​\(s\)q\\in\\Pi\\cap P\_\{v\(s\)\}such thats∈cl\(Ba3​\(v\)​\(q\)\)⊂cl\(E\)s\\in\\operatorname\*\{cl\}\(B\_\{a\_\{3\}\(v\)\}\(q\)\)\\subset\\operatorname\*\{cl\}\(E\)\. Ifb​\(s\)\>a3​\(v​\(s\)\)b\(s\)\>a\_\{3\}\(v\(s\)\), then there is no pointq∈Π∩Pv​\(s\)q\\in\\Pi\\cap P\_\{v\(s\)\}such thats∈cl\(Ba3​\(q\)\)s\\in\\operatorname\*\{cl\}\(B\_\{a\_\{3\}\}\(q\)\), and sossis not a member of the closure ofEE\. Put together, we can write the closure ofEEas

cl\(E\)=\{s∈ℝn:v​\(s\)≥0,b​\(s\)≤a3​\(v​\(s\)\)\}\\operatorname\*\{cl\}\(E\)=\\\{s\\in\\mathbb\{R\}^\{n\}:v\(s\)\\geq 0,b\(s\)\\leq a\_\{3\}\(v\(s\)\)\\\}
and thusD=cl\(E\)−ED=\\operatorname\*\{cl\}\(E\)\-Ecan be rewritten as

D=\{s∈ℝn:v​\(s\)≥0,b​\(s\)=a3​\(v​\(s\)\)\}\.D=\\\{s\\in\\mathbb\{R\}^\{n\}:v\(s\)\\geq 0,b\(s\)=a\_\{3\}\(v\(s\)\)\\\}\.
Therefore,s∈Ds\\in Dif and only ifb​\(s\)=a3​\(v​\(s\)\)b\(s\)=a\_\{3\}\(v\(s\)\)\.

Now, we will show that∀s,r∈D,s⊀r\\forall s,r\\in D,s\\not\\prec r\. Suppose for the sake of contradiction that there are pointss,r∈Ds,r\\in Dsuch thats≺rs\\prec r\. Then, it must be true thatv​\(s\)<v​\(r\)v\(s\)<v\(r\)\. Letq∈Π∩Pv​\(s\)q\\in\\Pi\\cap P\_\{v\(s\)\}be a point such thatd​\(s,q\)=d​\(s,Π∩Pv​\(s\)\)=a3​\(v​\(s\)\)d\(s,q\)=d\(s,\\Pi\\cap P\_\{v\(s\)\}\)=a\_\{3\}\(v\(s\)\)\. We know thats,q∈Pv​\(s\)s,q\\in P\_\{v\(s\)\}andr∈Pv​\(r\)r\\in P\_\{v\(r\)\}\. Using the definition of the planesPvP\_\{v\}, we find

∑i\(qi\+ri−si\)−si∗=v​\(r\)\+∑i\(qi−si∗\)−∑i\(si−si∗\)=v​\(r\)\.\\sum\_\{i\}\(q\_\{i\}\+r\_\{i\}\-s\_\{i\}\)\-s\_\{i\}^\{\*\}=v\(r\)\+\\sum\_\{i\}\(q\_\{i\}\-s\_\{i\}^\{\*\}\)\-\\sum\_\{i\}\(s\_\{i\}\-s\_\{i\}^\{\*\}\)=v\(r\)\.
So,q\+r−sq\+r\-sis a member ofΠ∩Pv​\(r\)\\Pi\\cap P\_\{v\(r\)\}\. But then this gives

a3​\(v​\(r\)\)=d​\(r,Π∩Pv​\(r\)\)≤d​\(r,q\+r−s\)=d​\(q,s\)=a3​\(v​\(s\)\)\.a\_\{3\}\(v\(r\)\)=d\(r,\\Pi\\cap P\_\{v\(r\)\}\)\\leq d\(r,q\+r\-s\)=d\(q,s\)=a\_\{3\}\(v\(s\)\)\.
Buta3a\_\{3\}is strictly increasing, making this a contradiction\. This proves that no members ofDDcan precede each other\.

Finally, we show that for alls∈ℝns\\in\\mathbb\{R\}^\{n\}, there is a uniquek∈ℝk\\in\\mathbb\{R\}such thats−k​𝟏∈Ds\-k\\bm\{1\}\\in D\. Consider, fors∈ℝns\\in\\mathbb\{R\}^\{n\}, the line defined byl​\(k\)=s\+k​𝟏l\(k\)=s\+k\\bm\{1\}for all realkk\. There is ak0k\_\{0\}suchl​\(k0\)∈P0l\(k\_\{0\}\)\\in P\_\{0\}and akΠk\_\{\\Pi\}such thatl​\(kΠ\)l\(k\_\{\\Pi\}\)is on the boundary ofΠ\\Pi\. Thus, we haveb​\(l​\(k0\)\)≥0b\(l\(k\_\{0\}\)\)\\geq 0andb​\(l​\(kΠ\)\)=0b\(l\(k\_\{\\Pi\}\)\)=0\. Additionally, over the range\[k0,kΠ\]\[k\_\{0\},k\_\{\\Pi\}\],b​\(l​\(k\)\)b\(l\(k\)\)is decreasing\. To see why, suppose thatqqis the nearest point in the setΠ∩Pv​\(l​\(k\)\)\\Pi\\cap P\_\{v\(l\(k\)\)\}tol​\(k\)l\(k\)\. Then, for allt\>0t\>0,q′=q\+t​𝟏q^\{\\prime\}=q\+t\\bm\{1\}is a member ofΠ∩Pv​\(l​\(k\+t\)\)\\Pi\\cap P\_\{v\(l\(k\+t\)\)\}and thus,b​\(l​\(k\)\)=d​\(l​\(k\),q\)=d​\(l​\(k\+t\),q′\)≥b​\(l​\(k\+t\)\)b\(l\(k\)\)=d\(l\(k\),q\)=d\(l\(k\+t\),q^\{\\prime\}\)\\geq b\(l\(k\+t\)\)which shows thatb​\(l​\(k\)\)b\(l\(k\)\)is decreasing withkk\.

Sincea3a\_\{3\}is continuous and strictly increasing,a3​\(v​\(l​\(k\)\)\)a\_\{3\}\(v\(l\(k\)\)\)is continuous and strictly increasing withkk\. Additionally sinceb​\(l​\(k\)\)b\(l\(k\)\)is continuous and decreasing withkk, there must be a uniquekk\-value such thatb​\(l​\(k\)\)=a3​\(v​\(l​\(k\)\)\)b\(l\(k\)\)=a\_\{3\}\(v\(l\(k\)\)\)\. In other words, there is a uniquekk\-value such thats\+k​𝟏=s−\(−k\)​𝟏∈Ds\+k\\bm\{1\}=s\-\(\-k\)\\bm\{1\}\\in Das desired\.

With all of the needed properties satisfied,DDforms a valid level surface of a strictly increasing continuous functionVVaccording to[LemmaA\.3](https://arxiv.org/html/2607.28881#A1.ThmLemma3)\.

Considerr∈ℝnr\\in\\mathbb\{R\}^\{n\}withV​\(r\)\>V​\(s∗\)V\(r\)\>V\(s^\{\*\}\)\. Then, there is a uniquekksuch thatr\+k​𝟏∈Dr\+k\\bm\{1\}\\in Dwhich givesV​\(r\+k​𝟏\)=V​\(s∗\)V\(r\+k\\bm\{1\}\)=V\(s^\{\*\}\)\. SinceVVis strictly increasing,kkmust be negative and sinceb​\(r\+k​𝟏\)b\(r\+k\\bm\{1\}\)is decreasing withkkanda3​\(v​\(r\+k​𝟏\)\)a\_\{3\}\(v\(r\+k\\bm\{1\}\)\)is strictly increasing withkk, it must be the case thatb​\(r\)<a3​\(v​\(r\)\)b\(r\)<a\_\{3\}\(v\(r\)\)which showsr∈Er\\in E\. SinceE∩cl\(S\)=∅E\\cap\\operatorname\*\{cl\}\(S\)=\\emptyset, the highest\-valued level hypersurface that intersectscl\(S\)\\operatorname\*\{cl\}\(S\)isDD, thusVVachieves its maximum overcl\(S\)\\operatorname\*\{cl\}\(S\)ats∗s^\{\*\}\. By[LemmaA\.4](https://arxiv.org/html/2607.28881#A1.ThmLemma4),g​\(x\)=V​\(L​\(x\)\)g\(x\)=V\(L\(x\)\)is then anη\\eta\-catastrophic tradeoff proxy forff\. ∎

## Acknowledgments

We would like to thank Léo Cymbalista for their help with the proof of[Theorem5\.1](https://arxiv.org/html/2607.28881#S5.ThmTheorem1)and for designing the value functions and distribution used in[Fig\.3](https://arxiv.org/html/2607.28881#S5.F3)and[Fig\.4](https://arxiv.org/html/2607.28881#S5.F4)\. We’d also like to thank Santiago Cifuentes, Alfred Harwood, and Alex Altair for their comments and suggestions\.

## References

- S\. Armstrong and S\. Mindermann \(2018\)Occam’s razor is insufficient to infer the preferences of irrational agents\.InAdvances in Neural Information Processing Systems,Vol\.31,pp\. 5603–5614\.Cited by:[§1](https://arxiv.org/html/2607.28881#S1.p2.1),[§2](https://arxiv.org/html/2607.28881#S2.p1.1)\.
- B\. Baker, J\. Huizinga, L\. Gao, Z\. Dou, M\. Y\. Guan, A\. Madry, W\. Zaremba, J\. Pachocki, and D\. Farhi \(2025\)Monitoring reasoning models for misbehavior and the risks of promoting obfuscation\.Note:arXiv preprint arXiv:2503\.11926Cited by:[§1](https://arxiv.org/html/2607.28881#S1.p3.1)\.
- C\. Bottomley and T\. L\. Williamson \(2025\)On the offense against fanaticism\.Ethics135\(2\),pp\. 320–332\.External Links:[Document](https://dx.doi.org/10.1086/732617)Cited by:[§3](https://arxiv.org/html/2607.28881#S3.p5.2)\.
- K\. Dingle, G\. Valle Pérez, and A\. A\. Louis \(2020\)Generic predictions of output probability based on complexities of inputs and outputs\.Scientific Reports10\(1\),pp\. 4415\.External Links:[Document](https://dx.doi.org/10.1038/s41598-020-61135-7)Cited by:[§7](https://arxiv.org/html/2607.28881#S7.p2.1)\.
- T\. Everitt, V\. Krakovna, L\. Orseau, M\. Hutter, and S\. Legg \(2017\)Reinforcement learning with a corrupted reward channel\.InProceedings of the Twenty\-Sixth International Joint Conference on Artificial Intelligence \(IJCAI\-17\),pp\. 4705–4713\.External Links:[Document](https://dx.doi.org/10.24963/ijcai.2017/656)Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p3.1)\.
- L\. Gao, J\. Schulman, and J\. Hilton \(2023\)Scaling laws for reward model overoptimization\.InProceedings of the 40th International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.202,pp\. 10835–10866\.Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p2.1)\.
- D\. Hadfield\-Menell and G\. K\. Hadfield \(2019\)Incomplete contracting and AI alignment\.InProceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society \(AIES\),pp\. 417–422\.External Links:[Document](https://dx.doi.org/10.1145/3306618.3314250)Cited by:[§1](https://arxiv.org/html/2607.28881#S1.p2.1)\.
- T\. Kwa, D\. Thomas, and A\. Garriga\-Alonso \(2024\)Catastrophic Goodhart: regularizing RLHF with KL divergence does not mitigate heavy\-tailed reward misspecification\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 14608–14633\.Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p2.1)\.
- J\. Lehman, J\. Clune, D\. Misevic, C\. Adami, L\. Altenberg, J\. Beaulieu, P\. J\. Bentley, S\. Bernard, G\. Beslon, D\. M\. Bryson, N\. Cheney, P\. Chrabaszcz, A\. Cully, S\. Doncieux, F\. C\. Dyer, K\. O\. Ellefsen, R\. Feldt, S\. Fischer, S\. Forrest, A\. Frénoy, C\. Gagné, L\. Le Goff, L\. M\. Grabowski, B\. Hodjat, F\. Hutter, L\. Keller, C\. Knibbe, P\. Krcah, R\. E\. Lenski, H\. Lipson, R\. MacCurdy, C\. Maestre, R\. Miikkulainen, S\. Mitri, D\. E\. Moriarty, J\. Mouret, A\. Nguyen, C\. Ofria, M\. Parizeau, D\. Parsons, R\. T\. Pennock, W\. F\. Punch, T\. S\. Ray, M\. Schoenauer, E\. Schulte, K\. Sims, K\. O\. Stanley, F\. Taddei, D\. Tarapore, S\. Thibault, R\. Watson, W\. Weimer, and J\. Yosinski \(2020\)The surprising creativity of digital evolution: a collection of anecdotes from the evolutionary computation and artificial life research communities\.Artificial Life26\(2\),pp\. 274–306\.External Links:[Document](https://dx.doi.org/10.1162/artl%5Fa%5F00319)Cited by:[§1](https://arxiv.org/html/2607.28881#S1.p3.1)\.
- D\. Manheim and S\. Garrabrant \(2018\)Categorizing variants of Goodhart’s law\.Note:arXiv preprint arXiv:1803\.04585Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p2.1)\.
- S\. Neth \(2026\)Against proxy optimization\.Philosophy and Phenomenological Research\.Note:Early View\.[https://doi\.org/10\.1111/phpr\.70138](https://doi.org/10.1111/phpr.70138)Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p3.1),[§6\.1](https://arxiv.org/html/2607.28881#S6.SS1.p5.1)\.
- S\. Pitis \(2023\)Consistent aggregation of objectives with diverse time preferences requires non\-Markovian rewards\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 2877–2893\.Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p1.1)\.
- D\. Silver, S\. Singh, D\. Precup, and R\. S\. Sutton \(2021\)Reward is enough\.Artificial Intelligence299,pp\. 103535\.External Links:[Document](https://dx.doi.org/10.1016/j.artint.2021.103535)Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p1.1)\.
- J\. Skalse and A\. Abate \(2023\)On the limitations of Markovian rewards to express multi\-objective, risk\-sensitive, and modal tasks\.InProceedings of the Thirty\-Ninth Conference on Uncertainty in Artificial Intelligence \(UAI\),Proceedings of Machine Learning Research, Vol\.216,pp\. 1974–1984\.Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p1.1)\.
- J\. Skalse, N\. Howe, D\. Krasheninnikov, and D\. Krueger \(2022\)Defining and characterizing reward gaming\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 9460–9471\.Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p3.1)\.
- J\. Taylor \(2016\)Quantilizers: a safer alternative to maximizers for limited optimization\.InAI, Ethics, and Society: Papers from the 2016 AAAI Workshop,AAAI Technical Report\.Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p4.2),[§7](https://arxiv.org/html/2607.28881#S7.p3.1)\.
- H\. Wilkinson \(2022\)In defense of fanaticism\.Ethics132\(2\),pp\. 445–477\.External Links:[Document](https://dx.doi.org/10.1086/716869)Cited by:[§3](https://arxiv.org/html/2607.28881#S3.p5.2)\.
- E\. Yudkowsky \(2009\)Value is fragile\.Note:LessWrong,[https://www\.lesswrong\.com/posts/GNnHHmm8EzePmKzPk/value\-is\-fragile](https://www.lesswrong.com/posts/GNnHHmm8EzePmKzPk/value-is-fragile)Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p1.1),[§2](https://arxiv.org/html/2607.28881#S2.p3.1),[§6\.3](https://arxiv.org/html/2607.28881#S6.SS3.p1.1)\.
- S\. Zhuang and D\. Hadfield\-Menell \(2020\)Consequences of misaligned AI\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 15763–15773\.Cited by:[§2](https://arxiv.org/html/2607.28881#S2.p3.1),[§6\.1](https://arxiv.org/html/2607.28881#S6.SS1.p5.1),[§6\.3](https://arxiv.org/html/2607.28881#S6.SS3.p1.1),[§6](https://arxiv.org/html/2607.28881#S6.p1.1)\.