Jev 风格的类型化决策模型中，标签压过定义

# Labels Override Definitions in Jev-Style Typed Decision Models
Source: [https://arxiv.org/html/2610.02586](https://arxiv.org/html/2610.02586)
Seyedarmin Azizi††thanks:Corresponding author\.Erfan Baghaei Potraghloo & Massoud PedramAffiliation:\{seyedarm, baghaeip, pedram\}@usc\.edu

###### Abstract

A typed decision model answers a fixed question about an input by returning a probability for each of several caller\-defined options\. Each option carries a short label and a written definition, which is where a developer states the rule the model should apply\. Jev introduced this interface for routing, moderation and triage, open implementations followed, and the same operation occurs whenever a language model is used as a classifier by scoring label strings\. We study the open implementations, whose weights we can inspect and patch, and ask whether the probability follows the definitions or the labels\. A preference for the label we call*option\-label bias*\. Across four open\-weight typed decision models, three ways of reading an answer from a Qwen2\.5 backbone, eleven classification tasks and PolicyBench, a synthetic routing suite we introduce in which the rule appears only in the definitions, the answer is mostly the labels\. Deleting every definition leaves accuracy unchanged \(laya\-td: 0\.8559 against 0\.8487\), although those definitions support 0\.7971 on their own, and renaming the options toAandB*raises*accuracy by \+0\.1511\[\+0\.1377,\+0\.1646\]\[\+0\.1377,\\,\+0\.1646\]\. One system,von, is unaffected, and the two code bases differ in one expression:layawrites each option as"\{label\}: \{definition\}", whilevonwrites only the definition\. Changing that expression in both directions, with no weight changed, makes all threelayacheckpoints exactly invariant \(\+0\.0000\+0\.0000\[\+0\.0000,\+0\.0000\]\[\+0\.0000,\+0\.0000\]\) and creates the effect invon, whose accuracy falls from 0\.8511 to 0\.2281 when a label contradicts its definition\. Earlier work attributed this failure to the constrained decision head these models use in place of a text decoder; our results locate it in the prompt rendering\. We give a two\-call test that tells a practitioner which case applies to their model, and measure what four mitigations are worth\.

Figure 1:PolicyBench, the synthetic routing suite of §[4\.3](https://arxiv.org/html/2610.02586#S4.SS3), in its controlled form \(XF, in which the two options carry definitions of comparable specificity\)\. The decision rule is stated only in the option definition, so the label carries no information by construction and the two right\-hand groups are controls\. Series are three typed decision models \(laya\-td,laya\-en,von\) and one language\-model readout \(Qwen2\.5\-7B scoring the option strings\);laya\-mlis near chance on this suite throughout and is reported in Table[2](https://arxiv.org/html/2610.02586#S5.T2)rather than plotted\. Every series exceptvonloses accuracy when the options are given meaningful names rather thanAandB, andlaya\-enfalls to 0\.5297 under names that point to the wrong class, close to the chance level of0\.50\.5\.vonis flat across all three label conditions because it never places the label in the model’s input\.n=3500n=3500\{\}; error bars are bootstrap 95% intervals over items\.## 1Introduction

During 2026 several vendors released small models that do not generate text\. Figure[1](https://arxiv.org/html/2610.02586#S0.F1)previews the central result\. Given an input and a question, such a model returns a probability for each of a set of options that the caller supplies with the request\. A typical call asks which of three departments should handle a support ticket, or whether a contract clause contains a particular kind of unfair term\. The appeal is cost and speed: because nothing is generated, one decision costs a single forward pass instead of an autoregressive decode, and reported costs are one to two orders of magnitude below those of a frontier language model asked the same question\([Deußer et al\., 2026](https://arxiv.org/html/2610.02586#bib.bib3)\)\. These systems are usually called typed decision models\. The first of them, Jev, was released in September 2026 for routing, content moderation and triage, and open implementations of the same interface followed within days\.

Interfaces of this kind ask the caller to supply two things for each option: a short label, such asbilling, and a written definition, such as “invoices, payments and refunds”\. The definition is the only place in the request where a developer can state the rule that the model is supposed to apply\. If the rule is “escalate when the customer is on the Enterprise plan and asks for a refund”, that sentence goes in the definition field, and the label is a name used to read the answer back afterwards\. The interface therefore implies that the returned probability is a posterior over the definitions, with the labels serving as identifiers\.

Readers who have used a language model as a classifier will recognize the setting\. Scoring a fixed set of label strings under a prompt, and taking the argmax, is the same operation; a typed decision model is a dedicated model for it, trained to return calibrated probabilities and to do so in one forward pass\. We measure both in this paper, and §[2\.3](https://arxiv.org/html/2610.02586#S2.SS3)states the correspondence precisely\. The effect we report is present in both, so the findings apply to a prompted classification pipeline as much as to a dedicated model\.

This paper tests that implication, and finds that it holds for one of the systems we study and fails for the others\. On threelayacheckpoints and on language\-model readouts over a Qwen2\.5 backbone, the answer is determined mainly by the labels\. Deleting every definition from a classification request leaves accuracy unchanged, even though the same definitions are sufficient on their own when the labels are replaced byAandB\. On a synthetic routing suite where the rule appears only in the definitions, giving the options meaningful names is worse than giving them arbitrary ones\. A fourth system,von, returns exactly the same probabilities whatever the labels are\.

[Sun et al\. \(2026\)](https://arxiv.org/html/2610.02586#bib.bib9)reported a related observation on a hosted model, that renaming an option set changes its answers, and attributed the effect to the constrained decision head that these models use in place of a decoder\. Our measurements do not support that attribution\. An ordinary language model scoring the same option strings shows the effect at least as strongly, and the effect can be removed from a typed decision head, or added to one that does not have it, by changing how the options are written into the input and nothing else\. What separates the systems that are affected from the one that is not is visible in the released code: whether the option label is written into the model input alongside its definition\. We establish this by changing that one expression in both directions rather than by comparing the two systems, so the result does not depend on their other differences\.

An evaluation audit of the first twenty\-eight papers on these models counted how many applied two basic checks\. Of those for which each check was applicable, only 5 of 26 included a label\-probability baseline from an ordinary language model, and only 1 of 27 tested whether a decision is invariant to the names given to the options\([Tang & Zheng, 2026](https://arxiv.org/html/2610.02586#bib.bib10)\)\. The two omissions are related, because without the invariance test the effect is not visible, and without the baseline it cannot be attributed to anything in particular\.

##### Contributions\.

1. 1\.An ablation that separates the two channels a request carries, the option label and the option definition, and measures what each contributes \(§[5\.1](https://arxiv.org/html/2610.02586#S5.SS1)\)\.
2. 2\.PolicyBench, a synthetic routing suite in which the decision rule appears only in the option definitions, the correct answer is computed from the generating attributes, and the labels are varied from aligned through arbitrary to misleading \(§[4\.3](https://arxiv.org/html/2610.02586#S4.SS3), §[5\.2](https://arxiv.org/html/2610.02586#S5.SS2)\)\.
3. 3\.An intervention in both directions on two independently trained systems: removing the option label from the input of threelayacheckpoints eliminates the effect, and adding it tovoncreates it, with no change to weights, decision head or calibration \(§[5\.5](https://arxiv.org/html/2610.02586#S5.SS5)\)\.
4. 4\.Evidence against the published attribution of this failure to the constrained decision head, from a language\-model readout over the same option strings and from three readouts over one fixed backbone \(§[5\.3](https://arxiv.org/html/2610.02586#S5.SS3)\)\.
5. 5\.A two\-call test that tells a caller which regime a given model is in, requiring no labels, no held\-out data and no access to weights \(§[5\.6](https://arxiv.org/html/2610.02586#S5.SS6)\)\.
6. 6\.Measurements of a second and independent failure, in which a negated yes/no question is not read at all \(§[5\.7](https://arxiv.org/html/2610.02586#S5.SS7)\), and of the behavior of the models’ own confidence scores when a decision has been corrupted \(§[5\.8](https://arxiv.org/html/2610.02586#S5.SS8)\)\.
7. 7\.Four mitigations, each measured rather than recommended on principle \(§[6](https://arxiv.org/html/2610.02586#S6)\)\.

##### What to do about it\.

A reader who uses one of these interfaces can act on three findings without reading the rest of the paper\. First, run the two\-call test of §[5\.6](https://arxiv.org/html/2610.02586#S5.SS6): ask one question twice, changing only the option labels and leaving the definitions alone, and compare the returned distributions\. If they differ, the labels are affecting the decision\. Second, if they differ and the rule you care about lives in the definitions rather than in the class names, rename the options toA,Band so on; on our routing suite this is worth \+0\.1511 accuracy\[\+0\.1377,\+0\.1646\]\[\+0\.1377,\\,\+0\.1646\]\. Third, do not rely on a confidence threshold to catch this: when a label and its definition disagree, the confidence score of the models we tested ranks their errors in the wrong direction \(§[5\.8](https://arxiv.org/html/2610.02586#S5.SS8)\)\. Section[6](https://arxiv.org/html/2610.02586#S6)gives the measured value of four mitigations, including what to do when the labels have to keep their meaning\.

##### Scope\.

Every model we study has open weights, and every number in this paper can be reproduced without a vendor API key\. We do not measure Jev itself\. The audit cited above notes that its “internals remain closed, so claims about mechanism apply only to open systems”\([Tang & Zheng, 2026](https://arxiv.org/html/2610.02586#bib.bib10)\), and the claims we make are about mechanism\. The open implementations are also what is deployed locally by a substantial part of this ecosystem\([Ling et al\., 2026](https://arxiv.org/html/2610.02586#bib.bib7)\)\.

## 2Background

### 2\.1What a typed decision model does

A request consists of a*state*, which is the input to be judged, and one or more*questions*\. Each question has a type, an instruction, and, for most types, a set of options with definitions\. The model returns one answer per question\. No text is produced, and the set of possible answers is fixed by the request, so the output can be parsed without error handling\. Three question types are common to the interfaces we study:

choiceselects one of a set of named options, and the reply contains the selected option, a probability for each option, and a confidence score\.scoreplaces the state on an ordered scale of named levels, and the reply contains a numeric position and a distribution over levels\.booleanjudges whether a statement about the state is true, and the reply is a single probability\.

Figure[2](https://arxiv.org/html/2610.02586#S2.F2)is a completechoicecall tolaya\-typed\-decisionswith the reply it produces\. Thecriteriafield carries the definitions\.

state=\("Hi,wewerebilledtwiceforMarch\.Pleaserefundtheduplicate"

"todayorwewillcancelourplan\."\)

question=\{"type":"choice",

"instructions":"Whichdepartmentshouldhandlethisticket?",

"criteria":\{"billing":"invoices,paymentsandrefunds",

"technical":"bugs,outagesandsystemerrors",

"other":"anythingelse"\}\}

\{"choice":"billing",

"probabilities":\{"billing":0\.8436,"technical":0\.0803,"other":0\.0761\},

"confidence":0\.5067\}

Figure 2:A complete request and reply\. The caller supplies a state and one typed question; the model returns a selected option, a probability for each option, and a confidence score\. Thecriteriafield is where the meaning of each option is given\.In this example the labelbillingand the definition bound to it both point to the same answer, which is the usual situation and the reason the distinction is easy to overlook\. The experiments in this paper separate the two\.

### 2\.2The two channels of a request

Each option in achoicerequest reaches the model through two pieces of text: its*label*, such asbilling, and its*definition*, such as “invoices, payments and refunds”\. We call these the twochannelsof the request\. Either one on its own can indicate which option is correct\. The label does so when it is an ordinary name for the class, asbillingis for a billing ticket\. The definition does so when it states the rule, which is the case the interface is designed for and the only case available when the rule is something a pretrained model could not already know\. Throughout the paper we vary the two channels independently and report what each contributes\.

### 2\.3How an answer is obtained without generating text

We use*readout*for the procedure that turns one forward pass into a probability over the caller’s options\. Writessfor the state,qqfor the question, ando1,…,oko\_\{1\},\\dots,o\_\{k\}for the options\. Three readouts appear in this paper, and the difference between them is what §[5\.3](https://arxiv.org/html/2610.02586#S5.SS3)rests on\.

Atyped decision headinserts a marker token into the input for each option, reads a scalarziz\_\{i\}from the encoder at theii\-th marker position, and normalizes across markers:

p⁡\(oi∣s,q\)=exp⁡\(zi/τ\)∑j=1kexp⁡\(zj/τ\),p\(o\_\{i\}\\mid s,q\)\\;=\\;\\frac\{\\exp\(z\_\{i\}/\\tau\)\}\{\\sum\_\{j=1\}^\{k\}\\exp\(z\_\{j\}/\\tau\)\},\(1\)withτ\\taua calibration temperature\. Allkkscores come from one pass, which is what makes a decision cost a single forward pass\. This is the readout used bylayaandvon\.

Alabel log\-probability readoutuses an ordinary language model\. It rendersssandqqinto a promptπ⁡\(s,q\)\\pi\(s,q\)and scores each option string as a continuation, normalizing by the number of tokens\|oi\|\|o\_\{i\}\|so that longer option names are not penalized:

ℓi=1\|oi\|​log⁡PLM​\(oi∣π⁡\(s,q\)\),p⁡\(oi∣s,q\)=exp⁡ℓi∑j=1kexp⁡ℓj\.\\ell\_\{i\}\\;=\\;\\frac\{1\}\{\|o\_\{i\}\|\}\\log P\_\{\\mathrm\{LM\}\}\\\!\\left\(o\_\{i\}\\mid\\pi\(s,q\)\\right\),\\qquad p\(o\_\{i\}\\mid s,q\)\\;=\\;\\frac\{\\exp\\ell\_\{i\}\}\{\\sum\_\{j=1\}^\{k\}\\exp\\ell\_\{j\}\}\.\(2\)Afirst\-token readoutis the same construction restricted to the first token at which the options differ, so thatℓi=log⁡PLM​\(oi\(1\)∣π⁡\(s,q\)\)\\ell\_\{i\}=\\log P\_\{\\mathrm\{LM\}\}\(o\_\{i\}^\{\(1\)\}\\mid\\pi\(s,q\)\)\. Both are standard ways of using a language model as a classifier\.

Equations \([1](https://arxiv.org/html/2610.02586#S2.E1)\) and \([2](https://arxiv.org/html/2610.02586#S2.E2)\) share a property that matters here: the answer is obtained by comparing the option strings with one another\. We call thesescoring readouts\.

Agenerative readoutinstead samples text from the model and matches the reply back to an option:

o^=match⁡\(decode⁡\(π⁡\(s,q\)\),\{o1,…,ok\}\)\.\\hat\{o\}\\;=\\;\\operatorname\{match\}\\\!\\big\(\\operatorname\{decode\}\(\\pi\(s,q\)\),\\;\\\{o\_\{1\},\\dots,o\_\{k\}\\\}\\big\)\.\(3\)It returns a decision rather than a distribution, and it does not compare option strings against one another\. Holding the model fixed and switching between \([2](https://arxiv.org/html/2610.02586#S2.E2)\) and \([3](https://arxiv.org/html/2610.02586#S2.E3)\) is what isolates the second half of our result in §[5\.3](https://arxiv.org/html/2610.02586#S5.SS3)\.

### 2\.4Systems studied

layaandvonare open\-weight implementations of the Jev interface: they accept the same request format, expose the samecriteriafield, and return the same three answer types\. We study them rather than Jev itself because the mechanism we identify is a property of how a request is rendered into the model’s input, which can only be read and patched in a system whose code and weights are available\.layais a family built on ModernBERT\([Warner et al\., 2024](https://arxiv.org/html/2610.02586#bib.bib12)\)and released under Apache\-2\.0\. We use three checkpoints, and refer to each by the short name used in every table and figure:laya\-typed\-decisions\(421M parameters, writtenlaya\-td\),laya\(421M, writtenlaya\-en\) andlaya\-multilingual\(322M, writtenlaya\-ml\)\.vonis a separate open implementation, also built on ModernBERT, with 395M parameters\. Where a statement applies to all threelayacheckpoints we say so explicitly; an unqualified short name always means that one checkpoint\. For the language\-model readouts we use Qwen2\.5 at 0\.5B, 1\.5B and 7B parameters, and Llama\-3\.1\-8B\-Instruct and Mistral\-7B\-Instruct\-v0\.3 to check that the result is not specific to one pretraining run\.

One architectural detail is relevant later, which is thatlayaconstructs one input sequence per question, so a request carrying several questions is a batch of sequences rather than a single shared context\. Questions in one request therefore cannot influence one another, and none of the effects we report can be explained that way\.

## 3Related work

##### Evaluations of typed decision models\.

A body of evaluation work appeared within weeks of Jev’s release, most of it evaluating the hosted model\.[Deußer et al\. \(2026\)](https://arxiv.org/html/2610.02586#bib.bib3)report accuracy across a wide set of classification and reading\-comprehension tasks\.[Li et al\. \(2026a\)](https://arxiv.org/html/2610.02586#bib.bib5)show that the returned probabilities violate the probability axioms when logically linked questions are asked\.[Xu \(2026\)](https://arxiv.org/html/2610.02586#bib.bib14)change decisions by adding naturally phrased context,[Hu et al\. \(2026\)](https://arxiv.org/html/2610.02586#bib.bib4)by appending an unverified opinion, and[Wu & Lim \(2026\)](https://arxiv.org/html/2610.02586#bib.bib13)by prompt injection\.[Ling et al\. \(2026\)](https://arxiv.org/html/2610.02586#bib.bib7)survey public projects built on these models\.[Tang & Zheng \(2026\)](https://arxiv.org/html/2610.02586#bib.bib10)review this literature and report the two evaluation gaps quoted in §[1](https://arxiv.org/html/2610.02586#S1)\.

##### The effect we study\.

[Sun et al\. \(2026\)](https://arxiv.org/html/2610.02586#bib.bib9)renamed option sets and reported large changes in the answers of Jev, the hosted model, which they attributed to the constrained decision head\. We reproduce the effect on open weights, measure it against a language\-model baseline that the earlier work did not include, and locate it in the prompt rendering rather than in the decision head, by changing that rendering in both directions\. We also separate it from a second failure, insensitivity to question polarity, which is present in every typed model we test including the one that is invariant to labels\.

##### Input\-independent label priors\.

That a language model’s label probabilities carry a large input\-independent prior is established, and contextual calibration removes it by dividing out the prediction on a content\-free input\([Zhao et al\., 2021](https://arxiv.org/html/2610.02586#bib.bib15)\)\. We evaluate that correction here and report the conditions under which it helps and the conditions under which it does not \(§[6](https://arxiv.org/html/2610.02586#S6)\)\. That prior and the effect we study are distinct\. An input\-independent prior is a tendency to return one option whatever the state is, and it is present even when the options carry no definitions\. Option\-label bias is a tendency to answer from an option’s name in preference to the definition supplied for it, and it is only defined when the two channels of §[2\.2](https://arxiv.org/html/2610.02586#S2.SS2)can disagree\. Equation \([5](https://arxiv.org/html/2610.02586#S4.E5)\) measures the first; the contrast between aligned and arbitrary labels measures the second\.

## 4Method

### 4\.1Varying the two channels

A*variant*is a rewriting of the same request that changes how the options are presented while leaving what they mean unchanged\. In every variant, the definition attached to a presented label still correctly describes the class that label stands for, so a model that answers from the definitions is unaffected by any of them\. We record, for each variant, which presented label carries the definition of each true class, so a class probability is always read back by the right name\.

Table 1:The variants used throughout\. The first column gives the identifier used in the released code and in the raw result files; the plain names introduced below are used in all prose, tables and figures\.Table[1](https://arxiv.org/html/2610.02586#S4.T1)lists every variant with the identifier used in the released code\. We use one name for each throughout the paper, and the tables use the same names:

Inalignedthe options keep the task’s own class names, which is the ordinary way a request is written\. Inarbitrarythe names are replaced byA,Band so on, which carry no meaning; only the definitions are informative, so this condition also measures the definition channel on its own\.Arbitrary, exchangeduses the same arbitrary names with the definitions attached the other way round\. Inmisleadingeach class is presented under a different class’s name, while its definition still describes it correctly\. Inlabel onlythe definitions are replaced by a placeholder, leaving the labels as the sole informative channel\. Inneitherboth channels are stripped; this is the no\-information control, and a model should be at chance here\.

The comparison betweenarbitraryandarbitrary, exchangedcarries most of the argument, because neither request contains a contradiction\. Both present labels with no meaning of their own alongside correct definitions, so the correct class is unambiguous in each, and a model that answers from the definitions scores the same under both\.

We vary the state as well, so that in therealcondition the model receives the actual input; innullit receives an empty state, which also supplies the prior used in §[6](https://arxiv.org/html/2610.02586#S6); inmismatchit receives another item’s state, which checks whether the model is using*this*input rather than responding to the question alone\.

### 4\.2Classification tasks

Eleven public tasks supply the ordinary classification setting\. Three of them are*fixed\-label choice*tasks, meaning every item is achoicequestion over the same option set, so the labels and the definitions can be varied independently across the whole task: SST\-2\([Socher et al\., 2013](https://arxiv.org/html/2610.02586#bib.bib8)\), RTE and CB\([Wang et al\., 2019](https://arxiv.org/html/2610.02586#bib.bib11)\)\. Seven are yes/no tasks used for the polarity measurement of §[5\.7](https://arxiv.org/html/2610.02586#S5.SS7): BoolQ\([Clark et al\., 2019](https://arxiv.org/html/2610.02586#bib.bib2)\), WiC\([Wang et al\., 2019](https://arxiv.org/html/2610.02586#bib.bib11)\), yes/no forms of SST\-2 and RTE, and three clause types from UNFAIR\-ToS\([Chalkidis et al\., 2022](https://arxiv.org/html/2610.02586#bib.bib1)\)\. One is ordinal, SST\-5\([Socher et al\., 2013](https://arxiv.org/html/2610.02586#bib.bib8)\), and is used only where an ordered scale is relevant\. Per\-task sizes and class prevalence are given in Appendix[B](https://arxiv.org/html/2610.02586#A2)\.

### 4\.3PolicyBench

On an ordinary classification task the label already names the answer, so a model can score well without reading the definition at all\. PolicyBench is constructed so that this is not possible\. Support tickets are generated from templates over six controlled attributes\. Each policy is a boolean function of those attributes, stated in natural language*only*in the definition bound to the true branch\. Ground truth is computed from the generating attributes, so there is no label noise, no annotator disagreement and no possibility of contamination, and classes are exactly balanced by rejection sampling\. Seven policies span single\-term, conjunctive, disjunctive and negated rules\.

The label conditions of §[4\.1](https://arxiv.org/html/2610.02586#S4.SS1)apply directly\. In the aligned condition the options are namedescalateandhold; in the arbitrary condition they are namedAandB; in the misleading condition the option whose definition is the policy is namedholdand the other is namedescalate\.

A control suite,PolicyBench\-XF\(for*explicit false branch*\), replaces the false branch’s catch\-all definition \(“none of the above applies”\) with an explicit negation of the policy, so that the two options carry definitions of comparable substance\. §[5\.2\.1](https://arxiv.org/html/2610.02586#S5.SS2.SSS1)shows why this control is necessary\. We report XF as the primary suite throughout\.

### 4\.4Models

Typed decision heads:laya\-typed\-decisions\(421M\),laya\(421M\),laya\-multilingual\(322M\), andvon\(395M\)\. Readouts over one Qwen2\.5 backbone, which separates the typed head from the task format: length\-normalized label log\-probability, constrained first\-token, and generate\-then\-parse, at 0\.5B, 1\.5B and 7B\.

### 4\.5Metrics

We report accuracy throughout, and AUROC where a confidence score is being evaluated as a ranking of the model’s own correctness \(§[5\.8](https://arxiv.org/html/2610.02586#S5.SS8)\)\. Two further statistics are specific to this paper\.

Theflip ratebetween two variantsuuandvvis the fraction of items on which the decision changes, where the decision is the highest\-probability class:

flip\(u,v\)=1n∑m=1n\[argmaxipu\(oi∣sm\)≠argmaxipv\(oi∣sm\)\]\.\\mathrm\{flip\}\(u,v\)\\;=\\;\\frac\{1\}\{n\}\\sum\_\{m=1\}^\{n\}\\mathbf\{1\}\\\!\\left\[\\arg\\max\_\{i\}p\_\{u\}\(o\_\{i\}\\mid s\_\{m\}\)\\neq\\arg\\max\_\{i\}p\_\{v\}\(o\_\{i\}\\mid s\_\{m\}\)\\right\]\.\(4\)It measures whether a rewriting changes what the model decides, independently of whether the decision was correct\.

Thestate\-blindnessof a variant is the fraction of items on which the model’s decision equals the decision it makes for the same request with an empty state:

blind\(u\)=1n∑m=1n\[argmaxipu\(oi∣sm\)=argmaxipu\(oi∣∅\)\]\.\\mathrm\{blind\}\(u\)\\;=\\;\\frac\{1\}\{n\}\\sum\_\{m=1\}^\{n\}\\mathbf\{1\}\\\!\\left\[\\arg\\max\_\{i\}p\_\{u\}\(o\_\{i\}\\mid s\_\{m\}\)=\\arg\\max\_\{i\}p\_\{u\}\(o\_\{i\}\\mid\\varnothing\)\\right\]\.\(5\)A value near11means the model is returning nearly the same answer whatever the input is, so a high accuracy in that condition would have to come from the class balance rather than from reading the state\.

All intervals are percentile bootstrap over items with 4000 resamples, paired when two conditions are compared on the same items\. Per\-task sizes and class prevalence are in Appendix[B](https://arxiv.org/html/2610.02586#A2)\.

## 5Results

### 5\.1What each channel contributes

Figure 3:What each channel contributes, on the fixed\-label choice tasks \(SST\-2, RTE, CB;n=833n=833\{\}\)\. For every model exceptvon, supplying only the labels \(second bar\) is as accurate as supplying both channels \(first bar\), so the definitions add nothing when a meaningful label is present\.vonshows the opposite pattern: it matches its full accuracy from the definitions alone and falls to chance when they are removed\. Exact values with intervals are in Table[9](https://arxiv.org/html/2610.02586#A1.T9)of Appendix[A](https://arxiv.org/html/2610.02586#A1)\. Error bars are bootstrap 95% intervals\.Figure[3](https://arxiv.org/html/2610.02586#S5.F3)reports the ablation\. Forlaya\-typed\-decisions, deleting every definition does not reduce accuracy \(0\.8559 against 0\.8487, intervals overlapping\), and the same holds forlaya\-en\. Forlaya\-mlthe two conditions differ by 0\.034 with intervals that overlap\. The definitions are not unreadable to the model, because when they are supplied as the only informative channel, with the labels replaced by letters, they support 0\.7971, well above the 0\.5690 control in which neither channel carries information\. The definitions are therefore legible to the model and are not being used when a label is available\. When a label and its definition point to different classes, accuracy falls to 0\.1357, below the no\-information control, which means the decision is following the label\.

Thevonrow behaves differently, in that its accuracy is 0\.8511 whether the labels are the class names, arbitrary letters, or the names of the wrong classes, and 0\.5102 in both conditions where the definitions have been removed\. Becausevondoes not use the labels, these three requests are the same input to it\. When the definitions are removed it has no information left and performs at chance, whereaslayastill reaches 0\.8559 from the labels alone\. Together these two observations describe a trade: reading only the definitions costs nothing in accuracy on this suite \(0\.8511 against 0\.8487\), but it requires the caller to have written definitions that are sufficient on their own\.

##### Thescoreprimitive admits no such ablation\.

Achoicequestion carries labels and definitions in separate fields, which is what makes this ablation possible\. Ascorequestion does not: itscriteriais the ordered list of level labels, so the label and the definition are the same string and there is no second channel to remove\. We therefore exclude the ordinal task from this ablation\. This is a property of the interface rather than of any model\. It has a consequence for callers: a rubric level cannot be given a definition that its label does not already carry, so the guidance of §[6](https://arxiv.org/html/2610.02586#S6)for choosing neutral labels cannot be applied to score questions at all\.

### 5\.2Results on PolicyBench

Table 2:PolicyBench\-XF accuracy by label condition\. The decision rule appears only in the option definition\. Chance is 0\.500\.label onlyandneitherare controls establishing that the definition is the sole information source\.Table 3:Paired contrasts on PolicyBench\-XF, bootstrap over items\.Forlaya\-td, accuracy is \+0\.1511\[\+0\.1377,\+0\.1646\]\[\+0\.1377,\\,\+0\.1646\]higher when the options are calledAandBthan when they are named after the decision \(n=3500n=3500\{\}\), and labels that point to the wrong class still give higher accuracy than correct ones, by \+0\.0942\[\+0\.0754,\+0\.1129\]\[\+0\.0754,\\,\+0\.1129\]\. The control cells are within seven points of chance, so the suite measures definition following rather than any property of the labels\. The gain does not depend on which uninformative label is used: unrelated words, digits and letters all recover most of the gap, at 0\.8537, 0\.8740 and 0\.8931 against 0\.7420 for the meaningful names, andvonreturns the same accuracy under all of them\. The state\-blindness column of Table[7](https://arxiv.org/html/2610.02586#S6.T7)shows thatlaya\-tdreturns its no\-state decision on 0\.634 of items under aligned labels against 0\.524 under arbitrary ones, so a meaningful label displaces the state rather than supplementing it\. This accounts for aligned labels being worse than arbitrary ones\. It does not by itself predict the ordering of the misleading condition, which for this checkpoint is higher than aligned; the stringsescalateandholdare not neutral with respect to the policies, and which way they push depends on the policy\.

##### The direction of the effect differs between checkpoints\.

All threelayacheckpoints depend on the labels, but they do not respond to misleading labels in the same way\. Forlaya\-typed\-decisions, misleading labels give higher accuracy than aligned ones \(\+0\.0942\[\+0\.0754,\+0\.1129\]\[\+0\.0754,\\,\+0\.1129\]\)\. Forlaya, they reduce accuracy from 0\.7911 to 0\.5297 \(\-0\.2618\[−0\.2851,−0\.2386\]\[\-0\.2851,\\,\-0\.2386\]\)\. We therefore report the contrast between arbitrary and aligned labels as the primary quantity, because it measures whether the labels affect the decision at all without depending on the direction in which a particular checkpoint responds\.

#### 5\.2\.1An artifact of the uncontrolled design

Our first version of PolicyBench gave the false branch the catch\-all definition “none of the above applies”\. In that version, exchanging which letter carries the policy changes accuracy by a clear margin \(Table[12](https://arxiv.org/html/2610.02586#A1.T12)\), which would ordinarily be read as a preference for one option position over another, but that reading does not survive the control\. When both options carry definitions of comparable length and specificity, the same contrast is indistinguishable from zero forlaya\-tdandlaya\-ml\(Table[3](https://arxiv.org/html/2610.02586#S5.T3)\)\. It remains small but non\-zero for the other systems, which we return to in §[5\.3](https://arxiv.org/html/2610.02586#S5.SS3)\. What the first version measured was a preference for whichever option carried the more specific definition\. The contrast between arbitrary and aligned labels survives the control for the two systems that are well above chance on this suite, and is larger under it: \+0\.1511 forlaya\-tdand \+0\.1203 for the 7B readout, against \+0\.1201 and \+0\.0080 on the uncontrolled suite\. We therefore report the controlled suite throughout\. We describe the artifact because the uncontrolled design is the one a reader is likely to write first\.

### 5\.3Language\-model readouts show the same effect

If the constrained decision head were responsible for this effect, an ordinary language model scoring the same option strings would not be expected to show it\. It shows it\. On PolicyBench\-XF \(Table[2](https://arxiv.org/html/2610.02586#S5.T2)\) the Qwen2\.5\-7B label\-log\-probability readout gains \+0\.1203\[\+0\.1019,\+0\.1395\]\[\+0\.1019,\\,\+0\.1395\]from replacing meaningful labels withAandB\(n=2100n=2100; Table[3](https://arxiv.org/html/2610.02586#S5.T3)\), an effect of the same kind and comparable size to the \+0\.1511\[\+0\.1377,\+0\.1646\]\[\+0\.1377,\\,\+0\.1646\]we measure forlaya\-td\(n=3500n=3500\{\}\)\. Its label\-only and neither cells sit at chance, so on this suite it too is reading the definitions and being displaced by the labels\.

The 1\.5B row should not be read as evidence that scale removes the effect\. Qwen2\.5\-1\.5B is at chance in*every*condition of PolicyBench\-XF, including the ones where the task is solvable, so its near\-null contrast is a capacity floor rather than robustness: a model that cannot do the task cannot display a label effect\. The 7B readout is the first language model in our set competent enough for the effect to be visible, and it is also the one that shows it most strongly\.

The result is not specific to one pretraining run\. We repeat the measurement with two further language\-model families, Llama\-3\.1\-8B and Mistral\-7B, using the same readout and the same items\. Mistral\-7B shows the effect in the same direction and at a comparable size, \+0\.1531\[\+0\.1343,\+0\.1724\]\[\+0\.1343,\\,\+0\.1724\]from replacing meaningful labels withAandB\. On the classification tasks of §[5\.1](https://arxiv.org/html/2610.02586#S5.SS1), where all three families are accurate under ordinary labels, all three lose a large amount of accuracy when a label contradicts its definition: \-0\.5482, \-0\.3235 and \-0\.3027 for Qwen2\.5\-1\.5B, Llama\-3\.1\-8B and Mistral\-7B respectively, every interval excluding zero\.

Llama\-3\.1\-8B is the exception on PolicyBench, and the state\-blindness statistic of Equation \([5](https://arxiv.org/html/2610.02586#S4.E5)\) identifies why\. Under arbitrary labels it answers one class on 98\.6% of items, and under contradicting labels its state\-blindness is 1\.0000: it returns the same decision for every input, so there is no label effect to measure\. This is the capacity floor described above for Qwen2\.5\-1\.5B rather than a model that resists the effect, and we read nothing from its contrast on that suite\. Its classification rows, where it is accurate, behave like the others\.

The readout is also sensitive to which letter carries which definition even when nothing in the request is contradictory: exchanging the two letters changes Qwen2\.5\-7B accuracy by \+0\.0305\[\+0\.0210,\+0\.0405\]\[\+0\.0210,\\,\+0\.0405\]\(Table[3](https://arxiv.org/html/2610.02586#S5.T3)\), where the same contrast is exactly zero forvon\. On the classification tasks of §[5\.1](https://arxiv.org/html/2610.02586#S5.SS1)both families lose accuracy when a label and its definition disagree, though not by the same amount:laya\-tdfalls by 0\.713 and the 1\.5B readout by 0\.548\. The 7B readout was not run on those tasks\.

##### The two scoring readouts agree; the generative readout depends on the suite\.

Table[11](https://arxiv.org/html/2610.02586#A1.T11)and Figure[6](https://arxiv.org/html/2610.02586#A1.F6)in Appendix[A](https://arxiv.org/html/2610.02586#A1)compare the three readouts of §[2\.3](https://arxiv.org/html/2610.02586#S2.SS3)on one fixed backbone, on both PolicyBench suites, and two of its results hold on both\. Scoring the option strings by length\-normalized log probability and scoring them by a single constrained token give results that agree within their intervals at every cell, so the particular scoring rule makes no difference\. This also rules out the length normalization of Equation \([2](https://arxiv.org/html/2610.02586#S2.E2)\) as a cause, because the first\-token readout applies no normalization and behaves the same way\.

The comparison between scoring and generation does*not*hold across suites, and we report this because the uncontrolled result is the more striking one\. On the uncontrolled suite the two scoring readouts fall to chance under misleading labels \(0\.5010 at 7B\) while the generative readout reaches 0\.8490\. On the controlled suite the same comparison reverses: the scoring readout reaches 0\.8352 and the generative readout 0\.7933\. The collapse to chance is therefore a property of the uncontrolled design, in which the false branch carries the catch\-all definition, and not a general property of scoring readouts\. It is the same artifact described in §[5\.2\.1](https://arxiv.org/html/2610.02586#S5.SS2.SSS1)\.

We therefore do not claim that the readout is a second condition for the effect\. What the controlled suite supports is narrower: the choice of scoring rule does not matter, and the label effect itself is present in a language\-model readout that has no typed decision head at all \(0\.7886 under aligned labels against 0\.8352 under misleading ones, and \+0\.1203 from arbitrary labels\)\. That is what bears on the attribution in §[5\.4](https://arxiv.org/html/2610.02586#S5.SS4)and §[5\.5](https://arxiv.org/html/2610.02586#S5.SS5)\.

### 5\.4A model that is unaffected, and why

Table[4](https://arxiv.org/html/2610.02586#S5.T4)showsvonanswering one routing request under five labelings, with probabilities read back by which definition each label carries\. The request is written in the PolicyBench format rather than drawn from the generated set, so that all five labelings can be displayed together\.

Table 4:vonanswering one routing request under five labelings of the same question\. Probabilities are read back by which definition each label carries\. The returned values are identical to six decimal places in every row\.The spread across renderings is00to machine precision\. With both definitions replaced by the same placeholder,vonreturns exactly\{0\.5,0\.5\}\\\{0\.5,0\.5\\\}\. Across the seven policies its accuracy under aligned, arbitrary and misleading labels is identical to four decimal places, and its label\-only and neither cells sit at exactly0\.50000\.5000, so it takes nothing at all from the labels\.

Figure 4:What reaches the encoder for the same request\.layaprefixes each definition with its option label \(shaded\), so the label competes with the definition the caller wrote\.vonpasses only the definitions and uses the label solely as a key for returning the answer\. §[5\.5](https://arxiv.org/html/2610.02586#S5.SS5)removes this prefix fromlayaand shows that the behavior reported in §[5\.1](https://arxiv.org/html/2610.02586#S5.SS1)and §[5\.2](https://arxiv.org/html/2610.02586#S5.SS2)goes with it\.Figure[4](https://arxiv.org/html/2610.02586#S5.F4)shows what reaches the encoder in each case, and the cause is one line in each code base, shown together below\.layaconcatenates the label in front of the definition, so both enter the encoder\.vonpacks only the definition; there the label is a dictionary key used to retrieve the definition and to name the answer, and it reaches the model only as a fallback when no definition was supplied\.

return\[str\(k\)ifvisNoneorv==""else"%s:%s"%\(k,render\_criterion\(v\)\)

fork,vincrit\.items\(\)\]

desc=q\.criteria\.get\(opt\)

descriptions\.append\(desc\.strip\(\)ifdescelseopt\.strip\(\)\)

packed\_text=model\.pack\_sequence\(state\_text,q\.instructions,descriptions\)

vonon its own would be weak evidence about the decision head, because a model that never receives the label cannot display a label effect; it shows that such a head*can*be label\-invariant, not that the head is irrelevant\. The intervention in §[5\.5](https://arxiv.org/html/2610.02586#S5.SS5)is what settles this: it removes the label fromlaya’s input while leaving its decision head, weights and calibration untouched, and the effect disappears\. The design also carries a cost, becausevonnever receives the label, it cannot use a label that is informative, and a caller who supplies short or missing definitions receives a uniform distribution\.

### 5\.5Adding and removing the label

Table 5:The rendering changed in both directions, on both suites\. For eachlayacheckpoint the patched row renders an option as its definition alone instead of"\{label\}: \{definition\}"\. Forvonthe patched row does the reverse and prepends the label\. Weights, decision head and calibration are unchanged within each pair, and the items are the same\.Comparinglayawithvoncompares two systems that differ in their weights, their training data and their rendering at once\. To isolate the rendering, we patched it in both directions\. In the threelayacheckpoints we changed the expression quoted above so that an option is rendered as its definition alone\. Invonwe did the reverse, prepending each option’s label to its definition, which is whatlayadoes\. Each patch is a change to one expression; nothing else about either system was touched\.

Table[5](https://arxiv.org/html/2610.02586#S5.T5)gives the result, and removing the label makes all threelayacheckpoints exactly invariant: aligned, arbitrary and misleading labels return the same accuracy on both suites, and the contrast between arbitrary and aligned labels is\+0\.0000\+0\.0000with a bootstrap interval of\[\+0\.0000,\+0\.0000\]\[\+0\.0000,\+0\.0000\], because the three requests have become the same input\. The collapse under conflicting labels goes with it\. On the classification taskslaya\-tdfalls to 0\.1357 when a label contradicts its definition, and the patched model stays at 0\.8067\.

Adding the label tovonproduces the effect\. Its contrast on PolicyBench moves from exactly zero to \+0\.0333\[\+0\.0246,\+0\.0423\]\[\+0\.0246,\\,\+0\.0423\], and on the classification tasks it acquires the same collapse as thelayacheckpoints, falling from 0\.8511 under aligned labels to 0\.2281 when a label contradicts its definition\. A model that is invariant to option labels by construction becomes label\-sensitive when the label is written into its input, with no change to its weights\.

The two directions together identify the rendering as the cause\. The effect is removed from three checkpoints of one system by deleting the label, and created in a second, independently trained system by inserting it\.

Removing the label is not free, and it moves accuracy in opposite directions on the two suites\. On PolicyBench, where the labels carry no information about the policy, removing them raiseslaya\-tdfrom 0\.7420 to 0\.8914\. On the classification tasks, where the class names are informative, it lowers accuracy from 0\.8487 to 0\.8067\. This is the trade that separatesvonfromlayain §[5\.1](https://arxiv.org/html/2610.02586#S5.SS1), reproduced inside a single system\.

### 5\.6A two\-call test

The measurements above reduce to a test that a caller can run in two calls\. Ask the same question twice, changing only the option labels and leaving the definitions unchanged, then compare the two returned distributions\. If they are identical, the model is in the position ofvon: the definitions are its entire input and need to be written so that they are sufficient on their own\. If they differ, the model is in the position oflaya, and the mitigations of §[6](https://arxiv.org/html/2610.02586#S6)apply\. The test requires no gold labels, no held\-out data and no access to the weights, so it can be run against a hosted model\.

### 5\.7Negated questions

Figure 5:Accuracy on yes/no questions before and after negating the question, pooled over seven tasks\. The returned probability is re\-oriented for the negated form, so a model that reads the negation would score the same in both conditions\. Every model instead falls from above chance to below it by about the same margin, which is what happens when the returned probability does not change and only the correct answer has been inverted\. Exact values and flip rates are in Table[10](https://arxiv.org/html/2610.02586#A1.T10)of Appendix[A](https://arxiv.org/html/2610.02586#A1)\.Prefixing a yes/no question with “Is it false that” changes the decision on 0\.9825\[0\.977,0\.988\]\[0\.977,\\,0\.988\]of items forlaya\-td\. The flip rate lies between 0\.9656 and 0\.9825 across the threelayacheckpoints\. The returned probability is essentially unchanged by the negation\. Accuracy moves from 0\.6419 to 0\.3576, and0\.6419\+0\.3576≈10\.6419\{\}\+0\.3576\{\}\\approx 1, which is what results when the returned probability is unchanged and only the gold label has been inverted\. An explicit instruction to attend to the polarity of the question leaves this untouched \(0\.3546\)\.

Negated phrasing is common in ordinary requests\. A developer who writes a check as “is it false that the request is authorized” rather than “is the request unauthorized” receives an inverted answer, reported with normal confidence\.

##### Invariance to labels does not imply that the question is read\.

vonis the most accurate of the four typed models on these tasks \(0\.7382\) and is invariant to the option labels, but it changes its decision on 0\.9711\[0\.964,0\.979\]\[0\.964,\\,0\.979\]of negated items, which is within the range we measure for the checkpoints that do use labels\. The two failures are independent of each other, because the first depends on whether the label is written into the model input, which the implementer controls; the second is unaffected by that choice and is present in every typed model we tested\. Adopting a label\-invariant model therefore removes one of these failures and leaves the other unchanged\.

##### Comparison with the language\-model baseline\.

The Qwen2\.5\-1\.5B label\-log\-probability readout changes its decision on 0\.7972 of negated items, against 0\.9656 to 0\.9825 for the typed heads\. It is also far from reading polarity correctly, but it is affected less, and it is the kind of system that a typed decision model is usually offered as a cheaper replacement for\. Combined with §[5\.1](https://arxiv.org/html/2610.02586#S5.SS1), we do not find the typed interface to be better than this baseline on either property we measured: the two are comparable on label dependence and the typed models are worse on polarity\. The reported case for typed decision models rests on latency and cost\([Deußer et al\., 2026](https://arxiv.org/html/2610.02586#bib.bib3)\)\. We did not measure latency or cost, and nothing here bears on that case\.

### 5\.8Behavior of the confidence scores

Table 6:Whether a model’s own confidence separates its correct from its incorrect answers, by condition, on fixed\-label choice tasks\. An AUROC below0\.50\.5means the signal is actively misleading about its own errors\.Under ordinary labels, the confidence score returned bylaya\-typed\-decisionsseparates its correct from its incorrect answers at AUROC 0\.8123, so it carries usable information about which decisions to trust\. When a label and its definition disagree, and accuracy falls to 0\.1357, the same score reaches AUROC 0\.2433\. A value below0\.50\.5means the ranking is reversed, so the decisions the model reports as most confident are the ones more likely to be wrong\.laya\-enbehaves the same way \(Table[6](https://arxiv.org/html/2610.02586#S5.T6)\)\.

vonprovides the control, with an AUROC of 0\.8055 under ordinary labels and 0\.8055 under misleading ones, which are equal because its decisions in the two conditions are identical\. The confidence score degrades only in the conditions where the labels change the decision, which indicates that the degradation follows the corrupted decision rather than the manipulation itself\.

This bears directly on the deployment pattern recommended throughout this literature, in which a typed decision is accepted when confidence is high and escalated to a stronger model when it is low\([Li et al\., 2026b](https://arxiv.org/html/2610.02586#bib.bib6)\)\. Under a corrupted rendering that gate routes the wrong items: it forwards the cases the model got right and accepts the ones it got wrong\. A confidence threshold cannot detect this failure, which is why we propose the rendering\-disagreement check of §[6](https://arxiv.org/html/2610.02586#S6)instead\.

## 6Mitigations

This section is written for a reader deciding what to do about the effect in their own system\. The order to work through is: run the test in §[5\.6](https://arxiv.org/html/2610.02586#S5.SS6)to find out whether the labels are affecting your decisions at all; if they are, apply M1 where the task allows it; if the labels have to keep their meaning, apply M2 and gate the remaining decisions with M4\. Each mitigation below is reported with what it was worth in our measurements rather than recommended on principle\.

Table 7:Null\-state prior correction on PolicyBench\-XF\.*state\-blind*is the fraction of decisions equal to the decision the same rendering makes with no state at all\.Table 8:Running the decision under a second rendering and flagging disagreement\. The detector needs two renderings that behave differently, a precondition the state\-blindness statistic checks\.##### M1: arbitrary labels\.

Name the optionsA,Band so on, and put the whole of the rule in the definitions\. This is worth \+0\.1511 on PolicyBench\-XF\. On the classification tasks it costs accuracy instead, because there the class name is itself a good predictor of the class\. The choice therefore depends on the task, and the test in §[5\.6](https://arxiv.org/html/2610.02586#S5.SS6)is how to decide which case applies\.

##### M2: null\-state prior correction\.

Send the same question once with an empty state, which gives the distribution the model returns when it has no evidence, then divide it out of every subsequent answer and renormalize:

p~​\(oi∣s\)∝p⁡\(oi∣s\)p​\(oi∣∅\)α,α=1\.\\tilde\{p\}\(o\_\{i\}\\mid s\)\\;\\propto\\;\\frac\{p\(o\_\{i\}\\mid s\)\}\{\\,p\(o\_\{i\}\\mid\\varnothing\)^\{\\alpha\}\},\\qquad\\alpha=1\.\(6\)Equation \([6](https://arxiv.org/html/2610.02586#S6.E6)\) is applied once per question and the empty\-state call is cached, so the cost is negligible\. This is contextual calibration\([Zhao et al\., 2021](https://arxiv.org/html/2610.02586#bib.bib15)\)applied to a typed request\.

Table[7](https://arxiv.org/html/2610.02586#S6.T7)shows that the correction helps the models whose errors come from a label\-induced preference, and harms the model that does not have one\. It is worth \+0\.0391 forlaya\-tdunder aligned labels and \+0\.0748 for the Qwen2\.5\-7B readout, and it costs accuracy once the labels are already arbitrary\. Forvonit is actively damaging, reducing accuracy to exactly0\.50000\.5000in all three conditions\. The reason is specific to how the correction is defined\. The empty\-state call removes the state but keeps the option definitions, and the definitions are the whole ofvon’s input, sop⁡\(oi∣∅\)p\(o\_\{i\}\\mid\\varnothing\)already contains the asymmetry thatvonuses to answer\. Dividing it out removes the signal rather than a bias\. The correction should therefore be applied only to models that the test in §[5\.6](https://arxiv.org/html/2610.02586#S5.SS6)places in the label\-using regime\.

##### M3: an instruction to ignore the labels\.

Adding a sentence to the instructions stating that the labels are arbitrary identifiers and that the definitions determine the answer is worth \+0\.0138\[\+0\.0077,\+0\.0200\]\[\+0\.0077,\\,\+0\.0200\]forlaya\-tdon PolicyBench\-XF, against \+0\.1511 for relabeling the options, so it is not a substitute for M1\.

##### M4: two\-rendering disagreement\.

Forlaya\-td, flagging the items on which the aligned and arbitrary renderings disagree raises accuracy on the items that remain from 0\.7420 to 0\.8921, at 0\.8100 coverage and 0\.8977 precision, for one extra call and no training\. Where the task permits arbitrary labels, M1 is the better choice: it reaches 0\.8931 on the same model at full coverage\. M4 is for the case where the labels must keep their meaning\.

The flag rate of this procedure is the same quantity as the test in §[5\.6](https://arxiv.org/html/2610.02586#S5.SS6), and the two are reported together in Table[8](https://arxiv.org/html/2610.02586#S6.T8)\. Forvonthe flag rate is0\.00000\.0000, because its two renderings produce identical decisions\. A flag rate of zero indicates that the procedure is unnecessary for that model, not that the procedure has failed, so a caller can decide what to do from this one number\. The procedure is uninformative in one other case: if a model cannot perform the task at all, both renderings produce near\-constant output, they agree with each other, and both are wrong\. The neither control separates that case from the case of a model that is reading the definitions\.

##### Summary of the mitigations\.

Where the task allows it, use arbitrary labels and state the rule in the definitions; this is the largest effect we measured and it costs one edit to the request\. Where the labels have to carry meaning, because they are themselves good predictors of the class, apply the cached null\-state correction instead\. Where individual decisions are worth checking, run a second rendering and review the items on which the two disagree\. Do not use a confidence threshold on its own for this purpose: under a label that contradicts its definition the confidence score ranks errors in the wrong direction \(§[5\.8](https://arxiv.org/html/2610.02586#S5.SS8)\), so the threshold forwards the decisions the model got right and accepts the ones it got wrong\.

## 7Limitations

We did not measure Jev, the hosted commercial model, so our claims cover open\-weight implementations of its interface and language\-model readouts; for Jev itself we rely on[Sun et al\. \(2026\)](https://arxiv.org/html/2610.02586#bib.bib9)\. The mechanism claim is scoped accordingly: we exhibit the responsible line in two open code bases and cannot inspect a closed one\.

PolicyBench is synthetic and templated, which buys exact ground truth, exact class balance and immunity to contamination at the cost of naturalness; the natural\-task suite covers the other side of that trade\.

The misleading condition presents a request whose label and definition disagree, which a reader could call under\-specified\. The argument therefore rests on the arbitrary condition against the arbitrary\-exchanged condition, where nothing is contradictory, and on the contrast between arbitrary and aligned labels, where no request contains a conflict at all\.

The definitions are our own wording, although the control cells establish that they are legible rather than decorative, but different phrasing would move the absolute numbers\.

laya\-multilingualis near chance throughout PolicyBench and we draw no conclusion from its contrast\.

## 8Conclusion

A typed decision model’scriteriafield is a contract: the caller writes what each option means, and expects the returned probability to be a posterior over those meanings\. Across open\-weight models we find that one of the four implementations satisfies this and three do not\. Architecture, training objective and parameter count do not separate the two groups\. What separates them is whether an option’s label is written into the model’s input alongside its definition\. We show this directly rather than by comparison: removing the label from the three systems that have the effect eliminates it, and adding it to the system that does not have it creates it, with no change to any weight\. The change costs nothing in latency, and practitioners can determine which regime their own model is in with two calls\.

The two evaluation gaps identified by[Tang & Zheng \(2026\)](https://arxiv.org/html/2610.02586#bib.bib10)are related to each other\. Without an invariance test over option names, the effect reported here is not visible at all, and without a language\-model baseline it cannot be separated from the architecture that the models introduce\. Reporting both is what allowed us to locate the behavior in the prompt rendering rather than in the decision head\.

## Ethics statement

This work evaluates publicly released model weights on public datasets and on synthetic data we generate\. It involves no human subjects, no personal data and no new data collection\. PolicyBench tickets are generated from templates and describe fictional customers\. The failure modes we report are already present in deployed systems and the paper gives detection and mitigation procedures for them, so we judge the risk of publication to be low relative to the benefit of disclosure\. We evaluate third\-party models under their licenses \(Apache\-2\.0 for the typed decision models\) and name specific checkpoints only to make the results reproducible, not to rank vendors\.

## Reproducibility statement

All models are open\-weight and all datasets are public, so no vendor API key is needed to reproduce any number in this paper\. Every reported measurement is generated from cached per\-item predictions bysrc/make\_tables\.pyand inserted through a macro, so a rerun of the grid updates the manuscript\. The only literals in the prose are values that are exactly zero or exactly one half, quoted as such\. Section[4](https://arxiv.org/html/2610.02586#S4)specifies the surface variants, the state ablations and the bootstrap procedure; Section[4\.3](https://arxiv.org/html/2610.02586#S4.SS3)specifies the PolicyBench generator, whose ground truth is computed from the generating attributes\. Code, the PolicyBench generator, and the raw per\-item model outputs for every grid cell are released\.

## References

- Chalkidis et al\. \(2022\)Ilias Chalkidis, Abhik Jana, Dirk Hartung, et al\.LexGLUE: A benchmark dataset for legal language understanding in English\.In*ACL*, 2022\.
- Clark et al\. \(2019\)Christopher Clark, Kenton Lee, Ming\-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova\.BoolQ: Exploring the surprising difficulty of natural yes/no questions\.In*NAACL*, 2019\.
- Deußer et al\. \(2026\)Tobias Deußer, Lorenz Sparrenberg, and Rafet Sifa\.Evaluating and benchmarking the system one model Jev, 2026\.arXiv:2609\.37647\.
- Hu et al\. \(2026\)Jianyi Hu, Hangtao Zhang, Yi Liu, Yeqi Zeng, Li Zeng, Xianlong Wang, Rui Wang, and Leo Yu Zhang\.JevAdvBench: A benchmark and black\-box attacks for reinforcement learning for calibrated decisions models, 2026\.arXiv:2609\.31142\.
- Li et al\. \(2026a\)Keyi Li, Yihao He, and Quanyi Li\.Beyond calibration: Do a typed\-decision model’s probabilities obey the probability axioms?, 2026a\.arXiv:2609\.33209\.
- Li et al\. \(2026b\)Yubo Li, Yidi Miao, Ramayya Krishnan, and Rema Padman\.JEV\-as\-a\-judge: Accept when confident, escalate when unsure, 2026b\.arXiv:2609\.26550\.
- Ling et al\. \(2026\)Guoming Ling, Muen Xue, and Zijian Ye\.Jev in the wild: A data\-driven analysis of the Jev model’s functionality, applications and ecosystem, 2026\.arXiv:2609\.30216\.
- Socher et al\. \(2013\)Richard Socher, Alex Perelygin, Jean Wu, et al\.Recursive deep models for semantic compositionality over a sentiment treebank\.In*EMNLP*, 2013\.
- Sun et al\. \(2026\)Yu Sun, Junhao Xu, Jiajia Shi, and Zijin Yang\.Type\-safe is not error\-free: A constrained decision head follows the option name, not the rubric bound to it, 2026\.arXiv:2609\.26758\.
- Tang & Zheng \(2026\)Lijuan Tang and Yuemeng Zheng\.Typed decision models: An early evidence audit and evaluation checklist, 2026\.arXiv:2609\.32160\.
- Wang et al\. \(2019\)Alex Wang, Yada Pruksachatkun, Nikita Nangia, et al\.SuperGLUE: A stickier benchmark for general\-purpose language understanding systems\.In*NeurIPS*, 2019\.
- Warner et al\. \(2024\)Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, et al\.Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference, 2024\.arXiv:2412\.13663\.
- Wu & Lim \(2026\)Tiantong Wu and Wei Yang Bryan Lim\.Decision hijacking: Prompt injection attacks on Jev’s typed probabilistic decisions, 2026\.arXiv:2609\.28613\.
- Xu \(2026\)Zixiang Xu\.JevOut: Natural context can flip decision models, 2026\.arXiv:2609\.30243\.
- Zhao et al\. \(2021\)Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh\.Calibrate before use: Improving few\-shot performance of language models\.In*ICML*, 2021\.

## Appendix ASupplementary tables

Figure 6:Accuracy on PolicyBench for three readouts over one fixed backbone, by label condition\. Shown on the uncontrolled suite, where the gap between the scoring and generative readouts is largest; §[5\.3](https://arxiv.org/html/2610.02586#S5.SS3)reports what happens under the control\. Error bars are bootstrap 95% intervals over items\.Table 9:Exact values for Figure[3](https://arxiv.org/html/2610.02586#S5.F3): what each channel contributes, on the fixed\-label choice tasks \(SST\-2, RTE, CB;n=833n=833\{\}\)\. The ordinal task is excluded because ascorequestion has no label channel separate from its definition\.Table 10:Exact values for Figure[5](https://arxiv.org/html/2610.02586#S5.F5), with the decision flip rate of Equation \([4](https://arxiv.org/html/2610.02586#S4.E4)\) in the final column\.Table 11:One backbone, three readouts, on both PolicyBench suites\. The two scoring readouts agree throughout; the comparison with the generative readout does not hold across suites \(§[5\.3](https://arxiv.org/html/2610.02586#S5.SS3)\)\.Table 12:Contrasts on the uncontrolled PolicyBench suite, in which the false branch carries the catch\-all definition “none of the above applies”\. The third column is the artifact discussed in §[5\.2\.1](https://arxiv.org/html/2610.02586#S5.SS2.SSS1); compare it with the same column of Table[3](https://arxiv.org/html/2610.02586#S5.T3)\.
## Appendix BTasks

Table[13](https://arxiv.org/html/2610.02586#A2.T13)lists every task, its question type, the number of items used and the class prevalence after any rebalancing\. UNFAIR\-ToS clause types are class\-balanced before use because their raw splits are about 97% negative, where a model that always answers “no” reaches 0\.97 accuracy while carrying no information\.

Table 13:Tasks used in the paper, with the number of items evaluated and the class prevalence after any rebalancing\.