ThinkJev：一次思考，适配各种预算的校准类型化决策

# One Thought, Every Budgetfor Calibrated Typed Decisions
Source: [https://arxiv.org/html/2610.37732](https://arxiv.org/html/2610.37732)
## ThinkJev: One Thought, Every Budget for Calibrated Typed Decisions

Kai YangDongdong Liu††thanks:Corresponding author:[liudongdong@bosssoft\.com\.cn](mailto:[email protected])\.Affiliation:Fujian Bosssoft Co\., Ltd\., Fuzhou, China

###### Abstract

Many uses of language models end in a typed decision: a choice among a closed set of options, whose probabilities code can act on\. Reading those probabilities from one forward pass is fast but often wrong on decisions that take several steps, such as arithmetic, counting, and date comparison\. Letting the model think first fixes most of these errors, but costs seconds per decision instead of milliseconds\. To pay for thinking only where it helps, ThinkJev builds a fast and a slow system from one open hybrid\-thinking model that can answer directly or after a thought\. System 1 reads the model’s probabilities for the option letters when it answers directly; System 2 reads the same letters after the model has thought on the same cached prompt, so escalating repeats no work and both systems give probabilities over the same options\. Because the letters can be read after any number of thought tokens, one sampled thought also gives the answer under every thinking budget\. Without training, ThinkJev lets System 1 answer alone only where a statistical test on labeled dev data guarantees that its accepted answers stay under a target error rate\. At a 5% target, Qwen3\.5\-27B answers 24\.5% of 8359 test decisions this way with a realized error of 3\.9% \(Qwen3\.5\-4B and 9B: 6\.5% and 0\.0%\)\. For the rest, stopping each thought as soon as its answer is confident reaches 81\.6% accuracy, against 82\.7% for always thinking, with 72% of the thought tokens\. Choosing when to think from System 1’s confidence alone saves less, because the decisions System 1 is least sure of are often not the ones thinking fixes\. On the public items of JevBench, an independent benchmark of typed\-decision services, Qwen3\.5\-27B thinks on 29\.4% and is not distinguishable in accuracy from the two thinking APIs listed \(exploratory\)\. Code and data generators are available at[https://github\.com/ding\-xiaolu/ThinkJev](https://github.com/ding-xiaolu/ThinkJev)\.

## 1Introduction

Many uses of language models end in a decision that code acts on rather than a person reads: which queue a ticket goes to, whether a message breaks a policy, whether an extracted record satisfies a rule\. We call these typed decisions: the answer is one of a fixed set of options, as in a multiple\-choice, yes/no, or rating question\. For them, a probability for each option is more useful than generated text\. It needs no parsing, and its confidence can decide whether to act, ask a person, or escalate\. Commercial decision APIs return answers in this form, with a probability per option and a confidence derived from them\[[27](https://arxiv.org/html/2610.37732#bib.bib27),[28](https://arxiv.org/html/2610.37732#bib.bib28)\]\. Open methods such as SemIf\[[26](https://arxiv.org/html/2610.37732#bib.bib26)\]and AnyJev\[[15](https://arxiv.org/html/2610.37732#bib.bib15)\]obtain calibrated option probabilities from any open model in one forward pass, by reading the model’s next\-token logits for the option letters\.

One forward pass is not enough for decisions that take several steps: date arithmetic, counting under conditions, chained rules, stacked negation\. One vendor lists exactly these as weak spots of its decision model\[[29](https://arxiv.org/html/2610.37732#bib.bib29)\]\. On the multi\-step decisions we build \(Section[4](https://arxiv.org/html/2610.37732#S4)\), a single forward pass answers only 52\.3% of them correctly with Qwen3\.5\-27B \(40\.3% with Qwen3\.5\-4B\)\. SemIf and AnyJev stop at this single pass\.

Letting the model think before it answers fixes most of these errors: after thinking, Qwen3\.5\-27B answers 87\.5% of the same decisions correctly\. Thinking is expensive, though\. On one GPU, a direct answer takes Qwen3\.5\-4B about 101 ms to process the prompt and 53 ms to read the answer, while thinking takes a median of 30\.3 s \(Section[5\.8](https://arxiv.org/html/2610.37732#S5.SS8)\), and on routine decisions thinking adds little accuracy \(Table[4](https://arxiv.org/html/2610.37732#S5.T4), block A\)\. A deployment therefore wants both: a direct answer for the decisions it can settle, and thinking for the rest\. This raises three questions\. Which decisions may skip thinking, and with what guarantee on their error rate? How long should the rest think, when the right budget differs from one decision to the next? And can the two be combined without running two models or processing the prompt twice?

Our starting point is an observation about hybrid\-thinking models such as the Qwen3 family\[[17](https://arxiv.org/html/2610.37732#bib.bib17)\], which answer either directly or after a thought, depending on how the chat template is set\. For the same decision, the direct prompt and the thinking prompt are identical up to the thinking block, and in both the answer can be read at the same place: the model’s next\-token logits for the option letters A, B, … at the answer position, which we call the answer slots\. One model therefore provides two systems on one prompt\. System 1 is the direct answer: the answer slots read after an empty thought\. System 2 is the answer after thinking: the same slots read after a sampled thought\. If the thought is cut after its firsttttokens and closed with the model’s own early\-stop sentence, reading the slots gives exactly the answer the model would have given under a thinking budget oftttokens\. One sampled thought therefore gives a calibrated answer at every budget, with System 1 at budget zero \(Figure[1](https://arxiv.org/html/2610.37732#S1.F1)\)\.

ThinkJev111The name refers to the typed\-decision request format of TypeSafe’s Jev, which ThinkJev’s server accepts \(Appendix[E](https://arxiv.org/html/2610.37732#A5)\)\. ThinkJev is independent of TypeSafe, and Jev is not evaluated in this paper\.builds a cascade on this observation\. It needs no training and answers the three questions in turn\. System 1 answers on its own only when its confidence reaches a threshold\. The threshold is chosen on a labeled development \(dev\) split with a statistical test, Learn\-then\-Test, which guarantees that the error rate of the answers System 1 gives stays under a target the user sets\. Every other decision is escalated: the model thinks, continuing from the prompt it has already processed, and the answer is read from the same option letters, again as calibrated probabilities\. The user can fix the thinking budget, or let the model stop as soon as its answer is confident\. No generated text is parsed\. We make the following contributions; each is tied to a measurement in Section[5](https://arxiv.org/html/2610.37732#S5)\.

1. 1\.Direct answers with an error guarantee\(Section[5\.5](https://arxiv.org/html/2610.37732#S5.SS5)\)\. System 1 gives calibrated option probabilities from one forward pass\. A threshold on its confidence, chosen by Learn\-then\-Test\[[1](https://arxiv.org/html/2610.37732#bib.bib1)\]on dev, bounds the error rate of the decisions it answers on its own; decisions generated from one template count as one sample, so near\-duplicates do not inflate the guarantee\. On every Qwen3\.5 model the error rate on test stays under each target for which the threshold accepts any decision\. How many decisions System 1 may answer depends on the model: at the strictest targets the smaller models accept none\. On Qwen3\.8\-27B the test error rate slightly exceeds the loosest target \(Figure[7](https://arxiv.org/html/2610.37732#S5.F7), Table[6](https://arxiv.org/html/2610.37732#S5.T6)\)\.
2. 2\.An answer at every thinking budget\(Section[5\.4](https://arxiv.org/html/2610.37732#S5.SS4)\)\. Cutting one sampled thought aftertttokens and reading the answer slots gives the answer under a budget oftt, so one run gives the whole curve of accuracy and calibration against the budget, with System 1 at budget zero\. An anytime policy can then stop thinking at the first checkpoint where the answer is confident \(Figure[6](https://arxiv.org/html/2610.37732#S5.F6)\)\.
3. 3\.One model as its own cascade, and the limit of gating on confidence\(Sections[3](https://arxiv.org/html/2610.37732#S3)and[5\.7](https://arxiv.org/html/2610.37732#S5.SS7)\)\. System 1 and System 2 come from the same model and the same processed prompt and read the same option letters, so the cascade needs no second model, no trained weights, and no parsing\. Deciding when to think from System 1’s confidence sends thinking to the decisions where System 1 is weak, but saves little thinking overall\. We trace this to a difference between two targets: predicting where System 1 is wrong is not the same as predicting where thinking fixes the answer \(Figure[3](https://arxiv.org/html/2610.37732#S5.F3), Table[4](https://arxiv.org/html/2610.37732#S5.T4), Section[5\.2](https://arxiv.org/html/2610.37732#S5.SS2)\)\.
4. 4\.A generated benchmark of multi\-step decisions\(Section[4](https://arxiv.org/html/2610.37732#S4)\)\. Eight families of decisions \(date arithmetic, conditional counting, chained rules, stacked negation, multi\-step arithmetic, logic grids, graph reachability, and scheduling\), each at four difficulty levels, written natively in English and in Chinese, plus chess puzzles\. The decisions are generated by code and every label is computed by code \(Figures[4](https://arxiv.org/html/2610.37732#S5.F4)and[5](https://arxiv.org/html/2610.37732#S5.F5)\)\.
5. 5\.An open server\(Appendix[E](https://arxiv.org/html/2610.37732#A5)\) for typed decisions that accepts the requests and returns the responses of a public decision API \(TypeSafe’sPOST /v1/systemone\), so that the API’s own SDK can call a local ThinkJev model\.

The text and figures use Qwen3\.5\-27B, the most accurate of the four models we run\. Every quantity is also reported for the other three models \(Section[4](https://arxiv.org/html/2610.37732#S4)\), and every criterion that fails on a model is reported as failed\.

A typed decisionAn order was placed on Thursday, August 5, 2027\. Orders ship on the 5th business day after the order date\. Business days are Monday to Friday; the order date itself does not count\.
On what date does the order ship?
AAugust 11, 2027BAugust 13, 2027CAugust 12, 2027DAugust 10, 2027
gold answer:Csysstate, question, optionsturnforkshared prefix: processed*once*System 1: no thoughtABCDB 0\.35picksB, confidence 0\.35
belowτ=0\.814\\tau=0\.814, so the model thinksSystem 2: one thought, read at every budgettt00ABCDB 0\.44128128ABCDB 0\.44256256ABCDB 0\.45512512ABCDC 0\.9510241024ABCDC 0\.8720482048ABCDC 0\.9314021402ABCDC 0\.90t=t\{=\}anytime stop att=512t=512: confidence 0\.95≥τ\\geq\\tau, answerC\(gold C\), using 512 of 1402 thought tokensAll 8359 test decisionsaccuracy at thinking budgettt60%70%80%01282565121k2kfullFigure 1:ThinkJev on one test decision, Qwen3\.5\-27B\. Left: the part of the prompt that the two systems share is processed once\. Top: System 1, the direct answer, gives calibrated option probabilities; its confidence is below the thresholdτ\\tauchosen on dev, so the model thinks\. Bottom: one sampled thought, with the same answer slots read after its firsttttokens \(each read calibrated with its own temperature; checkpoints not drawn to scale\)\. Each read is the answer under a thinking budget oftt, so the policy can stop at the first confident one\. Right: accuracy at each budget over all test decisions, with System 1 att=0t=0\. The example was chosen for illustration and is not typical; all values are generated from the released outputs \(example\.tex\)\.
## 2Background and related work

Deciding per query whether to think, and how much, is an active topic\. Table[1](https://arxiv.org/html/2610.37732#S2.T1)places ThinkJev among the closest lines of work; the paragraphs below say what each line does and what is different here\.

#### Typed\-decision models\.

TypeSafe’s Jev, a commercial decision model, answers choice, yes/no, and score questions with probabilities and a confidence computed from them\[[27](https://arxiv.org/html/2610.37732#bib.bib27),[28](https://arxiv.org/html/2610.37732#bib.bib28)\], and TypeSafe’s own documentation recommends gating actions on that confidence, for example routing low\-confidence decisions to a person\[[30](https://arxiv.org/html/2610.37732#bib.bib30)\]\. SemIf\[[26](https://arxiv.org/html/2610.37732#bib.bib26)\]reproduces the readout on open models by reading the logits of single\-token option letters after a fixed direct prompt; ThinkJev reuses its prompt and code unchanged\. AnyJev\[[15](https://arxiv.org/html/2610.37732#bib.bib15)\]adds cyclic\-shift debiasing, batch priors, coverage at a target risk, and hidden\-state heads\. Both are System 1 only\. In concurrent work, OpenJev\[[20](https://arxiv.org/html/2610.37732#bib.bib20)\]lets a request set a thought budget in advance, after which the answers are read; ThinkJev reads every budget from one thought and decides per decision whether to think\. ThinkJev’s gate is TypeSafe’s confidence\-gated routing pattern with the low\-confidence branch sent to the same model’s thinking mode, which answers through the same slots\.

#### Option readout and calibration\.

Language models prefer some option positions over others\[[36](https://arxiv.org/html/2610.37732#bib.bib36)\]; averaging the read over cyclic shifts of the option list removes any position bias that is additive in logit space\. Temperature scaling\[[5](https://arxiv.org/html/2610.37732#bib.bib5)\]fit on a dev split calibrates each readout; we fit one temperature per readout \(System 1, each anytime checkpoint, and the final System 2\)\.

#### Adaptive thinking\.

Training\-based methods change the weights so that the model itself allocates reasoning: When2Think uses RLVR post\-training with a difficulty\-aware efficiency bonus\[[22](https://arxiv.org/html/2610.37732#bib.bib22)\], AdapThink uses RL post\-training with adaptive thinking preferences\[[33](https://arxiv.org/html/2610.37732#bib.bib33)\], and CAT folds the model’s self\-certainty into preference optimization\[[6](https://arxiv.org/html/2610.37732#bib.bib6)\]\. ThinkJev trains nothing\. Sonata, from Adaptive Thinking\[[9](https://arxiv.org/html/2610.37732#bib.bib9)\], trains a light adapter on prefill hidden states to predict self\-consistency and sets the thinking budget before any thought; our probe router belongs to this family, reading the same model’s hidden states at the last position of the shared prefix\. DART\[[8](https://arxiv.org/html/2610.37732#bib.bib8)\]is training\-free and closest in spirit: it samples two no\-think drafts, answers directly when they agree, and otherwise predicts a budget from draft entropy, and it applies to free\-form answers such as math and code\. For closed\-set decisions, ThinkJev replaces the sampled drafts with one forward pass that already yields the full calibrated option distribution\. DEER\[[34](https://arxiv.org/html/2610.37732#bib.bib34)\]exits a reasoning trace early when a trial answer is confident; with fixed option slots, a trial answer is one short forward pass that appends the early\-stop sentence, and the anytime curve starts at System 1\. Studies of hybrid models report that reasoning leaks into the no\-think mode\[[31](https://arxiv.org/html/2610.37732#bib.bib31)\]and compare mode\-selection strategies empirically\[[25](https://arxiv.org/html/2610.37732#bib.bib25),[14](https://arxiv.org/html/2610.37732#bib.bib14)\]\. In our System 1 read, nothing is generated: the slots are read directly after the closed empty thinking block\.

#### Cascades, routing, and risk control\.

FrugalGPT\[[4](https://arxiv.org/html/2610.37732#bib.bib4)\]cascades across models with a learned answer scorer, and RouteLLM\[[16](https://arxiv.org/html/2610.37732#bib.bib16)\]trains a router between a strong and a weak model on preference data\. ThinkJev cascades within one model on one cached prefix, so escalation reuses the prefill and System 2’s answer is calibrated on the same scale\. Learn\-then\-Test\[[1](https://arxiv.org/html/2610.37732#bib.bib1)\]turns a threshold choice into a multiple\-testing problem with a finite\-sample guarantee; we apply it to the System 1 acceptance threshold\.

Table 1:Closest related work\. Descriptions follow each work’s own abstract or documentation\.

## 3Method

A decisionxxconsists of a state \(the facts to decide on\), a question, andKKoptions \(2≤K≤162\\leq K\\leq 16\), each with an ID and a description\. The model is a hybrid\-thinking language model, whose chat template selects either a direct answer or an answer after a thought\. Figure[2](https://arxiv.org/html/2610.37732#S3.F2)shows the pipeline\. Three requirements shape it\. Both systems must return probabilities over the same options, so that either can answer and code can act on the result the same way\. Escalating to System 2 must not redo the work already spent on System 1\. And the user, not the calibration, should decide how long a decision may think\. Reading the same answer slots after one shared prompt prefix \(Sections[3\.1](https://arxiv.org/html/2610.37732#S3.SS1)to[3\.3](https://arxiv.org/html/2610.37732#S3.SS3)\) meets the first two; reading them at checkpoints inside one thought \(Section[3\.4](https://arxiv.org/html/2610.37732#S3.SS4)\) meets the third\.

Decisionstate,question,KKoptionsShared prefixup to the fork,prefilled once→\\toKV cacheSystem 1copy the cache, read theanswer slots A, B, …overKKoption rotationsAccept S1?calibratedconfidencec1​\(x\)≥τc\_\{1\}\(x\)\\geq\\tauAnswer with S1calibratedp1\(⋅∣x\)p\_\{1\}\(\\cdot\\mid x\)cost: one prefill\+K\+\\,Kshort forwardsyesSystem 2: one sampled thoughtθ\\thetaoriginalcachet=0t\{=\}01281282562565125121024102420482048fullat each checkpointtt: copy the cache, append the early\-stop sentence, and read the*same*answer slots, which gives the answer under a thinking budget ofttno: thinkAnswer with S2calibratedp2,t\(⋅∣x\)p\_\{2,t\}\(\\cdot\\mid x\)cost:\+t\+\\,tthoughttokens, no new prefillThree policies, one rule\.Certified System 1:τ=τα\\tau=\\tau\_\{\\alpha\}from Learn\-then\-Test on dev, so the accepted answers’ error is at most the targetα\\alpha; the rest think to the full thought\.Gate:τ\\tauis the loosest dev threshold whose system accuracy is withinϵ\\epsilonof always thinking; the rest think to the full thought\.Anytime: the sameτ\\tau, and a rejected decision stops at the first checkpoint whose calibrated confidencec2,t​\(x\)c\_\{2,t\}\(x\)reachesτ\\tau\. No generated text is parsed and no weights are trained\.Figure 2:ThinkJev on one decision: the same answer slots read twice after one processed prompt prefix\. System 1 is read on a copy of the key\-value cache; if its calibrated confidence reachesτ\\tau, it answers\. Otherwise the model thinks, continuing on the original cache, and the same slots are read after the firstttthought tokens \(checkpoints not to scale\)\. The three policies of Sections[3\.5](https://arxiv.org/html/2610.37732#S3.SS5)and[3\.6](https://arxiv.org/html/2610.37732#S3.SS6)\(direct answers under an error guarantee, the gate, and the anytime policy\) differ only in howτ\\tauis chosen and where an escalated decision stops thinking\.### 3\.1System 1: reading the answer slots

System 1 is SemIf’sdirect\-options\-v1readout\[[26](https://arxiv.org/html/2610.37732#bib.bib26)\], whose code we copy unchanged\. The prompt lists the options under the letters A, B, …, and is rendered with thinking disabled, which for Qwen3\.5 closes an empty thinking block before the answer position\. Each letter must be exactly one token, and appending it must not change the tokenization of the prompt\. One forward pass gives the logitsz∈ℝKz\\in\\mathbb\{R\}^\{K\}of theKKletter tokens at the answer position, and

p1​\(k∣x\)=softmax⁡\(z/T1\)k,c1​\(x\)=maxk⁡p1​\(k∣x\),p\_\{1\}\(k\\mid x\)=\\operatorname\{softmax\}\(z/T\_\{1\}\)\_\{k\},\\qquad c\_\{1\}\(x\)=\\max\_\{k\}p\_\{1\}\(k\\mid x\),\(1\)with the temperatureT1T\_\{1\}fit on dev by negative log\-likelihood\[[5](https://arxiv.org/html/2610.37732#bib.bib5)\]\. Because SemIf’s prompt is used byte for byte, the hashes of our System 1 prompts equal those of SemIf’s published direct predictions \(Appendix[F](https://arxiv.org/html/2610.37732#A6)\)\.

#### Option rotations\.

Models favor some option positions\[[36](https://arxiv.org/html/2610.37732#bib.bib36)\], so a single read’s confidence partly reflects where an option is listed rather than how sure the model is, which makes it a poorer signal for deciding when to think\. Following AnyJev, we also read the prompt with the option list cyclically shiftedK−1K\-1more times\. The letters stay in place and each option moves with its description, so every option appears under every letter once\. Withℓk\(j\)\\ell^\{\(j\)\}\_\{k\}the log\-probability of optionkkunder shiftjj, the debiased read is

p¯1​\(k∣x\)∝exp⁡\(1K​∑j=0K−1ℓk\(j\)\),\\bar\{p\}\_\{1\}\(k\\mid x\)\\propto\\exp\\Bigl\(\\tfrac\{1\}\{K\}\\textstyle\\sum\_\{j=0\}^\{K\-1\}\\ell^\{\(j\)\}\_\{k\}\\Bigr\),\(2\)a renormalized geometric mean that removes any position bias additive in logit space\. It has its own dev temperature\. The shifted prompts share every token up to the first option description, so the extra cost isK−1K\-1short forward passes over the rest of the prompt\. The gate we report \(confidence\_rotin the released code\) uses this read, and the decisions it accepts are answered with its most probable option\.

### 3\.2Shared prefix and fork

An escalated decision should not pay for its prompt twice\. The two systems therefore share one pass over the prompt \(the prefill, which fills the model’s key\-value cache\), which requires finding where their prompts diverge\.

The System 1 prompt and the thinking prompt for the same decision are tokenized separately, and the fork is the end of their longest common token prefix\. It covers the system and user turns and ends inside the assistant’s opening; for Qwen3\.5 the text boundary after<think\>is not a token boundary, so the fork is located on tokens rather than text\. The shared prefix is prefilled once\. Qwen3\.5 mixes full\-attention layers with Gated DeltaNet layers\[[35](https://arxiv.org/html/2610.37732#bib.bib35)\], whose recurrent state cannot be rolled back to an earlier position\. System 1 is therefore read on a copy of the cache, and System 2 continues on the original, so escalation never repeats the prefill\.

### 3\.3System 2: the same slots after a thought

System 2 renders the same messages with thinking enabled and samples a thought with Qwen’s recommended thinking settings \(temperature 0\.6, top\-pp0\.95, top\-kk20\) and a seed derived from the decision’s ID\. If the thought closes by itself, we append</think\>and the tokens that follow the thinking block in the System 1 prompt\. If it reaches the thinking budgetBB, we append Qwen’s early\-stop sentence, which ends with</think\>\. In either case we read the logits of the same letter tokens at the same answer position, givingp2​\(k∣x\)=softmax⁡\(z2/T2\)kp\_\{2\}\(k\\mid x\)=\\operatorname\{softmax\}\(z\_\{2\}/T\_\{2\}\)\_\{k\}with its own temperatureT2T\_\{2\}, also fit on dev\. No generated text is parsed, so System 2 has no invalid outputs, and its answer has the same form as System 1’s: a probability over the same options, which the caller can use in place of System 1’s without any conversion\.

#### Batch System 2\.

For thousands of decisions, thoughts are sampled with vLLM from the exact token IDs of the thinking prompt, with the same settings and seeds, and the answer slots are then read in one forward pass over prompt, thought, and closing tokens\. On a replayed thought, this read reproduces the System 2 logits of the single\-process implementation that the server uses \(Appendix[F](https://arxiv.org/html/2610.37732#A6)\)\. Thoughts from vLLM are a different sample from the same distribution, so latency is measured with the single\-process implementation only\.

### 3\.4Anytime readout

The thinking budget a decision needs is not known in advance, and a caller may need to answer by a deadline\. Sampling one thought per budget would multiply the cost; instead, one thought is read at several points along the way\.

Letθ=\(θ1,…,θn\)\\theta=\(\\theta\_\{1\},\\dots,\\theta\_\{n\}\)be the sampled thought,n≤Bn\\leq B\. For a checkpointt<nt<n,*System 2 attt*appends the early\-stop sentence toθ1:t\\theta\_\{1:t\}and reads the same slots; fort≥nt\\geq nit is the final System 2 read\. Under a thinking budget oftt, the same sampler with the same seed emits the same firsttttokens and is then closed with the same sentence, so System 2 atttis the answer under budgetttfor this sampled thought \(Appendix[A](https://arxiv.org/html/2610.37732#A1)states the conditions\)\. One run therefore gives the answer at every budget up to the full thought\. In practice the thought is fed in chunks on one cache, and at each checkpoint a copy of the cache receives the early\-stop sentence and the slots are read\. We uset∈\{0,128,256,512,1024,2048\}t\\in\\\{0,128,256,512,1024,2048\\\}and the full thought \(≤4096\\leq 4096\)\. Each checkpoint has its own temperatureT2,tT\_\{2,t\}, fit on dev\.

Att=0t=0the read is the thinking template with an empty thought closed by the early\-stop sentence\. It differs from System 1 only by that sentence inside the thinking block, which is why we treat System 1 as the curve’s point at budget zero; Section[5\.4](https://arxiv.org/html/2610.37732#S5.SS4)measures how close the two reads are\.

### 3\.5Calibrated gating and the anytime policy

#### Gate\.

The gate thinks only where System 1 is unsure\. Accept System 1 ifc1​\(x\)≥τc\_\{1\}\(x\)\\geq\\tau; otherwise think and answer with System 2\. The thresholdτ\\tauis chosen on dev: among the thresholds whose accuracy on dev is at mostϵ\\epsilonbelow that of always thinking, take the one with the fewest thought tokens per decision\. Our main setting, the operating point used throughout, isϵ=1\\epsilon=1point; Table[3](https://arxiv.org/html/2610.37732#S5.T3)also reportsϵ∈\{0,2\}\\epsilon\\in\\\{0,2\\\}\. Test decisions never enter any fit or selection\.

#### Anytime policy\.

The anytime policy also stops thinking once the answer is confident\. Read System 1; ifc1​\(x\)<τc\_\{1\}\(x\)<\\tau, read System 2 at successive checkpoints and stop at the first whose calibrated confidencemaxk⁡softmax⁡\(z2,t/T2,t\)k\\max\_\{k\}\\operatorname\{softmax\}\(z\_\{2,t\}/T\_\{2,t\}\)\_\{k\}reachesτ\\tau, else use the full thought\. Its cost is the thought tokens up to the checkpoint where it stops; the short forward passes that read earlier checkpoints are not counted \(Section[6](https://arxiv.org/html/2610.37732#S6)\)\.τ\\tauis chosen on dev by the same rule as the gate\.

### 3\.6Risk guarantee for System 1

Calibration makes System 1’s confidence right on average, but an application that lets System 1 act alone needs a statement about the answers it accepts: of those, at most a fractionα\\alphais wrong\. Learn\-then\-Test\[[1](https://arxiv.org/html/2610.37732#bib.bib1)\]gives such a statement without training: it picks the threshold on dev so that, with probability at least1−δ1\-\\deltaover the draw of the dev data, the error rate of the answers System 1 gives on new data from the same distribution is at mostα\\alpha\. For a target error rate \(risk\)α∈\{1%,2%,5%,10%\}\\alpha\\in\\\{1\\%,2\\%,5\\%,10\\%\\\}we choose the System 1 acceptance threshold this way atδ=0\.05\\delta=0\.05\. Candidate thresholds are tested on dev from strictest to loosest with fixed\-sequence testing, which needs no multiplicity correction, and testing stops at the first failure\. At each candidate, the error rate of the decisions it accepts is tested with a one\-sided Clopper–Pearson upper bound\. Decisions from the same group \(e\.g\., variants of one template\) are not independent, so the sample size is the number of distinct groups accepted and the error count is scaled to it, rounded up\. The sequence starts at the first threshold whose group count would let zero errors pass,n≥log⁡δ/log⁡\(1−α\)n\\geq\\log\\delta/\\log\(1\-\\alpha\); this depends only on counts, so the guarantee is unaffected\. The dev threshold is applied unchanged to test, where we report its coverage \(the share of decisions System 1 answers\) and its realized risk \(the error rate of those answers\)\.

### 3\.7A server for typed decisions

thinkjev\-serveanswersPOST /v1/systemonein the request and response shapes of TypeSafe’s public API\[[27](https://arxiv.org/html/2610.37732#bib.bib27)\]\. Each choice, yes/no, or score question becomes one decision, runs through the gated cascade, and is answered with the calibrated probabilities of whichever system answered; temperatures and the threshold are read from an evaluation report\. Appendix[E](https://arxiv.org/html/2610.37732#A5)gives the mapping, an example, and the differences from Jev\.

## 4Experimental setup

#### Models\.

Four hybrid\-thinking models, all in BF16 and pinned by exact revision \(Appendix[F](https://arxiv.org/html/2610.37732#A6)\): Qwen3\.5\-4B\[[19](https://arxiv.org/html/2610.37732#bib.bib19)\], Qwen3\.5\-9B, Qwen3\.5\-27B\[[18](https://arxiv.org/html/2610.37732#bib.bib18)\], and Qwen3\.8\-27B, all run on the same decisions with the same settings and analysis code\. Qwen3\.8’s chat template inserts a reasoning\-effort note before the user turn in thinking mode, so its thinking prompt is rendered withreasoning\_effort="medium", the one setting that adds no note; only then do its two prompts share the decision’s prefix\. Thinking uses a budget ofB=4096B=4096tokens, one sampled thought per decision \(seed index 0\), and the settings of Section[3\.3](https://arxiv.org/html/2610.37732#S3.SS3)\. The text and figures use Qwen3\.5\-27B, the most accurate of the four; every quantity is reported for every model \(Tables[7](https://arxiv.org/html/2610.37732#S5.T7),[6](https://arxiv.org/html/2610.37732#S5.T6), and[8](https://arxiv.org/html/2610.37732#S5.T8)\), and the repository holds the same figures for every model\.

#### Data\.

The decisions come in three blocks \(Table[2](https://arxiv.org/html/2610.37732#S4.T2)\)\. Block A holds routine decisions that a direct answer often settles: intent routing over BANKING77\[[3](https://arxiv.org/html/2610.37732#bib.bib3)\]and CLINC150\[[7](https://arxiv.org/html/2610.37732#bib.bib7)\], detecting whether a request should be escalated to a person \(as AnyJev defines the task\), natural language inference \(NLI\) from WANLI\[[11](https://arxiv.org/html/2610.37732#bib.bib11)\], the hand\-written examples that ship with SemIf, a public set of labeled judgment questions that SemIf also uses \(“Every judgments”\), and the typed\-decisions dataset\[[12](https://arxiv.org/html/2610.37732#bib.bib12)\]\. Block B holds public reasoning benchmarks with fixed options, which let readers compare the two systems on established tasks \(Section[5\.3](https://arxiv.org/html/2610.37732#S5.SS3)\): the multiple\-choice tasks of BIG\-Bench Hard\[[24](https://arxiv.org/html/2610.37732#bib.bib24)\], MMLU\-Pro\[[32](https://arxiv.org/html/2610.37732#bib.bib32)\], GPQA Diamond\[[21](https://arxiv.org/html/2610.37732#bib.bib21)\], SuperGPQA\[[13](https://arxiv.org/html/2610.37732#bib.bib13)\], and MuSR\[[23](https://arxiv.org/html/2610.37732#bib.bib23)\]\. Block C is ours and holds multi\-step decisions: the eight generated families of Section[1](https://arxiv.org/html/2610.37732#S1)at four difficulty levels each, written in English and in Chinese; an earlier set of 200 hard generated decisions \(hard200\); and chess puzzles from the Lichess database\[[10](https://arxiv.org/html/2610.37732#bib.bib10)\]\(pick the best of four legal moves, with difficulty set by the puzzle’s rating\)\. Only block C has Chinese decisions\. Appendix[B](https://arxiv.org/html/2610.37732#A2)describes the generators\. All decisions are split into dev \(40%\) and test by a hash of their source and group, so variants of one template never fall on both sides of the split\. The repository stores only decision IDs; third\-party text is rebuilt locally from upstream versions pinned by commit\.

BlockSourceDevTestLanguageLicenseAEscalation \(CLINC\)210290enCC\-BY\-3\.0AEvery judgments7084enunstatedAIntent routing \(BANKING77\)313486enCC\-BY\-4\.0AIntent routing \(CLINC\)242358enCC\-BY\-3\.0ANLI \(WANLI\)97159enCC\-BY\-4\.0ASemIf authored6084enMITATyped decisions8351165enApache\-2\.0BBIG\-Bench Hard400620enMITBGPQA Diamond81117enCC\-BY\-4\.0BMMLU\-Pro475715enMITBMuSR217383enCC\-BY\-4\.0BSuperGPQA428652enODC\-BY\-1\.0CChess \(Lichess\)467733en, zhCC0\-1\.0CGenerated families16082392en, zhMITChard200 fixture79121enMITAll55828359Table 2:Decisions per source and split, from the released split file \(benchmarks/manifests/release\-split\.jsonl\); licenses as stated upstream \(THIRD\_PARTY\.md\)\. Block A: routine decisions; B: public reasoning; C: generated multi\-step decisions and chess\. Every source is used by all four models\.
#### Criteria\.

We fixed two success criteria and two comparisons before the runs\. The*savings criterion*: at the threshold chosen on dev, the gate’s test accuracy is within 1\.5 points of always thinking while it escalates at most half of the decisions\. The*risk criterion*: the test error rate of the decisions that the Learn\-then\-Test threshold lets System 1 answer stays within each target\. The anytime curve is reported descriptively, without a criterion\. The*ranking comparison*sets the gate against pre\-routers \(below\)\. A planned comparison with Jev on our decisions was dropped before any run, because we do not call Jev’s API \(Section[6](https://arxiv.org/html/2610.37732#S6)\)\. Each criterion is reported for every model, and a criterion that fails is reported as failed\. Temperatures, probes, and thresholds are fit on dev, and operating points are chosen on dev; test decisions never enter any fit or selection\.

#### Policies and baselines\.

A policy decides, for each decision, whether System 1 answers or the decision is escalated, i\.e\., sent to System 2\. All policies are simulated offline from one run that reads both System 1 and System 2 on every decision\. The two endpoints are always answering directly \(always\-S1\) and always thinking \(always\-S2\)\. The gate thresholds System 1’s calibrated confidence, read either once \(confidencein the released code\) or averaged over option rotations \(confidence\_rot, the gate we report\); the anytime policy is described in Section[3\.5](https://arxiv.org/html/2610.37732#S3.SS5)\. Pre\-routers are the usual alternative to gating on an answer’s confidence: they make the same choice from the decision’s text alone, before any answer is read\. We compare three\. The*probe router*\(router\_probe\) is a ridge regression on the same model’s hidden states at the last position of the shared prefix \(layer and penalty chosen by group cross\-validation on dev\)\. The*zero\-shot router*\(router\_zero\_shot\) asks the same model, with the System 1 prompt, whether the decision can be made at a glance \(Appendix[D](https://arxiv.org/html/2610.37732#A4)\)\. The*nn\-gram router*\(router\_supervised\) is a logistic regression on hashednn\-grams of the decision’s text, trained on dev to predict whether System 1 is right\. The oracle escalates exactly the decisions where System 1 is wrong and System 2 is right\.

#### Metrics\.

Accuracy; mean thought tokens per decision, counting zero for decisions System 1 answers; expected calibration error \(ECE: top label, 10 equal\-width bins\); coverage and realized risk at each target \(Section[3\.6](https://arxiv.org/html/2610.37732#S3.SS6)\); and, for the ranking comparison, the area under the test accuracy–coverage curve, normalized so that random escalation scores 0 and the oracle 1:\(policy−random\)/\(oracle−random\)\(\\text\{policy\}\-\\text\{random\}\)/\(\\text\{oracle\}\-\\text\{random\}\)\. Differences from the gate get 95% confidence intervals from a bootstrap over groups \(1,000 resamples\)\. Realized risk is shown with an exact binomial 95% interval over the accepted test decisions; it treats decisions as independent, so it is narrower than an interval over groups\. We also count the test decisions where System 1 and System 2 differ in correctness, which bounds what any gate can gain \(Section[6](https://arxiv.org/html/2610.37732#S6)\)\.

#### Compute and latency\.

Thoughts are sampled with vLLM across GPUs; answer slots are read in a separate process with one GPU each\. Latency is measured for Qwen3\.5\-4B with the single\-process implementation, on one H200 at batch size 1, over a stratified subset of 600 test decisions, with nothing else running on the machine\.

## 5Results

All numbers below come from the evaluation reports of complete runs; each table and figure names the report fields it reads\. Thresholds were chosen on dev and applied unchanged to test\. The text uses Qwen3\.5\-27B \(Section[4](https://arxiv.org/html/2610.37732#S4)\); Section[5\.6](https://arxiv.org/html/2610.37732#S5.SS6)and Tables[6](https://arxiv.org/html/2610.37732#S5.T6)and[8](https://arxiv.org/html/2610.37732#S5.T8)give the same quantities for every model\.

### 5\.1Main result

On 8359 test decisions, Qwen3\.5\-27B answers 62\.2% correctly when it always answers directly and 82\.7% when it always thinks, spending 1856 thought tokens per decision on average\. With the threshold chosen on dev \(at most one point below always thinking on dev\), the gate reaches 81\.6% with 1511 thought tokens per decision, 81% of what always thinking spends; the anytime policy reaches 81\.6% with 1344 \(72%\)\.

The savings criterion fails on every model\.Its accuracy half holds: the gate is 1\.1 points below always thinking on Qwen3\.5\-27B, inside the 1\.5\-point margin, and inside it on the other three models\. Its savings half fails: the gate escalates 62\.4% of test decisions on Qwen3\.5\-27B and between 60\.6% and 75\.1% on the other three, against a limit of 50% \(Table[7](https://arxiv.org/html/2610.37732#S5.T7)\)\. A looser tolerance does not rescue it \(Table[3](https://arxiv.org/html/2610.37732#S5.T3)\): allowed two points below always thinking, the gate still escalates 53\.6% of decisions, and at zero tolerance it escalates 89\.5%, nearly always thinking\. Confidence gating keeps accuracy but does not let most decisions skip thinking\. Figure[3](https://arxiv.org/html/2610.37732#S5.F3)shows the full trade\-off and Table[4](https://arxiv.org/html/2610.37732#S5.T4)breaks it down by block\.

#### Rotations\.

Always\-S1 is the single read, without rotations\. Averaged over rotations, the read that the gate uses answers 64\.2% correctly, against 62\.2% for the single read \(ECE 0\.024 against 0\.027\)\. The single read is fragile: its most probable option changes under some cyclic shift on 37\.5% of test decisions \(55\.9% on Qwen3\.5\-4B\)\. The rotations costK−1K\-1short forward passes, and they are why the gate’s curve in Figure[3](https://arxiv.org/html/2610.37732#S5.F3)starts above the always\-S1 marker\. AnyJev’s batch prior, applied on top of the rotation average, lowers accuracy to 63\.6% and raises ECE to 0\.086, so it is not used\. Temperature scaling does most of the calibration work: without it, the rotation average’s ECE is 0\.115\.

Table 3:Qwen3\.5\-27B, test: the gate and the anytime policy at each toleranceϵ\\epsilon\. The threshold is chosen on dev to stay at mostϵ\\epsilonbelow always thinking, then applied unchanged to test\. “Thinking” is thought tokens per decision as a share of always thinking’s\. The 1\-point row is the setting used everywhere else\. Source:dev\_selected\_operating\_points\(gate\),anytime\_test\.dev\_selected\_operating\_points\(anytime\)\.Figure 3:Test accuracy against thought tokens per decision for Qwen3\.5\-27B\. Each curve sweeps the threshold of one policy: the gate, the anytime policy, and the probe router\. The oracle escalates exactly the decisions that System 1 gets wrong and System 2 gets right\. Always answering directly \(always\-S1\) and always thinking \(always\-S2\) are the endpoints; the marked point is the gate’s threshold chosen on dev and applied unchanged to test\. Source fields:cost\_curves\_test,anytime\_test\.curves,anytime\_test\.dev\_selected\_operating\_points\.Table 4:Qwen3\.5\-27B test accuracy by block, and the share of decisions the gate escalates\. The gate’s threshold is the one inper\_family\_test\_gate\(at most one point below always thinking on dev, with the most decisions answered directly\); values are weighted by the number of decisions in each entry ofper\_family\_test\. The numbers in the text use the threshold with the fewest thought tokens on dev \(anytime\_test\.dev\_selected\_operating\_points\), so the “All” row can differ from them slightly\. Appendix[C](https://arxiv.org/html/2610.37732#A3)breaks this down per source for every model\.

### 5\.2Where the thinking goes

Figure[4](https://arxiv.org/html/2610.37732#S5.F4)groups test decisions by task: the families of block C and the sources of blocks A and B\. The gate sends most of block C to System 2 \(77\.1% of its decisions\), where System 1 is weakest \(52\.3% correct\) and thinking helps the most \(87\.5%\)\. Date comparison and multi\-step arithmetic are escalated on 99\.7% and 99\.7% of decisions, and System 2 answers 99\.0% and 100\.0% of them correctly, against 35\.5% and 37\.5% for System 1\. For intent routing on CLINC150, where System 1 is already at 95\.5%, the gate escalates only 9\.8%\.

Two groups show the limit of gating on System 1’s confidence\. The gate escalates 97\.8% of the chess puzzles, but thinking lifts their accuracy only from 40\.4% to 45\.8%; on the typed\-decisions dataset it escalates 61\.5% for a change from 71\.2% to 73\.3%\. In both, System 1 is often wrong and its low confidence shows it, yet thinking fixes few of those errors and breaks some correct answers\. The gate follows where System 1 is wrong, not where thinking helps, and that difference is what the oracle exploits \(Section[5\.7](https://arxiv.org/html/2610.37732#S5.SS7)\)\.

Language makes no difference within block C\. On the generated families, System 1 answers 54\.1% of the English and 54\.8% of the Chinese decisions, System 2 99\.5% and 99\.7%, and the gate escalates 71\.7% and 74\.8% \(Qwen3\.5\-4B: System 1 40\.9% and 41\.0%\)\. Chinese decisions score lower overall only because all of them are multi\-step decisions \(Appendix[C](https://arxiv.org/html/2610.37732#A3)\)\.

Figure[5](https://arxiv.org/html/2610.37732#S5.F5)plots escalation against difficulty level for each generated family\. Escalation rises with level in 7 of the 9 families \(rank correlation above zero\), and over all generated decisions it goes from 60\.0% at level 1 to 94\.1% at level 4, while System 1’s accuracy falls from 66\.6% to 39\.3%\. Qwen3\.5\-4B shows the same pattern in 7 of 9 families\.

Figure 4:Per task group: accuracy of System 1, System 2, and the gate \(left\), and the share of decisions the gate sent to System 2 \(right\), Qwen3\.5\-27B, test\. Source:per\_family\_test\.Figure 5:Escalation rate and System 1 accuracy by difficulty level, one panel per generated family, Qwen3\.5\-27B, test\. Source:per\_family\_test\.
### 5\.3Public benchmarks

Eight of our sources are public benchmarks: three classification sets in block A and five reasoning benchmarks in block B \(Section[4](https://arxiv.org/html/2610.37732#S4)\)\. Table[5](https://arxiv.org/html/2610.37732#S5.T5)gives System 1, System 2, the gate, and the gate’s escalation rate on each, for every model\. Two patterns hold on all four models\. On the reasoning benchmarks, thinking adds the most and the gate escalates most decisions: on Qwen3\.5\-27B, BIG\-Bench Hard goes from 70\.6% with System 1 to 94\.8% with System 2, MMLU\-Pro from 56\.9% to 82\.9%, and GPQA Diamond from 60\.7% to 85\.5%, with 78\.6% of GPQA Diamond escalated\. On the classification sets, thinking adds little \(BANKING77: 85\.0% to 86\.4%\) and the gate lets System 1 answer most decisions \(it escalates 9\.8% of CLINC150\)\. Everywhere, the gate’s accuracy stays within a few points of always thinking, and closest where it escalates most, so the thinking it skips is mostly thinking that would not have changed the answer\.

These scores are not comparable with the scores reported for the same models on the same benchmarks elsewhere\. We read one option letter from SemIf’s prompt \(Appendix[D](https://arxiv.org/html/2610.37732#A4)\), cap thinking at 4,096 tokens, use one sampled thought, offer the classification sets a subset of their labels \(Appendix[D](https://arxiv.org/html/2610.37732#A4)\), and score only our test split of each benchmark \(Table[2](https://arxiv.org/html/2610.37732#S4.T2)\); published scores typically parse a generated answer after a longer thought and use the full set\.

Table 5:Public benchmarks, test split: accuracy \(%\) of System 1, System 2, and the gate, and the share of decisions the gate escalates \(%\), for every model\. The gate uses the threshold inper\_family\_test\_gate, as in Table[4](https://arxiv.org/html/2610.37732#S5.T4); scores follow our protocol \(one option letter, a 4,096\-token budget\) and are not comparable with published scores\. Source:per\_family\_test\.
### 5\.4The anytime curve

Figure[6](https://arxiv.org/html/2610.37732#S5.F6)shows test accuracy and calibration error of System 2 at each think budget, each with its own dev temperature\. Att=0t=0, the thinking template with an empty thought reaches 61\.8% \(ECE 0\.035\), next to 62\.2% \(ECE 0\.027\) for System 1 read without rotations: the two reads nearly coincide, as Section[3\.4](https://arxiv.org/html/2610.37732#S3.SS4)expects\. The curve stays flat over the first 256 thought tokens \(62\.7%\), rises through 512 \(68\.9%\) and 1,024 \(77\.4%\) tokens, and is at 81\.4% by 2,048, close to the full thought’s 82\.7%; 81\.0% of thoughts end by themselves within the 4,096\-token budget\. Mid\-thought reads stay calibrated: ECE lies between 0\.017 and 0\.036 at every checkpoint\. They stay calibrated only because each checkpoint has its own temperature\. The fitted temperature rises with the budget on every model, from 1\.61 att=0t=0to 4\.07 at the full thought on Qwen3\.5\-27B \(Qwen3\.5\-4B: 1\.61 to 3\.78; Qwen3\.8\-27B: 1\.68 to 5\.31\), against 1\.68 for System 1’s rotation average\. The raw logits grow more confident faster than the answers improve: the longer the model has thought, the more its raw confidence overstates its accuracy, and one temperature cannot serve every budget\. The Qwen3\.5 models share the shape of the accuracy curve\. Qwen3\.8\-27B thinks far less \(722 mean thought tokens\), so its curve rises earlier \(69\.5% at 256 tokens\), and its calibration error is higher at long budgets, from 0\.029 att=0t=0to 0\.058 at the full thought\. We report the anytime curve descriptively and draw no directional conclusion from it\.

Reading every budget from one thought is what makes the anytime policy possible: on Qwen3\.5\-27B it spends 72% of always thinking’s thought tokens, against the gate’s 81%, at the same accuracy\. The saving comes from stopping early\. Withτ\\tauchosen on dev, the policy answers 35\.5% of test decisions with System 1, 25\.5% at a checkpoint before the full thought, and 39\.0% after the full thought; 39\.6% of the escalated decisions stop early\. The saving shrinks where thoughts are short: on Qwen3\.8\-27B only 12\.5% of escalated decisions stop early, because few checkpoints fall before the end of its thoughts, and the two policies spend 85% and 86%\.

Figure 6:System 2 accuracy and expected calibration error under each thinking budgettt, read from one sampled thought per decision, Qwen3\.5\-27B, test\. Source:anytime\_test\.per\_checkpoint,anytime\_test\.full\_thought\.
### 5\.5Risk guarantee

At a 5% target risk, the Learn\-then\-Test threshold chosen on dev lets System 1 answer 24\.5% of Qwen3\.5\-27B’s test decisions, and those answers have a test error rate \(realized risk\) of 3\.9% \[3\.1, 4\.8\]%\. Table[6](https://arxiv.org/html/2610.37732#S5.T6)gives all four targets for every model and Figure[7](https://arxiv.org/html/2610.37732#S5.F7)plots them for Qwen3\.5\-27B\.

On the three Qwen3\.5 models, realized risk stays at or under the target wherever the threshold accepts any decision, so the risk criterion holds\. It holds only trivially where the threshold accepts nothing, because then System 1 answers nothing\. Qwen3\.5\-27B accepts decisions at 3 of the four targets \(all but 1%\), Qwen3\.5\-4B at 2 \(5% and 10%\), and Qwen3\.5\-9B at 1 \(10% only\)\. Accepting nothing follows from the test design: fixed\-sequence testing starts at the strictest threshold and stops at its first failure, and with groups as the sample size a 1% target needs about 300 accepted groups before even an error\-free set of accepted decisions can pass\. If System 1’s most confident dev answers already contain errors, the sequence stops before accepting anything\. On Qwen3\.8\-27B the threshold accepts decisions at all four targets, but at the 10% target the realized risk is 10\.5% \[9\.5, 11\.7\]%, above the target\. Its interval contains the target, and Learn\-then\-Test bounds risk only with probability1−δ1\-\\deltaover dev draws, but by the risk criterion this is a violation, and we report it as one\.

Table 6:System 1 answers accepted by the Learn\-then\-Test threshold \(δ=0\.05\\delta=0\.05\), test: the share of decisions answered by System 1 and the error rate of those answers \(realized risk\), with an exact binomial 95% interval over the accepted decisions\. “0\.0%” answered with “–” risk means the threshold accepts no decision\. Source:s1\_readouts\_test\.rotation\.coverage\_at\_risk\.Figure 7:Target risk against realized test risk of System 1 answers accepted under the Learn\-then\-Test threshold \(δ=0\.05\\delta=0\.05\), Qwen3\.5\-27B, with exact binomial 95% intervals\. The hollow marker is a target at which no decision is accepted\. Source:s1\_readouts\_test\.rotation\.coverage\_at\_risk\.
### 5\.6Scale

Table[7](https://arxiv.org/html/2610.37732#S5.T7)repeats the headline quantities for all four models with the same settings\. Across the Qwen3\.5 models, accuracy rises with scale for both systems, from 50\.9% to 62\.2% for System 1 and from 77\.4% to 82\.7% for System 2, and the gap that thinking closes narrows\. The risk guarantee does not improve steadily with scale: Qwen3\.5\-9B accepts decisions at fewer targets than Qwen3\.5\-4B\. Qwen3\.8\-27B is close to Qwen3\.5\-27B in accuracy with less than half the thinking, which leaves the anytime policy little to cut\.

QuantityQwen3\.5\-4BQwen3\.5\-9BQwen3\.5\-27BQwen3\.8\-27BTest decisions8359835983598359Always S1: accuracy50\.9%54\.1%62\.2%60\.2%Always S2: accuracy77\.4%80\.0%82\.7%81\.8%Always S2: thought tokens per decision212620881856722Oracle accuracy81\.5%83\.5%85\.8%86\.2%Gate: accuracy76\.5%79\.1%81\.6%80\.9%Gate: share of always\-S2 tokens86%89%81%86%Savings: points below always S2 \(limit 1\.5\)1\.01\.01\.11\.0Savings: escalated \(limit 50%\)71\.6%75\.1%62\.4%60\.6%Anytime: accuracy76\.5%78\.9%81\.6%80\.8%Anytime: share of always\-S2 tokens83%82%72%85%System 1 ECE \(calibrated\)0\.0290\.0370\.0240\.029LTT: targets with any acceptance \(of 4\)2134LTT at 5%: answered by S16\.5%0\.0%24\.5%23\.2%LTT at 5%: realized risk1\.1%–3\.9%4\.2%Table 7:Headline quantities for every model, test\. Always S1: always answer directly; always S2: always think\. Gate and anytime values use the thresholds chosen on dev \(dev\_selected\_operating\_points, tolerance one point\)\. “Savings” rows are the two halves of the savings criterion; LTT rows are the Learn\-then\-Test threshold for System 1\. Each value’s report field is named inresults/figures/<model\>/numbers\.tex\.
### 5\.7Gating versus pre\-routing

A good policy escalates first the decisions where thinking helps\. Table[8](https://arxiv.org/html/2610.37732#S5.T8)measures this with the normalized area under the accuracy–coverage curve \(Section[4](https://arxiv.org/html/2610.37732#S4)\), comparing the gate with each pre\-router\. A positive difference favors the gate, and we claim a difference only when its bootstrap interval excludes zero\. The zero\-shot router, which asks the model whether a decision can be made at a glance, ranks decisions better than the gate on Qwen3\.5\-9B, Qwen3\.5\-27B, and Qwen3\.8\-27B, and ties with it on Qwen3\.5\-4B\. The probe router, which reads the model’s hidden states, is worse than the gate on Qwen3\.5\-4B and Qwen3\.5\-27B and ties on the other two models\. Thenn\-gram router is worse everywhere\. All policies stay far from the oracle: the best normalized area is 0\.491 on Qwen3\.5\-27B, so no policy captures even half of the oracle’s advantage over random escalation\. Section[5\.2](https://arxiv.org/html/2610.37732#S5.SS2)suggests why a System 1 signal cannot: the decisions where System 1 is least sure include many that thinking does not fix\. The zero\-shot router costs one more short read on the same prefix, so it is as cheap as the gate\. The probe router appears in Figure[3](https://arxiv.org/html/2610.37732#S5.F3)for reference\.

#### Combining the two signals \(post hoc\)\.

After these results had been read, we fit a logistic regression on the logits of the gate’s confidence and the zero\-shot score, trained on dev to predict System 1’s correctness and cross\-fitted by group \(Appendix[F](https://arxiv.org/html/2610.37732#A6)\)\. It is ahead of the gate on every model by normalized area \(Qwen3\.5\-27B: 0\.449 against 0\.436, difference−\-0\.013 \[−\-0\.014,−\-0\.011\]; last row of Table[8](https://arxiv.org/html/2610.37732#S5.T8)\), but on the three larger models it stays below the zero\-shot router alone \(0\.491 on Qwen3\.5\-27B\)\. The fit says why\. Trained to predict whether System 1 is right, it puts almost all of its weight on the confidence \(standardized weights 1\.355 and 0\.076 on Qwen3\.5\-27B\) and matches the confidence’s AUROC for System 1 correctness \(0\.803 against 0\.803; the zero\-shot score alone has 0\.705\)\. The zero\-shot router ranks worse by that target and better by area because its low scores pick out the decisions that thinking fixes, not the decisions System 1 gets wrong\. With its threshold chosen on dev, the combination escalates 60\.1% of decisions against the gate’s 62\.4% \(Qwen3\.8\-27B: 56\.6% against 60\.6%\), so it does not change the verdict on the savings criterion\. A combination trained on the gain from thinking needs System 2 labels on dev and is left to future work\.

Qwen3\.5\-4BQwen3\.5\-9BQwen3\.5\-27BQwen3\.8\-27BGate: normalized area0\.3600\.3670\.4360\.421Gate−\-zero\-shot router\+0\.011−\-0\.063−\-0\.054−\-0\.067\[−\-0\.017, \+0\.037\]\[−\-0\.091,−\-0\.038\]\[−\-0\.080,−\-0\.027\]\[−\-0\.092,−\-0\.041\]Gate−\-probe router\+0\.043−\-0\.014\+0\.040\+0\.021\[\+0\.018, \+0\.068\]\[−\-0\.039, \+0\.009\]\[\+0\.014, \+0\.068\]\[−\-0\.005, \+0\.050\]Gate−\-nn\-gram router\+0\.161\+0\.159\+0\.121\+0\.110\[\+0\.133, \+0\.190\]\[\+0\.131, \+0\.189\]\[\+0\.092, \+0\.152\]\[\+0\.080, \+0\.141\]Gate−\-combination−\-0\.018−\-0\.018−\-0\.013−\-0\.029\[−\-0\.023,−\-0\.013\]\[−\-0\.020,−\-0\.015\]\[−\-0\.014,−\-0\.011\]\[−\-0\.032,−\-0\.025\]Table 8:Ranking comparison: normalized area under the test accuracy–coverage curve,\(policy−random\)/\(oracle−random\)\(\\text\{policy\}\-\\text\{random\}\)/\(\\text\{oracle\}\-\\text\{random\}\), of the gate, and the gate’s difference from each pre\-router with a 95% interval from a bootstrap over groups \(1,000 resamples\); a negative difference means the router ranks better\. The last row is the combination of the gate’s confidence and the zero\-shot score, fit post hoc \(Section[5\.7](https://arxiv.org/html/2610.37732#S5.SS7)\)\. Source:normalized\_area\_test\.

### 5\.8Latency

On one H200 at batch size 1, with a median shared prefix of 261 tokens, Qwen3\.5\-4B spends 101 ms processing the shared prefix \(the prefill\) and 53 ms copying the cache and reading System 1’s answer slots; the rotations behind the gate’s confidence add 164 ms\. An escalated decision continues from the same prefill, so sharing the prefix saves one prefill per escalation\. That saving is small next to thinking itself: System 2 takes a median of 30\.3 s \(1038 thought tokens\) on the same decisions\. Whether a decision thinks, not the prefill, decides how long it takes\. Latency was measured on Qwen3\.5\-4B only, over 600 stratified test decisions, all of them thinking\.

### 5\.9External check on JevBench

Our own test set cannot compare ThinkJev with commercial decision services, because we do not call their APIs\. JevBench\[[2](https://arxiv.org/html/2610.37732#bib.bib2)\], an open benchmark of typed\-decision services maintained independently of us, makes that comparison possible: it publishes, next to its items, each listed system’s outcome on every item\. Its 231 public decisions come in three tiers\. The easy and standard tiers hold short business decisions \(intent and ticket routing, fact checks, field extraction, tool selection, policy compliance, adequacy and ordinal ratings\); the 111 hard items add long policies, multi\-hop and temporal or numeric reasoning, probability estimates, ambiguous trade\-offs, and adversarial phrasing\. Most are choice questions; the rest are yes/no and score questions\. The listed systems include Jev 1\.13\.0, the thinking APIs DeepSeek V4\.1 Flash and GPT\-5\.6 Luna, and four further systems, among them SemIf on Qwen3\.5\-4B \(Table[9](https://arxiv.org/html/2610.37732#S5.T9)\)\.

Each item is sent as the request JevBench’s own harness builds \(a state, instructions, and labeled options\), turned into a decision by the same mapping the server applies \(Appendix[E](https://arxiv.org/html/2610.37732#A5)\), and read with the same prompts as every other decision \(Appendix[D](https://arxiv.org/html/2610.37732#A4)shows one\)\. It is scored by JevBench’s rule: the top label must equal the expected one\. Nothing is fit on JevBench: the temperatures and the gate and anytime thresholds are each model’s values chosen on our dev split \(Section[5\.1](https://arxiv.org/html/2610.37732#S5.SS1)\), so the check also tests whether they transfer to a new domain\. This check was added after the main results had been read and is exploratory\. The easy and standard items are near ceiling for most listed systems, so differences come from the hard tier\. Table[9](https://arxiv.org/html/2610.37732#S5.T9)gives the accuracies, and we compare each policy with each listed system by an exact McNemar test on the same items\.

System 1 alone is level with Jev\. Qwen3\.5\-27B’s System 1 answers 207 of 231 items correctly against Jev’s 200 \(12 items only ThinkJev gets right, 5 only Jev,p=0\.143p=0\.143\); Qwen3\.5\-4B’s System 1 is below Jev \(185,p=0\.014p=0\.014\), close to SemIf’s own entry on the same model \(187\), as expected from the same prompt\. Thinking where System 1 is unsure changes this\. On Qwen3\.5\-27B the gate answers 224 items \(26 only ThinkJev, 2 only Jev,p<0\.001p<0\.001\) while thinking on 29\.4% of them, 742 thought tokens per item against 1307 for always thinking \(226 correct\)\. Every thinking policy is above Jev on every model, on all items and on the hard tier alone, with the smallest margin on Qwen3\.5\-4B \(gate 211,p=0\.043p=0\.043\)\.

The most accurate listed systems are two thinking APIs, DeepSeek V4\.1 Flash with its default thinking \(226\) and GPT\-5\.6 Luna at low reasoning effort \(225\)\. Qwen3\.5\-27B’s gate cannot be told apart from either \(p=0\.754p=0\.754andp=1\.000p=1\.000\), nor can Qwen3\.8\-27B’s \(226;p=1\.000p=1\.000andp=1\.000p=1\.000\)\. Qwen3\.5\-4B’s gate is below both \(p=0\.003p=0\.003andp=0\.003p=0\.003\); Qwen3\.5\-9B’s is not distinguishable from them over all items \(p=0\.077p=0\.077andp=0\.092p=0\.092\) but below them on the hard tier\. So an open 27B model, without training and thinking on fewer than a third of the decisions, is in the same accuracy band as these APIs here\. “Cannot be told apart” is not a tie: with near\-ceiling accuracy and 111 hard items, the test resolves only large gaps\.

Unlike on our test set, the gate thinks on a minority of these decisions, between 24\.7% and 42\.0% across the four models, presumably because JevBench holds fewer multi\-step decisions than our block C\. Three caveats\. The listed systems were measured by JevBench, through their own servers and at their own settings, not by us\. JevBench’s own score also weighs speed and cost, and we report accuracy only: an escalated decision takes seconds \(Section[5\.8](https://arxiv.org/html/2610.37732#S5.SS8)\) where Jev answers in under one\. Contamination is unlikely: JevBench froze its items on 2026\-09\-19, after every model version we use was released \(2026\-08\-14 for Qwen3\.8\-27B, March and April 2026 for the Qwen3\.5 models\)\.

PolicyQwen3\.5\-4BQwen3\.5\-9BQwen3\.5\-27BQwen3\.8\-27BS1 only80\.1% / 62\.2%82\.3% / 64\.9%89\.6% / 81\.1%88\.7% / 77\.5%Gate91\.3% / 83\.8%94\.4% / 88\.3%97\.0% / 93\.7%97\.8% / 95\.5%Anytime91\.3% / 83\.8%93\.5% / 87\.4%97\.0% / 93\.7%96\.5% / 92\.8%Always S293\.1% / 85\.6%94\.8% / 89\.2%97\.8% / 95\.5%99\.6% / 99\.1%Escalated by the gate32\.9%42\.0%29\.4%24\.7%Listed on JevBench \(same items, from its per\-task file\)Jev 1\.13\.086\.6% / 73\.0%DeepSeek V4\.1 Flash \(thinking\)97\.8% / 96\.4%GPT\-5\.6 Luna \(low reasoning\)97\.4% / 96\.4%OpenJev \(thinking\)88\.7% / 76\.6%djev \(thinking\)87\.4% / 76\.6%SemIf \(Qwen3\.5\-4B\)81\.0% / 61\.3%reflex\-27b \(Qwen3\.8\-27B\)87\.0% / 75\.7%Table 9:JevBench public items: accuracy on all 231 / on the 111 hard ones, under JevBench’s scoring rule, with each model’s release temperatures and thresholds \(nothing fit on JevBench\)\. Listed systems’ outcomes are JevBench’s own measurements at the pinned commit \(Appendix[F](https://arxiv.org/html/2610.37732#A6)\)\. Exploratory\. Source:results/jevbench\-<model\>\-report\-v2\.json\.

## 6Analysis and limitations

#### Power\.

The gate can only make a difference on decisions where System 1 and System 2 differ in correctness\. For Qwen3\.5\-27B there are 1971 test decisions where System 1 is wrong and System 2 right, and 260 where thinking turns a correct answer into a wrong one\. Most fixes are in block C \(1212 fixed, 70 broken\), then block B \(576 and 91\); in block A thinking fixes 183 answers but breaks 99\. Every model has at least 2231 such test decisions \(Qwen3\.5\-4B: 2900\), so the comparisons between policies do not lack decisions where the choice matters\.

#### One thought per decision\.

System 2 is sampled once per decision, so its accuracy and every anytime point carry sampling noise that a second seed would reduce\. The equivalence between a checkpoint and a thinking budget holds for the thought that was sampled, not for the expected answer under that budget\.

#### Data\.

Public benchmarks may be in the model’s training data; the generated block C is not, and we read the two side by side rather than pooling conclusions\. The labels of the typed\-decisions dataset come from a single teacher model, not from people\. Chinese decisions exist only in block C\.

#### Calibration and thresholds\.

Temperatures and thresholds are fit on a labeled dev split from the same sources as the test decisions\. They are not expected to transfer across domains, and a deployment needs labeled decisions from its own domain\. Nor does one temperature serve every thinking budget: the raw logits grow more overconfident as the thought lengthens \(Section[5\.4](https://arxiv.org/html/2610.37732#S5.SS4)\), so a deployment that changes its budget must refit\. The risk guarantee assumes that test groups are exchangeable with dev groups, i\.e\., drawn from the same distribution\.

#### Scope of the readout\.

Decisions have at most 16 options, each read through one letter token, so free\-form answers are out of scope\. Rotations cover cyclic shifts of the listed order, not arbitrary permutations\.

#### Cost accounting\.

Thinking, once triggered, is still slow: thought tokens dominate the cost of an escalated decision\. The anytime policy’s cost counts thought tokens up to the checkpoint where it stops, but not the short forward passes that read earlier checkpoints\.

#### Comparisons we did not run\.

We did not call Jev’s API, so Jev is not evaluated on our test set\. The only comparison with Jev is on JevBench’s public items \(Section[5\.9](https://arxiv.org/html/2610.37732#S5.SS9)\), where Jev’s outcomes are JevBench’s published measurements; other statements about Jev cite TypeSafe’s own pages\. We also did not run DART\[[8](https://arxiv.org/html/2610.37732#bib.bib8)\]or Sonata\[[9](https://arxiv.org/html/2610.37732#bib.bib9)\]; our probe router stands in for the family of routers that read hidden states\. The gate that combines System 1’s confidence with the zero\-shot score \(Section[5\.7](https://arxiv.org/html/2610.37732#S5.SS7)\) was fit post hoc, after the planned analyses had been read, and only on System 1’s correctness; a combination trained on the gain from thinking, which needs System 2 labels on dev, was not run\.

## 7Conclusion

A typed decision can often be settled by a direct answer, but not when it takes several steps, and thinking on every decision is slow\. ThinkJev shows that one open hybrid\-thinking model can serve both cases: reading the same option letters with and without a thought, after one processed prompt, gives a fast and a slow system whose probabilities are directly comparable, and reading them part\-way through one thought gives the answer under every thinking budget\. No training is needed\.

Three findings matter for a deployment\. First, direct answers can be used on their own with a guaranteed error rate: at a 5% target, Qwen3\.5\-27B answers 24\.5% of decisions from one forward pass, with an error rate of 3\.9% on test, although smaller models can answer far fewer this way \(Qwen3\.5\-4B: 6\.5%; Qwen3\.5\-9B: 0\.0%\)\. Second, stopping a thought once its answer is confident keeps most of thinking’s accuracy \(81\.6% against 82\.7%\) with 72% of the thought tokens\. Third, System 1’s confidence is a weak guide to when to think: our target of thinking on at most half of the decisions failed on every model, because the decisions System 1 is least sure of are often ones that thinking does not fix\. On JevBench, an independent benchmark, the same settings put an open 27B model in the accuracy band of the thinking APIs listed there\.

The clearest next step is a router trained to predict where thinking helps, rather than where System 1 is wrong, which needs System 2 labels on dev; sampling more than one thought per decision would also reduce the noise in the anytime curve\. Code, data generators, the IDs and splits of every decision, evaluation reports, and a server for typed decisions are released at[https://github\.com/ding\-xiaolu/ThinkJev](https://github.com/ding-xiaolu/ThinkJev)\.

## References

- \[1\]Anastasios N\. Angelopoulos, Stephen Bates, Emmanuel J\. Candès, Michael I\. Jordan, and Lihua Lei\.Learn then test: Calibrating predictive algorithms to achieve risk control\.arXiv preprint arXiv:2110\.01052, 2021\.
- \[2\]Benchmark Heaven\.JevBench\.[https://github\.com/fstandhartinger/jevbench](https://github.com/fstandhartinger/jevbench), 2026\.Commit 7625ecb1b075cfe4d09d34141d353325825eea1d\.
- \[3\]Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić\.Efficient intent detection with dual sentence encoders\.arXiv preprint arXiv:2003\.04807, 2020\.
- \[4\]Lingjiao Chen, Matei Zaharia, and James Zou\.FrugalGPT: How to use large language models while reducing cost and improving performance\.arXiv preprint arXiv:2305\.05176, 2023\.
- \[5\]Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q\. Weinberger\.On calibration of modern neural networks\.In*International Conference on Machine Learning \(ICML\)*, 2017\.
- \[6\]Qizhi Jiang, Shuo Wang, Pei Ke, Yuhang Song, and Ke Qin\.CAT: Confidence\-adaptive thinking for efficient reasoning of large reasoning models\.arXiv preprint arXiv:2607\.00862, 2026\.
- \[7\]Stefan Larson, Anish Mahendran, Joseph J\. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K\. Kummerfeld, Kevin Leach, Michael A\. Laurenzano, Lingjia Tang, and Jason Mars\.An evaluation dataset for intent classification and out\-of\-scope prediction\.In*Proceedings of EMNLP\-IJCNLP 2019*, 2019\.
- \[8\]Jungseob Lee, Seongtae Hong, Seungjun Lee, Jaehyung Seo, Junyoung Son, Sugyeong Eo, Chanjun Park, Hyeongju Park, Hyeonseok Moon, and Heuiseok Lim\.DART: Draft\-agreement routing for training\-free adaptive thinking budgets in hybrid reasoning models\.arXiv preprint arXiv:2606\.23181, 2026\.
- \[9\]Pingzhi Li, Bairu Hou, Yun Zhu, Yihao Feng, Ke Ye, Tao Lei, Zhifeng Chen, Tianlong Chen, and Xianzhi Du\.Adaptive thinking: Large language models know when to think in latent space\.In*International Conference on Learning Representations \(ICLR\)*, 2026\.URL[https://openreview\.net/forum?id=2i6Rp0gCq6](https://openreview.net/forum?id=2i6Rp0gCq6)\.
- \[10\]Lichess\.Lichess puzzle database\.[https://database\.lichess\.org/\#puzzles](https://database.lichess.org/#puzzles)\.CC0 1\.0; SHA\-256 of the dump recorded per row\.
- \[11\]Alisa Liu, Swabha Swayamdipta, Noah A\. Smith, and Yejin Choi\.WANLI: Worker and AI collaboration for natural language inference dataset creation\.In*Findings of the Association for Computational Linguistics: EMNLP 2022*, 2022\.
- \[12\]LocalLLaMA\.typed\-decisions\.[https://huggingface\.co/datasets/LocalLLaMA/typed\-decisions](https://huggingface.co/datasets/LocalLLaMA/typed-decisions)\.Revision pinned in THIRD\_PARTY\.md\.
- \[13\]M\-A\-P Team, Xinrun Du, Yifan Yao, Kaijing Ma, et al\.SuperGPQA: Scaling LLM evaluation across 285 graduate disciplines\.arXiv preprint arXiv:2502\.14739, 2025\.
- \[14\]Yansong Ning, Mianpeng Liu, Jingwen Ye, Weidong Zhang, and Hao Liu\.HRBench: Benchmarking and understanding thinking\-mode switch strategies in hybrid\-reasoning LLMs\.arXiv preprint arXiv:2605\.28398, 2026\.
- \[15\]Nokia Applied Research\.AnyJev\.[https://github\.com/nokia\-applied\-research/AnyJev](https://github.com/nokia-applied-research/AnyJev), 2026\.Commit 10d5db91dda38dbde74c6abc1c075ce6463723d1\.
- \[16\]Isaac Ong, Amjad Almahairi, Vincent Wu, Wei\-Lin Chiang, Tianhao Wu, Joseph E\. Gonzalez, M\. Waleed Kadous, and Ion Stoica\.RouteLLM: Learning to route LLMs with preference data\.arXiv preprint arXiv:2406\.18665, 2024\.
- \[17\]Qwen Team\.Qwen3 technical report\.arXiv preprint arXiv:2505\.09388, 2025\.
- \[18\]Qwen Team\.Qwen3\.5\-27B\.[https://huggingface\.co/Qwen/Qwen3\.5\-27B](https://huggingface.co/Qwen/Qwen3.5-27B), 2026a\.
- \[19\]Qwen Team\.Qwen3\.5\-4B\.[https://huggingface\.co/Qwen/Qwen3\.5\-4B](https://huggingface.co/Qwen/Qwen3.5-4B), 2026b\.Revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a\.
- \[20\]razorback16\.OpenJev\.[https://github\.com/razorback16/openjev](https://github.com/razorback16/openjev), 2026\.Commit dba783d9ce5327c1f97932a3074fd0c5bf5cb0d1\.
- \[21\]David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R\. Bowman\.GPQA: A graduate\-level google\-proof Q&A benchmark\.arXiv preprint arXiv:2311\.12022, 2023\.
- \[22\]Jaejun Shim, HyunJin Kim, Young Jin Kim, and JinYeong Bak\.When2Think: Learning when and how much to reason\.arXiv preprint arXiv:2609\.19671, 2026\.
- \[23\]Zayne Sprague, Xi Ye, Kaj Bostrom, Swarat Chaudhuri, and Greg Durrett\.MuSR: Testing the limits of chain\-of\-thought with multistep soft reasoning\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.
- \[24\]Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V\. Le, Ed H\. Chi, Denny Zhou, and Jason Wei\.Challenging BIG\-Bench tasks and whether chain\-of\-thought can solve them\.arXiv preprint arXiv:2210\.09261, 2022\.
- \[25\]Yuqiao Tan, Shizhu He, Kang Liu, and Jun Zhao\.The zero\-step thinking: An empirical study of mode selection as harder early exit in reasoning models\.arXiv preprint arXiv:2510\.19176, 2025\.NeurIPS 2025 Efficient Reasoning Workshop\.
- \[26\]TheoLeeCJ\.SemIf\.[https://github\.com/TheoLeeCJ/SemIf](https://github.com/TheoLeeCJ/SemIf), 2026\.Commit 1f2dea3e25379f9dfc98cb83c324f00ab5deda37\.
- \[27\]TypeSafe AI\.TypeSafe API reference: POST /v1/systemone\.[https://docs\.typesafe\.ai/api](https://docs.typesafe.ai/api), 2026a\.Accessed 2026\-10\-03\.
- \[28\]TypeSafe AI\.Confidence\.[https://docs\.typesafe\.ai/confidence](https://docs.typesafe.ai/confidence), 2026b\.Accessed 2026\-10\-03\.
- \[29\]TypeSafe AI\.Jev 1\.13 jaggedness\.[https://docs\.typesafe\.ai/model\-jaggedness/jev\-1\.13](https://docs.typesafe.ai/model-jaggedness/jev-1.13), 2026c\.Accessed 2026\-10\-03\.
- \[30\]TypeSafe AI\.Confidence\-gated routing\.[https://docs\.typesafe\.ai/patterns/confidence\-routing](https://docs.typesafe.ai/patterns/confidence-routing), 2026d\.Accessed 2026\-10\-03\.
- \[31\]Shouren Wang, Wang Yang, Xianxuan Long, Qifan Wang, Vipin Chaudhary, and Xiaotian Han\.Demystifying hybrid thinking: Can LLMs truly switch between think and no\-think?arXiv preprint arXiv:2510\.12680, 2025\.
- \[32\]Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen\.MMLU\-Pro: A more robust and challenging multi\-task language understanding benchmark\.In*Advances in Neural Information Processing Systems \(Datasets and Benchmarks Track\)*, 2024\.
- \[33\]Wenyue Xu, Xu Wan, Wei Wang, Wenqi Huang, Wotao Yin, Shengjie Zhao, and Mingyang Sun\.AdapThink: Adaptive thinking preferences for reasoning language models\.In*Findings of the Association for Computational Linguistics: ACL 2026*, pages 9808–9825, 2026\.URL[https://aclanthology\.org/2026\.findings\-acl\.477/](https://aclanthology.org/2026.findings-acl.477/)\.
- \[34\]Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Qiaowei Li, Minghui Chen, Zheng Lin, and Weiping Wang\.Dynamic early exit in reasoning models\.arXiv preprint arXiv:2504\.15895, 2025a\.
- \[35\]Songlin Yang, Jan Kautz, and Ali Hatamizadeh\.Gated delta networks: Improving Mamba2 with delta rule\.In*International Conference on Learning Representations \(ICLR\)*, 2025b\.
- \[36\]Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang\.Large language models are not robust multiple choice selectors\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.

## Appendix AThe anytime readout equals a thinking budget

Fix a decision, the thinking prompt’s token IDs, the sampler \(temperature, top\-pp, top\-kk\), and the decision’s seed\. Letθ\\thetabe the thought this sampler emits when it may run to the budgetBB, andn=\|θ\|n=\|\\theta\|\.

#### Claim\.

Fort<nt<n, running the same sampler with budgetttemitsθ1:t\\theta\_\{1:t\}, and closing it with the early\-stop sentenceeeand reading the answer slots gives the same logits as the checkpoint read onθ1:t∥e\\theta\_\{1:t\}\\\|e\. Fort≥nt\\geq nthe thought closes by itself before the budget binds, so the answer under budgetttis the final System 2 read\.

#### Conditions\.

\(i\) Sampling is autoregressive and the random draw for positioniidepends only on the seed andii, so the firsttttokens do not depend on the budget\. \(ii\) The early\-stop sentence is appended only when the thought did not end by itself\. \(iii\) The logits used for sampling and for reading are numerically identical in both runs; with batched GPU kernels this holds up to floating\-point differences, which the tests bound \(Appendix[F](https://arxiv.org/html/2610.37732#A6)\)\. In the batch pipeline,θ\\thetacomes from vLLM and the reads from a separate forward pass, so the equivalence is with a budget\-ttrun of the vLLM sampler, and the read’s agreement with the single\-process implementation is checked on replayed thoughts\.

#### Implementation\.

The thought is fed in chunks on one cache\. At each checkpointtta copy of the cache receiveseeand the answer slots are read; the original continues\. The final System 2 read from the chunked cache matches a one\-shot read, and each checkpoint read matches a one\-shot read of the cut thought \(tests/test\_pipeline\_tiny\.py\)\.

## Appendix BGenerated decisions \(block C\)

benchmarks/generate\_owned\.pywrites decisions whose labels are computed by code: date arithmetic; conditional counting over a table; chained rules with exceptions and overrides; stacked negation and quantifiers, with each answer checked against every possible case; multi\-step arithmetic; logic grids whose unique solution is checked by brute force; graph reachability and shortest paths; and scheduling with overlap, capacity, and time zones\. Each family has four difficulty levels that add reasoning steps \(more rows, conditions, rules, entities, hops, or a unit conversion\) and is written natively in English and in Chinese\. Answer positions, answer classes, and for numeric options the answer’s rank among the values are balanced per family, language, and level; numeric distractors come from typical slips\. The default file has 2,400 English and 1,600 Chinese decisions; a test regenerates it byte for byte and re\-derives every label independently\. The chess decisions turn Lichess puzzles \(CC0\) with popularity at least 80 and at least 1,000 plays into “best move” questions with four legal SAN options, with difficulty from the rating bucket \(<<1200, 1200–1599, 1600–1999,≥\\geq2000\)\. Table[10](https://arxiv.org/html/2610.37732#A2.T10)shows the level\-1 English decision of each family, abridged; the full decisions areowned/<family\>/en/d1/000in the committedbenchmarks/data/owned\-v1\.jsonl\.

Table 10:One generated decision per family \(level 1, English\), abridged from the released file\.
## Appendix CResults per source

Tables[11](https://arxiv.org/html/2610.37732#A3.T11)–[14](https://arxiv.org/html/2610.37732#A3.T14)give, for every model, System 1, System 2, and gate accuracy and the gate’s escalation rate per source, language, and difficulty level\. The gate uses the threshold inper\_family\_test\_gate; each line is weighted by the number of decisions over the families ofper\_family\_test, and level “–” marks sources without difficulty levels\.

Table 11:Qwen3\.5\-27B: test accuracy and escalation per source, language, and difficulty level\.BlockSourceLang\.LevelRowsS1S2GateEscalatedAEscalation \(CLINC\)en–29060\.0%72\.4%71\.0%43\.1%AEvery judgmentsen–8495\.2%95\.2%95\.2%7\.1%AIntent routing \(BANKING77\)en–48685\.0%86\.4%86\.4%18\.9%AIntent routing \(CLINC\)en–35895\.5%97\.5%97\.2%9\.8%ANLI \(WANLI\)en–15975\.5%81\.1%79\.2%32\.7%ASemIf authoreden–8492\.9%94\.0%95\.2%10\.7%ATyped decisionsen–116571\.2%73\.3%73\.7%61\.5%BBIG\-Bench Harden–62070\.6%94\.8%94\.4%53\.9%BGPQA Diamonden–11760\.7%85\.5%85\.5%78\.6%BMMLU\-Proen–71556\.9%82\.9%82\.8%71\.9%BMuSRen–38362\.1%68\.9%68\.7%51\.2%BSuperGPQAen–65247\.7%62\.1%62\.0%83\.4%CChess \(Lichess\)en112164\.5%71\.1%71\.1%92\.6%CChess \(Lichess\)en212534\.4%42\.4%42\.4%98\.4%CChess \(Lichess\)en312634\.9%37\.3%37\.3%100\.0%CChess \(Lichess\)en412232\.8%37\.7%37\.7%100\.0%CChess \(Lichess\)zh15850\.0%58\.6%58\.6%91\.4%CChess \(Lichess\)zh25940\.7%40\.7%40\.7%100\.0%CChess \(Lichess\)zh36033\.3%46\.7%46\.7%100\.0%CChess \(Lichess\)zh46229\.0%29\.0%29\.0%100\.0%CGenerated familiesen135669\.9%100\.0%99\.4%49\.4%CGenerated familiesen236266\.6%100\.0%93\.1%60\.8%CGenerated familiesen335839\.7%100\.0%95\.8%84\.1%CGenerated familiesen436940\.7%98\.1%95\.7%91\.9%CGenerated familieszh123266\.8%100\.0%99\.1%51\.3%CGenerated familieszh224564\.5%100\.0%95\.5%67\.3%CGenerated familieszh324744\.1%100\.0%96\.0%87\.9%CGenerated familieszh422343\.5%98\.7%96\.4%92\.8%Chard200 fixtureen–12182\.6%100\.0%95\.9%34\.7%Table 12:Qwen3\.5\-4B: test accuracy and escalation per source, language, and difficulty level\.BlockSourceLang\.LevelRowsS1S2GateEscalatedAEscalation \(CLINC\)en–29069\.0%84\.1%77\.2%34\.8%AEvery judgmentsen–8496\.4%95\.2%94\.0%4\.8%AIntent routing \(BANKING77\)en–48679\.2%81\.1%80\.5%21\.0%AIntent routing \(CLINC\)en–35889\.9%95\.5%95\.3%15\.9%ANLI \(WANLI\)en–15964\.2%73\.6%69\.8%35\.2%ASemIf authoreden–8482\.1%95\.2%94\.0%23\.8%ATyped decisionsen–116563\.3%71\.0%70\.3%59\.1%BBIG\-Bench Harden–62055\.3%89\.8%89\.0%72\.6%BGPQA Diamonden–11730\.8%71\.8%71\.8%95\.7%BMMLU\-Proen–71537\.8%73\.6%73\.1%83\.2%BMuSRen–38353\.8%62\.4%61\.4%55\.6%BSuperGPQAen–65229\.8%45\.4%44\.8%88\.5%CChess \(Lichess\)en112147\.1%55\.4%55\.4%98\.3%CChess \(Lichess\)en212534\.4%35\.2%35\.2%99\.2%CChess \(Lichess\)en312624\.6%28\.6%28\.6%100\.0%CChess \(Lichess\)en412227\.0%29\.5%29\.5%100\.0%CChess \(Lichess\)zh15846\.6%62\.1%62\.1%98\.3%CChess \(Lichess\)zh25928\.8%25\.4%25\.4%100\.0%CChess \(Lichess\)zh36040\.0%35\.0%35\.0%100\.0%CChess \(Lichess\)zh46227\.4%27\.4%27\.4%100\.0%CGenerated familiesen135655\.6%99\.7%96\.6%77\.2%CGenerated familiesen236245\.0%97\.2%96\.1%92\.8%CGenerated familiesen335830\.4%95\.0%94\.4%98\.6%CGenerated familiesen436932\.8%92\.7%92\.7%99\.5%CGenerated familieszh123256\.9%100\.0%98\.7%75\.0%CGenerated familieszh224540\.8%99\.2%96\.3%91\.0%CGenerated familieszh324733\.6%95\.5%95\.1%99\.6%CGenerated familieszh422332\.7%88\.8%88\.8%100\.0%Chard200 fixtureen–12166\.9%97\.5%95\.0%65\.3%Table 13:Qwen3\.5\-9B: test accuracy and escalation per source, language, and difficulty level\.BlockSourceLang\.LevelRowsS1S2GateEscalatedAEscalation \(CLINC\)en–29053\.4%84\.1%72\.1%36\.6%AEvery judgmentsen–8494\.0%95\.2%95\.2%8\.3%AIntent routing \(BANKING77\)en–48678\.6%82\.9%82\.7%31\.5%AIntent routing \(CLINC\)en–35891\.3%97\.8%97\.8%27\.4%ANLI \(WANLI\)en–15967\.3%75\.5%74\.8%54\.7%ASemIf authoreden–8492\.9%94\.0%94\.0%32\.1%ATyped decisionsen–116563\.3%71\.8%71\.0%66\.4%BBIG\-Bench Harden–62059\.4%92\.7%92\.6%74\.7%BGPQA Diamonden–11746\.2%73\.5%73\.5%94\.0%BMMLU\-Proen–71544\.5%76\.2%75\.9%86\.7%BMuSRen–38354\.8%63\.4%61\.6%63\.4%BSuperGPQAen–65235\.6%51\.5%51\.2%93\.7%CChess \(Lichess\)en112157\.0%66\.1%66\.1%97\.5%CChess \(Lichess\)en212534\.4%40\.0%40\.0%99\.2%CChess \(Lichess\)en312630\.2%35\.7%35\.7%100\.0%CChess \(Lichess\)en412231\.1%32\.0%32\.0%100\.0%CChess \(Lichess\)zh15858\.6%60\.3%60\.3%96\.6%CChess \(Lichess\)zh25935\.6%40\.7%40\.7%100\.0%CChess \(Lichess\)zh36033\.3%35\.0%35\.0%100\.0%CChess \(Lichess\)zh46222\.6%25\.8%25\.8%100\.0%CGenerated familiesen135657\.6%99\.7%98\.0%68\.3%CGenerated familiesen236254\.7%98\.9%97\.8%91\.7%CGenerated familiesen335839\.1%100\.0%99\.4%97\.2%CGenerated familiesen436936\.3%96\.5%96\.2%98\.9%CGenerated familieszh123261\.2%100\.0%98\.7%78\.0%CGenerated familieszh224549\.4%100\.0%100\.0%97\.6%CGenerated familieszh324734\.8%98\.8%98\.8%98\.8%CGenerated familieszh422336\.3%94\.6%94\.6%100\.0%Chard200 fixtureen–12174\.4%97\.5%91\.7%62\.8%Table 14:Qwen3\.8\-27B: test accuracy and escalation per source, language, and difficulty level\.BlockSourceLang\.LevelRowsS1S2GateEscalatedAEscalation \(CLINC\)en–29065\.2%80\.7%80\.7%44\.5%AEvery judgmentsen–8496\.4%97\.6%97\.6%7\.1%AIntent routing \(BANKING77\)en–48685\.6%87\.2%87\.2%14\.2%AIntent routing \(CLINC\)en–35895\.0%96\.9%96\.4%8\.7%ANLI \(WANLI\)en–15974\.2%77\.4%76\.7%24\.5%ASemIf authoreden–8494\.0%97\.6%97\.6%13\.1%ATyped decisionsen–116572\.8%73\.2%74\.0%57\.9%BBIG\-Bench Harden–62069\.5%93\.1%92\.9%56\.5%BGPQA Diamonden–11748\.7%75\.2%75\.2%83\.8%BMMLU\-Proen–71551\.3%78\.7%78\.3%67\.1%BMuSRen–38359\.0%67\.9%66\.1%43\.6%BSuperGPQAen–65242\.0%53\.7%54\.4%79\.0%CChess \(Lichess\)en112154\.5%62\.0%64\.5%81\.0%CChess \(Lichess\)en212530\.4%41\.6%42\.4%93\.6%CChess \(Lichess\)en312629\.4%44\.4%44\.4%96\.0%CChess \(Lichess\)en412225\.4%44\.3%44\.3%99\.2%CChess \(Lichess\)zh15853\.4%69\.0%70\.7%87\.9%CChess \(Lichess\)zh25945\.8%47\.5%47\.5%96\.6%CChess \(Lichess\)zh36035\.0%45\.0%45\.0%100\.0%CChess \(Lichess\)zh46227\.4%43\.5%43\.5%100\.0%CGenerated familiesen135668\.8%100\.0%98\.3%49\.2%CGenerated familiesen236260\.2%99\.4%93\.4%61\.9%CGenerated familiesen335838\.5%98\.9%95\.0%86\.9%CGenerated familiesen436939\.6%98\.1%95\.1%90\.5%CGenerated familieszh123266\.8%100\.0%99\.1%52\.2%CGenerated familieszh224561\.2%99\.6%95\.5%69\.0%CGenerated familieszh324740\.1%99\.6%93\.9%88\.3%CGenerated familieszh422340\.4%98\.7%96\.9%93\.3%Chard200 fixtureen–12180\.2%100\.0%93\.4%40\.5%
## Appendix DPrompts

Every read in this paper, on every source, uses the prompts below; nothing is tuned per benchmark\.

#### System 1\.

SemIf’sdirect\-options\-v1\[[26](https://arxiv.org/html/2610.37732#bib.bib26)\], unchanged\. A decision \(state, question, options\) becomes two chat messages\. The system message is \(one line in the prompt\):

```
Apply the supplied criterion to the supplied evidence. Choose exactly one
listed option. Respond with only its uppercase letter, with no explanation
or reasoning.
```

The user message is one JSON object with the decision’s state asevidence, its question ascriterion, and the options under the letters A, B, … in listed order\. For the request in Appendix[E](https://arxiv.org/html/2610.37732#A5), the questiondeptbecomes \(one line in the prompt\):

```
{"evidence": {"ticket": "I was charged twice for March."},
 "criterion": "Which team?",
 "options": [{"letter": "A", "description": "Payments"},
             {"letter": "B", "description": "tech"}]}
```

An option without a description is listed under its ID\. The messages are rendered with the model’s chat template with thinking disabled, which for Qwen3\.5 ends the prompt with an empty thinking block after the assistant’s opening, and System 1 is the next\-token distribution over the letters at that position \(Section[3\.1](https://arxiv.org/html/2610.37732#S3.SS1)\)\. Option IDs are never shown to the model, so the rotations of Section[3\.1](https://arxiv.org/html/2610.37732#S3.SS1)move descriptions between letters without other changes\.

#### System 2\.

The same two messages, rendered with thinking enabled, so the prompt ends inside an open thinking block; the shared prefix is the longest common token prefix of the two renderings \(Section[3\.2](https://arxiv.org/html/2610.37732#S3.SS2)\)\. A thought that ends by itself is followed by the tokens that follow the thinking block in the System 1 prompt\. A thought cut at a checkpoint or at the budget is closed with Qwen’s early\-stop sentence:

```
Considering the limited time by the user, I have to give the solution based
on the thinking directly now.
</think>
```

\(one line, preceded by a blank line and followed by one\)\. The answer slots are then read at the same letters as System 1’s\.

#### Zero\-shot router\.

The same System 1 prompt, applied to a new decision whose evidence is the original decision \(its state, question, and option descriptions, without letters\), with the question

```
Before answering it, judge this decision task: can the criterion be decided
reliably at a glance, without step-by-step work such as calculation,
counting, date arithmetic, or chaining several rules?
```

and two options, “Yes: it can be decided reliably at a glance” and “No: it needs step\-by\-step work before it can be decided”\. Its score is the probability of the first\.

#### Benchmark decisions\.

Multiple\-choice benchmarks \(block B\) keep their question and options\. Classification sets list labels as options, within the 16\-option limit: a BANKING77 decision offers 16 of the 20 most frequent intents \(the gold one and 15 others chosen by hash\), a CLINC150 decision the 15 intents of the gold intent’s domain, so every distractor is a near miss; out\-of\-scope CLINC queries form the separate escalation task\. These subsets make the tasks differ from the usual full\-label protocols\. JevBench items become decisions through the server’s mapping \(Appendix[E](https://arxiv.org/html/2610.37732#A5)\), so a JevBench choice item is read exactly like the example above\.benchmarks/build\_cascade\.pybuilds every decision, andtests/test\_semif\_parity\.pypins the System 1 prompt\.

## Appendix EServer interface

A request names a state and typed questions, as in TypeSafe’s API\[[27](https://arxiv.org/html/2610.37732#bib.bib27)\]:

```
POST /v1/systemone
{"state": {"ticket": "I was charged twice for March."},
 "questions": {
   "dept": {"type": "choice", "instructions": "Which team?",
            "criteria": {"billing": "Payments", "tech": null}},
   "mood": {"type": "score", "instructions": "How angry?",
            "criteria": ["Calm", "Angry", "Furious"]}},
 "thinkjev": {"think": "auto", "include_thought": false}}
```

A Choice keeps its options, a Noul becomes the optionstrue/false, and a Score becomes the levels0,…,n−10,\\dots,n\-1, whose value is the probability\-weighted level\. Each answer has the shape of the corresponding TypeSafe answer type \(choice or score with probabilities and confidence; noul as a probability\); confidence follows TypeSafe’s published formulas\[[28](https://arxiv.org/html/2610.37732#bib.bib28)\], for a Choice withnnoptions\(pmax−1/n\)/\(1−1/n\)\(p\_\{\\max\}\-1/n\)/\(1\-1/n\)\. The optionalthinkjevfield selectsauto\(gated\),never\(System 1 only\), oralways\(System 2\), and the response adds athinkjevmap with per\-question traces: which system answered, System 1’s confidence, and the threshold\. Differences from Jev: at most 16 options or levels per question, questions run one after another, andusage\.output\_tokenscounts thought tokens\. A test round\-trips a request through TypeSafe’s SDK \(tests/test\_api\.py\)\. With"stream": truethe same request answers as a server\-sent event stream: System 1’s calibrated probabilities, then the answer read at each checkpoint as the thought passes it, so the server’s demo page draws Figure[1](https://arxiv.org/html/2610.37732#S1.F1)for a live decision\. ThinkJev is independent of TypeSafe; only the public request format is shared\.

## Appendix FReproduction

Models are pinned by Hugging Face revision:

Qwen3\.5\-4B851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0aQwen3\.5\-9Bc202236235762e1c871ad0ccb60c8ee5ba337b9aQwen3\.5\-27Bfc05daec18b0a78c049392ed2e771dde82bdf654Qwen3\.8\-27B1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
SemIf’s code is copied from commit1f2dea3; a smoke run reproduces theprompt\_sha256of SemIf’s committed direct predictions, andtests/test\_semif\_parity\.pypins the prompt\. The release split isbenchmarks/manifests/release\-split\.jsonl; rebuilding it withbuild\_cascade\.py assemblemust reproduce the file byte for byte\. Commands for one model end to end are inscripts/run\_model\.sh; every figure and number in this paper is regenerated by

```
python benchmarks/make_figures.py --report results/<model>-b4096-report-v2.json \
    --label <Model> --out results/figures/<model> --regenerate \
    --manifest benchmarks/manifests/release-split.jsonl \
    --jevbench results/jevbench-<model>-report-v2.json
```

\(with\-\-latencyresults/qwen3\.5\-4b\-latency\.jsonlfor Qwen3\.5\-4B, and\-\-exampleowned/dates/en/d2/050for the Qwen3\.5\-27B decision in Figure[1](https://arxiv.org/html/2610.37732#S1.F1)\)\. The release reports areresults/<model\>\-b4096\-report\.jsonforqwen3\.5\-4b,qwen3\.5\-9b,qwen3\.5\-27b, andqwen3\.8\-27b, next to each model’s per\-decision cascade and router outputs\. They are attached to releasev0\.1\.0of the repository asresults\.tar\.gz, which unpacks toresults/, andresults/SHA256SUMSlists the SHA\-256 of every report, cascade, router, and latency file \(shasum \-a 256 \-c SHA256SUMSinresults/checks them\)\. Theresults/<model\>\-b4096\-report\-v2\.jsonfiles, re\-evaluated from the same cascade and router outputs, add the post\-hoccombinedpolicy of Section[5\.7](https://arxiv.org/html/2610.37732#S5.SS7)\(a logistic regression on the logits ofconfidence\_rotandrouter\_zero\_shot, fit on dev with five group folds\) and reproduce every other field of the release reports exactly; the figures and numbers in this paper are generated from them\.

The JevBench check \(Section[5\.9](https://arxiv.org/html/2610.37732#S5.SS9)\) reads JevBench at commit 7625ecb1b075cfe4d09d34141d353325825eea1d:benchmarks/export\_jevbench\.pyturns its public items into decisions \(IDs inbenchmarks/manifests/jevbench\-public\.jsonl\), the decisions run throughscripts/run\_model\.shunchanged, andbenchmarks/evaluate\_jevbench\.pywritesresults/jevbench\-<model\>\-report\-v2\.jsonfrom each model’s release report and JevBench’sresults/v1\.2/jevbench\-v1\.2\-per\-task\.json\. As a pipeline check, JevBench’s owntypesafeadapter was pointed atthinkjev\-serveon five items, and the served System 1 probabilities matched the batch run to within10−610^\{\-6\}\.