PACT：面向单令牌类型化决策的配对锚定校准微调

# Pairwise-Anchored Calibrated Tuningfor Single-Token Typed Decisions
Source: [https://arxiv.org/html/2609.35865](https://arxiv.org/html/2609.35865)
Yida Lin

###### Abstract

Single\-token typed\-decision models answer a schema question by reading the logits of a few one\-letter answer codes at a single position: they are fast and return a probability for every allowed answer, but they are trained with plain cross\-entropy that ignores most of the structure in their training data\. We study such a model whose data is curated as*contrastive pairs*—two contexts that differ in one edited fact that flips the answer—each carrying a machine\-checked certificate that deleting the decisive sentence makes the fact unknown\. We proposePACT, which turns this structure into four training terms that need no new annotation: a difference\-in\-differences margin over each pair that is invariant to any shared logit offset, a permutation\-consistency term against answer\-code position bias, an evidence\-necessity term on certificate\-verified ablated contexts, and an ordinal transport cost for rubric fields, plus a three\-parameter contextual temperature\. On a frozen 324\-item holdout with three seeds, PACT matches the published recipe in accuracy \(84\.6 vs\. 85\.2%; McNemarp≥0\.50p\\geq 0\.50at every seed\) while giving the lowest position bias of all runs \(answer flips under relabelling 9\.8 vs\. 13\.8%\) and the lowest ordinal error on rubric fields \(MAE 0\.232 vs\. 0\.311\)\. Against a control with the same optimiser and schedule but cross\-entropy only, PACT is significantly more accurate at two of three seeds, halves the seed\-to\-seed spread and lowers NLL by 26%\. Seed\-matched ablations and pre\-specified falsification tests locate these gains precisely: no single term raises raw accuracy, and the method’s value lies in robustness and stability rather than headline accuracy\. Code, data splits, trained adapters and all run records are available at[https://github\.com/BennyLinntu/PACT\-Pairwise\-Anchored\-Calibrated\-Tuning\-for\-Single\-Token\-Typed\-Decisions](https://github.com/BennyLinntu/PACT-Pairwise-Anchored-Calibrated-Tuning-for-Single-Token-Typed-Decisions)\.

###### Index Terms:

large language models, typed decisions, contrastive data, counterfactual supervision, position bias, calibration, parameter\-efficient fine\-tuning

## IIntroduction

A*typed decision*is a classification whose label space is supplied at inference time by a schema: pick one of five outcomes, answer a true/false question, or place a case on a five\-level rubric\. Such decisions can be served efficiently by rendering the context and schema once, assigning each allowed answer a one\-token code, reading the logits of those codes at one position, and normalising over them\[[1](https://arxiv.org/html/2609.35865#bib.bib2),[2](https://arxiv.org/html/2609.35865#bib.bib1)\]\. There is no generated JSON to parse and no sampling, and every allowed answer receives a probability\.

The training side of this interface is less settled\. The open Nimble recipe\[[2](https://arxiv.org/html/2609.35865#bib.bib1)\]fine\-tunes a 9B\-parameter base model with LoRA\[[3](https://arxiv.org/html/2609.35865#bib.bib26)\]on 2,676 curated examples using cross\-entropy over the candidate logits\. Its curation, however, is far richer than its objective\. Every example belongs to a*contrastive pair*produced by editing at most eight words of one evidence sentence so that the reference answer flips, and every pair carries a certificate that records the decisive fact and verifies that removing either focus sentence leaves that fact*unknown*\. The objective sees each example on its own: the pairing, the certificate and the arbitrariness of the letter assigned to each answer never enter the loss\.

This paper asks what a training objective looks like when it does use that structure\. Our premise is that the most informative supervision in contrastive data lies not in either example but in the*difference*between them—and in the counterfactual absence of the evidence—neither of which cross\-entropy can express\. We make three contributions\.

1. 1\.A structure\-derived objective \(PACT\)\.Four terms that use only structure already present in the data \(Section[IV](https://arxiv.org/html/2609.35865#S4)\): a difference\-in\-differences margin whose value is provably unchanged by any logit offset the pair shares \(Proposition[1](https://arxiv.org/html/2609.35865#Thmproposition1)\); a Jensen–Shannon consistency term across answer\-code relabellings; an evidence\-necessity term on certificate\-verified ablated contexts, which the released recipe discards; and a squared earth\-mover cost for ordinal rubric fields\. A three\-parameter contextual temperature conditions calibration on the number of choices and the prompt length\. Serving prompt, answer codes and adapter format are unchanged, so PACT is a drop\-in replacement\.
2. 2\.A controlled, falsifiable evaluation\.We reproduce the published recipe inside one harness, add a control that shares PACT’s optimiser but not its loss terms, run every leave\-one\-out ablation, state in advance which observation would falsify each claim \(Section[V\-E](https://arxiv.org/html/2609.35865#S5.SS5)\), and report paired significance tests rather than leaderboard deltas\.
3. 3\.An honest account of what the terms buy\.PACT does not raise accuracy over the published recipe\. It does produce the lowest position bias and ordinal error of all runs, stabilises an aggressive optimiser that on its own is significantly worse, and makes calibration a cheap post\-hoc step \(Section[VI](https://arxiv.org/html/2609.35865#S6)\); the ablations show which terms carry which effect and where the method does not help\.

Fig\.[1](https://arxiv.org/html/2609.35865#S1.F1)summarises the method\.

\(1\) Views built from one curated pairxax\_\{a\}base:context containing evidence sentenceee; referenceyay\_\{a\}xbx\_\{b\}counterfactual:eeedited toe′e^\{\\prime\}\(≤\\leq8 words\); referenceyb≠yay\_\{b\}\\neq y\_\{a\}xaπx\_\{a\}^\{\\pi\}relabelled:same text, answer codes permuted \(A,B,C→\\toC,A,B\)x∘x^\{\\circ\}ablated:focus sentence deleted; certificate says fact is*unknown*\(2\) Shared scorerBase LLM \+ LoRArender context \+ schema,read theCCcode\-tokenlogits at one positioncanonical logitsz⁡\(x\)∈ℝCz\(x\)\\in\\mathbb\{R\}^\{C\}code mapκ\\kappaundone\(3\) Training objectiveℒCE\\mathcal\{L\}\_\{\\mathrm\{CE\}\}:−log⁡py\-\\log p\_\{y\}onxax\_\{a\}andxbx\_\{b\}ℒCF=softplus⁡\(m−d\)\\mathcal\{L\}\_\{\\mathrm\{CF\}\}=\\mathrm\{softplus\}\(m\-d\),dd: log\-odds shift over the pairℒPC=JSD\(p\(xa\)∥p\(xaπ\)\)\\mathcal\{L\}\_\{\\mathrm\{PC\}\}=\\mathrm\{JSD\}\\big\(p\(x\_\{a\}\)\\,\\\|\\,p\(x\_\{a\}^\{\\pi\}\)\\big\)ℒNR=\(zya​\(x∘\)−zyb​\(x∘\)\)2\\mathcal\{L\}\_\{\\mathrm\{NR\}\}=\\big\(z\_\{y\_\{a\}\}\(x^\{\\circ\}\)\-z\_\{y\_\{b\}\}\(x^\{\\circ\}\)\\big\)^\{2\}ℒEMD\\mathcal\{L\}\_\{\\mathrm\{EMD\}\}: squared CDF distance on Score fields\(4\) ServingContextual temperatureT⁡\(x\)=softplus⁡\(a\+b​log⁡CCLOSET\(x\)=\\mathrm\{softplus\}\\big\(a\+b\\log COPEN\+c​log⁡\(L/1000\)\)\+\\,c\\log\(L/1000\)\\big\)3 parameters, fitted onheld\-out training familiesSingle\-pass decisionp=softmax⁡\(z/T⁡\(x\)\)p=\\mathrm\{softmax\}\\big\(z/T\(x\)\\big\)prompt, codes, adapter unchangedafter training, the same scorer is served

Fig\. 1:PACT at a glance\.\(1\) Each curated pair yields two labelled views \(base, counterfactual\), a copy of the base with its answer codes relabelled, and a certificate\-verified ablated context with the decisive sentence removed\. \(2\) All views share one forward pass per view through the LoRA\-adapted model and are mapped to canonical answer order\. \(3\) Four terms are added to cross\-entropy, each using structure the released recipe discards\. \(4\) A three\-parameter temperature, fitted on training\-side families, calibrates the served single\-pass distribution\. The colours of the views and loss terms are used consistently throughout the paper\.
## IIRelated Work

Serving typed decisions\.Scoring candidate\-token logits at a single position instead of generating and parsing structured text underlies the single\-token decision interface described for this model family\[[1](https://arxiv.org/html/2609.35865#bib.bib2),[2](https://arxiv.org/html/2609.35865#bib.bib1)\]; MiniCheck\[[4](https://arxiv.org/html/2609.35865#bib.bib4)\]applies the same idea to one Boolean grounding decision\. These works focus on the serving interface and data; we keep both fixed and change only the objective\.

Counterfactual and contrastive supervision\.Counterfactually augmented data\[[5](https://arxiv.org/html/2609.35865#bib.bib5)\]shows that minimal label\-flipping edits reduce reliance on spurious features, and contrast sets\[[6](https://arxiv.org/html/2609.35865#bib.bib6)\]use the same construction for evaluation\. Both usually feed edited examples to an unchanged objective\. Teney et al\.\[[7](https://arxiv.org/html/2609.35865#bib.bib7)\]supervise the input\-gradient direction between counterfactual pairs; our margin instead supervises the*output*geometry of a pair and is exactly invariant to a shared offset, in the spirit of difference\-in\-differences estimation\[[8](https://arxiv.org/html/2609.35865#bib.bib9)\]and pairwise ranking losses\[[9](https://arxiv.org/html/2609.35865#bib.bib10)\]\. Evidence removal is standard for*evaluating*rationales\[[10](https://arxiv.org/html/2609.35865#bib.bib8)\]; we use certified removals as a*training*constraint\.

Option\-order and label bias\.LLM multiple\-choice answers depend on option order and label identity\[[11](https://arxiv.org/html/2609.35865#bib.bib11),[12](https://arxiv.org/html/2609.35865#bib.bib12),[13](https://arxiv.org/html/2609.35865#bib.bib13)\], and on label priors more generally\[[14](https://arxiv.org/html/2609.35865#bib.bib14)\]\. Debiasing is typically applied at inference by permuting and averaging or by re\-normalising\. We additionally train against the bias with a bounded Jensen–Shannon consistency loss\[[15](https://arxiv.org/html/2609.35865#bib.bib16)\], analogous to the consistency term of AugMix\[[16](https://arxiv.org/html/2609.35865#bib.bib15)\], which keeps single\-pass serving—the reason this model family exists—sound\.

Calibration\.Temperature scaling\[[17](https://arxiv.org/html/2609.35865#bib.bib17)\]is the standard post\-hoc method, with richer maps such as Dirichlet calibration\[[18](https://arxiv.org/html/2609.35865#bib.bib19)\]; LLM confidence is known to be miscalibrated in task\-dependent ways\[[19](https://arxiv.org/html/2609.35865#bib.bib20),[20](https://arxiv.org/html/2609.35865#bib.bib21)\]\. We keep the temperature\-scaling objective but let the temperature depend on two observable prompt statistics, which preserves every argmax\.

Ordinal targets and efficient tuning\.Ordered labels benefit from losses that respect the order\[[21](https://arxiv.org/html/2609.35865#bib.bib25)\]; squared earth\-mover losses\[[22](https://arxiv.org/html/2609.35865#bib.bib24)\]are one such choice, which we apply to rubric fields whose expected level the serving path already reports\. The adapter uses LoRA\[[3](https://arxiv.org/html/2609.35865#bib.bib26)\]with rank\-stabilised scaling\[[23](https://arxiv.org/html/2609.35865#bib.bib28)\]and LoRA\+ learning\-rate ratios\[[24](https://arxiv.org/html/2609.35865#bib.bib27)\]; a QLoRA\[[25](https://arxiv.org/html/2609.35865#bib.bib29)\]configuration is provided for 24 GB GPUs\.

## IIIProblem Setting

### III\-ASingle\-token typed decisions

A field has allowed answersv1,…,vCv\_\{1\},\\dots,v\_\{C\}with1≤C≤261\\leq C\\leq 26\. The prompt renders the context, the schema and the requested field, and shows answervkv\_\{k\}under a code letter\. Withhhthe final\-position hidden state andtkt\_\{k\}the token of thekk\-th code, the model produceszk=⟨wtk,h⟩z\_\{k\}=\\langle w\_\{t\_\{k\}\},h\\rangleand servesp=softmax⁡\(z1,…,zC\)p=\\mathrm\{softmax\}\(z\_\{1\},\\dots,z\_\{C\}\)\. Fields are of three kinds:*Choice*\(an enumeration\),*Boolean*\(true/false\) and*Score*\(an ordered rubric with levels0,…,C−10,\\dots,C\-1\)\. Two properties matter below\. First, the map from a*meaning*to a*letter*is arbitrary, so any dependence ofppon the letter is error\. Second, since one position is scored, the decision is a linear read\-out of one hidden state, which makes the geometry ofzz—differences between candidate logits—the natural object to supervise\.

### III\-BContrastive curation with certificates

The training file contains 1,338 families of exactly two records: a base context and a counterfactual that differs in one edited evidence sentence, with the reference answer flipped and a byte\-identical question and criteria block\. Each record carries a certificate naming the two sentences that jointly determine the focus fact, their base and edited wording, and the verified statement that with either focus sentence removed the focus fact becomesunknown\. The released recipe trains on the two records independently and does not construct the removed\-evidence contexts, correctly noting that “missing evidence” has no reference label\. Our observation is that the absence of a label is not the absence of information: the certificate establishes*indifference*, which is a constraint on the logit geometry rather than a target\. Fig\.[2](https://arxiv.org/html/2609.35865#S3.F2)shows a real pair from the training file and the three views PACT derives from it\.

Context: seven turns, identical in both members except turn 61 Account reviewer:a $24 retention offer is recorded and valid for the disputed Sept\. 14 annual renewal\.2 Communications reviewer:before renewal, the subscriber accepted the offer on condition that the credit be applied\.3 Invoice reviewer:invoice INV\-8841 omitted the $24 credit\.4 Subscriber request:if the credit is ineligible, cancel the annual plan, move to monthly, refund the unused term\.5 Audit reviewer\(focus\): “Audit record AR\-7319 states that its field E records the number of calendar days between the subscriber’s disputed September 14 annual renewal and the correction request in the current contact\.”6 Audit record\(focus, edited\): “Field E in audit record AR\-7319 contains the value6/9\.”7 Payment reviewer:exactly one settled charge for the renewal; no second charge\.Choice field“Choose the authorized resolution” \(policy: credit only if the correction request is≤\\leq7 days after renewal\): A apply\_credit\_keep\_annual B execute\_fallback\_cancellation C none\_of\_above D route\_duplicate\_payment\_review E seek\_intent\_clarificationxax\_\{a\}\(turn 6 = 6 days\):focus fact*request within 7 days*issupported⇒\\Rightarrowyay\_\{a\}= Axbx\_\{b\}\(turn 6 = 9 days\):focus factrefuted⇒\\Rightarrowyby\_\{b\}= B\(one word edited\)x∘x^\{\\circ\}\(turn 6 deleted\):certificate verifies the focus fact becomesunknown⇒\\RightarrowℒNR\\mathcal\{L\}\_\{\\mathrm\{NR\}\}askszA​\(x∘\)≈zB​\(x∘\)z\_\{A\}\(x^\{\\circ\}\)\\approx z\_\{B\}\(x^\{\\circ\}\)and says nothing about C, D, E, which the other turns still constrain \(e\.g\. turn 7 excludes D\)\.

Fig\. 2:A curated contrastive pair from the training file\(familyscale\-diverse\-013\-002, commerce domain; turns 1–4 and 7 abridged, focus sentences verbatim\)\. Changing one token of the second focus sentence \(6→\\to9 days\) flips the certified focus fact and the reference answer from A to B; deleting that sentence makes the fact*unknown*\. Cross\-entropy seesxax\_\{a\}andxbx\_\{b\}as two unrelated examples; PACT uses the pair \(ℒCF\\mathcal\{L\}\_\{\\mathrm\{CF\}\}\), a relabelled copy \(ℒPC\\mathcal\{L\}\_\{\\mathrm\{PC\}\}\) and the ablated context \(ℒNR\\mathcal\{L\}\_\{\\mathrm\{NR\}\}\)\.
### III\-CNotation

For an examplexxwithCCallowed answers,z⁡\(x\)∈ℝCz\(x\)\\in\\mathbb\{R\}^\{C\}denotes the candidate logits in*canonical*order \(the order in which the schema declares the answers\) andp⁡\(x\)p\(x\)the corresponding distribution\. A pair is\(xa,xb\)\(x\_\{a\},x\_\{b\}\)with referencesya≠yby\_\{a\}\\neq y\_\{b\}from the same canonical set;x∘x^\{\\circ\}isxax\_\{a\}with one focus evidence sentence deleted;π\\piis a permutation of code positions andzπz^\{\\pi\}the canonical logits obtained when the prompt is rendered underπ\\pi\.

## IVMethod

### IV\-ACanonical choice space

Every loss and metric is computed after mapping code positions back to canonical answer indices\. Per view, the encoder records a vectorκ\\kappawithκk\\kappa\_\{k\}the canonical index of the answer shown at code positionkk; logits are scattered from code space into canonical space and unused positions are masked to−∞\-\\infty\. This makes the pair terms well defined when the two members are rendered under different permutations, and makes “the model changed its answer when the letters changed” a measurable event\.

### IV\-BCounterfactual difference\-in\-differences margin

For a pair define

d⁡\(xa,xb\)=\[zya​\(xa\)−zyb​\(xa\)\]\+\[zyb​\(xb\)−zya​\(xb\)\],d\(x\_\{a\},x\_\{b\}\)=\\big\[z\_\{y\_\{a\}\}\(x\_\{a\}\)\-z\_\{y\_\{b\}\}\(x\_\{a\}\)\\big\]\+\\big\[z\_\{y\_\{b\}\}\(x\_\{b\}\)\-z\_\{y\_\{a\}\}\(x\_\{b\}\)\\big\],\(1\)i\.e\. the change of theya:yby\_\{a\}\{:\}y\_\{b\}log\-odds between the two members, and penaliseℒCF=softplus⁡\(m−d\)\\mathcal\{L\}\_\{\\mathrm\{CF\}\}=\\mathrm\{softplus\}\(m\-d\)\.

###### Proposition 1\(Offset invariance\)

For anyc∈ℝCc\\in\\mathbb\{R\}^\{C\}, replacingz⁡\(xa\)z\(x\_\{a\}\)byz⁡\(xa\)\+cz\(x\_\{a\}\)\+candz⁡\(xb\)z\(x\_\{b\}\)byz⁡\(xb\)\+cz\(x\_\{b\}\)\+cleavesddunchanged\. Moreover, if both members are correct under the binary restriction to\{ya,yb\}\\\{y\_\{a\},y\_\{b\}\\\}, thend\>0d\>0\.

###### Proof:

Each bracket in \([1](https://arxiv.org/html/2609.35865#S4.E1)\) is a difference of two logits of the same input, socya−cybc\_\{y\_\{a\}\}\-c\_\{y\_\{b\}\}enters the first bracket andcyb−cyac\_\{y\_\{b\}\}\-c\_\{y\_\{a\}\}the second; they cancel\. Correctness ofxax\_\{a\}\(xbx\_\{b\}\) under the restriction means the first \(second\) bracket is positive\. ∎

A per\-pair offsetccis exactly what a model learns when it acquires a prior over outcomes from the topic, the policy text or the family’s surface form\.ℒCF\\mathcal\{L\}\_\{\\mathrm\{CF\}\}therefore cannot be reduced by such a prior, only by making the logit gap move in the right direction when the one edited fact moves\. Cross\-entropy lacks this property: a model that assigns 0\.51 toyay\_\{a\}on the base and 0\.49 on the counterfactual satisfies both cross\-entropy terms nearly as well as a model with a decisive, evidence\-driven gap\. Becauseddcouples both members, it can only be evaluated when both are in the same micro\-batch \(Section[IV\-G](https://arxiv.org/html/2609.35865#S4.SS7)\)\. We usem=2m=2nats and a softplus hinge so that gradients vanish smoothly once the pair is separated\.

### IV\-CPermutation consistency

For two code permutationsπ,π′\\pi,\\pi^\{\\prime\}of the same example we penalise

ℒPC=JSD\(pπ∥pπ′\)=12KL\(pπ∥p¯\)\+12KL\(pπ′∥p¯\),\\mathcal\{L\}\_\{\\mathrm\{PC\}\}=\\mathrm\{JSD\}\\big\(p^\{\\pi\}\\,\\\|\\,p^\{\\pi^\{\\prime\}\}\\big\)=\\tfrac\{1\}\{2\}\\mathrm\{KL\}\(p^\{\\pi\}\\\|\\bar\{p\}\)\+\\tfrac\{1\}\{2\}\\mathrm\{KL\}\(p^\{\\pi^\{\\prime\}\}\\\|\\bar\{p\}\),\(2\)withp¯=12​\(pπ\+pπ′\)\\bar\{p\}=\\tfrac\{1\}\{2\}\(p^\{\\pi\}\+p^\{\\pi^\{\\prime\}\}\)\. JSD is bounded and symmetric, so no view is privileged as a teacher and a confidently wrong view cannot produce an unbounded gradient; the term is label\-free\. At inference the same invariance supports aKK\-view ensemble that averages canonical log\-probabilities\. We treat the gain fromK=1K\{=\}1toK=4K\{=\}4as a measure of the bias that training failed to remove: a perfectly permutation\-invariant model gains nothing\.

### IV\-DEvidence\-necessity regularisation

The certificate states that inx∘x^\{\\circ\}the focus fact is unknown given all remaining text, and the decision rules that separateyay\_\{a\}fromyby\_\{b\}take the focus fact as input\. Hence the evidence inx∘x^\{\\circ\}does not discriminateyay\_\{a\}fromyby\_\{b\}—although it may still rule out other answers\. We encode exactly that and nothing more:

ℒNR=\(relu⁡\(\|zya​\(x∘\)−zyb​\(x∘\)\|−ε\)\)2\.\\mathcal\{L\}\_\{\\mathrm\{NR\}\}=\\Big\(\\mathrm\{relu\}\\big\(\|z\_\{y\_\{a\}\}\(x^\{\\circ\}\)\-z\_\{y\_\{b\}\}\(x^\{\\circ\}\)\|\-\\varepsilon\\big\)\\Big\)^\{2\}\.\(3\)The term is silent aboutzj​\(x∘\)z\_\{j\}\(x^\{\\circ\}\)forj∉\{ya,yb\}j\\notin\\\{y\_\{a\},y\_\{b\}\\\}, about the entropy ofp⁡\(x∘\)p\(x^\{\\circ\}\), and about which ofya,yby\_\{a\},y\_\{b\}is “correct”—claims the certificate does not support\. To buildx∘x^\{\\circ\}we locate the focus sentence by certificate path, then exact leaf match, then substring, then the counterfactual wording, and delete the whole conversational turn if it carries nothing else\. On the released training file this succeeds for all 2,676 records, verified by checking that the sentence no longer occurs anywhere in the serialised context\. We useε=0\\varepsilon=0\.

### IV\-EOrdinal transport for rubric fields

Score fields expose levels0,…,C−10,\\dots,C\-1and the serving code reports the expected level∑jj​pj\\sum\_\{j\}j\\,p\_\{j\}\. Cross\-entropy is invariant to the ordering of the levels, so nothing in training prefers a near miss to a far one\. We add the squared earth\-mover distance between predicted and reference CDFs,

ℒEMD=∑j=0C−1\(∑i≤jpi−𝟏\[j≥y\]\)2,\\mathcal\{L\}\_\{\\mathrm\{EMD\}\}=\\sum\_\{j=0\}^\{C\-1\}\\Big\(\\sum\_\{i\\leq j\}p\_\{i\}\-\\mathbf\{1\}\[\\,j\\geq y\\,\]\\Big\)^\{2\},\(4\)which is differentiable and minimised by mass concentrated*near*yyrather than merely*on*yy\.

### IV\-FContextual temperature

The released model ships without calibration\. A single global temperature has the wrong shape here: a two\-choice Boolean and a six\-choice enumeration do not share a miscalibration factor, and neither do a 300\-token and a 1,100\-token prompt\. We fit

T⁡\(x\)=softplus⁡\(a\+b​log⁡C\+c​log⁡\(L/1000\)\),T\(x\)=\\mathrm\{softplus\}\\big\(a\+b\\log C\+c\\log\(L/1000\)\\big\),\(5\)withLLthe prompt length in tokens, by minimising NLL with L\-BFGS on a*calibration split*of training\-side families never used to fit the adapter\. Withb=c=0b=c=0this is standard temperature scaling\[[17](https://arxiv.org/html/2609.35865#bib.bib17)\]\. BecauseT⁡\(x\)\>0T\(x\)\>0rescales each item’s logits by a positive constant, it never changes an argmax, so accuracy and pair accuracy are untouched; it can, however, change the*ranking*of confidences across items and hence selective risk\.

### IV\-GObjective, batching and cost

The full objective is

ℒ=ℒCE\+λcf​ℒCF\+λpc​ℒPC\+λnr​ℒNR\+λemd​ℒEMD,\\mathcal\{L\}=\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\+\\lambda\_\{\\mathrm\{cf\}\}\\mathcal\{L\}\_\{\\mathrm\{CF\}\}\+\\lambda\_\{\\mathrm\{pc\}\}\\mathcal\{L\}\_\{\\mathrm\{PC\}\}\+\\lambda\_\{\\mathrm\{nr\}\}\\mathcal\{L\}\_\{\\mathrm\{NR\}\}\+\\lambda\_\{\\mathrm\{emd\}\}\\mathcal\{L\}\_\{\\mathrm\{EMD\}\},\(6\)with\(λcf,λpc,λnr,λemd\)=\(0\.5,0\.5,0\.2,0\.3\)\(\\lambda\_\{\\mathrm\{cf\}\},\\lambda\_\{\\mathrm\{pc\}\},\\lambda\_\{\\mathrm\{nr\}\},\\lambda\_\{\\mathrm\{emd\}\}\)=\(0\.5,0\.5,0\.2,0\.3\), chosen for scale comparability and*not*tuned on any evaluation data\. The regularisers ramp in linearly over the first 10% of steps\. SinceℒCF\\mathcal\{L\}\_\{\\mathrm\{CF\}\}needs both pair members andℒPC\\mathcal\{L\}\_\{\\mathrm\{PC\}\}needs both views, the unit of batching is a*group*: one family with all of its views \(Algorithm[1](https://arxiv.org/html/2609.35865#alg1)\)\. Groups are shuffled, sorted by length inside buckets of 64 groups \(prompt lengths vary by more than 3×\\times, so padding dominates waste\) and cut into micro\-batches\. Each pair contributes two primary, two permuted and one ablated view, i\.e\. 2\.5×\\timesthe rows of the baseline\.

Algorithm 1One PACT optimiser step1:groups

𝒢\\mathcal\{G\}\(pair

\(xa,xb\)\(x\_\{a\},x\_\{b\}\), permuted

xaπ,xbπx\_\{a\}^\{\\pi\},x\_\{b\}^\{\\pi\}, ablated

x∘x^\{\\circ\}\); ramp

rt∈\[0,1\]r\_\{t\}\\in\[0,1\]
2:foreach micro\-batch

B⊂𝒢B\\subset\\mathcal\{G\}\(length\-bucketed\)do

3:one forward pass over all views in

BB; read code logits

4:scatter to canonical order with

κ\\kappa; mask unused slots

5:

ℓ←ℒCE\+rt​λemd​ℒEMD\\ell\\leftarrow\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\+r\_\{t\}\\lambda\_\{\\mathrm\{emd\}\}\\mathcal\{L\}\_\{\\mathrm\{EMD\}\}⊳\\trianglerightlabelled views

6:

ℓ←ℓ\+rt​\(λcf​ℒCF\+λpc​ℒPC\+λnr​ℒNR\)\\ell\\leftarrow\\ell\+r\_\{t\}\\big\(\\lambda\_\{\\mathrm\{cf\}\}\\mathcal\{L\}\_\{\\mathrm\{CF\}\}\+\\lambda\_\{\\mathrm\{pc\}\}\\mathcal\{L\}\_\{\\mathrm\{PC\}\}\+\\lambda\_\{\\mathrm\{nr\}\}\\mathcal\{L\}\_\{\\mathrm\{NR\}\}\\big\)
7:accumulate

∇ℓ\\nabla\\ell
8:endfor

9:clip global norm to 1; AdamW step on LoRA

AA\(

η\\eta\) and

BB\(

8​η8\\eta\)

## VExperimental Setup

### V\-AData and splits

Training uses the released training file \(2,676 records, 1,338 pairs, 34 source families, 10 domains\)\. Four source families are held out from training: two form the*selection split*used for checkpoint selection and two the*calibration split*used to fitT⁡\(x\)T\(x\)\(Table[I](https://arxiv.org/html/2609.35865#S5.T1)\)\. The frozen*holdout*\(324 records, 162 pairs, 6 source families from 6 domains\) is used only to report final numbers—never for selection, calibration or early stopping\. Family\-level disjointness and the file hashes of the dataset manifest are re\-verified at load time\. Holdout fields haveC∈\{2,…,6\}C\\in\\\{2,\\dots,6\\\}allowed answers, and the longest rendered prompt has 1,098 tokens\.

TABLE I:Data splits\. Splits are disjoint at the level of source families; every record belongs to a two\-record contrastive pair\. The holdout is never used for selection or calibration\.
### V\-BModel and training

The base model is Qwen3\.5\-9B\[[26](https://arxiv.org/html/2609.35865#bib.bib3)\]at the revision pinned by the published recipe, in BF16 with a 2,048\-token limit\. The adapter is LoRA of rank 16 \(α=32\\alpha\{=\}32, dropout 0\.05\) on all attention and MLP projections of the language model—the modules the published recipe adapts—with rsLoRA scaling\. We train with AdamW\[[27](https://arxiv.org/html/2609.35865#bib.bib30)\]atη=5×10−5\\eta=5\\times 10^\{\-5\}, LoRA\+ ratio 8, cosine schedule with 10% warm\-up, two epochs \(584 steps\) and an effective batch of 8 primary rows \(4 pairs\) per step, matching the published recipe\. The selection split is scored every 146 steps and the checkpoint with the lowest selection NLL is kept\. Jobs ran one per GPU on two 48 GB GPUs\.

### V\-CCompared runs

Table[II](https://arxiv.org/html/2609.35865#S5.T2)lists the runs\. Nimble recipe reproduces the published recipe inside our harness\. CE\-only shares*everything*with PACT except the four terms and the extra views, so PACT vs\. CE\-only isolates the objective, while Nimble recipe vs\. CE\-only isolates the optimisation changes\. The three main arms use seeds 17, 18 and 19; each leave\-one\-out ablation uses seed 17 and is compared against PACT at the*same*seed \(Section[VI\-D](https://arxiv.org/html/2609.35865#S6.SS4)\)\.

TABLE II:Compared runs\. All share data, prompt, adapter family and checkpoint selection\. “Opt\. A”: 1 epoch, linear schedule, plain LoRA \(published\)\. “Opt\. B”: 2 epochs, cosine, rsLoRA \+ LoRA\+\.RunObjectiveViews‡Opt\.SeedsBase model– \(adapter off\)–––Nimble recipeℒCE\\mathcal\{L\}\_\{\\mathrm\{CE\}\}2/0/0A3CE\-onlyℒCE\\mathcal\{L\}\_\{\\mathrm\{CE\}\}2/0/0B3PACTEq\. \([6](https://arxiv.org/html/2609.35865#S4.E6)\)2/2/1B3−ℒCF\-\\mathcal\{L\}\_\{\\mathrm\{CF\}\},−ℒNR\-\\mathcal\{L\}\_\{\\mathrm\{NR\}\},−ℒEMD\-\\mathcal\{L\}\_\{\\mathrm\{EMD\}\}PACT minus one term2/2/1B1−ℒPC\-\\mathcal\{L\}\_\{\\mathrm\{PC\}\}PACT minus JSD2/2∗/1B1−\-permuted viewsPACT withoutℒPC\\mathcal\{L\}\_\{\\mathrm\{PC\}\}2/0/1B1
‡Primary / permuted / ablated views per pair\.∗Permuted views kept as plain cross\-entropy augmentation\.

### V\-DMetrics and statistics

We report accuracy;pair accuracy, the share of the 162 holdout pairs with*both*members correct; NLL; multiclass Brier score\[[28](https://arxiv.org/html/2609.35865#bib.bib22)\]; ECE over 15 equal\-width bins\[[29](https://arxiv.org/html/2609.35865#bib.bib18)\]; the area under the confidence\-ranked risk–coverage curve \(AURC\)\[[30](https://arxiv.org/html/2609.35865#bib.bib23)\]; expected\-level MAE on Score fields; and, forK=4K\{=\}4decoding, the mean pairwisetotal variation \(TV\)between the canonical distributions of different code orderings and theflip rate, the share of items whose argmax changes across orderings\. Because all runs are scored on the same items, we compare accuracies with the exact McNemar test\[[31](https://arxiv.org/html/2609.35865#bib.bib31)\]and a 10,000\-resample paired bootstrap\[[32](https://arxiv.org/html/2609.35865#bib.bib32)\], per seed\.

### V\-EPre\-specified falsification criteria

The following were written before the runs, so the ablations read as tests rather than a search over cells\.

- H1*Pair margin\.*RemovingℒCF\\mathcal\{L\}\_\{\\mathrm\{CF\}\}lowers pair accuracy by more than it lowers accuracy\.
- H2*Permutation consistency\.*PACT has lower TV and flip rate than CE\-only and gains less fromK=4K\{=\}4ensembling\.
- H3*Necessity\.*WithoutℒNR\\mathcal\{L\}\_\{\\mathrm\{NR\}\}the model is more confident on ablated contexts and has no better pair accuracy\.
- H4*Ordinal transport\.*WithoutℒEMD\\mathcal\{L\}\_\{\\mathrm\{EMD\}\}, Score MAE is higher at similar Score accuracy\.
- H5*Calibration\.*Holdout ECE falls after applyingT⁡\(x\)T\(x\)without a loss of accuracy\.

## VIResults

TABLE III:Holdout results, single\-pass decoding \(K=1K\{=\}1\), mean±\\pms\.d\. over seeds 17/18/19 \(324 items, 162 pairs\)\. Accuracy, pair accuracy, ECE and AURC in %\. ECETis after the contextual temperatureT⁡\(x\)T\(x\)\(not fitted for the base model\)\.Shaded: best mean per column\.Fig\. 3:Holdout comparison of the three multi\-seed arms and the base model\. Bars: mean; dots: individual seeds; whiskers:±\\pm1 s\.d\. \(a–c\) single\-pass decoding; in \(c\) hollow diamonds give ECE after the contextual temperature\. \(d\) Permutation TV underK=4K\{=\}4relabellings \(lower is less position bias\)\.Fig\. 4:Position bias\. \(a\) Distribution of per\-item permutation TV for the shipped PACT model underK=4K\{=\}4\(log scale\)\. \(b, c\) Local 4\-bit sweep over the ensemble sizeKKfor the shipped model and the base model: accuracy \(b\) and mean pairwise TV between orderings \(c\)\.Fig\. 5:Calibration\. \(a\) Reliability diagram of the shipped model before and afterT⁡\(x\)T\(x\); marker area grows with bin size\. \(b\) Risk–coverage curves, items ranked by confidence\. \(c\) Holdout ECE before \(hollow\) and after \(filled\)T⁡\(x\)T\(x\)for every run and seed\.### VI\-AMain comparison

Table[III](https://arxiv.org/html/2609.35865#S6.T3)and Fig\.[3](https://arxiv.org/html/2609.35865#S6.F3)give the headline numbers, and Table[IV](https://arxiv.org/html/2609.35865#S6.T4)the paired tests\. Three findings stand out\.

*PACT matches, but does not beat, the published recipe in accuracy\.*PACT reaches 84\.6±\\pm2\.8% against 85\.2±\\pm2\.1% for Nimble recipe\. At every seed the difference is inside the bootstrap interval and McNemarp≥0\.50p\\geq 0\.50; at two seeds the accuracies are identical\. Properly reproduced with checkpoint selection, the published recipe is a strong accuracy baseline, and it is also the best\-calibrated run \(ECET5\.4% vs\. 7\.9% for PACT\)\.

*The optimisation changes alone hurt; the PACT terms repair them\.*CE\-only—PACT’s optimiser and schedule without the four terms—is the weakest and least stable fine\-tuned arm \(82\.2±\\pm6\.0%, NLL 0\.760, ECE 14\.3%\)\. Adding the terms raises mean accuracy by 2\.4 pp, significantly so at seeds 18 and 19 \(p=0\.008p=0\.008and0\.0350\.035; seed 17 favours CE\-only,p=0\.099p=0\.099\), halves the seed\-to\-seed standard deviation \(6\.0→\\to2\.8 pp\) and lowers NLL by 26% and ECE by 25%\. The terms thus act as regularisers that make a more aggressive optimiser safe, rather than as a source of accuracy beyond a well\-tuned baseline\.

*Where PACT does lead, the lead matches the structure it encodes\.*PACT has the highest mean pair accuracy \(73\.3%, not significant given the 6 pp seed spread\), the lowest Score MAE of all multi\-seed arms \(0\.232 vs\. 0\.311 and 0\.322\), and the lowest position bias \(Section[VI\-B](https://arxiv.org/html/2609.35865#S6.SS2)\)\. PACT clears the base model by 18–24 pp \(p<0\.001p<0\.001at every seed\)\.

TABLE IV:Paired tests on holdout accuracy \(%\), same 324 items\.b/cb/c: items only PACT / only the control gets right; exact two\-sided McNemarpp; 95% CI of the accuracy difference from a 10,000\-resample paired bootstrap \(pp\)\.Highlighted:p<0\.05p<0\.05\.†Base\-model predictions are from a local 4\-bit re\-evaluation \(acc\. 63\.9%\)\.TABLE V:Permutation\-ensembled decoding \(K=4K\{=\}4\), %\. TV: mean pairwise total variation between theKKcanonical distributions; Flip: share of items whose argmax changes across orderings\.ΔK\\Delta\_\{K\}: accuracy gain ofK=4K\{=\}4overK=1K\{=\}1\(pp\); a position\-invariant model should gain nothing\.TABLE VI:Seed\-matched leave\-one\-out ablations \(seed 17 for every row; single\-pass except the last two columns, which useK=4K\{=\}4\)\. Cells are coloured by the change relative to the full method in the first row \(seed 17\):removing the term clearly hurts,slightly hurts,slightly helps,clearly helps; uncoloured: change below a quarter of the “clear” threshold \(2 pp for accuracy\-type metrics\)\. One seed per row: read colours as directions, not as significance\.TABLE VII:Pre\-specified hypotheses \(Section[V\-E](https://arxiv.org/html/2609.35865#S5.SS5)\) against the evidence\. “Seed\-matched” values compare against PACT at seed 17; multi\-seed values are means over three seeds\.
### VI\-BPosition bias and permutation ensembling

Table[V](https://arxiv.org/html/2609.35865#S6.T5)reportsK=4K\{=\}4decoding\. PACT has the lowest TV \(5\.6% vs\. 6\.9% for Nimble recipe, 7\.4% for CE\-only and 12\.0% for the base model\) and the lowest flip rate \(9\.8% vs\. 13\.8%, 12\.1% and 23\.5%\)\. It is also the only arm that gains nothing from ensembling \(ΔK=−0\.7\\Delta\_\{K\}=\-0\.7pp vs\.\+2\.4\+2\.4pp for CE\-only\): its single pass already captures what the ensemble would recover, which is precisely the behaviour H2 predicts and the property that matters for single\-pass serving\.

Fig\.[4](https://arxiv.org/html/2609.35865#S6.F4)looks closer\. The bias of the shipped model is concentrated: the median item has TV of 1\.9% and 31% of items are below 1%, while a small tail of items carries TV above 30%\. A local 4\-bit sweep overK∈\{1,…,4\}K\\in\\\{1,\\dots,4\\\}\(single seed; supplementary\) shows the residual TV of PACT plateauing at 7\.3–7\.6% against 16\.7–18\.1% for the base model, 2\.2–2\.4×\\timeslower at everyKK, and accuracy peaking atK=2K\{=\}2for both models\. The non\-monotone accuracy curve is not a quantisation artefact: the BF16 multi\-seed results show the same pattern \(84\.6% atK=1K\{=\}1, 83\.8% atK=4K\{=\}4\)\.

### VI\-CCalibration

The contextual temperature lowers holdout ECE in all nine multi\-seed runs \(Fig\.[5](https://arxiv.org/html/2609.35865#S6.F5)c; per\-run values in Table[X](https://arxiv.org/html/2609.35865#A1.T10), Appendix[A](https://arxiv.org/html/2609.35865#A1)\), on average from 10\.8% to 7\.9% for PACT, from 9\.5% to 5\.4% for Nimble recipe and from 14\.3% to 10\.3% for CE\-only; accuracy is unchanged by construction\. For the shipped model the fitted slopes are negative for both the number of choices \(b=−0\.42b=\-0\.42\) and the prompt length \(c=−0\.48c=\-0\.48\), i\.e\. fields with many choices and long prompts are*under*\-confident relative to short binary ones, which a single scalar temperature cannot express \(Fig\.[6](https://arxiv.org/html/2609.35865#S6.F6)a\)\.

Three caveats are reported rather than omitted\. First, the*shape*of the fitted temperature is not stable across runs \(Fig\.[6](https://arxiv.org/html/2609.35865#S6.F6)b\): over the nine multi\-seed runs the slope onlog⁡C\\log Cranges from−1\.60\-1\.60to\+2\.73\+2\.73and the slope onlog⁡\(L/1000\)\\log\(L/1000\)from−3\.49\-3\.49to\+3\.58\+3\.58\. With 226 calibration records the two slopes are weakly identified; what transfers to the holdout is most plausibly the overall*level*of the temperature, so a scalar temperature is a reasonable default and the contextual form should be regularised or fitted on a larger split\. Second, on the calibration split of the shipped seed the fit leaves ECE essentially unchanged \(5\.5%→\\to5\.7%\); the gain appears only on the holdout\. Third, becauseT⁡\(x\)T\(x\)re\-ranks confidences across items, it slightly worsens selective risk \(AURC 3\.95→\\to4\.21 for PACT; Fig\.[5](https://arxiv.org/html/2609.35865#S6.F5)b\)\.

Fig\. 6:Fitted contextual temperature\. \(a\) Shipped model:T⁡\(x\)T\(x\)against prompt length for each number of allowed answersCCin the holdout\. \(b\)TTagainstCCatL=700L=700for every multi\-seed run: for most runsT\>1T\>1\(raw logits are over\-confident\), but the slopes differ in sign between seeds\.
### VI\-DSeed\-matched ablations

Table[VI](https://arxiv.org/html/2609.35865#S6.T6)compares each leave\-one\-out variant with PACT*at the same seed*\. Two caveats frame the reading\. Each row is a single run, and the seed\-17 PACT reference is itself a relatively weak draw \(84\.0% accuracy vs\. 84\.6% mean, and the highest ECE of its three seeds\), so colours indicate directions, not significance\. With that in mind, the pattern is consistent:

- •No single term raises raw accuracy\.RemovingℒCF\\mathcal\{L\}\_\{\\mathrm\{CF\}\}orℒEMD\\mathcal\{L\}\_\{\\mathrm\{EMD\}\}leaves accuracy unchanged \(84\.3% vs\. 84\.0%\); removingℒPC\\mathcal\{L\}\_\{\\mathrm\{PC\}\}even raises it \(87\.7%\), as does removing all four terms \(CE\-only, 87\.3% at this seed, though not at the other two\)\. Every removal also improves uncalibrated NLL and ECE, i\.e\. at this seed the regularisers trade calibration of the raw logits for other properties—a cost the post\-hoc temperature largely absorbs \(Section[VI\-C](https://arxiv.org/html/2609.35865#S6.SS3)\)\.
- •The robustness properties are carried by the terms\.RemovingℒNR\\mathcal\{L\}\_\{\\mathrm\{NR\}\},ℒEMD\\mathcal\{L\}\_\{\\mathrm\{EMD\}\}, the permuted views or all four terms raises the flip rate \(8\.0%→\\to13\.0–17\.6%\) and TV; removingℒCF\\mathcal\{L\}\_\{\\mathrm\{CF\}\},ℒNR\\mathcal\{L\}\_\{\\mathrm\{NR\}\},ℒEMD\\mathcal\{L\}\_\{\\mathrm\{EMD\}\}or the permuted views raises Score MAE \(0\.229→\\to0\.282–0\.356\)\.
- •The necessity term is the most load\-bearing\.RemovingℒNR\\mathcal\{L\}\_\{\\mathrm\{NR\}\}degrades accuracy \(−\-3\.1 pp\), pair accuracy \(−\-8\.0 pp\), AURC \(4\.44→\\to6\.63\), Score MAE and position bias \(TV 5\.0→\\to11\.7%\) at once\.

### VI\-EWhere the differences are

Pooled accuracy hides a structured, seed\-consistent pattern \(Fig\.[8](https://arxiv.org/html/2609.35865#S6.F8)\)\. Relative to Nimble recipe, PACT is better on Score fields at all three seeds \(\+6\.2 to \+15\.6 pp\) and worse on Choice \(−\-3\.4 to−\-4\.8 pp\) and Boolean fields \(−\-0\.9 to−\-4\.4 pp\) at all three seeds; per\-kind accuracies are in Table[IX](https://arxiv.org/html/2609.35865#A1.T9)\(Appendix[A](https://arxiv.org/html/2609.35865#A1)\)\. The Score advantage is what the ordinal term targets\. By domain, PACT is better on*public services*and worse on*education*and*media*at every seed\. We had no pre\-specified hypothesis about domains and report this as an observation for future work: the gains of a structure\-derived objective are heterogeneous, and a pooled accuracy delta near zero can conceal offsetting per\-kind effects of 5–15 pp\.

The same split appears in calibration \(Fig\.[7](https://arxiv.org/html/2609.35865#S6.F7)c\)\. Averaged over seeds, PACT halves the ECE of Score fields relative to Nimble recipe \(10\.8% vs\. 21\.2%; 10\.1% vs\. 16\.6% afterT⁡\(x\)T\(x\)\) but is worse on Choice fields \(16\.0% vs\. 10\.3%\), which is where its overall calibration deficit comes from\. \(Per\-kind ECE is computed on 64–146 items and is biased upwards relative to the pooled value\.\)

How pairs fail\.Fig\.[7](https://arxiv.org/html/2609.35865#S6.F7)b breaks the shipped model’s 162 holdout pairs down by outcome\. Both members are right in 67% of pairs\. When a pair is only half right, it is usually the counterfactual that is missed \(35 pairs vs\. 13 in which only the base is missed; for Boolean fields all 11 half\-right pairs miss the counterfactual\)\. Most strikingly, in 39 pairs \(24%; 33% for Choice fields\) the model gives the*same*answer to both members, i\.e\. it ignores the edited fact altogether—74% of all imperfect pairs\. This is exactly the failureℒCF\\mathcal\{L\}\_\{\\mathrm\{CF\}\}targets, and it remains the dominant error mode after training: the mean margin on training pairs ends far abovemm\(Fig\.[9](https://arxiv.org/html/2609.35865#S6.F9)b\), but the behaviour does not fully transfer to new source families\. Errors are also not always hesitant \(Fig\.[7](https://arxiv.org/html/2609.35865#S6.F7)a\): the median wrong answer has confidence 0\.74 against 0\.995 for right answers, yet 16 of the 58 errors \(28%\) are made with confidence≥0\.9\\geq 0\.9, against 56 of 117 \(48%\) for the base model\.

Fig\. 7:Error analysis on the holdout\. \(a\) Uncalibrated confidence of correct and wrong answers of the shipped model \(log scale\)\. \(b\) Outcomes of the 162 contrastive pairs by field kind \(numbers: % of pairs; right: share of pairs where both members receive the same answer, which guarantees at least one error\)\. \(c\) ECE by field kind, mean over three seeds: wide light bars uncalibrated, narrow dark bars afterT⁡\(x\)T\(x\)\.Fig\. 8:Per\-kind and per\-domain accuracy difference, PACT minus Nimble recipe, on the holdout\. Markers: seeds; bars: mean\.∗Same sign at all three seeds\.
### VI\-FTraining dynamics

Fig\.[9](https://arxiv.org/html/2609.35865#S6.F9)shows that the terms are optimised as intended\. The pair marginddstarts negative \(−0\.3\-0\.3nats: the base model moves the log\-odds the wrong way\), first exceeds the marginm=2m=2at step 20 and, after early oscillation, ends near 10–13 nats\. The logit gap on ablated contexts falls from 3\.4 to below 0\.1 nats, i\.e\. the model learns to be indifferent betweenyay\_\{a\}andyby\_\{b\}exactly where the certificate says it should\.ℒNR\\mathcal\{L\}\_\{\\mathrm\{NR\}\}is also the noisiest term, with isolated spikes when a batch contains an ablated context the model still separates strongly\.

Fig\. 9:Training dynamics of the shipped run \(EMA\-smoothed\)\. \(a\) Loss terms\. \(b\) Mean pair marginddof Eq\. \([1](https://arxiv.org/html/2609.35865#S4.E1)\) \(dashed:m=2m=2\) and mean gap\|zya−zyb\|\|z\_\{y\_\{a\}\}\-z\_\{y\_\{b\}\}\|on ablated contextsx∘x^\{\\circ\}\.
### VI\-GHypothesis scorecard

Table[VII](https://arxiv.org/html/2609.35865#S6.T7)confronts each pre\-specified criterion with the evidence\. Two hypotheses are supported, two partly, and one is inconclusive; none is contradicted outright, but H1—the motivating idea of the paper—does not receive clean support from a single\-seed ablation\.

## VIIDiscussion

What the evidence supports\.The results do not support the proposition that structure\-derived terms raise accuracy over a well\-reproduced cross\-entropy baseline\. They do support three narrower claims: \(i\) the terms reduce answer\-code position bias and ordinal error, the two failure modes they were designed around; \(ii\) they stabilise an optimiser that is otherwise significantly worse and highly seed\-sensitive; and \(iii\) post\-hoc calibration of this model family is cheap and lowers ECE in every run, although only the overall temperature level, not its contextual shape, is reliably estimated\. For a practitioner who serves single\-pass decisions over rubric fields, or who cannot affordKKforward passes to average out position bias, these are the properties that matter; for a practitioner who only needs pooled accuracy, the published recipe is sufficient and about five times cheaper to train\.

What remains open\.The dominant holdout failure is still the one the pair margin was designed for: in a quarter of the pairs the model answers both members identically, ignoring the edited fact \(Section[VI\-E](https://arxiv.org/html/2609.35865#S6.SS5)\)\. The mean margin is met on training pairs but the behaviour does not fully transfer to unseen source families, which points to margin schedules, harder negative edits or pair\-level data augmentation as the most promising next steps\.

Cost\.Each pair contributes 2\.5×\\timesthe rows of the baseline \(Table[VIII](https://arxiv.org/html/2609.35865#S7.T8)\)\. Measured wall\-clock per run is 1\.37 h for PACT against 0\.50 h for CE\-only and 0\.26 h for the one\-epoch Nimble recipe, and peak GPU memory is 28\.8 GiB against 21\.8 GiB\. Cost scales with the number of views rather than with the loss terms: droppingℒPC\\mathcal\{L\}\_\{\\mathrm\{PC\}\}alone saves nothing, while dropping the permuted views saves 45% of the time\. Serving cost is unchanged\.

TABLE VIII:Training cost per run \(one 48 GB GPU per job\): optimiser steps, training views per pair, mean wall\-clock time, peak GPU memory, and mean holdout accuracy \(%\)\. Serving cost is identical for all runs\.Selection noise\.Checkpoint selection is itself a source of variance comparable to the effects studied here \(Fig\.[10](https://arxiv.org/html/2609.35865#S7.F10)\)\. The shipped model is the PACT seed with the lowest selection\-split NLL \(seed 18\), yet it has the lowest holdout accuracy of the three PACT seeds \(82\.1%\)\. More strikingly, the checkpoint with the lowest selection NLL of*all*nine runs \(CE\-only seed 18: NLL 0\.112, 97\.4% selection accuracy\) has the*lowest*holdout accuracy of all fine\-tuned runs \(75\.6%\)\. With only two source families and 116 records, the selection split rewards fitting those families rather than transfer, which is one more reason to report seeds individually and to test differences in pairs\.

Fig\. 10:Selection\-split trajectories of the three multi\-seed arms \(line style: seed 17 solid, 18 dashed, 19 dotted; stars: selected checkpoints\)\. CE\-only converges slowly and reaches the best selection scores, which do not carry over to the holdout \(Table[IV](https://arxiv.org/html/2609.35865#S6.T4)\)\.
## VIIILimitations

The holdout contains 324 synthetic, model\-labelled items from six source families\. It is narrow, and its labels come from the same family of models as the training labels, so shared errors are possible and undetectable from inside the benchmark\.ℒNR\\mathcal\{L\}\_\{\\mathrm\{NR\}\}andℒCF\\mathcal\{L\}\_\{\\mathrm\{CF\}\}trust the certificate: a wrong necessity check teaches indifference where the evidence actually decides\. We apply the necessity term to one ablated view per pair to bound cost\. The ablations use one seed each, so per\-term claims are directional; the significance tests cover only the three multi\-seed arms\. The loss weights were fixed a priori and not tuned, so the negative accuracy findings are for this setting, not for every weighting\. All claims concern one base model at one scale with one adapter rank\.

## IXConclusion

Contrastive curation with certificates contains supervision that cross\-entropy cannot express: the offset\-free effect of an edited fact, the arbitrariness of answer codes, the indifference certified by evidence removal, and the order of rubric levels\. PACT turns each into a training term without changing the serving contract\. Evaluated under pre\-specified falsification criteria, the terms do not make the model more accurate than a well\-reproduced baseline, but they make it less sensitive to answer\-code order, better on ordinal fields and more stable across seeds, and a three\-parameter temperature makes its probabilities usable\. We hope the protocol—a reproduced baseline, a matched\-optimiser control, seed\-matched ablations and paired tests—is as useful as the method\.

## References

- \[1\]N\. Rogge\(2026\)Single\-token decoding for typed decisions\.Note:Online technical postCited by:[§I](https://arxiv.org/html/2609.35865#S1.p1.1),[§II](https://arxiv.org/html/2609.35865#S2.p1.1)\.
- \[2\]Bespoke Labs\(2026\)Nimble: data, model, and recipe for an open typed\-decision model\.Note:[https://github\.com/bespokelabsai/nimble](https://github.com/bespokelabsai/nimble)Model:[https://huggingface\.co/bespokelabs/Bespoke\-Nimble\-9B](https://huggingface.co/bespokelabs/Bespoke-Nimble-9B)Cited by:[§I](https://arxiv.org/html/2609.35865#S1.p1.1),[§I](https://arxiv.org/html/2609.35865#S1.p2.1),[§II](https://arxiv.org/html/2609.35865#S2.p1.1)\.
- \[3\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen\(2022\)LoRA: low\-rank adaptation of large language models\.InProc\. Int\. Conf\. Learning Representations \(ICLR\),Cited by:[§I](https://arxiv.org/html/2609.35865#S1.p2.1),[§II](https://arxiv.org/html/2609.35865#S2.p5.1)\.
- \[4\]L\. Tang, P\. Laban, and G\. Durrett\(2024\)MiniCheck: efficient fact\-checking of LLMs on grounding documents\.InProc\. Conf\. Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p1.1)\.
- \[5\]D\. Kaushik, E\. Hovy, and Z\. C\. Lipton\(2020\)Learning the difference that makes a difference with counterfactually\-augmented data\.InProc\. Int\. Conf\. Learning Representations \(ICLR\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p2.1)\.
- \[6\]M\. Gardner, Y\. Artzi, V\. Basmov, J\. Berant,et al\.\(2020\)Evaluating models’ local decision boundaries via contrast sets\.InFindings of the Association for Computational Linguistics: EMNLP,Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p2.1)\.
- \[7\]D\. Teney, E\. Abbasnejad, and A\. van den Hengel\(2020\)Learning what makes a difference from counterfactual examples and gradient supervision\.InProc\. European Conf\. Computer Vision \(ECCV\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p2.1)\.
- \[8\]J\. D\. Angrist and J\. Pischke\(2009\)Mostly harmless econometrics: an empiricist’s companion\.Princeton University Press\.Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p2.1)\.
- \[9\]C\. Burges, T\. Shaked, E\. Renshaw, A\. Lazier, M\. Deeds, N\. Hamilton, and G\. Hullender\(2005\)Learning to rank using gradient descent\.InProc\. Int\. Conf\. Machine Learning \(ICML\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p2.1)\.
- \[10\]J\. DeYoung, S\. Jain, N\. F\. Rajani, E\. Lehman, C\. Xiong, R\. Socher, and B\. C\. Wallace\(2020\)ERASER: a benchmark to evaluate rationalized NLP models\.InProc\. Annu\. Meeting Association for Computational Linguistics \(ACL\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p2.1)\.
- \[11\]C\. Zheng, H\. Zhou, F\. Meng, J\. Zhou, and M\. Huang\(2024\)Large language models are not robust multiple choice selectors\.InProc\. Int\. Conf\. Learning Representations \(ICLR\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p3.1)\.
- \[12\]P\. Pezeshkpour and E\. Hruschka\(2024\)Large language models sensitivity to the order of options in multiple\-choice questions\.InFindings of the Association for Computational Linguistics: NAACL,Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p3.1)\.
- \[13\]J\. Robinson and D\. Wingate\(2023\)Leveraging large language models for multiple choice question answering\.InProc\. Int\. Conf\. Learning Representations \(ICLR\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p3.1)\.
- \[14\]Z\. Zhao, E\. Wallace, S\. Feng, D\. Klein, and S\. Singh\(2021\)Calibrate before use: improving few\-shot performance of language models\.InProc\. Int\. Conf\. Machine Learning \(ICML\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p3.1)\.
- \[15\]J\. Lin\(1991\)Divergence measures based on the Shannon entropy\.IEEE Trans\. Inf\. Theory37\(1\),pp\. 145–151\.Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p3.1)\.
- \[16\]D\. Hendrycks, N\. Mu, E\. D\. Cubuk, B\. Zoph, J\. Gilmer, and B\. Lakshminarayanan\(2020\)AugMix: a simple data processing method to improve robustness and uncertainty\.InProc\. Int\. Conf\. Learning Representations \(ICLR\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p3.1)\.
- \[17\]C\. Guo, G\. Pleiss, Y\. Sun, and K\. Q\. Weinberger\(2017\)On calibration of modern neural networks\.InProc\. Int\. Conf\. Machine Learning \(ICML\),pp\. 1321–1330\.Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p4.1),[§IV\-F](https://arxiv.org/html/2609.35865#S4.SS6.p1.2)\.
- \[18\]M\. Kull, M\. Perello\-Nieto, M\. Kängsepp, T\. Silva Filho, H\. Song, and P\. Flach\(2019\)Beyond temperature scaling: obtaining well\-calibrated multi\-class probabilities with Dirichlet calibration\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p4.1)\.
- \[19\]Z\. Jiang, J\. Araki, H\. Ding, and G\. Neubig\(2021\)How can we know when language models know? On the calibration of language models for question answering\.Trans\. Assoc\. Comput\. Linguistics9,pp\. 962–977\.Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p4.1)\.
- \[20\]S\. Kadavath, T\. Conerly, A\. Askell,et al\.\(2022\)Language models \(mostly\) know what they know\.Note:arXiv:2207\.05221External Links:2207\.05221Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p4.1)\.
- \[21\]E\. Frank and M\. Hall\(2001\)A simple approach to ordinal classification\.InProc\. European Conf\. Machine Learning \(ECML\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p5.1)\.
- \[22\]L\. Hou, C\. Yu, and D\. Samaras\(2017\)Squared earth mover’s distance\-based loss for training deep neural networks\.Note:arXiv:1611\.05916External Links:1611\.05916Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p5.1)\.
- \[23\]D\. Kalajdzievski\(2023\)A rank stabilization scaling factor for fine\-tuning with LoRA\.Note:arXiv:2312\.03732External Links:2312\.03732Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p5.1)\.
- \[24\]S\. Hayou, N\. Ghosh, and B\. Yu\(2024\)LoRA\+: efficient low rank adaptation of large models\.InProc\. Int\. Conf\. Machine Learning \(ICML\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p5.1)\.
- \[25\]T\. Dettmers, A\. Pagnoni, A\. Holtzman, and L\. Zettlemoyer\(2023\)QLoRA: efficient finetuning of quantized LLMs\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§II](https://arxiv.org/html/2609.35865#S2.p5.1)\.
- \[26\]Qwen Team\(2026\)Qwen3\.5\-9B\.Note:[https://huggingface\.co/Qwen/Qwen3\.5\-9B](https://huggingface.co/Qwen/Qwen3.5-9B)Cited by:[§V\-B](https://arxiv.org/html/2609.35865#S5.SS2.p1.1)\.
- \[27\]I\. Loshchilov and F\. Hutter\(2019\)Decoupled weight decay regularization\.InProc\. Int\. Conf\. Learning Representations \(ICLR\),Cited by:[§V\-B](https://arxiv.org/html/2609.35865#S5.SS2.p1.1)\.
- \[28\]G\. W\. Brier\(1950\)Verification of forecasts expressed in terms of probability\.Monthly Weather Review78\(1\),pp\. 1–3\.Cited by:[§V\-D](https://arxiv.org/html/2609.35865#S5.SS4.p1.1)\.
- \[29\]M\. P\. Naeini, G\. F\. Cooper, and M\. Hauskrecht\(2015\)Obtaining well calibrated probabilities using Bayesian binning\.InProc\. AAAI Conf\. Artificial Intelligence,pp\. 2901–2907\.Cited by:[§V\-D](https://arxiv.org/html/2609.35865#S5.SS4.p1.1)\.
- \[30\]Y\. Geifman and R\. El\-Yaniv\(2017\)Selective classification for deep neural networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§V\-D](https://arxiv.org/html/2609.35865#S5.SS4.p1.1)\.
- \[31\]Q\. McNemar\(1947\)Note on the sampling error of the difference between correlated proportions or percentages\.Psychometrika12\(2\),pp\. 153–157\.Cited by:[§V\-D](https://arxiv.org/html/2609.35865#S5.SS4.p1.1)\.
- \[32\]B\. Efron and R\. J\. Tibshirani\(1993\)An introduction to the bootstrap\.Chapman & Hall\.Cited by:[§V\-D](https://arxiv.org/html/2609.35865#S5.SS4.p1.1)\.
- \[33\]C\. Thiery and B\. Scherrer\(2009\)Building controllers for Tetris\.ICGA Journal32\(1\),pp\. 3–11\.Cited by:[Appendix C](https://arxiv.org/html/2609.35865#A3.p1.1)\.
- \[34\]R\. J\. Williams\(1992\)Simple statistical gradient\-following algorithms for connectionist reinforcement learning\.Machine Learning8\(3–4\),pp\. 229–256\.Cited by:[Appendix C](https://arxiv.org/html/2609.35865#A3.p2.1)\.

## Appendix AAdditional Tables

Table[IX](https://arxiv.org/html/2609.35865#A1.T9)breaks accuracy down by field kind for every run and seed; Table[X](https://arxiv.org/html/2609.35865#A1.T10)gives the fitted temperatures and ECE before and after calibration on both the calibration split and the holdout\.

TABLE IX:Holdout accuracy \(%\) by field kind, single\-pass decoding \(Choicen=146n\{=\}146, Booleann=114n\{=\}114, Scoren=64n\{=\}64\), and expected\-level MAE on Score fields\.TABLE X:Contextual temperature per run: mean \[min, max\] ofT⁡\(x\)T\(x\)on the calibration split, and ECE \(%\) before/after on the calibration split \(fit\) and on the holdout \(test\)\.⋆Shipped model\.Red:T⁡\(x\)T\(x\)raised ECE\.
## Appendix BImplementation and Reproducibility

All code, configurations, data splits, trained adapters, run records and the scripts that regenerate every figure and table of this paper are released at[https://github\.com/BennyLinntu/PACT\-Pairwise\-Anchored\-Calibrated\-Tuning\-for\-Single\-Token\-Typed\-Decisions](https://github.com/BennyLinntu/PACT-Pairwise-Anchored-Calibrated-Tuning-for-Single-Token-Typed-Decisions)\. The training driver records, per run, the resolved configuration, the data audit and file hashes, the view\-cache fingerprint, library versions, device information, the training plan, the full step log, the selection\-split history, the fitted calibrator and every per\-item holdout prediction with its probabilities\. Adapters are saved with the recipe’sschema\_config\.jsoncontract, including the SHA\-256 of the scoring\-prompt implementation, so that a checkpoint is rejected if the prompt code it was trained against has changed\. All figures and data tables in this paper are regenerated from these records by a single script\. A 4\-bit NF4 configuration for 24 GB GPUs is provided but its results are not mixed with the BF16 numbers reported here; the only 4\-bit numbers in the paper are the supplementaryKK\-sweep and the base\-model predictions used for the paired tests\. Table[XI](https://arxiv.org/html/2609.35865#A2.T11)lists every hyper\-parameter; none was tuned on the holdout\.

TABLE XI:Hyper\-parameters of PACT \(read from the resolved run configuration\)\. The Nimble recipe differs only in: 1 epoch, linear schedule, no rsLoRA, LoRA\+ ratio 1, no extra views or terms\.
## Appendix CApplied Demonstration: Sequential Decisions in Tetris

Everything above evaluates the adapter on a static document plus a schema question\. As a qualitative check of how general the serving interface is, we reused it unmodified for a task with no document at all: choosing where to place falling pieces in a from\-scratch Tetris implementation\. At every piece the game engine—not the model—enumerates the legal placements and computes their consequences two plies deep \(rows cleared, holes, stack height and bumpiness, now and after the best placement of the visible next piece; standard Tetris features\[[33](https://arxiv.org/html/2609.35865#bib.bib34)\]\)\. Each option becomes one choice description in an ordinary schema call; the model reads one token and the highest\-logit option is played\.

Policy\-gradient adaptation\.Unmodified, the adapter does not reliably choose the option the two\-ply search ranks best: it was trained to classify documents, not to trade off sequential consequences\. We therefore further fine\-tuned the same LoRA adapter with single\-step REINFORCE\[[34](https://arxiv.org/html/2609.35865#bib.bib33)\]\(a contextual bandit over candidate placements\), using the two\-ply score of the sampled placement as reward and the mean score over that state’s candidates as a per\-state baseline, with an entropy bonus of 0\.01, learning rate10−510^\{\-5\}and 32 moves per update on 2×\\times40 GB GPUs\. Prompt construction is shared between training and serving, so there is no train/serve skew\.

Results\.The sampled top\-pick rate—agreement with the search’s best\-ranked option—rose from 19% at the first update to 87\.5% by step 16, and its 100\-step average stayed between 88% and 99% from step 200 to the end of the 1,574\-step run \(Fig\.[11](https://arxiv.org/html/2609.35865#A3.F11)a\)\. The training game itself never topped out after step 14, clearing 19,975 lines over roughly 50,000 pieces\. Greedy evaluation with a 400\-piece cap \(Fig\.[11](https://arxiv.org/html/2609.35865#A3.F11)b\) is flat at a mean of 17,000–18,000 points and about 158 lines, close to the ceiling of 160 lines that 400 pieces permit, so this protocol saturates and cannot show improvement; one uncapped greedy game at the final checkpoint reached 1,500 pieces and 598 lines \(score 65,700\) before it was stopped\.

Live play in the served app\.The repository \(Appendix[B](https://arxiv.org/html/2609.35865#A2)\) includes a three\-minute recording at 2×\\timesspeed \(media/pact\_rl\_tetris\_2x\.mp4\) of the final checkpoint driving the unmodified browser app through the ordinary serving path, in 4\-bit on a single 16 GB consumer GPU\. In six minutes of real play the model cleared more than 130 lines in one continuous game and reached level 13 without topping out, although gravity accelerates with every level and each move requires a fresh forward pass\.

Reading this appendix\.This is a single run without seeds, held\-out repetitions or a greedy pre\-training baseline, and the policy learns to follow a ranking that the engine already computes\. It demonstrates only that the single\-token interface is a general typed\-choice interface and that the same lightweight adapter can be re\-targeted to a new decision task; it carries no evidence about the holdout claims above\.

Fig\. 11:REINFORCE on Tetris placement choices\. \(a\) Share of sampled moves that match the two\-ply search’s top\-ranked option \(faint: per update; bold: 50\-step mean\)\. \(b\) Greedy evaluation every 50 steps, 5 games capped at 400 pieces \(line: mean; dashed: best game\)\.