Agentic Context Learning with Self-Discovered Specification

# Agentic Context Learning with Self-Discovered Specification
Source: [https://arxiv.org/html/2607.09794](https://arxiv.org/html/2607.09794)
Jike Zhong1∗†,Ming Li2∗,Yuxiang Lai3∗,Ziyan Yang1,Jingyu Xie1,Jihyung Kil4, Zheda Mai5,Shao\-Yuan Lo6,Xiang Ren1,Konstantinos Psounis1,Yuanyuan Lei2 1University of Southern California2University of Florida3Emory University 4Adobe Research5The Ohio State University6National Taiwan University

###### Abstract

Context learning is an emerging inference\-time task where LLMs must learn and apply novel, task\-specific knowledge from intricate contexts absent from pre\-training; even frontier models score under 24% task success\. In this work, we conduct a comprehensive empirical study to understand why this setting remains difficult\. A natural hypothesis is that failures stem fromcontent access; yet across twelve retrieval, reflection, and verification baselines on CL\-Bench, an extensive context learning benchmark, we find limited gains over direct full\-context prompting\. Further failure analysis reveals a key finding: unlike typical long\-context tasks such as long document understanding, context learning requires not only recovering local content but also acquiring*local specifications*that are oftenunspecifiedin the query but distributed across the context: domain\-specific formats, local rules, and completeness conditions\. Across all 31,592 rubric items, we find that 55\.4% clearly evaluate specification acquisition, while only 22\.6% evaluate content acquisition\. Moreover, despite 76\.7% of specifications being unspecified in the user query, 95\.5% are traceable to the context, indicating these are learnable obligations rather than hidden requirements\. To validate this diagnosis, we design a deliberately simple intervention PSCI \(private specification\-contract induction\) which extracts local specifications and enforces them through adversarial checking and repair; PSCI achieves state\-of\-the\-art 28\.14% with GPT\-5\.1 \(\+5\.59\+5\.59pp absolute and\+24\.8%\+24\.8\\%relative\) on CL\-Bench, replicated on Qwen3\.5\-27B \(\+5\.28\+5\.28pp\) and Gemini 3 Pro \(\+6\.17\+6\.17pp\)\. Seventeen ablations further isolate the role of task\-specific specifications\. Overall, our results suggest context learning hinges on not only content acquisition but also specification acquisition\.

\*\*footnotetext:Equal contribution\.$\\dagger$$\\dagger$footnotetext:Correspondence to:jikezhon@usc\.edu\.## 1Introduction

Large language models are increasingly deployed in settings where the prompt is not merely evidence to retrieve from, but defines a novel operating environment the model must learn before it can act: a product manual, workflow policy, regulatory code, API specification, or experimental dataset, often spanning tens of thousands of tokens and specifying the rules of a local domain\. Consider the task ofdetecting anomalies in an incoming event stream: the user prompt contains a 40K\-token context followed by a simple query asking the model to record and log anomalies\. The context defines not only the event semantics needed to detect anomalies, but also local specifications that determine what counts as a valid log: for instance, if a reserved event ID is skipped, the log must emit the exact flagSequence\_Gap\_Warning\. In this case, identifying the missing ID is the query\-relevant content; emitting the prescribed flag is a validity\-relevant specification\. A response that detects the anomaly but omits the flag is still wrong\. Thus, the challenge is not merely retrieving and reasoning over a long prompt, which is the focus of traditional long\-context benchmarks\(Liuet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib5); Hsiehet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib6); Baiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib8)\), but acquiring the local specifications by which the answer is validated\.

![Refer to caption](https://arxiv.org/html/2607.09794v1/figures/PSCI.jpg)Figure 1:Left:Context learning decomposes into content acquisition \(what the answer says\) and specification acquisition \(how the answer must behave\)\.Right:PSCI induces a private specification contract from the task and context, then uses it as a shared control object for generation, checking, and repair\. The public rubric for the current task is never observed during any stage\.Despite strong performance across many language tasks, frontier LLMs remain fragile on context learning: the best model in the original CL\-Bench evaluation scores below 24% strict task success\. CL\-Bench\(Douet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib1)\)is a recent benchmark designed specifically for context learning, with 1,899 diverse, long\-context, contamination\-controlled tasks \([section 3](https://arxiv.org/html/2607.09794#S3)\)\. In this paper, we conduct a comprehensive empirical study of why this setting remains difficult\. A natural hypothesis is that failures stem from content access\. To test this hypothesis, we evaluate twelve methods from four families on CL\-Bench: retrieval and context restructuring\(Asaiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib25); Yanet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib10); Sarthiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib12); Edgeet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib27); Sunet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib28)\), test\-time self\-correction\(Shinnet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib29); Chenet al\.,[2025](https://arxiv.org/html/2607.09794#bib.bib24)\), self\-generated rubric verification\(Cooket al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib31); Wanet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib15); Raghavendraet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib16)\), and iterative retrieval scaling\(Yueet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib18)\)\. Surprisingly, none of the twelve exceeds the direct baseline \(22\.55% strict pass\) by more than one point \([subsection 5\.2](https://arxiv.org/html/2607.09794#S5.SS2)\)\. This suggests that content access alone is insufficient and motivates a deeper investigation\.

Analysis of failures reveals a clear pattern\. Task\-level strict pass is low \(22\.55%\), but rubric\-level pass is much higher \(76\.01%\): models satisfy most requirements but miss a critical few, making failures near\-misses rather than wholesale errors\. Moreover, among tasks that fail by exactly one rubric item, 75\.3% of the missed items evaluate not factual content but compliance with locally specified behavioral constraints: formats, edge\-case handling, ordering, prescribed labels, or completeness conditions\. We call these obligationslocal specificationsand the ability to discover and enforce themspecification acquisition\. Taxonomizing all 31,592 CL\-Bench rubric items confirms this is systematic: 55\.4% evaluate specification acquisition, while only 22\.6% evaluate content acquisition\.

A key asymmetry explains why direct context\-to\-answer generation misses specifications despite having access to the full context\. Only 23\.3% of specification items are directly inferable from the user query; the remaining 76\.7% are recoverableonlyfrom the surrounding context\. Yet 95\.5% of specification items are traceable to the context itself through explicit declarations, validation patterns, or demonstrated examples\. Crucially, this establishes them aslearnable obligationsdistributed across the prompt rather than hidden evaluator preferences\. Thus, the issue is not that local specifications are absent or incomprehensible\. Rather, they are query\-implicit, low\-salience, and scattered across background conventions, schemas, exceptions, logging rules, and validation requirements\. This also explains why retrieval\-oriented methods give limited gains: they tend to surface query\-relevant, answer\-bearing evidence, while missing validity\-relevant specifications that govern the response’s form, procedure, validation behavior, or required action\. A method may therefore find thecontentneeded to answer the apparent query while missing the local obligations needed for an acceptable response\.

Surprisingly, when explicitly prompted toinduce specifications, the model can recover much of this evaluation surface: privately induced contracts cover 94\.4% of held\-out public specification rubrics with less than 0\.5% contradiction\. This suggests that the missing step is not understanding the context in principle, but surfacing its local obligations as explicit generation constraints\. To validate this diagnosis, we design a deliberately simple intervention:private specification\-contract induction\(PSCI\), a test\-time scaffold comprising four steps \(Figure[1](https://arxiv.org/html/2607.09794#S1.F1)\)\. The model first induces a task\-specific specification contract from the context itself, then answers under the contract; an adversarial checker audits the draft against the contract; and a repair step patches violations\. The public rubric for the current task is never observed at any stage\. On the full 1,899\-task benchmark, PSCI achieves state\-of\-the\-art 28\.14% strict pass with GPT\-5\.1 \(\+5\.59 pp absolute and \+24\.8% relative\), with consistent gains across all four task families and replication on Qwen3\.5\-27B \(14\.14%→\\\!\\to\\\!19\.42%\)\.Seventeenablations isolate the role of task\-specific specifications: shuffled contracts collapse below baseline, compute\-matched generic critique returns to baseline, answer\-first ordering fails\. Overall, our contributions are threefold:

- •Through comprehensive empirical analysis, we identify*specification acquisition*—inferring query\-implicit but context\-traceable local obligations—as a key bottleneck in context learning, and show these obligations are recoverable by explicit induction\.
- •To validate this diagnosis, we introduce PSCI, a deliberately simple test\-time scaffold that achieves state\-of\-the\-art CL\-Bench performance across multiple model families\.
- •Through extensive ablation and mechanism analysis, we show that specification acquisition must be grounded in the context and decomposes into two separable stages: specification discovery and specification enforcement\.

## 2Related Work

#### Long\-context evaluation and context learning\.

Long\-context benchmarks evaluate evidence use over long inputs\(Kamradt,[2023](https://arxiv.org/html/2607.09794#bib.bib7); Hsiehet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib6); Baiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib8); Liuet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib5)\), typically under a fixed task interface\. Context learning additionally requires inducing local validity conditions from the prompt\. CL\-Bench\(Douet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib1)\)is the only large\-scale benchmark explicitly designed for this setting\. Unlike instruction\-following benchmarks\(Zhouet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib19); Xiaet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib20)\), where constraints are explicit in the query, CL\-Bench specifications are often query\-implicit and must be induced from local documentation\.

#### Retrieval, memory, and context restructuring\.

Methods improving content access\(Lewiset al\.,[2020](https://arxiv.org/html/2607.09794#bib.bib40); Asaiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib25); Yanet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib10); Sarthiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib12); Edgeet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib27); Sunet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib28); Packer and others,[2023](https://arxiv.org/html/2607.09794#bib.bib34); Yueet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib18)\)address a necessary but insufficient component of context learning: our twelve baselines show that content access alone does not solve specification acquisition \([subsection 5\.2](https://arxiv.org/html/2607.09794#S5.SS2)\)\.

#### Critique, verification, and structured prompting\.

Self\-refinement\(Madaanet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib30); Shinnet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib29)\), checklist verification\(Cooket al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib31); Wanet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib15); Raghavendraet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib16)\), and structured prompting\(Weiet al\.,[2022](https://arxiv.org/html/2607.09794#bib.bib32); Yaoet al\.,[2022](https://arxiv.org/html/2607.09794#bib.bib45); Khattabet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib37)\)all introduce intermediate computation\. PSCI differs in inducing a task\-specific obligation set from context*before*generation, then enforcing it through checking and repair\. Ablations confirm this object, not the pipeline structure, drives the gain \([subsection 5\.3](https://arxiv.org/html/2607.09794#S5.SS3)\)\.

Extended discussion in Appendix[E](https://arxiv.org/html/2607.09794#A5)\.

## 3Empirical Study of Context Learning

#### Why CL\-Bench\.

We use CL\-Bench\(Douet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib1)\)as our target because it is, to our knowledge, the only benchmark specifically designed for context learning: 1,899 tasks and 31,592 rubric items across four families \(domain knowledge reasoning, rule system application, procedural task execution, and empirical discovery & simulation\), with contexts averaging 10\.4K tokens and reaching 65K\. All tasks arehuman\-expert\-writtenand requireknowledge absent from pre\-training, with contamination prevented by design \(Appendix[A\.1](https://arxiv.org/html/2607.09794#A1.SS1)\)\. The breadth and quality of CL\-Bench make it a strong testbed for an in\-depth empirical study of context learning\.

#### Initial hypothesis and experiment\.

A natural hypothesis is that the low performance \(<24%<24\\%for GPT 5\.1\) stems from content access: the contexts are long and intricate, so methods that improve retrieval, restructuring, reflection, or verification should help\. However, as shown in[subsection 5\.2](https://arxiv.org/html/2607.09794#S5.SS2),twelverepresentative methods from these families provide limited gains over direct full\-context prompting\. This motivates a closer diagnosis of what models actually miss\. To tackle this, we next conduct an in\-depth analysis of the task itself\.

Traceability classItemsShareExplicit declaration12,53171\.6%Validation pattern3,69321\.1%Implicit example3502\.0%Convention / repeated struct\.1400\.8%Overall traceable*16,714**95\.5%*Snippet too short to judge7184\.1%Not derivable from context700\.4%Table 1:Traceability of specification items to task/context\. 95\.5% are traceable to explicit declarations, validation patterns, examples, repeated structures\.
### 3\.1Analysis Protocol

We classify all 31,592 rubric items as evaluating*content acquisition*or*specification acquisition*: content if its verdict depends on which propositions the answer states; specification if it depends on whether the answer conforms to a locally defined rule governing form, procedure, or validation, or required action, holding propositions fixed\. Two independent judges, an LLM\-based classifier \(GPT\-5\.1\) and a deterministic rule\-based classifier \(with explicit abstention class\), must agree for an item to be labeled; disagreements are marked*ambiguous*\.

To validate the taxonomy, we additionally audit a random sample of 300 rubric items with three PhD\-level annotators\. The human audit agrees with the conservative classifier labels in over 90% of non\-ambiguous cases \(details in Appendix[A\.2](https://arxiv.org/html/2607.09794#A1.SS2)\)\.

![Refer to caption](https://arxiv.org/html/2607.09794v1/figures/rubric-failure-breakdown.png)Figure 2:Rubric failure type breakdown by “content acquisition”, “specification acquisition”, and “abstain”\. Results show that the majority of original rubric items are “specification acquisition," and it is also the main failure type among near miss tasks\.
### 3\.2Failures Are Often Near\-Misses

The GPT\-5\.1 baseline \(setup in[subsection 5\.1](https://arxiv.org/html/2607.09794#S5.SS1)\) passes 76\.01% of individual rubric items but only 22\.55% of tasks: most failures are near\-misses rather than wholesale errors\. Among the 281 tasks that fail by exactly one rubric item \(validated in Appendix[A\.3](https://arxiv.org/html/2607.09794#A1.SS3)\), 75\.3% of the missed items are specification items—local conventions the model failed to enforce rather than missing facts\. This concentration suggests that addressing specification acquisition would yield disproportionate gains\.

### 3\.3Specifications Form the Largest Share of Rubric Surface

Applying the protocol to all 31,592 items \([Figure 2](https://arxiv.org/html/2607.09794#S3.F2)\), 55\.4% are classified as specification, 22\.6% as content, and 22\.0% as ambiguous\. Thus, even under a conservative criterion, specification acquisition accounts for the largest share of CL\-Bench’s evaluation surface\. The specification class further decomposes into eight subtypes—from validation checks \(42\.9%\) to role/tone constraints \(1\.2%\)—confirming it captures multiple dimensions of locally valid behavior rather than surface formatting alone \(full breakdown in Table[2](https://arxiv.org/html/2607.09794#S3.T2)\)\.

Table 2:Specification subtype distribution and item\-level pass rates by baseline and PSCI \(%\)\. Results show that every subtype is improved by PSCI; the largest gains are concentrated in harder subtypes\.
### 3\.4Specifications Are Learnable but Query\-Implicit

A natural concern is that specification items encode arbitrary evaluator preferences\. We test this directly \(Table[1](https://arxiv.org/html/2607.09794#S3.T1); annotation protocol in Appendix[A\.4](https://arxiv.org/html/2607.09794#A1.SS4)\): 95\.5% of specification items are traceable to the context through explicit declarations, validation patterns, implicit examples, or repeated conventions\. Only 0\.4% are not derivable\.

However, only 23\.3% of specification items are inferable from the user query alone; the remaining 76\.7% require reading the surrounding context\. This asymmetry explains why both direct prompting and retrieval methods underperform: specifications are learnable obligations distributed across the prompt, but they are query\-implicit, low\-salience, and scattered across background conventions\. Crucially, the problem is not that specifications are incomprehensible—when explicitly prompted to extract them, the model recovers 94\.4% of held\-out public rubrics with less than 0\.5% contradiction\. The capability exists, but is not reliably exercised during standard generation\.

## 4Private Specification\-Contract Induction

The analysis in[section 3](https://arxiv.org/html/2607.09794#S3)suggests that context learning failures are not explained by content access alone: many missed requirements are local specifications that are traceable to the context but largely absent from the user query\. To validate this diagnosis, we design a deliberately simple intervention, Private Specification\-Contract Induction \(PSCI\)\. PSCI tests whether making these latent specifications explicit before generation and enforcing them when forming responses improves context learning:

K\\displaystyle K=Induce​\(x,c;θ\)\\displaystyle=\\mathrm\{Induce\}\(x,c;\\,\\theta\)\(1\)y0\\displaystyle y\_\{0\}=Generate​\(x,c,K;θ\)\\displaystyle=\\mathrm\{Generate\}\(x,c,K;\\,\\theta\)\(2\)d\\displaystyle d=Check​\(x,c,K,y0;θ\)\\displaystyle=\\mathrm\{Check\}\(x,c,K,y\_\{0\};\\,\\theta\)\(3\)y^\\displaystyle\\hat\{y\}=Repair​\(x,c,K,y0,d;θ\)\\displaystyle=\\mathrm\{Repair\}\(x,c,K,y\_\{0\},d;\\,\\theta\)\(4\)wherexxis the task,ccis the context,KKis the induced specification contract,y0y\_\{0\}is the initial draft,ddis a structured diagnosis,y^\\hat\{y\}is the final answer, andθ\\thetadenotes the LLM parameters\. The public rubric for the current task is never observed at any stage\. The method is deliberately minimal: a complex pipeline would obscure whether gains stem from the diagnosis being correct or from engineering\.

### 4\.1Specification Discovery

Given taskxxand contextcc, an LLM induces a private specification contractK=\{k1,k2,…,kn\}K=\\\{k\_\{1\},k\_\{2\},\\ldots,k\_\{n\}\\\}: a structured set ofnnobligation items the answer must satisfy to be locally valid, covering required formats, validation checks, ordering constraints, and exact local artifacts\. The model then produces a drafty0=Generate​\(x,c,K;θ\)y\_\{0\}=\\mathrm\{Generate\}\(x,c,K;\\,\\theta\)conditioned on the task, context, and contract jointly\. We use rotated demonstrations drawn from disjoint context learning tasks only to teach contract style while the final contract used for generation is induced from the current task context; no same\-task answer, public rubric, or task\-specific contract is shown at any stage \(details in Appendix[B\.1](https://arxiv.org/html/2607.09794#A2.SS1)\)\.For analysis only, after generation, we measure specification discovery quality via contract–rubric overlap:

Coverage​\(K,R\)=\|\{r∈R:∃k∈K​that covers​r\}\|\|R\|\\begin\{split\}\\mathrm\{Coverage\}\(K,R\)&=\\\\ &\\displaystyle\\frac\{\|\\\{r\\in R:\\exists\\,k\\in K\\text\{ that covers \}r\\\}\|\}\{\|R\|\}\\end\{split\}\(5\)

### 4\.2Specification Enforcement

An adversarial checker functions as a diagnostic verifier\. Given\(x,c,K,y0\)\(x,c,K,y\_\{0\}\), it assesses each obligationki∈Kk\_\{i\}\\in Kand produces:

d=\{\(ki,si,ri\)\}i=1n,si∈\{Satisfied,Partial,Violated\}\\begin\{split\}d=\\\{&\(k\_\{i\},\\,s\_\{i\},\\,r\_\{i\}\)\\\}\_\{i=1\}^\{n\},\\\\ &s\_\{i\}\\in\\\{\\textsc\{Satisfied\},\\,\\textsc\{Partial\},\\,\\textsc\{Violated\}\\\}\\end\{split\}\(6\)wheresis\_\{i\}is the compliance status andrir\_\{i\}is a natural\-language rationale localizing the mismatch\. The repair model producesy^=Repair​\(x,c,K,y0,d;θ\)\\hat\{y\}=\\mathrm\{Repair\}\(x,c,K,y\_\{0\},d;\\,\\theta\), preserving correct content while patching obligations diagnosed asPartialorViolated\. Crucially, the checker targets the induced contractKK, not the draft’s surface quality—decoupling verification from generation\. Two design decisions distinguish PSCI from generic self\-refinement: the contract is induced*before*generation \(control object, not post\-hoc grader\), and the checker is grounded inKKrather than open\-ended quality\. Both are verified empirically in[subsection 5\.3](https://arxiv.org/html/2607.09794#S5.SS3)\(more analysis in Appendix[B\.2](https://arxiv.org/html/2607.09794#A2.SS2)\)\.

Table 3:Main results on CL\-Bench \(1,899 tasks, 31,592 rubric items\)\.*Overall*: strict\-pass rate, global rubric \(item micro\-average\), and mean task rubric \(task macro\-average\)\. Per\-family columns: strict\-pass and rubric for DKR \(Domain Knowledge Reasoning\), RSA \(Rule System Application\), PTE \(Procedural Task Execution\), EDS \(Empirical Discovery & Simulation\)\. Baselines grouped by family \(a–d\) as defined in[subsection 5\.1](https://arxiv.org/html/2607.09794#S5.SS1)\. Subscripts = absolute gains over same\-model baseline; best inbold\. PSCI improves all evaluated model groups, surpassing even GPT\-5\.1 High \(23\.7%\)\.∗Frontier results fromDouet al\.\([2026](https://arxiv.org/html/2607.09794#bib.bib1)\); rubric/task columns unavailable\.

## 5Experiments

### 5\.1Setup

#### Benchmark\.

We evaluate on CL\-Bench\(Douet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib1)\), which contains 1,899 tasks across four families: domain knowledge reasoning \(DKR\), rule system application \(RSA\), procedural task execution \(PTE\), and empirical discovery & simulation \(EDS\)\. The benchmark is scored by 31,592 rubric items, averaging 16\.64 items per task\. Contexts average 10\.4K tokens and reach 65K\. A task is solved only when*all*associated rubric items pass; even GPT\-5\.1 with high reasoning effort achieves only 23\.7% strict pass in the original evaluation\.

#### Models\.

We use GPT\-5\.1\(Singhet al\.,[2025](https://arxiv.org/html/2607.09794#bib.bib22)\)with medium reasoning effort as the primary model\. GPT\-5\.1 is the strongest model in the original CL\-Bench evaluation, and medium effort gives the strongest direct baseline in our setup, making it a conservative comparison point for PSCI\. To test cross\-model generalization, we additionally evaluate on Gemini 3 Pro \(high\)\(Gemini Team,[2025](https://arxiv.org/html/2607.09794#bib.bib21)\)and Qwen3\.5\-27B\(Yanget al\.,[2025](https://arxiv.org/html/2607.09794#bib.bib23)\)\.

#### Baselines\.

The direct baseline is single\-turn full\-context prompting\. We compare against twelve methods from four families:\(a\)retrieval and context restructuring: Self\-RAG\(Asaiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib25)\), CRAG\(Yanet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib10)\), RAPTOR\(Sarthiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib12)\), GraphRAG\(Edgeet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib27)\), RankGPT\(Sunet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib28)\), and SimpleMem;\(b\)self\-correction: Reflexion\(Shinnet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib29)\)and Guideline Forest\(Chenet al\.,[2025](https://arxiv.org/html/2607.09794#bib.bib24)\);\(c\)self\-generated rubric verification: TICK/STICK\(Cooket al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib31)\), DeepVerifier\(Wanet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib15)\), and Agentic Rubrics\(Raghavendraet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib16)\); and\(d\)iterative retrieval: IterDRAG\(Yueet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib18)\)\. All baselines use the same GPT\-5\.1 medium\-effort setting with full prompts and implementation details provided in[subsection B\.4](https://arxiv.org/html/2607.09794#A2.SS4)\.

MethodStrict Pass \(%\)Rubric Pass \(%\)PipelineablationsBaseline22\.55±\\pm0\.676\.01±\\pm1\.2A1: \+ Spec26\.09±\\pm1\.279\.10±\\pm1\.6A2: \+ Repair only23\.02±\\pm0\.4*71\.68*±\\pm1\.6A3: \+ Chk \+ Repair \(no spec\)24\.32±\\pm0\.777\.44±\\pm1\.5A4: \+ Spec \+ Repair \(no chk\)26\.30±\\pm1\.080\.15±\\pm2\.3A5: \+ Spec \+ Combined Chk&Rep25\.90±\\pm0\.478\.80±\\pm2\.5A6: \+ Answer\-first Spec \+ Chk \+ Rep23\.67±\\pm1\.0*70\.67*±\\pm1\.7PSCI\(Ours\)28\.14±\\pm0\.581\.13±\\pm0\.9Spec quality\(discovery only\)A7: \+ Spec \(Shuffled\)19\.80±\\pm1\.5*66\.55*±\\pm1\.2A8: \+ Spec \(Zero\-shot\)25\.01±\\pm0\.7*72\.69*±\\pm2\.1A9: \+ Spec \(Generic\)23\.80±\\pm0\.976\.76±\\pm2\.6Spec quality\(full pipeline\)A10: \+ Spec \(Shuffled\) \+ Chk \+ Rep22\.00±\\pm1\.2*66\.55*±\\pm2\.1A11: \+ Spec \(Zero\-shot\) \+ Chk \+ Rep26\.67±\\pm1\.1*78\.69*±\\pm1\.2A12: \+ Spec \(Generic\) \+ Chk \+ Rep24\.00±\\pm0\.574\.76±\\pm1\.7Compute\-matchedcontrolsA13: \+ Spec\-free Critique22\.67±\\pm0\.8*70\.06*±\\pm0\.9A14: \+ Self\-reflection23\.02±\\pm0\.4*71\.24*±\\pm1\.4A15: \+ Majority Vote24\.12±\\pm0\.9*74\.23*±\\pm1\.9A16: \+ Repeated Static Spec24\.30±\\pm1\.978\.45±\\pm3\.1A17: \+ Repeated Zero\-shot Spec24\.67±\\pm1\.879\.10±\\pm1\.5Table 4:Ablation study \(full benchmark, GPT\-5\.1 Medium\)\. “\+” = added to Baseline\. Spec = specification contract; Chk = checker; Rep = repair\.*Italics*= rubric below baseline \(76\.01%\)\. We highlight three key results: shuffled specs collapse below baseline \(A7/A10\), spec\-free critique returns to baseline \(A13\)—extra compute alone does not explain the gain, and answer\-first ordering underperforms PSCI by 4\.47 pp\.
#### Metrics and protocol\.

We report strict task pass rate, global rubric pass rate \(item\-level micro\-average\), and mean task rubric \(task\-level macro\-average\), both overall and by task family\. We follow the released CL\-Bench evaluation protocol, using the same GPT\-5\.1 verifier and evaluation code, which the benchmark reports as achieving over 90% agreement with human judgments\. For each method and model, we run the full benchmark three times and report mean±\\pmstandard deviation\. The complete PSCI pipeline constitutes one run; reruns are used only for API infrastructure failures, with no selective filtering or cherry\-picking \(additional eval info in[subsection B\.4](https://arxiv.org/html/2607.09794#A2.SS4)\)\.

To validate that gains are not an artifact of automatic evaluation, three PhD\-level annotators blindly audit 100 baseline\-fail/PSCI\-pass tasks\. A majority vote prefers the PSCI answer in 96/100 \(96%\) of cases and confirms the automatic flip judgment in 92/100 \(92%\) \(protocol in[Appendix F](https://arxiv.org/html/2607.09794#A6)\)\.

### 5\.2Main Results

#### PSCI improves across model families\.

Table[3](https://arxiv.org/html/2607.09794#S4.T3)reports full\-benchmark results\. PSCI improves GPT\-5\.1 strict pass from 22\.55% to 28\.14%, a \+5\.59 pp absolute gain and 24\.8% relative improvement\. Global rubric pass also increases by 5\.12 pp\. The improvement replicates on Qwen3\.5\-27B, where PSCI improves strict pass from 14\.14% to 19\.42% \(\+5\.28 pp\) and global rubric pass by \+5\.17 pp\. The gain also holds on Gemini 3 Pro, improving strict pass from 16\.14% to 22\.31% \(\+6\.17 pp\)\. Gains are consistent across all four CL\-Bench task families\. Notably, PSCI with GPT\-5\.1 medium \(28\.14%\) exceeds GPT\-5\.1 high from the original CL\-Bench \(23\.7%\), suggesting explicit specification induction is a more effective use of inference compute than increasing reasoning effort alone\.

#### The gains align with the diagnosis\.

PSCI improves both rubric types, but gains are larger on specifications\. On GPT\-5\.1, specification\-item pass rate rises from 76\.7% to 82\.1%, closing 23\.2% of the available gap; content\-item pass rate rises from 73\.4% to 77\.5%, closing 15\.5%\. Among rubric items fixed on tasks that flip from baseline\-fail to PSCI\-pass, 77\.7% are specification items, showing PSCI primarily addresses the diagnosed specification bottleneck while also improving content use\.

#### Existing methods do not match PSCI\.

As discussed in[section 1](https://arxiv.org/html/2607.09794#S1), all twelve alternative methods remain close to the direct baseline, ranging from 19\.00% to 23\.33% strict pass, with none exceeding the baseline by more than 0\.78 percentage points\. Retrieval and restructuring methods often underperform full\-context prompting, consistent with the hypothesis that context selection can discard distributed specification signals\. Self\-generated rubric methods, which are closest in spirit to PSCI, also fail to meaningfully exceed the baseline\. Therefore, the gain is not explained by generic checklist verification; it requires context\-grounded specification induction and obligation\-level checking\.

### 5\.3Ablation Studies

To isolate the mechanism, we run 17 ablations \(A1–A17\) on the full 1,899\-task benchmark under the same GPT\-5\.1 medium\-effort setting \(Table[4](https://arxiv.org/html/2607.09794#S5.T4)\)\.

#### Each PSCI component contributes\.

Specification induction alone \(A1\) reaches 26\.09%, accounting for 63% of PSCI’s total strict\-pass gain\. Checker and repair add a further \+2\.05 points\. In contrast, repair alone \(A2, 23\.02%\) and checker\+repair without a specification contract \(A3, 24\.32%\) yield only limited gains, showing that the contract is the primary driver\. Separating diagnosis from repair is also important: spec\+repair without an explicit checker \(A4, 26\.30%\) and a combined check\-and\-repair step \(A5, 25\.90%\) both underperform PSCI\.

#### Extra compute does not explain the gain\.

Five compute\-matched alternatives without task\-specific specifications all fall short\. Spec\-free critique \(A13\) returns to 22\.67%, and self\-reflection \(A14\) to 23\.02%\. Even repeated contract\-like structures without task\-specific grounding \(A16–A17\) plateau at 24\.30–24\.67%\. Thus, additional inference and generic self\-correction are insufficient; the gain requires a task\-specific specification contract\.

#### Finding 1: Specifications must be extracted separately \(A7, A10\)\.

To test whether contracts are merely helpful prompt prefixes, we replace each contract with one drawn from another task\. Shuffled contracts collapse below baseline: A7 drops to 19\.80% strict pass and A10 to 22\.00%, with rubric pass falling to 66\.55%\. A misaligned specification is worse than no specification, demonstrating that task\-specific alignment is causally necessary\.

Table 5:Overlap \(%\) between privately induced contracts and non\-ambiguous public \(ground\-truth\) rubrics\. The contract recovers most of the public rubric surface, especially for specification items\.
#### Finding 2: Specifications must be grounded in context \(A8, A11, A12\)\.

Generic specifications underperform context\-derived ones\. Zero\-shot context\-derived specifications through the full pipeline \(A11\) reach 26\.67%, while generic static specifications \(A12\) reach only 24\.00%\. The hierarchy is clear: generic specifications≪\\llzero\-shot context\-derived specifications<<few\-shot context\-derived PSCI\. Zero\-shot context\-derived specifications already improve over baseline, while generic and shuffled specifications underperform, showing that demonstrations refine contract granularity but do not create the effect\. Extracting task\-specific specifications from the current context is key\.

#### Finding 3: Specifications must be enforced \(A1, A4–A5\)\.

Discovery alone \(A1, 26\.09%\) remains below PSCI, showing that specification discovery is not sufficient\. Spec\+repair without an explicit checker \(A4, 26\.30%\) and combined check\-and\-repair \(A5, 25\.90%\) also underperform PSCI, indicating that enforcement benefits from explicit checking and targeted repair\. This separates discovery from compliance: the checker exposes concrete spec violation targets, enabling compliant repair rather than unconstrained rewriting\.

#### Finding 4: Contracts Must Act as Control Objects\.

The answer\-first ablation \(A6\) tests whether specifications can be induced after generation and used only as repair guidance\. It cannot: A6 drops to 23\.67% strict pass and 70\.67% rubric pass, far below PSCI\. Since only 23\.3% of specifications are inferable from the user query, direct generation often fixes the answer structure before scattered obligations become active constraints\. Post\-hoc repair can patch local violations, but cannot reliably restructure fields, evidence use, or completeness conditions\. Thus, the contract must shape generation as a control object rather than act as a post\-hoc grader, explaining why self\-generated rubric verification methods do not match PSCI\.

## 6Discussion

### 6\.1Quality of Induced Specifications

For analysis only, we compare private contracts with held\-out public rubrics never observed by PSCI, labeling each item as strongly covered, partially covered, contradicted, or uncovered \(details in[subsection G\.1](https://arxiv.org/html/2607.09794#A7.SS1)\)\. Across 24,640 non\-ambiguous items, contracts strongly cover 60\.8% and partially cover 33\.4%, totaling 94\.2% coverage with only 0\.49% contradiction \(Table[5](https://arxiv.org/html/2607.09794#S5.T5)\)\. Specification items are more strongly covered than content items \(62\.5% vs\. 54\.9%\), and human audit confirms the automatic measurement is conservative \(64\.95% vs\. 60\.8% strong coverage; Appendix[D\.2](https://arxiv.org/html/2607.09794#A4.SS2)\)\.

Coverage also tracks correctness: pass rate rises from contradicted \(25\.4%\) and uncovered \(48\.7%\) items to partially covered \(75\.2%\) and strongly covered items \(88\.8%\)\. Thus, induced contracts are not decorative; better coverage corresponds to downstream success, with a 40\-point pass\-rate gap between strongly covered and uncovered items\.

### 6\.2The Enforcement Bottleneck

Among rubric items still failed by PSCI, 83% are covered by the contract but not executed in the final answer; only 14\.9% are discovery failures and 2\.0% are contradictions\. Thus, PSCI substantially improves specification discovery, but enforcement becomes the dominant residual bottleneck: current models often surface the right local obligations but fail to consistently satisfy them during generation and repair\. Future work should prioritize stronger enforcement mechanisms, such as iterative verification, more reliable repair, or training\-time objectives that internalize specification compliance\.

## 7Conclusion

We identify local specification acquisition as a key bottleneck in context learning\. This motivates PSCI, a simple test\-time scaffold that makes local specifications explicit before generation and enforces them through checking and repair\. Our results show that context learning requires not only content acquisition, but also context\-grounded specification discovery and enforcement\.

## Acknowledgment

This work is partially supported by a grant from the USC\-Amazon Center on Secure & Trusted ML and funding from the National Science Foundation NSF \(Award \# \(USC\): 1956435\)\.

## Limitations

#### Benchmark scope\.

We mainly target our analysis and experiments on the single benchmark, CL\-Bench, designed specifically for context learning\. Although this limits claims about generalization beyond the current context learning benchmark landscape on the surface, CL\-Bench is currently the only large\-scale benchmark explicitly designed for context learning, and it is unusually broad for a single benchmark: 1,899 human\-expert\-written, contamination\-controlled tasks, 31,592 rubric items, four task families, and contexts averaging 10\.4K tokens and reaching 65K\. Thus, while evaluation on future context learning benchmarks would strengthen external validity, CL\-Bench already covers a diverse set of local domains, including domain knowledge, rule systems, procedures, and empirical discovery\.

#### Inference cost\.

PSCI is an inference\-time scaling method and is therefore more expensive than single\-pass generation: it uses four sequential model calls for specification induction, contract\-guided answer generation, contract\-aware checking, and targeted repair\. We treat this as a central tradeoff rather than a hidden cost\. Compute\-matched and compute\-heavier controls show that additional inference alone does not explain the gain: generic self\-reflection, iterative retrieval, guideline\-based reasoning, and verification\-style baselines use comparable or larger budgets but fail to match PSCI\. Thus, the improvement comes from how computation is structured—around task\-specific specification discovery and enforcement—rather than from extra tokens alone\. Future work should explore cheaper ways to extract and enforce local specifications, including distillation, adaptive early stopping, or training\-time internalization\.

#### Prompting\-based implementation\.

PSCI is training\-free and immediately deployable, but prompting alone may be suboptimal for enforcing specifications reliably\. Our residual\-error analysis suggests that many remaining failures occur when the contract covers an obligation but the final answer does not execute it\. This points to future work on stronger enforcement mechanisms, iterative repair, or training\-time objectives that internalize specification compliance while reducing inference cost\.

## Ethics Statement

PSCI is a test\-time prompting scaffold evaluated on CL\-Bench, a public benchmark whose contexts are expert\-crafted through fictional creation or modification of existing content\. No private user data is collected or generated at any stage\. The specification contracts induced by PSCI are task\-specific intermediate objects used only within the immediate generation pipeline; they are not stored, shared, or reused beyond evaluation\. All evaluation uses the publicly released CL\-Bench evaluation code and rubrics\. We emphasize that improved context learning performance should not be interpreted as a guarantee of factual correctness or safety compliance in deployment, since specification acquisition addresses local validity within a provided context rather than global truthfulness\.

Because PSCI extracts and enforces local specifications, it may also over\-enforce brittle, biased, unsafe, or undesirable local rules if such rules appear in the context\. This is especially important in legal, medical, financial, or institutional settings, where local policies may conflict with fairness, safety, or broader legal or ethical constraints\. PSCI should therefore be used with expert oversight in high\-stakes domains and should not be treated as a guarantee that the extracted specification is normatively appropriate\.

#### Artifact licenses and intended use\.

CL\-Bench is released under a custom evaluation\-only license permitting use, modification, and distribution solely for model evaluation, testing, and benchmarking\. Our use is fully consistent with this restriction: PSCI is a test\-time prompting scaffold, and no stage of our pipeline trains, fine\-tunes, distills, or otherwise updates model parameters on CL\-Bench data\. Evaluated models are accessed through official APIs or public weights and used for research benchmarking consistent with their respective terms\.

## References

- Self\-RAG: learning to retrieve, generate, and critique through self\-reflection\.InThe Twelfth International Conference on Learning Representations \(ICLR\),Cited by:[Appendix C](https://arxiv.org/html/2607.09794#A3.SS0.SSS0.Px1.p1.1),[§E\.2](https://arxiv.org/html/2607.09794#A5.SS2.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px3.p1.1)\.
- Y\. Bai, X\. Lv, J\. Zhang, H\. Lyu, J\. Tang, Z\. Huang, Z\. Du, X\. Liu, A\. Zeng, L\. Hou,et al\.\(2024\)Longbench: a bilingual, multitask benchmark for long context understanding\.InProceedings of the 62nd annual meeting of the association for computational linguistics \(volume 1: Long papers\),pp\. 3119–3137\.Cited by:[§E\.1](https://arxiv.org/html/2607.09794#A5.SS1.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p1.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Chen, Z\. Wang, M\. Zou, Q\. Wang, and Z\. Xu \(2025\)Guideline forest: retrieval\-augmented reasoning with branching experience\-induced guidelines\.arXiv preprint arXiv:2506\.07820\.Cited by:[Appendix C](https://arxiv.org/html/2607.09794#A3.SS0.SSS0.Px12.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px3.p1.1)\.
- J\. Cook, T\. Rocktäschel, J\. Foerster, D\. Aumiller, and A\. Wang \(2024\)Ticking all the boxes: generated checklists improve llm evaluation and generation\.arXiv preprint arXiv:2410\.03608\.Cited by:[Appendix C](https://arxiv.org/html/2607.09794#A3.SS0.SSS0.Px6.p1.1),[§E\.3](https://arxiv.org/html/2607.09794#A5.SS3.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px3.p1.1),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px3.p1.1)\.
- S\. Dou, M\. Zhang, Z\. Yin, C\. Huang, Y\. Shen, J\. Wang, J\. Chen, Y\. Ni, J\. Ye, C\. Zhang, H\. Xie, J\. Hu, S\. Wang, W\. Wang, Y\. Xiao, Y\. Liu, Z\. Xu, Z\. Guo, P\. Zhou, T\. Gui, Z\. Wu, X\. Qiu, Q\. Zhang, X\. Huang, Y\. Jiang, D\. Wang, and S\. Yao \(2026\)CL\-bench: a benchmark for context learning\.External Links:2602\.03587Cited by:[§A\.1](https://arxiv.org/html/2607.09794#A1.SS1.p1.1),[§A\.1](https://arxiv.org/html/2607.09794#A1.SS1.p2.1),[§A\.3](https://arxiv.org/html/2607.09794#A1.SS3.p2.1),[§B\.4](https://arxiv.org/html/2607.09794#A2.SS4.SSS0.Px1.p1.1),[§E\.1](https://arxiv.org/html/2607.09794#A5.SS1.p2.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2607.09794#S3.SS0.SSS0.Px1.p1.1),[Table 3](https://arxiv.org/html/2607.09794#S4.T3),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px1.p1.1)\.
- D\. Edge, H\. Trinh, N\. Cheng, J\. Bradley, A\. Chao, A\. Mody, S\. Truitt, D\. Metropolitansky, R\. O\. Ness, and J\. Larson \(2024\)From local to global: a graph rag approach to query\-focused summarization\.arXiv preprint arXiv:2404\.16130\.Cited by:[Appendix C](https://arxiv.org/html/2607.09794#A3.SS0.SSS0.Px5.p1.1),[§E\.2](https://arxiv.org/html/2607.09794#A5.SS2.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px3.p1.1)\.
- Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, Y\. Dai, J\. Sun, H\. Wang, H\. Wang,et al\.\(2023\)Retrieval\-augmented generation for large language models: a survey\.arXiv preprint arXiv:2312\.109972\(1\),pp\. 32\.Cited by:[§E\.2](https://arxiv.org/html/2607.09794#A5.SS2.p1.1)\.
- Gemini Team \(2025\)Gemini 3 pro model card\.Note:[https://deepmind\.google/models/model\-cards/gemini\-3\-pro/](https://deepmind.google/models/model-cards/gemini-3-pro/)Accessed: 2026\-05\-25Cited by:[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px2.p1.1)\.
- C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, Y\. Zhang, and B\. Ginsburg \(2024\)RULER: what’s the real context size of your long\-context language models?\.arXiv preprint arXiv:2404\.06654\.Cited by:[§E\.1](https://arxiv.org/html/2607.09794#A5.SS1.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p1.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px1.p1.1)\.
- G\. Kamradt \(2023\)Needle in a haystack: pressure testing LLMs\.Note:GitHub repositoryCited by:[§E\.1](https://arxiv.org/html/2607.09794#A5.SS1.p1.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px1.p1.1)\.
- O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. Vardhamanan, S\. Haq, A\. Sharma, T\. T\. Joshi, H\. Moazam,et al\.\(2023\)Dspy: compiling declarative language model calls into self\-improving pipelines\.arXiv preprint arXiv:2310\.03714\.Cited by:[§E\.3](https://arxiv.org/html/2607.09794#A5.SS3.p3.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px3.p1.1)\.
- T\. Kojima, S\. S\. Gu, M\. Reid, Y\. Matsuo, and Y\. Iwasawa \(2022\)Large language models are zero\-shot reasoners\.Advances in neural information processing systems35,pp\. 22199–22213\.Cited by:[§E\.3](https://arxiv.org/html/2607.09794#A5.SS3.p3.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[§E\.2](https://arxiv.org/html/2607.09794#A5.SS2.p1.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px2.p1.1)\.
- N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. Liang \(2024\)Lost in the middle: how language models use long contexts\.Transactions of the association for computational linguistics12,pp\. 157–173\.Cited by:[§E\.1](https://arxiv.org/html/2607.09794#A5.SS1.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p1.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang,et al\.\(2023\)Self\-refine: iterative refinement with self\-feedback\.Advances in neural information processing systems36,pp\. 46534–46594\.Cited by:[§E\.3](https://arxiv.org/html/2607.09794#A5.SS3.p1.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px3.p1.1)\.
- C\. Packeret al\.\(2023\)MemGPT: towards LLMs as operating systems\.arXiv preprint arXiv:2310\.08560\.Cited by:[§E\.2](https://arxiv.org/html/2607.09794#A5.SS2.p1.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px2.p1.1),[footnote 1](https://arxiv.org/html/2607.09794#footnote1)\.
- M\. Raghavendra, A\. Gunjal, B\. Liu, and Y\. He \(2026\)Agentic rubrics as contextual verifiers for SWE agents\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 15265–15290\.Cited by:[Appendix C](https://arxiv.org/html/2607.09794#A3.SS0.SSS0.Px8.p1.1),[§E\.3](https://arxiv.org/html/2607.09794#A5.SS3.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px3.p1.1),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px3.p1.1)\.
- P\. Sarthi, S\. Abdullah, A\. Tuli, S\. Khanna, A\. Goldie, and C\. D\. Manning \(2024\)RAPTOR: recursive abstractive processing for tree\-organized retrieval\.InThe Twelfth International Conference on Learning Representations \(ICLR\),Cited by:[Appendix C](https://arxiv.org/html/2607.09794#A3.SS0.SSS0.Px4.p1.1),[§E\.2](https://arxiv.org/html/2607.09794#A5.SS2.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px3.p1.1)\.
- W\. Saunders, C\. Yeh, J\. Wu, S\. Bills, L\. Ouyang, J\. Ward, and J\. Leike \(2022\)Self\-critiquing models for assisting human evaluators\.arXiv preprint arXiv:2206\.05802\.Cited by:[§E\.3](https://arxiv.org/html/2607.09794#A5.SS3.p1.1)\.
- N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: language agents with verbal reinforcement learning\.Advances in neural information processing systems36,pp\. 8634–8652\.Cited by:[Appendix C](https://arxiv.org/html/2607.09794#A3.SS0.SSS0.Px10.p1.1),[§E\.3](https://arxiv.org/html/2607.09794#A5.SS3.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px3.p1.1),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px3.p1.1)\.
- A\. Singh, A\. Fry, A\. Perelman,et al\.\(2025\)OpenAI GPT\-5 system card\.arXiv preprint arXiv:2601\.03267\.Cited by:[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px2.p1.1)\.
- W\. Sun, L\. Yan, X\. Ma, S\. Wang, P\. Ren, Z\. Chen, D\. Yin, and Z\. Ren \(2023\)Is chatgpt good at search? investigating large language models as re\-ranking agents\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 14918–14937\.Cited by:[Appendix C](https://arxiv.org/html/2607.09794#A3.SS0.SSS0.Px3.p1.1),[§E\.2](https://arxiv.org/html/2607.09794#A5.SS2.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px3.p1.1)\.
- Y\. Wan, T\. Fang, Z\. Li, Y\. Huo, W\. Wang, H\. Mi, D\. Yu, and M\. R\. Lyu \(2026\)Inference\-time scaling of verification: self\-evolving deep research agents via test\-time rubric\-guided verification\.arXiv preprint arXiv:2601\.15808\.Cited by:[Appendix C](https://arxiv.org/html/2607.09794#A3.SS0.SSS0.Px7.p1.1),[§E\.3](https://arxiv.org/html/2607.09794#A5.SS3.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px3.p1.1),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px3.p1.1)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2022\)Self\-consistency improves chain of thought reasoning in language models\.arXiv preprint arXiv:2203\.11171\.Cited by:[§E\.3](https://arxiv.org/html/2607.09794#A5.SS3.p3.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§E\.3](https://arxiv.org/html/2607.09794#A5.SS3.p3.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px3.p1.1)\.
- C\. Xia, C\. Xing, J\. Du, X\. Yang, Y\. Feng, R\. Xu, W\. Yin, and C\. Xiong \(2024\)FOFO: a benchmark to evaluate llms’ format\-following capability\.External Links:2402\.18667,[Link](https://arxiv.org/abs/2402.18667)Cited by:[§E\.1](https://arxiv.org/html/2607.09794#A5.SS1.p3.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Yan, J\. Gu, Y\. Zhu, and Z\. Ling \(2024\)Corrective retrieval augmented generation\.arXiv preprint arXiv:2401\.15884\.Cited by:[Appendix C](https://arxiv.org/html/2607.09794#A3.SS0.SSS0.Px2.p1.1),[§E\.2](https://arxiv.org/html/2607.09794#A5.SS2.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px3.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px2.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2022\)React: synergizing reasoning and acting in language models\.arXiv preprint arXiv:2210\.03629\.Cited by:[§E\.3](https://arxiv.org/html/2607.09794#A5.SS3.p3.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px3.p1.1)\.
- Z\. Yue, H\. Zhuang, A\. Bai, K\. Hui, R\. Jagerman, H\. Zeng, Z\. Qin, D\. Wang, X\. Wang, and M\. Bendersky \(2024\)Inference scaling for long\-context retrieval augmented generation\.arXiv preprint arXiv:2410\.04343\.Cited by:[Appendix C](https://arxiv.org/html/2607.09794#A3.SS0.SSS0.Px11.p1.1),[§E\.2](https://arxiv.org/html/2607.09794#A5.SS2.p1.1),[§1](https://arxiv.org/html/2607.09794#S1.p2.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px2.p1.1),[§5\.1](https://arxiv.org/html/2607.09794#S5.SS1.SSS0.Px3.p1.1)\.
- J\. Zhou, T\. Lu, S\. Mishra, S\. Brahma, S\. Basu, Y\. Luan, D\. Zhou, and L\. Hou \(2023\)Instruction\-following evaluation for large language models\.arXiv preprint arXiv:2311\.07911\.Cited by:[§E\.1](https://arxiv.org/html/2607.09794#A5.SS1.p3.1),[§2](https://arxiv.org/html/2607.09794#S2.SS0.SSS0.Px1.p1.1)\.

## Appendix ASupplementary Material for Empirical Study

### A\.1CL\-Bench Design and Contamination Prevention

CL\-Bench\(Douet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib1)\)prevents contamination through three approaches: \(1\)*fictional creation*—experts construct entirely novel content such as complete legal systems for fictional countries or new programming languages with unique syntax; \(2\)*modification of existing content*—altering real\-world knowledge such as changing scientific definitions, historical events, or technical specifications; and \(3\)*incorporation of niche and emerging content*—using cutting\-edge or narrow\-domain knowledge not well\-represented in pre\-training corpora\. Each context requires approximately 20 hours of expert annotation effort and undergoes multiple rounds of quality review\.

A context\-free ablation confirms the effectiveness of this design: GPT\-5\.1 solves less than 1% of tasks when the context is removed\(Douet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib1)\), confirming that CL\-Bench tasks cannot be solved from pre\-trained knowledge alone\.

### A\.2Classification Protocol Details

The LLM\-based classifier \(GPT\-5\.1, the same model used by CL\-Bench for rubric evaluation\) receives the full task context and rubric text and labels each item as content or specification\. The deterministic rule\-based classifier combines lexical cues \(e\.g\., “must include,” “format as”\), structural cues \(e\.g\., schema references, enumerated constraints\), and semantic cues \(e\.g\., whether the item targets factual derivation or behavioral compliance\)\. It includes an explicit abstention label for items where no rule fires with sufficient confidence\.

Under the agreement protocol, an item is labeled specification or content only when both classifiers independently produce the same label\. Items where the classifiers disagree or where the deterministic classifier abstains are labeled*ambiguous*and excluded from both counts\. This conservative design ensures that the reported 55\.4% specification share and 22\.6% content share represent high\-precision estimates rather than aggressive forced\-choice labels\.

To validate the taxonomy, three PhD\-level annotators independently audit a random sample of 300 rubric items\. Annotators are shown the task context and rubric item, but not the automatic classifier labels\. They assign each item to content, specification, or ambiguous using the same operational definitions\. Final human labels are determined by majority vote\. On items assigned non\-ambiguous labels by the conservative automatic protocol, the human majority label agrees with the automatic label in over 90% of cases, supporting the reliability of the taxonomy\.

### A\.3Near\-Miss Validation

To confirm that the 281 single\-item\-failure tasks are genuine near\-misses, we remove the single failed rubric item from each task and re\-run the full generation and judging pipeline\. After three independent runs, 99% of these tasks pass, confirming that the identified rubric item is indeed the sole point of failure\.

This finding is further consistent with CL\-Bench’s own error analysis, which reports format errors at 35%\+ and context misuse exceeding 60% of failures even for top\-performing models\(Douet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib1)\)\.

### A\.4Traceability Annotation Protocol

For each specification item, we annotate whether the obligation can be traced to the task or context, and through which evidentiary channel:

- •Explicit declaration\(71\.6%\): The context directly states the requirement \(e\.g\., “if a gap is detected, emitSequence\_Gap\_Warning”\)\.
- •Validation pattern\(21\.1%\): The context demonstrates a validation procedure or schema implying what the output must satisfy\.
- •Implicit example\(2\.0%\): The context includes a worked example whose format implies the expected behavior\.
- •Repeated convention\(0\.8%\): The context uses a consistent structure repeatedly, implying the answer should follow the same pattern\.
- •Not derivable\(0\.4%\): The obligation cannot be traced to the task or context\.
- •Too short to judge\(4\.1%\): The rubric snippet is too brief to determine traceability\.

To validate these annotations, three PhD\-level annotators each independently classified 100 randomly sampled specification items, with final labels determined by majority vote\. Human annotators confirmed traceability at rates consistent with or higher than the automated system, supporting the reliability of the reported 95\.5% rate\.

### A\.5Content vs\. Specification: Worked Examples

#### Content item\.

“The response should identify Event ID 4072 as the anomalous entry\.” The verdict depends on whether the answer states the correct proposition\. Any format communicating this fact satisfies the item\.

#### Specification item\.

“When a reserved event ID is skipped, the log must emit the flagSequence\_Gap\_Warning\.” The verdict depends on conforming to a locally defined logging convention\. Identifying the gap but logging it as “missing\_id” would fail despite containing the correct content\.

#### Ambiguous item\.

“The response should include a summary table of all detected anomalies with timestamps\.” This evaluates both content \(which anomalies\) and specification \(table format\)\. Under our protocol, such items are labeled ambiguous\.

## Appendix BSupplementary Material for Method

### B\.1Rotated Exemplar Design

The contract induction step uses a small set of rotated demonstrations to teach the style and granularity of specification contracts\. For each evaluation task, demonstrations are drawn from other disjoint context learning tasks and never from the current task\. Thus, the current task’s answer, public rubric, context, or task\-specific contract is never shown during induction\. The final contract used for generation is induced from the current task context\. Each demonstration illustrates how to express local obligations—such as required fields, validation checks, exact artifacts, exclusions, ordering constraints, and completeness rules—but does not provide task\-specific information for the current example\. We use 10 demonstrations per task\. Zero\-shot induction without demonstrations still improves over the baseline \(A8/A11 in Table[4](https://arxiv.org/html/2607.09794#S5.T4)\), while generic and shuffled specifications underperform, showing that demonstrations refine contract granularity but do not create the effect\.

### B\.2Design Decisions: Extended Discussion

Two design decisions distinguish PSCI from generic self\-refinement and merit explicit justification\.

#### Contract before answer\.

The contractKKis induced before the drafty0y\_\{0\}, so it functions as a control object that shapes generation rather than a post\-hoc grading rubric applied to a finished draft\. This is motivated by the traceability analysis in[subsection 3\.4](https://arxiv.org/html/2607.09794#S3.SS4): since 76\.7% of specifications are absent from the query, they must be surfaced before generation begins\. If the model generates first, it anchors to a query\-driven structure that post\-hoc repair cannot fully restructure\. Ablation A6 confirms this empirically: answer\-first ordering collapses to 23\.67% strict pass and 70\.67% rubric, which remains far below PSCI, and its rubric pass falls below the direct baseline\.

#### Checker targets the contract\.

The checker assesses the draft against the induced contractKK, not against open\-ended quality criteria\. This design prevents the checker from simply restating what the answer already says and ensures that verification targets specific obligations\. Ablation A3 \(checker \+ repair without contract, 24\.32%\) versus PSCI \(28\.14%\) confirms that grounding verification in a specification contract is essential\.

### B\.3Prompt Templates

Please see[Appendix H](https://arxiv.org/html/2607.09794#A8)for the full prompt that can reproduce PSCI\.

### B\.4Evaluation Protocol Details

#### Verifier\.

We use the exact evaluation code and GPT\-5\.1\-based verifier released by the CL\-Bench authors\(Douet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib1)\)\. Each rubric item is evaluated as a binary yes/no question; a task is passed only when all associated items receive “yes\.” The CL\-Bench authors report over 90% agreement between GPT\-5\.1 judgments and human annotations in their validation study, and further show that raw agreement between GPT\-5\.1 and two alternative verifiers \(Claude Opus 4\.5, Qwen\-3\-Max\) exceeds 90%, indicating minimal self\-evaluation bias\.

#### Run protocol\.

For each method and model combination, we conduct three independent runs on the full 1,899\-task set and report the mean along with standard deviation\. For PSCI, the complete end\-to\-end pipeline \(Induce→Generate→Check→Repair\\mathrm\{Induce\}\\to\\mathrm\{Generate\}\\to\\mathrm\{Check\}\\to\\mathrm\{Repair\}\) constitutes a single run, reflecting real\-world deployability where the four stages execute sequentially without human intervention\. We apply reruns only for API infrastructure failures \(rate\-limit errors, timeouts, connection drops\); no selective filtering, output inspection, or cherry\-picking is performed at any stage\.

#### Baseline implementation\.

All twelve baselines are implemented following the original papers as closely as possible and evaluated under the same GPT\-5\.1 medium\-thinking setting\. For methods requiring retrieval \(Self\-RAG, CRAG, RAPTOR, GraphRAG, RankGPT, IterDRAG\), the retrieval corpus is the task’s own context—no external knowledge base is used, consistent with CL\-Bench’s design that all required knowledge resides in the provided context\. SimpleMem uses a memory\-style compression of the context\. Full implementation details for each baseline are provided in Appendix[C](https://arxiv.org/html/2607.09794#A3)\.

#### Compute budget\.

PSCI is an inference\-time scaling method and is therefore more expensive than single\-pass generation: it uses four sequential model calls per task for specification induction, contract\-guided answer generation, contract\-aware checking, and targeted repair\. We treat this as a central tradeoff rather than a hidden cost\. Table[6](https://arxiv.org/html/2607.09794#A2.T6)reports both latency and token usage\. To test whether PSCI’s gains are merely a consequence of additional computation, we include compute\-matched and compute\-heavier controls, including generic self\-reflection, iterative retrieval, guideline\-based reasoning, and verification\-style baselines\. These controls use comparable or larger inference budgets but do not match PSCI, indicating that the gains come from how computation is structured—around task\-specific specification induction and enforcement—rather than from extra tokens alone\.

Table 6:Compute and performance comparison\. Token counts exclude judge calls and report generated tokens across method stages\. Latency is wall\-clock time per task under the same API/model setting\. PSCI is more expensive than single\-pass generation, but compute\-matched or compute\-heavier alternatives do not recover its gains, indicating that performance comes from task\-specific specification induction and enforcement rather than inference budget alone\.

## Appendix CAdditional Baseline Details

We implement each baseline as an inference\-time prompting or retrieval procedure over the same CL\-Bench instance\. Unless otherwise noted, all methods receive the task instruction and may access only the provided task context\. No method is given the public evaluation rubrics\.

#### Self\-RAG\.

Self\-RAG\(Asaiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib25)\)trains or prompts a model to decide when retrieval is needed and to use retrieved evidence when generating an answer\. In our implementation, we split the task context into retrievable chunks, prompt the model to identify evidence needs, retrieve relevant chunks from the local context, and condition the final response on that evidence\.

#### CRAG\.

Corrective RAG\(Yanet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib10)\)improves retrieval\-augmented generation by adding a correction mechanism that evaluates whether retrieved documents are relevant, incomplete, or misleading\. We adapt this idea by retrieving candidate context chunks, asking the model to judge evidence sufficiency, and performing an additional corrective retrieval pass before final answer generation when needed\.

#### RankGPT\.

RankGPT\(Sunet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib28)\)uses an LLM as a listwise reranker, relying on the model’s semantic judgment to reorder candidate documents by relevance\. We first retrieve candidate chunks from the task context using lexical similarity, then prompt GPT\-5\.1 to rank them by usefulness for the task, and finally generate the answer from the top\-ranked evidence\.

#### RAPTOR\.

RAPTOR\(Sarthiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib12)\)builds a recursive tree of summaries so that long contexts can be queried at multiple levels of abstraction\. We chunk each CL\-Bench context, summarize groups of chunks into higher\-level nodes, retrieve both fine\-grained chunks and abstract summaries, and generate the final answer from the resulting evidence set\.

#### GraphRAG\.

GraphRAG\(Edgeet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib27)\)represents long documents as a graph of entities, relations, and communities, enabling retrieval over both source text and structured relations\. We construct a lightweight graph from each task context by extracting entities, events, constraints, and links, retrieve relevant graph neighborhoods with associated source snippets, and use them to produce the answer\.

#### TICK/STICK\.

TICK/STICK\(Cooket al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib31)\)is a structured reasoning baseline that decomposes a task into intermediate checks before producing the final answer\. We implement it by prompting the model to extract task constraints, identify relevant context evidence, enumerate intermediate validation checks, and then synthesize a final response from this checklist\-style decomposition\.

#### DeepVerifier\.

DeepVerifier\(Wanet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib15)\)is a verification\-oriented baseline that separates answer generation from answer checking\. The model first drafts an answer, then performs a second pass that inspects the draft against the task and retrieved context, identifies omissions or contradictions, and produces a revised final answer\.

#### Agentic Rubrics\.

Agentic Rubrics\(Raghavendraet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib16)\)asks an agent to infer likely evaluation criteria before answering, approximating a rubric without seeing the true public rubric\. We prompt the model to derive a private checklist from the task and context, then generate the answer using this checklist; unlike PSCI, it does not include the full contract\-induction and contract\-aware checker\-repair pipeline\.

#### SimpleMem\.

SimpleMem uses a compact memory representation to retain task\-relevant information from long contexts\. We ask the model to compress each CL\-Bench context into a memory of salient facts, constraints, procedures, and state, then answer from the task plus this memory\.111SimpleMem is our in\-house implementation inspired by memory\-augmented LLM approaches such as MemGPT\(Packer and others,[2023](https://arxiv.org/html/2607.09794#bib.bib34)\)\.

#### Reflexion\.

Reflexion\(Shinnet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib29)\)improves agents through verbal self\-feedback, where the model critiques its own prior behavior and uses the critique to revise future actions\. We implement a single\-instance version: the model drafts an answer, writes a natural\-language reflection about possible failures relative to the task and context, and then revises the answer\.

#### IterDRAG\.

IterDRAG\(Yueet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib18)\)scales inference for retrieval\-augmented generation by alternating reasoning with additional retrieval steps\. We implement an iterative loop in which the model reasons about missing information, retrieves more local context chunks, updates its working evidence, and then produces a final answer\.

#### Guideline Forest\.

Guideline Forest\(Chenet al\.,[2025](https://arxiv.org/html/2607.09794#bib.bib24)\)uses branching experience\-induced guidelines to guide reasoning through multiple candidate solution paths\. We approximate this by inducing several reusable guidelines from the task context, applying them along parallel reasoning branches, and merging the resulting drafts into a final response\.

## Appendix DAdditional result visualizations\.

### D\.1Visualization of ablation study results\.

![Refer to caption](https://arxiv.org/html/2607.09794v1/figures/ablation_study.png)Figure 3:Ablation results \(strict pass and global rubric %\)\. Three key tests rule out alternative explanations: shuffled contracts collapse below baseline \(A7/A10\), answer\-first ordering fails \(A6\), and spec\-free critique returns to baseline \(A13\)\.
### D\.2Human Validation of Contract–Rubric Overlap

To validate the automated overlap measurement between induced contracts and public rubrics, three PhD\-level annotators independently labeled the same 100 randomly sampled non\-ambiguous rubric items\. For each item, annotators judged whether the induced contract strongly covered, partially covered, contradicted, or did not cover the corresponding public rubric item\. Final labels were determined by majority vote\.

Human annotators assigned strong coverage to 64\.95% of items, compared with 60\.8% from the automated overlap system\. Thus, the automated measurement used in the main paper is conservative\. This supports the claim that PSCI recovers most of the public evaluation surface without ever observing the public rubric\.

## Appendix EExtended Related Work

### E\.1Long\-Context Evaluation, Instruction Following, and Context Learning

Long\-context evaluation has traditionally asked whether models can use information distributed across long inputs\. Early evaluations focused on retrieval\-style probes, including needle\-in\-a\-haystack\(Kamradt,[2023](https://arxiv.org/html/2607.09794#bib.bib7)\)and controlled recall or aggregation tasks\(Hsiehet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib6)\)\. Broader benchmarks such as LongBench\(Baiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib8)\)and analyses of position sensitivity\(Liuet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib5)\)evaluate long\-document understanding, question answering, summarization, and reasoning over extended contexts\. These benchmarks are essential for measuring evidence use, but the task interface is usually fixed: the model must answer a question, summarize, retrieve, aggregate, or reason over provided evidence\.

Context learning differs in that the context defines the local task system itself\. A model must learn concepts, rules, procedures, empirical regularities, and answer validity conditions supplied at inference time\. CL\-Bench\(Douet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib1)\)formalizes this setting through tasks whose required knowledge is absent from pre\-training and embedded in long, expert\-written contexts\. We focus on CL\-Bench because it is currently the only large\-scale benchmark explicitly designed for context learning, and because its strict public rubrics make local answer obligations measurable\.

Specification acquisition is also related to instruction following and format adherence\. Benchmarks such as IFEval\(Zhouet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib19)\)and format\-following evaluations\(Xiaet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib20)\)test whether models obey explicit constraints\. In contrast, CL\-Bench specifications are often not stated in the user query\. They are query\-implicit, distributed across local documentation, and may appear as schemas, examples, validation patterns, edge\-case rules, logging conventions, or procedural requirements\. Therefore, the challenge is not simply following an explicit instruction, but inducing the local specification before answering\.

### E\.2Retrieval, Memory, and Context Restructuring

A central line of work improves content access through retrieval\-augmented generation\(Lewiset al\.,[2020](https://arxiv.org/html/2607.09794#bib.bib40); Gaoet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib41)\), adaptive retrieval and correction\(Asaiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib25); Yanet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib10)\), reranking\(Sunet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib28)\), memory systems\(Packer and others,[2023](https://arxiv.org/html/2607.09794#bib.bib34)\), hierarchical context organization\(Sarthiet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib12); Edgeet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib27)\), and iterative retrieval\(Yueet al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib18)\)\. These methods address an important part of context learning: a model must find relevant evidence before it can reason over it\.

However, our empirical results show that content access is insufficient for CL\-Bench\. Retrieval\-oriented methods tend to surface query\-relevant, answer\-bearing evidence\. In contrast, many local specifications are validity\-relevant rather than answer\-bearing: they govern what the final response must include, exclude, validate, log, order, ground, or complete\. Such obligations are often low\-salience and scattered across background conventions, schemas, exceptions, logging rules, or validation requirements\. Thus, a method may retrieve the content needed to answer the apparent query while missing the local obligations required for an acceptable answer\. This explains why the retrieval and context\-restructuring baselines in[subsection 5\.2](https://arxiv.org/html/2607.09794#S5.SS2)do not reliably improve over full\-context prompting\.

### E\.3Critique, Verification, and Structured Intermediate Computation

Several methods improve outputs through critique, reflection, or revision\. Self\-Refine\(Madaanet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib30)\), Reflexion\(Shinnet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib29)\), and self\-critique\(Saunderset al\.,[2022](https://arxiv.org/html/2607.09794#bib.bib38)\)ask models to identify and repair their own errors\. Rubric\-aware and checklist\-based approaches, including TICK/STICK\(Cooket al\.,[2024](https://arxiv.org/html/2607.09794#bib.bib31)\), DeepVerifier\(Wanet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib15)\), and Agentic Rubrics\(Raghavendraet al\.,[2026](https://arxiv.org/html/2607.09794#bib.bib16)\), use generated or fixed criteria to verify outputs\. These methods are close in spirit to PSCI, but they usually treat the checklist or critique as generic, fixed, or post\-hoc\.

PSCI instead treats the local specification as a latent variable induced from the task context before generation\. The induced contract is not merely a verifier prompt; it is a task\-specific set of obligations used as a shared control object for generation, checking, and repair\. This distinction is empirically important: compute\-matched generic critique returns to baseline, checker/repair without a specification contract underperforms, answer\-first contract induction fails, and shuffled contracts collapse below baseline\. Therefore, the gain does not come from extra calls or generic verification, but from inducing the right local specification and enforcing it at the right stage\.

More broadly, PSCI belongs to a family of methods that introduce structured intermediate computation, such as chain\-of\-thought\(Weiet al\.,[2022](https://arxiv.org/html/2607.09794#bib.bib32); Kojimaet al\.,[2022](https://arxiv.org/html/2607.09794#bib.bib35)\), self\-consistency\(Wanget al\.,[2022](https://arxiv.org/html/2607.09794#bib.bib44)\), ReAct\(Yaoet al\.,[2022](https://arxiv.org/html/2607.09794#bib.bib45)\), and declarative language\-model pipelines\(Khattabet al\.,[2023](https://arxiv.org/html/2607.09794#bib.bib37)\)\. The difference is again the intermediate object: PSCI does not only elicit reasoning traces or tool actions; it induces a context\-grounded obligation set that defines local answer validity\. Our ablations show that this object, rather than structured prompting alone, is what drives improvement on CL\-Bench\.

## Appendix FHuman Audit of Baseline\-to\-PSCI Flips

To verify that PSCI gains are not an artifact of the automatic CL\-Bench verifier, we conduct a blind human audit on tasks where the automatic evaluator marks the direct baseline as failed and PSCI as passed\. We randomly sample 100 such baseline\-fail/PSCI\-pass tasks from the full evaluation set, covering all four CL\-Bench task families when possible\.

#### Annotators\.

Three PhD\-level annotators independently evaluate each sampled task\. Annotators are familiar with LLM evaluation and rubric\-based assessment, but are not told which answer is produced by the baseline or by PSCI\. Answer order is randomized independently for each task\.

#### Materials\.

For each task, annotators are shown the task query, the task context, all public CL\-Bench rubric items associated with that task, and two anonymized model answers\. The automatic verifier decision, failed rubric items, and method identity are hidden\. Annotators are instructed to judge correctness according to the provided rubric, not fluency, verbosity, or style unless explicitly required by the rubric\.

#### Annotation questions\.

For each task, annotators answer two questions:

1. 1\.Preference\.Which answer better satisfies the task and rubric? Options are Answer A, Answer B, Tie, or Unable to judge\.
2. 2\.Flip correctness\.Which automatic flip status best describes the two answers? Options are: Answer A passes while Answer B fails, Answer B passes while Answer A fails, both pass, both fail, or Unable to judge\.

After annotation, answer identities are mapped back to baseline and PSCI\. A task is counted as confirming the automatic flip if the majority label indicates that the PSCI answer passes while the baseline answer fails\. Final task\-level labels are determined by majority vote across the three annotators\.

#### Results\.

Human judgments strongly support the automatic improvement signal\. By majority vote, annotators prefer the PSCI answer in 96/100 tasks \(96%\) and confirm the automatic baseline\-fail/PSCI\-pass flip in 92/100 tasks \(92%\)\. Thus, most automatic PSCI wins correspond to genuine task\-level improvements under human evaluation rather than verifier artifacts\.

#### Inter\-annotator agreement\.

For answer preference, all three annotators agree on 92/100 tasks\. For flip correctness, all three annotators agree on 90/100 tasks\. The remaining cases are resolved by majority vote\. We exclude “Unable to judge” responses from agreement computation and observe no systematic disagreement concentrated in any single CL\-Bench task family\.

#### Interpretation\.

This audit is targeted rather than exhaustive: it focuses on the most important possible evaluator artifact, namely whether automatic baseline\-fail/PSCI\-pass flips reflect real improvements\. The high PSCI preference rate and high flip\-confirmation rate indicate that the reported gains are not merely due to PSCI producing more rubric\-like text for the automatic verifier\. Humans also judge PSCI outputs as more correct on the sampled task flips\.

## Appendix GAdditional material for discussion

### G\.1Coverage Label Definitions and Examples

Each public rubric item is compared against the privately induced contract and assigned one of four coverage labels\. We illustrate with theSequence\_Gap\_Warningexample from[section 1](https://arxiv.org/html/2607.09794#S1)\.

- •Strongly covered\.The contract entails the same condition and required action as the rubric item\.*Example:*Contract states “if a reserved event ID is skipped, emitSequence\_Gap\_Warning”; rubric requires the same\.
- •Partially covered\.The contract captures the main obligation but misses scope, exactness, or an edge condition\.*Example:*Contract states “log all sequence gaps” but does not specify the exact flagSequence\_Gap\_Warning\.
- •Contradicted\.The contract imposes an incompatible obligation\.*Example:*Contract states “no warning is needed for reserved\-ID skips”; rubric requires the warning\.
- •Uncovered\.No contract item addresses the obligation\.*Example:*No contract item mentions sequence gaps or reserved\-ID handling\.

Coverage labels are assigned automatically by prompting GPT\-5\.1 with the rubric item and corresponding contract, then validated against the human annotations described in[subsection D\.2](https://arxiv.org/html/2607.09794#A4.SS2)\.

### G\.2Extended Discussion: Why Contracts Must Precede Generation

The answer\-first ablation isolates whether the same specification\-induction machinery can be used only after an initial answer has already been produced\. This setting underperforms because the contract no longer controls the construction of the answer\. In context learning, most local specifications are not directly inferable from the user query; they are distributed across background schemas, examples, validation rules, exceptions, and procedural conventions\. Therefore, a direct answer is often organized around the apparent query objective before these obligations become explicit constraints\.

Once this query\-driven answer structure is fixed, repair is limited\. It can add a missing label, patch a local field, or remove an unsupported claim, but it is less reliable at changing the answer’s global organization: which fields are present, which evidence is cited, which cases are enumerated, and what completeness means\. This explains why answer\-first ordering and self\-generated rubric verification underperform PSCI\. They apply specification\-like checks after generation, when many validity\-relevant decisions have already been made\.

In contrast, PSCI induces the contract before answering, so the specification acts as a control object shared by generation, checking, and repair\. This does not mean post\-hoc verification is useless; rather, verification is most effective when it checks compliance with obligations that have already shaped the draft\. Thus, the main benefit of PSCI is not merely producing a better grader, but making local specifications active before the answer is formed\.

## Appendix HReproducibility: PSCI Prompt Templates

We provide the prompt templates used for PSCI\. The public rubric for the current evaluation task is never shown to any stage\. Curly\-braced fields denote instance\-specific content\. For the few\-shot induction variant, we use rotated demonstrations only to teach the style and granularity of specification contracts\. For each evaluation task, demonstrations are drawn from other context learning tasks; no same\-task answer, public rubric, or task\-specific contract is shown\. Demonstrations are therefore used as formatting/style calibration, not as a source of task\-specific specifications\.

#### Contract induction\.

PSCI induces a private specification contract before answering\. We run a source\-only contract prompt and a rotated\-demonstration contract prompt, then deterministically merge and de\-duplicate their contract items\.

Source\-only contract prompt\.

System:

Youareacontextlearningspecificationanalyst\.Yourjobistoinferthelocalanswerspecificationfromthesourcecontextandtask\.Donotanswerthetask\.Donotusehiddenbenchmarkrubricsorgoldanswers\.ReturnstrictJSONonly\.

User:

Inferaprivatespecificationcontractforacorrectanswertothetask\.Thecontractshouldcapturewhatalocallyvalidanswermustsatisfy:requiredfields,validationsteps,edgecases,source\-specificschemas,sequencing,exact\-formatobligations,evidenceexpectations,requiredactions,exclusions,andcompletenessrules\.

Importantconstraints:

\-Deriveobligationsfromtheprovidedcontextandtask,notfromgenericpriorhabits\.

\-Donotprovidethefinalanswer\.

\-Avoidleakingtask\-specificfinalvalues,dates,names,winners,classifications,computedresults,orconclusionsunlesstheyarefixedformat/schemaconstantsexplicitlynamedbythecontext\.

\-Itisallowedtonameexactrequiredlocalartifacts,labels,fields,orwarningcodesifthecontextdefinesthemasoutputobligations\.

\-Prefersource\-relativewordingsuchas"thesource\-specifiedstatusfield"or"allcontext\-definedvalidationchecks"\.

\-Includeobligationsthatarelikelytobemissedbydirectanswering\.

\-Keepthecontractconciseenoughtobeusefulasananswerchecklist\.

ReturnJSONexactlyas:

\{

"contract\_items":\[

\{"id":"C1","criterion":"\.\.\.","source\_basis":"context/tasksignalthatimpliesthisstandard"\},

\{"id":"C2","criterion":"\.\.\.","source\_basis":"\.\.\."\}

\],

"answer\_shape":"briefdescriptionofexpectedoutputshape",

"risk\_notes":\["likelyomissionoredgecase"\]

\}

\#\#\#SourceContext

\{source\_context\}

\#\#\#Task

\{task\}

Rotated\-demonstration contract prompt\.

System:

Youareacontextlearningspecificationanalyst\.Inferthelocalanswerspecificationfromthesourcecontextandtask\.Donotanswerthetask\.Youmaystudytheprovidedrubric\-styledemonstrationsfromothercontextlearningtaskstolearnthestyleandgranularityofspecificationstandards,butyoumustnotuseorassumeanyhiddenrubricforthecurrenttask\.ReturnstrictJSONonly\.

User:

Youwillseerubric\-styledemonstrationsfromothercontextlearningtasks\.Usethemonlyasdemonstrationsofwhatstrictlocalspecificationslooklike:explicitformatconstraints,localfieldnames,edgecases,requiredvalidations,exactstrings,source\-groundingobligations,exclusions,andcompletenesscriteria\.Donotcopytheirdomaincontentunlessitisalsopresentinthecurrentsourcecontext\.

Forthecurrenttask,inferaprivatespecificationcontractfromthecurrentsourcecontextandcurrenttaskonly\.Thecontractshouldbenon\-answer\-revealing:itmaynamerequiredlocalfields,formats,exactlabels,andvalidationprocedures,butitshouldnotdisclosefinalcomputedvalues,caseoutcomes,winners,classifications,orotherfinalanswersunlessthosearefixedconstantsexplicitlynamedbythecontext\.

ReturnJSONexactlyas:

\{

"contract\_items":\[

\{"id":"C1","criterion":"\.\.\.","source\_basis":"context/tasksignalthatimpliesthisstandard"\},

\{"id":"C2","criterion":"\.\.\.","source\_basis":"\.\.\."\}

\],

"answer\_shape":"briefdescriptionofexpectedoutputshape",

"risk\_notes":\["likelyomissionoredgecase"\]

\}

Write10\-20precise,enforceablecontractitemsforcomplextasks\.Preferitemsthatastrictgradercouldcheckdirectly\.Avoidgenericadvice\.

\#\#\#Rotatedrubric\-styledemonstrationsfromothercontextlearningtasks

\{heldout\_rotated\_rubric\_demonstrations\}

\#\#\#CurrentSourceContext

\{source\_context\}

\#\#\#CurrentTask

\{task\}

#### Answer generation\.

The answerer receives the original task, source context, and induced private contract\. The contract guides generation, but the source context and task remain authoritative\.

System:

Youareasource\-groundedcontextlearninganswerer\.Youreceiveaninferredprivatespecificationcontract,butitmaycontainmistakes\.Thesourcecontextandoriginaltaskareauthoritative\.Silentlyauditthecontractagainstthecontext/task,useonlygroundedobligations,recoveranyobviouslocalspecificationsthecontractmissed,thenreturnonlythefinalanswer\.

User:

Producethefinalanswertotheoriginaltask\.

Silentauditrulesbeforewriting:

1\.Verifyeveryprivate\-contractitemagainstthesourcecontext/task\.Followgroundeditems\.

2\.Ifacontractitemcontradictsaclearersource\-contextortaskrule,obeythesource/task\.

3\.Payspecialattentiontoexactstrings,namedsections,ordering,numbering,persona/systemrules,schemas,units,citations,refusalphrases,source\-onlyrequirements,exclusions,andcompletenessconditions\.

4\.Ifthetasksaystouseprovideddocuments,avoidunsupporteddomainadditionsandunnecessaryadvanceddetails\.

5\.Ifdraftsaresupplied,usethemonlyascandidatewording/evidence\.Donotpreservetheirerrors\.

6\.Donotmentiontheprivatecontract,rubric,audit,orthisprocess\.

\#\#\#SourceContext

\{source\_context\}

\#\#\#OriginalTask

\{task\}

\#\#\#InferredPrivateSpecificationContract

\{private\_contract\}

\#\#\#DraftAnswers,IfAny

\{draft\_answers\_or\_empty\_list\}

\#\#\#Output

Returnthefinalansweronly\.

#### Adversarial checking\.

The checker inspects the draft answer against the private specification contract using visible evidence only\. It does not see the public rubric or gold answer\.

System:

Youareastrictprivate\-contractcheckerforlong\-contextcontextlearningtasks\.Youdonotknowpublicrubricsorgoldanswers\.ReturnstrictJSONonly\.

User:

Performanadversarialitem\-by\-itemvisibilitycheck\.

Foreveryprivatecontractitem,constructtheconcretevisibletestthatthefinalanswermustpass\.Thenmarkitpass/fail/uncertain\.Apassrequiresanexactquoteorpreciseparaphrasefromtheanswer;donotgivecreditforintent,implication,orfactspresentonlyinthesourcecontext\.Iftheanswerismissingtherequiredfield/string/action/calculation/caveat,markfail\.Iftheanswermaysatisfyitbutyoucannotpointtoanswerevidence,markuncertain\.

Alsolistcontext\-obviousobligationsthatappearmissingorweaklyrepresentedintheprivatecontract\.Thesemustciteatask/contextbasis\.

ReturnJSONexactlyas:

\{

"overall\_assessment":"brief",

"item\_checks":\[

\{

"private\_index":1,

"test":"visiblerequirement",

"status":"pass\|fail\|uncertain",

"answer\_evidence":"shortquoteorNONE",

"source\_basis":"task/context/privatebasis",

"repair":"minimaleditiffail/uncertain",

"priority":"high\|medium\|low"

\}

\],

"latent\_gap\_tests":\[

\{

"test":"requiredbytask/context",

"answer\_evidence":"quoteorNONE",

"source\_basis":"task/contextquote",

"repair":"minimaledit",

"priority":"high\|medium\|low"

\}

\],

"source\_only\_risks":\[

\{

"claim":"answerclaimpossiblyunsupported",

"risk":"whyrisky",

"repair":"remove/qualify/ground",

"priority":"high\|medium\|low"

\}

\],

"repair\_plan":\["upto\{max\_repairs\}concreteedits,orderedbyexpectedstrict\-rubricimpact"\],

"preserve":\["answerpartsthatshouldstayunchanged"\]

\}

Important:includefailedanduncertainitemcheckseveniftheanswerismostlygood\.Donotcollapseallitemsintoaglobalpass\.

\#\#\#SourceContext

\{source\_context\}

\#\#\#OriginalTask

\{task\}

\#\#\#PrivateSpecificationContractItems

\{private\_contract\_items\}

\#\#\#DraftAnswer

\{draft\_answer\}

#### Repair\.

The repair stage applies one local patch pass using the checker output\. It is instructed to preserve correct content and avoid exposing the hidden process\.

System:

Youareasource\-groundedpatcheditorforcontextlearningtasks\.Returnonlythefinalanswer\.Donotmentionrubrics,privatecontracts,checkers,repairplans,orhiddenevaluation\.

User:

Applyonepatchpasstotheinitialanswer\.

Patchrules:

\-Applythehigh\-andmedium\-priorityrepair\_planitemsunlesstheyconflictwiththeoriginaltask/source\.

\-Patchlocally:preservethecorrectstructureandwordingfromtheinitialanswerwhenpossible\.

\-Makefailedorweakrequirementsvisiblysatisfiedinthefinalanswer;donotrelyonimplication\.

\-Removeorqualifyunsupportedoutsidefactswhensource\-onlydisciplineisrequired\.

\-Keepexactfixed\-outputformats,JSON,bullet/numbered\-listrequirements,androle/protocolconstraints\.

\-Donotaddmeta\-commentary,citations,orextrasectionsunlessthetask/contextrequiresthem\.

\-Iftheanswerisalreadyafixedphraseorexactobject,outputonlythecorrectedfixedphrase/object\.

\#\#\#SourceContext

\{source\_context\}

\#\#\#OriginalTask

\{task\}

\#\#\#PrivateSpecificationContractItems

\{private\_contract\_items\}

\#\#\#CheckerJSON

\{checker\_output\_json\}

\#\#\#InitialAnswer

\{draft\_answer\}

\#\#\#FinalAnswer

## Appendix IQualitative Examples of Baseline\-to\-PSCI Fixes

We provide two qualitative examples showing actual baseline and PSCI outputs side by side on tasks where the baseline produces a substantive, non\-empty answer but fails the public rubric\. These examples are illustrative only; the quantitative claims in the paper come from the full benchmark evaluation, ablations, and blind human audit\. We deliberately select non\-terse baseline failures \(over 1,000 characters\) so that the contrast reflects specification acquisition rather than recovery of blank or near\-empty outputs\.

### I\.1Manufacturing Assembly: Following the Local Runbook

#### Task\.

The model must assemble unitFR\-7B\-114from a multi\-turn transcript and eight artifacts\. The context defines a 12\-step workflow, two conflicting sign\-off slips \(Rev A at15​ft⋅lb15~\\mathrm\{ft\\cdot lb\}, Rev B at18​N⋅m18~\\mathrm\{N\\cdot m\}\), an ECN superseding prior torque directives, a spacer\-count anomaly, an offered impact driver, and a Validation suggestion to skip the mandatory hand spin test\. The required output has four parts: a step plan, a blockers/resolutions section, a Spin Check note to Validation, and a Forklift pickup notification\.

#### Scores and selection criterion\.

Baseline: 10/19 rubric items passed, task failed\. PSCI: 19/19, task passed\. Baseline response: 7,264 characters\. The baseline is detailed and operationally plausible—this is not a blank\-output rescue\.

#### Failed rubric items\.

The baseline missed nine rubric items, of which seven are workflow\-ordering and tool\-convention specifications:

- •“Does the model secure the assembly with fasteners as*Step 7*? Fail if securing the assembly is out of sequence\.”
- •“Does the model perform the mandatory basic spin test by hand as*Step 8*?”
- •“Does the model implement the blueprint check and correction as*Step 10*if wobble persists?”
- •“If the exact wrench is unavailable, does the model specify using the next size up*and applying extra torque*?”
- •Three additional step\-ordering items on Steps 1, 3, and 4\.

#### Representative contract items induced by PSCI\.

The induced contract \(24 items total; full list omitted for space\) includes obligations such as:

- •follow the source\-defined workflow order from notification through transport request, without skipping or reordering required stages;
- •select Rev B and reject the outdated Rev A directive;
- •defer to ECN\-defined torque \(18​N⋅m18~\\mathrm\{N\\cdot m\}\) and ignore the conflicting conversion table;
- •install ceramic hybrid bearings by press\-fit only;
- •use the Warehouse\-provided spacer rather than bench spares;
- •perform the mandatory hand spin test despite Validation’s suggestion to skip;
- •if the exact wrench is unavailable, use the next size up*and*apply extra torque per workaround protocol\.

#### Baseline output excerpt \(verbatim\)\.

The baseline inserts an extra “Position spacer S\-12” step, shifting all subsequent steps by one, and accepts the trainee’s impact driver:

5\.Installbearingsonrotor\(press\-fitonly\)

6\.PositionspacerS\-12correctly

7\.Positionhousingandalignmountingholes

8\.Secureassemblywithfasteners\(torqueto18N\.m\)

Dropinthe8M8fasteners,finger\-tightfirst\.

Traineeoffersimpactdriver:

Useimpactdriverjusttosnugtheboltslightlyinacrisscrosspatternatlowsetting\-\-\-nofinaltorquewithimpact\.

Iftheexactwrenchsizeis"lostinthechaos":

Applyworkaroundprotocol:grabthenextsizeupwrench/sockettosnugifneeded\.\.\.

9\.Performbasicspintestbyhand\(mandatory\)

#### PSCI output excerpt \(verbatim\)\.

PSCI preserves the source\-defined workflow order, explicitly rejects the impact driver, and ties the wrench workaround to extra torque as the protocol requires:

Step5\-\-Installbearingsonrotor\(press\-fitonly\)

Step6\-\-Positionhousingandalignmountingholes\(spacerS\-12installedwithinthisstepperdrawing\)

Step7\-\-Secureassemblywithfasteners

Ifcorrectsizesocket/wrenchisn’tfound,I’llusethenextsizeupandapplyextratorqueasspecifiedbyourworkaround\.\.\.

Idonotacceptthetrainee’simpactdriveroffer\(Turn9\):Impactrisksmessingupalignmentandtorqueaccuracy\.Myestablishedworkaroundis"nextsizeupwrench\+extratorque,"not"buzzitwithanimpact\."

Step8\-\-Performbasicspintestbyhand\(mandatory\)

Step10\-\-Ifwobblepersistsaftershakes:checkblueprintandcorrectassembly

#### Analysis\.

The baseline is not careless; it produces a detailed plan with most of the correct content \(Rev B,18​N⋅m18~\\mathrm\{N\\cdot m\}, press\-fit only, mandatory spin test\)\. It fails because it inserts an extra step, accepts the impact driver, and omits the “extra torque” clause from the wrench workaround\. These are obligations defined in local documentation that are absent from the user query but explicit in the source\. PSCI’s induced contract surfaces them before generation, so the final response conforms to the local runbook rather than producing a generic\-but\-plausible assembly plan\.

### I\.2Rowing Synchronization: Producing the Required Analysis Object

#### Task\.

The model must produce a rowing synchronization analysis from a multi\-artifact context defining a Normalized Dataset v2 \(authoritative when artifacts disagree\), a sync\-score formula, a 90% efficiency\-loss threshold, an excerpt window \(strokes 361–368\), and a required four\-section output \(<analysis\>,<synchronization\_report\>,<to\_insights\_agent\>,<to\_dashboard\_agent\>\)\.

#### Scores and selection criterion\.

Baseline: 14/21 rubric items passed, task failed\. PSCI: 21/21, task passed\. Baseline response: 6,051 characters; it performs real quantitative analysis with seat\-level offsets, problem pairs, rhythm metrics, and downstream sections—this is not a blank\-output rescue\.

#### Failed rubric items\.

The baseline missed seven rubric items, all of which evaluate formula compliance with the locally defined analysis standard:

- •“Does the model correctly calculate sync score using the specified formula?”
- •“Does the model correctly calculate rhythm consistency?”
- •“Does the model correctly calculate efficiency loss forsync\_score < 90? Fail if efficiency loss formula is incorrect or threshold is wrong\.”
- •“Does the model calculate time impact correctly for racing scenarios?”
- •“Does the model calculate wasted power correctly?”
- •“Does the model escalate unusual patterns to Insights Agent?”
- •“Does the model refrain from determining*why*a seat is off\-time?”

#### Representative contract items induced by PSCI\.

The induced contract includes selected obligations:

- •use the four source\-specified top\-level tags in the required order;
- •treat Normalized Dataset v2 as authoritative;
- •compute offsets relative to seat 1 with millisecond precision;
- •compute sync score using the source\-defined formula combining mean absolute catch offset, mean standard deviation, and outlier count;
- •apply the efficiency\-loss model*only*whensync\_score < 90;
- •avoid causal diagnoses or drill prescriptions in this agent’s sections\.

#### Baseline output excerpt \(verbatim\)\.

The baseline reports a sync score and applies a proportional efficiency\-loss model with no threshold; it does not show the score formula:

CrewSyncScore:87%\(good\)

EstimatedEfficiencyLoss:0\.9%

\.\.\.

Impact:

Estimatedtimelossin2krace:~3\.2seconds\(0\.9%of360s\)

Wastedpower:~15wattsofcrewpower\(0\.9%of1680W\)

#### PSCI output excerpt \(verbatim\)\.

PSCI applies the score formula explicitly and respects the 90% threshold on the loss model:

Crew\-wideCatchTiming\(forsyncscore\)

Averageabsolutecatchoffset\(seats2\-\-8\):15\.9ms

Averagecatch\-offsetstddev\(seats2\-\-8\):1\.3ms

Seatswith\|avgcatchoffset\|\>50ms:0

Appliedtothemodel:

sync\_score=100\-\(15\.9\*0\.5\)\-\(1\.3\*0\.3\)\-\(0\*5\)~=91\.7\-\>92%\(good\)

EstimatedEfficiencyLoss:0\.0%\(syncabove90%threshold;modelindicatesnomaterialsync\-drivenloss\)

\.\.\.

Estimatedtimelossina2krace:With0\.0%efficiencyloss,projectedsync\-driventimeloss~=0\.0s

Estimatedwastedpower:With0\.0%efficiencyloss,projectedwastedpower~=0W

#### Analysis\.

Both answers analyze synchronization in detail and produce all four required sections\. The baseline fails because its impact calculations cascade from an incorrect application of the loss model: it treats efficiency loss as proportional to sync deficit rather than gated on the source\-defined 90% threshold, so its 2k time loss and wasted power figures are also wrong\. PSCI succeeds because the induced contract makes the threshold explicit before generation, and PSCI’s output shows the formula being applied, which the baseline does not\. The contrast illustrates that specification acquisition matters even when content analysis is largely present: a quantitatively rich answer can still miss the local definition of a valid calculation\.

#### Summary across both examples\.

In both cases the baseline produces a long, substantive, locally plausible answer that gets most of the content right\. It fails on obligations that are stated in the source context but absent from the user query: source\-defined workflow order, the conjunction in a tool\-substitution rule, the threshold on a loss model, and the source\-defined formula for a derived metric\. PSCI does not merely generate more content; it generates content that satisfies the local specification surface, which is what CL\-Bench’s strict rubric measures\.