MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoning

# MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoning
Source: [https://arxiv.org/html/2607.07391](https://arxiv.org/html/2607.07391)
\\nameCharbel Al Bateh\\emailcharbel\.albateh@lau\.edu \\addrDepartment of Electrical and Computer Engineering Lebanese American University Byblos, Lebanon\\nameSamer Saab Jr\.\\emailsamer\.saabjr@lau\.edu\.lb \\addrDepartment of Electrical and Computer Engineering Lebanese American University Byblos, Lebanon

###### Abstract

Mathematical reasoning benchmarks typically provide all facts needed to solve each problem, while interactive benchmarks often mix reasoning with tools, retrieval, and long\-horizon dialogue\. We introduceMIRA\-Math, a benchmark for a narrower diagnostic capability: solving mathematical problems whose full latent state has a unique answer, but whose solver\-facing view is missing exactly one necessary atomic fact\. The solver must request the missing information in natural language under a strict budget and then integrate the returned fact into an exact final answer\. A fixed constrained LLM responder sees only the dataset\-provided atomic fact and must eitherofferthe quoted fact when the request matches it, ordeclineotherwise\. Thus, instance generation, typed hint specifications, validation, and final\-answer verification are deterministic, while request metrics are measured under a fixed LLM\-mediated responder channel\.MIRA\-Mathcontains 2,310 generated instances from 22 typed mathematical families spanning algebra, probability, linear systems, discrete structures, signal processing, Markov chains, circuits, interpolation, and numerical boundary\-value problems\. Experiments across frontier and small models show that request success and final\-answer accuracy are separable: models may ask for the right fact yet fail the downstream computation, or fail before obtaining the canonical hint\. We release generators, verifiers, prompts, run metadata, and dataset documentation to support reproducible evaluation of minimal information requesting in mathematical reasoning\.

Keywords:benchmark, dataset generation, mathematical reasoning, information acquisition, clarification, partial observability, large language models

## 1Introduction

Large language models are increasingly evaluated on mathematical reasoning, tool use, and interactive problem solving\. Yet many high\-profile math benchmarks assess only the final answer under full information\(Cobbeet al\.,[2021](https://arxiv.org/html/2607.07391#bib.bib14); Hendryckset al\.,[2021](https://arxiv.org/html/2607.07391#bib.bib15); Balunovićet al\.,[2025](https://arxiv.org/html/2607.07391#bib.bib16)\), while broad interactive benchmarks mix multiple sources of difficulty, including tool selection, retrieval, web navigation, external APIs, and long\-context management\(Mialonet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib11); Liuet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib12); Qinet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib13)\)\. These settings are valuable, but they make it difficult to isolate a basic bottleneck that appears whenever a model does not initially possess all relevant information: can the model recognize the missing fact, ask for that fact precisely, and use the answer correctly?

MIRA\-Mathis designed to isolate this capability\. Each benchmark instance is generated from a complete mathematical state with a unique answer, but the solver model receives only a private view that is deliberately insufficient, missing exactly one necessary atomic fact\. A fixed information\-holder model receives only the missing fact and is constrained by a structured\-output protocol: if the solver’s request semantically matches the information it holds, it returns that fact; otherwise, it returns a declination stating that it does not have the requested information\. The information holder does not solve, explain, negotiate, or volunteer extra hints\. The benchmark therefore measures minimal information requesting under a controlled responder channel, not open\-ended multi\-agent collaboration\.

This framing is intentionally data\-centric\. Following the emphasis on public benchmark infrastructure and responsible dataset design in recent data\-centric machine\-learning work\(Oalaet al\.,[2024](https://arxiv.org/html/2607.07391#bib.bib1); Orr and Crawford,[2024](https://arxiv.org/html/2607.07391#bib.bib2)\),MIRA\-Mathincludes generator code, exact verifiers, canonical hint specifications, prompt templates, run metadata, and dataset documentation\. The benchmark is closest to diagnostic benchmark datasets that expose hidden structure or evaluation gaps through carefully controlled data construction\(Matsubaraet al\.,[2024](https://arxiv.org/html/2607.07391#bib.bib3); Zhanget al\.,[2024a](https://arxiv.org/html/2607.07391#bib.bib4),[b](https://arxiv.org/html/2607.07391#bib.bib5); Telyatnikovet al\.,[2025](https://arxiv.org/html/2607.07391#bib.bib6)\), but its target capability is different: information acquisition under partial mathematical observability\.

#### Contributions\.

This paper makes three contributions:

- •We define a reproducible protocol for minimal information requesting in mathematical reasoning, with typed atomic hints, a fixed constrained LLM responder using structuredoffer/declineoutputs, deterministic dataset validation, and exact answer checking\.
- •We report baseline evaluations across multiple solver models and prompt regimes, over a benchmark of 2,310 instances showing that first\-request success, request hit rate, and final accuracy capture distinct failure modes\.
- •We provide dataset documentation, intended\-use guidance, and a maintenance plan\.

## 2Related Work

MIRA\-Mathis positioned at the intersection of mathematical reasoning benchmarks, clarification and information\-acquisition benchmarks, active reasoning under incomplete information, and data\-centric benchmark design\. Table[1](https://arxiv.org/html/2607.07391#S2.T1)summarizes the closest literature streams and the distinction betweenMIRA\-Mathand prior work\.

Table 1:Positioning ofMIRA\-Mathrelative to nearby benchmark streams\.StreamTypical focusHowMIRA\-MathdiffersFull\-information math and diagnostic benchmarks\(Cobbeet al\.,[2021](https://arxiv.org/html/2607.07391#bib.bib14); Hendryckset al\.,[2021](https://arxiv.org/html/2607.07391#bib.bib15); Balunovićet al\.,[2025](https://arxiv.org/html/2607.07391#bib.bib16); Matsubaraet al\.,[2024](https://arxiv.org/html/2607.07391#bib.bib3)\)Solve fully specified problems, often with exact final\-answer scoring or controlled generators\.The solver’s private view is intentionally incomplete; success requires requesting and then using a missing atomic fact\.Clarification and incomplete\-information reasoning\(Minet al\.,[2020](https://arxiv.org/html/2607.07391#bib.bib19); Ganet al\.,[2024](https://arxiv.org/html/2607.07391#bib.bib18); Liet al\.,[2025](https://arxiv.org/html/2607.07391#bib.bib17); Zhouet al\.,[2025](https://arxiv.org/html/2607.07391#bib.bib29); Huang and others,[2025](https://arxiv.org/html/2607.07391#bib.bib28)\)Ask useful questions or detect missing information in underspecified tasks\.MIRA\-Mathuses typed mathematical families, a fixed constrained information holder, and exact verification of final mathematical integration\.Agent and tool\-use benchmarks\(Mialonet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib11); Liuet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib12); Qinet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib13); Zhouet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib32)\)Evaluate web navigation, tools, retrieval, APIs, and long\-horizon workflows\.MIRA\-Mathremoves tools and environment management to isolate minimal mathematical information requesting\.Multi\-agent communication\(Foersteret al\.,[2016](https://arxiv.org/html/2607.07391#bib.bib20); Sukhbaataret al\.,[2016](https://arxiv.org/html/2607.07391#bib.bib21); Daset al\.,[2019](https://arxiv.org/html/2607.07391#bib.bib22); Wanget al\.,[2020](https://arxiv.org/html/2607.07391#bib.bib23); Wuet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib24); Liet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib25); Chenet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib26)\)Study coordination and message passing under partial observability\.The responder is a fixed constrained information holder, not a strategic teammate, so failures are more localized to request precision, responder matching, or mathematical integration\.Data\-centric benchmark design\(Oalaet al\.,[2024](https://arxiv.org/html/2607.07391#bib.bib1); Orr and Crawford,[2024](https://arxiv.org/html/2607.07391#bib.bib2); Gebruet al\.,[2021](https://arxiv.org/html/2607.07391#bib.bib9); Mitchellet al\.,[2019](https://arxiv.org/html/2607.07391#bib.bib10)\)Emphasize public artifacts, documentation, intended use, and reproducible evaluation\.MIRA\-Mathreleases generators, verifiers, typed hint schemas, responder specifications, prompts, run metadata, and documentation\.

### 2\.1Final\-answer mathematical reasoning is not enough

Mathematical reasoning benchmarks such as GSM8K and MATH have been central for measuring whether language models can produce correct final answers from complete problem statements\(Cobbeet al\.,[2021](https://arxiv.org/html/2607.07391#bib.bib14); Hendryckset al\.,[2021](https://arxiv.org/html/2607.07391#bib.bib15)\)\. More recent evaluations increase difficulty, reduce contamination, or track competition\-level performance\(Balunovićet al\.,[2025](https://arxiv.org/html/2607.07391#bib.bib16)\)\. These benchmarks are important, but they usually assume that the model receives all information required to solve the problem\. As a result, they do not isolate the ability to notice that a problem is underdetermined, identify the missing fact, ask for it, and then use it correctly\.

MIRA\-Mathtargets this missing capability directly\. It should therefore not be read as a replacement for full\-information mathematical benchmarks\. Instead, it complements them by turning information acquisition into an explicitly measured step\. A model that is strong on ordinary math benchmarks may still fail inMIRA\-Mathif it cannot identify the missing atomic fact or if it asks for an irrelevant quantity\.

### 2\.2Controlled diagnostic benchmarks

A second relevant stream uses controlled generation and exact evaluation to reveal hidden model weaknesses\. Symbolic and scientific benchmark work has shown the value of generator\-controlled tasks, exact verifiers, and diagnostic decompositions\(Matsubaraet al\.,[2024](https://arxiv.org/html/2607.07391#bib.bib3); Somasekharanet al\.,[2026](https://arxiv.org/html/2607.07391#bib.bib8)\)\. Data\-centric benchmark papers similarly emphasize that the dataset, generator, evaluation protocol, and documentation are part of the scientific contribution\(Zhanget al\.,[2024a](https://arxiv.org/html/2607.07391#bib.bib4),[b](https://arxiv.org/html/2607.07391#bib.bib5); Telyatnikovet al\.,[2025](https://arxiv.org/html/2607.07391#bib.bib6)\)\.

MIRA\-Mathfollows this diagnostic tradition but uses controlled generation for a different purpose\. Each instance is generated from a latent complete mathematical state that fixes a unique answer, while the solver’s private view omits a necessary atomic fact\. This global\-well\-posed/local\-underdetermined construction lets us evaluate not only whether the model eventually gets the answer right, but also whether it requested the correct missing information along the way\.

### 2\.3Clarification and information acquisition

Clarification benchmarks study whether models ask useful questions when instructions, dialogue states, or user requests are ambiguous or underspecified\(Minet al\.,[2020](https://arxiv.org/html/2607.07391#bib.bib19); Ganet al\.,[2024](https://arxiv.org/html/2607.07391#bib.bib18); Zhanget al\.,[2025](https://arxiv.org/html/2607.07391#bib.bib31); Zhaoet al\.,[2026](https://arxiv.org/html/2607.07391#bib.bib30)\)\. This line of work is closely related in motivation in that practical assistants should not always guess when required information is absent\. However, many clarification settings involve subjective ambiguity, natural human preferences, or dialogue\-level uncertainty\. These properties are valuable for realism but make it difficult to assign exact credit for a particular request\.

MIRA\-Mathreduces this ambiguity by construction\. The full latent state has a unique target answer, and the missing information is represented by typed atomic hints specified by the generator\. The solver may still ask in free\-form natural language, but the response channel is constrained\. A fixed information\-holder LLM sees only its private constraints and must choose between a structuredofferand a structureddecline\. The accepted/declined events are therefore logged in an auditable schema, while the semantic matching decision itself remains LLM\-mediated\. The deterministic components are the generator invariants, canonical hint specification, instance validation, and final\-answer verification\.

### 2\.4Active and missing\-information reasoning benchmarks

The closest recent work studies reasoning under incomplete information\. QuestBench formalizes underspecified reasoning as missing variable assignments and evaluates whether models can identify the minimal necessary question in logic, planning, and math\-derived tasks\(Liet al\.,[2025](https://arxiv.org/html/2607.07391#bib.bib17)\)\. AR\-Bench broadens the setting to active reasoning, where models must ask questions to acquire missing evidence in interactive environments\(Zhouet al\.,[2025](https://arxiv.org/html/2607.07391#bib.bib29)\)\. CRITIC\-math focuses directly on incomplete mathematical problems and evaluates whether large reasoning models proactively ask for information rather than hallucinating an answer\(Huang and others,[2025](https://arxiv.org/html/2607.07391#bib.bib28)\)\.

MIRA\-Mathis closest to this family of benchmarks, but it makes a different design choice\. QuestBench emphasizes selecting the right clarification from a finite set of options;MIRA\-Mathrequires open\-ended natural\-language requests that must be precise enough to be accepted by a fixed constrained information\-holder LLM\. AR\-Bench emphasizes broader active reasoning in interactive scenarios;MIRA\-Mathrestricts the environment to a single controlled mathematical information channel\. CRITIC\-math studies whether models detect incompleteness in mathematical problems;MIRA\-Mathadditionally evaluates whether the model identifies the exact missing atomic slot and integrates the returned value into an exact solution\. These differences makeMIRA\-Matha narrower but more controlled diagnostic for minimal mathematical information requesting\.

### 2\.5Agent, tool\-use, and communication benchmarks

Broad agent benchmarks evaluate capabilities such as tool selection, API use, retrieval, web navigation, and long\-horizon planning\(Mialonet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib11); Liuet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib12); Qinet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib13); Zhouet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib32)\)\. LLM multi\-agent frameworks and multi\-agent reinforcement\-learning work further study how agents coordinate and communicate under partial observability\(Foersteret al\.,[2016](https://arxiv.org/html/2607.07391#bib.bib20); Sukhbaataret al\.,[2016](https://arxiv.org/html/2607.07391#bib.bib21); Daset al\.,[2019](https://arxiv.org/html/2607.07391#bib.bib22); Wanget al\.,[2020](https://arxiv.org/html/2607.07391#bib.bib23); Wuet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib24); Liet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib25); Chenet al\.,[2023](https://arxiv.org/html/2607.07391#bib.bib26)\)\. These settings are useful, but they conflate information acquisition with many other sources of difficulty\.

MIRA\-Mathintentionally avoids this conflation\. The second role is not an autonomous collaborator and does not solve, negotiate, explain, or strategically communicate\. It is a fixed LLM\-mediated information holder over dataset\-provided atomic hints: the holder sees only its private constraints, must quote an explicit constraint when offering information, and otherwise returns a structured declination\. This design makesMIRA\-Matha benchmark of request precision and mathematical integration, not a benchmark of general multi\-agent collaboration\.

### 2\.6Data\-centric benchmark design

Finally,MIRA\-Mathis a data\-centric benchmark contribution\. Work on dataset documentation, model cards, and responsible benchmark design emphasizes the need to specify intended uses, limitations, quality\-control procedures, and maintenance plans\(Gebruet al\.,[2021](https://arxiv.org/html/2607.07391#bib.bib9); Mitchellet al\.,[2019](https://arxiv.org/html/2607.07391#bib.bib10); Orr and Crawford,[2024](https://arxiv.org/html/2607.07391#bib.bib2); Oalaet al\.,[2024](https://arxiv.org/html/2607.07391#bib.bib1)\)\. Following this view, theMIRA\-Mathrelease is intended to include generatedJSONLfiles, family generators, exact verifiers, canonical hint specifications, prompt templates, run metadata, raw logs, and dataset documentation\. The benchmark’s contribution is therefore not only a collection of problems, but a reproducible evaluation protocol for a specific capability: asking for the minimal missing mathematical fact needed to solve an underdetermined problem\.

## 3Benchmark Design

### 3\.1Task Formalization

AMIRA\-Mathinstance is generated from a latent complete statezz\. The full state determines a unique target answery=f​\(z\)y=f\(z\)\. The solver receives a private viewvA=ϕA​\(z\)v\_\{A\}=\\phi\_\{A\}\(z\)that omits one necessary atomic fact\. The hint source stores that atomic hinth∈H​\(z\)h\\in H\(z\), each with a typed identifier such as a missing coefficient, boundary value, congruence, transition probability, or interpolation point\. The benchmark enforces two conditions:

global well\-posedness:\|\{y:y​is consistent with​vA∪H​\(z\)\}\|\\displaystyle\\left\|\\\{y:y\\text\{ is consistent with \}v\_\{A\}\\cup H\(z\)\\\}\\right\|=1,\\displaystyle=1,\(1\)local underdetermination:\|\{y:y​is consistent with​vA\}\|\\displaystyle\\left\|\\\{y:y\\text\{ is consistent with \}v\_\{A\}\\\}\\right\|\>1\.\\displaystyle\>1\.\(2\)

### 3\.2Interaction Protocol

At evaluation time the solver is shown its private view, the target question, and the request budgetBB\. At each turn the solver may either issue a request or provide a final answer\. Requests are free\-form natural language\. A fixed information\-holder model receives its own private view containing the missing atomic fact and must respond using a structured schema\. If the request semantically matches the fact it holds, the responder returns that fact; otherwise, it returns a declination and the solver may try again until the request budget is exhausted\.

For example, in a recurrence instance the solver may see the recurrence rule anda​\(0\)a\(0\)but nota​\(1\)a\(1\)\. A request such as “What is the missing value ofa​\(1\)a\(1\)?” maps to the canonical initial\-condition hint and receives the stored value\. A vague request such as “Can you help me solve the recurrence?” is declined because it does not identify an allowed atomic hint\. In a variable\-slot family, such as a linear system with one missing coefficient, the request must identify the missing coefficient slot rather than merely ask for “the missing information\.”

![Refer to caption](https://arxiv.org/html/2607.07391v1/Workflow.jpg)Figure 1:Overview of theMIRA\-Mathprotocol, illustrated with a successfulcircuit\_missing\_resistancetrace\. Agent A receives a private view with one atomic fact hidden, asks for the missing slot in natural language, and receives either a structuredofferordeclinefrom a fixed constrained information\-holder that sees only its private constraint\. After receiving the hint, Agent A must still compute the final answer and pass the family\-specific exact verifier\.This constrained responder channel is a central design choice\. In an unconstrained two\-model dialogue, the responder could solve part of the problem, volunteer extra information, negotiate, or provide explanations\.MIRA\-Mathremoves these behaviors by giving the information holder only its private constraints and requiring a structured offer\-or\-decline response\. The resulting request metrics should be interpreted as measuring the solver’s information\-requesting behavior under a controlled LLM responder, not as a measure of general multi\-agent teamwork or a purely deterministic request\-matching oracle\.

#### Responder implementation\.

In the released reference runner, the information holder is implemented as a fixed LLM responder with a flat structured\-output schema\. Its only valid outputs areoffer, withhas\_exact\_match=true, a quoted private constraint, and a hint string; ordecline, with a fixed declination message\. If the structured output cannot be parsed, the runner attempts conservative raw\-output extraction and otherwise records a declination\. This design prevents the responder from solving the task, volunteering extra information, or giving partial clues, but it does not make semantic matching itself deterministic\. Consequently, request metrics should be interpreted as solver performance under this fixed constrained responder channel\. Exact mathematical correctness is still evaluated by deterministic family\-specific verifiers\.

### 3\.3Family Types

The 22 families are partitioned into two types\.Type Afamilies have a fixed hint slot: the missing information is structurally determined by the family\. For example, a Bayes\-rule instance always withholds the prior, and a recurrence instance always withholds one initial condition\.Type Bfamilies have a variable hint slot: the missing position changes by instance, such as a missing transition probability, boundary value, coefficient, interpolation point, resistor, or grid cell\. Type B therefore requires the solver to locate which slot is missing before phrasing the request\.

This design supports diagnosis of two separate subskills\. Type A primarily tests whether the model knows what kind of fact is needed and can compute after receiving it\. Type B additionally tests whether the model can inspect its local view, identify the absent slot, and request that slot precisely\.

### 3\.4Metrics

We report the following metrics\.

- •Accuracy: fraction of instances whose final answer normalizes to the ground truth under type\-aware comparison\.
- •Request hit rate: mean per\-instance ratio of accepted requests to fixed\-responder messages,offers/\(offers\+declines\)\\text\{offers\}/\(\\text\{offers\}\+\\text\{declines\}\)\.
- •First\-request success: fraction of instances in which the solver’s first request receives a structured offer from the fixed responder\.
- •Average requests: mean number of requests the solver issues per instance\.
- •Average declinations: mean number of declinations from the fixed responder per instance\.
- •Average hints: mean number of accepted offers the solver receives per instance\.
- •Average tokens: approximate transcript cost, computed as mean transcript word\-count × 1\.3 per instance and summed across the solver and responder messages\.

#### Trace\-level decomposition\.

To separate information\-acquisition failures from downstream mathematical failures, we also classify each transcript using three binary indicators\. LetOiO\_\{i\}indicate that instanceiireceived at least one structuredoffer; letCiC\_\{i\}indicate that an offered hint matches the instance’s canonical atomic\-hint specification under an offline audit of the quoted constraint and machine\-readable hint fields; and letFiF\_\{i\}indicate that the final answer is accepted by the exact verifier\. We report:

pno​\-​canonical=1N​∑i\(1−Ci\),pintegration​\-​fail=1N​∑iCi​\(1−Fi\),psuccess=1N​∑iFi\.p\_\{\\mathrm\{no\\text\{\-\}canonical\}\}=\\frac\{1\}\{N\}\\sum\_\{i\}\(1\-C\_\{i\}\),\\qquad p\_\{\\mathrm\{integration\\text\{\-\}fail\}\}=\\frac\{1\}\{N\}\\sum\_\{i\}C\_\{i\}\(1\-F\_\{i\}\),\\qquad p\_\{\\mathrm\{success\}\}=\\frac\{1\}\{N\}\\sum\_\{i\}F\_\{i\}\.
The first quantity captures failures to acquire the canonical missing fact under the fixed responder channel\. The second captures cases where the missing fact was acquired but the solver did not integrate it into a correct mathematical answer\. The third is the usual final\-answer accuracy\. We separately log raw offersOiO\_\{i\}so that any rare noncanonical offer can be distinguished from a genuine solver request success\.

Accuracy measures mathematical completion, while request metrics measure information acquisition under the fixed responder channel\. Their separation is important: a model may fail to elicit the canonical missing fact, may elicit it but still fail the mathematics, or may solve correctly after receiving it\. The trace\-level decomposition above makes these cases explicit rather than relying only on aggregate accuracy and request hit rate\.

## 4Dataset Construction

### 4\.1Current Release

The currentMIRA\-Mathrelease is the*20/50 typed*dataset: 20 instances per difficulty level for each Type A family and 50 instances per difficulty level for each Type B family\. This yields 660 Type A instances, 1,650 Type B instances, and 2,310 instances in total\. Each instance stores the family name, type, difficulty, random seed, solver prompt, target question, typed atomic\-hint specification, canonical hint values, exact answer, and verifier metadata\.

The 22 families \(described in Appendix[C](https://arxiv.org/html/2607.07391#A3)\) cover Bayesian inference, Chinese remainder reconstruction, coordinate geometry, triangle path\-sum inversion, linear systems with a separator variable, rank\-one matrix completion, moment recovery, phase retrieval, piecewise functions, rank\-deficient linear systems with a shared variable, recurrences, birth\-death chains, series\-parallel resistor networks, deconvolution, discrete tomography, eigenvector\-constrained matrix questions, discrete Laplace equations, linear systems with a missing coefficient, Markov chains, polynomial interpolation, portfolio variance, and hidden Markov models\.

### 4\.2Generator Invariants and Quality Control

Every generator enforces family\-specific nondegeneracy conditions\. Examples include rejecting singular linear systems, parallel lines, nonunique phase\-retrieval instances, ambiguous tomography targets without a sufficient hint cell, and rational answers whose denominators exceed family\-specific thresholds\. All numeric answers are represented exactly as integers, tuples, or reduced rational values\. Verifiers compare normalized exact values rather than surface strings\.

For families where local underdetermination can be checked algebraically, the generator uses rank, uniqueness, or constraint\-satisfaction tests\. For finite discrete families, the generator can enumerate consistent completions\. These checks are part of the dataset artifact, so users can regenerate the release, create larger splits, or audit any instance from its seed\.

### 4\.3Data Format, Access, and Maintenance

The public release includes generatedJSONLfiles, family generator code, exact verifiers, prompt templates, responder specifications, model\-run scripts, and raw logs\. The repository and dataset pages are:

Code and documentation:[https://github\.com/cedar\-lau/mira\-math](https://github.com/cedar-lau/mira-math)

Generated data:[https://huggingface\.co/datasets/samersaabjr/MIRA\-MATH/](https://huggingface.co/datasets/samersaabjr/MIRA-MATH/)

The generated benchmark data are released under CC BY 4\.0, and the generator, verifier, runner, and evaluation code are released under the MIT License\.

The maintenance plan is to version releases by generator version, seed set, and prompt protocol\. Future releases will preserve the current 20/50 typed split as a regression suite while adding larger held\-out seed sets and optional multi\-hint variants\. Each release will include checksums, schema documentation, and a changelog documenting changes to generators, prompts, matching rules, and verifiers\.

## 5Experimental Protocol

We evaluate solver models in two prompt regimes: zero\-shot \(ZS\) and four\-shot \(4S\)\. The four\-shot prompt provides examples of valid minimal requests and final\-answer formatting without reusing test instances\. In all reported experiments, the information\-holder model is fixed togpt\-4o\-mini; the solver model is the variable under evaluation\. The responder receives only its private constraints and is prompted to return a flat structured output with one of two types:offer, when it judges the request to semantically match one of its private constraints, ordecline, otherwise\. A valid offer must includehas\_exact\_match=true, the exact quoted private constraint, and the extracted hint string\. A declination uses a fixed message and must not reveal what other information the responder holds\. Thus the responder is fixed and constrained, but it is not a deterministic matching oracle\.

The released runner stores the full transcript for each instance, including Agent A requests, Agent B offer/decline decisions, quoted constraints, extracted hints, parsing errors when present, final answers, and exact\-verifier outcomes\. This makes it possible to recompute aggregate metrics and the trace\-level decomposition in Section[6](https://arxiv.org/html/2607.07391#S6)without rerunning model calls, provided the raw transcripts are retained\.

The main results report six solver models\. Exact decoding settings, model identifiers, timestamps, prompts, and raw transcripts are included with the release artifacts so that future users can reproduce the runs or update the baselines as model APIs and open\-weight checkpoints change\.

## 6Results

#### Requesting and solving are distinct\.

Figure[2](https://arxiv.org/html/2607.07391#S6.F2)summarizes final\-answer accuracy across solver models and prompting regimes, while Appendix[B\.1](https://arxiv.org/html/2607.07391#A2.SS1)retains the full numeric request metrics\. The aggregate results show that high request performance does not necessarily imply high final\-answer accuracy\. For example, in the zero\-shot setting,gemini\-2\.5\-flashhas the highest request hit rate and first\-request success among the completed models, but its final accuracy remains belowgpt\-5\.1andGemma\-4\. Conversely,Gemma\-4obtains the highest zero\-shot final accuracy while not achieving the highest request metrics\. This validates the benchmark’s central diagnostic split: asking for the right fact and using it correctly are separable skills\.

![Refer to caption](https://arxiv.org/html/2607.07391v1/mira_bar_overall_accuracy_2.png)Figure 2:Overall final\-answer accuracy by solver model and prompting regime\. Bars are grouped by solver model, with separate zero\-shot \(ZS\) and four\-shot \(4S\) results\. The full numeric counterpart, including request hit rate, first\-request success, average requests, declinations, and token estimates, is retained in Appendix[B\.1](https://arxiv.org/html/2607.07391#A2.SS1)\.
#### Trace snapshots make the diagnostic split concrete\.

To make the aggregate metrics easier to interpret, Appendix[D](https://arxiv.org/html/2607.07391#A4)gives short, offline\-audited transcript snapshots for three representative outcomes: no canonical hint acquired, canonical hint acquired but final answer wrong, and full request–resolve–solve success\. These snapshots are not used to estimate prevalence and should not be read as representative samples\. Instead, they show what the logged metrics mean at the transcript level\. For example, a no\-hint trace shows a solver cycling through plausible but wrong missing rates until the request budget is exhausted; a hint\-but\-wrong trace shows the solver receiving the prior needed for Bayes’ rule but still submitting an incorrect posterior; and a full\-success trace shows the solver requesting the missing resistor value and then computing the equivalent resistance exactly\. Table[2](https://arxiv.org/html/2607.07391#S6.T2)summarizes the examples and their diagnostic role\.

Table 2:Illustrative transcript snapshots used to interpret the aggregate metrics\. These are hand\-audited examples, not a frequency estimate\. Full transcripts and raw logs are released with the benchmark artifacts\.TagOutcome classExample familyInstanceDiagnostic roleT1No canonical hint acquiredbirth\_deathmissing\_ratebir\-000768The solver recognizes that a rate is missing but repeatedly asks for the wrong rate, so the responder only declines and no final answer is submitted\.T2Canonical hint acquired, final answer wrongbayes\_missingpriorbay\-000044The solver asks for and receives the priorP​\(H\)P\(H\), but the downstream Bayes calculation is wrong\. This is an integration failure rather than a request failure\.T3Canonical hint acquired, final answer correctcircuit\_missingresistancecir\-000924The solver identifies the missing resistor, receives it, and computes the exact equivalent resistance\. This illustrates the intended successful loop\.
#### Request acquisition versus mathematical integration\.

The transcript decomposition confirms that final accuracy should not be read as a pure requesting score\. In the reference protocol, an instance can fail because the solver never obtains the canonical atomic hint, or because it obtains the hint but performs the subsequent computation incorrectly\. We therefore report the disjoint trace\-level categories in Table[3](https://arxiv.org/html/2607.07391#S6.T3): no canonical hint acquired, canonical hint acquired but final answer wrong, and final answer correct\. This decomposition is especially important for families such as Chinese\-remainder reconstruction, phase retrieval, polynomial interpolation, and portfolio variance, where models can often obtain the relevant hint but still fail exact arithmetic, symbolic reconstruction, or answer normalization\.

Table 3:Trace\-level decomposition of failures\. “No canonical hint” means that the transcript did not contain an accepted offer matching the instance’s canonical atomic\-hint specification under the offline hint audit\. “Hint but wrong” means that the canonical hint was acquired but the exact final\-answer verifier rejected the solver’s answer\. “Correct” is the standard final\-answer accuracy\. All values are computed from raw transcripts; no additional model calls are required to reproduce this decomposition from the released logs\.Agent AmodelFormatNocanonicalhintHintbutwrongCorrectAcc\.givencanonicalhintgpt\-4o\-miniZS0\.2130\.5400\.2480\.314gpt\-5\.1ZS0\.2000\.1110\.6890\.861gemini\-2\.5\-flashZS0\.1260\.3850\.4900\.560Gemma\-4ZS0\.1870\.0750\.7390\.908Grok\-4\.20ZS0\.2310\.2660\.5030\.654Llama\-3\.1\-8B\-InstZS0\.3930\.5740\.0330\.054gpt\-4o\-mini4S0\.2100\.5570\.2330\.295gpt\-5\.14S0\.1970\.1160\.6870\.855gemini\-2\.5\-flash4S0\.1960\.4770\.3280\.407Gemma\-44S0\.2160\.0740\.7100\.906Grok\-4\.204S0\.2500\.2670\.4830\.644Llama\-3\.1\-8B\-Inst4S0\.3750\.5810\.0440\.070![Refer to caption](https://arxiv.org/html/2607.07391v1/mira_bar_type_accuracy_2.png)Figure 3:Type A versus Type B final\-answer accuracy by solver model and prompting regime\. Type A families have a fixed missing\-information slot, whereas Type B families require instance\-level localization of the missing slot\. The full numeric tables for Type A/Type B accuracy and request quality are retained in Appendix[B\.1](https://arxiv.org/html/2607.07391#A2.SS1)\.
#### Variable\-slot families are not uniformly harder\.

Figure[3](https://arxiv.org/html/2607.07391#S6.F3)shows that Type B families, despite requiring the solver to identify which slot is absent, are not uniformly harder in final\-answer accuracy\. Strong models often score higher on Type B than on Type A\. This is not a contradiction: Type A contains several families where the request is easy but the downstream computation is brittle, such as exact Chinese\-remainder reconstruction, path\-sum inversion, moment recovery, and phase retrieval\. The Type A/Type B distinction should therefore be interpreted as a missing\-slot diagnostic, not as a total difficulty ranking\. Appendix[B\.1](https://arxiv.org/html/2607.07391#A2.SS1)gives the corresponding request\-quality metrics, including hit rate, first\-request success, and average number of requests by family type\.

![Refer to caption](https://arxiv.org/html/2607.07391v1/mira_bar_difficulty_accuracy_2.png)Figure 4:Overall final\-answer accuracy by difficulty level, solver model, and prompting regime\. Bars are grouped by solver model and split by difficulty level D1–D3 for zero\-shot and four\-shot prompting\. The full numeric difficulty tables, including average request counts and the difficulty\-by\-family\-type accuracy split, are retained in Appendix[B\.1](https://arxiv.org/html/2607.07391#A2.SS1)\.
#### Difficulty interacts with family structure\.

Figure[4](https://arxiv.org/html/2607.07391#S6.F4)shows that smaller models can degrade sharply with difficulty, while stronger models are less monotonic\. This reflects the benchmark design: difficulty scaling may increase coefficient ranges, dimensions, denominators, or target\-index depth depending on the family\. Per\-family and per\-type reporting remain essential, because a single aggregate score can hide whether a model fails at requesting, exact arithmetic, linear algebra, combinatorial search, or answer normalization\. The full difficulty\-by\-type breakdown is provided in Appendix[B\.1](https://arxiv.org/html/2607.07391#A2.SS1)\.

#### Four\-shot prompting is not uniformly beneficial\.

Figures[2](https://arxiv.org/html/2607.07391#S6.F2)–[4](https://arxiv.org/html/2607.07391#S6.F4)show that the tested four\-shot prompt does not uniformly improve final\-answer accuracy\. The Type A/Type B split helps interpret this pattern\. Demonstrations can help when the missing slot is structurally stable: in Type A families, the solver can often reuse a demonstrated pattern such as asking for a prior, initial condition, threshold, or missing equation\. By contrast, Type B families require instance\-level slot localization: the solver must inspect the current problem and identify which coefficient, transition, boundary value, grid cell, resistor, interpolation point, or emission entry is absent\. A small four\-shot prompt necessarily covers only a few request archetypes, and may therefore induce exemplar anchoring: the solver may imitate the demonstrated request forms instead of adapting to the specific missing slot in the current instance\. We treat this as a prompt\-sensitivity hypothesis rather than a causal claim\. The main result is that demonstrations of the interaction protocol do not automatically improve minimal\-information reasoning; in some families they improve first\-request compliance while leaving mathematical integration unchanged or worse\. Appendix[D](https://arxiv.org/html/2607.07391#A4)includes an illustrative prompt\-anchoring snapshot \(T4\), and Appendix[B\.1](https://arxiv.org/html/2607.07391#A2.SS1)gives the full per\-family four\-shot deltas\.

Taken together, these results show thatMIRA\-Mathshould be read less as a single leaderboard and more as a diagnostic decomposition\. The same aggregate accuracy can arise from different mechanisms: failure to localize the missing slot, successful acquisition followed by incorrect computation, or correct end\-to\-end request and solution\. The trace\-level logs, per\-family metrics, and Type A/Type B split are therefore intended to be used jointly when comparing models or prompt strategies\.

## 7Failure Modes and Diagnostic Use

The benchmark exposes at least two recurring failure modes\.*Request\-acquisition failures*occur when the solver does not obtain the canonical missing fact under the fixed responder channel\. These can happen because the solver asks for a wrong slot, asks too vaguely, or repeatedly probes quantities that are already present in its private view\.*Integration failures*occur when the solver receives the canonical hint but still computes, formats, or normalizes the final answer incorrectly\.

Several families make this distinction visible\. In fixed\-slot families such asbayes\_missing\_prior, the request can be easy while exact arithmetic remains brittle\. In variable\-slot families such asbirth\_death\_missing\_rate, the solver must first localize the absent rate before it can apply a standard computation\. Other families, such ascircuit\_missing\_resistance, show the intended successful pattern: identify the missing slot, request it precisely, and integrate the returned value into the exact verifier\-compatible answer\.

Appendix[D](https://arxiv.org/html/2607.07391#A4)gives compact transcript snapshots for these cases\. The examples are illustrative rather than representative; aggregate claims are based on the tables above\. Their purpose is to show how the same final incorrect score can correspond to different underlying failures: no canonical hint acquired, canonical hint acquired but wrong downstream computation, or successful request and solution\. This is why the release includes both exact verifiers and raw interaction logs\.

## 8Intended Uses and Limitations

#### Intended uses\.

MIRA\-Mathis intended for evaluating models, prompts, and solver loops that must operate under partial mathematical information\. Appropriate uses include measuring request precision, comparing communication\-efficiency strategies, auditing whether a model can identify missing facts, and studying the relationship between clarification success and final mathematical accuracy\.

#### Out\-of\-scope uses\.

The benchmark should not be used as evidence that a model can collaborate with another autonomous agent\. The second role is a fixed, constrained information holder, not a teammate\. The benchmark should also not be treated as a general measure of tool use, web navigation, social reasoning, or real\-world decision\-making\.

#### Limitations\.

The tasks are synthetic and mathematically structured\. This improves controllability and exact verification but limits realism\. The current release uses one required atomic hint per instance; many real tasks require multiple facts, or uncertain evidence\. The typed hint schema and constrained responder protocol is useful for reproducibility but may understate the difficulty of communicating with humans or open\-ended systems\. The responder channel is controlled but not deterministic: in the reported experiments, a fixed LLM information holder performs the semantic offer\-or\-decline decision under structured\-output constraints\. This means request metrics can still reflect occasional responder matching errors, although the final\-answer verifier and dataset validity checks are deterministic\. Finally, model results can become stale as APIs and open\-weight checkpoints change; the release is designed to support re\-running baselines rather than treating the reported ranking as permanent\.

## 9Conclusion

MIRA\-Mathintroduces a controlled benchmark for minimal information requesting in mathematical reasoning\. Each instance is generated from a complete latent mathematical state with a unique answer, but the solver receives only a locally underdetermined view and must request exactly the missing atomic fact needed to complete the solution\. This construction turns information acquisition into an explicit, measurable step rather than leaving it implicit inside final\-answer accuracy\.

The results show that this step is not redundant with ordinary mathematical solving\. Across the tested models, request hit rate, first\-request success, and final\-answer accuracy diverge substantially: some models obtain the canonical hint but fail exact downstream computation, while others fail before acquiring the resolving fact\. The Type A/Type B split and transcript\-level decomposition further show that fixed missing slots, variable missing slots, arithmetic integration, and answer normalization can fail independently\.

The benchmark is intentionally narrow\. It does not claim to measure open\-ended collaboration, tool use, or human clarification behavior\. Instead, it provides a reproducible diagnostic setting with typed atomic hints, a fixed constrained LLM responder, deterministic validation, exact family\-specific verifiers, and released transcripts\. We hope this makesMIRA\-Mathuseful both as a baseline suite for future models and as an analysis tool for studying when language models ask for missing mathematical information rather than guessing\.

Broader Impact Statement

MIRA\-Mathhas positive potential as a controlled diagnostic for safer and more transparent use of language models in settings where systems should ask for missing information instead of guessing\. The main risks are overclaiming and benchmark overfitting\. The benchmark does not measure open\-ended collaboration, human communication, or real\-world decision\-making, and we state these limitations explicitly\. Since the data are synthetically generated mathematical problems, privacy and consent risks are minimal\. We mitigate misuse by releasing generator code, verifiers, documentation, and intended\-use guidance, and by recommending held\-out seed sets for future evaluations\.

Acknowledgments and Disclosure of Funding

The authors are responsible for the dataset design, generated instances, and evaluation protocol\.

## References

- MathArena: evaluating llms on uncontaminated math competitions\.arXiv preprint arXiv:2505\.23281\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2505.23281)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.07391#S2.SS1.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.2.1.1.1)\.
- W\. Chen, Y\. Su, C\. Zuo, C\. Yang, C\. Yuan, C\. Chan, H\. Yu, Y\. Lu, Y\. Hung, C\. Qian, Y\. Qin, X\. Cong, R\. Xie, Z\. Liu, M\. Sun, and J\. Zhou \(2023\)AgentVerse: facilitating multi\-agent collaboration and exploring emergent behaviors\.arXiv preprint arXiv:2308\.10848\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2308.10848)Cited by:[§2\.5](https://arxiv.org/html/2607.07391#S2.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.5.1.1.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman \(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2110.14168)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.07391#S2.SS1.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.2.1.1.1)\.
- A\. Das, T\. Gervet, J\. Romoff, D\. Batra, D\. Parikh, M\. Rabbat, and J\. Pineau \(2019\)TarMAC: targeted multi\-agent communication\.InProceedings of the 36th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.97,pp\. 1538–1546\.External Links:[Link](https://proceedings.mlr.press/v97/das19a.html)Cited by:[§2\.5](https://arxiv.org/html/2607.07391#S2.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.5.1.1.1)\.
- J\. N\. Foerster, I\. A\. Assael, N\. de Freitas, and S\. Whiteson \(2016\)Learning to communicate with deep multi\-agent reinforcement learning\.arXiv preprint arXiv:1605\.06676\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1605.06676)Cited by:[§2\.5](https://arxiv.org/html/2607.07391#S2.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.5.1.1.1)\.
- Y\. Gan, C\. Li, J\. Xie, L\. Wen, M\. Purver, and M\. Poesio \(2024\)ClarQ\-LLM: a benchmark for models clarifying and requesting information in task\-oriented dialog\.arXiv preprint arXiv:2409\.06097\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2409.06097)Cited by:[§2\.3](https://arxiv.org/html/2607.07391#S2.SS3.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.3.1.1.1)\.
- T\. Gebru, J\. Morgenstern, B\. Vecchione, J\. W\. Vaughan, H\. Wallach, H\. Daumé III, and K\. Crawford \(2021\)Datasheets for datasets\.Communications of the ACM64\(12\),pp\. 86–92\.External Links:[Document](https://dx.doi.org/10.1145/3458723)Cited by:[§2\.6](https://arxiv.org/html/2607.07391#S2.SS6.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.6.1.1.1)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021\)Measuring mathematical problem solving with the MATH dataset\.arXiv preprint arXiv:2103\.03874\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2103.03874)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.07391#S2.SS1.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.2.1.1.1)\.
- Y\. Huanget al\.\(2025\)Beyond solving math quiz: evaluating the ability of large reasoning models to ask for information in incomplete mathematical problems\.arXiv preprint arXiv:2508\.11252\.Cited by:[§2\.4](https://arxiv.org/html/2607.07391#S2.SS4.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.3.1.1.1)\.
- B\. Z\. Li, B\. Kim, and Z\. Wang \(2025\)QuestBench: can llms ask the right question to acquire information in reasoning tasks?\.arXiv preprint arXiv:2503\.22674\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2503.22674)Cited by:[§2\.4](https://arxiv.org/html/2607.07391#S2.SS4.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.3.1.1.1)\.
- G\. Li, H\. A\. A\. K\. Hammoud, H\. Itani, D\. Khizbullin, and B\. Ghanem \(2023\)CAMEL: communicative agents for “mind” exploration of large language model society\.arXiv preprint arXiv:2303\.17760\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2303.17760)Cited by:[§2\.5](https://arxiv.org/html/2607.07391#S2.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.5.1.1.1)\.
- X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang, S\. Zhang, X\. Deng, A\. Zeng, Z\. Du, C\. Zhang, S\. Shen, T\. Zhang, Y\. Su, H\. Sun, M\. Huang, Y\. Dong, and J\. Tang \(2023\)AgentBench: evaluating llms as agents\.arXiv preprint arXiv:2308\.03688\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2308.03688)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p1.1),[§2\.5](https://arxiv.org/html/2607.07391#S2.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.4.1.1.1)\.
- Y\. Matsubara, N\. Chiba, R\. Igarashi, and Y\. Ushiku \(2024\)Rethinking symbolic regression datasets and benchmarks for scientific discovery\.Journal of Data\-centric Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=qrUdrXsiXX)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.07391#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.2.1.1.1)\.
- G\. Mialon, C\. Fourrier, C\. Swift, T\. Wolf, Y\. LeCun, and T\. Scialom \(2023\)GAIA: a benchmark for general ai assistants\.arXiv preprint arXiv:2311\.12983\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2311.12983)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p1.1),[§2\.5](https://arxiv.org/html/2607.07391#S2.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.4.1.1.1)\.
- S\. Min, J\. Michael, H\. Hajishirzi, and L\. Zettlemoyer \(2020\)AmbigQA: answering ambiguous open\-domain questions\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing,pp\. 5783–5797\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.466),[Link](https://aclanthology.org/2020.emnlp-main.466/)Cited by:[§2\.3](https://arxiv.org/html/2607.07391#S2.SS3.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.3.1.1.1)\.
- M\. Mitchell, S\. Wu, A\. Zaldivar, P\. Barnes, L\. Vasserman, B\. Hutchinson, E\. Spitzer, I\. D\. Raji, and T\. Gebru \(2019\)Model cards for model reporting\.InProceedings of the Conference on Fairness, Accountability, and Transparency,pp\. 220–229\.External Links:[Document](https://dx.doi.org/10.1145/3287560.3287596)Cited by:[§2\.6](https://arxiv.org/html/2607.07391#S2.SS6.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.6.1.1.1)\.
- L\. Oala, M\. Maskey, L\. Bat\-Leah, A\. Parrish, N\. M\. Gürel, T\. Kuo, Y\. Liu, R\. Dror, D\. Brajovic, X\. Yao, M\. Bartolo, W\. A\. Gaviria Rojas, R\. Hileman, R\. Aliment, M\. W\. Mahoney, M\. Risdal, M\. Lease, W\. Samek, D\. Dutta, C\. G\. Northcutt, C\. Coleman, B\. Hancock, B\. Koch, G\. A\. Tadesse, B\. Karlaš, A\. Alaa, A\. B\. Dieng, N\. Noy, V\. J\. Reddi, J\. Zou, P\. Paritosh, M\. van der Schaar, K\. Bollacker, L\. Aroyo, C\. Zhang, J\. Vanschoren, I\. Guyon, and P\. Mattson \(2024\)DMLR: data\-centric machine learning research—past, present and future\.Journal of Data\-centric Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=2kpu78QdeE)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p3.1),[§2\.6](https://arxiv.org/html/2607.07391#S2.SS6.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.6.1.1.1)\.
- W\. Orr and K\. Crawford \(2024\)Building better datasets: seven recommendations for responsible design from dataset creators\.Journal of Data\-centric Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=6bd8BrRKTW)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p3.1),[§2\.6](https://arxiv.org/html/2607.07391#S2.SS6.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.6.1.1.1)\.
- Y\. Qin, S\. Liang, Y\. Ye, K\. Zhu, L\. Yan, Y\. Lu, Y\. Lin, X\. Cong, X\. Tang, B\. Qian, S\. Zhao, L\. Hong, R\. Tian, R\. Xie, J\. Zhou, M\. Gerstein, D\. Li, Z\. Liu, and M\. Sun \(2023\)ToolLLM: facilitating large language models to master 16000\+ real\-world apis\.arXiv preprint arXiv:2307\.16789\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2307.16789)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p1.1),[§2\.5](https://arxiv.org/html/2607.07391#S2.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.4.1.1.1)\.
- N\. Somasekharan, L\. Yue, Y\. Cao, W\. LI, P\. Emami, P\. S\. Bhargav, A\. Acharya, X\. Xie, and S\. Pan \(2026\)CFDLLMBench: a benchmark suite for evaluating large language models in computational fluid dynamics\.Journal of Data\-centric Machine Learning Research\.External Links:[Link](https://data.mlr.press/volumes/03.html)Cited by:[§2\.2](https://arxiv.org/html/2607.07391#S2.SS2.p1.1)\.
- S\. Sukhbaatar, A\. Szlam, and R\. Fergus \(2016\)Learning multiagent communication with backpropagation\.arXiv preprint arXiv:1605\.07736\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1605.07736)Cited by:[§2\.5](https://arxiv.org/html/2607.07391#S2.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.5.1.1.1)\.
- L\. Telyatnikov, G\. Bernardez, M\. Montagna, M\. Hajij, M\. Carrasco, P\. Vasylenko, M\. Papillon, G\. Zamzmi, M\. T\. Schaub, J\. Verhellen, P\. Snopov, B\. Miquel\-Oliver, M\. Gil\-Sorribes, A\. Molina, V\. GUALLAR, T\. Long, J\. Suk, P\. Rygiel, A\. V\. Nikitin, G\. Escalona, M\. Banf, D\. Filipiak, L\. Imasheva, M\. Schattauer, A\. L\. Martinez, H\. Fritze, M\. Masden, V\. Sánchez, M\. Lecha, A\. Cavallo, C\. Battiloro, M\. Piekenbrock, M\. Tec, G\. Dasoulas, N\. Miolane, S\. Scardapane, and T\. Papamarkou \(2025\)TopoBench: a framework for benchmarking topological deep learning\.Journal of Data\-centric Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=07sTzyEVtY)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.07391#S2.SS2.p1.1)\.
- R\. Wang, X\. He, R\. Yu, W\. Qiu, B\. An, and Z\. Rabinovich \(2020\)Learning efficient multi\-agent communication: an information bottleneck approach\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,pp\. 9908–9918\.External Links:[Link](https://proceedings.mlr.press/v119/wang20i.html)Cited by:[§2\.5](https://arxiv.org/html/2607.07391#S2.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.5.1.1.1)\.
- Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu, A\. H\. Awadallah, R\. W\. White, D\. Burger, and C\. Wang \(2023\)AutoGen: enabling next\-gen llm applications via multi\-agent conversation\.arXiv preprint arXiv:2308\.08155\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2308.08155)Cited by:[§2\.5](https://arxiv.org/html/2607.07391#S2.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.5.1.1.1)\.
- J\. Zhang, Y\. Chen, G\. Canal, A\. M\. Das, G\. Bhatt, S\. Mussmann, Y\. Zhu, J\. Bilmes, S\. S\. Du, K\. Jamieson, and R\. D\. Nowak \(2024a\)LabelBench: a comprehensive framework for benchmarking adaptive label\-efficient learning\.Journal of Data\-centric Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=Y2QcZfwHE7)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.07391#S2.SS2.p1.1)\.
- J\. Zhang, J\. Yang, P\. Wang, H\. Wang, Y\. Lin, H\. Zhang, Y\. Sun, X\. Du, Y\. Li, Z\. Liu, Y\. Chen, and H\. Li \(2024b\)OpenOOD v1\.5: enhanced benchmark for out\-of\-distribution detection\.Journal of Data\-centric Machine Learning Research\.Note:Dataset CertificationExternal Links:[Link](https://openreview.net/forum?id=cnnTnJQigs)Cited by:[§1](https://arxiv.org/html/2607.07391#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.07391#S2.SS2.p1.1)\.
- M\. J\. Q\. Zhang, W\. B\. Knox, and E\. Choi \(2025\)Modeling future conversation turns to teach llms to ask clarifying questions\.Proceedings of the International Conference on Learning Representations\.Cited by:[§2\.3](https://arxiv.org/html/2607.07391#S2.SS3.p1.1)\.
- J\. Zhao, K\. Fang, and L\. Cheng \(2026\)When and what to ask: askbench and rubric\-guided rlvr for llm clarification\.arXiv preprint arXiv:2602\.11199\.Cited by:[§2\.3](https://arxiv.org/html/2607.07391#S2.SS3.p1.1)\.
- S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, and G\. Neubig \(2023\)WebArena: a realistic web environment for building autonomous agents\.arXiv preprint arXiv:2307\.13854\.Cited by:[§2\.5](https://arxiv.org/html/2607.07391#S2.SS5.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.4.1.1.1)\.
- Z\. Zhou, X\. Feng, Z\. Zhu, J\. Yao, S\. Koyejo, and B\. Han \(2025\)From passive to active reasoning: can large language models ask the right questions under incomplete information?\.InProceedings of the International Conference on Machine Learning,Cited by:[§2\.4](https://arxiv.org/html/2607.07391#S2.SS4.p1.1),[Table 1](https://arxiv.org/html/2607.07391#S2.T1.3.1.1.1.1.1.3.1.1.1)\.

## Dataset Certification

MIRA\-Mathis a synthetically generated benchmark\. Each instance is produced from a recorded generator family, difficulty level, random seed, typed atomic\-hint specification, exact answer, and verifier metadata\. The release includes generated data, generator code, exact verifiers, prompt templates, responder specifications, model\-run scripts, raw interaction logs, and documentation\. The dataset contains mathematical problem text and synthetic numeric values rather than personal data\. Intended uses and out\-of\-scope uses are described in Section[8](https://arxiv.org/html/2607.07391#S8)\.

## Appendix AImplementation and Reproducibility Details

#### Released artifacts\.

The release contains the family generators, instance validator, exact family\-specific solvers, prompt templates, structured\-output schemas, reference LangGraph runner, scoring code, generated JSONL files, run metadata, and raw transcripts\. Each generated instance records the family name, difficulty, random seed, agent\-private constraints, machine\-readable canonical atomic\-hint specification, global solution, answer type, and verifier metadata\.

#### Instance validation\.

The validator checks three properties for each instance: global uniqueness of the full latent problem, local underdetermination of each agent’s private view, and sufficiency of the canonical atomic hint when applied to the solver’s view\. These checks are family\-specific and deterministic\. The final\-answer verifier also uses normalized exact values rather than surface\-form string matching\.

#### Responder protocol\.

The reference responder is a fixed information\-holder LLM\. It receives only its private constraints and Agent A’s current request\. It must emit a flat structured object with eitherofferordecline\. Anoffermust sethas\_exact\_match=true, quote the exact private constraint, and provide the extracted hint\. Adeclinemust not reveal alternative information\. If structured parsing fails, the runner attempts conservative raw\-output extraction; if that also fails, the response is recorded as a declination\. Therefore, semantic matching is controlled and auditable but LLM\-mediated\.

#### Transcript scoring\.

For each transcript, the scorer records final\-answer accuracy, request attempts, offer count, declination count, first\-request success, requests before first offer, token estimate, and exact expected/actual normalized answers\. We additionally compute the trace\-level indicators used in Table[3](https://arxiv.org/html/2607.07391#S6.T3): whether a canonical hint was acquired, whether the solver produced a correct final answer after acquiring it, and whether a noncanonical offer occurred\.

#### Versioning\.

To make future reruns comparable, each release records the generator version, seed set, prompt\-protocol version, responder model identifier, solver model identifier, decoding settings, code commit, dataset checksum, and transcript checksum\. Model rankings should be interpreted as a snapshot under these settings because API\-backed model behavior can change over time\.

#### Exact run configuration\.

Tables[4](https://arxiv.org/html/2607.07391#A1.T4)and[5](https://arxiv.org/html/2607.07391#A1.T5)record the concrete settings used for all reported runs so that the main reproducibility facts are available without consulting the release artifacts\. All runs use greedy decoding \(temperature=0=0\) for both agents, a single pass per instance, and a fixed request seed of12341234; residual provider\-side nondeterminism may remain where the backend does not honor the seed\. Models are referenced by provider alias on the dates shown; the underlying API snapshots and open\-weight checkpoints are not version\-pinned and may change\.

Table 4:Shared run configuration, identical across all reported solver models and both prompting regimes\. The request budget scales with instance difficultyddas3​d3d\. Datasets are released as JSONL with a fixed generation seed; each instance additionally stores its own generation seed\.SettingValueResponder \(Agent B\)gpt\-4o\-mini\(OpenAI\)Decoding temperature0\.00\.0\(Agent A and Agent B\)Max new tokensunset \(provider/model default\)Request budget \(max attempts\)3/6/93/6/9at difficulty1/2/31/2/3Runs per instance11\(single pass\)Inference\-time seed12341234Per\-instance timeout3600 s \(parallel runner\)Zero\-shot datasetfamily\_types\_20\_50\.jsonl\(2,310 inst\.\)Four\-shot datasetfamily\_types\_20\_50\_few\_shot\.jsonl\(2,310 inst\.\)Dataset generation seed12341234Protocol / methodllm\(single\-shot request, then final answer\)Table 5:Per\-solver exact model identifiers, serving backend, and run dates\. The responder \(Agent B\) isgpt\-4o\-miniin every configuration; for thegpt\-4o\-minisolver, Agents A and B are the same model\. Date ranges span the initial run and any subsequent resumption waves for instances that timed out or errored\.Solver \(Agent A\)Exact model IDBackendRun dates \(2026\)gpt\-4o\-minigpt\-4o\-miniOpenAI APIMar 13gpt\-5\.1gpt\-5\.1OpenAI APIMar 23gemini\-2\.5\-flashgemini\-2\.5\-flashGoogle GenAI / OpenRouterApr 09–21Gemma\-4google/gemma\-4\-31b\-itOpenRouterApr 17–19Grok\-4\.20x\-ai/grok\-4\.20OpenRouterApr 22–26Llama\-3\.1\-8B\-InstMeta\-Llama\-3\.1\-8B\-Instruct\(quantized, 16k ctx\)Ollama \(local\)May 07–09

## Appendix BAdditional Results and Full Numeric Tables

### B\.1Full numeric tables for main\-text figures

Tables[6](https://arxiv.org/html/2607.07391#A2.T6)–[10](https://arxiv.org/html/2607.07391#A2.T10)provide the full numeric values underlying Figures[2](https://arxiv.org/html/2607.07391#S6.F2)–[4](https://arxiv.org/html/2607.07391#S6.F4)\. The main text visualizes the central accuracy patterns for readability, while these appendix tables retain the complete request\-quality and token\-usage metrics\.

Table 6:Full overall performance metrics onMIRA\-Math\. This table is the numeric counterpart of Figure[2](https://arxiv.org/html/2607.07391#S6.F2)and includes request hit rate, first\-request success, average requests, declinations, and token estimates\.Agent A ModelTypeAccHit Rate1st\-Req OKAvg ReqAvg DeclAvg Tokensgpt\-4o\-miniZS0\.2480\.5960\.4932\.551\.77507gpt\-5\.10\.6900\.7010\.6432\.191\.391065gemini\-2\.5\-flash0\.4900\.7930\.7351\.690\.82533Gemma\-40\.7390\.7200\.6522\.121\.301099Grok\-4\.200\.5030\.6610\.5872\.341\.571536Llama\-3\.1\-8B\-Inst0\.0330\.4330\.3293\.352\.74165gpt\-4o\-mini4S0\.2330\.6040\.5102\.601\.81494gpt\-5\.10\.6870\.7190\.6652\.091\.291013gemini\-2\.5\-flash0\.3280\.7220\.6702\.061\.25672Gemma\-40\.7100\.7030\.6482\.261\.471107Grok\-4\.200\.4830\.6340\.5552\.461\.711513Llama\-3\.1\-8B\-Inst0\.0440\.4500\.3473\.352\.72173

Table 7:Full performance by family type\. Type A denotes fixed missing\-information slots and Type B denotes variable missing\-information slots\. This table is the numeric counterpart to Figure[3](https://arxiv.org/html/2607.07391#S6.F3)\.Accuracy1st\-Req OKAvg RequestsAgent A ModelTypeType AType BType AType BType AType Bgpt\-4o\-miniZS0\.3590\.2030\.5420\.4732\.362\.63gpt\-5\.10\.6080\.7230\.6150\.6552\.522\.06gemini\-2\.5\-flash0\.3480\.5460\.6580\.7652\.231\.48Gemma\-40\.6270\.7830\.5980\.6732\.451\.98Grok\-4\.200\.3640\.5580\.5910\.5862\.302\.36Llama\-3\.1\-8B\-Inst0\.0770\.0150\.3640\.3153\.023\.48gpt\-4o\-mini4S0\.3260\.1960\.6120\.4702\.362\.70gpt\-5\.10\.6290\.7100\.6610\.6672\.322\.00gemini\-2\.5\-flash0\.3320\.3260\.6760\.6682\.231\.99Gemma\-40\.6140\.7480\.5500\.6872\.622\.11Grok\-4\.200\.3380\.5410\.6560\.5152\.302\.52Llama\-3\.1\-8B\-Inst0\.0790\.0300\.5470\.2672\.943\.51

Table 8:Full accuracy and average\-request breakdown by difficulty level\. This table is the numeric counterpart to Figure[4](https://arxiv.org/html/2607.07391#S6.F4)\.AccuracyAvg RequestsAgent A ModelFormatD1D2D3D1D2D3gpt\-4o\-miniZS0\.3120\.2200\.1911\.902\.563\.39gpt\-5\.10\.6710\.7200\.6841\.702\.242\.78gemini\-2\.5\-flash0\.4700\.5060\.4991\.561\.701\.86Gemma\-40\.7230\.7660\.7311\.622\.162\.72Grok\-4\.200\.4480\.5460\.5301\.762\.343\.10Llama\-3\.1\-8B\-Inst0\.0440\.0260\.0262\.163\.474\.76gpt\-4o\-mini4S0\.2910\.2090\.1821\.812\.693\.54gpt\-5\.10\.6650\.7330\.6701\.682\.152\.56gemini\-2\.5\-flash0\.3810\.3290\.2571\.682\.092\.52Gemma\-40\.7020\.7510\.6791\.632\.313\.01Grok\-4\.200\.4160\.5260\.5261\.812\.473\.28Llama\-3\.1\-8B\-Inst0\.0710\.0300\.0212\.093\.474\.88

Table 9:Full accuracy split by difficulty level and family type\. This table supports the discussion of how difficulty interacts with fixed\-slot and variable\-slot family structure\.Type AType BAgent A ModelFormatD1D2D3D1D2D3gpt\-4o\-miniZS0\.3380\.3800\.3650\.3020\.1560\.122gpt\-5\.10\.5120\.6500\.6900\.7350\.7480\.678gemini\-2\.5\-flash0\.2770\.3900\.4000\.5480\.5520\.538Gemma\-40\.5190\.6900\.7050\.8050\.7960\.742Grok\-4\.200\.3080\.4000\.4000\.5050\.6040\.582Llama\-3\.1\-8B\-Inst0\.0690\.0800\.0850\.0340\.0040\.002gpt\-4o\-mini4S0\.2850\.3300\.3750\.2940\.1600\.104gpt\-5\.10\.5120\.6950\.7150\.7260\.7480\.652gemini\-2\.5\-flash0\.3000\.3550\.3500\.4140\.3180\.220Gemma\-40\.4810\.6850\.7150\.7910\.7780\.664Grok\-4\.200\.2920\.3650\.3700\.4660\.5900\.588Llama\-3\.1\-8B\-Inst0\.0810\.0850\.0700\.0680\.0080\.002

Table 10:Full request\-quality breakdown by family type, including hit rate, first\-request success, and average request count\. This table supports the claim that request acquisition and final\-answer accuracy capture different failure modes\.Hit Rate1st\-Req OKAvg RequestsAgent A ModelFormatType AType BType AType BType AType Bgpt\-4o\-miniZS0\.6530\.5730\.5420\.4732\.362\.63gpt\-5\.10\.6590\.7150\.6150\.6552\.522\.06gemini\-2\.5\-flash0\.7040\.8290\.6580\.7652\.231\.48Gemma\-40\.6590\.7450\.5980\.6732\.451\.98Grok\-4\.200\.6810\.6530\.5910\.5862\.302\.36Llama\-3\.1\-8B\-Inst0\.4980\.4070\.3640\.3153\.023\.48gpt\-4o\-mini4S0\.6710\.5770\.6120\.4702\.362\.70gpt\-5\.10\.7020\.7260\.6610\.6672\.322\.00gemini\-2\.5\-flash0\.7230\.7220\.6760\.6682\.231\.99Gemma\-40\.6180\.7370\.5500\.6872\.622\.11Grok\-4\.200\.7040\.6060\.6560\.5152\.302\.52Llama\-3\.1\-8B\-Inst0\.5870\.3960\.5470\.2672\.943\.51

### B\.2Per\-family and prompt\-effect results

The following tables provide additional per\-family result summaries and four\-shot effect analyses for each reported solver model and prompt regime\.

Table 11:Per\-family accuracy on the 20/50 typed dataset \(gpt\-4o\-mini, 2310 instances\)\. Families sorted by Type then name\.nn= instances per family\.zero\-shotfour\-shotTypeFamilynnAcc1st\-ReqAcc1st\-ReqAbayes\_missing\_prior600\.7501\.0000\.8001\.000crt\_reconstruction600\.0001\.0000\.0001\.000geometry\_coordinates600\.0330\.5000\.0670\.567graph\_path\_sums600\.0000\.0000\.0000\.000linear\_system\_separator600\.9330\.2830\.7830\.350matrix\_completion600\.1670\.9670\.1331\.000moment\_problem600\.0000\.0000\.0000\.000phase\_retrieval600\.0000\.6500\.0330\.967piecewise\_missing\_thresh\.600\.9501\.0000\.8001\.000rankdef\_linear\_shared600\.1330\.3500\.0330\.300recurrence\_missing\_init600\.9831\.0000\.9331\.000Bbirth\_death\_missing\_rate1500\.2270\.3870\.2800\.400circuit\_missing\_resistance1500\.5601\.0000\.5400\.993deconvolution1500\.1270\.1270\.1270\.080discrete\_tomography1500\.0800\.0130\.1470\.113eigenvector\_missing\_entry1500\.5930\.5470\.5930\.300laplace\_grid1500\.0000\.0200\.0000\.007linear\_sys\_missing\_coeff1500\.0930\.0070\.1400\.047markov\_missing\_transition1500\.1530\.2070\.1000\.193poly\_interpolation1500\.0810\.8190\.0270\.780portfolio\_var\_missing\_c\.1500\.1530\.9600\.1410\.832steady\_state\_missing\_em\.1500\.1671\.0000\.0600\.993

Table 12:Full per\-family metrics \(gpt\-4o\-mini, zero\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior600\.7501\.0001\.0001\.000\.001\.00345crt\_reconstruction600\.0001\.0001\.0001\.000\.001\.00312geometry\_coordinates600\.0330\.5970\.5002\.171\.180\.78465graph\_path\_sums600\.0000\.0030\.0006\.005\.980\.02826linear\_system\_separator600\.9330\.4210\.2832\.721\.721\.00607matrix\_completion600\.1670\.9830\.9671\.020\.021\.00311moment\_problem600\.0000\.0000\.0006\.006\.000\.00908phase\_retrieval600\.0000\.6500\.6501\.350\.350\.65497piecewise\_missing\_thresh\.600\.9501\.0001\.0001\.000\.001\.00342rankdef\_linear\_shared600\.1330\.4190\.3502\.681\.780\.62555recurrence\_missing\_init600\.9831\.0001\.0001\.000\.001\.00358Bbirth\_death\_missing\_rate1500\.2270\.4050\.3872\.791\.950\.65566circuit\_missing\_resistance1500\.5601\.0001\.0001\.000\.001\.00489deconvolution1500\.1270\.1330\.1274\.233\.600\.34560discrete\_tomography1500\.0800\.0870\.0132\.912\.730\.16424eigenvector\_missing\_entry1500\.5930\.5800\.5472\.251\.330\.77555laplace\_grid1500\.0000\.1020\.0204\.413\.720\.53597linear\_sys\_missing\_coeff1500\.0930\.0360\.0074\.323\.830\.36688markov\_missing\_transition1500\.1530\.3590\.2073\.042\.210\.63559poly\_interpolation1490\.0810\.7840\.8191\.960\.970\.61389portfolio\_var\_missing\_c\.1500\.1530\.9600\.9601\.040\.040\.96535steady\_state\_missing\_em\.1500\.1671\.0001\.0001\.000\.001\.00548

Table 13:Full per\-family metrics \(gpt\-4o\-mini, four\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior600\.8001\.0001\.0001\.000\.001\.00248crt\_reconstruction600\.0001\.0001\.0001\.000\.001\.00271geometry\_coordinates600\.0670\.6580\.5172\.071\.220\.85358graph\_path\_sums600\.0000\.0100\.0005\.975\.930\.03747linear\_system\_separator600\.7830\.5030\.2332\.451\.451\.00625matrix\_completion600\.1331\.0001\.0001\.000\.001\.00258moment\_problem600\.0000\.0000\.0006\.006\.000\.00725phase\_retrieval600\.0330\.9830\.9671\.030\.031\.00307piecewise\_missing\_thresh\.600\.8001\.0001\.0001\.000\.001\.00185rankdef\_linear\_shared600\.0330\.2280\.0173\.432\.720\.72778recurrence\_missing\_init600\.9331\.0001\.0001\.000\.001\.00341Bbirth\_death\_missing\_rate1500\.2800\.5440\.3532\.811\.890\.91580circuit\_missing\_resistance1500\.5400\.9970\.9931\.010\.011\.00298deconvolution1500\.1270\.2060\.0934\.544\.010\.53736discrete\_tomography1500\.1470\.1900\.1072\.692\.380\.31486eigenvector\_missing\_entry1500\.5930\.6400\.5332\.691\.850\.85513laplace\_grid1500\.0000\.1760\.0134\.874\.200\.67908linear\_sys\_missing\_coeff1500\.1400\.4010\.2604\.173\.530\.64629markov\_missing\_transition1500\.1000\.5290\.3132\.872\.000\.87572poly\_interpolation1500\.0270\.7280\.6071\.830\.890\.95392portfolio\_var\_missing\_c\.1500\.1400\.9390\.9001\.160\.170\.99314steady\_state\_missing\_em\.1500\.0600\.9960\.9931\.010\.011\.00234

Table 14:Four\-shot effect: accuracy change per family \(gpt\-4o\-mini, 20/50 typed dataset\)\.Δ\\Delta= four\-shot accuracy−\-baseline accuracy\. Positive values indicate improvement\.Accuracy1st\-Request SuccessTypeFamilynnBase4SΔ\\DeltaBase4SΔ\\DeltaAbayes\_missing\_prior600\.7500\.800\+0\.0501\.0001\.0000\.000crt\_reconstruction600\.0000\.0000\.0001\.0001\.0000\.000geometry\_coordinates600\.0330\.067\+0\.0330\.5000\.567\+0\.067graph\_path\_sums600\.0000\.0000\.0000\.0000\.0000\.000linear\_system\_separator600\.9330\.783−\-0\.1500\.2830\.350\+0\.067matrix\_completion600\.1670\.133−\-0\.0330\.9671\.000\+0\.033moment\_problem600\.0000\.0000\.0000\.0000\.0000\.000phase\_retrieval600\.0000\.033\+0\.0330\.6500\.967\+0\.317piecewise\_missing\_thresh\.600\.9500\.800−\-0\.1501\.0001\.0000\.000rankdef\_linear\_shared600\.1330\.033−\-0\.1000\.3500\.300−\-0\.050recurrence\_missing\_init600\.9830\.933−\-0\.0501\.0001\.0000\.000Type A average6600\.3590\.326−\-0\.0330\.5420\.612\+0\.070Bbirth\_death\_missing\_rate1500\.2270\.280\+0\.0530\.3870\.400\+0\.013circuit\_missing\_resistance1500\.5600\.540−\-0\.0201\.0000\.993−\-0\.007deconvolution1500\.1270\.1270\.0000\.1270\.080−\-0\.047discrete\_tomography1500\.0800\.147\+0\.0670\.0130\.113\+0\.100eigenvector\_missing\_entry1500\.5930\.5930\.0000\.5470\.300−\-0\.247laplace\_grid1500\.0000\.0000\.0000\.0200\.007−\-0\.013linear\_sys\_missing\_coeff1500\.0930\.140\+0\.0470\.0070\.047\+0\.040markov\_missing\_transition1500\.1530\.100−\-0\.0530\.2070\.193−\-0\.013poly\_interpolation1500\.0810\.027−\-0\.0540\.8190\.780−\-0\.039portfolio\_var\_missing\_c\.1500\.1530\.141−\-0\.0130\.9600\.832−\-0\.128steady\_state\_missing\_em\.1500\.1670\.060−\-0\.1071\.0000\.993−\-0\.007Type B average16500\.2030\.196−\-0\.0070\.4730\.470−\-0\.003Overall23100\.2480\.233−\-0\.0150\.4930\.510\+0\.017

Table 15:Per\-family accuracy on the 20/50 typed dataset \(gpt\-5\.1\)\. Families sorted by Type then name\.nn= instances per family\.zero\-shotfour\-shotTypeFamilynnAcc1st\-ReqAcc1st\-ReqAbayes\_missing\_prior601\.0001\.0001\.0001\.000crt\_reconstruction600\.0001\.0000\.0001\.000geometry\_coordinates600\.1170\.0670\.3170\.267graph\_path\_sums600\.0170\.0170\.0500\.083linear\_system\_separator601\.0000\.1331\.0001\.000matrix\_completion600\.7000\.6670\.7170\.683moment\_problem600\.0000\.0000\.0000\.017phase\_retrieval600\.9331\.0000\.9831\.000piecewise\_missing\_thresh\.601\.0001\.0001\.0001\.000rankdef\_linear\_shared600\.9170\.8830\.8500\.217recurrence\_missing\_init601\.0001\.0001\.0001\.000Bbirth\_death\_missing\_rate1500\.9800\.9670\.9530\.920circuit\_missing\_resistance1501\.0001\.0000\.9801\.000deconvolution1500\.1530\.0330\.1530\.107discrete\_tomography1500\.5670\.2330\.5130\.173eigenvector\_missing\_entry1500\.9330\.8530\.9600\.927laplace\_grid1500\.2270\.5330\.2270\.520linear\_sys\_missing\_coeff1500\.6930\.5200\.5730\.427markov\_missing\_transition1500\.8330\.6400\.8470\.647poly\_interpolation1500\.8070\.4670\.8470\.660portfolio\_var\_missing\_c\.1500\.9600\.9800\.9730\.993steady\_state\_missing\_em\.1500\.8000\.9730\.7870\.967

Table 16:Full per\-family metrics \(gpt\-5\.1, zero\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior601\.0001\.0001\.0001\.000\.001\.00392crt\_reconstruction600\.0001\.0001\.0001\.000\.001\.00791geometry\_coordinates600\.1170\.1660\.0674\.724\.370\.351161graph\_path\_sums600\.0170\.0310\.0175\.755\.700\.051211linear\_system\_separator601\.0000\.4250\.1332\.721\.721\.001035matrix\_completion600\.7000\.7180\.6672\.221\.370\.85774moment\_problem600\.0000\.0020\.0005\.975\.950\.021154phase\_retrieval600\.9331\.0001\.0001\.000\.001\.00957piecewise\_missing\_thresh\.601\.0001\.0001\.0001\.000\.001\.00451rankdef\_linear\_shared600\.9170\.9030\.8831\.370\.430\.93998recurrence\_missing\_init601\.0001\.0001\.0001\.000\.001\.00619Bbirth\_death\_missing\_rate1500\.9800\.9740\.9671\.090\.100\.99804circuit\_missing\_resistance1501\.0001\.0001\.0001\.000\.001\.00561deconvolution1500\.1530\.0660\.0335\.595\.400\.191250discrete\_tomography1500\.5670\.3660\.2332\.411\.840\.57847eigenvector\_missing\_entry1500\.9330\.8750\.8531\.520\.590\.93675laplace\_grid1500\.2270\.6070\.5332\.621\.850\.771470linear\_sys\_missing\_coeff1500\.6930\.6220\.5202\.351\.550\.791513markov\_missing\_transition1500\.8330\.8100\.6401\.470\.480\.991752poly\_interpolation1500\.8070\.6040\.4672\.471\.610\.851486portfolio\_var\_missing\_c\.1500\.9600\.9860\.9801\.050\.060\.99732steady\_state\_missing\_em\.1500\.8000\.9820\.9731\.070\.071\.001501

Table 17:Full per\-family metrics \(gpt\-5\.1, four\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior601\.0001\.0001\.0001\.000\.001\.00404crt\_reconstruction600\.0001\.0001\.0001\.000\.001\.00783geometry\_coordinates600\.3170\.3490\.2673\.873\.320\.551084graph\_path\_sums600\.0500\.0940\.0835\.375\.250\.121149linear\_system\_separator601\.0001\.0001\.0001\.000\.001\.00676matrix\_completion600\.7170\.7080\.6832\.331\.570\.77721moment\_problem600\.0000\.0220\.0175\.875\.820\.051165phase\_retrieval600\.9831\.0001\.0001\.000\.001\.001081piecewise\_missing\_thresh\.601\.0001\.0001\.0001\.000\.001\.00453rankdef\_linear\_shared600\.8500\.5480\.2172\.071\.170\.901149recurrence\_missing\_init601\.0001\.0001\.0001\.000\.001\.00529Bbirth\_death\_missing\_rate1500\.9530\.9390\.9201\.230\.270\.96816circuit\_missing\_resistance1500\.9801\.0001\.0001\.000\.001\.00554deconvolution1500\.1530\.1310\.1075\.305\.070\.231223discrete\_tomography1500\.5130\.3160\.1732\.481\.970\.51828eigenvector\_missing\_entry1500\.9600\.9380\.9271\.310\.350\.96624laplace\_grid1500\.2270\.5940\.5132\.591\.830\.761421linear\_sys\_missing\_coeff1500\.5730\.5310\.4272\.722\.010\.711421markov\_missing\_transition1500\.8470\.8180\.6471\.420\.430\.991622poly\_interpolation1500\.8470\.7400\.6601\.870\.980\.891338portfolio\_var\_missing\_c\.1500\.9730\.9960\.9931\.010\.011\.00743steady\_state\_missing\_em\.1500\.7870\.9810\.9671\.050\.051\.001337

Table 18:Four\-shot effect: accuracy change per family \(gpt\-5\.1, 20/50 typed dataset\)\.Δ\\Delta= four\-shot accuracy−\-baseline accuracy\.Accuracy1st\-Request SuccessTypeFamilynnBase4SΔ\\DeltaBase4SΔ\\DeltaAbayes\_missing\_prior601\.0001\.0000\.0001\.0001\.0000\.000crt\_reconstruction600\.0000\.0000\.0001\.0001\.0000\.000geometry\_coordinates600\.1170\.317\+0\.2000\.0670\.267\+0\.200graph\_path\_sums600\.0170\.050\+0\.0330\.0170\.083\+0\.067linear\_system\_separator601\.0001\.0000\.0000\.1331\.000\+0\.867matrix\_completion600\.7000\.717\+0\.0170\.6670\.683\+0\.017moment\_problem600\.0000\.0000\.0000\.0000\.017\+0\.017phase\_retrieval600\.9330\.983\+0\.0501\.0001\.0000\.000piecewise\_missing\_thresh\.601\.0001\.0000\.0001\.0001\.0000\.000rankdef\_linear\_shared600\.9170\.850−\-0\.0670\.8830\.217−\-0\.667recurrence\_missing\_init601\.0001\.0000\.0001\.0001\.0000\.000Type A average6600\.6080\.629\+0\.0210\.6150\.661\+0\.046Bbirth\_death\_missing\_rate1500\.9800\.953−\-0\.0270\.9670\.920−\-0\.047circuit\_missing\_resistance1501\.0000\.980−\-0\.0201\.0001\.0000\.000deconvolution1500\.1530\.1530\.0000\.0330\.107\+0\.073discrete\_tomography1500\.5670\.513−\-0\.0530\.2330\.173−\-0\.060eigenvector\_missing\_entry1500\.9330\.960\+0\.0270\.8530\.927\+0\.073laplace\_grid1500\.2270\.2270\.0000\.5330\.520−\-0\.013linear\_sys\_missing\_coeff1500\.6930\.573−\-0\.1200\.5200\.427−\-0\.093markov\_missing\_transition1500\.8330\.847\+0\.0130\.6400\.647\+0\.007poly\_interpolation1500\.8070\.847\+0\.0400\.4670\.660\+0\.193portfolio\_var\_missing\_c\.1500\.9600\.973\+0\.0130\.9800\.993\+0\.013steady\_state\_missing\_em\.1500\.8000\.787−\-0\.0130\.9730\.967−\-0\.007Type B average16500\.7230\.710−\-0\.0130\.6550\.667\+0\.013Overall23100\.6900\.687−\-0\.0030\.6430\.665\+0\.022

Table 19:Per\-family accuracy on the 20/50 typed dataset \(gemini\-2\.5\-flash, 2310 instances\)\. Families sorted by Type then name\.nn= instances per family\.zero\-shotfour\-shotTypeFamilynnAcc1st\-ReqAcc1st\-ReqAbayes\_missing\_prior600\.9831\.0000\.8671\.000crt\_reconstruction600\.0001\.0000\.0001\.000geometry\_coordinates600\.6330\.5170\.3000\.550graph\_path\_sums600\.0000\.2670\.0000\.050linear\_system\_separator600\.0000\.0330\.0000\.050matrix\_completion600\.2330\.4170\.5670\.850moment\_problem600\.0000\.0330\.0000\.017phase\_retrieval600\.0000\.9830\.0001\.000piecewise\_missing\_thresh\.601\.0001\.0000\.9501\.000rankdef\_linear\_shared600\.0000\.9830\.0000\.917recurrence\_missing\_init600\.9831\.0000\.9671\.000Bbirth\_death\_missing\_rate1500\.6400\.9870\.2670\.807circuit\_missing\_resistance1500\.8271\.0000\.5730\.993deconvolution1500\.0000\.2400\.0000\.027discrete\_tomography1500\.4730\.2200\.3000\.113eigenvector\_missing\_entry1500\.9400\.9800\.6670\.840laplace\_grid1500\.0670\.5400\.0000\.527linear\_sys\_missing\_coeff1500\.1930\.8670\.1070\.513markov\_missing\_transition1500\.3930\.6070\.4270\.567poly\_interpolation1500\.8730\.9800\.2130\.967portfolio\_var\_missing\_c\.1500\.8601\.0000\.5471\.000steady\_state\_missing\_em\.1500\.7401\.0000\.4870\.993

Table 20:Full per\-family metrics \(gemini\-2\.5\-flash, zero\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior600\.9831\.0001\.0001\.000\.001\.00285crt\_reconstruction600\.0001\.0001\.0001\.000\.001\.00335geometry\_coordinates600\.6330\.6170\.5172\.371\.600\.77424graph\_path\_sums600\.0000\.2670\.2674\.524\.250\.27641linear\_system\_separator600\.0000\.3560\.0332\.931\.931\.00516matrix\_completion600\.2330\.4670\.4172\.872\.250\.62445moment\_problem600\.0000\.0540\.0335\.785\.680\.10671phase\_retrieval600\.0000\.9920\.9831\.020\.021\.00762piecewise\_missing\_thresh\.601\.0001\.0001\.0001\.000\.001\.00216rankdef\_linear\_shared600\.0000\.9880\.9831\.050\.051\.00497recurrence\_missing\_init600\.9831\.0001\.0001\.000\.001\.00273Bbirth\_death\_missing\_rate1500\.6400\.9900\.9871\.020\.030\.99451circuit\_missing\_resistance1500\.8271\.0001\.0001\.000\.001\.00311deconvolution1500\.0000\.4260\.2402\.912\.240\.67487discrete\_tomography1500\.4730\.3810\.2202\.341\.750\.59540eigenvector\_missing\_entry1500\.9400\.9880\.9801\.030\.031\.00245laplace\_grid1500\.0670\.6170\.5402\.391\.670\.721048linear\_sys\_missing\_coeff1500\.1930\.9280\.8671\.170\.171\.00617markov\_missing\_transition1500\.3930\.8000\.6071\.380\.390\.99830poly\_interpolation1500\.8730\.9860\.9801\.030\.040\.99671portfolio\_var\_missing\_c\.1500\.8601\.0001\.0001\.000\.001\.00441steady\_state\_missing\_em\.1500\.7401\.0001\.0001\.000\.001\.00547

Table 21:Full per\-family metrics \(gemini\-2\.5\-flash, four\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior600\.8671\.0001\.0001\.000\.001\.00261crt\_reconstruction600\.0001\.0001\.0001\.000\.001\.00501geometry\_coordinates600\.3000\.6990\.5502\.221\.330\.88406graph\_path\_sums600\.0000\.0500\.0505\.755\.700\.05658linear\_system\_separator600\.0000\.3540\.0502\.921\.950\.97544matrix\_completion600\.5670\.8790\.8501\.550\.570\.98268moment\_problem600\.0000\.0170\.0175\.975\.950\.02676phase\_retrieval600\.0001\.0001\.0001\.000\.001\.00894piecewise\_missing\_thresh\.600\.9501\.0001\.0001\.000\.001\.00223rankdef\_linear\_shared600\.0000\.9530\.9171\.150\.151\.00567recurrence\_missing\_init600\.9671\.0001\.0001\.000\.001\.00270Bbirth\_death\_missing\_rate1500\.2670\.8840\.8071\.270\.290\.98476circuit\_missing\_resistance1500\.5730\.9970\.9931\.010\.011\.00289deconvolution1500\.0000\.0710\.0275\.435\.150\.28724discrete\_tomography1500\.3000\.2100\.1132\.632\.300\.33422eigenvector\_missing\_entry1500\.6670\.8930\.8401\.460\.510\.95332laplace\_grid1500\.0000\.5780\.5272\.782\.120\.661813linear\_sys\_missing\_coeff1500\.1070\.5630\.5132\.732\.080\.651149markov\_missing\_transition1500\.4270\.7730\.5671\.480\.490\.991039poly\_interpolation1500\.2130\.9730\.9671\.090\.110\.98887portfolio\_var\_missing\_c\.1500\.5471\.0001\.0001\.000\.001\.00470steady\_state\_missing\_em\.1500\.4870\.9970\.9931\.010\.011\.00634

Table 22:Four\-shot effect: accuracy change per family \(gemini\-2\.5\-flash, 20/50 typed dataset\)\.Δ\\Delta= four\-shot accuracy−\-baseline accuracy\. Positive values indicate improvement\.Accuracy1st\-Request SuccessTypeFamilynnBase4SΔ\\DeltaBase4SΔ\\DeltaAbayes\_missing\_prior600\.9830\.867−\-0\.1171\.0001\.0000\.000crt\_reconstruction600\.0000\.0000\.0001\.0001\.0000\.000geometry\_coordinates600\.6330\.300−\-0\.3330\.5170\.550\+0\.033graph\_path\_sums600\.0000\.0000\.0000\.2670\.050−\-0\.217linear\_system\_separator600\.0000\.0000\.0000\.0330\.050\+0\.017matrix\_completion600\.2330\.567\+0\.3330\.4170\.850\+0\.433moment\_problem600\.0000\.0000\.0000\.0330\.017−\-0\.017phase\_retrieval600\.0000\.0000\.0000\.9831\.000\+0\.017piecewise\_missing\_thresh\.601\.0000\.950−\-0\.0501\.0001\.0000\.000rankdef\_linear\_shared600\.0000\.0000\.0000\.9830\.917−\-0\.067recurrence\_missing\_init600\.9830\.967−\-0\.0171\.0001\.0000\.000Type A average6600\.3480\.332−\-0\.0170\.6580\.676\+0\.018Bbirth\_death\_missing\_rate1500\.6400\.267−\-0\.3730\.9870\.807−\-0\.180circuit\_missing\_resistance1500\.8270\.573−\-0\.2531\.0000\.993−\-0\.007deconvolution1500\.0000\.0000\.0000\.2400\.027−\-0\.213discrete\_tomography1500\.4730\.300−\-0\.1730\.2200\.113−\-0\.107eigenvector\_missing\_entry1500\.9400\.667−\-0\.2730\.9800\.840−\-0\.140laplace\_grid1500\.0670\.000−\-0\.0670\.5400\.527−\-0\.013linear\_sys\_missing\_coeff1500\.1930\.107−\-0\.0870\.8670\.513−\-0\.353markov\_missing\_transition1500\.3930\.427\+0\.0330\.6070\.567−\-0\.040poly\_interpolation1500\.8730\.213−\-0\.6600\.9800\.967−\-0\.013portfolio\_var\_missing\_c\.1500\.8600\.547−\-0\.3131\.0001\.0000\.000steady\_state\_missing\_em\.1500\.7400\.487−\-0\.2531\.0000\.993−\-0\.007Type B average16500\.5460\.326−\-0\.2200\.7650\.668−\-0\.098Overall23100\.4900\.328−\-0\.1620\.7350\.670−\-0\.065

Table 23:Per\-family accuracy on the 20/50 typed dataset \(Gemma\-4, 2310 instances\)\. Families sorted by Type then name\.nn= instances per family\.zero\-shotfour\-shotTypeFamilynnAcc1st\-ReqAcc1st\-ReqAbayes\_missing\_prior601\.0001\.0000\.9831\.000crt\_reconstruction600\.0001\.0000\.0001\.000geometry\_coordinates600\.2000\.0830\.1330\.117graph\_path\_sums600\.0000\.0000\.0000\.000linear\_system\_separator601\.0000\.0670\.9500\.367matrix\_completion600\.9500\.8830\.8670\.583moment\_problem600\.0170\.2000\.0330\.000phase\_retrieval600\.8501\.0000\.9830\.983piecewise\_missing\_thresh\.601\.0001\.0001\.0001\.000rankdef\_linear\_shared600\.8830\.3500\.8170\.000recurrence\_missing\_init601\.0001\.0000\.9831\.000Bbirth\_death\_missing\_rate1500\.9800\.9470\.9200\.907circuit\_missing\_resistance1501\.0001\.0000\.9871\.000deconvolution1500\.4130\.2330\.3200\.060discrete\_tomography1500\.9470\.6200\.9270\.580eigenvector\_missing\_entry1500\.8870\.8530\.8470\.800laplace\_grid1500\.1470\.3200\.1070\.247linear\_sys\_missing\_coeff1500\.7730\.6470\.6670\.553markov\_missing\_transition1500\.8930\.6070\.8730\.740poly\_interpolation1500\.7000\.2000\.7730\.727portfolio\_var\_missing\_c\.1500\.9601\.0000\.9801\.000steady\_state\_missing\_em\.1500\.9130\.9730\.8330\.940

Table 24:Full per\-family metrics \(Gemma\-4, zero\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior601\.0001\.0001\.0001\.000\.001\.00314crt\_reconstruction600\.0001\.0001\.0001\.000\.001\.00460geometry\_coordinates600\.2000\.1050\.0835\.184\.980\.20893graph\_path\_sums600\.0000\.0000\.0006\.006\.000\.002019linear\_system\_separator601\.0000\.4080\.0672\.681\.681\.00588matrix\_completion600\.9500\.9070\.8831\.300\.330\.97480moment\_problem600\.0170\.2140\.2004\.724\.470\.251771phase\_retrieval600\.8501\.0001\.0001\.000\.001\.00712piecewise\_missing\_thresh\.601\.0001\.0001\.0001\.000\.001\.00288rankdef\_linear\_shared600\.8830\.6170\.3502\.071\.180\.88740recurrence\_missing\_init601\.0001\.0001\.0001\.000\.001\.00323Bbirth\_death\_missing\_rate1500\.9800\.9630\.9471\.090\.110\.98561circuit\_missing\_resistance1501\.0001\.0001\.0001\.000\.001\.00404deconvolution1500\.4130\.2990\.2333\.853\.420\.432866discrete\_tomography1500\.9470\.7640\.6201\.550\.600\.95839eigenvector\_missing\_entry1500\.8870\.8740\.8531\.590\.690\.91633laplace\_grid1500\.1470\.3360\.3204\.394\.030\.363704linear\_sys\_missing\_coeff1500\.7730\.7400\.6471\.941\.110\.831044markov\_missing\_transition1500\.8930\.8020\.6071\.400\.401\.00914poly\_interpolation1500\.7000\.4240\.2002\.972\.210\.751082portfolio\_var\_missing\_c\.1500\.9601\.0001\.0001\.000\.001\.00605steady\_state\_missing\_em\.1500\.9130\.9870\.9731\.030\.031\.00844

Table 25:Full per\-family metrics \(Gemma\-4, four\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior600\.9831\.0001\.0001\.000\.001\.00309crt\_reconstruction600\.0001\.0001\.0001\.000\.001\.00455geometry\_coordinates600\.1330\.1210\.1175\.185\.050\.13816graph\_path\_sums600\.0000\.0000\.0006\.006\.000\.001953linear\_system\_separator600\.9500\.6000\.3672\.131\.131\.00534matrix\_completion600\.8670\.6550\.5832\.421\.520\.90746moment\_problem600\.0330\.0170\.0005\.785\.720\.072049phase\_retrieval600\.9830\.9920\.9831\.020\.021\.00850piecewise\_missing\_thresh\.601\.0001\.0001\.0001\.000\.001\.00265rankdef\_linear\_shared600\.8170\.4140\.0002\.331\.500\.83659recurrence\_missing\_init600\.9831\.0001\.0001\.000\.001\.00312Bbirth\_death\_missing\_rate1500\.9200\.9160\.9071\.470\.540\.93700circuit\_missing\_resistance1500\.9871\.0001\.0001\.000\.001\.00406deconvolution1500\.3200\.1450\.0604\.834\.490\.342986discrete\_tomography1500\.9270\.7340\.5801\.620\.690\.93802eigenvector\_missing\_entry1500\.8470\.8180\.8001\.881\.020\.86728laplace\_grid1500\.1070\.2560\.2474\.634\.350\.273385linear\_sys\_missing\_coeff1500\.6670\.6290\.5532\.621\.910\.711327markov\_missing\_transition1500\.8730\.8670\.7401\.290\.291\.00930poly\_interpolation1500\.7730\.7750\.7271\.810\.970\.84783portfolio\_var\_missing\_c\.1500\.9801\.0001\.0001\.000\.001\.00571steady\_state\_missing\_em\.1500\.8330\.9690\.9401\.070\.071\.00848

Table 26:Four\-shot effect: accuracy change per family \(Gemma\-4, 20/50 typed dataset\)\.Δ\\Delta= four\-shot accuracy−\-baseline accuracy\. Positive values indicate improvement\.Accuracy1st\-Request SuccessTypeFamilynnBase4SΔ\\DeltaBase4SΔ\\DeltaAbayes\_missing\_prior601\.0000\.983−\-0\.0171\.0001\.0000\.000crt\_reconstruction600\.0000\.0000\.0001\.0001\.0000\.000geometry\_coordinates600\.2000\.133−\-0\.0670\.0830\.117\+0\.033graph\_path\_sums600\.0000\.0000\.0000\.0000\.0000\.000linear\_system\_separator601\.0000\.950−\-0\.0500\.0670\.367\+0\.300matrix\_completion600\.9500\.867−\-0\.0830\.8830\.583−\-0\.300moment\_problem600\.0170\.033\+0\.0170\.2000\.000−\-0\.200phase\_retrieval600\.8500\.983\+0\.1331\.0000\.983−\-0\.017piecewise\_missing\_thresh\.601\.0001\.0000\.0001\.0001\.0000\.000rankdef\_linear\_shared600\.8830\.817−\-0\.0670\.3500\.000−\-0\.350recurrence\_missing\_init601\.0000\.983−\-0\.0171\.0001\.0000\.000Type A average6600\.6270\.614−\-0\.0140\.5980\.550−\-0\.048Bbirth\_death\_missing\_rate1500\.9800\.920−\-0\.0600\.9470\.907−\-0\.040circuit\_missing\_resistance1501\.0000\.987−\-0\.0131\.0001\.0000\.000deconvolution1500\.4130\.320−\-0\.0930\.2330\.060−\-0\.173discrete\_tomography1500\.9470\.927−\-0\.0200\.6200\.580−\-0\.040eigenvector\_missing\_entry1500\.8870\.847−\-0\.0400\.8530\.800−\-0\.053laplace\_grid1500\.1470\.107−\-0\.0400\.3200\.247−\-0\.073linear\_sys\_missing\_coeff1500\.7730\.667−\-0\.1070\.6470\.553−\-0\.093markov\_missing\_transition1500\.8930\.873−\-0\.0200\.6070\.740\+0\.133poly\_interpolation1500\.7000\.773\+0\.0730\.2000\.727\+0\.527portfolio\_var\_missing\_c\.1500\.9600\.980\+0\.0201\.0001\.0000\.000steady\_state\_missing\_em\.1500\.9130\.833−\-0\.0800\.9730\.940−\-0\.033Type B average16500\.7830\.748−\-0\.0350\.6730\.687\+0\.014Overall23100\.7390\.710−\-0\.0290\.6520\.648−\-0\.004

Table 27:Per\-family accuracy on the 20/50 typed dataset \(Grok\-4\.20, 2310 instances\)\. Families sorted by Type then name\.nn= instances per family\.zero\-shotfour\-shotTypeFamilynnAcc1st\-ReqAcc1st\-ReqAbayes\_missing\_prior600\.9001\.0000\.9671\.000crt\_reconstruction600\.0001\.0000\.0171\.000geometry\_coordinates600\.2170\.1330\.1500\.217graph\_path\_sums600\.0000\.1170\.0170\.083linear\_system\_separator600\.0000\.3170\.0500\.983matrix\_completion600\.2830\.9330\.3170\.917moment\_problem600\.0000\.0000\.0000\.017phase\_retrieval600\.5671\.0000\.4671\.000piecewise\_missing\_thresh\.601\.0001\.0000\.7171\.000rankdef\_linear\_shared600\.2330\.0000\.1670\.000recurrence\_missing\_init600\.8001\.0000\.8501\.000Bbirth\_death\_missing\_rate1500\.8930\.6600\.8330\.373circuit\_missing\_resistance1500\.9931\.0000\.9601\.000deconvolution1500\.0470\.0930\.0200\.093discrete\_tomography1500\.1070\.1670\.0730\.020eigenvector\_missing\_entry1500\.5270\.7870\.6400\.640laplace\_grid1500\.0800\.2130\.1200\.267linear\_sys\_missing\_coeff1500\.6530\.2930\.6330\.233markov\_missing\_transition1500\.8070\.5270\.7070\.573poly\_interpolation1500\.4200\.7130\.3270\.500portfolio\_var\_missing\_c\.1500\.9200\.9930\.9200\.987steady\_state\_missing\_em\.1500\.6931\.0000\.7130\.980

Table 28:Full per\-family metrics \(Grok\-4\.20, zero\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior600\.9001\.0001\.0001\.000\.001\.00267crt\_reconstruction600\.0001\.0001\.0001\.000\.001\.00401geometry\_coordinates600\.2170\.4670\.1332\.671\.870\.805595graph\_path\_sums600\.0000\.1190\.1175\.185\.050\.132802linear\_system\_separator600\.0000\.5530\.3172\.321\.321\.00636matrix\_completion600\.2830\.9380\.9331\.250\.300\.95330moment\_problem600\.0000\.0000\.0006\.006\.000\.001791phase\_retrieval600\.5671\.0001\.0001\.000\.001\.001563piecewise\_missing\_thresh\.601\.0001\.0001\.0001\.000\.001\.00288rankdef\_linear\_shared600\.2330\.4190\.0002\.902\.000\.90961recurrence\_missing\_init600\.8001\.0001\.0001\.000\.001\.00325Bbirth\_death\_missing\_rate1500\.8930\.7650\.6601\.740\.790\.951304circuit\_missing\_resistance1500\.9931\.0001\.0001\.000\.001\.00478deconvolution1500\.0470\.1040\.0935\.355\.230\.133273discrete\_tomography1500\.1070\.2200\.1672\.612\.310\.30697eigenvector\_missing\_entry1500\.5270\.8100\.7872\.041\.180\.861602laplace\_grid1500\.0800\.3250\.2134\.203\.710\.493024linear\_sys\_missing\_coeff1500\.6530\.4290\.2933\.562\.880\.682460markov\_missing\_transition1500\.8070\.7270\.5271\.820\.870\.952088poly\_interpolation1500\.4200\.8070\.7131\.600\.640\.96892portfolio\_var\_missing\_c\.1500\.9200\.9950\.9931\.030\.031\.00594steady\_state\_missing\_em\.1500\.6931\.0001\.0001\.000\.001\.001262

Table 29:Full per\-family metrics \(Grok\-4\.20, four\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior600\.9671\.0001\.0001\.000\.001\.00266crt\_reconstruction600\.0171\.0001\.0001\.000\.001\.00433geometry\_coordinates600\.1500\.5670\.2172\.271\.350\.921563graph\_path\_sums600\.0170\.1030\.0835\.325\.180\.133235linear\_system\_separator600\.0500\.9890\.9831\.030\.031\.00475matrix\_completion600\.3170\.9290\.9171\.370\.420\.95485moment\_problem600\.0000\.0280\.0175\.875\.820\.052008phase\_retrieval600\.4671\.0001\.0001\.000\.001\.001039piecewise\_missing\_thresh\.600\.7171\.0001\.0001\.000\.001\.00305rankdef\_linear\_shared600\.1670\.1280\.0004\.433\.830\.60945recurrence\_missing\_init600\.8501\.0001\.0001\.000\.001\.00383Bbirth\_death\_missing\_rate1500\.8330\.5830\.3732\.321\.430\.892332circuit\_missing\_resistance1500\.9601\.0001\.0001\.000\.001\.00481deconvolution1500\.0200\.0930\.0935\.515\.420\.091937discrete\_tomography1500\.0730\.0810\.0202\.902\.730\.17333eigenvector\_missing\_entry1500\.6400\.7110\.6402\.561\.730\.832335laplace\_grid1500\.1200\.3730\.2674\.213\.690\.523747linear\_sys\_missing\_coeff1500\.6330\.4030\.2333\.532\.850\.682720markov\_missing\_transition1500\.7070\.7610\.5731\.660\.690\.971923poly\_interpolation1500\.3270\.6750\.5001\.951\.020\.931027portfolio\_var\_missing\_c\.1500\.9200\.9900\.9871\.040\.041\.00594steady\_state\_missing\_em\.1500\.7130\.9900\.9801\.020\.021\.001424

Table 30:Four\-shot effect: accuracy change per family \(Grok\-4\.20, 20/50 typed dataset\)\.Δ\\Delta= four\-shot accuracy−\-baseline accuracy\. Positive values indicate improvement\.Accuracy1st\-Request SuccessTypeFamilynnBase4SΔ\\DeltaBase4SΔ\\DeltaAbayes\_missing\_prior600\.9000\.967\+0\.0671\.0001\.0000\.000crt\_reconstruction600\.0000\.017\+0\.0171\.0001\.0000\.000geometry\_coordinates600\.2170\.150−\-0\.0670\.1330\.217\+0\.083graph\_path\_sums600\.0000\.017\+0\.0170\.1170\.083−\-0\.033linear\_system\_separator600\.0000\.050\+0\.0500\.3170\.983\+0\.667matrix\_completion600\.2830\.317\+0\.0330\.9330\.917−\-0\.017moment\_problem600\.0000\.0000\.0000\.0000\.017\+0\.017phase\_retrieval600\.5670\.467−\-0\.1001\.0001\.0000\.000piecewise\_missing\_thresh\.601\.0000\.717−\-0\.2831\.0001\.0000\.000rankdef\_linear\_shared600\.2330\.167−\-0\.0670\.0000\.0000\.000recurrence\_missing\_init600\.8000\.850\+0\.0501\.0001\.0000\.000Type A average6600\.3640\.338−\-0\.0260\.5910\.656\+0\.065Bbirth\_death\_missing\_rate1500\.8930\.833−\-0\.0600\.6600\.373−\-0\.287circuit\_missing\_resistance1500\.9930\.960−\-0\.0331\.0001\.0000\.000deconvolution1500\.0470\.020−\-0\.0270\.0930\.0930\.000discrete\_tomography1500\.1070\.073−\-0\.0330\.1670\.020−\-0\.147eigenvector\_missing\_entry1500\.5270\.640\+0\.1130\.7870\.640−\-0\.147laplace\_grid1500\.0800\.120\+0\.0400\.2130\.267\+0\.053linear\_sys\_missing\_coeff1500\.6530\.633−\-0\.0200\.2930\.233−\-0\.060markov\_missing\_transition1500\.8070\.707−\-0\.1000\.5270\.573\+0\.047poly\_interpolation1500\.4200\.327−\-0\.0930\.7130\.500−\-0\.213portfolio\_var\_missing\_c\.1500\.9200\.9200\.0000\.9930\.987−\-0\.007steady\_state\_missing\_em\.1500\.6930\.713\+0\.0201\.0000\.980−\-0\.020Type B average16500\.5580\.541−\-0\.0180\.5860\.515−\-0\.071Overall23100\.5030\.483−\-0\.0200\.5870\.555−\-0\.032

Table 31:Per\-family accuracy on the 20/50 typed dataset \(Llama\-3\.1\-8B\-Instruct, 2310 instances\)\. Families sorted by Type then name\.nn= instances per family\.zero\-shotfour\-shotTypeFamilynnAcc1st\-ReqAcc1st\-ReqAbayes\_missing\_prior600\.0171\.0000\.0331\.000crt\_reconstruction600\.0000\.9830\.0001\.000geometry\_coordinates600\.1330\.0170\.1330\.083graph\_path\_sums600\.0000\.0000\.0000\.000linear\_system\_separator600\.0500\.0000\.0170\.967matrix\_completion600\.1830\.0000\.0000\.000moment\_problem600\.0000\.0000\.0000\.000phase\_retrieval600\.0000\.0000\.0671\.000piecewise\_missing\_thresh\.600\.4501\.0000\.6001\.000rankdef\_linear\_shared600\.0000\.0000\.0000\.000recurrence\_missing\_init600\.0171\.0000\.0170\.967Bbirth\_death\_missing\_rate1500\.0000\.2000\.0070\.140circuit\_missing\_resistance1500\.0071\.0000\.0270\.460deconvolution1500\.0000\.0000\.0000\.000discrete\_tomography1500\.0930\.0000\.1800\.113eigenvector\_missing\_entry1500\.0530\.1930\.0530\.280laplace\_grid1500\.0000\.0400\.0000\.013linear\_sys\_missing\_coeff1500\.0000\.0000\.0070\.113markov\_missing\_transition1500\.0000\.0600\.0130\.053poly\_interpolation1500\.0000\.3070\.0130\.253portfolio\_var\_missing\_c\.1500\.0070\.7330\.0270\.627steady\_state\_missing\_em\.1500\.0070\.9330\.0000\.880

Table 32:Full per\-family metrics \(Llama\-3\.1\-8B\-Instruct, zero\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior600\.0171\.0001\.0001\.000\.001\.00154crt\_reconstruction600\.0000\.9830\.9831\.030\.050\.98251geometry\_coordinates600\.1330\.4470\.0172\.351\.351\.00209graph\_path\_sums600\.0000\.0000\.0006\.006\.000\.00167linear\_system\_separator600\.0500\.4060\.0002\.571\.571\.00235matrix\_completion600\.1830\.5000\.0002\.001\.001\.00118moment\_problem600\.0000\.0000\.0006\.006\.000\.00167phase\_retrieval600\.0000\.1380\.0004\.233\.570\.67173piecewise\_missing\_thresh\.600\.4501\.0001\.0001\.000\.001\.00179rankdef\_linear\_shared600\.0000\.0020\.0006\.005\.980\.02169recurrence\_missing\_init600\.0171\.0001\.0001\.000\.001\.00160Bbirth\_death\_missing\_rate1500\.0000\.4090\.2003\.452\.640\.81190circuit\_missing\_resistance1500\.0071\.0001\.0001\.000\.001\.0092deconvolution1500\.0000\.0010\.0005\.975\.970\.01167discrete\_tomography1500\.0930\.0570\.0002\.912\.790\.1396eigenvector\_missing\_entry1500\.0530\.3750\.1933\.502\.790\.71184laplace\_grid1500\.0000\.1110\.0405\.515\.250\.26191linear\_sys\_missing\_coeff1500\.0000\.0860\.0005\.415\.170\.23203markov\_missing\_transition1500\.0000\.1460\.0604\.944\.550\.39190poly\_interpolation1500\.0000\.4910\.3072\.972\.220\.75156portfolio\_var\_missing\_c\.1500\.0070\.8380\.7331\.520\.521\.00134steady\_state\_missing\_em\.1500\.0070\.9660\.9331\.070\.071\.00145

Table 33:Full per\-family metrics \(Llama\-3\.1\-8B\-Instruct, four\-shot, 20/50 typed dataset\)\.TypeFamilynnAccHit Rate1st\-ReqAvg ReqAvg DeclAvg HintsAvg TokAbayes\_missing\_prior600\.0331\.0001\.0001\.000\.001\.00117crt\_reconstruction600\.0001\.0001\.0001\.000\.001\.00192geometry\_coordinates600\.1330\.4780\.0832\.301\.301\.00207graph\_path\_sums600\.0000\.0000\.0006\.006\.000\.00197linear\_system\_separator600\.0170\.9830\.9671\.030\.031\.00195matrix\_completion600\.0000\.0000\.0006\.006\.000\.00167moment\_problem600\.0000\.0000\.0006\.006\.000\.00167phase\_retrieval600\.0671\.0001\.0001\.000\.001\.00175piecewise\_missing\_thresh\.600\.6001\.0001\.0001\.000\.001\.00182rankdef\_linear\_shared600\.0000\.0080\.0005\.985\.930\.05172recurrence\_missing\_init600\.0170\.9830\.9671\.030\.031\.00172Bbirth\_death\_missing\_rate1500\.0070\.3370\.1403\.833\.050\.79212circuit\_missing\_resistance1500\.0270\.7290\.4601\.550\.551\.00174deconvolution1500\.0000\.0000\.0006\.006\.000\.00166discrete\_tomography1500\.1800\.1830\.1132\.702\.410\.29108eigenvector\_missing\_entry1500\.0530\.4480\.2803\.312\.560\.75176laplace\_grid1500\.0000\.1010\.0135\.335\.060\.27193linear\_sys\_missing\_coeff1500\.0070\.2830\.1134\.333\.700\.63200markov\_missing\_transition1500\.0130\.1440\.0535\.044\.640\.40195poly\_interpolation1500\.0130\.4390\.2533\.392\.670\.71169portfolio\_var\_missing\_c\.1500\.0270\.7650\.6271\.850\.860\.99167steady\_state\_missing\_em\.1500\.0000\.9260\.8801\.320\.340\.98127

Table 34:Four\-shot effect: accuracy change per family \(Llama\-3\.1\-8B\-Instruct, 20/50 typed dataset\)\.Δ\\Delta= four\-shot accuracy−\-baseline accuracy\. Positive values indicate improvement\.Accuracy1st\-Request SuccessTypeFamilynnBase4SΔ\\DeltaBase4SΔ\\DeltaAbayes\_missing\_prior600\.0170\.033\+0\.0171\.0001\.0000\.000crt\_reconstruction600\.0000\.0000\.0000\.9831\.000\+0\.017geometry\_coordinates600\.1330\.1330\.0000\.0170\.083\+0\.067graph\_path\_sums600\.0000\.0000\.0000\.0000\.0000\.000linear\_system\_separator600\.0500\.017−\-0\.0330\.0000\.967\+0\.967matrix\_completion600\.1830\.000−\-0\.1830\.0000\.0000\.000moment\_problem600\.0000\.0000\.0000\.0000\.0000\.000phase\_retrieval600\.0000\.067\+0\.0670\.0001\.000\+1\.000piecewise\_missing\_thresh\.600\.4500\.600\+0\.1501\.0001\.0000\.000rankdef\_linear\_shared600\.0000\.0000\.0000\.0000\.0000\.000recurrence\_missing\_init600\.0170\.0170\.0001\.0000\.967−\-0\.033Type A average6600\.0770\.079\+0\.0020\.3640\.547\+0\.183Bbirth\_death\_missing\_rate1500\.0000\.007\+0\.0070\.2000\.140−\-0\.060circuit\_missing\_resistance1500\.0070\.027\+0\.0201\.0000\.460−\-0\.540deconvolution1500\.0000\.0000\.0000\.0000\.0000\.000discrete\_tomography1500\.0930\.180\+0\.0870\.0000\.113\+0\.113eigenvector\_missing\_entry1500\.0530\.0530\.0000\.1930\.280\+0\.087laplace\_grid1500\.0000\.0000\.0000\.0400\.013−\-0\.027linear\_sys\_missing\_coeff1500\.0000\.007\+0\.0070\.0000\.113\+0\.113markov\_missing\_transition1500\.0000\.013\+0\.0130\.0600\.053−\-0\.007poly\_interpolation1500\.0000\.013\+0\.0130\.3070\.253−\-0\.053portfolio\_var\_missing\_c\.1500\.0070\.027\+0\.0200\.7330\.627−\-0\.107steady\_state\_missing\_em\.1500\.0070\.000−\-0\.0070\.9330\.880−\-0\.053Type B average16500\.0150\.030\+0\.0150\.3150\.267−\-0\.048Overall23100\.0330\.044\+0\.0110\.3290\.347\+0\.018

## Appendix CProblem Family Generators

Each family below defines a parameterised class of two\-agent reasoning problems\. A*generator*samples an instance at random given a difficulty leveld∈\{1,2,3\}d\\in\\\{1,2,3\\\}and a random seed\. For each instance, Agent A and Agent B each receive a disjoint subset of the constraints; neither subset alone determines the answer, but together they do, and exactly one atomic hint from B to A resolves A’s ambiguity\.

The 22 families are split into two types:

- •Type A — Fixed hint slot \(11 families\)\.The missing information is structurally determined by the family: it is always the same kind/position regardless of the specific instance\. Agent A can predict*what*to request without scanning its constraints\.
- •Type B — Variable hint slot \(11 families\)\.The missing slot varies per instance — it could be any one of multiple possible positions\. Agent A must first identify*which*slot is absent before formulating its request\.

### A1\. Bayesian Inference with Missing Prior \(bayes\_missing\_prior\)

#### Setup\.

Three rational parameters are sampled:

P​\(H\)=pnpd,P​\(E∣H\)=snsd,P​\(E∣¬H\)=fnfd,P\(H\)=\\tfrac\{p\_\{n\}\}\{p\_\{d\}\},\\quad P\(E\\mid H\)=\\tfrac\{s\_\{n\}\}\{s\_\{d\}\},\\quad P\(E\\mid\\neg H\)=\\tfrac\{f\_\{n\}\}\{f\_\{d\}\},with denominators drawn fromUnif​\[2,3\+2​d\]\\mathrm\{Unif\}\[2,\\,3\+2d\]and numerators chosen so all three lie strictly in\(0,1\)\(0,1\)andP​\(E∣H\)≠P​\(E∣¬H\)P\(E\\mid H\)\\neq P\(E\\mid\\neg H\)\. The posterior is computed via Bayes’ rule:

P​\(H∣E\)=P​\(E∣H\)​P​\(H\)P​\(E∣H\)​P​\(H\)\+P​\(E∣¬H\)​\(1−P​\(H\)\)\.P\(H\\mid E\)\\;=\\;\\frac\{P\(E\\mid H\)\\,P\(H\)\}\{P\(E\\mid H\)\\,P\(H\)\\;\+\\;P\(E\\mid\\neg H\)\\,\(1\-P\(H\)\)\}\.Instances where the posterior denominator exceeds 200 are rejected\.

#### Information split\.

- •Agent A: the likelihoodP​\(E∣H\)P\(E\\mid H\)and the false\-positive rateP​\(E∣¬H\)P\(E\\mid\\neg H\)\.
- •Agent B: the priorP​\(H\)P\(H\)\.

B’s atomic hint is always the priorP​\(H\)P\(H\)\.

#### Answer\.

The posteriorP​\(H∣E\)P\(H\\mid E\)as a reduced fraction\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

Fraction denominators drawn from\[2,3\+2​d\]\[2,\\,3\+2d\]\(max 5, 7, 9\), controlling numerical complexity\.

### A2\. Chinese Remainder Theorem Reconstruction \(crt\_reconstruction\)

#### Setup\.

Three distinct primes\(m1,m2,m3\)\(m\_\{1\},m\_\{2\},m\_\{3\}\)are drawn uniformly at random \(without replacement\) from the pool\{3,5,7,11,13,17,19\}\\\{3,5,7,11,13,17,19\\\}\. An integerx∼Unif​\[0,M\)x\\sim\\mathrm\{Unif\}\[0,M\)is sampled, whereM=m1​m2​m3M=m\_\{1\}m\_\{2\}m\_\{3\}, and the three residuesri=xmodmir\_\{i\}=x\\bmod m\_\{i\}are computed\. By the Chinese Remainder Theorem the solution in\[0,M\)\[0,M\)is unique\.

#### Information split\.

- •Agent A: congruencesx≡r1\(modm1\)x\\equiv r\_\{1\}\\pmod\{m\_\{1\}\}andx≡r2\(modm2\)x\\equiv r\_\{2\}\\pmod\{m\_\{2\}\}\.
- •Agent B: congruencex≡r3\(modm3\)x\\equiv r\_\{3\}\\pmod\{m\_\{3\}\}\.

B’s atomic hint is always the third congruence\(m3,r3\)\(m\_\{3\},r\_\{3\}\)\.

#### Answer\.

The integerxx\.

#### Difficulty scaling\.

This family does not scale with difficulty: the prime pool and problem structure are identical for alldd\. Accordingly, inmixgeneration it is always run atd=1d=1regardless of the requested difficulty\.

### A3\. Coordinate Geometry with Missing Line \(geometry\_coordinates\)

#### Setup\.

An intersection point\(x,y\)\(x,y\)is drawn uniformly from\[−\(4\+d\),4\+d\]2\[\-\(4\+d\),\\,4\+d\]^\{2\}\. Two non\-parallel lines through\(x,y\)\(x,y\)are generated with integer coefficients in\[−\(3\+d\),3\+d\]\[\-\(3\+d\),\\,3\+d\]:

a1​x\+b1​y=c1,a2​x\+b2​y=c2,a\_\{1\}x\+b\_\{1\}y=c\_\{1\},\\qquad a\_\{2\}x\+b\_\{2\}y=c\_\{2\},subject toa1​b2−a2​b1≠0a\_\{1\}b\_\{2\}\-a\_\{2\}b\_\{1\}\\neq 0\(non\-parallel\)\. A reference pointQ=\(x\+Δ​x,y\+Δ​y\)Q=\(x\+\\Delta x,\\;y\+\\Delta y\)is placed at an integer Euclidean distance from\(x,y\)\(x,y\)using a Pythagorean pair\(Δ​x,Δ​y\)∈\{\(3,4\),\(4,3\),\(5,12\),\(12,5\)\}\(\\Delta x,\\Delta y\)\\in\\\{\(3,4\),\(4,3\),\(5,12\),\(12,5\)\\\}\. The target isd​\(\(x,y\),Q\)=Δ​x2\+Δ​y2d\(\(x,y\),Q\)=\\sqrt\{\\Delta x^\{2\}\+\\Delta y^\{2\}\}, guaranteed to be an integer\.

#### Information split\.

- •Agent A: the first line equation and the coordinates ofQQ\.
- •Agent B: the second line equation and the coordinates ofQQ\.

B’s atomic hint is always the second line equation\.

#### Answer\.

The integer distanced​\(\(x,y\),Q\)d\(\(x,y\),Q\)\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Intersection point drawn from\[−\(4\+d\),4\+d\]2\[\-\(4\+d\),\\,4\+d\]^\{2\}\(range±\\pm5,±\\pm6,±\\pm7\)\.
- •Line coefficients drawn from\[−\(3\+d\),3\+d\]\[\-\(3\+d\),\\,3\+d\]\(range±\\pm4,±\\pm5,±\\pm6\)\.

### A4\. Triangle Path Sums \(graph\_path\_sums\)

#### Setup\.

A triangle graph has three edges with positive integer weightse01,e12,e20∼Unif​\[1,2\+d\]e\_\{01\},e\_\{12\},e\_\{20\}\\sim\\mathrm\{Unif\}\[1,\\,2\+d\]\. The three path sums \(one per pair of edges sharing a vertex\) are:

s1=e01\+e12,s2=e12\+e20,s3=e01\+e20\.s\_\{1\}=e\_\{01\}\+e\_\{12\},\\qquad s\_\{2\}=e\_\{12\}\+e\_\{20\},\\qquad s\_\{3\}=e\_\{01\}\+e\_\{20\}\.Any two path sums leave one degree of freedom, while all three path sums determine the three edge weights uniquely\.

#### Information split\.

- •Agent A: path sumss1s\_\{1\}ands2s\_\{2\}\.
- •Agent B: path sums3=e01\+e20s\_\{3\}=e\_\{01\}\+e\_\{20\}\.

B’s atomic hint is always the equatione01\+e20=s3e\_\{01\}\+e\_\{20\}=s\_\{3\}\.

#### Answer\.

The triple\(e01,e12,e20\)\(e\_\{01\},e\_\{12\},e\_\{20\}\)\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

Edge weights drawn from\[1,2\+d\]\[1,\\,2\+d\]\(max 3, 4, 5\)\.

### A5\. Linear System with Separator Variable \(linear\_system\_separator\)

#### Setup\.

Unknowns\(a,b,c\)\(a,b,c\)are drawn uniformly from\[−\(5\+d\),5\+d\]3\[\-\(5\+d\),\\,5\+d\]^\{3\}\. Two linearly independent equations in\(a,b,c\)\(a,b,c\)are sampled with integer coefficients in\[−\(3\+d\),3\+d\]\[\-\(3\+d\),\\,3\+d\], subject to the constraint that, whenccis substituted, the residual2×22\\times 2system in\(a,b\)\(a,b\)is uniquely solvable \(i\.e\. the2×22\\times 2submatrix on columnsa,ba,bhas rank 2\)\.

#### Information split\.

- •Agent A: the two linear equations\.
- •Agent B: the exact value ofcc\.

B’s atomic hint is alwaysc=valuec=\\text\{value\}\.

#### Answer\.

The triple\(a,b,c\)\(a,b,c\)\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Solution drawn from\[−\(5\+d\),5\+d\]3\[\-\(5\+d\),\\,5\+d\]^\{3\}\(range±\\pm6,±\\pm7,±\\pm8\)\.
- •Coefficients drawn from\[−\(3\+d\),3\+d\]\[\-\(3\+d\),\\,3\+d\]\(range±\\pm4,±\\pm5,±\\pm6\)\.

### A6\. Rank\-1 Matrix Completion \(matrix\_completion\)

#### Setup\.

A2×22\\times 2rank\-1 matrixM=𝐮𝐯⊤M=\\mathbf\{u\}\\mathbf\{v\}^\{\\top\}is constructed from vectors𝐮=\(u0,u1\)⊤\\mathbf\{u\}=\(u\_\{0\},u\_\{1\}\)^\{\\top\}and𝐯=\(v0,v1\)⊤\\mathbf\{v\}=\(v\_\{0\},v\_\{1\}\)^\{\\top\}withui,vi∼Unif​\[1,2\+d\]u\_\{i\},v\_\{i\}\\sim\\mathrm\{Unif\}\[1,\\,2\+d\]:

M=\(u0​v0u0​v1u1​v0u1​v1\)=\(abcd\)\.M=\\begin\{pmatrix\}u\_\{0\}v\_\{0\}&u\_\{0\}v\_\{1\}\\\\ u\_\{1\}v\_\{0\}&u\_\{1\}v\_\{1\}\\end\{pmatrix\}=\\begin\{pmatrix\}a&b\\\\ c&d\\end\{pmatrix\}\.By the rank\-1 identity,m11=m01​m10/m00m\_\{11\}=m\_\{01\}\\,m\_\{10\}/m\_\{00\}\(valid sincem00=u0​v0≥1m\_\{00\}=u\_\{0\}v\_\{0\}\\geq 1\)\.

#### Information split\.

- •Agent A: entriesm00=am\_\{00\}=aandm01=bm\_\{01\}=b\(first row\)\.
- •Agent B: entrym10=cm\_\{10\}=c\(position\[1,0\]\[1,0\]\)\.

B’s atomic hint is alwaysm10=cm\_\{10\}=c\.

#### Answer\.

The integerm11=b​c/am\_\{11\}=bc/a\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

Factor entriesui,viu\_\{i\},v\_\{i\}drawn from\[1,2\+d\]\[1,\\,2\+d\]\(max 3, 4, 5\)\.

### A7\. Moment Problem on Three\-Point Support \(moment\_problem\)

#### Setup\.

A discrete probability distribution on support\{0,1,2\}\\\{0,1,2\\\}is specified by unnormalised weights\(q0,q1,q2\)\(q\_\{0\},q\_\{1\},q\_\{2\}\)summing to a common denominatorD=6\+2​dD=6\+2d\. Three linear constraints determine the distribution uniquely:

\(sum\)​q0\+q1\+q2=D,\(1st moment\)​q1\+2​q2=m1,\(2nd moment\)​q1\+4​q2=m2\.\\text\{\(sum\)\}\\;\\;q\_\{0\}\+q\_\{1\}\+q\_\{2\}=D,\\qquad\\text\{\(1st moment\)\}\\;\\;q\_\{1\}\+2q\_\{2\}=m\_\{1\},\\qquad\\text\{\(2nd moment\)\}\\;\\;q\_\{1\}\+4q\_\{2\}=m\_\{2\}\.Valuesq0,q1∼Unif​\[1,D−2\]q\_\{0\},q\_\{1\}\\sim\\mathrm\{Unif\}\[1,D\-2\]are drawn subject toq2=D−q0−q1\>0q\_\{2\}=D\-q\_\{0\}\-q\_\{1\}\>0, andm1,m2m\_\{1\},m\_\{2\}are derived accordingly\.

#### Information split\.

- •Agent A: the sum constraint and the first\-moment equationq1\+2​q2=m1q\_\{1\}\+2q\_\{2\}=m\_\{1\}\.
- •Agent B: the sum constraint and the second\-moment equationq1\+4​q2=m2q\_\{1\}\+4q\_\{2\}=m\_\{2\}\.

Each two\-equation subsystem has rank 2 in three unknowns \(ill\-posed\)\. B’s atomic hint is always the second\-moment equation\.

#### Answer\.

The triple\(q0,q1,q2\)\(q\_\{0\},q\_\{1\},q\_\{2\}\)\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

Common denominatorD=6\+2​dD=6\+2d\(values 8, 10, 12\), controlling the magnitude of the weights and moments\.

### A8\. Phase Retrieval via DFT Magnitudes \(phase\_retrieval\)

#### Setup\.

A signal𝐱=\(x0,x1,x2,x3\)∈ℤ4\\mathbf\{x\}=\(x\_\{0\},x\_\{1\},x\_\{2\},x\_\{3\}\)\\in\\mathbb\{Z\}^\{4\}is drawn withx0≠0x\_\{0\}\\neq 0and\|xi\|≤2\+min⁡\(d,2\)\|x\_\{i\}\|\\leq 2\+\\min\(d,2\)\. The magnitude\-squared spectrum is computed using closed\-form expressions for a length\-4 real DFT:

\|x^0\|2\\displaystyle\|\\hat\{x\}\_\{0\}\|^\{2\}=\(x0\+x1\+x2\+x3\)2,\\displaystyle=\(x\_\{0\}\+x\_\{1\}\+x\_\{2\}\+x\_\{3\}\)^\{2\},\|x^1\|2\\displaystyle\|\\hat\{x\}\_\{1\}\|^\{2\}=\(x0−x2\)2\+\(x3−x1\)2,\\displaystyle=\(x\_\{0\}\-x\_\{2\}\)^\{2\}\+\(x\_\{3\}\-x\_\{1\}\)^\{2\},\|x^2\|2\\displaystyle\|\\hat\{x\}\_\{2\}\|^\{2\}=\(x0−x1\+x2−x3\)2,\\displaystyle=\(x\_\{0\}\-x\_\{1\}\+x\_\{2\}\-x\_\{3\}\)^\{2\},\|x^3\|2\\displaystyle\|\\hat\{x\}\_\{3\}\|^\{2\}=\(x0−x2\)2\+\(x1−x3\)2\.\\displaystyle=\(x\_\{0\}\-x\_\{2\}\)^\{2\}\+\(x\_\{1\}\-x\_\{3\}\)^\{2\}\.Note that\|x^1\|2=\|x^3\|2\|\\hat\{x\}\_\{1\}\|^\{2\}=\|\\hat\{x\}\_\{3\}\|^\{2\}in this formulation\. By rejection sampling, instances are kept only when \(i\) the magnitude vector alone admits multiple solutions \(i\.e\. A is locally ill\-posed\), and \(ii\) the magnitude vector together withsign​\(x0\)\\mathrm\{sign\}\(x\_\{0\}\)admits exactly one solution\.

#### Information split\.

- •Agent A: the four magnitude\-squared values\(\|x^0\|2,\|x^1\|2,\|x^2\|2,\|x^3\|2\)\(\|\\hat\{x\}\_\{0\}\|^\{2\},\|\\hat\{x\}\_\{1\}\|^\{2\},\|\\hat\{x\}\_\{2\}\|^\{2\},\|\\hat\{x\}\_\{3\}\|^\{2\}\)\.
- •Agent B: the sign bitsign​\(x0\)∈\{−1,\+1\}\\mathrm\{sign\}\(x\_\{0\}\)\\in\\\{\-1,\+1\\\}\.

B’s atomic hint is alwayssign​\(x0\)\\mathrm\{sign\}\(x\_\{0\}\)\.

#### Answer\.

The full signal\(x0,x1,x2,x3\)\(x\_\{0\},x\_\{1\},x\_\{2\},x\_\{3\}\)\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

Signal value range\|xi\|≤2\+min⁡\(d,2\)\|x\_\{i\}\|\\leq 2\+\\min\(d,2\)\(bounds 3, 4, 4 — caps atd=2d=2, sod=3d=3is identical tod=2d=2\)\.

### A9\. Piecewise Linear Function with Missing Threshold \(piecewise\_missing\_threshold\)

#### Setup\.

A two\-branch piecewise linear function is defined:

f​\(x\)=\{a0​x\+b0if​x<t,a1​x\+b1if​x≥t,f\(x\)=\\begin\{cases\}a\_\{0\}\\,x\+b\_\{0\}&\\text\{if \}x<t,\\\\ a\_\{1\}\\,x\+b\_\{1\}&\\text\{if \}x\\geq t,\\end\{cases\}with slopesai∼Unif​\[−\(2\+d\),2\+d\]a\_\{i\}\\sim\\mathrm\{Unif\}\[\-\(2\+d\),\\,2\+d\], interceptsbi∼Unif​\[−\(4\+2​d\),4\+2​d\]b\_\{i\}\\sim\\mathrm\{Unif\}\[\-\(4\+2d\),\\,4\+2d\], query pointx∼Unif​\[−\(5\+3​d\),5\+3​d\]x\\sim\\mathrm\{Unif\}\[\-\(5\+3d\),\\,5\+3d\], and thresholdttdrawn uniformly from a symmetric interval, withx≠tx\\neq tenforced and the two branch values atxxrequired to be distinct \(so the active branch is not degenerate\)\.

#### Information split\.

- •Agent A: the query pointxxand both branch formulas, but*not*tt\.
- •Agent B: the query pointxxand the thresholdtt\.

Withouttt, A cannot determine which branch is active\. B’s atomic hint is alwaystt\.

#### Answer\.

The integerf​\(x\)f\(x\)\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Slope range±\(2\+d\)\\pm\(2\+d\)\(max±\\pm3,±\\pm4,±\\pm5\)\.
- •Query and threshold range±\(5\+3​d\)\\pm\(5\+3d\)\(max±\\pm8,±\\pm11,±\\pm14\)\.

### A10\. Rank\-Deficient Linear System, Shared Variable \(rankdef\_linear\_shared\)

#### Setup\.

A solution\(x,y,z\)∈ℤ3\(x,y,z\)\\in\\mathbb\{Z\}^\{3\}is drawn uniformly from\[−\(5\+d\),5\+d\]3\[\-\(5\+d\),\\,5\+d\]^\{3\}\. Four integer\-coefficient equations are generated: rows 1 and 2 form a rank\-2 subsystem; rows 3 and 4 are each independently sufficient to complete the system to rank 3\. Formally, all four rows have coefficients in\[−\(3\+d\),3\+d\]∖\{0\}\[\-\(3\+d\),\\,3\+d\]\\setminus\\\{0\\\}and are generated by rejection to satisfy:

rank​\(rows​1,2\)=2,rank​\(rows​1,2,3\)=3,rank​\(rows​1,2,4\)=3\.\\mathrm\{rank\}\(\\text\{rows \}1,2\)=2,\\quad\\mathrm\{rank\}\(\\text\{rows \}1,2,3\)=3,\\quad\\mathrm\{rank\}\(\\text\{rows \}1,2,4\)=3\.

#### Information split\.

- •Agent A: equations 1 and 2 \(rank 2 in 3 unknowns, ill\-posed\)\.
- •Agent B: equations 3 and 4 \(either one suffices for A\)\.

B’s atomic hint is always equation 3\.

#### Answer\.

The triple\(x,y,z\)\(x,y,z\)\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Solution drawn from\[−\(5\+d\),5\+d\]3\[\-\(5\+d\),\\,5\+d\]^\{3\}\(range±\\pm6,±\\pm7,±\\pm8\)\.
- •Coefficients drawn from\[−\(3\+d\),3\+d\]∖\{0\}\[\-\(3\+d\),\\,3\+d\]\\setminus\\\{0\\\}\(range±\\pm4,±\\pm5,±\\pm6\)\.

### A11\. Recurrence Missing Initial Condition \(recurrence\_missing\_init\)

#### Setup\.

A second\-order linear recurrence over the integers:

a​\(n\+2\)=r1​a​\(n\+1\)\+r2​a​\(n\),n≥0\.a\(n\+2\)\\;=\\;r\_\{1\}\\,a\(n\+1\)\\;\+\\;r\_\{2\}\\,a\(n\),\\qquad n\\geq 0\.Parameters are sampled asr1∼Unif​\[1,3\+d\]r\_\{1\}\\sim\\mathrm\{Unif\}\[1,\\,3\+d\],r2∼Unif​\[−2,2\]r\_\{2\}\\sim\\mathrm\{Unif\}\[\-2,2\], and initial conditionsa​\(0\),a​\(1\)∼Unif​\[−3−d,3\+d\]a\(0\),a\(1\)\\sim\\mathrm\{Unif\}\[\-3\-d,\\,3\+d\]\. A target indexn∗∼Unif​\[4,7\+d\]n^\{\*\}\\sim\\mathrm\{Unif\}\[4,\\,7\+d\]is drawn, and the target valuea​\(n∗\)a\(n^\{\*\}\)is computed by forward iteration\.

#### Information split\.

- •Agent A: the recurrence coefficientsr1,r2r\_\{1\},r\_\{2\}and the initial conditiona​\(0\)a\(0\)\.
- •Agent B: the recurrence coefficientsr1,r2r\_\{1\},r\_\{2\}and the initial conditiona​\(1\)a\(1\)\.

Neither agent can computea​\(n∗\)a\(n^\{\*\}\)alone\. B’s atomic hint is alwaysa​\(1\)a\(1\)\.

#### Answer\.

The integera​\(n∗\)a\(n^\{\*\}\)\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Recurrence coefficientr1r\_\{1\}drawn from\[1,3\+d\]\[1,\\,3\+d\]\(max 4, 5, 6\)\.
- •Initial conditions and coefficients drawn from\[−3−d,3\+d\]\[\-3\-d,\\,3\+d\]\(range±\\pm4,±\\pm5,±\\pm6\)\.
- •Target indexn∗n^\{\*\}drawn from\[4,7\+d\]\[4,\\,7\+d\]\(max 8, 9, 10\)\.

Type B — Variable Hint Slot

### B1\. Birth\-Death Chain with Missing Rate \(birth\_death\_missing\_rate\)

#### Setup\.

A birth\-death Markov chain on states\{0,1,…,n−1\}\\\{0,1,\\ldots,n\-1\\\}withn=d\+2n=d\+2is defined by birth ratesλi\\lambda\_\{i\}\(transitionsi→i\+1i\\to i\+1fori=0,…,n−2i=0,\\ldots,n\-2\) and death ratesμi\\mu\_\{i\}\(transitionsi→i−1i\\to i\-1fori=1,…,n−1i=1,\\ldots,n\-1\), all drawn fromUnif​\[1,3\+2​d\]\\mathrm\{Unif\}\[1,\\,3\+2d\]\.

The unique stationary distribution satisfies detailed balance:

πi=π0⋅∏k=0i−1λkμk\+1,i=1,…,n−1,\\pi\_\{i\}\\;=\\;\\pi\_\{0\}\\cdot\\prod\_\{k=0\}^\{i\-1\}\\frac\{\\lambda\_\{k\}\}\{\\mu\_\{k\+1\}\},\\qquad i=1,\\ldots,n\-1,normalised so∑iπi=1\\sum\_\{i\}\\pi\_\{i\}=1\. One rate—either someλi\\lambda\_\{i\}or someμi\\mu\_\{i\}—is chosen uniformly at random to be hidden\. A target states∗∼Unif​\[0,n−1\]s^\{\*\}\\sim\\mathrm\{Unif\}\[0,n\-1\]is drawn\. Instances whereπs∗\\pi\_\{s^\{\*\}\}has denominator exceeding 300 are rejected\.

#### Information split\.

- •Agent A: all birth and death rates*except*the one missing rate, in natural\-language form\.
- •Agent B: the single missing rate \(which rate it is varies per instance\)\.

#### Answer\.

The stationary probabilityπs∗\\pi\_\{s^\{\*\}\}as a reduced fraction\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Number of statesn=d\+2n=d\+2\(3, 4, 5 states\)\.
- •Rates drawn from\[1,3\+2​d\]\[1,\\,3\+2d\]\(max 5, 7, 9\)\.

### B2\. Series\-Parallel Resistor Network \(circuit\_missing\_resistance\)

#### Setup\.

A random binary tree withn=2​d\+1n=2d\+1leaves is built by recursive splitting\. Each leaf is a named resistorR1,…,RnR\_\{1\},\\ldots,R\_\{n\}with a positive integer resistance drawn fromUnif​\[1,5\+3​\(d−1\)\]\\mathrm\{Unif\}\[1,\\,5\+3\(d\-1\)\]\. Each internal node is independently labelled*series*or*parallel*with equal probability\. The equivalent resistance is evaluated recursively:

Rseries=RL\+RR,Rparallel=RL​RRRL\+RR\.R\_\{\\text\{series\}\}=R\_\{L\}\+R\_\{R\},\\qquad R\_\{\\text\{parallel\}\}=\\frac\{R\_\{L\}\\,R\_\{R\}\}\{R\_\{L\}\+R\_\{R\}\}\.One resistorRkR\_\{k\}\(chosen uniformly\) is withheld\. The circuit topology is described to both agents in natural language\.

#### Information split\.

- •Agent A: the full circuit topology and all resistance values*except*RkR\_\{k\}\(which resistor varies per instance\)\.
- •Agent B: the value ofRkR\_\{k\}\.

#### Answer\.

The equivalent resistanceReqR\_\{\\text\{eq\}\}as a reduced fraction\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Number of resistorsn=2​d\+1n=2d\+1\(3, 5, 7\)\.
- •Resistance values drawn from\[1,5\+3​\(d−1\)\]\[1,\\,5\+3\(d\-1\)\]\(max 5, 8, 11\)\.

### B3\. Deconvolution with Missing Measurement \(deconvolution\)

#### Setup\.

A signal𝐱∈ℤn\\mathbf\{x\}\\in\\mathbb\{Z\}^\{n\}and a kernel𝐡∈ℤm\\mathbf\{h\}\\in\\mathbb\{Z\}^\{m\}are sampled with entries in\[−\(2\+d\),2\+d\]\[\-\(2\+d\),\\,2\+d\]andh0≠0h\_\{0\}\\neq 0\. The full output vector𝐲=𝐱∗𝐡∈ℤn\+m−1\\mathbf\{y\}=\\mathbf\{x\}\*\\mathbf\{h\}\\in\\mathbb\{Z\}^\{n\+m\-1\}is computed via linear convolution:

yk=∑i=0m−1hi​xk−i,k=0,…,n\+m−2\.y\_\{k\}=\\sum\_\{i=0\}^\{m\-1\}h\_\{i\}\\,x\_\{k\-i\},\\qquad k=0,\\ldots,n\+m\-2\.Then\+m−1n\+m\-1equations in𝐱\\mathbf\{x\}form an overdetermined linear system\. By rejection, an indexk∗k^\{\*\}is found such that removingyk∗y\_\{k^\{\*\}\}makes the system rank\-deficient, while the full system uniquely determines𝐱\\mathbf\{x\}\.

#### Information split\.

- •Agent A: the kernel𝐡\\mathbf\{h\}and all output valuesyky\_\{k\}*except*yk∗y\_\{k^\{\*\}\}\(which index varies per instance\)\.
- •Agent B: the missing output valueyk∗y\_\{k^\{\*\}\}\.

#### Answer\.

The signal𝐱=\(x0,…,xn−1\)\\mathbf\{x\}=\(x\_\{0\},\\ldots,x\_\{n\-1\}\)\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Signal lengthn=3\+min⁡\(d,2\)n=3\+\\min\(d,2\)\(4, 5, 5 — caps atd=2d=2\)\.
- •Kernel lengthm=2m=2ford≤1d\\leq 1, elsem=3m=3\.
- •Entry magnitudes bounded by2\+d2\+d\(max 3, 4, 5\)\.

### B4\. Discrete Tomography with Ambiguous Cell \(discrete\_tomography\)

#### Setup\.

A3×33\\times 3binary gridG∈\{0,1\}3×3G\\in\\\{0,1\\\}^\{3\\times 3\}is sampled uniformly at random\. Its row sumsri=∑jGi​jr\_\{i\}=\\sum\_\{j\}G\_\{ij\}and column sumscj=∑iGi​jc\_\{j\}=\\sum\_\{i\}G\_\{ij\}are computed\. The row and column sums alone are typically consistent with multiple binary grids\. A*target cell*\(it,jt\)\(i\_\{t\},j\_\{t\}\)is chosen from those that are ambiguous \(i\.e\. their value differs across consistent completions\)\. A*hint cell*\(ih,jh\)≠\(it,jt\)\(i\_\{h\},j\_\{h\}\)\\neq\(i\_\{t\},j\_\{t\}\)is found by search: revealing its value must reduce the consistent completions to exactly one\.

#### Information split\.

- •Agent A: all three row sumsr0,r1,r2r\_\{0\},r\_\{1\},r\_\{2\}and all three column sumsc0,c1,c2c\_\{0\},c\_\{1\},c\_\{2\}\.
- •Agent B: the value of the hint cellGih​jhG\_\{i\_\{h\}j\_\{h\}\}\(which cell varies per instance\)\.

#### Answer\.

The binary valueGit​jt∈\{0,1\}G\_\{i\_\{t\}j\_\{t\}\}\\in\\\{0,1\\\}\.

#### Difficulty scaling\.

This family does not scale with difficulty: the grid is always3×33\\times 3and the generation logic is identical for alldd\. Accordingly, inmixgeneration it is always run atd=1d=1regardless of the requested difficulty\.

### B5\. Matrix Entry Hidden from Eigenvector \(eigenvector\_missing\_entry\)

#### Setup\.

Ann×nn\\times ninteger matrixMM\(withn=d\+1n=d\+1\) is constructed such that a known integer vector𝐯\\mathbf\{v\}is an eigenvector with eigenvalueλ\\lambda:M​𝐯=λ​𝐯M\\mathbf\{v\}=\\lambda\\mathbf\{v\}\.

The key construction: one componentvj∗=0v\_\{j^\{\*\}\}=0, so the eigenvector equation for rowi∗i^\{\*\},

∑c=0n−1Mi∗​c​vc=λ​vi∗,\\sum\_\{c=0\}^\{n\-1\}M\_\{i^\{\*\}c\}\\,v\_\{c\}=\\lambda\\,v\_\{i^\{\*\}\},contains no term involvingMi∗​j∗M\_\{i^\{\*\}j^\{\*\}\}\(that term vanishes\)\. ThereforeMi∗​j∗M\_\{i^\{\*\}j^\{\*\}\}is unconstrained by the eigenvector equations\. A target columnt∗t^\{\*\}is drawn ensuringMj∗​t∗≠0M\_\{j^\{\*\}t^\{\*\}\}\\neq 0, so that A cannot compute\(M2\)i∗​t∗=∑kMi∗​k​Mk​t∗\(M^\{2\}\)\_\{i^\{\*\}t^\{\*\}\}=\\sum\_\{k\}M\_\{i^\{\*\}k\}M\_\{kt^\{\*\}\}without knowingMi∗​j∗M\_\{i^\{\*\}j^\{\*\}\}\.

#### Information split\.

- •Agent A: all entries ofMM*except*Mi∗​j∗M\_\{i^\{\*\}j^\{\*\}\}, and all components of𝐯\\mathbf\{v\}\(which entry is missing varies per instance\)\.
- •Agent B: the entryMi∗​j∗M\_\{i^\{\*\}j^\{\*\}\}\.

#### Answer\.

The integer\(M2\)i∗​t∗\(M^\{2\}\)\_\{i^\{\*\}t^\{\*\}\}\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Matrix dimensionn=d\+1n=d\+1\(2×22\\times 2,3×33\\times 3,4×44\\times 4\)\.
- •Entry magnitudes bounded by3\+2​\(d−1\)3\+2\(d\-1\)\(max 3, 5, 7\)\.
- •Eigenvalueλ\\lambdadrawn from\[1,3\+2​\(d−1\)\]\[1,\\,3\+2\(d\-1\)\]\(max 3, 5, 7\)\.

### B6\. Discrete Laplace Equation on a Grid \(laplace\_grid\)

#### Setup\.

Ann×nn\\times ngrid has unknown valuesui​ju\_\{ij\}on every cell\. Interior cells satisfy the discrete Laplace equation:

4​ui​j=ui−1,j\+ui\+1,j\+ui,j−1\+ui,j\+1,\(i,j\)​interior\.4\\,u\_\{ij\}\\;=\\;u\_\{i\-1,j\}\+u\_\{i\+1,j\}\+u\_\{i,j\-1\}\+u\_\{i,j\+1\},\\qquad\(i,j\)\\text\{ interior\.\}Boundary values are drawn as integer multiples of a scaling factor \(24 forn=4n=4, 224 forn=5n=5\) chosen to ensure integer interior solutions\. One boundary cell\(i∗,j∗\)\(i^\{\*\},j^\{\*\}\)is chosen uniformly at random and withheld; it is verified that A’s system is rank\-deficient without this value\.

#### Information split\.

- •Agent A: all discrete Laplace equations and all boundary values*except*ui∗​j∗u\_\{i^\{\*\}j^\{\*\}\}\(which cell varies per instance\)\.
- •Agent B: the missing boundary valueui∗​j∗u\_\{i^\{\*\}j^\{\*\}\}\.

#### Answer\.

The integer valueuit​jtu\_\{i\_\{t\}j\_\{t\}\}at a target interior cell\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Grid sizen=4n=4ford≤1d\\leq 1,n=5n=5ford≥2d\\geq 2\(4×44\\times 4grid with 4 interior cells;5×55\\times 5with 9 interior cells\)\.
- •Boundary value magnitudes scale linearly withddwithin each grid size\.

### B7\. Linear System with Missing Coefficient \(linear\_system\_missing\_coeff\)

#### Setup\.

Ann×nn\\times nsystemA​𝐱=𝐛A\\mathbf\{x\}=\\mathbf\{b\}\(withn=d\+1n=d\+1\) is generated with a random non\-singular integer matrixAAand right\-hand side𝐛\\mathbf\{b\}\(entries in\[−8,8\]\[\-8,8\]\)\. The unique solution𝐱\\mathbf\{x\}is computed by Gaussian elimination\. One coefficientAi∗​j∗A\_\{i^\{\*\}j^\{\*\}\}\(chosen uniformly\) is withheld\. Instances wherexk∗x\_\{k^\{\*\}\}has denominator exceeding 300 are rejected\.

#### Information split\.

- •Agent A: all coefficientsAi​jA\_\{ij\}*except*Ai∗​j∗A\_\{i^\{\*\}j^\{\*\}\}, and all right\-hand\-side valuesbib\_\{i\}, in natural language \(which coefficient is missing varies per instance\)\.
- •Agent B: the missing coefficientAi∗​j∗A\_\{i^\{\*\}j^\{\*\}\}\.

#### Answer\.

The solution componentxk∗x\_\{k^\{\*\}\}as a reduced fraction\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •System sizen=d\+1n=d\+1\(2×22\\times 2,3×33\\times 3,4×44\\times 4\)\.
- •Coefficient magnitudes bounded by4\+3​\(d−1\)4\+3\(d\-1\)\(max 4, 7, 10\)\.

### B8\. Markov Chain with Missing Transition Probability \(markov\_missing\_transition\)

#### Setup\.

Annn\-state ergodic Markov chain is generated with a random row\-stochastic transition matrixPP\(each row sampled by drawing positive integer parts and normalising\)\. One row indexr∗r^\{\*\}and a column partition are chosen uniformly at random:

- •A knows\(n−2\)\(n\-2\)entries of rowr∗r^\{\*\}\.
- •B knows exactly one entry of rowr∗r^\{\*\}at hint columnc∗c^\{\*\}\.
- •The remaining entry is derivable from∑jPr∗​j=1\\sum\_\{j\}P\_\{r^\{\*\}j\}=1\.

All other rows are fully known to A\. The unique stationary distribution𝝅\\boldsymbol\{\\pi\}satisfies𝝅​P=𝝅\\boldsymbol\{\\pi\}P=\\boldsymbol\{\\pi\},∑iπi=1\\sum\_\{i\}\\pi\_\{i\}=1\.

#### Information split\.

- •Agent A: all rows ofPPcompletely, except rowr∗r^\{\*\}where onlyn−2n\-2entries are known \(which row and column vary per instance\)\.
- •Agent B: entryPr∗​c∗P\_\{r^\{\*\}c^\{\*\}\}\.

#### Answer\.

The stationary probabilityπs∗\\pi\_\{s^\{\*\}\}as a reduced fraction\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Number of statesn=min⁡\(d\+1,4\)n=\\min\(d\+1,4\)\(2, 3, 4 — caps atd=3d=3\)\.
- •Transition probability denominators up ton⋅\(2\+d\)n\\cdot\(2\+d\), controlling fractional complexity\.

### B9\. Polynomial Interpolation with Missing Point \(poly\_interpolation\)

#### Setup\.

A polynomialppof degreeδ=2\+min⁡\(d,2\)\\delta=2\+\\min\(d,2\)is generated by samplingδ\+1\\delta\+1integer coefficients from\[−\(3\+d\),3\+d\]\[\-\(3\+d\),\\,3\+d\]\. A set ofδ\+1\\delta\+1distinct integer evaluation points is drawn, and the valuesp​\(xi\)p\(x\_\{i\}\)computed\. By Lagrange interpolation,δ\+1\\delta\+1distinct points uniquely determine a degree\-δ\\deltapolynomial, soδ\\deltapoints leave it underdetermined\. A fresh query pointx∗x^\{\*\}\(outside the interpolation set\) is sampled andp​\(x∗\)p\(x^\{\*\}\)computed as the target\.

#### Information split\.

- •Agent A: theδ\\deltaevaluation pairs\(x0,p​\(x0\)\),…,\(xδ−1,p​\(xδ−1\)\)\(x\_\{0\},p\(x\_\{0\}\)\),\\ldots,\(x\_\{\\delta\-1\},p\(x\_\{\\delta\-1\}\)\)\(which point is held back varies per instance\)\.
- •Agent B: the one remaining pair\(xδ,p​\(xδ\)\)\(x\_\{\\delta\},p\(x\_\{\\delta\}\)\)\.

#### Answer\.

The integerp​\(x∗\)p\(x^\{\*\}\)\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Degreeδ=2\+min⁡\(d,2\)\\delta=2\+\\min\(d,2\)\(3, 4, 4 — caps atd=2d=2, sod=3d=3is identical tod=2d=2in structure but with wider coefficient range\)\.
- •Coefficient magnitudes bounded by3\+d3\+d\(max 4, 5, 6\)\.

### B10\. Portfolio Variance with Missing Correlation \(portfolio\_variance\_missing\_corr\)

#### Setup\.

A portfolio ofn=d\+1n=d\+1assets has weightswiw\_\{i\}, standard deviationsσi∼Unif​\[1,5\]\\sigma\_\{i\}\\sim\\mathrm\{Unif\}\[1,5\], and pairwise correlationsρi​j∈\(−1,1\)\\rho\_\{ij\}\\in\(\-1,1\)fori<ji<j\(drawn as rational multiples of1/D1/DforD∈\{2,3,4\}D\\in\\\{2,3,4\\\}\)\. Portfolio variance is:

σp2=∑i=0n−1∑j=0n−1wi​wj​σi​σj​ρi​j,ρi​i=1\.\\sigma\_\{p\}^\{2\}=\\sum\_\{i=0\}^\{n\-1\}\\sum\_\{j=0\}^\{n\-1\}w\_\{i\}\\,w\_\{j\}\\,\\sigma\_\{i\}\\,\\sigma\_\{j\}\\,\\rho\_\{ij\},\\qquad\\rho\_\{ii\}=1\.One correlation pair\(a,b\)\(a,b\)witha<ba<bis chosen uniformly at random and withheld\. Instances whereσp2\\sigma\_\{p\}^\{2\}has denominator exceeding 500 are rejected\.

#### Information split\.

- •Agent A: all weightswiw\_\{i\}, all standard deviationsσi\\sigma\_\{i\}, and all\(n2\)\\binom\{n\}\{2\}pairwise correlations*except*ρa​b\\rho\_\{ab\}\(which pair varies per instance\)\.
- •Agent B: the correlationρa​b\\rho\_\{ab\}\.

#### Answer\.

The portfolio varianceσp2\\sigma\_\{p\}^\{2\}as a reduced fraction\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

Number of assetsn=d\+1n=d\+1\(2, 3, 4\), giving\(n2\)\\binom\{n\}\{2\}correlation pairs \(1, 3, 6 — of which one is always missing\)\.

### B11\. Hidden Markov Model with Missing Emission \(steady\_state\_missing\_emission\)

#### Setup\.

A Hidden Markov Model hasn=d\+1n=d\+1states andm=min⁡\(d\+1,3\)m=\\min\(d\+1,3\)observation symbols\. The transition matrixPPis fully specified \(each row a random distribution over states\), and the emission matrix has entriesE​\[s\]​\[o\]E\[s\]\[o\]= probability of observing symboloofrom statess\(each row a random distribution over symbols\)\.

The stationary distribution𝝅\\boldsymbol\{\\pi\}ofPPis computed by solving𝝅​P=𝝅\\boldsymbol\{\\pi\}P=\\boldsymbol\{\\pi\}\. The marginal observation probability for a target symbolo∗o^\{\*\}is:

P​\(observe​o∗\)=∑s=0n−1πs​E​\[s\]​\[o∗\]\.P\(\\mathrm\{observe\}\\;o^\{\*\}\)=\\sum\_\{s=0\}^\{n\-1\}\\pi\_\{s\}\\,E\[s\]\[o^\{\*\}\]\.One emission entryE​\[s∗\]​\[o∗\]E\[s^\{\*\}\]\[o^\{\*\}\]\(chosen uniformly\) is withheld\.

#### Information split\.

- •Agent A: the full transition matrixPPand all emission entries*except*E​\[s∗\]​\[o∗\]E\[s^\{\*\}\]\[o^\{\*\}\]\(which state/symbol entry varies per instance\)\.
- •Agent B: the missing emissionE​\[s∗\]​\[o∗\]E\[s^\{\*\}\]\[o^\{\*\}\]\.

#### Answer\.

The observation probabilityP​\(observe​o∗\)P\(\\mathrm\{observe\}\\;o^\{\*\}\)as a reduced fraction\.

#### Difficulty scaling \(d∈\{1,2,3\}d\\in\\\{1,2,3\\\}\)\.

- •Number of statesn=d\+1n=d\+1\(2, 3, 4\)\.
- •Number of symbolsm=min⁡\(d\+1,3\)m=\\min\(d\+1,3\)\(2, 3, 3 — caps atd=2d=2\)\.

## Appendix DIllustrative Transcript Snapshots

This appendix gives compact snapshots of representative transcripts\. The snapshots are included to make the qualitative failure modes in Section 7 concrete\. They are not a sampling protocol and are not used to estimate prevalence\. Aggregate conclusions should be drawn from the quantitative tables in the main text and Appendix A\. The full transcripts and raw JSON logs are included with the released artifacts\.

For each snapshot, Agent A is the solver and Agent B is the fixed constrained information holder\. We report only the fields needed to interpret the trace: the instance, the canonical missing fact held by Agent B, the salient request/offer/decline sequence, and the exact\-verifier outcome\.

We exclude noncanonical offers from the integration\-failure examples\. If the responder offers a constraint that does not match the canonical atomic hint, the trace is categorized separately as a responder\-matching anomaly rather than as a solver integration failure\.

#### T1: No canonical hint acquired\.

This trace illustrates a request\-acquisition failure\. The solver identifies the general family and realizes that a birth–death rate is missing, but repeatedly asks for wrong rate slots\. Agent B never issues an offer, so the solver exhausts its request budget without the resolving constraint\.

```
Model/mode: gpt-4o-mini, zero-shot
Instance: bir-000768
Family: birth_death_missing_rate
Target: pi_2
Canonical missing fact held by B: mu_4 = 2

A requests: "the birth rate lambda_2"
B declines.
A requests: "the death rate mu_0"
B declines.
A requests: "the death rate mu_2"
B declines.
A requests: "the birth rate lambda_2"
B declines.
A requests: "the birth rate lambda_1"
B declines.
A requests: "the birth rate lambda_3"
B declines.
A requests: "the birth rate lambda_0"
B declines.
A requests: "the death rate mu_3"
B declines.
A requests: "the birth rate lambda_4"
B declines.

Outcome: no answer submitted.
Ground truth: 1/9.
Failure type: request-acquisition failure.
```

This example shows why average request count and declination count are informative: the solver is not merely failing the final arithmetic, but failing to localize the hidden slot under the request budget\.

#### T2: Canonical hint acquired, but final answer wrong\.

This trace illustrates an integration failure\. The solver asks for the correct missing prior and receives it, but then submits an incorrect posterior\.

```
Model/mode: gpt-4o-mini, zero-shot
Instance: bay-000044
Family: bayes_missing_prior
Target: P_H_given_E
A’s private facts: P(E|H) = 1/9, P(E|not_H) = 1/2
Canonical missing fact held by B: P(H) = 5/8

A requests: "the prior P(H)"
B offers: P(H) = 5/8

A final answer: 5/29
Ground truth: 10/37.
Failure type: integration failure after successful hint acquisition.
```

The request was successful, so this transcript should not be counted as a gap\-identification failure\. The incorrect score comes from the downstream computation or simplification step\.

#### T3: Canonical hint acquired and final answer correct\.

This trace illustrates the intended successful loop\. The solver identifies the absent resistor, receives the missing value, and computes the exact equivalent resistance\.

```
Model/mode: gpt-4o-mini, zero-shot
Instance: cir-000924
Family: circuit_missing_resistance
Target: R_eq
Canonical missing fact held by B: R3 = 7 ohms

A requests: "the resistance of R3"
B offers: R3 = 7 ohms
A computes:
    G1 = R3 + R7 = 7 + 6 = 13.
    G2 = R2 + R5 = 11 + 4 = 15.
    G3 = R1 + G2 = 8 + 15 = 23.
    G4 = G3 + R6 = 23 + 9 = 32.
    G5 = G4 + R4 = 32 + 1 = 33.
    R_eq = (13 * 33) / (13 + 33) = 429/46.

A final answer: 429/46
Ground truth: 429/46.
Failure type: none; full request--resolve--solve success.
```

#### T4: Optional prompt\-anchoring snapshot for four\-shot analysis\.

This trace is included only to illustrate a prompt\-sensitivity hypothesis\. It should not be interpreted as a prevalence estimate\. In this moment\-recovery instance, the solver repeatedly asks for generic named moments rather than asking for the concrete missing equation held by Agent B\.

```
Model/mode: gemini-2.5-flash, four-shot
Instance: mom-000404
Family: moment_problem
Target: q0, q1, q2
A’s private equations: q0 + q1 + q2 = 12 q1 + 2q2 = 3
Canonical missing fact held by B: q1 + 4q2 = 5

A requests: "the 2nd moment"
B declines.
A requests: "the 1st moment"
B declines.
A requests: "the 0th moment"
B declines.
A requests: "the value of moment_0"
B declines.
A requests: "the value of moment_1"
B declines.
A requests: "the value of moment_2"
B declines.
A requests: "the 0th moment"
B declines.
A requests: "the 2nd moment"
B declines.
A requests: "the 3rd moment"
B declines.

Outcome: no answer submitted.
Ground truth: q0 = 10, q1 = 1, q2 = 1.
Interpretation: possible exemplar anchoring or mismatch between generic
moment labels and the concrete constraint held by the responder.
```

This example supports the qualitative interpretation in Section 6: four\-shot demonstrations may encourage protocol\-shaped requests without guaranteeing instance\-specific slot localization\. It is not used as evidence that this behavior is frequent\.