Exploring Structures in Physics Problems: Can AI Agents Discover Statistical Mechanical Mappings?

# Exploring Structures in Physics Problems: Can AI Agents Discover Statistical Mechanical Mappings?
Source: [https://arxiv.org/html/2607.26367](https://arxiv.org/html/2607.26367)
Wanyu Zhao Siebel School of Computing and Data Science University of Illinois Urbana\-Champaign wanyu2@illinois\.edu&Wanbing Zhao11footnotemark:1 Department of Physics and Astronomy Rice University wz56@rice\.edu

###### Abstract

An important skill in theoretical physics is to recognize when a new problem can be transformed into a known model\. We study this skill as an AI\-agent task: can LLM\-based agents discover statistical mechanical mappings from a raw partition function to a tractable representation? To probe this question, we introduceStatMechBench\-v0, a benchmark of six Ising\-type problems covering transfer\-matrix methods, gauge\-removable disorder, and planar/Pfaffian structure\. We evaluate a simple propose\-verify\-revise agent across multiple LLMs and problem phrasings\. The results show that numerical feedback often helps agents repair code and recover correct partition functions\. However, agents can also pass the numerical checks while misidentifying the underlying tractable class or understating computational complexity\. This both reveals limitations in current LLM reasoning and calls for a verification stack that goes beyond numerical agreement, incorporating, for example, symbolic checks and structural invariants\. Our study provides an early evaluation and design directions for AI agents aimed at structural discovery in theoretical physics\.

## 1Introduction

A common mindset in theoretical physics is to relate a new problem to existing theories, such as mapping newly formulated models to well\-studied ones\. Statistical mechanics, a key branch in physics, has developed a collection of paradigmatic models to understand how microscopic degrees of freedom give rise to macroscopic phases and critical phenomena\(Baxter,[1982](https://arxiv.org/html/2607.26367#bib.bib41)\)\. Examples include the Ising model\(Lenz,[1920](https://arxiv.org/html/2607.26367#bib.bib67); Ising,[1925](https://arxiv.org/html/2607.26367#bib.bib48)\), the Potts model\(Potts,[1952](https://arxiv.org/html/2607.26367#bib.bib39)\), and many other exactly solved lattice models\(Baxter,[1982](https://arxiv.org/html/2607.26367#bib.bib41)\)\. Developed and solved by generations of physicists over the past century, these models encode invaluable expert knowledge, and physicists seek transformations to connect new statistical mechanical problems to existing models, allowing known results, intuitions, and tools to be reused\.

While successfully relating one statistical mechanical model to another can itself be a profound physical result, as exemplified by the Kramers–Wannier duality\(Kramers and Wannier,[1941a](https://arxiv.org/html/2607.26367#bib.bib40)\), it also serves as one step in a broader research workflow\. For example, physicists identify a mapping from a new problem to an existing model and use it to compute quantities of interest, simplify the analysis, and gain qualitative understanding of the original problem\. However, recognizing or constructing such mappings is nontrivial: it often requires familiarity with existing models, mathematical or physical intuition about the structure of the original problem, and the ability to apply suitable transformations\. Historically, many important mappings have required substantial expert effort, and this difficulty can become a key bottleneck in research, especially for those outside the relevant subfield\.

In this work, we ask:Can we build an LLM\-based AI agent to discover statistical mechanical mappings?Formally, we view a mapping as a transformation from the original problem representation to a reference model whose solution structure is already known \(§[2](https://arxiv.org/html/2607.26367#S2)\)\. Our key insight is that LLMs could have learned about all the existing physics models as they are described both mathematically and in natural language across textbooks, papers, and lecture notes, and an LLM\-based agent may be able to propose plausible mapping hypotheses by combining model descriptions, symbolic patterns, and high\-level physical analogies, with its reasoning and self\-reflection abilities\.

We first build our insight and understanding of current LLMs’ ability in this domain through a systematic evaluation \(§[3](https://arxiv.org/html/2607.26367#S3)\)\. Specifically, we collect a set of representative statistical mechanical mapping problems and group them by difficulty into three tiers \(§[3\.1](https://arxiv.org/html/2607.26367#S3.SS1)\)\. Our preliminary evaluation on a subset of problems shows that LLMs can find the mappings and construct tractable partition\-function representations correctly with feedback loops, under both canonical and paraphrased problem descriptions \(§[3\.2](https://arxiv.org/html/2607.26367#S3.SS2), §[3\.3](https://arxiv.org/html/2607.26367#S3.SS3)\)\. We also identify several failure modes\. These observations both reveal the ability of current AI and shed light on our next step of building a discovery agent\.

Based on our problem categories and evaluation findings, we propose future statistical mechanical discovery agent designs and challenges \(§[4](https://arxiv.org/html/2607.26367#S4)\)\. Our ultimate goal is to support frontier research, e\.g\., error thresholds for quantum codes\(Denniset al\.,[2002](https://arxiv.org/html/2607.26367#bib.bib1); Chubb and Flammia,[2021](https://arxiv.org/html/2607.26367#bib.bib2)\)and phase\-transition questions in monitored quantum dynamics\(Skinneret al\.,[2019](https://arxiv.org/html/2607.26367#bib.bib3); Baoet al\.,[2020](https://arxiv.org/html/2607.26367#bib.bib47)\)\. More broadly, as an early attempt to concretely apply AI agents to open\-ended theoretical physics problems, we hope this work encourages physicists to further explore the role of LLMs in research\. By highlighting both the strengths and limitations of current agentic AI systems in assisting scientific discovery, this work aims to contribute to accelerating progress in science\.

## 2Background and Problem Formulation

The aim ofstatistical mechanicsis to explain and predict macroscopic behavior of a system from its microscopic degrees of freedom and interactions\.

In this section, we provide necessary background on statistical mechanics, following Ref\.\(Baxter,[1982](https://arxiv.org/html/2607.26367#bib.bib41)\), and then formalize the problem of “discovering statistical mechanical mappings"\. We end the section with a review of recent progress in AI agents for scientific discovery that inspires this work\.

### 2\.1Partition function and exactly solvable models

We begin with the partition functionZZ, which allows macroscopic thermodynamic observables to be obtained from microscopic configurations and will serve as our starting point for constructing a statistical\-mechanical mapping\.

Classical equilibrium statistical mechanics is formulated in terms of thepartition function111For concreteness, we describe the canonical ensemble at fixed temperature\. Other equilibrium ensembles have analogous partition functions\.:

Z=∑se−β​H​\(s\),Z=\\sum\_\{s\}e^\{\-\\beta H\(s\)\},\(1\)wheressdenotes a microscopic state, or configuration, of the system, e\.g\., for a system withnnbinary degrees of freedom,s=\(s1,…,sn\)s=\(s\_\{1\},\\dots,s\_\{n\}\)withsi∈\{−1,\+1\}s\_\{i\}\\in\\\{\-1,\+1\\\}fori∈\{1,…,n\}i\\in\\\{1,\\dots,n\\\};H​\(s\)H\(s\)denotes the energy of the configurationss; andβ\\betais the inverse temperature\.ZZis understood to depend on the physical model parameters enteringH​\(s\)H\(s\)and on temperature; this dependence is left implicit here\. The sum—for discrete systems—becomes an integral and a trace for continuous and quantum systems respectively\. Partition function is central to statistical mechanics because normalizinge−β​H​\(s\)e^\{\-\\beta H\(s\)\}byZZgives the probability distribution over microscopic configurationsss\(i\.e\.,p​\(s\)=e−β​H​\(s\)Zp\(s\)=\\frac\{e^\{\-\\beta H\(s\)\}\}\{Z\}\); then, macroscopic observables can be obtained by taking weighted averages over microscopic states\.

AlthoughZZlinks microscopic states to macroscopic observables, directly evaluating it as in Eq\.[1](https://arxiv.org/html/2607.26367#S2.E1)is intractable for realistic interacting222Here, “interacting” means that the energy functionH​\(s\)H\(s\)contains terms involving more than one degree of freedom, for example, pairwise interactions between theii\-th andjj\-th components ofs=\(s1,…,sn\)s=\(s\_\{1\},\\dots,s\_\{n\}\)\.systems of macroscopic size, which contain an enormous number of degrees of freedom, typically on the order of Avogadro’s number \(102310^\{23\}\)\. For example, consider a system ofnnbinary degrees of freedom, i\.e\.,s=\(s1,…,sn\)s=\(s\_\{1\},\\ldots,s\_\{n\}\)withsi∈\{0,1\}s\_\{i\}\\in\\\{0,1\\\}, a direct computation of Eq\.[1](https://arxiv.org/html/2607.26367#S2.E1)involves a sum over2n2^\{n\}\(n∼1023n\\sim 10^\{23\}\) configurations\. Consequently, one or both of the following strategies is adopted: \(i\) replacing the real system by a simplified idealization \(amodel\), specified by the statesssand the energy functionH​\(s\)H\(s\)chosen to capture the relevant physics while giving the partition function a structure that makes it tractable\. \(ii\) introducing approximations to evaluate the sum\.

In special cases, the structure of the model allows the partition function to be evaluated exactly; such models are usually called*exactly solvable models*\. To date, the catalog of exactly solvable models remains limited, but it forms a valuable theoretical library for physicists to use, marking the current boundary of analytical control\. Table[1](https://arxiv.org/html/2607.26367#S3.T1)summarizes four groups of models that have been exactly solved\.

ModelI\. TransformationsII\. Solvability structuresIII\. EvaluationZZPhysicists’reasoning

Figure 1:General logic underlying the exact evaluation of a partition functionZZ\.
### 2\.2Statistical mechanical mapping

We extract the general logic for physicists evaluating a partition functionZZin Figure[1](https://arxiv.org/html/2607.26367#S2.F1): one maps a model to an equivalent formulation in which solvable structures become manifest, thereby enabling an explicit evaluation of the partition function\. Similarly, this logic also guides the study of new models: when a model is first introduced to explain a new phenomenon, one natural first step in analyzing the model is to seek transformations that reveal whether it has solvable structure or can be connected to a known model\. To scale the search for transformations that may open tractable solution paths, we ask:

*Can an LLM\-based AI agent uncover the solvability structure of a problem by mapping it onto a known statistical mechanical model?*

We refer to such a task as a*statistical mechanical mapping*\. As an initial probe, this work focuses on cases where the quantity of interest is presented as a raw partition function or a raw partition\-function\-type sum of the form in Eq\.[1](https://arxiv.org/html/2607.26367#S2.E1)\(i\.e\., a sum over exponentially many configurations arising from the problem\); accordingly, “mapping” means the mathematical transformations bringing the raw partition function into the partition function of a known model, e\.g\., changes of variables, gauge transformations, and duality transformations\.

Generally, a successful partition\-function mapping has both physical and methodological value\. From a physical perspective, it can place a new problem within an existing universality333In particular, if a model with the same dimensionality, symmetry, and interaction range as a given physical system can be solved, universality implies identical critical behavior, characterized by the same critical exponents near phase transitions\.class\. From a methodological perspective, analytical control over the mapped model can be transferred back to the original problem\. Even when the mapped partition function is not exactly solvable, the mapping still provides physical insight and allows one to apply model\-specific numerical methods, such as Monte Carlo or tensor network methods\.

To the best of our knowledge, no prior work have investigated using LLM\-based agent to identify statistical mechanical mappings\. The closest work\(Guptaet al\.,[2025](https://arxiv.org/html/2607.26367#bib.bib65)\)uses machine learning algorithms to learn the resulting function mapped by duality \(a specific transformation\), while this work asks if an LLM agent can discover the mapping itself\.

### 2\.3AI agents for scientific discovery

Several LLM\-based agents have been proposed for scientific and mathematical discovery through program search\. FunSearch\(Romera\-Paredeset al\.,[2024](https://arxiv.org/html/2607.26367#bib.bib19)\)introduced an LLM\-guided evolutionary search framework that pairs a pretrained LLM with an automated evaluator to discover programs for mathematical and algorithmic problems\. AlphaEvolve\(Novikov and others,[2025](https://arxiv.org/html/2607.26367#bib.bib18)\)extends this line of work to a more general evolutionary coding agent that can modify larger codebases and optimize algorithms across scientific, mathematical, and engineering tasks\. In theoretical physics,Brenner and Colwell \([2026](https://arxiv.org/html/2607.26367#bib.bib14)\)combine Gemini Deep Think, tree search, and automated numerical feedback to derive analytical solutions for a cosmic\-string radiation problem by evaluating a core integral\. It builds on a broader LLM–tree\-search systems for scientific software generation called ERA\(Aygünet al\.,[2025](https://arxiv.org/html/2607.26367#bib.bib20)\), which formulate scientific software development as a scorable search problem and demonstrate results across multiple scientific domains\.

Several benchmarks have evaluated AI agents for physics problems\.Songet al\.\([2025](https://arxiv.org/html/2607.26367#bib.bib37)\)evaluate LLMs on scenario\-grounded scientific\-discovery tasks across biology, chemistry, materials science, and physics \(including solving an Ising model\)\. PRL\-Bench\(Miaoet al\.,[2026](https://arxiv.org/html/2607.26367#bib.bib77)\)provides a benchmark for evaluating theoretical and computational physics research tasks derived from recentPhysical Review Letterspapers, covering major subfields in modern physics including statistical physics\. This work targets the finer\-grained problem of statistical mechanical mappings, where success requires identification of the underlying tractable structure, enabling a more precise diagnosis of whether an AI agent can discover a physically meaningful reduction rather than merely producing a plausible final answer or automating a broad research workflow\.

## 3Probing LLM Agents for Statistical Mechanical Mapping

Table 1:Representative classes of exactly solved models, adapted from Ref\.\(Baxter,[1982](https://arxiv.org/html/2607.26367#bib.bib41)\)\.As statistical mechanical mapping \(§[2\.2](https://arxiv.org/html/2607.26367#S2.SS2)\) spans broad problem domains, touches diverse exactly solvable models and mathematical transformations, and can serve different purposes \(e\.g\., computing quantity of interest vs\. analyzing physics\), and as today’s LLM\-based agent has a large design space \(from prompt design to tool uses\) with the LLM capability remaining opaque \(e\.g\., memorization vs\. reasoning\), directly building a full end\-to\-end agent to achieve our task is unwieldy and challenging\. Therefore, we conduct a probe evaluation to first build an understanding of LLM capabilities and guide the agent design\. We first categorize statistical mechanical mapping problems into three tiers by difficulty and then evaluate a naive propose\-verify\-refine agent on a small benchmark comprising the easiest\-tier problems \(rediscover known ones\)\. Our evaluation shows promising results of using feedback\-guided LLM\-based agents to identify mappings\. We also reveal several failure modes that inform the agent design directions in §[4](https://arxiv.org/html/2607.26367#S4)\.

### 3\.1Three\-tier problems

We classify the statistical mechanical model mapping problems into three tiers by difficulty\. We instantiate and evaluate a Tier 1 benchmark in §[3\.2](https://arxiv.org/html/2607.26367#S3.SS2), with Tier 2 and 3 problem study left for future\.

#### Tier 1: canonical models with known exact solution mechanisms\.

Tier 1 consists of textbook canonical examples of exactly solved statistical mechanical models as listed in Table[1](https://arxiv.org/html/2607.26367#S3.T1)\. These models naturally compose a sanity\-check benchmark because the mechanisms for evaluating their partition functions—e\.g\., transfer\-matrix form, gauge\-removable disorder, or planarity/Pfaffian structure—are well understood and analytically controlled\. In this tier, the mapping agent’s task is to recover a known exact solution/reduction or exact tractable representation given the model’s original formulation\.

#### Tier 2: established mappings in quantum information literature\.

Tier 2 consists of known statistical mechanical mappings from the quantum information literature which are substantially more challenging to recover than those in Tier 1\. Examples include toric\-code threshold mappings with perfect or imperfect syndrome\(Denniset al\.,[2002](https://arxiv.org/html/2607.26367#bib.bib1)\), correlated\-noise or subsystem\-code generalizations\(Chubb and Flammia,[2021](https://arxiv.org/html/2607.26367#bib.bib2)\), coherent\-noise extensions\(Behrends and Béri,[2025](https://arxiv.org/html/2607.26367#bib.bib68)\), and replica mappings for monitored random circuits in the large local Hilbert\-space dimension limit\(Baoet al\.,[2020](https://arxiv.org/html/2607.26367#bib.bib47)\)\. In this tier, the mapping agent’s task, given a raw partition function, is to recover an effective energy functionH​\(s\)H\(s\)and identify any special parameter limits that enable the mapping\.

#### Tier 3: open but structured research problems\.

Tier 3 consists of open research problems for which statistical mechanical reformulations are expected to be useful, but for which no sufficiently\-tractable mapping is known\. One example is maximum\-likelihood threshold problems for structured quantum code families under various noise models, for which useful reductions remain unclear after a raw partition\-function representation is obtained; another example is analytical approaches to measurement\-induced entanglement phase transitions at finite local Hilbert\-space dimension\. In this tier, a useful agent output may be a partial reduction, an effective Hamiltonian, a candidate universality class, a numerically testable observable, or evidence that no simple mapping is likely\.

### 3\.2Experiment design

Table 2:StatMechBench\-v0: six tasks, four tractable classes covered\.nnis the number of spins;LLis the linear lattice size when applicable \(n=L2n=L^\{2\}\)\.#### Task problems\.

We select six Tier 1 tasks, all classical Ising\-type partition functions, listed in Table[2](https://arxiv.org/html/2607.26367#S3.T2)\. We name this small benchmarkStatMechBench\-v0\. The tasks are chosen to span diverse mapping types: canonical transfer\-matrix solvability \(P01–P02\), canonical gauge equivalence \(P05\), broadly\-documented tractable reformulations via a two\-dimensional transfer matrix \(P03\), a Pfaffian reduction for zero\-field planar Ising instances \(P04\), and a documented negative case with no assumed efficient exact mechanism in our predefined library \(P06\)\. Each task is labeled with atractable class444Tractable class is a unified name in operation level, it can be a target tractable model class, an algebraic transformation, an algorithm, or a graph structural property\. Note that each task can be a multi\-class problem because multiple tractability mechanisms could be applied\. Here inStatMechBench\-v0, exactly one label under our consideration applies\.and the corresponding computational complexity\.

#### A naive propose\-verify\-refine mapping agent\.

- •Mapping proposer\.The proposer takes the natural\-language task description and is asked to return a JSON object with fieldstractable\_class,complexity\_O,confidence,justification, andZ\_efficient\(params, n\)\(Python function\)555We also do a simple integrity check of code at the Abstract Syntax Tree \(AST\) stage \(e\.g\., illegal library imports\)\.\. Task description includes the statistical mechanical mapping problem formulation with a list of tractable class labels \(system prompt\), the specific problem description \(initial user prompt\), and failure feedback from earlier runs \(feedback user prompt\)\. We make sure no tractable class label and corresponding complexity are leaked in the prompt\. We provide nine tractable classes for each task to be classified into one\. The full prompt template and tractable class list are provided in Appendix[A](https://arxiv.org/html/2607.26367#A1)\.
- •Brute\-force numerical verifier\.We build a simple verifier that checks numerical equality by brute\-force: aZ\_brute\(params, n\)function that returns the exact Z via summarization over all2n2^\{n\}configurations\. Given a proposedZ\_efficient\(params, n\), we launch an evaluation process with a timeout, write the candidate code into it, and pass a list of\(params, n\)probes\. A trial passes only if all probe results are within a tolerance \(relative errorεrel=\|Zeff−Zbrute\|/max⁡\(\|Zbrute\|,10−300\)<10−6\\varepsilon\_\{\\text\{rel\}\}=\|Z\_\{\\text\{eff\}\}\-Z\_\{\\text\{brute\}\}\|/\\max\\bigl\(\|Z\_\{\\text\{brute\}\}\|,10^\{\-300\}\\bigr\)<10^\{\-6\}\)\. On failure a brief textual report listing whether the code ran, the number and rate of failed probes, and an order\-of\-magnitude relative error \(but notZb​r​u​t​eZ\_\{brute\}itself\) is constructed as feedback of its round\. We limit the number of rounds tokk\.

### 3\.3Experiments and results

#### Setup and evaluation metrics\.

We cover eight model configurations across four vendors: Claude Haiku 4\.5\(Anthropic,[2025](https://arxiv.org/html/2607.26367#bib.bib69)\), Claude Sonnet 4\.6\(Anthropic,[2026b](https://arxiv.org/html/2607.26367#bib.bib70)\), Claude Opus 4\.7\(Anthropic,[2026a](https://arxiv.org/html/2607.26367#bib.bib71)\), DeepSeek V4\-Flash in Thinking mode\(DeepSeek AI,[2026a](https://arxiv.org/html/2607.26367#bib.bib72); DeepSeek,[2026](https://arxiv.org/html/2607.26367#bib.bib73)\), DeepSeek V4\-Pro\(DeepSeek AI,[2026b](https://arxiv.org/html/2607.26367#bib.bib74)\), Gemini 3\.1\-Pro\(Google,[2026](https://arxiv.org/html/2607.26367#bib.bib75)\), and GPT\-5\.4\-mini with minimal and high reasoning effort\(OpenAI,[2026](https://arxiv.org/html/2607.26367#bib.bib76)\)\. We usek=5k=5verifier rounds, five parameter samples per probednn, and a per\-trial timeout of 30 s\. Each task is associated with two problem descriptions: a canonical version and a paraphrased variant\. Both descriptions are generated using LLMs and subsequently reviewed by a domain expert\.

Table 3:LLM performance onStatMechBench\-v0across eight models with canonical and paraphrased problem descriptions\. Opus 4\.7 delivers the best overall performance\.†DeepSeek\-V4\-Pro wall\-clock includes connection\-stall overhead\.∗False check: P03 emitscomplexity\_O =O\(n​2nn\\,2^\{\\sqrt\{n\}\}\), which*is*an honest acknowledgement of2L=2n2^\{L\}=2^\{\\sqrt\{n\}\}, but the audit’s regex anchors on the literal letterL

#### How do different LLMs perform?

Table[3](https://arxiv.org/html/2607.26367#S3.T3)reports tractable class classification accuracy of the first round \(TC0\), numerical evaluation pass rate of the first round \(NM0\), post\-hoc complexity honesty check score \(O\), saturation round numberk⋆k^\{\\star\}where the agent succeeds with 100% numerical evaluation pass and stops iteration, total wall\-clock time and cost\. We omit numerical pass rate with the totalk=5k\\\!=\\\!5loops from the table as all reach100%100\\%, so on canonical Tier 1 tasks, the loop alone cannot discriminate frontier models from each other\.

Overall, Opus 4\.7 performs best, achieving the highest classification accuracy, numerical evaluation pass rate, and complexity honesty score, with lowk∗k^\{\*\}\. The results also show a clear improvement across Claude models from earlier to later generations and from DeepSeek V4\-Flash Thinking to stronger DeepSeek V4\-Pro\.

In several cases, models pass the probed numerical tests while submitting exponential row\-transfer constructions under polynomial\-time labels such aspfaffian/O​\(n3\)O\(n^\{3\}\)\. The O audit detects these complexity mismatches and lowers the corresponding complexity check scores \(<<100%\)\. Three models remain honest under O on both problem description variants: Haiku 4\.5, Opus 4\.7 and GPT\-5\.4\-mini high\. This also indicates that numerical correctness alone is insufficient to certify the claimed tractable class\.

#### Does iterative refinement loop help?

Figure[2](https://arxiv.org/html/2607.26367#S3.F2)shows that numerical recovery usually occurs quickly\. Across models and problem description variants, many saturate within one verifier round\. Only a small number of trajectories use a substantial fraction of thek=5k=5budget, typically when magnitude\-only feedback cannot localize the structural source of the error\.

The recovery traces also show the loop’s limitation\. We find that when the initial solution is numerically wrong, the verifier can often drive a successful fallback, such as replacing a flawed Pfaffian attempt with a row\-transfer matrix\. However, when the code is already numerically correct but carries the wrong tractable\-class label, the loop has little signal to correct the label \(e\.g\., DeepSeek V4\-Flash Thinking and GPT\-5\.4\-mini minimal on paraphrased description in Table[3](https://arxiv.org/html/2607.26367#S3.T3)\)\. Thus the verifier is effective for numerical repair, but not for detecting class\-only mismatches\.

![Refer to caption](https://arxiv.org/html/2607.26367v1/x1.png)Figure 2:Cumulative numerical evaluation pass rates across self\-refinement rounds, with eight LLMs on canonical \(left\) and paraphrased \(right\) problem descriptions\. A trial first passing at roundjjis counted as passed for allk≥jk\\geq j\. Most reasoning models saturate by round 1, while only a few trajectories use much of thek=5k=5budget\.
#### How does it perform when we paraphrase the problem description?

By looking atΔ​TC0≡TC0,paraphrased−TC0,canonical\\Delta\\mathrm\{TC\}\_\{0\}\\equiv\\mathrm\{TC\}\_\{0,\\text\{paraphrased\}\}\-\\mathrm\{TC\}\_\{0,\\text\{canonical\}\}, a finding is that reasoning\-enabled LLMs are largely paraphrase\-invariant, whereas low\-effort or non\-reasoning ones lose TC0accuracy under paraphrase\.

Another notable pattern is that paraphrasing can improve zero\-shot numerical success \(see NM0of Sonnet 4\.6, Opus 4\.7 and DeepSeek V4\-Flash Thinking in Table[3](https://arxiv.org/html/2607.26367#S3.T3)\)\. In P04, the canonical wording tends to trigger an ambitious Pfaffian construction, which some models fail to implement reliably\. The paraphrased wording instead encourages a simpler row\-transfer solution that passes at the probed sizes; however, some still claim thepfaffianstructure, yielding lower honesty\. This reveals that LLMs may still rely on memorization rather than genuinely interpreting the mathematics and reasoning through the problem\.

#### How did the agent go wrong?

The evaluation reveals several failure modes\. Sign errors produce large round\-0 numerical mismatches but are usually repairable by magnitude\-only feedback; class\-only errors can pass numerically while retaining the wrong tractable\-class label, leaving the refinement loop with no signal for correction and motivating richer verification checks such as symbolic checks\. Canonical wording can induce memorized labels without reliable execution, which paraphrasing avoids by classifying to class with reliable execution but falsely claims the complexity, and some runs exhibit complexity false confidence: exponential implementations are paired with polynomial\-time claims and high self\-confidence\. The negative control behaves as intended: models consistently identify the no\-known\-efficient case and use brute force rather than forcing a spurious tractable label\. Overall, these failures show that numerical verification alone is weak: it repairs many implementation bugs, but misses class\-only and complexity\-claim errors\.

## 4From v0 to Open Problems

The probing evaluation in §[3](https://arxiv.org/html/2607.26367#S3)demonstrates the initial promise of LLM\-based agents for identifying statistical mechanical mappings, yet the benchmark is limited, and several failures occurred\. Below, we outline howStatMechBench\-v0can be systematically extended and discuss possible directions for improving the agent architecture\.

#### Benchmark

- •Task completion\.StatMechBench\-v0only covers a few tasks in Tier 1\. Moving forward, the benchmark should cover a larger set of all three tiers’ problems\. Broader problems will expand the set of the “tractable classes" as well, e\.g\., Tier 2 problems can require we do some algebraic transformation before having intuition about which tractable models they can potentially map to\. Tier 3 open problems might not have mappings to existing physical models; thus, we may pivot the agent output to giving a bound instead, potentially by leveraging partial structure and approximate evaluation\.
- •Task description\.We manually paraphrased the canonical problem description inStatMechBench\-v0\. In later version of the benchmark, we could dynamically paraphrase with predefined templates\.
- •Task scale\.We only covern≤16n\\leq 16\. Solving practical problems needs to scalenn, which will disable our brute\-force numerical verifier\.

#### Agent

- •Action space\.One next step is to define a structured action library as the agent’s search space\. Given a candidate partition functionZZ, an LLM–CAS \(Computer Algebra System\) pipeline first analyzes its structure by extracting the interaction hypergraph and computing invariants such as treewidth, planarity, symmetry group, and connectivity\. These structural features determine a restricted set of admissible transformations, expressed in a domain\-specific language \(DSL\)\. Within this constrained space, the agent performs symbolic manipulations—including gauge transformations, duality mappings, replica\-limit evaluations, and Jordan–Wigner transformations—with each intermediate step checked for algebraic consistency\. Correspondingly, the overall design will integrate multiple validation layers\. The action library provides both hints and constraints, which could improve search efficiency yet should not overly constrain the space\.
- •Evaluation stack\.A multi\-layer evaluation stack combining static complexity analysis, symbolic verification via a CAS backend, and graph\-theoretic invariants \(e\.g\., treewidth, planarity, symmetry group, connectivity\) can better support the mapping discovery\. For ground\-truth validation, we could extend brute\-force enumeration with scalable numerical oracles, including MCMC sampling at the Nishimori temperature and tensor\-network contraction on the Tanner graph, to enable system size ofn\>16n\>16\. Additional diagnostic signals, such as traceback\-style checks, can be fed back into the agent context to improve robustness\.
- •Search and evolution\.We could directly integrate with AlphaEvolve\(Novikov and others,[2025](https://arxiv.org/html/2607.26367#bib.bib18)\)for evolutionary search or adopt a tree\-search procedure following ERA\(Aygünet al\.,[2025](https://arxiv.org/html/2607.26367#bib.bib20)\)\. With tree search, each node corresponds to a candidate mapping procedure: the LLM proposes symbolic derivations, the CAS executes algebraic steps, and numerical oracles provide verifiable feedback\. If the errorε<δtol\\varepsilon<\\delta\_\{\\mathrm\{tol\}\}, the candidate is promoted to larger system sizes and tested for consistency \(e\.g\., phase\-transition behavior\)\. Otherwise, failed derivations orε\>δtol\\varepsilon\>\\delta\_\{\\mathrm\{tol\}\}incur penalties, and the search explores alternative hypotheses\.
- •Prompting\.Stronger prompts could be designed and systematically explored\. For example, negative prompting\(Brenner and Colwell,[2026](https://arxiv.org/html/2607.26367#bib.bib14)\)could be used to encourage structural diversity across search branches and mitigate premature convergence\. In addition, expert\-provided hints could lead to more efficient search\.
- •Human oversight\.Human\-in\-the\-loop supervision can/should be incorporated to audit intermediate results, validate high\-level reasoning, and guide exploration in ambiguous regimes\.

## 5Conclusion

This work takes a first step toward formulating statistical mechanical mapping as an agentic task for AI\-assisted theoretical physics\. The focus is distinguished from rigor\-oriented tasks on exactly solvable models; here the agent task is to recover representations whose usefulness includes their physical relevance to the original problem, rather than by formal equivalence alone\.

We focus on a restricted setting—raw partition\-function expressions with known tractable structure—and ask whether LLM\-based agents can recover useful representations from such inputs\. To this end, we introduce a three\-tier taxonomy of statistical\-mechanical mapping problems and instantiate its first tier—canonical models with known exact solution mechanisms—inStatMechBench\-v0\. Although small in scale,StatMechBench\-v0provides a controlled testbed for studying the capabilities of LLM\-based agents on this class of problems\. Our experiments show that verifier feedback can help agents repair implementation errors and reproduce finite\-size partition functions\. At the same time, our failure studies reveal limitations in current LLM reasoning and point toward future mapping\-discovery agents equipped with a multi\-layer evaluation stack, a structured action space, and more effective evolutionary or search mechanisms\.

## References

- Anthropic \(2025\)Introducing claude haiku 4\.5\.Note:Accessed 2026\-05\-27External Links:[Link](https://www.anthropic.com/news/claude-haiku-4-5)Cited by:[§3\.3](https://arxiv.org/html/2607.26367#S3.SS3.SSS0.Px1.p1.2)\.
- Anthropic \(2026a\)Introducing claude opus 4\.7\.Note:Accessed 2026\-05\-27External Links:[Link](https://www.anthropic.com/news/claude-opus-4-7)Cited by:[§3\.3](https://arxiv.org/html/2607.26367#S3.SS3.SSS0.Px1.p1.2)\.
- Anthropic \(2026b\)Introducing claude sonnet 4\.6\.Note:Accessed 2026\-05\-27External Links:[Link](https://www.anthropic.com/news/claude-sonnet-4-6)Cited by:[§3\.3](https://arxiv.org/html/2607.26367#S3.SS3.SSS0.Px1.p1.2)\.
- E\. Aygün, A\. Belyaeva, G\. Comanici, M\. Coram, H\. Cui, J\. Garrison, R\. J\. A\. Kast, C\. Y\. McLean, P\. Norgaard, Z\. Shamsi,et al\.\(2025\)An ai system to help scientists write expert\-level empirical software\.arXiv preprint arXiv:2509\.06503\.Cited by:[§2\.3](https://arxiv.org/html/2607.26367#S2.SS3.p1.1),[3rd item](https://arxiv.org/html/2607.26367#S4.I2.i3.p1.2)\.
- Y\. Bao, S\. Choi, and E\. Altman \(2020\)Theory of the phase transition in random unitary circuits with measurements\.Physical Review B101\(10\),pp\. 104301\.Cited by:[§1](https://arxiv.org/html/2607.26367#S1.p5.1),[§3\.1](https://arxiv.org/html/2607.26367#S3.SS1.SSS0.Px2.p1.1)\.
- R\. J\. Baxter and F\. Wu \(1973\)Exact solution of an ising model with three\-spin interactions on a triangular lattice\.Physical Review Letters31\(21\),pp\. 1294–1297\.External Links:[Document](https://dx.doi.org/10.1103/PhysRevLett.31.1294)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.5.4.2.1.1)\.
- R\. J\. Baxter \(1963\)Statistical mechanics of a one\-dimensional coulomb system with a uniform charge background\.Mathematical Proceedings of the Cambridge Philosophical Society59\(4\),pp\. 779–787\.External Links:[Document](https://dx.doi.org/10.1017/S0305004100003790)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.2.1.2.1.1)\.
- R\. J\. Baxter \(1971\)Eight\-vertex model in lattice statistics\.Physical Review Letters26\(14\),pp\. 832–833\.External Links:[Document](https://dx.doi.org/10.1103/PhysRevLett.26.832)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.5.4.2.1.1)\.
- R\. J\. Baxter \(1982\)Exactly solved models in statistical mechanics\.Academic Press,London\.External Links:ISBN 978\-0\-12\-083180\-7Cited by:[§1](https://arxiv.org/html/2607.26367#S1.p1.1),[§2](https://arxiv.org/html/2607.26367#S2.p2.1),[Table 1](https://arxiv.org/html/2607.26367#S3.T1)\.
- J\. Behrends and B\. Béri \(2025\)Statistical mechanical mapping and maximum\-likelihood thresholds for the surface code under generic single\-qubit coherent errors\.PRX Quantum6\(4\),pp\. 040305\.Cited by:[§3\.1](https://arxiv.org/html/2607.26367#S3.SS1.SSS0.Px2.p1.1)\.
- T\. H\. Berlin and M\. Kac \(1952\)The spherical model of a ferromagnet\.Physical Review86\(6\),pp\. 821–835\.External Links:[Document](https://dx.doi.org/10.1103/PhysRev.86.821)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.4.3.2.1.1)\.
- M\. P\. Brenner and L\. Colwell \(2026\)Solving an open problem in theoretical physics using AI\-assisted discovery\.External Links:2603\.04735Cited by:[§2\.3](https://arxiv.org/html/2607.26367#S2.SS3.p1.1),[4th item](https://arxiv.org/html/2607.26367#S4.I2.i4.p1.1)\.
- C\. T\. Chubb and S\. T\. Flammia \(2021\)Statistical mechanical models for quantum codes with correlated noise\.Annales de l’Institut Henri Poincaré D8\(2\),pp\. 269–321\.External Links:[Document](https://dx.doi.org/10.4171/AIHPD/105)Cited by:[§1](https://arxiv.org/html/2607.26367#S1.p5.1),[§3\.1](https://arxiv.org/html/2607.26367#S3.SS1.SSS0.Px2.p1.1)\.
- DeepSeek AI \(2026a\)DeepSeek\-v4\-flash\.Note:Accessed 2026\-05\-27External Links:[Link](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash)Cited by:[§3\.3](https://arxiv.org/html/2607.26367#S3.SS3.SSS0.Px1.p1.2)\.
- DeepSeek AI \(2026b\)DeepSeek\-v4\-pro\.Note:Accessed 2026\-05\-27External Links:[Link](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)Cited by:[§3\.3](https://arxiv.org/html/2607.26367#S3.SS3.SSS0.Px1.p1.2)\.
- DeepSeek \(2026\)Thinking mode\.Note:Accessed 2026\-05\-27External Links:[Link](https://api-docs.deepseek.com/guides/thinking_mode)Cited by:[§3\.3](https://arxiv.org/html/2607.26367#S3.SS3.SSS0.Px1.p1.2)\.
- E\. Dennis, A\. Kitaev, A\. Landahl, and J\. Preskill \(2002\)Topological quantum memory\.Journal of Mathematical Physics43\(9\),pp\. 4452–4505\.External Links:[Document](https://dx.doi.org/10.1063/1.1499754)Cited by:[§1](https://arxiv.org/html/2607.26367#S1.p5.1),[§3\.1](https://arxiv.org/html/2607.26367#S3.SS1.SSS0.Px2.p1.1)\.
- Google \(2026\)Gemini 3\.1 pro\.Note:Accessed 2026\-05\-27External Links:[Link](https://blog.google/innovation-and-ai/models-and-research/gemini-3-1-pro/)Cited by:[§3\.3](https://arxiv.org/html/2607.26367#S3.SS3.SSS0.Px1.p1.2)\.
- P\. Gupta, A\. E\. V\. Ferrari, and N\. Iqbal \(2025\)A machine learning approach to duality in statistical physics\.InProceedings of the 42nd International Conference on Machine Learning,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267,pp\. 21322–21341\.External Links:[Link](https://proceedings.mlr.press/v267/gupta25b.html)Cited by:[§2\.2](https://arxiv.org/html/2607.26367#S2.SS2.p5.1)\.
- P\. C\. Hemmer, M\. Kac, and G\. E\. Uhlenbeck \(1964\)On the van der waals theory of the vapor\-liquid equilibrium\. iii\. discussion of the critical region\.Journal of Mathematical Physics5\(1\),pp\. 60–74\.External Links:[Document](https://dx.doi.org/10.1063/1.1704065)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.3.2.2.1.1)\.
- E\. Ising \(1925\)Beitrag zur theorie des ferromagnetismus\.Zeitschrift für Physik31,pp\. 253–258\.External Links:[Document](https://dx.doi.org/10.1007/BF02980577)Cited by:[§1](https://arxiv.org/html/2607.26367#S1.p1.1),[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.2.1.2.1.1)\.
- M\. Kac, G\. E\. Uhlenbeck, and P\. C\. Hemmer \(1963\)On the van der waals theory of the vapor\-liquid equilibrium\. i\. discussion of a one\-dimensional model\.Journal of Mathematical Physics4\(2\),pp\. 216–228\.External Links:[Document](https://dx.doi.org/10.1063/1.1703946)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.3.2.2.1.1)\.
- H\. A\. Kramers and G\. H\. Wannier \(1941a\)Statistics of the two\-dimensional ferromagnet\. part i\.Physical Review60\(3\),pp\. 252–262\.External Links:[Document](https://dx.doi.org/10.1103/PhysRev.60.252)Cited by:[§1](https://arxiv.org/html/2607.26367#S1.p2.1),[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.5.4.2.1.1)\.
- H\. A\. Kramers and G\. H\. Wannier \(1941b\)Statistics of the two\-dimensional ferromagnet\. part ii\.Physical Review60\(3\),pp\. 263–276\.External Links:[Document](https://dx.doi.org/10.1103/PhysRev.60.263)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.5.4.2.1.1)\.
- A\. Lenard \(1961\)Exact statistical mechanics of a one\-dimensional system with coulomb forces\.Journal of Mathematical Physics2\(5\),pp\. 682–693\.External Links:[Document](https://dx.doi.org/10.1063/1.1703742)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.2.1.2.1.1)\.
- W\. Lenz \(1920\)Beitrag zum verständnis der magnetischen erscheinungen in festen körpern\.Physikalische Zeitschrift21,pp\. 613–615\.Cited by:[§1](https://arxiv.org/html/2607.26367#S1.p1.1)\.
- E\. H\. Lieb \(1967a\)Exact solution of the f model of an antiferroelectric\.Physical Review Letters18\(24\),pp\. 1046–1048\.External Links:[Document](https://dx.doi.org/10.1103/PhysRevLett.18.1046)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.5.4.2.1.1)\.
- E\. H\. Lieb \(1967b\)Exact solution of the two\-dimensional slater kdp model of a ferroelectric\.Physical Review Letters19\(3\),pp\. 108–110\.External Links:[Document](https://dx.doi.org/10.1103/PhysRevLett.19.108)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.5.4.2.1.1)\.
- E\. H\. Lieb \(1967c\)Residual entropy of square ice\.Physical Review162\(1\),pp\. 162–172\.External Links:[Document](https://dx.doi.org/10.1103/PhysRev.162.162)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.5.4.2.1.1)\.
- T\. Miao, W\. Jin, M\. Zhang, J\. Tan, Y\. Hu, T\. Guo, J\. Zhang, Y\. Wang, W\. Li, Y\. Gao, S\. Chen, W\. Jiang, Y\. Hu, Z\. Lei, X\. Pang, Z\. Liu, Y\. Zhang, L\. Zhang, K\. Chen, W\. Wang, W\. E, and S\. Chen \(2026\)PRL\-bench: a comprehensive benchmark evaluating llms’ capabilities in frontier physics research\.External Links:2604\.15411,[Link](https://arxiv.org/abs/2604.15411)Cited by:[§2\.3](https://arxiv.org/html/2607.26367#S2.SS3.p2.1)\.
- E\. W\. Montroll \(1949\)Statistical mechanics of nearest neighbor systems\.Il Nuovo Cimento6,pp\. 264–281\.Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.4.3.2.1.1)\.
- A\. Novikovet al\.\(2025\)[AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://arxiv.org/abs/2506.13131)\.Cited by:[§2\.3](https://arxiv.org/html/2607.26367#S2.SS3.p1.1),[3rd item](https://arxiv.org/html/2607.26367#S4.I2.i3.p1.2)\.
- L\. Onsager \(1944\)Crystal statistics\. i\. a two\-dimensional model with an order\-disorder transition\.Physical Review65\(3–4\),pp\. 117–149\.External Links:[Document](https://dx.doi.org/10.1103/PhysRev.65.117)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.5.4.2.1.1)\.
- OpenAI \(2026\)GPT\-5\.4 mini\.Note:Accessed 2026\-05\-27External Links:[Link](https://openai.com/index/gpt-5-4-mini/)Cited by:[§3\.3](https://arxiv.org/html/2607.26367#S3.SS3.SSS0.Px1.p1.2)\.
- R\. B\. Potts \(1952\)Some generalized order\-disorder transformations\.Mathematical Proceedings of the Cambridge Philosophical Society48\(1\),pp\. 106–109\.External Links:[Document](https://dx.doi.org/10.1017/S0305004100027419)Cited by:[§1](https://arxiv.org/html/2607.26367#S1.p1.1)\.
- B\. Romera\-Paredes, M\. Barekatain, A\. Novikov, M\. Balog, M\. P\. Kumar, E\. Dupont, F\. J\. R\. Ruiz, J\. S\. Ellenberg, P\. Wang, O\. Fawzi, P\. Kohli, and A\. Fawzi \(2024\)Mathematical discoveries from program search with large language models\.Nature625\(7995\),pp\. 468–475\.External Links:ISSN 1476\-4687,[Document](https://dx.doi.org/10.1038/s41586-023-06924-6),[Link](https://doi.org/10.1038/s41586-023-06924-6)Cited by:[§2\.3](https://arxiv.org/html/2607.26367#S2.SS3.p1.1)\.
- B\. Skinner, J\. Ruhman, and A\. Nahum \(2019\)Measurement\-induced phase transitions in the dynamics of entanglement\.Physical Review X9\(3\),pp\. 031009\.External Links:[Document](https://dx.doi.org/10.1103/PhysRevX.9.031009)Cited by:[§1](https://arxiv.org/html/2607.26367#S1.p5.1)\.
- Z\. Song, J\. Lu, Y\. Du, B\. Yu, T\. M\. Pruyn, Y\. Huang, K\. Guo, X\. Luo, Y\. Qu, Y\. Qu, Y\. Wang, H\. Wang, J\. Guo, J\. Gan, P\. Shojaee, D\. Luo, A\. M\. Bran, G\. Li, Q\. Zhao, S\. L\. Luo, Y\. Zhang, X\. Zou, W\. Zhao, Y\. F\. Zhang, W\. Zhang, S\. Zheng, S\. Zhang, S\. T\. Khan, M\. Rajabi\-Kochi, S\. Paradi\-Maropakis, T\. Baltoiu, F\. Xie, T\. Chen, K\. Huang, W\. Luo, M\. Fang, X\. Yang, L\. Cheng, J\. He, S\. Hassoun, X\. Zhang, W\. Wang, C\. K\. Reddy, C\. Zhang, Z\. Zheng, M\. Wang, L\. Cong, C\. P\. Gomes, C\. Hsieh, A\. Nandy, P\. Schwaller, H\. J\. Kulik, H\. Jia, H\. Sun, S\. M\. Moosavi, and C\. Duan \(2025\)Evaluating large language models in scientific discovery\.External Links:2512\.15567,[Link](https://arxiv.org/abs/2512.15567)Cited by:[§2\.3](https://arxiv.org/html/2607.26367#S2.SS3.p2.1)\.
- H\. E\. Stanley \(1968\)Spherical model as the limit of infinite spin dimensionality\.Physical Review176\(2\),pp\. 718–722\.External Links:[Document](https://dx.doi.org/10.1103/PhysRev.176.718)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.4.3.2.1.1)\.
- G\. E\. Uhlenbeck, P\. C\. Hemmer, and M\. Kac \(1963\)On the van der waals theory of the vapor\-liquid equilibrium\. ii\. discussion of the distribution functions\.Journal of Mathematical Physics4\(2\),pp\. 229–247\.External Links:[Document](https://dx.doi.org/10.1063/1.1703947)Cited by:[Table 1](https://arxiv.org/html/2607.26367#S3.T1.7.3.2.2.1.1)\.

## Appendix

The appendices document everything needed to inspect the agent’s behavior and replicate the probe evaluation\. App\.[A](https://arxiv.org/html/2607.26367#A1)reproduces the proposer prompts verbatim\. App\.[B](https://arxiv.org/html/2607.26367#A2)gives one card per task inStatMechBench\-v0, with both the canonical and paraphrased descriptions\. App\.[C](https://arxiv.org/html/2607.26367#A3)documents the sandboxed verifier \(AST allowlist, subprocess construction, feedback string template\)\.

## Appendix AProposer prompts

The proposer is steered by a single system prompt plus one of two user templates \(initial vs\. feedback round\)\.

#### System prompt\.

The system prompt string:

Youarearesearchassistantparticipatinginabenchmarkofpartition\-function

mappingdiscovery\.

Yourjob:givenadescriptionofastatistical\-mechanicsmodelH\(s\),produce

aJSONobjectthatclassifiesitintoaknowntractableclassandprovides

anEXECUTABLEPythonfunction‘Z\_efficient\(params,n\)‘thatcomputesthe

partitionfunction

Z=sum\_\{s\}exp\(\-beta\*H\(s\)\)

ideallyintimepolynomialinn\(or,wheretheproblemisintractablein

theworstcase,anhonestbrute\-forcefallback\)\.

\{LIBRARY\}

OutputFORMAT\(asingleJSONobject;aMarkdown‘‘‘jsonfenceisfinebutnotrequired\):

\{

"tractable\_class":"<oneofthelibrarylabelsabove\>",

"complexity\_O":"<e\.g\.’O\(n\)’or’O\(n^3\)’or’O\(2^n\)’\>",

"confidence":<afloatin\[0,1\]\>,

"justification":"<2\-6sentences:whichtransformationmakesZtractableandwhy\>",

"code":"<purePythonsourcedefiningZ\_efficient\(params,n\)\-\>float;onlystandardlibs\+numpy\>"

\}

Rules:

\-ThecodeMUSTdefineatop\-levelfunction‘Z\_efficient\(params,n\)‘returningaPythonfloat\(oranyvaluelosslesslyconvertibletoone\)\.

\-Youmayimport‘math‘,‘numpy‘,‘scipy‘,‘itertools‘,‘functools‘\.DoNOTimport‘os‘,‘sys‘,‘subprocess‘,‘socket‘,oranynetworking/IOmodule,anddonotreadorwritefiles\.

\-Ifyougenuinelybelievenopolynomialalgorithmexists,set‘tractable\_class‘to‘"no\_efficient\_known"‘andprovideanhonestbrute\-forceimplementation;doNOTpretend\.

\-Yourcodewillbeexecutedinasubprocesswitha30stimeout;budgetyourenumerationaccordingly\.

\-Yourcodewillbecheckednumericallyagainstbrute\-forceenumerationwithrtol=1e\-6\.

#### Tractable\-class library \(LIBRARY\)\.

Substituted into the system prompt\. The labels here are the only legal values fortractable\_class\.

Libraryoftractablepartition\-functionclasses\(youmustclassifyintoone\):

\*"transfer\_matrix"\-\-1D/quasi\-1Dsystems;row\-by\-rowupdate;O\(n\)orO\(n\*2^L\)\.

\*"free\_fermion"\-\-quadraticfermionicHamiltonianafterJordan\-Wigner;O\(n^3\)viadiagonalization\.

\*"pfaffian"\-\-planarIsing/dimermodel;Kasteleynorientation;O\(n^3\)\.

\*"gauge"\-\-disorderremovablebyasign\-flipgauge\(e\.g\.Mattis\);reducestoknownmodel\.

\*"percolation"\-\-bond/sitepercolationuniversality;analyticcriticalpoints\.

\*"rbim\_nishimori"\-\-random\-bondIsingontheNishimoriline\(e\.g\.toriccodethreshold\)\.

\*"treewidth\_bounded"\-\-junction\-treealgorithm;O\(2^k\*n\)fortreewidthk\.

\*"exactly\_solvable\_other"\-\-ANYOTHERknownpolynomial\-timeclass;youmustnameitexplicitly\.

\*"no\_efficient\_known"\-\-noknownpolynomial\-timealgorithm;honestbruteforceisacceptable\.

The tractable class library mixes categories at different operational levels; Table[4](https://arxiv.org/html/2607.26367#A1.T4)further explains the underlying types\.

Table 4:Details of tractable classes selected in this work\.
#### Initial user prompt\.

Round 0 of every trial\.

Hereistheproblem\.

\{task\_description\}

PleaserespondwithONLYtheJSONobjectdescribedinthesysteminstructions\.

#### Feedback user prompt\.

Roundsk≥1k\\geq 1of every trial\. The\{prev\_json\}slot is filled with the proposer’s raw round\-\(k−1\)\(k\-1\)response;\{feedback\}is the verifier string from App\.[C](https://arxiv.org/html/2607.26367#A3)\.

Youpreviouslyrespondedtothisproblem:

\-\-\-problem\-\-\-

\{task\_description\}

\-\-\-endproblem\-\-\-

Yourpreviousresponsewas:

\{prev\_json\}

Theverifierreports:

\{feedback\}

Pleaserevise\.ReplywithaNEWJSONobjectonly\(sameschema\)\.Ifyoucan

identifythebuginyourpreviouscode,fixit\.Ifyourtractableclasswas

wrong,chooseadifferentone\.

## Appendix BTask cards \(StatMechBench\-v0\)

One card per task\. Each card shows the tractable class and complexity, the probed sizesnn, the canonical description, the paraphrased description, and the optional hint \(unused in the evaluation\)\.

#### How the paraphrases were produced\.

We followed a fixed rewrite rule to generate the paraphrased description:

1. 1\.Strip canonical model names\.“Ising”, “spin glass”, “Pfaffian”, “Mattis”, “Hamiltonian”, “ferromagnet”, “torus” are removed; geometric structure is described in plain words \(“L×LL\\times Lgrid of binary variables that wraps in both directions” in place of “L×LL\\times LIsing torus”\)\.
2. 2\.Rename the spin variable\.The canonical statement usessi∈\{−1,\+1\}s\_\{i\}\\in\\\{\-1,\+1\\\}; the paraphrase usesxix\_\{i\}to remove the near\-deterministic “spin” association\.
3. 3\.Preserve theZZexpression literally\.The number of configurations \(2n2^\{n\}\), the index ranges, the coupling structure, and the exponent insideexp⁡\(⋅\)\\exp\(\\cdot\)are reproduced unchanged\. P05 is the strongest illustration: the canonical statement names the Mattis model and gives the rank\-1 formJi​j=ξi​ξjJ\_\{ij\}=\\xi\_\{i\}\\xi\_\{j\}; the paraphrase substitutes the already\-contracted scalarM​\(x\)=∑iξi​xiM\(x\)=\\sum\_\{i\}\\xi\_\{i\}x\_\{i\}and asks for∑xexp⁡\(\(β/2\)​\(M​\(x\)2−n\)\)\\sum\_\{x\}\\exp\\\!\\bigl\(\(\\beta/2\)\(M\(x\)^\{2\}\-n\)\\bigr\)without naming the model\.
4. 4\.Do not add or remove information\.The paraphrase contains no extra hint and no extra constraint relative to the canonical description; the optional hint field is the only place hints live\.

The fixed\-string design makes the paraphrase set a reproducible benchmark artifact\. Its quality is bounded by the curation rules; we view this as acceptable for v0 and discuss*programmatic*rewriting \(templated variable renaming, structural isomorphism over a graph DSL\) as a v1 extension in §[4](https://arxiv.org/html/2607.26367#S4)\.

#### P01—1D Ising chain, OBC\.

Tractable classtransfer\_matrix, gold complexityO​\(n\)O\(n\)\. Probed sizesn∈\{6,10,14\}n\\in\\\{6,10,14\\\}\.

\[BLIND\]

Youaregiventhe\*\*1DIsingchainwithopenboundaryconditions\*\*\.

Hamiltonian:H\(s\)=\-J\*sum\_\{i=0\}^\{n\-2\}s\_i\*s\_\{i\+1\},withs\_iin\{\-1,\+1\}\.

Goal:writeaPythonfunction

defZ\_efficient\(params:dict,n:int\)\-\>float:

\#paramshaskeys"beta"\(float\)and"J"\(float\)

thatcomputesthepartitionfunction

Z\(beta,J,n\)=sum\_\{sin\{\-1,\+1\}^n\}exp\(\-beta\*H\(s\)\)

intimepolynomialinn\(NOTexponential\)\.

\[PARAPHRASED\]

Youaregivenasequenceofnbinaryvariablesx\_1,\.\.\.,x\_nwitheach

x\_iin\{\-1,\+1\},plustworealparametersbetaandJ\.Compute

Z\(beta,J,n\)=sumoverall2^nbinarytuples\(x\_1,\.\.\.,x\_n\)of

exp\(beta\*J\*sum\_\{i=1\}^\{n\-1\}x\_i\*x\_\{i\+1\}\)\.

Naiveenumerationofall2^ntuplesistooslow;yourimplementationshould

runintimepolynomialinn\.

#### P02—1D Ising chain with random site fields, OBC\.

Tractable classtransfer\_matrix,O​\(n\)O\(n\)\. Probedn∈\{6,10,12\}n\\in\\\{6,10,12\\\}\.

\[BLIND\]

Youaregivena\*\*1DIsingchainwithrandomsitefieldsandopenboundaryconditions\*\*\.

Hamiltonian:H\(s\)=\-J\*sum\_\{i=0\}^\{n\-2\}s\_is\_\{i\+1\}\-sum\_\{i=0\}^\{n\-1\}h\_is\_i\.

Implement

defZ\_efficient\(params:dict,n:int\)\-\>float:

\#paramshas"beta"\(float\),"J"\(float\),"h"\(listoflengthn\)

\[PARAPHRASED\]

Youaregivenasequenceofnbinaryvariablesx\_1,\.\.\.,x\_nin\{\-1,\+1\},

acouplingconstantJ,andanarrayofnper\-positionweightsh\_1,\.\.\.,h\_n\.

Compute

Z\(beta,J,h,n\)=sumoverall2^ntuplesxof

exp\(beta\*J\*sum\_\{i=1\}^\{n\-1\}x\_i\*x\_\{i\+1\}

\+beta\*sum\_\{i=1\}^\{n\}h\_i\*x\_i\)\.

#### P03—2D ferromagnetic Ising onL×LL\\\!\\times\\\!Ltorus\.

Tractable classtransfer\_matrix, gold complexitypoly​\(L\)⋅2L\\text\{poly\}\(L\)\\cdot 2^\{L\}\. Probedn∈\{9,16\}n\\in\\\{9,16\\\}\(i\.e\.L∈\{3,4\}L\\in\\\{3,4\\\}\)\.

\[BLIND\]

Youaregiventhe\*\*2DferromagneticIsingmodelonanLxLtorus\*\*

\(periodicboundaryconditionsinbothdirections\)\.

H\(s\)=\-J\*sum\_\{<i,j\>\}s\_is\_joverallnearest\-neighbourpairs\.

ZmustbecomputedintimepolynomialinL\(soexponentialinsqrt\(n\)is

acceptable,butexponentialinnisnot\)\.

\[PARAPHRASED\]

LxLgridofbinaryvariablesx\_\{r,c\}in\{\-1,\+1\};thegridwrapsinboth

directions\.Eisthesetofalluniqueneighbourpairs\(2\*L^2entries\)\.

ComputeZ\(beta,J,n\)=sumover2^nassignmentsof

exp\(beta\*J\*sum\_\{\(i,j\)inE\}x\_i\*x\_j\)\.

\[HINT,unused\]Considerarow\-to\-rowtransfermatrixofsize2^Lby2^L\.

#### P04—2D Ising spin glass on planar grid \(open\)\.

Tractable classpfaffian,O​\(n3\)O\(n^\{3\}\)\. Probedn∈\{9,16\}n\\in\\\{9,16\\\}\.

\[BLIND\]

Youaregivena\*\*2DIsingspinglassonanLxLOPEN\-boundaryplanar

lattice\*\*withrandombinarycouplingsoneachedge\.

H\(s\)=\-sum\_\{horizedges\}Jh\[r,c\]\*s\[r,c\]\*s\[r,c\+1\]

\-sum\_\{vertedges\}Jv\[r,c\]\*s\[r,c\]\*s\[r\+1,c\]\.

Zmustbepolynomialinn\(NOTexponential\)\.

\[PARAPHRASED\]

LxLgridofbinaryvariablesx\_\{r,c\};griddoesNOTwrap\.Eachhorizontal

edgecarriesabinaryweightJh\[r\]\[c\]in\{\-1,\+1\};eachverticaledge

Jv\[r\]\[c\]in\{\-1,\+1\}\.Compute

Z\(beta,Jh,Jv,n\)=sumoverall2^nassignmentsxof

exp\(beta\*sum\_\{r,c:c<L\-1\}Jh\[r\]\[c\]\*x\_\{r,c\}\*x\_\{r,c\+1\}

\+beta\*sum\_\{r,c:r<L\-1\}Jv\[r\]\[c\]\*x\_\{r,c\}\*x\_\{r\+1,c\}\)

\[HINT,unused\]Theinteractiongraphisplanar\.

#### P05—Mattis spin glass \(rank\-1 disorder\)\.

Tractable classgauge,O​\(n\)O\(n\)\. Probedn∈\{6,10,12\}n\\in\\\{6,10,12\\\}\.

\[BLIND\]

\*\*Mattis\-likespinglassonnspins\*\*withall\-to\-allcoupling\.

J\_\{ij\}=xi\_i\*xi\_jfori\!=j,withxiafixed\+/\-1pattern\.

H\(s\)=\-\(1/2\)\*sum\_\{i\!=j\}J\_\{ij\}s\_is\_j=\-\(1/2\)\*\[\(sum\_ixi\_is\_i\)^2\-n\]\.

Z\(beta,xi,n\)intimepolynomialinn\.

\[PARAPHRASED\]

nbinaryvariablesx\_1,\.\.\.,x\_nin\{\-1,\+1\}plusafixedpatternvectorxi\.

ForeachxdefineM\(x\)=sum\_ixi\_i\*x\_i\.Compute

Z\(beta,xi,n\)=sumover2^nassignmentsofexp\(\(beta/2\)\*\(M\(x\)^2\-n\)\)\.

\[HINT,unused\]ThecouplingmatrixJ\_\{ij\}=xi\_i\*xi\_jisrank\-1\.

#### P06—Random 3\-regular±1\\pm 1Ising glass\.

Tractable classno\_efficient\_known,O​\(2n\)O\(2^\{n\}\)\(best known on general instances\)\. Probedn∈\{8,10\}n\\in\\\{8,10\\\}\.

\[BLIND\]

\*\*Isingspinglassonarandom3\-regulargraph\*\*withbinary\+/\-1couplings\.

Edgesgivenasedgelist;H\(s\)=\-sum\_kJ\[k\]\*s\[edges\[k\]\[0\]\]\*s\[edges\[k\]\[1\]\]\.

"Ifyoudetermine,afteranalysingthestructureoftheinteractiongraph,

thatnopolynomial\-timealgorithminanyclassfromthelibraryapplies,

statesoexplicitlyinyourjustificationandprovideyourbest

implementation;theverifierwillonlycallyouatsmalln\."

\[PARAPHRASED\]

Graphonnverticesspecifiedbyedgelist,everyvertexhasdegree3\.

ComputeZ\(beta,edges,J,n\)=sumover2^nassignmentsof

exp\(beta\*sum\_kJ\[k\]\*x\[edges\[k\]\[0\]\]\*x\[edges\[k\]\[1\]\]\)\.

## Appendix CVerifier internals

The verifier is an integrity check on a cooperative proposer, not an adversarial sandbox\. Its job is to \(i\) reject obvious filesystem / network access at parse time, \(ii\) execute the candidateZ\_efficienton a list of\(n,params\)\(n,\\text\{params\}\)probes in isolation, and \(iii\) compare each returned value againstZbruteZ\_\{\\text\{brute\}\}atεrel=10−6\\varepsilon\_\{\\text\{rel\}\}=10^\{\-6\}\.

#### AST import allowlist\.

A static import check walks the candidate’s AST and rejects anyimportorfromstatement whose top\-level module is not in:

ALLOWED\_IMPORTS=\{

"math","numpy","scipy","itertools","functools",

"operator","fractions","collections","cmath","decimal",

"dataclasses","typing",

\}

Sub\-modules thereof \(e\.g\.scipy\.linalg\) are allowed\.os,sys,subprocess,socket,pathlib,requests, … are all rejected before the subprocess starts\. The allowlist only exists to surface accidental IO, so any adversarial bypasses \(\_\_import\_\_\(‘os’\), monkey\-patchedgetattrchains\) may still pass\.

#### Subprocess template\.

Every probe set is run in a freshly\-launched Python subprocess viasubprocess\.runwith a 30s wall\-clock timeout,stdincarrying a JSON list of calls andstdoutcarrying a single sentinel\-prefixed line:

importjson,sys,math,itertools,functools

importnumpyasnp

def\_to\_float\(x\):

try:returnfloat\(x\)

except\(TypeError,ValueError\):

arr=np\.asarray\(x\)

ifarr\.size==1:returnfloat\(arr\.reshape\(\(\)\)\.item\(\)\)

raiseTypeError\(f"Z\_efficientreturnednon\-scalarofshape\{arr\.shape\}"\)

\#\-\-\-\-\-\-\-\-\-\-candidatecodestarts\-\-\-\-\-\-\-\-\-\-

\_\_CANDIDATE\_CODE\_\_

\#\-\-\-\-\-\-\-\-\-\-candidatecodeends\-\-\-\-\-\-\-\-\-\-

if’Z\_efficient’notindir\(\):

print\("\_\_VERIFIER\_RESULT\_\_"\+json\.dumps\(\{"error":"missingZ\_efficient"\}\)\)

sys\.exit\(0\)

req=json\.loads\(sys\.stdin\.read\(\)\)

out=\[\]

forcallinreq\["calls"\]:

try:

z=Z\_efficient\(call\["params"\],call\["n"\]\)

out\.append\(\{"ok":True,"z":\_to\_float\(z\)\}\)

exceptExceptionase:

out\.append\(\{"ok":False,"error":f"\{type\(e\)\.\_\_name\_\_\}:\{e\}"\}\)

print\("\_\_VERIFIER\_RESULT\_\_"\+json\.dumps\(\{"results":out\}\)\)

The sentinel prefix\_\_VERIFIER\_RESULT\_\_isolates the result line from any debugprintthe candidate may emit; The subprocessstdoutparser scans the lines in reverse and returns the first line it encounters with this prefix\.

#### Environment\-variable allowlist\.

The subprocess inherits only:

PATH,HOME,LANG,LC\_ALL,LC\_CTYPE,

TMPDIR,TEMP,TMP,PYTHONPATH,

LD\_LIBRARY\_PATH,DYLD\_LIBRARY\_PATH\.

Every other environment variable is dropped, so API keys and other credentials in the parent shell are not exposed to candidate code\.

#### Probe construction\.

For eachnn, we draw some independent parameter samples \(default 5 samples\) with a random seed, so the probe set is deterministic given\(seed, task\)\.ZbruteZ\_\{\\text\{brute\}\}is computed in the parent process; only the candidate’sZeffZ\_\{\\text\{eff\}\}runs in the subprocess\.

#### Pass criterion\.

A trial*passes*only if every probe returned a finite float and satisfiedεrel=\|Zeff−Zbrute\|/max⁡\(\|Zbrute\|,10−300\)<10−6\\varepsilon\_\{\\text\{rel\}\}=\|Z\_\{\\text\{eff\}\}\-Z\_\{\\text\{brute\}\}\|/\\max\\bigl\(\|Z\_\{\\text\{brute\}\}\|,10^\{\-300\}\\bigr\)<10^\{\-6\}\.

#### Feedback string template\.

The verifier also produces a\{feedback\}string, which is pasted into the next round’s user prompt \(see App\.[A](https://arxiv.org/html/2607.26367#A1)\)\. Three branches:

\#\(a\)Codedidn’trunatall:

"Yourcodedidnotrun\.Error:\{report\.error\}"

\#\(b\)Coderanbuteveryprobeagreed:

"Verifierranon\{N\}\(n,params\)samples\.

Allsamplespassed\(maxrelativeerror\{max\_rel\_err:\.2e\}\)\."

\#\(c\)Coderanandatleastoneprobedisagreed:

"Verifierranon\{N\}\(n,params\)samples\.

\{k\}/\{N\}samplesfailed\.

\-relativeerror\{rel\_err:\.2e\}\>\>1e\-06\-\-yourcomputedZdisagreeswithbruteforce\.

\-relativeerror\{rel\_err:\.2e\}\>\>1e\-06\-\-yourcomputedZdisagreeswithbruteforce\.

\-relativeerror\{rel\_err:\.2e\}\>\>1e\-06\-\-yourcomputedZdisagreeswithbruteforce\."

At most three failed\-probe lines are listed\.ZbruteZ\_\{\\text\{brute\}\}itself is never disclosed\. This limits output verbosity and information leakage\.