NLData2Opt：对从自然语言和数据建模优化问题中的参数推导与预测进行基准测试

# NLData2Opt: Benchmarking Parameter Derivation and Prediction for Modeling Optimization Problems from Natural Language and Data
Source: [https://arxiv.org/html/2610.22889](https://arxiv.org/html/2610.22889)
Kyohei AtarashiJiyi LiKoh TakeuchiShunji UmetaniHisashi KashimaAffiliation:Kyoto UniversityAffiliation:Hokkaido UniversityAffiliation:Recruit Co\., Ltd\.

December 2025

###### Abstract

Formulating optimization problems from raw inputs of problem descriptions and data is time\-consuming and costly\. While Large Language Models \(LLMs\) offer a promising way to automate this process, existing benchmarks overestimate their capabilities by relying on idealized inputs where all data is relevant and complete\. This diverges from reality, where data\-centric reasoning is required to extract, compute, and predict parameters from noisy sources\. To address this, we introduce NLData2Opt, a benchmark to evaluate LLMs on the multi\-stage reasoning inherent in practical optimization workflows\. NLData2Opt decomposes problems into problem descriptions and data, and requires models to \(1\) extract relevant parameters from data containing misleading attributes, \(2\) compute missing parameters derived by transforming existing data attributes, and \(3\) predict unavailable future values from historical observations\. Evaluation results show substantial performance decline relative to existing benchmarks, indicating that benchmark progress does not directly translate to formulation accuracy in practical optimization workflows\. NLData2Opt is available at[https://github\.com/sumarivyn/NLData2Opt](https://github.com/sumarivyn/NLData2Opt)\.

## 1Introduction

Modeling real world problems through mathematical optimization to find optimal solutions has many applications, such as task scheduling[Cheng et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib35);[Pallagani et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib36), vehicle routing[Baty et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib31);[Luo et al\. \(2025\)](https://arxiv.org/html/2610.22889#bib.bib32), and healthcare planning[Tripathi et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib33);[Tang et al\. \(2024a\)](https://arxiv.org/html/2610.22889#bib.bib34)\. Recent works explore using Large Language Models \(LLMs\) to automatically translate natural language inputs into optimization models, making the time\-consuming and costly formulation process more accessible\. In this setting, the natural language input generally consists of two components that summarize the optimization problem: a problem description and its corresponding data\. The problem description defines the objective and constraints, and data provides the concrete parameters required to formulate the problem\. Aparameterdenotes a known attribute specified in the data, while avariabledenotes a decision quantity whose value is determined by solving the optimization problem\.

Current benchmarks designed to evaluate LLMs on optimization modeling generally assume clean and well\-structured inputs\. Specifically, they assume complete and relevant problem descriptions and data are provided\. For example, NL4Opt[Ramamonjison et al\. \(2023\)](https://arxiv.org/html/2610.22889#bib.bib12), Mamo[Huang et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib11), OptiGuide[Li et al\. \(2023a\)](https://arxiv.org/html/2610.22889#bib.bib28)and OPT\-Engine[Chen et al\. \(2026\)](https://arxiv.org/html/2610.22889#bib.bib15)assume a single unified natural language input can be directly converted into an optimization model, and explicitly provide all parameters required for the formulation within the problem description\. NLP4LP[AhmadiTeshnizi et al\. \(2023\)](https://arxiv.org/html/2610.22889#bib.bib1), ComplexOR[Wang et al\. \(2025\)](https://arxiv.org/html/2610.22889#bib.bib18)and ORThought[Yang et al\. \(2026\)](https://arxiv.org/html/2610.22889#bib.bib16)introduce problems with significantly longer natural language descriptions\. Advanced benchmarks like IndustryOR[Tang et al\. \(2024b\)](https://arxiv.org/html/2610.22889#bib.bib2)shift towards problems from different industrial sectors, while OptiVerse[Zhang et al\. \(2026\)](https://arxiv.org/html/2610.22889#bib.bib14)covers complex domains such as stochastic programming\. Optibench[Yang et al\. \(2025b\)](https://arxiv.org/html/2610.22889#bib.bib13)embed linear and nonlinear tabular data in the problem description to measure LLMs’ end\-to\-end problem solving abilities\.

Figure 1:An optimization task in NLData2Opt with its components\.However, despite recent progress, current benchmarks have two fundamental limitations\. Firstly, data is directly embedded in the problem description, and the model’s task is to locate and map them into an optimization formulation\. This diverges from real\-world modeling, in which the construction of the objective and constraints is usually a separate process from data collection, due to the scale and independent storage of the data\. Secondly, it is assumed that every parameter is provided in the data in a ready\-to\-use form\. This is rarely guaranteed with noisy and incomplete real\-world data[Bengio et al\. \(2021\)](https://arxiv.org/html/2610.22889#bib.bib17)\. Since raw data typically contains several pieces of contextually related information, we have to identify throughparameter extractionwhat information is actually necessary to solve the problem[Shi et al\. \(2023\)](https://arxiv.org/html/2610.22889#bib.bib37)\. Analysts must also frequently performparameter derivation, which involves transforming multiple existing data attributes into a single usable value, such as calculating net profit from revenue and expenses\. In cases such as demand forecasting or production scheduling, a predict\-then\-optimize pipeline is commonly adopted, whereparameter predictionestimates unavailable future data from historical trends, and the predicted values are subsequently used in deterministic planning models[Elmachtoub and Grigas \(2022\)](https://arxiv.org/html/2610.22889#bib.bib38);[Mandi et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib39)\. Due to these two limitations, it is difficult to evaluate LLMs’ optimization modeling capabilities in real\-world applications with existing benchmarks\.

To address this, we introduce NLData2Opt, a novel benchmark that incorporates multi\-stage reasoning through controlled and systematic tasks, as illustrated in Figure[1](https://arxiv.org/html/2610.22889#S1.F1)\. Tasks are decomposed into two structurally separate components: a natural language problem description and a collection of CSV\-style data tables\. Noisy and incomplete data is reflected in the need to extract, compute, and predict parameters\. Under the NLData2Opt framework, an LLM needs to fulfill three challenges to complete the optimization formulation: \(1\) extract relevant parameters from data containing misleading attributes, \(2\) compute missing parameters derived by transforming existing data attributes, and \(3\) predict unavailable future values from historical observations\. We comprehensively evaluate state\-of\-the\-art methods and ten open\-weight and proprietary LLMs\. Results show that current LLMs perform competitively in simplified tasks but achieve limited success on NLData2Opt\.

Contributions\.We make three contributions: \(1\) We introduce NLData2Opt, a benchmark that evaluates whether LLMs can perform multi\-stage reasoning for optimization formulation, including model–data separation, information retrieval, implicit derivation, and future\-value prediction\. \(2\) We propose an automatic construction pipeline that transforms NLP4LP and Mamo tasks while preserving the feasible region and optimal objective value\. \(3\) We provide comprehensive evaluations of prompting, agent\-based, and fine\-tuned baselines, which reveal substantial gaps between structural correctness and functional executability\.

## 2Related Work

Modeling an optimization problem into an accurate formulation is often time\-consuming and requires domain expertise\. To enhance this process, recent work has explored automating the formulation process, reducing the time and expertise needed\.

Chain\-of\-thought variants include Tree of Thoughts[Yao et al\. \(2023\)](https://arxiv.org/html/2610.22889#bib.bib19)and Autoformulation[Astorga et al\. \(2025\)](https://arxiv.org/html/2610.22889#bib.bib20), while multi\-agent systems include CoE[Wang et al\. \(2025\)](https://arxiv.org/html/2610.22889#bib.bib18), OptLLM[Xiao et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib29)and EoR[Yang et al\. \(2025a\)](https://arxiv.org/html/2610.22889#bib.bib30)\. Training\-free approaches[Chen et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib8);[Jiao et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib9);[Li et al\. \(2023b\)](https://arxiv.org/html/2610.22889#bib.bib10)provide step\-by\-step instruction and incorporate prior knowledge into prompt design to improve performance\. For learning\-based methods, several existing works explore applying SFT with synthetic data[Tang et al\. \(2024b\)](https://arxiv.org/html/2610.22889#bib.bib2);[Wang et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib5);[Wu et al\. \(2025\)](https://arxiv.org/html/2610.22889#bib.bib6)or adopt reinforcement learning frameworks[Chen et al\. \(2025\)](https://arxiv.org/html/2610.22889#bib.bib4);[Jiang et al\. \(2025b\)](https://arxiv.org/html/2610.22889#bib.bib3), while LLaMoCo[Ma et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib23)incorporates a contrastive learning\-based warm\-up procedure to enhance convergence behavior\.

Techniques to improve formulation quality include highlighting named entities with XML\-like tags[Gangwar and Kani \(2023\)](https://arxiv.org/html/2610.22889#bib.bib22)\. LLMOPT[Jiang et al\. \(2025a\)](https://arxiv.org/html/2610.22889#bib.bib21)constructs a five\-element formulation and reports an11\.08%11\.08\\%average improvement in solving accuracy over SOTA methods\. NED\-Tree[Hu et al\. \(2026\)](https://arxiv.org/html/2610.22889#bib.bib24)and LEAN\-LLM\-OPT[Liang et al\. \(2026\)](https://arxiv.org/html/2610.22889#bib.bib25)extend to handle complex nonlinear and large\-scale industrial formulations\. During inference, format control such as five\-element formulation, LaTeX code, or JSON outputs[Jiang et al\. \(2025a\)](https://arxiv.org/html/2610.22889#bib.bib21);[JU et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib7);[Wang et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib5)assists models to parse the problem into symbolic optimization models\.

## 3NLData2Opt

This section introduces NLData2Opt, a benchmark for evaluating LLMs on formulating optimization problems in realistic data conditions\. We introduce the class of optimization problems considered, then outline the NLData2Opt task structures, and finally provide a validity criterion to ensure ground\-truth solution validity\.

### 3\.1Mixed Integer Linear Program

A mathematical optimization problem aims to find an optimal solution of decision variables that minimizes the objective function while satisfying all constraints\. In this paper, we focus on modeling Linear Program \(LP\) and Mixed Integer Linear Programs \(MILP\), which is one of the most popular optimization problems having a wide variety of industrial applications\. Given a matrix𝑨∈ℝm×n\\bm\{A\}\\in\\mathbb\{R\}^\{m\\times n\},𝒃∈ℝ\+m\\bm\{b\}\\in\\mathbb\{R\}\_\{\+\}^\{m\},𝒄∈ℝn\\bm\{c\}\\in\\mathbb\{R\}^\{n\}, and a subsetI⊆N=\{1,…,n\}I\\subseteq N=\\\{1,\\dots,n\\\}, the corresponding MILP𝒫\\mathcal\{P\}is given by

min𝒄⊤​𝒙subject to𝑨​𝒙≥𝒃,xj≥0,j∈N,xj∈ℝ,j∈N∖I,xj∈ℤ,j∈I,\\begin\{array\}\[\]\{lll\}\\textnormal\{min\}&\\bm\{c\}^\{\\top\}\\bm\{x\}\\\\ \\textnormal\{subject to\}&\\bm\{A\}\\bm\{x\}\\geq\\bm\{b\},\\\\ &x\_\{j\}\\geq 0,&j\\in N,\\\\ &x\_\{j\}\\in\\mathbb\{R\},&j\\in N\\setminus I,\\\\ &x\_\{j\}\\in\\mathbb\{Z\},&j\\in I,\\end\{array\}\(1\)where we call LP in case ofI=∅I=\\emptyset\. Currently, many branch\-and\-cut solvers have been developed for MILP, where input data is written in the standard MPS \(Mathematical Programming System\) or LP formats\. Alternatively, mathematical optimization problems can be specified via several modeling languages such as AMPL[Fourer et al\. \(1990\)](https://arxiv.org/html/2610.22889#bib.bib26)and GAMS[McCarl et al\. \(2012\)](https://arxiv.org/html/2610.22889#bib.bib27)\.

### 3\.2Task Definition in NLData2Opt

Given a natural language problem description𝒫\\mathcal\{P\}and a set of data𝒯\\mathcal\{T\}, an LLM is required to formulate the task in AMPL\. We define𝒯=𝒯r​e​l∪𝒯i​r​r\\mathcal\{T\}=\\mathcal\{T\}\_\{rel\}\\cup\\mathcal\{T\}\_\{irr\}, where𝒯r​e​l\\mathcal\{T\}\_\{rel\}contains all parameters necessary for the optimization formulation, and𝒯i​r​r\\mathcal\{T\}\_\{irr\}denotes the attributes irrelevant to the problem\. For all cases except the base case, we consider both𝒯i​r​r=∅\\mathcal\{T\}\_\{irr\}=\\emptysetand settings where𝒯i​r​r\\mathcal\{T\}\_\{irr\}contains irrelevant attributes\. The objective is to produce an executable formulation that yields a solution to the optimization problem\. We denote the generated AMPL model file and data file byℳ\\mathcal\{M\}and𝒟\\mathcal\{D\}, respectively\.

Each instance consists of the following four standard cases, expanded into nine prompt settings\.

NL Case \(base case\)\.The problem is expressed entirely in natural language, with all numerical parameters embedded in the text\. The LLM directly generates\(ℳ,𝒟\)\(\\mathcal\{M\},\\mathcal\{D\}\)from𝒫\\mathcal\{P\}:

ℱ⁡\(𝒫\)→\(ℳ,𝒟\)\.\\mathcal\{F\}\(\\mathcal\{P\}\)\\rightarrow\(\\mathcal\{M\},\\mathcal\{D\}\)\.
NL\+Data Case\.The problem description𝒫\\mathcal\{P\}and dataset𝒯\\mathcal\{T\}are provided separately\. The LLM generates

ℱ⁡\(𝒫,𝒯\)→\(ℳ,𝒟\)\.\\mathcal\{F\}\(\\mathcal\{P\},\\mathcal\{T\}\)\\rightarrow\(\\mathcal\{M\},\\mathcal\{D\}\)\.
Derivation Case\.In addition to the separation of the problem description and data, eachk∈𝐤k\\in\\mathbf\{k\}in a subset of parameters𝐤⊂𝒯r​e​l\\mathbf\{k\}\\subset\\mathcal\{T\}\_\{rel\}is defined implicitly through intermediate quantitiesQkQ\_\{k\}\. The LLM must compute the parameters via an algebraic functional relationship:

k=ϕ⁡\(Qk\),∀k∈𝐤,k=\\phi\(Q\_\{k\}\),\\quad\\forall k\\in\\mathbf\{k\},before constructing the optimization model\. This transformation preserves the feasible region and the optimal solution while introducing derivation\-oriented reasoning\. The optimization problem is formulated as:

ℱ⁡\(𝒫,\(𝒯∖𝐤\)∪⋃k∈𝐤Qk\)→\(ℳ,𝒟\)\.\\mathcal\{F\}\(\\mathcal\{P\},\(\\mathcal\{T\}\\setminus\\mathbf\{k\}\)\\cup\\bigcup\_\{k\\in\\mathbf\{k\}\}Q\_\{k\}\)\\rightarrow\(\\mathcal\{M\},\\mathcal\{D\}\)\.
Prediction Case\.In addition to the separation of the problem description and data, a subset of parameters𝐭⊂𝒯r​e​l\\mathbf\{t\}\\subset\\mathcal\{T\}\_\{rel\}is masked from the data tables\. Instead, for each missing parametert∈𝐭t\\in\\mathbf\{t\}, a sequence of historical observationsHt=\{h1,…,hn\}H\_\{t\}=\\\{h\_\{1\},\\dots,h\_\{n\}\\\}\(n≥7n\\geq 7\) is provided\. We denote the full set of historical data asℋ\\mathcal\{H\}\. The LLM must first perform a forecasting step to estimate the missing values:

ψ⁡\(ℋ\)→𝐭^,\\psi\(\\mathcal\{H\}\)\\rightarrow\\hat\{\\mathbf\{t\}\},where𝐭^\\hat\{\\mathbf\{t\}\}is the set of predicted values for the masked parameters\. The optimization problem is formulated as:

OPENℱ⁡\(𝒫,\(𝒯∖𝐭\)∪⋃t∈𝐭Ht\)\)→\(ℳ,𝒟\)\.\\mathcal\{F\}\(\\mathcal\{P\},\(\\mathcal\{T\}\\setminus\\mathbf\{t\}\)\\cup\\bigcup\_\{t\\in\\mathbf\{t\}\}H\_\{t\}\)\)\\rightarrow\(\\mathcal\{M\},\\mathcal\{D\}\)\.

### 3\.3Validity Criterion

To maintain the validity of the benchmark, every transformation must satisfy the following criterion\. LetS∗​\(𝒪\)S^\{\*\}\(\\mathcal\{O\}\)denote the optimal objective value of an optimization problem𝒪\\mathcal\{O\}\. Let𝒪orig\\mathcal\{O\}\_\{\\text\{orig\}\}be the original problem and𝒪trans\\mathcal\{O\}\_\{\\text\{trans\}\}its transformed counterpart\. Each transformationΨ∈\{Base,Prediction,Derivation\}\\Psi\\in\\\{\\text\{Base\},\\text\{Prediction\},\\text\{Derivation\}\\\}is required to preserve the original feasible region and optimal solution\. Accordingly, we ensure that

S∗​\(𝒪trans\)=S∗​\(𝒪orig\),S^\{\*\}\(\\mathcal\{O\}\_\{\\text\{trans\}\}\)=S^\{\*\}\(\\mathcal\{O\}\_\{\\text\{orig\}\}\),\(2\)which guarantees that the ground\-truth solution of the optimization problem remains invariant under all transformations\.

## 4Automatic Construction of NLData2Opt

### 4\.1Separation of Problem Description and Data

Each task in NLData2Opt, except for the NL case, is decomposed into two structurally separate components to simulate practical optimization workflows:

Problem description\.A structured natural language description that specifies the objective and constraints, but does not include any concrete numerical values\. This component guides the LLM to construct the AMPL model file\.

Data\.A collection of CSV\-style tables that contain numerical parameters\. Each table includes a name, a brief description and multiple rows of numerical or categorical data\. Provided independently of the problem description, this component guides the LLM to construct the AMPL data file\.

![Refer to caption](https://arxiv.org/html/2610.22889v1/pipeline_pred_der.png)Figure 2:Transformation pipeline for prediction and derivation cases\.Figure 3:AMPL formulation of an example optimization task with model–data separation\.The transformation process for the prediction and derivation cases follows a multi\-stage pipeline, as illustrated in Figure[2](https://arxiv.org/html/2610.22889#S4.F2)\. In the single\-parameter prediction case, we first augment the data with historical observations\. An AMPL Formulator samples from predefined prompt templates to produce controlled temporal patterns that systematically evaluate whether models can predict parameters and integrate them into optimization formulations\. Patterns such as seasonality, abrupt changes, intermittent missing data and outliers are then introduced to simulate richer temporal dynamics\. Finally, the problem description is revised to explicitly specify the parameter to be predicted and the corresponding future time period\.

The multi\-parameter prediction case extends this procedure by applying the same transformation to multiple target parameters, incorporating historical data and prediction requirements for each parameter independently\.

For derivation case, directly transforming existing optimization problems to require intermediate calculations in a single step proved challenging for GPT\-4o\. Instead, we adopt a two\-stage transformation strategy\. The Context Generator first revises the task at the natural language level to introduce intermediate quantities from which the target parameterppcan be computed\. Only after this modification is the revised task separated into a problem description and corresponding data tables\. To further encourage valid derivation\-based optimization formulations, we employ few\-shot prompting during generation\. Detailed prompt templates are provided in Appendix[III](https://arxiv.org/html/2610.22889#A3)\-[IV](https://arxiv.org/html/2610.22889#A4)\.

### 4\.2Generating Gold Solutions in AMPL

To construct gold\-standard solutions for NLData2Opt, we adopt AMPL as the modeling language for mathematical programming\. This choice is motivated by several considerations\. \(1\) AMPL explicitly separates themodel, which details the sets, parameters, decision variables, objectives, and constraints, from thedata, which defines the specific set and parameter values, as shown in Figure[3](https://arxiv.org/html/2610.22889#S4.F3)\. This encourages more compact and readable model definitions, while mirroring practical scenarios where large\-scale data is stored separately\. \(2\) Treating data as a separate component allows modifications without altering the underlying model structure, allowing the same model to be evaluated on different datasets or vice versa[Fourer et al\. \(1990\)](https://arxiv.org/html/2610.22889#bib.bib26)\. \(3\) AMPL is a declarative language based on algebraic representations of objectives and constraints, which is natural and familiar to practitioners in large\-scale mathematical programming\.

![Refer to caption](https://arxiv.org/html/2610.22889v1/pipeline_ampl2.png)Figure 4:Ground\-truth AMPL formulation construction framework\.To generate AMPL formulations, we adopt a CoE framework, as illustrated in Figure[4](https://arxiv.org/html/2610.22889#S4.F4)\. Our implementation builds on prior work[Wang et al\. \(2025\)](https://arxiv.org/html/2610.22889#bib.bib18), with prompts adapted to produce AMPL files\. The system consists of multiple specialized experts, each responsible for a specific subtask\. A Problem Classifier first identifies the optimization problem class \(e\.g\., transshipment or max\-flow\)\. If a standard template is available, it is passed directly to the AMPL Formulator; otherwise, a Model Constructor and a Data Extractor generate the formulation from scratch\. The resulting code is then refined through an iterative Code Runtime and Self\-correction loop\. Detailed prompt templates for each expert are provided in Appendix[I](https://arxiv.org/html/2610.22889#A1)\.

### 4\.3Evaluation Metrics

Since generated formulations may be partially correct, we adopt three complementary metrics to evaluate outputs from different perspectives\.

Entity F1 Score \(EF1\)\.Structural correctness is measured using an entity\-level F1 score\. Each AMPL model is represented as a multisetℰ\\mathcal\{E\}of entity types drawn from

Ω=\{set,parameter,variable,constraint,objective\}\.\\Omega=\\\{\\text\{set\},\\text\{parameter\},\\text\{variable\},\\text\{constraint\},\\text\{objective\}\\\}\.
Letℰcand\\mathcal\{E\}\_\{\\text\{cand\}\}andℰgt\\mathcal\{E\}\_\{\\text\{gt\}\}denote the multisets extracted from the candidate and ground\-truth models, respectively\. The number of true positives is computed asTP=∑ω∈Ωmin⁡\(mult⁡\(ℰcand,ω\),mult⁡\(ℰgt,ω\)\)\\mathrm\{TP\}=\\sum\_\{\\omega\\in\\Omega\}\\min\\bigl\(\\mathrm\{mult\}\(\\mathcal\{E\}\_\{\\text\{cand\}\},\\omega\),\\mathrm\{mult\}\(\\mathcal\{E\}\_\{\\text\{gt\}\},\\omega\)\\bigr\), wheremult⁡\(ℰ,ω\):=\|\{e∈ℰ∣e=ω\}\|\\mathrm\{mult\}\(\\mathcal\{E\},\\omega\):=\|\\\{e\\in\\mathcal\{E\}\\mid e=\\omega\\\}\|denotes the multiplicity of entity typeω\\omegainℰ\\mathcal\{E\}\. Precision and recall are defined asP=TP\|ℰcand\|\\mathrm\{P\}=\\frac\{\\mathrm\{TP\}\}\{\|\\mathcal\{E\}\_\{\\text\{cand\}\}\|\}andR=TP\|ℰgt\|\\mathrm\{R\}=\\frac\{\\mathrm\{TP\}\}\{\|\\mathcal\{E\}\_\{\\text\{gt\}\}\|\}, and the F1 score becomesF1=2​P​RP\+R\\mathrm\{F1\}=\\frac\{2\\mathrm\{P\}\\mathrm\{R\}\}\{\\mathrm\{P\}\+\\mathrm\{R\}\}\. This captures structural correctness independent of variable or constraint naming\.

Execution\-Based Correctness \(EBC\)\.Functional correctness is evaluated by executing each model–data pair using the AMPL interpreter with the CBC solver\. If execution fails, the score is set to00\. Otherwise, a smoothed correctness score is computed:

Correctness=\{max⁡\(0,1−\|O−AA\|\)if​A≠0,1if​O=A=0,0otherwise,\\mathrm\{Correctness\}=\\begin\{cases\}\\max\\\!\\left\(0,1\-\\left\|\\frac\{O\-A\}\{A\}\\right\|\\right\)&\\text\{if \}A\\neq 0,\\\\ 1&\\hskip\-22\.76219pt\\text\{if \}O=A=0,\\\\ 0&\\hskip\-4\.2679pt\\text\{otherwise\},\\end\{cases\}\(3\)whereOOdenotes the objective value from the candidate formulation andAAis the ground\-truth value\.

LLM\-Based Evaluation Score \(LLM\-ES\)\.We employ GPT\-4o\-mini to perform a holistic evaluation of candidate formulations\. The evaluator compares the constraints specified in the AMPL model and the parameter values specified in the data against the corresponding components of the gold\-standard formulation, assigning a score on a00\-1010scale\. The full evaluation prompt is provided in Appendix[V](https://arxiv.org/html/2610.22889#A5)\.

### 4\.4Quality Control

We rigorously validate each generated AMPL formulation through a combination of execution\-based and semantic checks\. Every formulation is executed to detect runtime or modeling errors, and the resulting objective value is compared against the ground\-truth solution\. In addition, we employ LLM\-based reviewers to perform holistic verification of formulation soundness\. When validating different cases of natural language problem descriptions and data pairs, these reviewers ensure that all transformation rules are satisfied and that each transformed instance preserves the original feasible region and optimal solution\.

![Refer to caption](https://arxiv.org/html/2610.22889v1/heatmap.png)Figure 5:Average EBC scores across task cases\.The heatmap illustrates functional correctness achieved on the nine task cases of NLData2Opt \(rows, cases with irrelevant attributes are marked with "\+ Irr"\), when evaluated with an LLM model or baseline method \(columns\)\. The columns are ranked to reflect the worst to best performing models\. Higher saturation represents a higher EBC score\. While models with smaller capacity did not generate executable code, stronger models only reached scores as high as 0\.77, highlighting the challenges of the tasks\.

## 5Experiments

We develop NLData2Opt and comprehensively evaluate SOTA methods to assess their ability to formulate and solve realistic optimization problems from natural language and data\. The observed performance decline highlights the challenges posed by NLData2Opt and indicates that current methods are not yet capable of fully addressing such tasks\.

### 5\.1Experimental Setup

Benchmarks\.We construct NLData2Opt from two existing benchmarks: NLP4LP[AhmadiTeshnizi et al\. \(2023\)](https://arxiv.org/html/2610.22889#bib.bib1)and Mamo[Huang et al\. \(2024\)](https://arxiv.org/html/2610.22889#bib.bib11)\. NLP4LP consists of 269 LP and MILP tasks collected from optimization textbooks and lecture notes\. Mamo is divided into two subsets: Easy LP, which contains 647 high school\-level MILP problems, and Complex LP, which includes 210 undergraduate\-level tasks integrating both LP and MILP formulations\.

Baselines\.We evaluate NLData2Opt using a diverse set of methods\. Proprietary LLMs evaluated under direct prompting include GPT\-4o\-mini \(gpt\-4o\-mini\-2024\-07\-18\), GPT\-4o \(gpt\-4o\-2024\-11\-20\), Gemini 2\.5 Pro, Claude Haiku 4\.5 \(claude\-haiku\-4\-5\-20251001\), Claude Sonnet 4\.6 and Claude Opus 4\.6\. Open\-weight models include Llama\-3\.1\-8B\-Instruct, Ministral\-3\-8B\-2512, Llama\-3\.3\-70B\-Instruct and Mistral\-Large\-2411\. We use a temperature of0\.10\.1and limit each model to a single attempt per task\. Generated formulations are normalized to AMPL\.mod/\.datformat using the reformatting prompt in Appendix[V](https://arxiv.org/html/2610.22889#A5)and executed using the AMPL interpreter with the CBC solver\. Non\-executable outputs are assigned zero EBC\.

In addition, we evaluate several structured baselines: the five\-element formulation approach proposed by LLMOPT using GPT\-4o\-mini, CoE with its default backbone GPT\-3\.5\-turbo, and a SFT baseline trained on Qwen3\-1\.7B\.

### 5\.2Results and Analysis

Figure[5](https://arxiv.org/html/2610.22889#S4.F5)reports EBC across nine prompt settings: NL, NL\+Data, derivation, two prediction cases, and four cases with irrelevant attributes\. Comprehensive evaluation results for all metrics \(EF1, EBC and LLM\-ES\) are summarized in Tables[7](https://arxiv.org/html/2610.22889#A8.T7)\-[15](https://arxiv.org/html/2610.22889#A8.T15)in Appendix[VIII](https://arxiv.org/html/2610.22889#A8), averaged over a15%15\\%subset per dataset using a random seed of4242\. Additionally, we report full results in Tables[16](https://arxiv.org/html/2610.22889#A8.T16)\-[18](https://arxiv.org/html/2610.22889#A8.T18)for three representative models: claude\-opus\-4\.6, gpt\-4o\-mini, and llama\-3\.3\-70b\-instruct\. Overall, current LLMs and other baseline methods achieve limited success on NLData2Opt, with subset\-based evaluation showing trends consistent with the full benchmark\.

Impact of model–data separation\.Transitioning from NL \(Table[7](https://arxiv.org/html/2610.22889#A8.T7)\) to NL\+Data cases \(Table[8](https://arxiv.org/html/2610.22889#A8.T8)\), LLMs struggle with mapping declared variables in the model to instantiated values in the data\. Consequently, we observe a general decrease in EBC, even when EF1 remains relatively stable\. However, we note that all proprietary models achieve higher EBC on Mamo Complex for NL\+Data case\. As Mamo Complex contains more complicated tasks, extracting data into separate tables might have provided more clarity and structure\.

Qualitative analysis shows that LLMs tend to embed coefficients directly into constraints in the NL case\. In contrast, in the NL\+Data case, LLMs use a more elegant approach by defining the parameters in the model file and recording the corresponding values in the data file\. While both methods yield the same solution, parameterizing is more desirable as it allows for higher scalability and generalizability\.

Impact of irrelevant attributes\.Our experiments illustrate that the performance drop is not simply due to noise\. Rather, when multiple conceptually related pieces of information are present, models struggle to identify the correct data\. In our initial experiments, adding irrelevant attributes did not significantly affect results\. However, after introducing more semantically similar attributes, we observe performance declines across all three metrics\. Error analysis \(Figure[6](https://arxiv.org/html/2610.22889#A7.F6)\) further confirms this, revealing that irrelevant attributes increase undefined entities errors, including a rise from36\.3%36\.3\\%to47\.6%47\.6\\%for the derivation case\. This confirms that the presence of semantically similar yet irrelevant data hinders the model’s ability to maintain a mapping between declared variables and instantiated data\.

Derivation case analysis\.The derivation cases \(Table[10](https://arxiv.org/html/2610.22889#A8.T10)\) receive drops in EBC across baselines\. Table[4](https://arxiv.org/html/2610.22889#A7.T4)reports the percentage of correctly identified models, revealing that correct constraint structures are relatively low\. This indicates that models struggle to maintain structural consistency when parameters are not explicitly provided\.

Qualitative analysis further supports that models often struggle during optimization formulation that requires parameter derivation\. When a total quantity must be derived from multiple components, models may fail to aggregate, generating multiple false constraints instead\. Moreover, even when derivation is performed correctly, models introduce irrelevant constraints that unnecessarily restrict the feasible region\. This suggests that models struggle to integrate intermediate derived results into correct downstream optimization constraints\.

Prediction case analysis\.Performance drops in both cases, though one\-variable prediction \(Table[12](https://arxiv.org/html/2610.22889#A8.T12)\) does not show significant EBC improvement over multiple\-variable cases \(Table[14](https://arxiv.org/html/2610.22889#A8.T14)\)\. Table[5](https://arxiv.org/html/2610.22889#A7.T5)shows that even if an exact match is not achieved, predictions often lie in the ground truth value’s vicinity\. Simple forecasting and oracle\-prediction baselines \(Tables[1](https://arxiv.org/html/2610.22889#A6.T1)\-[3](https://arxiv.org/html/2610.22889#A6.T3)\) confirm that forecasting errors alone cannot explain the observed performance gap caused by formulation errors\.

Qualitative analysis shows that models occasionally predict incorrect values, such as directly copying from the second\-to\-last time period\. Others correctly predict the required parameters but fail to incorporate them into the formulation\. Across both derivation and prediction cases, common AMPL execution errors include undefined entities and undeclared data assignments\. Therefore, even when arithmetic or forecasting is correct, integrating intermediate results into downstream constraints remains a key challenge in multi\-step tasks\.

### 5\.3Human Alignment

Although human evaluation remains the gold standard, evaluating NLData2Opt presents scalability challenges\. Each task requires the evaluation of nine distinct cases, and assessing AMPL formulations requires domain expertise\. Consequently, comprehensive human evaluation on the entire benchmark across baselines is far too expensive\.

To provide a scalable and cost\-effective alternative, we adopt an LLM\-as\-a\-judge framework using the GPT\-4o\-mini\-based LLM\-ES metric\. To verify this approach, we manually evaluated 108 candidate formulations generated by Claude Opus 4\.6\. The instructions used were identical to the LLM judge’s prompt \(Appendix[V](https://arxiv.org/html/2610.22889#A5)\)\. The evaluation yields a Spearman’s rank correlation coefficient ofρ=0\.39\\rho=0\.39\(p=3\.90×10−6p=3\.90\\times 10^\{\-6\}\) and a Pearson linear correlation coefficient ofr=0\.30r=0\.30\(p=3\.90×10−4p=3\.90\\times 10^\{\-4\}\)\. These results indicate a positive alignment with human judgments, consistent with prior work on LLM\-based evaluators[Bavaresco et al\. \(2025\)](https://arxiv.org/html/2610.22889#bib.bib41);[Schroeder and Wood\-Doughty \(2024\)](https://arxiv.org/html/2610.22889#bib.bib42)\. Results suggest that evaluating specialized programming is more challenging than evaluating open\-ended natural language generation\. While LLM\-ES provides a scalable signal, its moderate correlation with human judgments suggests it should be interpreted with caution and used alongside other execution\-based metrics\.

## 6Conclusion

We introduced NLData2Opt, a benchmark designed to evaluate optimization modeling under realistic data conditions\. NLData2Opt reflects practical workflows by separating data from problem descriptions and incorporating irrelevant yet misleading attributes\. In addition, it requires the derivation of missing parameters and prediction of future values that are common in applied optimization but largely absent from prior benchmarks\.

Through comprehensive evaluations, we demonstrate that current LLMs and baseline methods achieve only limited success on NLData2Opt\. These findings underscore the need for more robust approaches to optimization problem formulation\.

## Limitations

NLData2Opt utilizes LLM\-based transformations for benchmark construction, which may contain hallucinated information, temporal inconsistencies, or other logical distortions\. They may also introduce synthetic parameters that differ systematically from real\-world data distributions\. Common mitigation strategies include human evaluation, comparison between real and synthetic data, error analysis, and execution\-based validation\. In NLData2Opt, we reduce these risks through execution\-based checks of generated optimization formulations and LLM\-based validation of transformed instances against predefined consistency criteria\. Nevertheless, automated validation cannot guarantee that all semantic or logical errors are detected, and certain distortions may remain in the benchmark\.

Because NLData2Opt is derived from existing datasets such as NLP4LP and Mamo, some source instances may have appeared in the pretraining data of evaluated models\. Our transformations substantially modify the original tasks, but potential data contamination cannot be fully ruled out\.

Our conclusions on industrial realism and general optimization are limited to the LP/MILP task families, data perturbations, and AMPL\-based formulation setting studied in this work, and should not be interpreted as directly representing the full complexity of industrial optimization problems\.

Finally, while NLData2Opt requires LLMs to predict or derive parameters before generating formal optimization formulations, this workflow may emphasize formalization ability over deeper conceptual reasoning\. For example, an LLM may correctly construct an AMPL formulation using a predicted parameter without fully understanding how that parameter affects the feasible region or optimal solution\. Future benchmarks could evaluate conceptual understanding and decision\-making independently from formulation ability, enabling more nuanced assessments of mathematical reasoning\.

## Ethical considerations

NLData2Opt is constructed by adapting open\-source datasets NLP4LP and Mamo\. Both datasets are licensed under CC BY NC 4\.0\. We respect their creators’ ownership and intellectual property, and our use of these artifacts is strictly consistent with their intended purpose of evaluating the mathematical modeling capabilities of LLMs\.

NLData2Opt is a benchmarking dataset designed to evaluate LLMs on optimization tasks\. The tasks presented focus solely on mathematical optimization modeling and do not contain personal or sensitive data\. We plan to release NLData2Opt under the same CC BY NC 4\.0 license, allowing only non\-commercial use for research purposes\.

AI assistants were utilized to support prompt template development and improve the clarity of this paper\. We maintain full responsibility for the content, experimental design, and conclusions presented\.

## Acknowledgments

This work was supported by yJST CREST Grant Number JPMJCR21D1\.

## References

- AhmadiTeshniziet al\.\(2023\)A\. AhmadiTeshnizi, W\. Gao, and M\. UdellOptiMUS: optimization modeling using MIP solvers and large language models\.CoRRabs/2310\.06116\.External Links:[Link](https://doi.org/10.48550/arXiv.2310.06116),[Document](https://dx.doi.org/10.48550/ARXIV.2310.06116),2310\.06116Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p2.1),[§5\.1](https://arxiv.org/html/2610.22889#S5.SS1.p1.1)\.
- Astorgaet al\.\(2025\)N\. Astorga, T\. Liu, Y\. Xiao, and M\. van der SchaarAutoformulation of mathematical optimization models using llms\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,External Links:[Link](https://openreview.net/forum?id=33YrT1j0O0)Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- Batyet al\.\(2024\)L\. Baty, K\. Jungel, P\. S\. Klein, A\. Parmentier, and M\. SchifferCombinatorial optimization\-enriched machine learning to solve the dynamic vehicle routing problem with time windows\.Transportation Science58\(4\),pp\. 708–725\.External Links:ISSN 1526\-5447,[Link](https://doi.org/10.1287/trsc.2023.0107),[Document](https://dx.doi.org/10.1287/trsc.2023.0107)Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p1.1)\.
- Bavarescoet al\.\(2025\)A\. Bavaresco, R\. Bernardi, L\. Bertolazzi, D\. Elliott, R\. Fernández, A\. Gatt, E\. Ghaleb, M\. Giulianelli, M\. Hanna, A\. Koller, A\. F\. T\. Martins, P\. Mondorf, V\. Neplenbroek, S\. Pezzelle, B\. Plank, D\. Schlangen, A\. Suglia, A\. K\. Surikuchi, E\. Takmaz, and A\. TestoniLLMs instead of human judges? A large scale empirical study across 20 NLP evaluation tasks\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\), ACL 2025, Vienna, Austria, July 27 \- August 1, 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),pp\. 238–255\.External Links:[Link](https://doi.org/10.18653/v1/2025.acl-short.20),[Document](https://dx.doi.org/10.18653/V1/2025.ACL-SHORT.20)Cited by:[§5\.3](https://arxiv.org/html/2610.22889#S5.SS3.p2.1)\.
- Bengioet al\.\(2021\)Y\. Bengio, A\. Lodi, and A\. ProuvostMachine learning for combinatorial optimization: a methodological tour d’horizon\.European Journal of Operational Research290\(2\),pp\. 405–421\.External Links:ISSN 0377\-2217,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.ejor.2020.07.063),[Link](https://www.sciencedirect.com/science/article/pii/S0377221720306895)Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p3.1)\.
- Chenet al\.\(2024\)H\. Chen, G\. E\. Constante\-Flores, and C\. LiDiagnosing infeasible optimization problems using large language models\.INFOR Inf\. Syst\. Oper\. Res\.62\(4\),pp\. 573–587\.External Links:[Link](https://doi.org/10.1080/03155986.2024.2385189),[Document](https://dx.doi.org/10.1080/03155986.2024.2385189)Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- Chenet al\.\(2026\)Y\. Chen, C\. Cheng, Y\. Sun, Z\. Ling, and D\. GeOPT\-engine: benchmarking the limits of llms in optimization modeling via complexity scaling\.CoRRabs/2601\.19924\.External Links:[Link](https://doi.org/10.48550/arXiv.2601.19924),[Document](https://dx.doi.org/10.48550/ARXIV.2601.19924),2601\.19924Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p2.1)\.
- Chenet al\.\(2025\)Y\. Chen, J\. Xia, S\. Shao, D\. Ge, and Y\. YeSolver\-informed RL: grounding large language models for authentic optimization modeling\.CoRRabs/2505\.11792\.External Links:[Link](https://doi.org/10.48550/arXiv.2505.11792),[Document](https://dx.doi.org/10.48550/ARXIV.2505.11792),2505\.11792Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- Chenget al\.\(2024\)Y\. Cheng, Z\. Cao, X\. Zhang, Q\. Cao, and D\. ZhangMulti objective dynamic task scheduling optimization algorithm based on deep reinforcement learning\.J\. Supercomput\.80\(5\),pp\. 6917–6945\.External Links:[Link](https://doi.org/10.1007/s11227-023-05714-1),[Document](https://dx.doi.org/10.1007/S11227-023-05714-1)Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p1.1)\.
- Choiet al\.\(2026\)J\. Choi, S\. Park, C\. Cho, H\. Park, and B\. KimDiagnosing the reliability of llm\-as\-a\-judge via item response theory\.CoRRabs/2602\.00521\.External Links:[Link](https://doi.org/10.48550/arXiv.2602.00521),[Document](https://dx.doi.org/10.48550/ARXIV.2602.00521),2602\.00521Cited by:[Appendix VIII](https://arxiv.org/html/2610.22889#A8.p1.1)\.
- Elmachtoub and Grigas \(2022\)A\. N\. Elmachtoub and P\. GrigasSmart "predict, then optimize"\.Manag\. Sci\.68\(1\),pp\. 9–26\.External Links:[Link](https://doi.org/10.1287/mnsc.2020.3922),[Document](https://dx.doi.org/10.1287/MNSC.2020.3922)Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p3.1)\.
- Foureret al\.\(1990\)R\. Fourer, D\. M\. Gay, and B\. W\. KernighanAMPL: a mathematical programming language\.Management Science36\(5\),pp\. 519–554\.Cited by:[§3\.1](https://arxiv.org/html/2610.22889#S3.SS1.p1.2),[§4\.2](https://arxiv.org/html/2610.22889#S4.SS2.p1.1)\.
- Gangwar and Kani \(2023\)N\. Gangwar and N\. KaniHighlighting named entities in input for auto\-formulation of optimization problems\.InIntelligent Computer Mathematics \- 16th International Conference, CICM 2023, Cambridge, UK, September 5\-8, 2023, Proceedings,C\. Dubois and M\. Kerber \(Eds\.\),Lecture Notes in Computer Science, Vol\.14101,pp\. 130–141\.External Links:[Link](https://doi.org/10.1007/978-3-031-42753-4/_9),[Document](https://dx.doi.org/10.1007/978-3-031-42753-4%5F9)Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p3.1)\.
- Huet al\.\(2026\)Z\. Hu, Y\. Deng, H\. Liu, and C\. FanNED\-tree: bridging the semantic gap with nonlinear element decomposition tree for LLM nonlinear optimization modeling\.CoRRabs/2604\.01588\.External Links:[Link](https://doi.org/10.48550/arXiv.2604.01588),[Document](https://dx.doi.org/10.48550/ARXIV.2604.01588),2604\.01588Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p3.1)\.
- Huanget al\.\(2024\)X\. Huang, Q\. Shen, Y\. Hu, A\. Gao, and B\. WangMamo: a mathematical modeling benchmark with solvers\.CoRR\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2405.13144)Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p2.1),[§5\.1](https://arxiv.org/html/2610.22889#S5.SS1.p1.1)\.
- Jianget al\.\(2025a\)C\. Jiang, X\. Shu, H\. Qian, X\. Lu, J\. Zhou, A\. Zhou, and Y\. YuLLMOPT: learning to define and solve general optimization problems from scratch\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=9OMvtboTJg)Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p3.1)\.
- Jianget al\.\(2025b\)X\. Jiang, Y\. Wu, M\. Li, Z\. Cao, and Y\. ZhangLarge language models as end\-to\-end combinatorial optimization solvers\.CoRRabs/2509\.16865\.External Links:[Link](https://doi.org/10.48550/arXiv.2509.16865),[Document](https://dx.doi.org/10.48550/ARXIV.2509.16865),2509\.16865Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- Jiaoet al\.\(2024\)Z\. Jiao, M\. Sha, H\. Zhang, X\. Jiang, and W\. QiCity\-leo: toward transparent city management using LLM with end\-to\-end optimization\.CoRRabs/2406\.10958\.External Links:[Link](https://doi.org/10.48550/arXiv.2406.10958),[Document](https://dx.doi.org/10.48550/ARXIV.2406.10958),2406\.10958Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- JUet al\.\(2024\)D\. JU, S\. Jiang, A\. Cohen, A\. Foss, S\. Mitts, A\. Zharmagambetov, B\. Amos, X\. Li, J\. T\. Kao, M\. Fazel\-Zarandi, and Y\. TianTo the globe \(TTG\): towards language\-driven guaranteed travel planning\.CoRRabs/2410\.16456\.External Links:[Link](https://doi.org/10.48550/arXiv.2410.16456),[Document](https://dx.doi.org/10.48550/ARXIV.2410.16456),2410\.16456Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p3.1)\.
- Liet al\.\(2023a\)B\. Li, K\. Mellou, B\. Zhang, J\. Pathuri, and I\. MenacheLarge language models for supply chain optimization\.CoRRabs/2307\.03875\.External Links:[Link](https://doi.org/10.48550/arXiv.2307.03875),[Document](https://dx.doi.org/10.48550/ARXIV.2307.03875),2307\.03875Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p2.1)\.
- Liet al\.\(2023b\)R\. Li, C\. Pu, J\. Tao, C\. Li, F\. Fan, Y\. Xiang, and S\. ChenLLM\-based frameworks for power engineering from routine to novel tasks\.External Links:2305\.11202,[Link](https://arxiv.org/abs/2305.11202)Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- Lianget al\.\(2026\)K\. Liang, Y\. Lu, J\. Mao, S\. Sun, C\. Yang, C\. Zeng, X\. Jin, H\. Qin, R\. Zhu, and C\. TeoLarge\-scale optimization model auto\-formulation: harnessing LLM flexibility via structured workflow\.CoRRabs/2601\.09635\.External Links:[Link](https://doi.org/10.48550/arXiv.2601.09635),[Document](https://dx.doi.org/10.48550/ARXIV.2601.09635),2601\.09635Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p3.1)\.
- Luoet al\.\(2025\)F\. Luo, X\. Lin, Y\. Wu, Z\. Wang, T\. Xialiang, M\. Yuan, and Q\. ZhangBoosting neural combinatorial optimization for large\-scale vehicle routing problems\.InInternational Conference on Representation Learning,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 14754–14780\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/264f2e10479c9370972847e96107db7f-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p1.1)\.
- Maet al\.\(2024\)Z\. Ma, H\. Guo, J\. Chen, G\. Peng, Z\. Cao, Y\. Ma, and Y\. GongLLaMoCo: instruction tuning of large language models for optimization code generation\.CoRRabs/2403\.01131\.External Links:[Link](https://doi.org/10.48550/arXiv.2403.01131),[Document](https://dx.doi.org/10.48550/ARXIV.2403.01131),2403\.01131Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- Mandiet al\.\(2024\)J\. Mandi, J\. Kotary, S\. Berden, M\. Mulamba, V\. Bucarey, T\. Guns, and F\. FiorettoDecision\-focused learning: foundations, state of the art, benchmark and future opportunities\.J\. Artif\. Intell\. Res\.80,pp\. 1623–1701\.External Links:[Link](https://doi.org/10.1613/jair.1.15320),[Document](https://dx.doi.org/10.1613/JAIR.1.15320)Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p3.1)\.
- McCarlet al\.\(2012\)B\. McCarl, A\. Meeraus, P\. van der Eijk, M\. Bussieck, S\. Dirkse, and F\. NelissenMcCarl expanded gams user guide version 23\.6\.GAMS Development, Washington, DC\.Cited by:[§3\.1](https://arxiv.org/html/2610.22889#S3.SS1.p1.2)\.
- Pallaganiet al\.\(2024\)V\. Pallagani, B\. C\. Muppasani, K\. Roy, F\. Fabiano, A\. Loreggia, K\. Murugesan, B\. Srivastava, F\. Rossi, L\. Horesh, and A\. P\. ShethOn the prospects of incorporating large language models \(llms\) in automated planning and scheduling \(APS\)\.InProceedings of the Thirty\-Fourth International Conference on Automated Planning and Scheduling, ICAPS 2024, Banff, Alberta, Canada, June 1\-6, 2024,S\. Bernardini and C\. Muise \(Eds\.\),pp\. 432–444\.External Links:[Link](https://doi.org/10.1609/icaps.v34i1.31503),[Document](https://dx.doi.org/10.1609/ICAPS.V34I1.31503)Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p1.1)\.
- Ramamonjisonet al\.\(2023\)R\. Ramamonjison, T\. T\. L\. Yu, R\. Li, H\. Li, G\. Carenini, B\. Ghaddar, S\. He, M\. Mostajabdaveh, A\. Banitalebi\-Dehkordi, Z\. Zhou, and Y\. ZhangNL4Opt competition: formulating optimization problems based on their natural language descriptions\.CoRRabs/2303\.08233\.External Links:[Link](https://doi.org/10.48550/arXiv.2303.08233),[Document](https://dx.doi.org/10.48550/ARXIV.2303.08233),2303\.08233Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p2.1)\.
- Schroeder and Wood\-Doughty \(2024\)K\. Schroeder and Z\. Wood\-DoughtyCan you trust LLM judgments? reliability of llm\-as\-a\-judge\.CoRRabs/2412\.12509\.External Links:[Link](https://doi.org/10.48550/arXiv.2412.12509),[Document](https://dx.doi.org/10.48550/ARXIV.2412.12509),2412\.12509Cited by:[§5\.3](https://arxiv.org/html/2610.22889#S5.SS3.p2.1)\.
- Shiet al\.\(2023\)F\. Shi, X\. Chen, K\. Misra, N\. Scales, D\. Dohan, E\. H\. Chi, N\. Schärli, and D\. ZhouLarge language models can be easily distracted by irrelevant context\.InInternational Conference on Machine Learning, ICML 2023, 23\-29 July 2023, Honolulu, Hawaii, USA,A\. Krause, E\. Brunskill, K\. Cho, B\. Engelhardt, S\. Sabato, and J\. Scarlett \(Eds\.\),Proceedings of Machine Learning Research,pp\. 31210–31227\.External Links:[Link](https://proceedings.mlr.press/v202/shi23a.html)Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p3.1)\.
- Tanget al\.\(2024a\)P\. M\. Tang, K\. K\. H\. Leong, N\. Shaik, and H\. C\. LauAutomated conversion of static to dynamic scheduler via natural language\.CoRRabs/2405\.06697\.External Links:[Link](https://doi.org/10.48550/arXiv.2405.06697),[Document](https://dx.doi.org/10.48550/ARXIV.2405.06697),2405\.06697Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p1.1)\.
- Tanget al\.\(2024b\)Z\. Tang, C\. Huang, X\. Zheng, S\. Hu, Z\. Wang, D\. Ge, and B\. WangORLM: training large language models for optimization modeling\.CoRRabs/2405\.17743\.External Links:[Link](https://doi.org/10.48550/arXiv.2405.17743),[Document](https://dx.doi.org/10.48550/ARXIV.2405.17743),2405\.17743Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p2.1),[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- Tripathiet al\.\(2024\)S\. Tripathi, R\. Sukumaran, and T\. S\. CookEfficient healthcare with large language models: optimizing clinical workflow and enhancing patient care\.Journal of the American Medical Informatics Association31\(6\),pp\. 1436–1440\.External Links:ISSN 1527\-974X,[Document](https://dx.doi.org/10.1093/jamia/ocad258),[Link](https://doi.org/10.1093/jamia/ocad258),https://academic\.oup\.com/jamia/article\-pdf/31/6/1436/57768962/ocad258\.pdfCited by:[§1](https://arxiv.org/html/2610.22889#S1.p1.1)\.
- Wanget al\.\(2024\)T\. Wang, W\. Yu, R\. She, W\. Yang, T\. Chen, and J\. ZhangLeveraging large language models for solving rare MIP challenges\.CoRRabs/2409\.04464\.External Links:[Link](https://doi.org/10.48550/arXiv.2409.04464),[Document](https://dx.doi.org/10.48550/ARXIV.2409.04464),2409\.04464Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1),[§2](https://arxiv.org/html/2610.22889#S2.p3.1)\.
- Wanget al\.\(2025\)Z\. Wang, R\. Pan, J\. Yao, R\. Csordas, L\. Li, L\. Yin, J\. Wu, T\. Zhang, M\. Li, and S\. LiuChain\-of\-experts: unlocking the communication power of mixture\-of\-experts models\.CoRRabs/2506\.18945\.External Links:[Link](https://doi.org/10.48550/arXiv.2506.18945),[Document](https://dx.doi.org/10.48550/ARXIV.2506.18945),2506\.18945Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p2.1),[§2](https://arxiv.org/html/2610.22889#S2.p2.1),[§4\.2](https://arxiv.org/html/2610.22889#S4.SS2.p2.1)\.
- Wuet al\.\(2025\)Y\. Wu, Y\. Zhang, Y\. Wu, Y\. Wang, J\. Zhang, and J\. ChengEvo\-step: evolutionary generation and stepwise validation for optimizing llms in or\.External Links:[Link](https://openreview.net/forum?id=aapUBU9U0D)Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- Xiaoet al\.\(2024\)Z\. Xiao, D\. Zhang, X\. Han, X\. Fu, W\. Y\. Yu, T\. Zhong, S\. Wu, Y\. Wang, J\. Yin, and G\. ChenEnhancing LLM reasoning via vision\-augmented prompting\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 \- 15, 2024,A\. Globersons, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. M\. Tomczak, and C\. Zhang \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2024/hash/328c922d068dd4ccb23cec5c64e6c7fc-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- Yanget al\.\(2026\)B\. Yang, Q\. Zhou, J\. Li, C\. Su, P\. Angeloudis, and S\. HuORThought: benchmarking and automating logistics optimization modeling via structured llm reasoning\.Artificial Intelligence for Transportation6,pp\. 100059\.External Links:ISSN 3050\-8606,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.ait.2026.100059),[Link](https://www.sciencedirect.com/science/article/pii/S3050860626000141)Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p2.1)\.
- Yanget al\.\(2025a\)Z\. Yang, Y\. Wang, Y\. Huang, Z\. Guo, W\. Shi, X\. Han, L\. Feng, L\. Song, X\. Liang, and J\. TangOptiBench meets resocratic: measure and improve llms for optimization modeling\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=fsDZwS49uY)Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- Yanget al\.\(2025b\)Z\. Yang, Y\. Wang, Y\. Huang, Z\. Guo, W\. Shi, X\. Han, L\. Feng, L\. Song, X\. Liang, and J\. TangOptiBench meets resocratic: measure and improve llms for optimization modeling\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,External Links:[Link](https://openreview.net/forum?id=fsDZwS49uY)Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p2.1)\.
- Yaoet al\.\(2023\)S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. Griffiths, Y\. Cao, and K\. NarasimhanTree of thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2023/hash/271db9922b8d1f4dd7aaef84ed5ac703-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2610.22889#S2.p2.1)\.
- Zhanget al\.\(2026\)X\. Zhang, B\. Zhang, Y\. Wan, L\. Zhang, Y\. Yao, B\. Wei, Y\. Wu, and J\. LiuOptiVerse: A comprehensive benchmark towards optimization problem solving\.CoRRabs/2604\.21510\.External Links:[Link](https://doi.org/10.48550/arXiv.2604.21510),[Document](https://dx.doi.org/10.48550/ARXIV.2604.21510),2604\.21510Cited by:[§1](https://arxiv.org/html/2610.22889#S1.p2.1)\.

## Appendix

We provide the details of the exact templates utilized in the automated\-generation of NLData2Opt that were omitted in the main text\. GPT\-4o\-mini \(gpt\-4o\-mini\-2024\-07\-18\) and GPT\-4o \(gpt\-4o\-2024\-11\-20\) were used with a temperature of0\.10\.1\. While GPT\-4o\-mini primarily constructed NL\+Data and prediction cases for NLP4LP, GPT\-4o was employed for more complex tasks, including derivative cases within NLP4LP and task instances that were challenging for GPT\-4o\-mini to process effectively\.

## Appendix ITemplates for AMPL formulation construction

To construct high\-quality and executable gold\-standard solutions for NLData2Opt, we utilize a CoE framework adapted to produce AMPL code\. This pipeline decomposes the complex modeling task into subtasks handled by respective experts\. The initial formulator proposed an initial AMPL formulation, and consists of the following experts: Problem Classifier: Identifies the precise mathematical model and objective \(e\.g\., transshipment or max\-flow\) while ensuring all numerical inputs are treated as generic parameters\. It also provides mathematical formulations for linking constraints in discrete decision problems\. If a standard template is available, it is passed directly to the AMPL Formulator\. AMPL Formulator: Synthesizes the provided components into an executable\.modfile, adhering to the blueprint and ensuring no numerical data values are included in the model logic\. Model Constructor: Collaborates with the Data Extractor to generate the formulation from scratch if a standard template is unavailable\. Data Extractor: Extracts data from the base case and synthesizes the final components into AMPL\.modand\.datfiles\.

After executing the generated AMPL formulation, the following experts revise the initial formulation through an iterative loop: Debugger: Identifies the source of numerical errors, determining if the discrepancy between predicted and expected values arises from model logic or data errors\. Code Reviewer: Conducts a detailed analysis of the previous AMPL formulation, identifying whether the discrepancy arises from model logic or data errors, and providing a corrected proposed fix\. Code Modifier: Implements the proposed modifications by the Code Reviewer to ensure the final AMPL code accurately models the optimization problem\.

Listing 1:Prompt template for Problem ClassifierAstheOptimizationModelerandKnowledgeRetriever,yourprimaryroleistocreateacomplete,mathematicallypreciseblueprintforthemodel,ensuringallnecessaryconstraintsareincluded\.YouroutputMUSTbegeneric,usingparametersforallnumericalinputs\(costs,demands,capacities\)\.

\#\#\#1\.ModelType&Objective:

Identifytheprecisemathematicalmodel\(e\.g\.,TSP,TransshipmentProblem,Min\-CostFlow\)andtheobjective\(Minimize/Maximize\)\.

\#\#\#2\.VariablesandSets:

DefineallnecessarySets\(e\.g\.,CITIES,I,J\)andVariables\(e\.g\.,binaryx\[i,j\],continuousflow\[i,j\],auxiliaryu\[i\]\)\.

\#\#\#3\.BinaryLinkingConstraintsCheck:

Foranyprobleminvolvingdiscretedecisions\(TSP,Fixed\-Charge,etc\.\),youMUSTprovidetheprecisemathematicalformulationforthe\*\*LINKINGCONSTRAINTS\*\*necessarytoconnectthebinaryvariablestothecontinuousvariables\.

\*ExampleforFixed\-Charge:Definetheconstraintensuringflowiszeroifafacilityisclosed\(usinggenericparameterslike’s\[j\]’and’y\[j\]’\)\.

\*ExampleforTSP:DefinetheMiller\-Tucker\-Zemlin\(MTZ\)SubtourEliminationConstraint,includingallrequiredindexexclusions\(e\.g\.,i\!=1andj\!=1\)andusing’N’or’card\(CITIES\)’forthescalingfactor\.

ProblemDescription:\{q\}

Outputonlythecomplete,mathematicallypreciselistofsets,variables,objective,andconstraints,strictlyadheringtotheinstructionsabove,andusinggenericparameternamesonly\.Donotincludeanyspecificnumericaldata\.

"""

Listing 2:Prompt template for Model FormulatorYouareahighlyspecializedAMPLCoder\.Yoursolefunctionistosynthesizetheprovidedcomponentsintoacomplete,executableAMPLmodel\(\.mod\)file\.

\-\-\-

\#\#\#1\.ModelBlueprint\(FromPeer\):

Youmuststrictlyadheretothemodeltype,parameters,andconstraintslistedhere:

\{constraint\_output\}

\#\#\#2\.SynthesisInstructions:

1\.DefineallsetsandparametersusinggenericAMPLsyntax\(e\.g\.,’paramcost\{\{SETS\}\};’\)\.

2\.Youmustnotincludeanydataassignmentoperators\(:=\)ordatavaluesinthe\.modfile\.Alldatabelongsinthe\.datfile\.

3\.Defineallvariables,Objectivefunction,andConstraints,integratingthecomplexlinkingconstraintsprovidedintheblueprint\.

ReturnonlytheAMPLmodelcodeforthe\.modfile\.Donotincludeexplanationsorcommentsanywhere\.

Listing 3:Prompt template for Data ExtractorYouarethefinal,highlyspecializedAMPLCoder\.Yoursolefunctionistosynthesizetheprovidedcomponentsintoacomplete,executableAMPLmodel\(\.mod\)anddata\(\.dat\)file\.

Payattentiontotheunitsrequiredinthefinaloutput\.

Theoptimizationproblemisthefollowing:

\{q\}

Themodelfileisalreadyattemptedbyyourpeer:

\{mod\}

Yourjobistocreatethemodelanddatafilesfortheoptimizationproblem\.

ReturnboththeAMPLmodelanddatacode\.Donotincludeexplanationsorcommentsanywhere\.

‘‘‘

1\.mod:

\[\.modfilecontenthere\]

2\.dat:

\[\.datfilecontenthere\]

Listing 4:Prompt template for DebuggerAsaspecializedOptimizationCodeReviewer,yourresponsibilityistoidentifythesourceofthenumericalerrorintheprovidedAMPLmodel\.

Hereistheoptimizationproblem:

‘‘‘

\{q\}

‘‘‘

HereisthepreviousAMPLcode:

‘‘‘

\{ampl\}

‘‘‘

ReturninafewshortlineswhytheAMPLcodegavethewronganswerof\{predicted\},whenthecorrectanswershouldbe\{expected\}\.

Proposeafixbriefly\.

Listing 5:Prompt template for Code ReviewerAsaspecializedOptimizationCodeReviewer,yourresponsibilityistoidentifythesourceofthenumericalerrorintheprovidedAMPLmodel\.

YouranalysismustdetermineifthediscrepancybetweenthePREDICTEDvalue\(\{predicted\}\)andtheEXPECTEDvalue\(\{expected\}\)isduetoamodelingerrororadataerror\.

Now,hereistheproblem:

‘‘‘

\{q\}

‘‘‘

HereistheAMPLcodeforreview:

‘‘‘

\{ampl\}

‘‘‘

Yourresponseshouldfollowthisformat:

1\.DiagnosisofErrorSource

\[StateclearlywhethertheerrorisintheModelLogic\(\.mod\)ortheData\(\.dat\)\.Ifthecodeiscorrect,statethattheexpectedvalueislikelyflawedduetosourcedata\.\]

2\.CodeReview:ModelLogic\(\.mod\)

\[Reviewstructure,variabledefinitions,andconstraints\.CheckforMTZexclusions,capacitylinks,andcorrectObjectivefunction\.\]

3\.ProposedFix

\[Iftheerrorisinthemodel,providethecorrectedAMPLcodeforthe\.modfile\.Iftheerrorisinthedata,specifywhichparameter\(e\.g\.,uniterror,transcriptionerror\)\.\]

Ifboththemodelandthedatafileiscorrect,andtheerrorliesintheoptimizationproblem,output1inthefinalline\.

Listing 6:Prompt template for Code ModifierAsaspecializedOptimizationCodeReviewer,yourresponsibilityistoimprovethecurrentAMPLmodel\.

Hereistheoptimizationproblem:

‘‘‘

\{q\}

‘‘‘

HereisthepreviousAMPLcode:

‘‘‘

\{ampl\}

‘‘‘

Hereisthecommentandproposedfixbyyourpeer:

‘‘‘

\{comment\}

‘‘‘

ModifytheAMPLcodesuchthatitmodelsthequestionaccurately,basedonthecommentandproposedfixbyyourpeer\.

ReturnonlythenewAMPLcode\.Donotincludeexplanationsorcommentsanywhere\.

Usetheexactformatbelow:

‘‘‘

1\.mod:

\[\.modfilecontenthere\]

2\.dat:

\[\.datfilecontenthere\]

‘‘‘

## Appendix IITemplates for model–data separation

To systematically separate the natural language description from its associated data, we employ a pipeline consisting of a NL\-Data Separator and a Verifier\. Model\-Data Separator: Transforms a given optimization problem into two distinct components: a problem statement in natural language and a dataset in CSV format\. The problem statement presents a structured task description and specifies the optimization objective without including any numerical values\. All quantitative information is captured exclusively in the dataset, which contains only the values explicitly provided in the original problem\. Verifier: Ensures the integrity of the model–data separation by checking that all numerical values required to solve the optimization problem are present in the data file\.

Listing 7:Prompt template for NL\-Data SeparatorThefollowingisanoptimizationproblem\.Pleaseconstruct:

1\.Aproblemstatementinnaturallanguage,beginningwith"Youare\.\.\."andfollowingastructuredtaskdescription\.Itshouldclearlyexplainwhattheobjectiveis,butmustNOTcontainanyspecificnumericalvalues\.Noconcretequantities,figures,orunitsofmeasurementshouldbementioned\.

2\.AdatasetcomposedofmultipleCSV\-styletablesthatdefineaspecificparameters\.

\-Allvariablesreferencedintheoptimizationproblemmustberepresentedinthetables\.

\-Alltablecellsshouldbenumeric,donotuseplaceholders\.

\-Onlyincludevaluesexplicitlymentionedintheproblem\.

Theoptimizationproblemisasfollows:

\{ques\}

Pleaseconstructaproblemspecificationandtabulardatasetaccordingtotheaboverequirements,andprovidetheresultsthefollowingformat:

‘‘‘

1\.Problemstatement

\[WritetheproblemspecificationinplainEnglish\.\]

2\.Tables

table1\_name\.csv\-Description

Column1,Column2,\.\.\.

value1,value2,\.\.\.

\.\.\.

table2\_name\.csv\-Description

Column1,Column2,\.\.\.

value1,value2,\.\.\.

\.\.\.

‘‘‘

Listing 8:Prompt template for VerifierYouareaskedtocompareanoptimizationproblemanditscorrespondingtabulardatasets\.

Theoptimizationproblemisasfollows:

\{ques\}

Thetabulardatasetscorrespondingareasfollows:

\{dat\}

Pleasecheckifthetabulardatasetscontainallvaluesneededtosolvetheoptimizationproblem\.

Ifthisissatisfied,return1\.Ifnot,return0\.Return1or0only\.

## Appendix IIITemplates for prediction case construction

To construct prediction cases for NLData2Opt, we transform NL\+Data cases into tasks that require LLMs to predict future parameter values from historical data\. To construct tasks that require different numbers of parameters to be predicted, we implement a pipeline that consists of the following components: AMPL Formulator: Modifies the data of a task from NL\+Data case to create prediction cases\. It handles single or multiple targets, and supports increasing and stable trends\. Historical data are generated for each target, with the final time period containing the true value\. After passing the Verifier, this value is subsequently blanked out in the historical data table using automated scripts\. Based on these patterns, seasonality, abrupt changes, intermittent missing data, and outliers are introduced to simulate richer temporal dynamics\. Verifier: Inspects the new optimization problem if all data in the NL\+Data case are included\. In addition, historical data are provided for a certain number of parameters, with the final time period containing the value to be predicted\. Two separate verifiers are used for single variable and multi\-variable prediction, respectively\. Description Writer: Generates a natural language problem description from the tabular datasets\. In addition to clearly stating the overall optimization objective, it explicitly specifies which variables must be predicted at which exact time period\.

Listing 9:Prompt template for AMPL Formulator \(increasing case, single variable\)Thefollowingareaproblemdescriptionanditsdatatables:

‘‘‘

\{base\}

‘‘‘

Modifythetabulardatasetsfollowingthebelowrequirements:

\-Chooseonevalueinthetabulardatasetsoftheoptimizationproblemasthepredictiontarget\.

\-Providehistoricaldataofatleast8timeperiodsthatshowarealistictrendwithsmallrandomnoise\.

\-Thehistoricaldatashowanincreasingtrendthatapproachesthetruetargetvaluenaturally,withoutovershootingit\.

\-Thehistoricaltrendmuststopimmediatelyatthetruetargetvalue\.

\-Thetimeperiodshouldhaveitsowncolumn,withrealisticnamesandvalues\.

\-Donotremoveorrenameanyexistingcolumnsorrowsintheoriginaltables\.However,thesingletruetargetvaluemustbevisiblyblankedoutexactlywhereitoriginallyappeared\.

Providetheresultsthefollowingformat,donotaddcomments:

‘‘‘

1\.Problemstatement

\[Theoriginalproblemstatement\]

2\.Tables

table1\_name\.csv\-Description

Column1,Column2,\.\.\.

value1,value2,\.\.\.

\.\.\.

table2\_name\.csv\-Description

Column1,Column2,\.\.\.

value1,value2,\.\.\.

\.\.\.

‘‘‘

Listing 10:Prompt template for AMPL Formulator \(stable case, multiple variables\)Thefollowingareaproblemdescriptionanditsdatatables:

‘‘‘

\{base\}

‘‘‘

Modifythetabulardatasetsfollowingthebelowrequirements:

\-Thegivenproblemdescriptionandtablesalreadycontainonevaluethathasbeenblankedoutasanexistingpredictiontarget\.Keepthathistoricaldatatableunchanged\.

\-Selectoneadditionalvalueinthetabulardatasetstoserveasasecondnewpredictiontarget\.Blankitoutexactlywhereitoriginallyappeared,withoutremovingorrenaminganycolumnsorrows\.

\-Forthenewpredictiontarget,provideatleast8timeperiodsofhistoricaldatainaseparatetable\.

\-Thehistoricalseriesforthenewtargetmustshownaturalsmallrandomfluctuationsaroundthepredictiontargetwhilemaintainingintegerconstraintswhennecessary\.

\-Thehistoricaldatashouldhoverbothslightlyaboveandbelowthenewpredictiontarget,centeringroughlyonit\.

\-Thefinalrowforallhistoricaldatashouldhavethetargetvalueexactly,donotleaveitblank\.

\-Thetimeperiodshouldhaveitsowncolumn,withrealisticnamesandvalues\.

\-Donotremoveorrenameanyexistingdataasidefromblankingoutthetwotruetargetvalues\.

Thefinalsentenceintheproblemdescriptionshouldspecifywhichtwoparametersneedtobepredictedatwhichtimeperiod\.Phraseitas"youneedtopredict\.\.\."\.

Providetheresultsthefollowingformat,donotaddcomments:

‘‘‘

1\.Problemstatement

\[Rewritetheoriginalproblemstatement\]

2\.Tables

table1\_name\.csv\-Description

Column1,Column2,\.\.\.

value1,value2,\.\.\.

\.\.\.

table2\_name\.csv\-Description

Column1,Column2,\.\.\.

value1,value2,\.\.\.

\.\.\.

‘‘‘

Listing 11:Prompt template for Verifier \(single variable\)Youareaskedtoverifyifthegiventabulardatasetssatisfycertainrules\.

Optimizationproblem:

\{ques\}

Tabulardatasets:

‘‘‘

\{dat\}

‘‘‘

Checkifthedatatablessatisfythefollowingrules:

1\.Historicaldataisonlyprovidedfor1variableinthedatatables\.

2\.Allvariablesmentionedintheoptimizationproblemappearinthemaindatatablesexceptforonemissingvalue\.

3\.Ahistoricaltableprovidesthecorrectvalueforonevariable,withthevalueinthefinaltimeperiodappearinginthequestion\.

Output"1"ifallrulesaresatisfied,otherwiseoutput"0"\.Donotprovideanyextratext\.

Listing 12:Prompt template for Verifier \(multiple variable\)Youareaskedtoverifyifthegiventabulardatasetssatisfycertainrules\.

Optimizationproblem:

\{ques\}

Tabulardatasets:

‘‘‘

\{dat\}

‘‘‘

Checkifthedatatablessatisfythefollowingrules:

1\.Historicaldataendingatthesamenumericalvalueastheoptimizationproblemareprovidedforatleast2variables\.Ignoretextualdifferenceswhencomparingnumericalvalues\.

2\.Allvariablesmentionedintheoptimizationproblemappearinthemaindatatablesexceptforthoseinthehistoricaltables\.

Output"1"ifallrulesaresatisfied,otherwiseoutput"0"\.Donotprovideanyextratext\.

Listing 13:Prompt template for Description WriterThefollowingisanoptimizationproblem,withitscorrespondingtabulardatasets\.Youareaskedtowriteaproblemdescription\.

Optimizationproblem:

\{ques\}

Tabulardatasets:

\{dat\}

Pleasewriteaproblemdescriptioninnaturallanguage,beginningwith"Youare\.\.\."andfollowingastructuredtaskdescription\.

Donotincludeanyspecificnumericalvalues\.Noconcretequantities,figures,orunitsofmeasurementshouldbementioned\.

Itshouldclearlyexplainwhattheoverallobjectiveoftheoptimizationproblemis\.

Specifywhichparametersneedstobepredictedatwhichtimeperiod\.Phraseitas"youneedtopredict\.\.\."\.Explicitlymentionthe\*\*exacttime\*\*tobepredictedasitappearsoftheprovidedtables\.Donotparaphraseitas"thefinalrow"or"thenexttime"\.

Provideonlytheproblemstatement,donotaddcomments\.

## Appendix IVTemplates for derivation case construction

To generate derivation cases for NLData2Opt, we transform straightforward optimization problems into new problems that require intermediate calculations\. Because directly transforming existing optimization problems to require intermediate calculations in a single step proved challenging for LLMs, we adopt a two\-stage transformation strategy\. The components of the pipeline are as follows: Context Generator: Produces a rewritten optimization problem where exactly one numerical quantity is expressed via meaningful intermediate quantities, without changing the total value or feasible region\. Example transformations are provided to encourage the LLM to generate valid decompositions\. Example transformations are provided to guide the LLM toward valid decompositions\. Two context generators are used sequentially, with context generator22employed after context generator11to process questions that were not successfully transformed according to the Verifier\. Verifier: Inspects the new optimization problem if it maintains the same feasible region and optimal solution as the problem before the transformation\. In addition, it makes sure the transformation successfully introduced intermediate quantities to require derivation\-based manipulation by making sure one numerical quantity is decomposed into possible components\. NL\-Data Separator: Separates the model from its associated data for the new optimization problem\. Compared to the standard NL\+Data Separator used for the NL\+Data case, this version specifies to only include the intermediate quantities when both aggregated parameters and intermediate quantities are provided, ensuring that derivation\-based reasoning is required while modeling the optimization problem\.

Listing 14:Prompt template for Context Generator 1Rewritethegivenoptimizationproblemsothatexactlyonenumericalvariableisoptionallyexpressedintermsofmeaningfulintermediatecomponents,withoutchangingthetotalvalueorthefeasibleregionoftheproblem\.Allotherpartsoftheproblemmustremainunchanged\.Thedecompositionshouldreflectderivation\-orientedreasoning,suchascomputingprofitfromrevenueandcosts,orbreakingdowntotalproductiontimeintoseparatetasks\.Returnonlytherewrittenproblem\.

\-\-\-

Theoptimizationproblemisasfollows:

Aschoolneedstoprovidelunchforatleast300students\.

Rewrite:

Aschoolneedstoprovidelunchforatleast150boysand150girls\.

\-\-\-

Theoptimizationproblemisasfollows:

Itmakesaprofitof$3perregularsandwichandaprofitof$4perspecialsandwich\.

Rewrite:

Aregularsandwichcosts$1tomakeandhasasellingpriceof$4\.Aspecialsandwichcosts$3tomakeandhasasellingpriceof$7\.

\-\-\-

Theoptimizationproblemisasfollows:

\{ques\}

Rewrite:

Listing 15:Prompt template for Context Generator 2YouareanOptimizationProblemDecomposer\.Yourtaskistotakeastraightforwardoptimizationproblemandrewriteitintoanew,moredetailednarrative\.

Youmustfollowtheserules:

1\.You\*must\*takethesimplenumbers\(constants\)intheproblem\(e\.g\.,10,6,150\)andbreakthemdownarithmeticallyintonontrivialnewcomponents\.

\*FAILURE:Atrivialsplitlike’100’\-\>’50\+50’isa\*\*completefailure\*\*\.

\*GOODEXAMPLE:Agoodsplitaddsnewconcepts:’4’\-\>’a1\-unitprepfeeand3unitsofapplicationtime’\.

2\.Youmustrewrite\*only\*thepartsofthesentencethatcontainthenumbersyouaredecomposing\.\*\*Donot\*\*addextra,unnecessarysentencesorrephrasepartsoftheproblemthatarenotbeingchanged\.

3\.Simplyrephrasingtheprobleminnewwordsortranslatingitintoitsalgebraicform\(like"minimize10x\+6y"\)isa\*\*completefailure\*\*\.Therewrite\*must\*containthearithmeticdecompositionfromRule1\.

4\.Thefinal,rewrittenproblemmustbe\*mathematicallyidentical\*totheoriginal\.Theobjectivefunctionandallconstraintsmustsimplifytothe\*exactsameequations\*\.

5\.Therewrittenproblemshouldpreservethenarrativestyle,sentencestructure,andcontextaroundthenumbers\.Itshouldreadlikeanaturalproblemstatement,notaformulaorexplanation\.

\-\-\-

\*\*Example1:ObjectiveFunction\*\*

Theoptimizationproblemisasfollows:

Acompanyearnsaprofitof$5perproduct\.

Rewrite:

Eachproductsellsfor$15andcosts$10toproduce\.

\-\-\-

\*\*Example2:SimpleConstraint\*\*

Theoptimizationproblemisasfollows:

Afactoryproduces800unitsperday\.

Rewrite:

Thefactoryoperatesfor10hoursaday\.Itspends2hourssettingupmachinesandthenproduces100unitsperhourfortheremaining8hours\.

\-\-\-

\*\*Example3:ConstraintwithMultipliers\*\*

Theoptimizationproblemisasfollows:

Fourtimestheman\-hoursspentontaskXplusthosespentontaskYshouldnotexceed16

Rewrite:

EachunitoftaskXcontributes4man\-hours,consistingof2hoursofplanningand2hoursofexecution;eachunitoftaskYcontributes1man\-hourofwork\.Then,summingthesecontributions,thetotalman\-hours4X\+Ycannotexceed16\.

\-\-\-

\*\*TASK\*\*

Theoptimizationproblemisasfollows:

\{ques\}

Rewrite:

Listing 16:Prompt template for VerifierComparethetwooptimizationproblemsbelowanddetermineiftherewrittenonecorrectlyfollowsthetransformationrules:

1\.Therewrittenproblemmusthavethesamefeasibleregionandoptimalsolutionastheoriginalproblemafterthetransformation\.

2\.Atleastonenumericalquantityfromtheoriginalproblemissplitintomeaningfulcomponents\.

Originalproblem:

\{ques\}

Rewrittenproblem:

\{new\}

Return1ifbothrulesaresatisfied,else0\.Onlyreturn1or0\.

If0,inthenextparagraph,changetherewrittenproblemtofulfilltherules\.

Listing 17:Prompt template for NL\-Data SeparatorThefollowingisanoptimizationproblem\.Pleaseconstruct:

1\.Aproblemstatementinnaturallanguage

\-Beginwith"Youare\.\.\."andfollowingastructuredtaskdescription\.

\-Includeonlythoseconstraintsthatarenotalreadyevidentornumericallyencodedinthedatasettables\.

\-Donotincludeanynumbers,units,orplaceholdersanywhereinthetext\.

2\.AdatasetcomposedofmultipleCSV\-styletablesthatdefineaspecificparameters\.

\-Allvariablesreferencedintheoptimizationproblemmustberepresentedinthetables\.

\-Alltablecellsshouldbenumeric,donotuseplaceholders\.

\-Onlyincludevaluesexplicitlymentionedintheproblem\.

\-Whenbothaggregatedparametersandintemediateparametersareprovided,onlyincludetheintermediateparameters\.

Theoptimizationproblemisasfollows:

\{ques\}

Pleaseconstructaproblemspecificationandtabulardatasetaccordingtotheaboverequirements,andprovidetheresultsthefollowingformat:

‘‘‘

1\.Problemstatement

\[WritetheproblemspecificationinplainEnglish\.\]

2\.Tables

table1\_name\.csv\-Description

Column1,Column2,\.\.\.

value1,value2,\.\.\.

\.\.\.

table2\_name\.csv\-Description

Column1,Column2,\.\.\.

value1,value2,\.\.\.

\.\.\.

‘‘‘

## Appendix VTemplates for LLM evaluation

In this section, instructions templates to assess the accuracy of AMPL formulation are introduced\. Given that LLM outputs may contain conversational headers or omit the target\.mod/\.datmarkers, we utilize the template detailed in Listing[18](https://arxiv.org/html/2610.22889#LST18)to extract the AMPL formulation and ensure the files are in a compatible format with the CBC solver\.

To perform a holistic evaluation, in addition to the EF1 and EBC, we employ LLM\-based evaluation with the template provided in Listing[19](https://arxiv.org/html/2610.22889#LST19)\. The rating of00\-1010is then normalized to a score between00and1010, which is consistent with the other two metrics\.

Listing 18:Prompt template for reformatting generated messy formulationYouaregivenanoutput\.ExtracttheAMPLformulationifpossible\.Forthedatafile,leave\#emptyifempty,butdonotomitthe"2\.dat"marker\.

Output:

\{AMPL\_Formulation\}

Returnstrictlyinthisformat:

‘‘‘1\.mod:

<AMPLmodel\>

2\.dat:

<AMPLdata\>‘‘‘

IfthetextdoesnotcontaintangibleAMPLformulation,return:

‘‘‘1\.mod:

\#empty

2\.dat:

\#empty‘‘‘

Listing 19:Prompt template for evaluating overall correctnessDoesthis\.modand\.datpairrepresentthesameoptimizationproblemastheground\-truth\.modand\.dat?Ratethissolutionfrom0to10:

\.modfile:

\{mod1\_content\}

groundtruth\.modfile:

\{mod2\_content\}

\.datfile:

\{dat1\_content\}

groundtruth\.datfile:

\{dat2\_content\}

Specifically,evaluate:

\-Whethertheconstraintsinthecandidate\.modfileareequivalenttothoseintheground\-truth\.modfile,

\-Whethertheparametervaluesinthecandidate\.datfileexactlymatchthoseintheground\-truth\.datfile\.

Returnanexplanation\.Thefinallineshouldbetherating\.

## Appendix VIControlled Prediction Baselines

We implement two additional controlled baselines to separate forecasting difficulty from formulation difficulty, focusing on three representative models: claude\-opus\-4\.6 \(Table[1](https://arxiv.org/html/2610.22889#A6.T1)\), gpt\-4o\-mini \(Table[2](https://arxiv.org/html/2610.22889#A6.T2)\), and llama\-3\.3\-70b\-instruct \(Table[3](https://arxiv.org/html/2610.22889#A6.T3)\)\. In the simple forecasting baseline, the LLM\-generated predictions are replaced with estimates from a linear trend extrapolation over historical series data, while the remaining formulation process remains unchanged\. In the oracle\-prediction baseline, the predicted values are replaced with the ground\-truth future values, thereby removing forecasting error and allowing us to examine errors arising from incorporating these values into the final optimization formulation\.

We observe that replacing the LLM\-generated predictions with the simple forecasting baseline generally reduces EBC\. However, using oracle\-prediction values yields only small improvements over the original LLM baseline\. This suggests that the forecasting capability of modern LLMs is already sufficient for many tasks and that forecasting errors alone cannot explain the observed performance gap\.

The oracle\-prediction baseline generally improves EBC and LLM\-ES only slightly\. For example, EBC of Llama 3\.3 70B Instruct on the One\-Variable Prediction case increases from 0\.1824 to just 0\.1847 under oracle prediction\. This indicates that formulation errors remain persistent even when forecasting difficulty is isolated\. In particular, lower\-capacity models continue to struggle with incorporating predicted values into the optimization instance and constructing the corresponding formulation correctly\.

Table 1:Performance breakdown for Claude Opus 4\.6 across prediction baselines \(Simple Forecasting, Oracle Prediction, and LLM\)\.Table 2:Performance breakdown for GPT\-4o Mini across prediction baselines \(Simple Forecasting, Oracle Prediction, and LLM\)\.Table 3:Performance breakdown for Meta LLaMA 3\.3 70B Instruct across prediction baselines \(Simple Forecasting, Oracle Prediction, and LLM\)\.
## Appendix VIIFurther Analysis

Table[4](https://arxiv.org/html/2610.22889#A7.T4)summarizes the percentage of tasks where the optimization model is correctly identified for the derivation case\. Here, a correct model is when the constraint relation types \(≤\\leq,≥\\geq,==\) and objective function type \(maximize or minimize\) exactly match the ground truth formulation, regardless of the values of the parameters\.

Table 4:Percentage of correctly identified optimization models on derivation case\.Table[5](https://arxiv.org/html/2610.22889#A7.T5)analyzes the performance of the prediction step prior to optimization\. It reports percentage of tasks where future prediction exactly matches the ground truth value \(EC\) and relative correctness \(RC\)\. We find that most methods obtain over30%30\\%on the EC metric and over60%60\\%on the RC metric\.

Table 5:Percentage of Exact Matches \(EM\) and continuous Smoothed Correctness \(SC\) across four prediction cases\.![Refer to caption](https://arxiv.org/html/2610.22889v1/ampl_error2.png)Figure 6:Distribution of AMPL execution errors across nine task cases\.The breakdown highlights the relative frequency of specific compilation and logical errors, such as undefined entities, undeclared data assignments, and syntax errors\.Table 6:Failure type breakdown across all task cases\.To further investigate the failure modes of the generated optimization models, we categorize AMPL execution errors encountered across all models and datasets\. Figure[6](https://arxiv.org/html/2610.22889#A7.F6)visualizes the proportional distribution of the error types across the nine distinct task cases, while Table[6](https://arxiv.org/html/2610.22889#A7.T6)reorganizes failures into tool\-specific syntax errors and structural errors\.

Across task cases, maintaining structural consistency between the model and the data is demonstrated to be a significant challenge for LLMs\. We identify three primary types of errors that are prevalent across tasks:

- •Undefined Entities:A structural error where the model \(\.mod\) attempts to use a set, parameter, or variable in a constraint or objective function that has not been formally declared or initialized\.
- •Undeclared Data Assignments:A structural error where the data file \(\.dat\) attempts to assign values for parameters or sets that do not exist in the corresponding\.modfile\.
- •Syntax Errors:Fundamental formatting violations, such as missing semicolons or incorrect assignment operators, prevent the AMPL compiler from parsing the scripts\. This includes dimension mismatches such as improper set index, reflecting a failure of mathematical indexing as well as grammar\.

Beyond compile\-time failures, semantic modeling errors cannot be caught during compilation but manifest as discrepancies in the final objective value\. As shown in Table[6](https://arxiv.org/html/2610.22889#A7.T6), the exact objective match rate is consistently lower than the compilation success rate, which indicates that many generated formulations are executable but still contain semantic modeling errors that lead to incorrect solutions\.

## Appendix VIIIComprehensive Evaluation Results

The experimental results reported in Tables[7](https://arxiv.org/html/2610.22889#A8.T7)\-[15](https://arxiv.org/html/2610.22889#A8.T15)are based on a randomly sampled15%15\\%subset of the full benchmark, selected using a random seed of4242\. The results are categorized across all cases, datasets, and methods to provide a comprehensive overview of baseline performance\. We report the three evaluation metrics: entity F1 score \(EF1\), execution\-based correctness \(EBC\) and LLM\-based evaluation score \(LLM\-ES\)\. To assess the variance and stability of LLM\-ES, we measure self\-consistency by evaluating each candidate formulation with prompt variations\. Specifically, we apply character\-level errors in high\-attention words, insert sentence\-level line breaks, and paraphrase certain verbs and adjectives[Choi et al\. \(2026\)](https://arxiv.org/html/2610.22889#bib.bib40)\. We report the mean score and standard deviation across four evaluation runs with different prompt variations\.

To assess whether the subset is representative of the full benchmark, we evaluated claude\-opus\-4\.6 \(Table[16](https://arxiv.org/html/2610.22889#A8.T16)\), gpt\-4o\-mini \(Table[17](https://arxiv.org/html/2610.22889#A8.T17)\), and llama\-3\.3\-70b\-instruct \(Table[18](https://arxiv.org/html/2610.22889#A8.T18)\) on the complete benchmark\. The full\-benchmark results exhibit trends consistent with those reported on the15%15\\%subset\. In particular, the relative ranking of models is preserved, with Claude Opus 4\.6 outperforming GPT\-4o Mini and Llama 3\.3 70B Instruct\. Similarly, the comparative difficulty of different task settings are preserved: task cases involving derivation, prediction, and irrelevant attributes remain more challenging than the base NL and NL\+Data cases\. These observations suggest that the subset\-based evaluation is representative of overall model performance\.

NLP4LPMamo EasyMamo ComplexModel/MethodEF1EBCLLM\-ESEF1EBCLLM\-ESEF1EBCLLM\-ESGPT\-4o Mini0\.5820\.8180\.568±0\.0560\.568\\pm 0\.0560\.7290\.8670\.481±0\.0550\.481\\pm 0\.0550\.8090\.1300\.548±0\.0610\.548\\pm 0\.061GPT\-4o0\.9680\.1500\.667±0\.0470\.667\\pm 0\.0470\.6870\.7670\.383±0\.0500\.383\\pm 0\.0500\.8700\.4680\.670±0\.0340\.670\\pm 0\.034Gemini 2\.5 Pro0\.4130\.0000\.282±0\.0560\.282\\pm 0\.0560\.3700\.0000\.203±0\.0590\.203\\pm 0\.0590\.3010\.0000\.390±0\.0470\.390\\pm 0\.047Claude Haiku 4\.50\.5880\.7020\.463±0\.0510\.463\\pm 0\.0510\.8250\.8450\.638±0\.060\\bm\{0\.638\\pm 0\.060\}0\.8650\.6190\.625±0\.0430\.625\\pm 0\.043Claude Sonnet 4\.60\.8800\.4500\.403±0\.0390\.403\\pm 0\.0390\.8500\.6000\.610±0\.0460\.610\\pm 0\.0460\.9050\.6160\.596±0\.0420\.596\\pm 0\.042Claude Opus 4\.60\.8980\.6500\.413±0\.0370\.413\\pm 0\.0370\.8900\.7500\.610±0\.0390\.610\\pm 0\.0390\.8720\.7410\.611±0\.0420\.611\\pm 0\.042LLaMA 3\.1 8B Inst\.0\.7760\.0000\.538±0\.0510\.538\\pm 0\.0510\.6020\.0000\.324±0\.0520\.324\\pm 0\.0520\.6900\.0000\.542±0\.0490\.542\\pm 0\.049Ministral 8B0\.8980\.0000\.626±0\.0510\.626\\pm 0\.0510\.6450\.0000\.356±0\.0530\.356\\pm 0\.0530\.8240\.0000\.601±0\.0290\.601\\pm 0\.029LLaMA 3\.3 70B Inst\.0\.7960\.0750\.493±0\.0530\.493\\pm 0\.0530\.6510\.9060\.367±0\.0560\.367\\pm 0\.0560\.7940\.3080\.609±0\.0390\.609\\pm 0\.039Mistral Large0\.9220\.0750\.588±0\.0610\.588\\pm 0\.0610\.6540\.1000\.347±0\.0410\.347\\pm 0\.0410\.8670\.4170\.650±0\.0410\.650\\pm 0\.041Five\-element0\.8720\.1930\.738±0\.0130\.738\\pm 0\.0130\.6910\.6250\.132±0\.0180\.132\\pm 0\.0180\.8270\.3490\.700±0\.024\\bm\{0\.700\\pm 0\.024\}CoE0\.5810\.6020\.747±0\.035\\bm\{0\.747\\pm 0\.035\}0\.6720\.7330\.132±0\.0070\.132\\pm 0\.0070\.5770\.0910\.249±0\.0350\.249\\pm 0\.035SFT0\.9020\.5070\.732±0\.0220\.732\\pm 0\.0220\.7260\.7480\.183±0\.0140\.183\\pm 0\.0140\.8170\.0520\.367±0\.0220\.367\\pm 0\.022Table 7:Comprehensive metrics for performance on NL case\.NLP4LPMamo EasyMamo ComplexModel/MethodEF1EBCLLM\-ESEF1EBCLLM\-ESEF1EBCLLM\-ESGPT\-4o Mini0\.9310\.4000\.421±0\.0490\.421\\pm 0\.0490\.7000\.3450\.419±0\.0490\.419\\pm 0\.0490\.8760\.3930\.529±0\.0490\.529\\pm 0\.049GPT\-4o0\.9400\.1500\.645±0\.0490\.645\\pm 0\.0490\.6700\.6390\.361±0\.0560\.361\\pm 0\.0560\.8870\.6120\.658±0\.049\\bm\{0\.658\\pm 0\.049\}Gemini 2\.5 Pro0\.4270\.0000\.317±0\.0460\.317\\pm 0\.0460\.2390\.0000\.147±0\.0440\.147\\pm 0\.0440\.3270\.0000\.396±0\.0610\.396\\pm 0\.061Claude Haiku 4\.50\.8960\.7330\.644±0\.0450\.644\\pm 0\.0450\.7340\.7530\.543±0\.059\\bm\{0\.543\\pm 0\.059\}0\.8720\.6300\.625±0\.0420\.625\\pm 0\.042Claude Sonnet 4\.60\.9450\.2900\.515±0\.0500\.515\\pm 0\.0500\.6700\.5530\.495±0\.0620\.495\\pm 0\.0620\.8790\.7940\.596±0\.0420\.596\\pm 0\.042Claude Opus 4\.60\.9380\.6990\.598±0\.0510\.598\\pm 0\.0510\.6790\.7620\.513±0\.0610\.513\\pm 0\.0610\.8850\.8530\.611±0\.0420\.611\\pm 0\.042LLaMA 3\.1 8B Inst\.0\.7780\.0000\.530±0\.0560\.530\\pm 0\.0560\.6000\.0000\.294±0\.0490\.294\\pm 0\.0490\.6500\.0000\.522±0\.0470\.522\\pm 0\.047Ministral 8B0\.8850\.0000\.596±0\.0470\.596\\pm 0\.0470\.6640\.0000\.382±0\.0630\.382\\pm 0\.0630\.8050\.0000\.615±0\.0440\.615\\pm 0\.044LLaMA 3\.3 70B Inst\.0\.8460\.0250\.593±0\.0530\.593\\pm 0\.0530\.6570\.4990\.385±0\.0590\.385\\pm 0\.0590\.7860\.1910\.620±0\.0430\.620\\pm 0\.043Mistral Large0\.9160\.1000\.657±0\.0500\.657\\pm 0\.0500\.6580\.2390\.357±0\.0550\.357\\pm 0\.0550\.8370\.2530\.628±0\.0380\.628\\pm 0\.038Five\-element0\.9340\.4000\.839±0\.026\\bm\{0\.839\\pm 0\.026\}0\.6840\.6000\.165±0\.0050\.165\\pm 0\.0050\.9220\.5340\.583±0\.0240\.583\\pm 0\.024CoE0\.7240\.5160\.698±0\.0050\.698\\pm 0\.0050\.6340\.3810\.122±0\.0110\.122\\pm 0\.0110\.7450\.0920\.388±0\.0300\.388\\pm 0\.030SFT0\.9050\.4830\.699±0\.0400\.699\\pm 0\.0400\.7020\.6030\.168±0\.0100\.168\\pm 0\.0100\.8300\.1290\.458±0\.0120\.458\\pm 0\.012Table 8:Comprehensive metrics for performance on NL\+Data case\.NLP4LPMamo EasyMamo ComplexModel/MethodEF1EBCLLM\-ESEF1EBCLLM\-ESEF1EBCLLM\-ESGPT\-4o Mini0\.7370\.2660\.421±0\.0490\.421\\pm 0\.0490\.6000\.4900\.419±0\.0490\.419\\pm 0\.0490\.8110\.3360\.529±0\.0490\.529\\pm 0\.049GPT\-4o0\.7230\.6270\.451±0\.0470\.451\\pm 0\.0470\.6270\.5130\.403±0\.0530\.403\\pm 0\.0530\.8010\.3850\.558±0\.0660\.558\\pm 0\.066Gemini 2\.5 Pro0\.8790\.2730\.347±0\.0550\.347\\pm 0\.0550\.6350\.2190\.299±0\.0590\.299\\pm 0\.0590\.8000\.0320\.411±0\.0550\.411\\pm 0\.055Claude Haiku 4\.50\.8240\.6130\.438±0\.0470\.438\\pm 0\.0470\.6670\.6970\.461±0\.0560\.461\\pm 0\.0560\.8750\.5510\.554±0\.0430\.554\\pm 0\.043Claude Sonnet 4\.60\.8610\.2230\.355±0\.0740\.355\\pm 0\.0740\.7010\.2840\.503±0\.0490\.503\\pm 0\.0490\.8760\.6780\.539±0\.0550\.539\\pm 0\.055Claude Opus 4\.60\.9120\.6550\.412±0\.0500\.412\\pm 0\.0500\.6830\.5860\.480±0\.0590\.480\\pm 0\.0590\.8720\.7870\.551±0\.0450\.551\\pm 0\.045LLaMA 3\.1 8B Inst\.0\.7200\.0000\.371±0\.0390\.371\\pm 0\.0390\.5370\.0000\.363±0\.0460\.363\\pm 0\.0460\.5930\.0320\.418±0\.0490\.418\\pm 0\.049Ministral 8B0\.8010\.0000\.417±0\.0600\.417\\pm 0\.0600\.5630\.0100\.398±0\.0610\.398\\pm 0\.0610\.7360\.0000\.483±0\.0500\.483\\pm 0\.050LLaMA 3\.3 70B Inst\.0\.6060\.1870\.398±0\.0450\.398\\pm 0\.0450\.5280\.3050\.420±0\.0480\.420\\pm 0\.0480\.8520\.1010\.473±0\.0520\.473\\pm 0\.052Mistral Large0\.5390\.1320\.453±0\.0560\.453\\pm 0\.0560\.5400\.1200\.416±0\.0480\.416\\pm 0\.0480\.8260\.2840\.503±0\.0520\.503\\pm 0\.052Five\-element0\.8520\.6330\.710±0\.014\\bm\{0\.710\\pm 0\.014\}0\.7420\.4400\.543±0\.007\\bm\{0\.543\\pm 0\.007\}0\.8210\.4150\.629±0\.010\\bm\{0\.629\\pm 0\.010\}CoE0\.5690\.0330\.031±0\.0160\.031\\pm 0\.0160\.5070\.0140\.254±0\.0220\.254\\pm 0\.0220\.9350\.0060\.245±0\.0290\.245\\pm 0\.029SFT0\.8120\.4950\.666±0\.0170\.666\\pm 0\.0170\.5420\.5800\.431±0\.0090\.431\\pm 0\.0090\.8210\.1800\.526±0\.0060\.526\\pm 0\.006Table 9:Comprehensive metrics for performance on NL\+Data case with irrelevant attributes\.NLP4LPMamo EasyMamo ComplexModel/MethodEF1EBCLLM\-ESEF1EBCLLM\-ESEF1EBCLLM\-ESGPT\-4o Mini0\.9010\.2590\.476±0\.0530\.476\\pm 0\.0530\.6370\.1510\.289±0\.0350\.289\\pm 0\.0350\.7300\.1320\.433±0\.0400\.433\\pm 0\.040GPT\-4o0\.9130\.1250\.553±0\.0620\.553\\pm 0\.0620\.6090\.3280\.310±0\.0500\.310\\pm 0\.0500\.7920\.4160\.502±0\.0520\.502\\pm 0\.052Gemini 2\.5 Pro0\.3780\.0000\.318±0\.0430\.318\\pm 0\.0430\.2220\.0000\.131±0\.0450\.131\\pm 0\.0450\.2820\.0000\.372±0\.0490\.372\\pm 0\.049Claude Haiku 4\.50\.7820\.5420\.486±0\.0410\.486\\pm 0\.0410\.6960\.5600\.459±0\.0400\.459\\pm 0\.0400\.8180\.4380\.511±0\.0370\.511\\pm 0\.037Claude Sonnet 4\.60\.8640\.2890\.416±0\.0340\.416\\pm 0\.0340\.6460\.2150\.408±0\.0380\.408\\pm 0\.0380\.8450\.5420\.543±0\.0420\.543\\pm 0\.042Claude Opus 4\.60\.8660\.5000\.441±0\.0430\.441\\pm 0\.0430\.6370\.5140\.410±0\.0310\.410\\pm 0\.0310\.8610\.6090\.556±0\.039\\bm\{0\.556\\pm 0\.039\}LLaMA 3\.1 8B Inst\.0\.7520\.0000\.476±0\.0530\.476\\pm 0\.0530\.5200\.0000\.289±0\.0350\.289\\pm 0\.0350\.6080\.0000\.433±0\.0400\.433\\pm 0\.040Ministral 8B0\.8620\.0000\.548±0\.0580\.548\\pm 0\.0580\.5800\.0000\.300±0\.0460\.300\\pm 0\.0460\.6940\.0000\.465±0\.0460\.465\\pm 0\.046LLaMA 3\.3 70B Inst\.0\.7870\.0000\.506±0\.0570\.506\\pm 0\.0570\.5570\.3450\.318±0\.044\\bm\{0\.318\\pm 0\.044\}0\.7000\.1480\.526±0\.0540\.526\\pm 0\.054Mistral Large0\.8980\.0500\.575±0\.0420\.575\\pm 0\.0420\.6270\.0920\.309±0\.0590\.309\\pm 0\.0590\.7150\.1920\.514±0\.0510\.514\\pm 0\.051Five\-element0\.9010\.3140\.793±0\.012\\bm\{0\.793\\pm 0\.012\}0\.5740\.1330\.072±0\.0280\.072\\pm 0\.0280\.8970\.1940\.342±0\.0110\.342\\pm 0\.011CoE0\.6390\.1860\.499±0\.0330\.499\\pm 0\.0330\.4910\.0340\.086±0\.0100\.086\\pm 0\.0100\.6320\.0320\.271±0\.0240\.271\\pm 0\.024SFT0\.8940\.3640\.612±0\.0090\.612\\pm 0\.0090\.6750\.2160\.136±0\.0360\.136\\pm 0\.0360\.8150\.3420\.206±0\.0190\.206\\pm 0\.019Table 10:Comprehensive metrics for performance on derivation case\.NLP4LPMamo EasyMamo ComplexModel/MethodEF1EBCLLM\-ESEF1EBCLLM\-ESEF1EBCLLM\-ESGPT\-4o Mini0\.6720\.1940\.389±0\.0410\.389\\pm 0\.0410\.4990\.1900\.353±0\.0400\.353\\pm 0\.0400\.6540\.0480\.453±0\.0370\.453\\pm 0\.037GPT\-4o0\.6850\.4440\.396±0\.0520\.396\\pm 0\.0520\.5370\.2830\.388±0\.0340\.388\\pm 0\.0340\.6950\.2750\.447±0\.0420\.447\\pm 0\.042Gemini 2\.5 Pro0\.9140\.1280\.290±0\.0650\.290\\pm 0\.0650\.7080\.0250\.209±0\.0630\.209\\pm 0\.0630\.8000\.0330\.373±0\.0520\.373\\pm 0\.052Claude Haiku 4\.50\.7660\.5100\.398±0\.0420\.398\\pm 0\.0420\.6530\.4780\.394±0\.0510\.394\\pm 0\.0510\.8270\.3570\.491±0\.0330\.491\\pm 0\.033Claude Sonnet 4\.60\.7950\.2250\.341±0\.0380\.341\\pm 0\.0380\.5610\.2760\.381±0\.0470\.381\\pm 0\.0470\.8510\.4260\.493±0\.049\\bm\{0\.493\\pm 0\.049\}Claude Opus 4\.60\.8040\.3150\.355±0\.0390\.355\\pm 0\.0390\.6430\.4170\.408±0\.049\\bm\{0\.408\\pm 0\.049\}0\.8440\.5400\.484±0\.0380\.484\\pm 0\.038LLaMA 3\.1 8B Inst\.0\.6100\.0000\.345±0\.0430\.345\\pm 0\.0430\.4830\.0000\.328±0\.0280\.328\\pm 0\.0280\.4420\.0000\.379±0\.0380\.379\\pm 0\.038Ministral 8B0\.7170\.0000\.368±0\.0440\.368\\pm 0\.0440\.4890\.0000\.340±0\.0440\.340\\pm 0\.0440\.6000\.0000\.423±0\.0410\.423\\pm 0\.041LLaMA 3\.3 70B Inst\.0\.6960\.0960\.392±0\.0400\.392\\pm 0\.0400\.4210\.1600\.359±0\.0390\.359\\pm 0\.0390\.5740\.0200\.428±0\.0450\.428\\pm 0\.045Mistral Large0\.6680\.1500\.398±0\.0420\.398\\pm 0\.0420\.4800\.0340\.382±0\.0390\.382\\pm 0\.0390\.7140\.0790\.439±0\.0400\.439\\pm 0\.040Five\-element0\.6520\.3200\.493±0\.016\\bm\{0\.493\\pm 0\.016\}0\.5120\.1100\.366±0\.0120\.366\\pm 0\.0120\.6480\.1450\.390±0\.0170\.390\\pm 0\.017CoE0\.6000\.0240\.085±0\.0080\.085\\pm 0\.0080\.6970\.3420\.218±0\.0160\.218\\pm 0\.0160\.6730\.0120\.191±0\.0190\.191\\pm 0\.019SFT0\.5420\.2800\.393±0\.0150\.393\\pm 0\.0150\.4120\.1950\.295±0\.0120\.295\\pm 0\.0120\.6210\.2850\.327±0\.0160\.327\\pm 0\.016Table 11:Comprehensive metrics for performance on derivation case with irrelevant attributes\.NLP4LPMamo EasyMamo ComplexModel/MethodEF1EBCLLM\-ESEF1EBCLLM\-ESEF1EBCLLM\-ESGPT\-4o Mini0\.3260\.2400\.531±0\.0580\.531\\pm 0\.0580\.3990\.3850\.471±0\.0440\.471\\pm 0\.0440\.4420\.2040\.559±0\.0480\.559\\pm 0\.048GPT\-4o0\.6760\.5540\.575±0\.049\\bm\{0\.575\\pm 0\.049\}0\.5460\.5480\.505±0\.0660\.505\\pm 0\.0660\.7100\.4040\.548±0\.0410\.548\\pm 0\.041Gemini 2\.5 Pro0\.4050\.3220\.472±0\.0430\.472\\pm 0\.0430\.2970\.2930\.385±0\.0570\.385\\pm 0\.0570\.1130\.0380\.424±0\.0460\.424\\pm 0\.046Claude Haiku 4\.50\.8910\.3550\.528±0\.0570\.528\\pm 0\.0570\.7090\.4730\.468±0\.0440\.468\\pm 0\.0440\.9020\.6290\.574±0\.0560\.574\\pm 0\.056Claude Sonnet 4\.60\.7690\.1640\.439±0\.0390\.439\\pm 0\.0390\.7120\.5570\.505±0\.0540\.505\\pm 0\.0540\.8440\.6020\.531±0\.0430\.531\\pm 0\.043Claude Opus 4\.60\.9090\.7200\.557±0\.0500\.557\\pm 0\.0500\.6830\.7210\.448±0\.0650\.448\\pm 0\.0650\.8960\.7590\.582±0\.049\\bm\{0\.582\\pm 0\.049\}LLaMA 3\.1 8B Inst\.0\.7190\.0000\.456±0\.0440\.456\\pm 0\.0440\.6300\.0000\.426±0\.0450\.426\\pm 0\.0450\.5860\.0000\.435±0\.0420\.435\\pm 0\.042Ministral 8B0\.5980\.0000\.507±0\.0600\.507\\pm 0\.0600\.5690\.0000\.530±0\.0430\.530\\pm 0\.0430\.6930\.0000\.484±0\.0480\.484\\pm 0\.048LLaMA 3\.3 70B Inst\.0\.3610\.2160\.466±0\.0390\.466\\pm 0\.0390\.3340\.3670\.502±0\.0560\.502\\pm 0\.0560\.3360\.0980\.526±0\.0480\.526\\pm 0\.048Mistral Large0\.3870\.2400\.576±0\.057\\bm\{0\.576\\pm 0\.057\}0\.1130\.1370\.521±0\.0620\.521\\pm 0\.0620\.3710\.2050\.542±0\.0520\.542\\pm 0\.052Five\-element0\.9040\.1000\.471±0\.0160\.471\\pm 0\.0160\.6830\.1000\.514±0\.0120\.514\\pm 0\.0120\.8110\.3000\.372±0\.0020\.372\\pm 0\.002CoE0\.8320\.1330\.560±0\.0020\.560\\pm 0\.0020\.7050\.1420\.575±0\.0180\.575\\pm 0\.0180\.7710\.0310\.423±0\.0100\.423\\pm 0\.010SFT0\.8840\.3510\.540±0\.0130\.540\\pm 0\.0130\.7640\.6760\.661±0\.014\\bm\{0\.661\\pm 0\.014\}0\.8010\.2850\.216±0\.0090\.216\\pm 0\.009Table 12:Comprehensive metrics for performance on prediction case \(one variable\)\.NLP4LPMamo EasyMamo ComplexModel/MethodEF1EBCLLM\-ESEF1EBCLLM\-ESEF1EBCLLM\-ESGPT\-4o Mini0\.4230\.2470\.418±0\.0490\.418\\pm 0\.0490\.3300\.2980\.426±0\.0430\.426\\pm 0\.0430\.4510\.2570\.482±0\.0440\.482\\pm 0\.044GPT\-4o0\.6700\.4800\.435±0\.0360\.435\\pm 0\.0360\.5270\.5850\.446±0\.0570\.446\\pm 0\.0570\.7540\.4100\.513±0\.0520\.513\\pm 0\.052Gemini 2\.5 Pro0\.3300\.0650\.312±0\.0680\.312\\pm 0\.0680\.2420\.2680\.315±0\.0680\.315\\pm 0\.0680\.0240\.0350\.411±0\.0450\.411\\pm 0\.045Claude Haiku 4\.50\.8000\.4010\.407±0\.0470\.407\\pm 0\.0470\.6840\.4220\.416±0\.0440\.416\\pm 0\.0440\.8510\.5040\.529±0\.054\\bm\{0\.529\\pm 0\.054\}Claude Sonnet 4\.60\.7510\.3170\.364±0\.0370\.364\\pm 0\.0370\.7160\.4930\.468±0\.0560\.468\\pm 0\.0560\.8540\.6940\.498±0\.0550\.498\\pm 0\.055Claude Opus 4\.60\.8530\.3910\.415±0\.0410\.415\\pm 0\.0410\.6690\.6900\.452±0\.0570\.452\\pm 0\.0570\.8720\.6800\.520±0\.0600\.520\\pm 0\.060LLaMA 3\.1 8B Inst\.0\.5710\.0000\.355±0\.0340\.355\\pm 0\.0340\.4650\.0000\.389±0\.0410\.389\\pm 0\.0410\.5010\.0000\.374±0\.0490\.374\\pm 0\.049Ministral 8B0\.6920\.0000\.388±0\.0410\.388\\pm 0\.0410\.5640\.0110\.417±0\.0570\.417\\pm 0\.0570\.5640\.0000\.436±0\.0520\.436\\pm 0\.052LLaMA 3\.3 70B Inst\.0\.1390\.1020\.385±0\.0390\.385\\pm 0\.0390\.2370\.3170\.442±0\.0510\.442\\pm 0\.0510\.1590\.0940\.454±0\.0520\.454\\pm 0\.052Mistral Large0\.2900\.1210\.443±0\.0470\.443\\pm 0\.0470\.0750\.0570\.438±0\.0540\.438\\pm 0\.0540\.2000\.0580\.470±0\.0520\.470\\pm 0\.052Five\-element0\.8640\.2670\.453±0\.0170\.453\\pm 0\.0170\.6820\.1000\.421±0\.0110\.421\\pm 0\.0110\.7290\.2000\.288±0\.0340\.288\\pm 0\.034CoE0\.7150\.1080\.592±0\.018\\bm\{0\.592\\pm 0\.018\}0\.5890\.2310\.522±0\.0120\.522\\pm 0\.0120\.7260\.0570\.374±0\.0110\.374\\pm 0\.011SFT0\.8690\.3250\.449±0\.0170\.449\\pm 0\.0170\.7320\.4170\.576±0\.007\\bm\{0\.576\\pm 0\.007\}0\.7950\.2850\.124±0\.0150\.124\\pm 0\.015Table 13:Comprehensive metrics for performance on prediction case \(one variable\) with irrelevant attributes\.NLP4LPMamo EasyMamo ComplexModel/MethodEF1EBCLLM\-ESEF1EBCLLM\-ESEF1EBCLLM\-ESGPT\-4o Mini0\.3340\.2830\.568±0\.0560\.568\\pm 0\.0560\.3230\.2980\.481±0\.0550\.481\\pm 0\.0550\.4610\.2420\.548±0\.0610\.548\\pm 0\.061GPT\-4o0\.7760\.5960\.600±0\.0500\.600\\pm 0\.0500\.5310\.5380\.503±0\.0620\.503\\pm 0\.0620\.7110\.4470\.574±0\.0470\.574\\pm 0\.047Gemini 2\.5 Pro0\.4100\.1990\.480±0\.0390\.480\\pm 0\.0390\.2450\.1840\.395±0\.0630\.395\\pm 0\.0630\.1370\.0000\.426±0\.0410\.426\\pm 0\.041Claude Haiku 4\.50\.8790\.6300\.640±0\.0390\.640\\pm 0\.0390\.6320\.3940\.454±0\.0400\.454\\pm 0\.0400\.9070\.4670\.557±0\.0560\.557\\pm 0\.056Claude Sonnet 4\.60\.8420\.3280\.489±0\.0510\.489\\pm 0\.0510\.6220\.4930\.465±0\.0460\.465\\pm 0\.0460\.8530\.5980\.528±0\.0460\.528\\pm 0\.046Claude Opus 4\.60\.8520\.5180\.493±0\.0490\.493\\pm 0\.0490\.6800\.6330\.460±0\.0540\.460\\pm 0\.0540\.9170\.7540\.593±0\.053\\bm\{0\.593\\pm 0\.053\}LLaMA 3\.1 8B Inst\.0\.7950\.0000\.498±0\.0490\.498\\pm 0\.0490\.5510\.0000\.404±0\.0340\.404\\pm 0\.0340\.5960\.0000\.420±0\.0420\.420\\pm 0\.042Ministral 8B0\.8290\.0000\.546±0\.0590\.546\\pm 0\.0590\.5590\.0000\.470±0\.0430\.470\\pm 0\.0430\.6700\.0000\.494±0\.0580\.494\\pm 0\.058LLaMA 3\.3 70B Inst\.0\.4290\.3110\.493±0\.0600\.493\\pm 0\.0600\.3890\.3990\.492±0\.0500\.492\\pm 0\.0500\.4040\.2200\.522±0\.0580\.522\\pm 0\.058Mistral Large0\.3080\.3240\.598±0\.0600\.598\\pm 0\.0600\.1230\.0900\.490±0\.0620\.490\\pm 0\.0620\.3410\.2960\.546±0\.0490\.546\\pm 0\.049Five\-element0\.8900\.2000\.745±0\.011\\bm\{0\.745\\pm 0\.011\}0\.6770\.1000\.418±0\.0120\.418\\pm 0\.0120\.7960\.2330\.267±0\.0180\.267\\pm 0\.018CoE0\.8260\.2980\.736±0\.0090\.736\\pm 0\.0090\.6910\.0340\.518±0\.008\\bm\{0\.518\\pm 0\.008\}0\.7680\.0330\.358±0\.0050\.358\\pm 0\.005SFT0\.8790\.3140\.610±0\.0340\.610\\pm 0\.0340\.7300\.3390\.321±0\.0140\.321\\pm 0\.0140\.7800\.2850\.160±0\.0180\.160\\pm 0\.018Table 14:Comprehensive metrics for performance on prediction case \(multiple variables\)\.NLP4LPMamo EasyMamo ComplexModel/MethodEF1EBCLLM\-ESEF1EBCLLM\-ESEF1EBCLLM\-ESGPT\-4o Mini0\.3970\.2640\.426±0\.0350\.426\\pm 0\.0350\.2500\.1810\.415±0\.0500\.415\\pm 0\.0500\.2920\.1400\.472±0\.0500\.472\\pm 0\.050GPT\-4o0\.7430\.5690\.453±0\.0370\.453\\pm 0\.0370\.4920\.5100\.419±0\.0590\.419\\pm 0\.0590\.7110\.4710\.485±0\.0520\.485\\pm 0\.052Gemini 2\.5 Pro0\.3000\.2060\.333±0\.0550\.333\\pm 0\.0550\.1920\.1300\.298±0\.0740\.298\\pm 0\.0740\.0900\.0340\.395±0\.0580\.395\\pm 0\.058Claude Haiku 4\.50\.7780\.5990\.428±0\.0480\.428\\pm 0\.0480\.6100\.3450\.409±0\.0450\.409\\pm 0\.0450\.8250\.3890\.503±0\.0450\.503\\pm 0\.045Claude Sonnet 4\.60\.8030\.2530\.331±0\.0690\.331\\pm 0\.0690\.6830\.4290\.444±0\.0520\.444\\pm 0\.0520\.7730\.6050\.473±0\.0480\.473\\pm 0\.048Claude Opus 4\.60\.9080\.6560\.432±0\.0470\.432\\pm 0\.0470\.6600\.5580\.441±0\.0450\.441\\pm 0\.0450\.8640\.6360\.518±0\.058\\bm\{0\.518\\pm 0\.058\}LLaMA 3\.1 8B Inst\.0\.5830\.0000\.384±0\.0340\.384\\pm 0\.0340\.4270\.0000\.337±0\.0370\.337\\pm 0\.0370\.4570\.0000\.354±0\.0450\.354\\pm 0\.045Ministral 8B0\.6760\.0000\.392±0\.0330\.392\\pm 0\.0330\.4930\.0110\.395±0\.0530\.395\\pm 0\.0530\.5910\.0000\.439±0\.0420\.439\\pm 0\.042LLaMA 3\.3 70B Inst\.0\.3190\.2440\.366±0\.0400\.366\\pm 0\.0400\.2730\.2300\.394±0\.0360\.394\\pm 0\.0360\.2630\.1390\.435±0\.0460\.435\\pm 0\.046Mistral Large0\.3530\.2700\.410±0\.0460\.410\\pm 0\.0460\.1170\.1110\.407±0\.0390\.407\\pm 0\.0390\.2850\.1900\.478±0\.0450\.478\\pm 0\.045Five\-element0\.8940\.3000\.507±0\.0150\.507\\pm 0\.0150\.6810\.1000\.329±0\.0100\.329\\pm 0\.0100\.7680\.2330\.269±0\.0190\.269\\pm 0\.019CoE0\.7220\.3090\.743±0\.003\\bm\{0\.743\\pm 0\.003\}0\.5760\.0900\.471±0\.033\\bm\{0\.471\\pm 0\.033\}0\.7190\.0940\.294±0\.0110\.294\\pm 0\.011SFT0\.9040\.1580\.439±0\.0220\.439\\pm 0\.0220\.7120\.1640\.286±0\.0130\.286\\pm 0\.0130\.7780\.0320\.141±0\.0190\.141\\pm 0\.019Table 15:Comprehensive metrics for performance on prediction case \(multiple variables\) with irrelevant attributes\.Table 16:Evaluation results for Claude Opus 4\.6 on the full benchmark\.Table 17:Evaluation results for GPT\-4o Mini on the full benchmark\.Table 18:Evaluation results for LLaMA 3\.3 70B Instruct on the full benchmark\.