Synthesizing Feature Extractors: An Agentic Approach for Algorithm Selection

# An Agentic Approach for Algorithm Selection
Source: [https://arxiv.org/html/2608.17170](https://arxiv.org/html/2608.17170)
## Synthesizing Feature Extractors: An Agentic Approach for Algorithm SelectionThanks:Code, data, and the full reproducibility archive:[https://doi\.org/10\.5281/zenodo\.20161743](https://doi.org/10.5281/zenodo.20161743)\.

Hai XiaCarlos AnsóteguiAffiliation:Logic & Optimization Group, University of Lleida, Lleida, Spain carlos\.ansotegui@udl\.catStefan SzeiderAffiliation:Algorithms and Complexity Group, TU Wien, Vienna, Austria \{hxia,sz\}@ac\.tuwien\.ac\.at

###### Abstract

Algorithm selection for constraint satisfaction problems requires extracting features that capture problem structure\. Manually designing feature extractors demands deep domain expertise and quickly becomes a bottleneck when new problem classes appear\. We present an automated approach that uses Large Language Models \(LLMs\) in an agentic check–fix–verify loop to synthesize executable Python scripts that act as interpretable, problem\-specific feature extractors\. Given a high\-level MiniZinc model and an instance, the LLM agent generates code that constructs a typed graph representation and computes structural properties such as graph density, variable clustering, and constraint tightness\. We evaluate our approach on three combinatorial problems \(vehicle routing, car sequencing, fixed\-length error\-correcting codes\) with a portfolio of five state\-of\-the\-art solvers\. The synthesized extractors yield algorithm selectors that consistently outperform both expert\-curated*mzn2feat*features \(up to8\.38\.3percentage points \(pp\) test\-set accuracy on FLECC\) and the best transformer\-based*trans2feat*variants\. In the meanwhile, the synthesized feature extractors remain inspectable\.

## 1Introduction

Combinatorial problems such as vehicle routing \(VRP\), car sequencing \(CS\), and fixed\-length error\-correcting codes \(FLECC\) are computationally hard: state\-of\-the\-art solvers can run for hours without finding proven optima, and the gap between the best and a poorly chosen solver on an instance can span orders of magnitude\([Kotthoff, 2016](https://arxiv.org/html/2608.17170#bib.bib12);[Kerschke et al\., 2019](https://arxiv.org/html/2608.17170#bib.bib10)\)\. The difficulty of an instance depends strongly on its structure \(customer density, time\-window tightness, constraint coupling\), and that structure determines which solver performs best\([Smith\-Miles and Lopes, 2012](https://arxiv.org/html/2608.17170#bib.bib23)\)\. Algorithm selection \(AS\) exploits this fact by mapping instances to the most effective solvers from a portfolio of complementary solvers\([Rice, 1976](https://arxiv.org/html/2608.17170#bib.bib20);[Kerschke et al\., 2019](https://arxiv.org/html/2608.17170#bib.bib10)\)\.

The standard AS pipeline depends on a*feature extractor*that computes an informative feature vector from the problem instance\. However, designing such an extractor typically requires substantial domain expertise and insight into which structural properties correlate with solver performance\. Moreover, validating an extractor’s efficiency and effectiveness across instances takes a long time\. Therefore, AS has been applied primarily to domains with mature formalisms, chiefly SAT\([Hoos et al\., 2021](https://arxiv.org/html/2608.17170#bib.bib9);[Shavit and Hoos, 2024](https://arxiv.org/html/2608.17170#bib.bib22)\)and constraint programming via*mzn2feat*\([Amadini et al\., 2013](https://arxiv.org/html/2608.17170#bib.bib1);[Amadini et al\., 2014](https://arxiv.org/html/2608.17170#bib.bib2)\)\. When it comes to problem classes outside classical formalisms, researchers either build a new extractor from scratch or translate the problem into one of them, with possible structural information loss\.

In general, there are several sides preventing the AS pipeline from being widely used\. On the design side, it requires expert\-curated extractors like*mzn2feat*\. But for these extractors, it is still possible to miss solver\-relevant structural properties that our experiments expose \(Section[4](https://arxiv.org/html/2608.17170#S4)\)\. The expertise\-necessary design also makes feature extractors generated without full autonomy\. However, LLMs \(Large Language Models\) are trained with diverse domain knowledge, making them suitable to curate feature extractors\. On the LLM side, a naive single\-shot prompt does not work in practice: weaker open\-source backends fail to produce a working extractor in10/1010/10trials, and removing any step in our loop below collapses success to zero even for strong models \(Sections[3](https://arxiv.org/html/2608.17170#S3)and[4\.9](https://arxiv.org/html/2608.17170#S4.SS9)\)\. The technical question is how to wrap an LLM so its outputs are reliably executable, AS\-relevant, and inspectable\.

#### Our approach: automating feature extraction via LLMs

We introduce an LLM\-based framework that automatically generates executable Python scripts as feature extractors\. The design is a*two\-level process*: an LLM agent, wrapped in a check–fix–verify error\-correcting loop, synthesizes the program, and the output program is then the extractor\. The agent first reads a high\-level MiniZinc problem description\([Stuckey et al\., 2014](https://arxiv.org/html/2608.17170#bib.bib25);[Marriott et al\., 2008](https://arxiv.org/html/2608.17170#bib.bib15)\)and produces a Python script\. Then the extractor \(the Python script\) constructs a graph representation from an instance and outputs a vector of interpretable features\. As MiniZinc is declarative formalisms with rich information, like give the LLM a compact view of the problem’s structural patterns, we use MiniZinc as our problem modeling environment\.

The framework produces explicit feature extractors \(Python programs\), and the output features are interpretable rather than opaque neural embeddings\. While recent work has explored deep learning for producing latent problem representations\([Pellegrino et al\., 2025](https://arxiv.org/html/2608.17170#bib.bib18);[Zhang et al\., 2024](https://arxiv.org/html/2608.17170#bib.bib30);[Loreggia et al\., 2016](https://arxiv.org/html/2608.17170#bib.bib14)\), such approaches sacrifice transparency for automation\. Our generated extractors produce graphical features, like graph density, variable clustering, constraint tightness, statistical summaries of data that domain experts can read, validate, and refine\. This “gray\-box” design keeps the automation accessible to human understanding and improvement\.

#### Empirical validation

We validate the approach on AS for three combinatorial problems \(VRP, CS, FLECC\) using a portfolio of five state\-of\-the\-art solvers \(Gurobi, CPLEX, SCIP, Gecode, OR\-Tools\)\. The synthesized extractors outperform both*mzn2feat*, the established expert\-curated extractor for MiniZinc problems\([Amadini et al\., 2013](https://arxiv.org/html/2608.17170#bib.bib1);[Amadini et al\., 2014](https://arxiv.org/html/2608.17170#bib.bib2)\), and the transformer\-based*trans2feat*\([Pellegrino et al\., 2025](https://arxiv.org/html/2608.17170#bib.bib18)\)\. The gain comes from capturing high\-level structural properties that flat representations and opaque embeddings miss\.

#### Contributions

1. 1\.We demonstrate that an LLM agent can reason about combinatorial problem structure and synthesize functional, interpretable feature extractors from MiniZinc problem descriptions, reducing the manual engineering cost of building AS pipelines for problems expressible in MiniZinc\.
2. 2\.We propose an agentic check–fix–verify pipeline whose intermediate artifact is an explicit Python program\. Unlike opaque neural embeddings, the generated extractors expose inspectable features\.
3. 3\.Across VRP, CS, and FLECC, selectors built from our synthesized features outperform selectors built from expert\-curated*mzn2feat*and transformer\-based*trans2feat*\. This suggests the LLM uncovers solver\-aware structural patterns that expert\-curated extractors and transformer\-based pipelines miss\.

## 2Related Work

The*Algorithm Selection Problem*\(AS\)\([Rice, 1976](https://arxiv.org/html/2608.17170#bib.bib20);[Kerschke et al\., 2019](https://arxiv.org/html/2608.17170#bib.bib10)\)considers a portfolio𝒫\\mathcal\{P\}of algorithms, a set of instancesII, a performance metricP​M​\(A,i\)PM\(A,i\), and a resource budgetBB\. Since the performance of an algorithmA∈𝒫A\\in\\mathcal\{P\}varies across instances, an AS strategy must predict,*before solving*, whichAAto run on a given instance\. To make this tractable, each instancei∈Ii\\in Iis described by a*feature vector*ϕ⁡\(i\)∈ℝd\\phi\(i\)\\in\\mathbb\{R\}^\{d\}obtained from a feature extractorΦ:i↦ϕ⁡\(i\)\\Phi:i\\mapsto\\phi\(i\)\. The AS task is to learn a selectorS:ℝd→𝒫S:\\mathbb\{R\}^\{d\}\\rightarrow\\mathcal\{P\}maximizing∑i∈IP​M​\(S⁡\(ϕ⁡\(i\)\),i\)\\sum\_\{i\\in I\}PM\(S\(\\phi\(i\)\),i\)subject toBB\. Two reference points calibrate AS performance:

###### Definition 2\.1\(Single Best Solver\)\.

The*Single Best Solver*\(SB\) is the algorithmASB=arg⁡maxA∈𝒫​P​M​\(A,I\)A^\{\\text\{SB\}\}=\\arg\\max\_\{A\\in\\mathcal\{P\}\}PM\(A,I\)that achieves the best overall performance across the entire instance setII\. The SB strategy appliesASBA^\{\\text\{SB\}\}to every instance\.

###### Definition 2\.2\(Virtual Best Solver\)\.

The*Virtual Best Solver*\(VBS\) is the \(hypothetical\) per\-instance selector that, for eachi∈Ii\\in I, chooses the algorithmA∈𝒫A\\in\\mathcal\{P\}that achieves the best performance on that instance:P​M​\(VBS,I\)=∑i∈ImaxA∈𝒫⁡P​M​\(A,\{i\}\)PM\(\\text\{VBS\},I\)=\\sum\_\{i\\in I\}\\max\_\{A\\in\\mathcal\{P\}\}PM\(A,\\\{i\\\}\)\. The VBS upper\-bounds the performance of any AS strategy\.

MiniZinc\([Nethercote et al\., 2007](https://arxiv.org/html/2608.17170#bib.bib17)\)is a high\-level, declarative modeling language for constraint satisfaction and optimization\. A model file \(\.mzn\) defines variables, constraints, and \(optionally\) an objective\. A data file \(\.dzn\) contains instance parameters\.

#### Expert\-curated features

Classical AS builds on hand\-engineered feature sets such as SATzilla’s\([Shavit and Hoos, 2024](https://arxiv.org/html/2608.17170#bib.bib22)\)and*mzn2feat*’s\([Amadini et al\., 2013](https://arxiv.org/html/2608.17170#bib.bib1);[Amadini et al\., 2014](https://arxiv.org/html/2608.17170#bib.bib2)\); hand\-crafted features likewise drive per\-instance policy selection inside solvers, e\.g\. in SAT\-based tree decomposition\([Xia and Szeider, 2024](https://arxiv.org/html/2608.17170#bib.bib28)\)\.

#### Graph\-based features without LLMs

Encoding combinatorial instances as graphs for feature extraction does not by itself require an LLM\.[Stone et al\., 2024](https://arxiv.org/html/2608.17170#bib.bib24)convert instances from three problem domains into domain\-agnostic graph and image encodings, extract generic graph metrics, and use the resulting representations for algorithm selection and other downstream tasks\. Their pipeline applies one fixed, hand\-specified encoding and feature set across all domains; our agent instead*writes a new extractor program per problem family*, deciding from the high\-level MiniZinc model which typed graph to construct and which problem\-adapted, solver\-aware quantities to materialize as named, editable features \(e\.g\. demand concentration and depot eccentricity for VRP, Section[4\.8](https://arxiv.org/html/2608.17170#S4.SS8)\)\. The approaches are complementary: generic encodings transfer at zero synthesis cost, synthesized extractors capture semantics that fixed encodings discard\.

#### LLM\-based feature engineering and program synthesis

CAAFE\([Hollmann et al\., 2023](https://arxiv.org/html/2608.17170#bib.bib8)\)uses LLMs to generate Python feature transformations for tabular ML datasets, and FeatLLM\([Han et al\., 2024](https://arxiv.org/html/2608.17170#bib.bib7)\)prompts LLMs to derive rule\-based features for few\-shot tabular learning\. Both operate on existing tabular inputs\. More broadly, LLM\-driven program search can yield executable artifacts that are competitive in combinatorial settings\([Romera\-Paredes et al\., 2024](https://arxiv.org/html/2608.17170#bib.bib21)\)\. Our work instead starts from declarative\.mzn/\.dznspecifications and synthesizes a complete, reusable extractor program that computes structural and semantic properties of the problem specification itself\.

#### Neural embeddings for algorithm selection

[Wu et al\., 2024](https://arxiv.org/html/2608.17170#bib.bib27)use LLMs to embed algorithm source code and documentation for AS\.[Zhang et al\., 2024](https://arxiv.org/html/2608.17170#bib.bib30)combine graph neural networks with expert knowledge to select SAT solvers, learning embeddings of CNF formula structure\.[Pellegrino et al\., 2025](https://arxiv.org/html/2608.17170#bib.bib18)apply transformer encoders directly to the high\-level textual representation of constraint optimization instances to learn features\. These neural approaches produce high\-dimensional embeddings whose individual dimensions lack clear semantic meaning and cannot easily be inspected or edited\. On the FLECC and CS instance sets for which[Pellegrino et al\., 2025](https://arxiv.org/html/2608.17170#bib.bib18)released*trans2feat*features, our*LLM2feat*selectors outperform the best*trans2feat*variant by7\.47\.4and5\.45\.4percentage points \(pp\) in test accuracy \(Section[4\.7](https://arxiv.org/html/2608.17170#S4.SS7)\)\.

## 3Problem\-Specific LLM\-Based Agent

We build on the agentic framework that bridges LLMs and constraint satisfaction solving\([Szeider, 2025](https://arxiv.org/html/2608.17170#bib.bib26)\)\.

###### Definition 3\.1\(LLM agent\)\.

A*Large Language Model agent*\([Yao et al\., 2023](https://arxiv.org/html/2608.17170#bib.bib29)\)is a tuple𝒜=\(L,T,M,π,E\)\\mathcal\{A\}=\(L,T,M,\\pi,E\), whereLLis a language model,TTis a set of external tools,MMis a memory module,π\\piis the prompting policy that maps observations and history to model inputs, andEEis the environment\. The agent operates in a loopot→𝜋pt→𝐿at→T,Eot\+1o\_\{t\}\\xrightarrow\{\\pi\}p\_\{t\}\\xrightarrow\{L\}a\_\{t\}\\xrightarrow\{T,E\}o\_\{t\+1\}, whereoto\_\{t\}is the observation at timett,ptp\_\{t\}is the constructed prompt,ata\_\{t\}is the action \(e\.g\. tool call\), andot\+1o\_\{t\+1\}is the next observation\.

Our agent takes a MiniZinc problem description \(\.mznand\.dznfiles\) and a data schema as input, and outputs a Python script that extracts a feature vector from any instance of that problem\. Two prompts control the loop in Figure[1](https://arxiv.org/html/2608.17170#S3.F1):*script\_system\_prompt*\(a general\-purpose Python\-script generation prompt that defines a strict workflow, technical requirements, and the available tools\) and*mzn\-tuning*\(a domain\-specialized prompt that instructs the agent to extract5050standardized structural features from constraint\-programming problems suitable for AS\)\.

Figure 1:Workflow for generating problem\-specific feature extractors\. The agent loops overClear,Insert,Check/Fix, andExecutesteps until an executable Python script that produces a validated feature vector is obtained\.#### General script prompt

The general Python\-script prompt \(script\-system\-prompt\.md\) enforces a four\-step workflow: \(i\)Clearall previous content; \(ii\)Insertthe complete script; \(iii\)Check/Fixsyntax and structural requirements, addressing validation errors if needed; \(iv\)Executethe script and validate its output\. The expected script structure is given in Listing[1](https://arxiv.org/html/2608.17170#LST1)\.

1

2importnecessary\_modules

3

4CONSTANTS=values

5

6defhelper\_functions\(\):

7pass

8

9if\_\_name\_\_=="\_\_main\_\_":

10

11result\_dict=\{"key":"value","results":data\}

12

13output\_results\(result\_dict\)

Listing 1:General Python script template guided byscript\-system\-prompt\.md\.
#### MiniZinc tuning prompt

The specialized*mzn\-tuning*prompt \(mzn\-tuning\-prompt\.md\) layers AS\-specific requirements on top of the general workflow: mandatory imports for framework integration, no file I/O \(instance data is accessed viainput\_data\(\)\), and a standardized output shape of exactly5050features ready for downstream selector training\. The template is given in Listing[2](https://arxiv.org/html/2608.17170#LST2)\. The size5050was fixed in pilot experiments as a balance between coverage and interpretability: comparable in scale to*mzn2feat*’s9595and*trans2feat*’s116116dimensions, yet small enough that each feature remains individually inspectable and selectors stay data\-efficient\. The*identity*of the5050features is chosen anew by the agent in every run: across the1010independent syntheses per problem \(Section[4\.7](https://arxiv.org/html/2608.17170#S4.SS7)\), core structural metrics such as graph density, degree statistics, and clustering recur consistently, while the remaining features vary in which problem\-specific quantities they materialize \(Section[4\.8](https://arxiv.org/html/2608.17170#S4.SS8)\); the per\-run feature lists and value tables ship with the supplement\. Each step of the check–fix–verify loop is load\-bearing: in pilot ablations, removing any one of the four steps produced zero executable extractors, and weaker LLM backends that drift from the protocol fail in10/1010/10trials \(Section[4\.9](https://arxiv.org/html/2608.17170#S4.SS9)\)\.

1

2fromlmtune\_helpersimportinput\_data,output\_results

3importnetworkxasnx

4importnumpyasnp

5defmain\(\):

6

7instance\_data=input\_data\(\)

8

9results=\{

10"README":"~200wordmethodologydescription",

11"characteristic\_1":0\.0,

12"characteristic\_2":0\.0,

13

14"characteristic\_50":0\.0

15\}

16

17

18

19

20

21

22

23output\_results\(results\)

24if\_\_name\_\_=="\_\_main\_\_":

25main\(\)

Listing 2:MiniZinc instance feature extraction template guided bymzn\-tuning\-prompt\.md\.

## 4Experimental Analyses

We evaluate the agentic framework on three problem\-specific AS benchmarks and analyze*why**LLM2feat*features yield better selectors than*mzn2feat*or*trans2feat*, looking at feature correlation, utilization efficiency, and selector accuracy\.

### 4\.1Experimental Settings

We use the top\-performing solvers from the MiniZinc challenge111[https://www\.minizinc\.org/challenge/2025/results/](https://www.minizinc.org/challenge/2025/results/): Gurobi \(12\.0\.3\), CPLEX \(22\.1\.2\), SCIP \(9\.2\.3\), Gecode \(6\.2\.0\), and OR\-Tools \(9\.3\.10497\)\. The portfolio spans solver paradigms: Gurobi and CPLEX are commercial mixed\-integer programming \(MIP\) solvers; SCIP combines CP and MIP techniques; Gecode is a constraint\-programming \(CP\) solver used throughout the MiniZinc Challenge history; OR\-Tools won gold medals across all major categories of the MiniZinc Challenge in 2023–2025\.

The instance setIIcontains minimization, maximization, and decision problems: “best solver” means the lowest/highest objective value at timeout for optimization problems, and the shortest solving time for decision problems; the performance metric is oriented so that larger values are better\. We use three problems with sufficient instance diversity: VRP\([Queiroga et al\., 2022](https://arxiv.org/html/2608.17170#bib.bib19)\), CS\([Pellegrino et al\., 2025](https://arxiv.org/html/2608.17170#bib.bib18)\), and FLECC\([Pellegrino et al\., 2025](https://arxiv.org/html/2608.17170#bib.bib18)\)\. Each problem is split7:37\{:\}3into training/test sets; the same split is reused across all extractor comparisons\. Solvers run with a2020\-minute timeout per instance, in line with the MiniZinc Challenge standard222[https://www\.minizinc\.org/challenge/](https://www.minizinc.org/challenge/); extractor synthesis is capped at11minute per attempt, with automatic restart on timeout\. All experiments run on a cluster with two AMD 7403 processors \(2424cores at 2\.8 GHz,3232GB RAM/core\)\. Two performance metrics are reported:A​c​cAcc, the fraction of instances on which the selector picks the truly best solver, andR​a​n​kRank, the average ranking of the selected solver \(lower is better\)\.

For LLM model selection, we use the commercial OpenAI o4\-mini\-2025\-04\-16 as the agent backend\. All LLM calls use the provider’s default generation parameters: the sampling temperature is never overridden \(o4\-mini is a reasoning model whose temperature is fixed at its default of1\.01\.0\), and the reasoning effort is the API default \(medium\); the exact call sites are in the supplement \(llm\_factory\.py\)\. Run\-to\-run stochasticity is thus a property of the backend itself and is quantified through1010independent synthesis runs per problem\. The open\-weight backends we evaluated \(Llama 3\.3, DeepSeek R1\) failed to follow the check–fix–verify protocol in our setup \(Section[4\.9](https://arxiv.org/html/2608.17170#S4.SS9)\); sensitivity analyses across LLM variants and the1010runs appear in Appendix[C](https://arxiv.org/html/2608.17170#A3)\.

### 4\.2Algorithm\-Selection Toolchains

The AS input is a feature table \(instance→\\rightarrowfeature vector\) and a performance table \(instance→\\rightarrowper\-solver performance\)\. We compare three feature extractors: our LLM\-synthesized*LLM2feat*, expert\-curated*mzn2feat*\([Amadini et al\., 2013](https://arxiv.org/html/2608.17170#bib.bib1);[Amadini et al\., 2014](https://arxiv.org/html/2608.17170#bib.bib2)\), and transformer\-based*trans2feat*\([Pellegrino et al\., 2025](https://arxiv.org/html/2608.17170#bib.bib18)\)\.

For selector training we use Random Forest \(RF\) following the standard recipe\([Kerschke et al\., 2019](https://arxiv.org/html/2608.17170#bib.bib10)\)and AutoSklearn \(AutoSK\)\([Feurer et al\., 2015](https://arxiv.org/html/2608.17170#bib.bib5);[Feurer et al\., 2022](https://arxiv.org/html/2608.17170#bib.bib4)\); hyperparameters are in Appendix[A](https://arxiv.org/html/2608.17170#A1)\. We also report AutoFolio \(AF\)\([Lindauer et al\., 2015](https://arxiv.org/html/2608.17170#bib.bib13)\)and LLAMA\([Kotthoff, 2013](https://arxiv.org/html/2608.17170#bib.bib11)\), top\-performing tools in AS surveys and challenges\([Kerschke et al\., 2019](https://arxiv.org/html/2608.17170#bib.bib10)\)\. RF and AutoSK are trained with two loss functions,A​c​cAccandR​a​n​kRank; AF and LLAMA use their defaultA​c​cAccloss\. All training uses 5\-fold cross\-validation\.

### 4\.3Research Questions

Q1:How diverse are the features generated by*LLM2feat*? Highly correlated features carry overlapping information and inflate the effective dimension of the feature space\.Q2:How efficiently does each feature contribute to the AS model? Distributions over feature importance shed light on quality and potential redundancy\.Q3:How accurate are the resulting AS models compared with*mzn2feat*\-based and*trans2feat*\-based selectors?

### 4\.4Feature Correlation Analysis \(Q1\)

For each feature set we train an RF selector as in Section[4\.1](https://arxiv.org/html/2608.17170#S4.SS1), extract the top2020features by importance for the random forest model, and compute Pearson correlation matrices\([Guyon and Elisseeff, 2003](https://arxiv.org/html/2608.17170#bib.bib6)\)on the selected features\. Figure[2](https://arxiv.org/html/2608.17170#S4.F2)shows the result for VRP:*LLM2feat*features have a48\.5%48\.5\\%lower average absolute correlation \(\|r\|=0\.221\|r\|=0\.221\) than*mzn2feat*\(\|r\|=0\.429\|r\|=0\.429\)\. The same pattern holds for FLECC \(\|r\|=0\.306\|r\|=0\.306vs\.0\.3300\.330\) and CS \(\|r\|=0\.551\|r\|=0\.551vs\.0\.7250\.725\)\. The corresponding heatmaps for FLECC and CS are in Appendix[B](https://arxiv.org/html/2608.17170#A2)\. LLM\-generated features capture a more diverse range of structural properties, reducing redundancy with fewer overlapping features\.

![Refer to caption](https://arxiv.org/html/2608.17170v1/vrp_random_forest_correlation_matrix.png)Figure 2:Feature correlation matrices for VRP \(feature names suppressed for clarity\)\.*LLM2feat*features show48\.5%48\.5\\%lower average correlation \(\|r\|=0\.221\|r\|=0\.221\) than*mzn2feat*\(\|r\|=0\.429\|r\|=0\.429\)\.
### 4\.5Feature Utilization Efficiency \(Q2\)

For both*mzn2feat*\+RF and*LLM2feat*\+RF we extract feature\-importance scores for the entire feature set and count features above a0\.0010\.001significance threshold as “effectively utilized\.”*mzn2feat*provides9595hand\-crafted features;*LLM2feat*produces5050\. Across the three problems,*LLM2feat*achieves significantly higher utilization \(Figure[3](https://arxiv.org/html/2608.17170#S4.F3)\): on VRP,96%96\\%vs\.56\.8%56\.8\\%\(a69%69\\%relative improvement\); on FLECC,58%58\\%vs\.23\.2%23\.2\\%; on CS,84%84\\%vs\.45\.1%45\.1\\%\. More of the LLM\-generated dimensions contribute meaningfully to the selector’s decisions\.

\(a\)FLECC\(b\)CS\(c\)VRP
Figure 3:Feature\-importance distribution\.*LLM2feat*achieves better spread of importance across features than*mzn2feat*on all three problems\.
### 4\.6Accuracy Analysis \(Q3\)

We evaluate AS accuracy as a function of feature\-set size, growing the set in order of decreasing RF importance\. Figure[4](https://arxiv.org/html/2608.17170#S4.F4)shows that*LLM2feat*\-based selectors reach higher accuracy with fewer features, and continue to benefit from additional features where*mzn2feat*plateaus: on VRP,*mzn2feat*flatlines near81%81\\%once1010features are used, while*LLM2feat*continues to improve and peaks beyond2020features\.

\(a\)FLECC\(b\)CS\(c\)VRP
Figure 4:AS test accuracy as a function of feature\-set size on the three benchmarks\.*LLM2feat*matches or exceeds*mzn2feat*at every size and reaches higher peak accuracy\.Table[1](https://arxiv.org/html/2608.17170#S4.T1)compares the three feature extractors across four toolchains \(AF, RF, LLAMA, AutoSK\)\. With the \(trivial\) exception of AF \(which falls back to the single best solver, SB, for all our extractors in this setting\),*LLM2feat*achieves the highest accuracy and the lowest ranking on every problem–toolchain pair\. Across VRP, CS, and FLECC,*LLM2feat*improves over the*mzn2feat*test accuracy by up to8\.38\.3pp, and over the best*trans2feat*variant by7\.47\.4pp on FLECC and5\.45\.4pp on CS \(see Section[4\.7](https://arxiv.org/html/2608.17170#S4.SS7)for the head\-to\-head with all2020*trans2feat*variants\)\. Training\-set results, ranking\-loss results, and full LLM\-sensitivity tables are in Appendix[C](https://arxiv.org/html/2608.17170#A3)\.

The three benchmarks include both classical AS regimes\. On CS the single best solver is weak \(49\.0%49\.0\\%\), and feature\-based selection adds up to15\.015\.0pp\. On VRP and FLECC the SB baseline is strong \(79\.5%79\.5\\%/78\.2%78\.2\\%\), yet*LLM2feat*still closes roughly30%30\\%of the remaining gap between SB and the virtual best solver \(\+6\.3\+6\.3pp on VRP,\+6\.5\+6\.5pp on FLECC over SB\)\. Because the feature extraction and selection add negligible overhead \(less than 1 minute\) relative to a2020\-minute solve, these gains convert directly into saved solver time\.

Table 1:Test\-set accuracy \(A​c​cAcc\) and average ranking \(R​a​n​kRank\) for AS toolchains using*mzn2feat*\(9595features\),*LLM2feat*\(5050features\), and*trans2feat*\(116116features\)\. Loss function=A​c​c=Acc\(AutoFolio, LLAMA, and*trans2feat*use their accuracy\-based defaults\)\. Each*LLM2feat*row reports the strongest of the three initial synthesis runs per problem; best/mean/worst over all1010runs appear in Table[2](https://arxiv.org/html/2608.17170#S4.T2)\.×\\timesmarks combinations unavailable because no released*trans2feat*features exist for VRP\. Best per column inbold\.
### 4\.7Head\-to\-Head Comparison with*trans2feat*

[Pellegrino et al\., 2025](https://arxiv.org/html/2608.17170#bib.bib18)release2020*trans2feat*feature variants \(one per neural\-network configuration\) for CS and FLECC\. Table[1](https://arxiv.org/html/2608.17170#S4.T1)compares only the*best**trans2feat*variant per toolchain\. To examine the comparison more carefully, we trained the same RF and AutoSK toolchains on every*trans2feat*variant, on every*LLM2feat*run from1010independent agent syntheses, and on the canonical*mzn2feat*extractor\. Table[2](https://arxiv.org/html/2608.17170#S4.T2)reports the best, mean, and worst test accuracy across each group\. Even the*worst**LLM2feat*run beats the*best**trans2feat*variant on FLECC under RF, and clearly dominates on CS as well\. Per\-variant tables for both toolchains and both problems are in Appendix[D](https://arxiv.org/html/2608.17170#A4)\.

Table 2:Head\-to\-head with*trans2feat*: test\-setA​c​cAccaggregated over1010independent*LLM2feat*runs and the2020released*trans2feat*variants\.*mzn2feat*is a single deterministic extractor\. Best/Mean/Worst refer to the best/mean/worst test accuracy across runs within each method; runs without a valid selector \(TO/F, Appendix[D](https://arxiv.org/html/2608.17170#A4)\) are excluded, leavingn=17n=17*trans2feat*variants andn=9n=9*LLM2feat*runs on CS/AutoSK andn=14n=14variants on FLECC/AutoSK\.
### 4\.8Qualitative Feature Analysis

To illustrate the solver\-awareness of LLM\-generated features, consider VRP: the agent synthesizes features such as demand\-distribution statistics \(avg\_demand\) and depot\-centrality metrics \(depot\_centrality\)\. Such features enable the selector to distinguish instances where MIP solvers excel from those better suited to CP solvers\([Moreno\-Scott et al\., 2016](https://arxiv.org/html/2608.17170#bib.bib16)\), and they account for the\+2\.9\+2\.9pp test\-accuracy gain over*mzn2feat*on VRP\. Note that the advantage cannot stem from feature\-set size:*LLM2feat*uses5050features against*mzn2feat*’s9595and*trans2feat*’s116116\. However, the agent synthesizes*relevant, complementary*feature combinations that the curated list lacks, consistent with the lower feature correlation \(Q1\) and the higher utilization \(Q2\) reported above\. We attribute the broader gains over classical baselines to three properties of the generated extractors\.

First, the extractors preserve structural information lost in flat encodings\.*mzn2feat*\([Amadini et al\., 2014](https://arxiv.org/html/2608.17170#bib.bib2)\)flattens the constraint model into a long list of primitive constraints, discarding much of the high\-level structure, whereas richer structural views often dominate cheap flat features in practice\([Dalla et al\., 2023](https://arxiv.org/html/2608.17170#bib.bib3);[Shavit and Hoos, 2024](https://arxiv.org/html/2608.17170#bib.bib22)\)\.

Second, the extractors materialize solver\-aware quantities that flat features miss\. The LLM reads the model and\.dznfiles \(objective, global constraints, parameter roles\) and the generated code computes proxy quantities that solvers implicitly exploit, such as tightness, propagation strength approximating domain\-reduction ratios, and supports per constraint family \(e\.g\.alldifferent,table\), along with distributional summaries \(mean/variance/skew\)\.

Third, the agent adapts feature definitions to the problem family\. For VRP it instantiates domain priors as interpretable features such as demand concentration, depot eccentricity, and route\-length lower bounds \(e\.g\. MST surrogates\), which separate MIP\-friendly from CP\-friendly instances by leveraging the solvers’ strengths\([Moreno\-Scott et al\., 2016](https://arxiv.org/html/2608.17170#bib.bib16)\)\.

### 4\.9Cost Analysis

To quantify the cost of extractor synthesis, we ran the agent without a wall\-clock cap on four LLM backends, two commercial and two open\-weight \(Table[4](https://arxiv.org/html/2608.17170#S4.T4)\)\. As already noted in Section[4\.1](https://arxiv.org/html/2608.17170#S4.SS1), o4\-mini and Claude Sonnet 4 produced a working extractor in all1010of1010trials, while the open\-weight DeepSeek R1 and Llama 3\.3\-8b\-instruct failed in all1010trials because they did not follow the check–fix–verify procedure\. No open\-weight backend we tried sustained the protocol at the time of our experiments; since the framework’sLM:interface runs local backends served via Ollama or LM Studio \(see supplement\), stronger future open\-weight models can be plugged in directly, and we consider a systematic open\-weight evaluation worthwhile future work\.

Table 3:LLM backends evaluated as synthesis agents\.
Table 4:Statistics from3030*LLM2feat*synthesis runs on o4\-mini\.

Table[4](https://arxiv.org/html/2608.17170#S4.T4)aggregates statistics over3030o4\-mini runs on different problems until an extractor is generated: the average single\-extractor synthesis cost is≈210\\approx 210s of wall\-clock time and≈$0\.27\\approx\\$0\.27in API charges, which is negligible relative to running each solver for up to2020minutes per instance\. Once synthesized, an extractor can be reused indefinitely across instances of the same problem family\.

## 5Conclusion and Future Work

We presented an LLM\-based framework that synthesizes graph\-theoretic, interpretable feature extractors from symbolic constraint models \(MiniZinc\) for algorithm selection\. The agentic check–fix–verify loop produces Python scripts that human experts can refine in a “gray\-box” manner, and extractor synthesis is cheap enough to be performed on demand\. On three problem\-specific benchmarks \(VRP, CS, FLECC\) with a five\-solver portfolio, our synthesized extractors yield AS models that outperform both expert\-curated*mzn2feat*and transformer\-based*trans2feat*baselines\. Four concrete limits define the current scope and the follow\-up roadmap: the framework produces one extractor per problem family rather than a universal one; the agent’s loop does not yet incorporate human feedback on the generated code; the input formalism is restricted to MiniZinc; and reliable synthesis currently requires strong commercial LLM backends, as the open\-weight models we evaluated failed the protocol \(Section[4\.9](https://arxiv.org/html/2608.17170#S4.SS9)\)\. Each of these is a concrete target for follow\-up work\.

## Impact Statement

This work aims to improve algorithm selection \(AS\) for constraint optimization by automating the creation of interpretable feature extractors from high\-level MiniZinc models\. The positive impact is reducing the manual feature\-engineering burden that currently limits AS to a small set of well\-studied domains: auditable Python extractors let practitioners build solver portfolios quickly for new problem classes, with applications in scheduling, routing, and resource allocation\.

The LLM\-generated code may carry errors or biases if deployed without validation\. Both are mitigated by the extractors’ interpretability: the code is explicit, testable, and auditable by domain experts\.

#### Acknowledgements\.

The authors acknowledge the support of the European Union’s Horizon 2020 research and innovation programme under the Maria Skłodowska\-Curie grant agreement No\. 101034440††margin:, the support of the Austrian Science Fund \(FWF\), projects 10\.55776/P36688, 10\.55776/P36420, and 10\.55776/COE12, and project PID2022\-138506NB\-C21 from Ministerio de Ciencia e Innovación\.

## References

- Amadini et al\., \(2013\)Amadini, R\., Gabbrielli, M\., and Mauro, J\. \(2013\)\.Features for building CSP portfolio solvers\.CoRR, abs/1308\.0227\.
- Amadini et al\., \(2014\)Amadini, R\., Gabbrielli, M\., and Mauro, J\. \(2014\)\.An enhanced features extractor for a portfolio of constraint solvers\.In Cho, Y\., Shin, S\. Y\., Kim, S\., Hung, C\., and Hong, J\., editors,Symposium on Applied Computing, SAC 2014, Gyeongju, Republic of Korea \- March 24 \- 28, 2014, pages 1357–1359\. ACM\.
- Dalla et al\., \(2023\)Dalla, M\., Provan\-Bessell, B\., Visentin, A\., and O’Sullivan, B\. \(2023\)\.SAT feature analysis for machine learning classification tasks\.InProceedings of the Sixteenth International Symposium on Combinatorial Search, SOCS 2023, pages 138–142\. AAAI Press\.
- Feurer et al\., \(2022\)Feurer, M\., Eggensperger, K\., Falkner, S\., Lindauer, M\., and Hutter, F\. \(2022\)\.Auto\-sklearn 2\.0: Hands\-free AutoML via meta\-learning\.Journal of Machine Learning Research, 23\(261\):1–61\.
- Feurer et al\., \(2015\)Feurer, M\., Klein, A\., Eggensperger, K\., Springenberg, J\. T\., Blum, M\., and Hutter, F\. \(2015\)\.Efficient and robust automated machine learning\.InAdvances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7\-12, 2015, Montreal, Quebec, Canada, pages 2962–2970\.
- Guyon and Elisseeff, \(2003\)Guyon, I\. and Elisseeff, A\. \(2003\)\.An introduction to variable and feature selection\.J\. Mach\. Learn\. Res\., 3:1157–1182\.
- Han et al\., \(2024\)Han, S\., Yoon, J\., Arik, S\. Ö\., and Pfister, T\. \(2024\)\.Large language models can automatically engineer features for few\-shot tabular learning\.InForty\-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21\-27, 2024, volume 235 ofProceedings of Machine Learning Research, pages 17454–17479\. PMLR\.
- Hollmann et al\., \(2023\)Hollmann, N\., Müller, S\., and Hutter, F\. \(2023\)\.Large language models for automated data science: Introducing CAAFE for context\-aware automated feature engineering\.In Oh, A\., Naumann, T\., Globerson, A\., Saenko, K\., Hardt, M\., and Levine, S\., editors,Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023\.
- Hoos et al\., \(2021\)Hoos, H\. H\., Hutter, F\., and Leyton\-Brown, K\. \(2021\)\.Automated configuration and selection of sat solvers\.InHandbook of Satisfiability, pages 481–507\. IOS Press\.
- Kerschke et al\., \(2019\)Kerschke, P\., Hoos, H\. H\., Neumann, F\., and Trautmann, H\. \(2019\)\.Automated algorithm selection: Survey and perspectives\.Evol\. Comput\., 27\(1\):3–45\.
- Kotthoff, \(2013\)Kotthoff, L\. \(2013\)\.LLAMA: leveraging learning to automatically manage algorithms\.Technical Report arXiv:1306\.1031, arXiv\.
- Kotthoff, \(2016\)Kotthoff, L\. \(2016\)\.Algorithm selection for combinatorial search problems: A survey\.InData Mining and Constraint Programming: Foundations of a Cross\-Disciplinary Approach, volume 10101 ofLecture Notes in Computer Science, pages 149–190\. Springer\.
- Lindauer et al\., \(2015\)Lindauer, M\., Hoos, H\. H\., Hutter, F\., and Schaub, T\. \(2015\)\.Autofolio: An automatically configured algorithm selector\.Journal of Artificial Intelligence Research, 53:745–778\.
- Loreggia et al\., \(2016\)Loreggia, A\., Malitsky, Y\., Samulowitz, H\., and Saraswat, V\. A\. \(2016\)\.Deep learning for algorithm portfolios\.In Schuurmans, D\. and Wellman, M\. P\., editors,Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12\-17, 2016, Phoenix, Arizona, USA, pages 1280–1286\. AAAI Press\.
- Marriott et al\., \(2008\)Marriott, K\., Nethercote, N\., Rafeh, R\., Stuckey, P\. J\., de la Banda, M\. G\., and Wallace, M\. \(2008\)\.The design of the zinc modelling language\.Constraints An Int\. J\., 13\(3\):229–267\.
- Moreno\-Scott et al\., \(2016\)Moreno\-Scott, J\. H\., Ortiz\-Bayliss, J\. C\., Terashima\-Marín, H\., and Conant\-Pablos, S\. E\. \(2016\)\.Experimental matching of instances to heuristics for constraint satisfaction problems\.Computational Intelligence and Neuroscience, 2016:7349070\.
- Nethercote et al\., \(2007\)Nethercote, N\., Stuckey, P\. J\., Becket, R\., Brand, S\., Duck, G\. J\., and Tack, G\. \(2007\)\.Minizinc: Towards a standard CP modelling language\.In Bessiere, C\., editor,Principles and Practice of Constraint Programming \- CP 2007, 13th International Conference, CP 2007, Providence, RI, USA, September 23\-27, 2007, Proceedings, volume 4741 ofLecture Notes in Computer Science, pages 529–543\. Springer\.
- Pellegrino et al\., \(2025\)Pellegrino, A\., Akgün, O\., Dang, N\., Kiziltan, Z\., and Miguel, I\. \(2025\)\.Transformer\-Based Feature Learning for Algorithm Selection in Combinatorial Optimisation\.In de la Banda, M\. G\., editor,31st International Conference on Principles and Practice of Constraint Programming \(CP 2025\), volume 340 ofLeibniz International Proceedings in Informatics \(LIPIcs\), pages 31:1–31:22, Dagstuhl, Germany\. Schloss Dagstuhl – Leibniz\-Zentrum für Informatik\.
- Queiroga et al\., \(2022\)Queiroga, E\., Sadykov, R\., Uchoa, E\., and Vidal, T\. \(2022\)\.10,000 optimal CVRP solutions for testing machine learning based heuristics\.InAAAI\-22 Workshop on Machine Learning for Operations Research \(ML4OR\)\.
- Rice, \(1976\)Rice, J\. R\. \(1976\)\.The algorithm selection problem\.In Rubinoff, M\. and Yovits, M\. C\., editors,Advances in Computers, volume 15 ofAdvances in Computers, pages 65–118\. Elsevier\.
- Romera\-Paredes et al\., \(2024\)Romera\-Paredes, B\., Barekatain, M\., Novikov, A\., Balog, M\., Kumar, M\. P\., Dupont, E\., Ruiz, F\. J\. R\., Ellenberg, J\. S\., Wang, P\., Fawzi, O\., Kohli, P\., and Fawzi, A\. \(2024\)\.Mathematical discoveries from program search with large language models\.Nature, 625\(7995\):468–475\.
- Shavit and Hoos, \(2024\)Shavit, H\. and Hoos, H\. H\. \(2024\)\.Revisiting satzilla features in 2024\.In Chakraborty, S\. and Jiang, J\. R\., editors,27th International Conference on Theory and Applications of Satisfiability Testing, SAT 2024, August 21\-24, 2024, Pune, India, volume 305 ofLIPIcs, pages 27:1–27:26\. Schloss Dagstuhl \- Leibniz\-Zentrum für Informatik\.
- Smith\-Miles and Lopes, \(2012\)Smith\-Miles, K\. and Lopes, L\. \(2012\)\.Measuring instance difficulty for combinatorial optimization problems\.Computers & Operations Research, 39\(5\):875–889\.
- Stone et al\., \(2024\)Stone, C\., Renau, Q\., Miguel, I\., and Hart, E\. \(2024\)\.An evaluation of domain\-agnostic representations to enable multi\-task learning in combinatorial optimisation\.InLearning and Intelligent Optimization \- 18th International Conference, LION 18, Ischia Island, Italy, June 9\-13, 2024, Revised Selected Papers, volume 14990 ofLecture Notes in Computer Science, pages 399–414\. Springer\.
- Stuckey et al\., \(2014\)Stuckey, P\. J\., Feydy, T\., Schutt, A\., Tack, G\., and Fischer, J\. \(2014\)\.The minizinc challenge 2008–2013\.AI Magazine, 35\(2\):55–60\.
- Szeider, \(2025\)Szeider, S\. \(2025\)\.Bridging language models and symbolic solvers via the model context protocol\.In Berg, J\. and Nordström, J\., editors,28th International Conference on Theory and Applications of Satisfiability Testing, SAT 2025, August 12\-15, 2025, Glasgow, Scotland, volume 341 ofLIPIcs, pages 30:1–30:12\. Schloss Dagstuhl \- Leibniz\-Zentrum für Informatik\.
- Wu et al\., \(2024\)Wu, X\., Zhong, Y\., Wu, J\., Jiang, B\., and Tan, K\. C\. \(2024\)\.Large language model\-enhanced algorithm selection: Towards comprehensive algorithm representation\.InProceedings of the Thirty\-Third International Joint Conference on Artificial Intelligence, IJCAI 2024, Jeju, South Korea, August 3\-9, 2024, pages 5235–5244\. ijcai\.org\.
- Xia and Szeider, \(2024\)Xia, H\. and Szeider, S\. \(2024\)\.SAT\-Based tree decomposition with iterative cascading policy selection\.In Wooldridge, M\. J\., Dy, J\. G\., and Natarajan, S\., editors,Thirty\-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty\-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2024, February 20\-27, 2024, Vancouver, Canada, pages 8191–8199\. AAAI Press\.
- Yao et al\., \(2023\)Yao, S\., Zhao, J\., Yu, D\., Du, N\., Shafran, I\., Narasimhan, K\. R\., and Cao, Y\. \(2023\)\.ReAct: Synergizing reasoning and acting in language models\.InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1\-5, 2023\. OpenReview\.net\.
- Zhang et al\., \(2024\)Zhang, Z\., Chételat, D\., Cotnareanu, J\., Ghose, A\., Xiao, W\., Zhen, H\., Zhang, Y\., Hao, J\., Coates, M\., and Yuan, M\. \(2024\)\.Grass: Combining graph neural networks with expert knowledge for SAT solver selection\.In Baeza\-Yates, R\. and Bonchi, F\., editors,Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25\-29, 2024, pages 6301–6311\. ACM\.

## Appendix AHyperparameters

Settings shared by Random Forest training and the LLAMA random\-forest classification mode are in Table[5](https://arxiv.org/html/2608.17170#A1.T5)\. AutoSklearn settings are in Table[6](https://arxiv.org/html/2608.17170#A1.T6)\.

Table 5:Random Forest hyperparameters\.Table 6:AutoSklearn standard configuration\.
## Appendix BAdditional Feature\-Correlation Heatmaps

Figures[5](https://arxiv.org/html/2608.17170#A2.F5)–[6](https://arxiv.org/html/2608.17170#A2.F6)show the correlation matrices for the FLECC and CS problems\.

![Refer to caption](https://arxiv.org/html/2608.17170v1/FLECC_random_forest_correlation_matrix.png)Figure 5:Feature correlation matrices for FLECC\.*LLM2feat*features have average absolute correlation\|r\|=0\.306\|r\|=0\.306vs\.\|r\|=0\.330\|r\|=0\.330for*mzn2feat*\.![Refer to caption](https://arxiv.org/html/2608.17170v1/car_sequencing_random_forest_correlation_matrix.png)Figure 6:Feature correlation matrices for CS\.*LLM2feat*features have\|r\|=0\.551\|r\|=0\.551vs\.\|r\|=0\.725\|r\|=0\.725for*mzn2feat*, a24%24\\%improvement in diversity\.
## Appendix CLLM Model Selection and Sensitivity

We evaluated several LLMs as the agent backend\. Strong agentic models \(o4\-mini, Claude Sonnet\) reliably followed the check–fix–verify protocol; smaller open\-source models \(Llama 3\.3, DeepSeek R1\) failed under our prompts \(see Section[4\.9](https://arxiv.org/html/2608.17170#S4.SS9)\)\. For the problem\-specific framework we used OpenAI o4\-mini\-2025\-04\-16 and ran1010independent syntheses per problem\. Tables[10](https://arxiv.org/html/2608.17170#A3.T10)–[15](https://arxiv.org/html/2608.17170#A3.T15)list the three highest\-scoring runs per problem and toolchain alongside*mzn2feat*baselines; full per\-variant tables \(10 LLM2feat runs vs\. 20*trans2feat*variants\) are in Appendix[D](https://arxiv.org/html/2608.17170#A4)\. Across the top\-three runs shown here, test accuracy fluctuates within roughly33pp and every shown LLM2feat run outperforms the corresponding*mzn2feat*baseline\. The full1010\-run distribution is wider on CS/AutoSK in particular, where two syntheses produced under\-fitting extractors \(test accuracy≈0\.528\\approx 0\.528, with correspondingly low training accuracy\); the formal significance analysis on the full distributions appears in Section[4\.7](https://arxiv.org/html/2608.17170#S4.SS7)\.

#### Training\-set and ranking\-loss results

Tables[7](https://arxiv.org/html/2608.17170#A3.T7)–[9](https://arxiv.org/html/2608.17170#A3.T9)report the results omitted from the main text: training\-set accuracy underA​c​cAccloss; test\-set and training\-set accuracy/ranking underR​a​n​kRankloss\.

Table 7:Training\-set accuracy/ranking withA​c​cAccas loss \(problem\-specific toolchains\)\.Table 8:Test\-set accuracy/ranking withR​a​n​kRankas loss\.Table 9:Training\-set accuracy/ranking withR​a​n​kRankas loss\.
#### Per\-extractor sensitivity tables

Table 10:*LLM2feat*\+RF performance for FLECC\.Table 11:*LLM2feat*\+RF performance for CS\.Table 12:*LLM2feat*\+RF performance for VRP\.Table 13:*LLM2feat*\+AutoSK performance for FLECC\.Table 14:*LLM2feat*\+AutoSK performance for CS\.Table 15:*LLM2feat*\+AutoSK performance for VRP\.

## Appendix DDetailed Transformer Comparison

We obtain2020*trans2feat*feature sets from the official repository333[https://github\.com/SeppiaBrilla/EFE\_project/tree/master/data/features](https://github.com/SeppiaBrilla/EFE_project/tree/master/data/features)\(one per neural\-network variant in[Pellegrino et al\., 2025](https://arxiv.org/html/2608.17170#bib.bib18)\) and train both RF and AutoSK toolchains on each\. Tables[16](https://arxiv.org/html/2608.17170#A4.T16)–[19](https://arxiv.org/html/2608.17170#A4.T19)give the full per\-variant results for CS and FLECC\. “TO” marks runs that exceeded the AutoSklearn18001800s training budget\([Feurer et al\., 2022](https://arxiv.org/html/2608.17170#bib.bib4)\)without producing a model; “F” marks runs whose AutoSklearn fit collapsed to a degenerate constant\-class model \(test accuracy<1%\{<\}1\\%\)\. Both are excluded from all aggregate statistics and significance tests; the raw result files for the F runs ship with the supplement\.

Table 16:Car Sequencing: Random Forest with accuracy loss\.Table 17:FLECC: Random Forest with accuracy loss\.Table 18:Car Sequencing: AutoSklearn with accuracy loss\. F==degenerate failed fit \(test accuracy<1%\{<\}1\\%\), excluded from all aggregates\.Table 19:FLECC: AutoSklearn with accuracy loss\. TO==18001800s timeout without a model; F==degenerate failed fit\.