FunL2O: LLM-Guided Feature Function Design for Learning to Optimize

# FunL2O: LLM-Guided Feature Function Design for Learning to Optimize
Source: [https://arxiv.org/html/2607.27389](https://arxiv.org/html/2607.27389)
###### Abstract

Learning\-to\-optimize \(L2O\) methods accelerate repeated optimization by training models to predict solutions, warm starts, branching decisions, or other forms of solver guidance\. A critical yet largely overlooked component of these pipelines is the feature function that maps problem instances to inputs for machine learning models\. Existing L2O methods typically rely on hand\-crafted features, making representation design manual and largely fixed across domains\. We introduce FunL2O, thefirstunified framework for automating feature design through LLM\-driven program evolution for L2O\. In a FunSearch\-style loop, an LLM proposes executable feature functions, while a fixed evaluation process retrains the original L2O model and measures downstream optimization performance\. We evaluate FunL2O on linear and quadratic programming tasks involving solution prediction and warm\-starting, as well as on mixed\-integer optimization tasks using GNN\-guided backdoor branching and Predict\-and\-Search\. Across continuous and discrete optimization tasks and four LLMs, the evolved featuresconsistently outperformhand\-crafted representations\. These results establish LLM\-driven feature evolution as a general and effective approach to automating representation design in L2O\.

## Introduction

Many optimization problems are solved repeatedly with the same mathematical structure but different numerical data\. Learning\-to\-optimize \(L2O\) methods exploit this regularity by training neural networks on previously solved instances\(Andrychowiczet al\.[2016](https://arxiv.org/html/2607.27389#bib.bib2); Chenet al\.[2022](https://arxiv.org/html/2607.27389#bib.bib1)\)\. Depending on the pipeline, the learned component may predict a solution, initialize an iterative algorithm, select a simplex basis or pivot, or guide mixed\-integer search\(Qianet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib22); Liet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib23); Fanet al\.[2023](https://arxiv.org/html/2607.27389#bib.bib25); Liuet al\.[2024b](https://arxiv.org/html/2607.27389#bib.bib26); Hanet al\.[2023](https://arxiv.org/html/2607.27389#bib.bib12); Caiet al\.[2025a](https://arxiv.org/html/2607.27389#bib.bib14)\)\. Despite these varied roles, every L2O method begins with the same fundamental design decision: how to represent an optimization instance using features that enable a learning model to train more effectively and capture the relevant problem structure\.

The feature function determines what information is directly available to the model and how that information is structured\. For linear and mixed\-integer programs, the representation may encode objective coefficients, variable bounds and types, graph structure, and properties of the LP relaxation\. When a useful relationship is made explicit, the model can exploit it directly; otherwise, it must infer that relationship from the available inputs\. Consequently, changing only the input representation can affect both the model’s expressive power and its downstream performance\(Sunet al\.[2023](https://arxiv.org/html/2607.27389#bib.bib29); Chenet al\.[2025](https://arxiv.org/html/2607.27389#bib.bib30); Caiet al\.[2025c](https://arxiv.org/html/2607.27389#bib.bib13)\)\.

Despite its importance, the feature function is typically hand\-designed and fixed across problem domains before training\. Most L2O research then focuses on the model architecture, learning objective, or solver interface while leaving the input representation unchanged\. Automating feature design, however, is not simply a matter of feature selection\. An L2O feature function is an executable program over structured optimization data: it must preserve a model\-specific tensor interface, satisfy semantic and dimensional invariants, and rely only on information available at deployment time\. Moreover, its quality can be evaluated only after retraining the learner and measuring downstream performance\. The relevant metric is role\-dependent: prediction methods prioritize feasibility and objective quality, whereas solver policies seek to reduce iterations, pivots, search effort, or wall\-clock time\.

To address this problem, we introduceFunL2O, the first LLM\-guided framework for automatically designing executable feature functions for L2O\. The framework defines clear rules for what information a feature function may use at deployment, what output format it must produce, and what conditions it must satisfy\. Each valid candidate replaces the original fixed hand\-crafted feature function in the L2O pipeline\. The model is then retrained, and its downstream optimization performance is used to guide the search toward better feature functions\. All other parts of the pipeline—including the model architecture, loss function, labels, training schedule, data, and solver—remain unchanged\. This allows us to isolate the effect of the input representation\. After the best feature function is selected, it is used as standard preprocessing code and does not require an LLM at deployment\.

![Refer to caption](https://arxiv.org/html/2607.27389v1/1.png)Figure 1:FunL2Osearches over executable feature functions\. Each proposal must satisfy a pipeline\-specific semantic contract, after which the original L2O model is retrained and evaluated\. Measured optimization outcomes determine the elite programs supplied to the next generation\.Our contributions are threefold\. First, we formulate feature\-function design as a program\-search problem within the original L2O pipeline\. We also introduce semantic contracts that define valid inputs and outputs for heterogeneous graphs, dense tensors, and solver states\. Second, we develop an L2O\-in\-the\-loop search process in which executable feature functions are selected based on measured feasibility and downstream optimization performance, rather than on LLM judgments\. Third, we evaluate the framework on eight L2O pipelines covering LP and QP solution prediction, constrained nonlinear prediction, primal–dual warm starting, simplex basis initialization, pivot selection, and MILP search\. Across these settings, the evolved features consistently outperform hand\-crafted representations, improving solution quality, feasibility, and solver efficiency without changing the learned model or the optimization pipeline\.

## Background

#### Learning to optimize\.

Learning\-to\-optimize \(L2O\) trains a model on related problem instances to predict a solution or guide an optimization algorithm\(Andrychowiczet al\.[2016](https://arxiv.org/html/2607.27389#bib.bib2); Chenet al\.[2022](https://arxiv.org/html/2607.27389#bib.bib1)\)\. Given an instancexx, an L2O pipeline can be written as

x→ϕX→𝑀a→𝒮y^,x\\xrightarrow\{\\phi\}X\\xrightarrow\{M\}a\\xrightarrow\{\\mathcal\{S\}\}\\widehat\{y\},\(1\)whereϕ\\phiis the feature function,MMis the learned model,aais a predicted solution or solver directive, and𝒮\\mathcal\{S\}is the downstream solver\. Depending on the learned role, the outcomey^\\widehat\{y\}is evaluated by feasibility, objective quality, iterations, pivots, or wall time\.

#### Feature functions\.

The feature function transforms structured optimization data into tensors consumed by the model\. For an LP or MILP,

min⁡\{c⊤​x:A​x≤b,ℓ≤x≤u,xj∈ℤ​\(j∈ℐ\)\},\\min\\left\\\{c^\{\\top\}x:Ax\\leq b,\\;\\ell\\leq x\\leq u,\\;x\_\{j\}\\in\\mathbb\{Z\}\\ \(j\\in\\mathcal\{I\}\)\\right\\\},\(2\)a common representation is a bipartite graph with variable and constraint nodes and an edge for each nonzero coefficient ofAA\(Gasseet al\.[2019](https://arxiv.org/html/2607.27389#bib.bib28)\)\. Features may encode objective coefficients, bounds, variable types, constraint statistics, and LP\-relaxation information\. These choices determine which numerical and structural relations are directly available to the learner\.

#### Learned solver guidance\.

Some L2O models predict solutions directly, while others guide a classical solver through warm starts, simplex bases, pivot scores, partial assignments, or branching priorities\. For solver\-facing roles, prediction accuracy is only an intermediate metric: the primary objective is to reduce the work performed by the solver\. Accordingly,FunL2Oevaluates each feature function using the native downstream metric of its host pipeline\.

#### Mixed\-integer Prediction\.

We consider two MILP\-related prediction tasks\. Predict\-and\-Search predicts binary\-variable marginals and searches a trust region around the resulting partial assignment\(Hanet al\.[2023](https://arxiv.org/html/2607.27389#bib.bib12); Huanget al\.[2024](https://arxiv.org/html/2607.27389#bib.bib11)\)\. Learned backdoors rank important variables and assign higher branching priorities to the top predictions\(Caiet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib9)\)\. In both cases,FunL2Ochanges only the feature function, leaving the model, training procedure, and solver fixed\.

## Methodology

FunL2Otakes an existing L2O pipeline𝒫\\mathcal\{P\}and its handcrafted feature functionϕ0\\phi\_\{0\}, and returns an evolved feature function as executable code\. The search changes only the representation: the model architecture, training procedure, hyperparameters, data, learned role, and downstream solver remain fixed\. As shown in Figure[1](https://arxiv.org/html/2607.27389#Sx1.F1), an LLM proposes candidate functions, a semantic contract checks their validity, and the original pipeline evaluates valid candidates by retraining the model and measuring the native optimization outcome\. The highest\-ranked programs guide subsequent proposals\.

### Feature\-Function Search

Consider an L2O pipeline𝒫\\mathcal\{P\}with a learned modelMθM\_\{\\theta\}, a training procedure, and an optional downstream solver operationS𝒫S\_\{\\mathcal\{P\}\}\. Letx∈𝒳𝒫x\\in\\mathcal\{X\}\_\{\\mathcal\{P\}\}denote an optimization instance or a solver state\. A feature function

ϕ:𝒳𝒫→𝒵𝒫​\(dϕ\)\\phi:\\mathcal\{X\}\_\{\\mathcal\{P\}\}\\rightarrow\\mathcal\{Z\}\_\{\\mathcal\{P\}\}\(d\_\{\\phi\}\)\(3\)mapsxxto a representation of widthdϕd\_\{\\phi\}\. The model uses this representation to produce a learned output:

zϕ​\(x\)=ϕ​\(x\),aϕ​\(x\)=Mθϕ⋆\(dϕ\)​\(zϕ​\(x\)\)\.\\begin\{split\}z\_\{\\phi\}\(x\)&=\\phi\(x\),\\\\ a\_\{\\phi\}\(x\)&=M\_\{\\theta\_\{\\phi\}^\{\\star\}\}^\{\(d\_\{\\phi\}\)\}\\bigl\(z\_\{\\phi\}\(x\)\\bigr\)\.\\end\{split\}\(4\)The learned output is then passed to the remaining optimization pipeline:

y^ϕ​\(x\)=S𝒫​\(x,aϕ​\(x\)\)\.\\widehat\{y\}\_\{\\phi\}\(x\)=S\_\{\\mathcal\{P\}\}\\bigl\(x,a\_\{\\phi\}\(x\)\\bigr\)\.\(5\)The outputaϕ​\(x\)a\_\{\\phi\}\(x\)may be a predicted solution, a primal–dual initialization, a basis\-status prediction, or a pivot score\. When the model directly predicts the final solution,S𝒫S\_\{\\mathcal\{P\}\}is the identity map\.

Existing L2O methods provide a handcrafted feature functionϕ0\\phi\_\{0\}\. FunL2O keeps the rest of𝒫\\mathcal\{P\}fixed and searches for a replacementϕ\\phi\. A model trained with one representation cannot be reused to evaluate another representation\. Therefore, every candidate induces a new training problem:

θϕ⋆=Train𝒫⁡\(ϕ;𝒟tr\)\.\\theta\_\{\\phi\}^\{\\star\}=\\operatorname\{Train\}\_\{\\mathcal\{P\}\}\\left\(\\phi;\\mathcal\{D\}\_\{\\mathrm\{tr\}\}\\right\)\.\(6\)Ifdϕd\_\{\\phi\}differs from the width of the handcrafted representation, the input layer is instantiated at the corresponding width\. This is a mechanical interface change: the model template, hidden architecture, loss, optimizer, training schedule, and other hyperparameters are not searched\.

After training, the candidate is evaluated on the validation set\. We write

𝐦𝒫​\(ϕ\)=Aggx∈𝒟val⁡μ𝒫​\(x,y^ϕ​\(x\)\)\\mathbf\{m\}\_\{\\mathcal\{P\}\}\(\\phi\)=\\operatorname\{Agg\}\_\{x\\in\\mathcal\{D\}\_\{\\mathrm\{val\}\}\}\\mu\_\{\\mathcal\{P\}\}\\left\(x,\\widehat\{y\}\_\{\\phi\}\(x\)\\right\)\(7\)for its measured outcome, whereμ𝒫\\mu\_\{\\mathcal\{P\}\}is defined by the original pipeline\. Depending on the learned role,𝐦𝒫\\mathbf\{m\}\_\{\\mathcal\{P\}\}may contain feasibility rate, constraint violation, objective gap, iteration count, running time, or pivot count\.

These outcomes are compared through a pipeline\-specific ranking key

ρ𝒫​\(ϕ\)=\(b𝒫​\(ϕ\),v𝒫​\(ϕ\),q𝒫​\(ϕ\)\),\\rho\_\{\\mathcal\{P\}\}\(\\phi\)=\\left\(b\_\{\\mathcal\{P\}\}\(\\phi\),v\_\{\\mathcal\{P\}\}\(\\phi\),q\_\{\\mathcal\{P\}\}\(\\phi\)\\right\),\(8\)ordered lexicographically with lower values preferred\. The first componentb𝒫b\_\{\\mathcal\{P\}\}indicates whether the required feasibility condition is violated\. The second componentv𝒫v\_\{\\mathcal\{P\}\}measures the degree of infeasibility, and the third componentq𝒫q\_\{\\mathcal\{P\}\}measures objective quality or solver work\. For tasks without a feasibility condition, the first two components are set to zero\. The sign ofq𝒫q\_\{\\mathcal\{P\}\}is chosen so that lower is always better; for example, it may be objective gap, iteration count, pivot count, or the negative of iteration saving\.

The feature\-design problem is therefore

ϕ⋆∈arg​minϕ∈Φ𝒫⁡ρ𝒫​\(ϕ\)\.\\phi^\{\\star\}\\in\\operatorname\*\{arg\\,min\}\_\{\\phi\\in\\Phi\_\{\\mathcal\{P\}\}\}\\rho\_\{\\mathcal\{P\}\}\(\\phi\)\.\(9\)The minimization in Equation[9](https://arxiv.org/html/2607.27389#Sx3.E9)follows the lexicographic order\. It prevents an infeasible prediction from being preferred only because it has a better objective value\. It also allows the same outer problem to represent prediction quality, warm\-start effectiveness, and learned solver decisions\.

Equation[9](https://arxiv.org/html/2607.27389#Sx3.E9)presents two difficulties\. First, the admissible spaceΦ𝒫\\Phi\_\{\\mathcal\{P\}\}differs across pipelines because their inputs and model interfaces differ\. Second, the outer objective is an expensive black box because evaluating one candidate requires model training and may also require a complete solver run\. FunL2O addresses the first difficulty through semantic feature contracts and the second through a feedback\-driven search over measured outcomes\.

### Semantic Feature Contracts

A semantic feature contract specifies which feature functions belong toΦ𝒫\\Phi\_\{\\mathcal\{P\}\}\. For pipeline𝒫\\mathcal\{P\}, we define

𝒞𝒫=\(𝒰𝒫,𝒵𝒫,ℐ𝒫,V𝒫\)\.\\mathcal\{C\}\_\{\\mathcal\{P\}\}=\\left\(\\mathcal\{U\}\_\{\\mathcal\{P\}\},\\mathcal\{Z\}\_\{\\mathcal\{P\}\},\\mathcal\{I\}\_\{\\mathcal\{P\}\},V\_\{\\mathcal\{P\}\}\\right\)\.\(10\)Here,𝒰𝒫\\mathcal\{U\}\_\{\\mathcal\{P\}\}contains the instance fields or solver\-state information available at deployment\. The output specification𝒵𝒫\\mathcal\{Z\}\_\{\\mathcal\{P\}\}defines the required return type, orientation, and maximum feature width\. The invariantsℐ𝒫\\mathcal\{I\}\_\{\\mathcal\{P\}\}describe properties that must be preserved, such as determinism, finite outputs, and required handcrafted channels\. Finally,V𝒫V\_\{\\mathcal\{P\}\}is an executable validator for these requirements\.

Table 1:Instantiation ofFunL2Oacross the host L2O pipelines\. In every case, the feature function is searched while the learned output, training procedure, and downstream solver interface remain fixed\.The contract defines the intended admissible space

Φ𝒫=\{ϕ∈ℋsafe\|Fields⁡\(ϕ\)⊆𝒰𝒫,ϕ​\(x\)∈𝒵𝒫​\(dϕ\),ℐ𝒫​\(ϕ,x\)=1\}\.\\Phi\_\{\\mathcal\{P\}\}=\\left\\\{\\phi\\in\\mathcal\{H\}\_\{\\mathrm\{safe\}\}\\;\\middle\|\\;\\begin\{subarray\}\{c\}\\operatorname\{Fields\}\(\\phi\)\\subseteq\\mathcal\{U\}\_\{\\mathcal\{P\}\},\\\\ \\phi\(x\)\\in\\mathcal\{Z\}\_\{\\mathcal\{P\}\}\(d\_\{\\phi\}\),\\\\ \\mathcal\{I\}\_\{\\mathcal\{P\}\}\(\\phi,x\)=1\\end\{subarray\}\\right\\\}\.\(11\)Here,ℋsafe\\mathcal\{H\}\_\{\\mathrm\{safe\}\}denotes executable functions allowed by the restricted environment\. A candidate may combine information already available to the original method, but it cannot access reference solutions, training labels, files, networks, or external solvers\.

The output specification remains pipeline\-specific\. A graph model may require separate variable and constraint features, whereas a constrained predictor may expect a dense tensor and a pivot model may expect one feature vector per solver\-state node\. This allows FunL2O to share one search procedure without forcing different L2O methods into a common representation\. Table[1](https://arxiv.org/html/2607.27389#Sx3.T1)summarizes how the semantic contract and native evaluation criterion are instantiated across the host pipelines\. Although the feature interface and learned role vary, the proposal, validation, retraining, and selection procedure remains unchanged\.

When the original representation must be retained, the contract requires the candidate to preserve its channels\. In the common append\-only case, the candidate has the form

ϕ​\(x\)=Concat⁡\(ϕ0​\(x\),ψ​\(x\)\),\\phi\(x\)=\\operatorname\{Concat\}\\left\(\\phi\_\{0\}\(x\),\\psi\(x\)\\right\),\(12\)whereψ\\psicontains the newly proposed features\. For structured outputs, the same requirement is applied separately to the relevant variable, constraint, edge, or global channels\.

In implementation,V𝒫V\_\{\\mathcal\{P\}\}first applies a static guard to the candidate source and then executes it on a small probe instance\. The validator checks the function signature, output structure, feature dimensions, numerical finiteness, and required seed channels\. A failed check returns the violated condition to the proposal model\. PassingV𝒫V\_\{\\mathcal\{P\}\}establishes interface validity only; whether a function is useful is determined by retraining and evaluating it according to Equations[6](https://arxiv.org/html/2607.27389#Sx3.E6)–[8](https://arxiv.org/html/2607.27389#Sx3.E8)\.

### Standardized Prompt Construction

The semantic contract determines whether a feature function can be used by the original pipeline\. To propose a useful function, the LLM must also know how candidates are evaluated and which relations have already been explored\. FunL2O organizes this information through one prompt template shared by all pipelines\.

At generationgg, the prompt is constructed as

pg=𝒯​\(ℛ,𝒞𝒫,𝒢,ℰg,ℱg\)\.p\_\{g\}=\\mathcal\{T\}\\left\(\\mathcal\{R\},\\mathcal\{C\}\_\{\\mathcal\{P\}\},\\mathcal\{G\},\\mathcal\{E\}\_\{g\},\\mathcal\{F\}\_\{g\}\\right\)\.\(13\)The shared instructionsℛ\\mathcal\{R\}state that only the feature function may change and that the returned function must be deterministic and executable\. The contract𝒞𝒫\\mathcal\{C\}\_\{\\mathcal\{P\}\}supplies the available inputs, function signature, output structure, and pipeline\-native evaluation rule\.

The shared vocabulary𝒢\\mathcal\{G\}describes broad relations that may be useful in optimization representations, including raw problem quantities, scale, residual, curvature, bound, cone, and graph statistics\. These groups guide the proposal model but do not define a fixed library of allowed features\. The generation\-specific contextℰg\\mathcal\{E\}\_\{g\}contains the source code, feature width, and validation outcome of the current elite functions\. The feedbackℱg\\mathcal\{F\}\_\{g\}contains a contract error or a summary of whether the preceding generation improved the current best\.

Given this prompt, the proposal modelQωQ\_\{\\omega\}generates a population of candidate feature functions:

Φ~g∼Qω\(⋅∣pg\)\.\\widetilde\{\\Phi\}\_\{g\}\\sim Q\_\{\\omega\}\\left\(\\,\\cdot\\mid p\_\{g\}\\right\)\.\(14\)The model is asked to identify a useful relation that the current representation may not expose and to implement it as one feature function\. It receives aggregate validation outcomes and candidate source code, but it does not receive validation instances, target solutions, or test results\. The LLM therefore proposes feature computations; it does not predict their performance\.

### L2O\-in\-the\-Loop Search

The search begins by evaluating the handcrafted functionϕ0\\phi\_\{0\}\. Its source code, feature width, measured outcome, and ranking key initialize the search memory:

ℳ0=\{\(ϕ0,dϕ0,𝐦𝒫​\(ϕ0\),ρ𝒫​\(ϕ0\)\)\}\.\\mathcal\{M\}\_\{0\}=\\left\\\{\\left\(\\phi\_\{0\},d\_\{\\phi\_\{0\}\},\\mathbf\{m\}\_\{\\mathcal\{P\}\}\(\\phi\_\{0\}\),\\rho\_\{\\mathcal\{P\}\}\(\\phi\_\{0\}\)\\right\)\\right\\\}\.\(15\)The handcrafted function therefore serves both as the performance baseline and as the first valid example of the semantic contract\.

Table 2:Results across continuous\-optimization L2O pipelines\. Prediction gaps and solver costs are lower\-is\-better; saving metrics and accuracy are higher\-is\-better\. Prediction results show clean\-test medians over four paired searches\. Feasibility is reported as handcrafted toFunL2O\. Positive IR always denotes improvement\.HostTask or metricHandcraftedFunL2OImprovementFeasibility \(H→\\toF\)\\rowcolorpanelcolorPanel A: LP solution predictionIPM\-MPNNIndependent set objective gap0\.003650\.0030616\.4%0\.914→\\to0\.920Set cover objective gap0\.012070\.0061149\.4%0\.957→\\to0\.942Combinatorial auction objective gap0\.022170\.0166025\.1%0\.874→\\to0\.883Facility location objective gap0\.023030\.0181321\.3%0\.988→\\to0\.941GNNLP objective gap0\.004990\.0038323\.2%0\.979→\\to0\.979\\rowcolorpanelcolorPanel B: Constrained solution predictionFSNetConvex QP gap0\.6850\.36347\.0%0\.938→\\to1\.000Convex QCQP gap14\.098\.6138\.9%1\.000→\\to1\.000Convex SOCP gap42\.3322\.5246\.8%1\.000→\\to1\.000Nonconvex QP gap5\.465\.460\.0%1\.000→\\to1\.000Nonconvex QCQP gap26\.4219\.2627\.1%1\.000→\\to1\.000Nonconvex SOCP gap933\.85141\.2884\.9%0\.250→\\to1\.000Nonsmooth nonconvex QP gap5\.205\.200\.0%1\.000→\\to1\.000Nonsmooth nonconvex QCQP gap22\.6922\.690\.0%1\.000→\\to1\.000Nonsmooth nonconvex SOCP gap332\.33159\.3852\.0%0\.750→\\to1\.000DC3Convex QP gap13\.0112\.652\.8%0\.986→\\to0\.986Nonconvex QP gap15\.6215\.312\.0%0\.848→\\to0\.849Nonsmooth nonconvex QP gap15\.9615\.284\.3%0\.855→\\to0\.861\\rowcolorpanelcolorPanel C: Solver\-facing learned rolesPDHG\-NetIteration saving↑\\uparrow42\.44%51\.05%20\.3%–Time saving↑\\uparrow34\.03%39\.28%15\.4%–Smart Initial BasisIteration saving↑\\uparrow59\.28%60\.24%1\.62%–Learning to PivotPivot\-count geometric mean↓\\downarrow1162\.86857\.7826\.2%–Top\-1 accuracy↑\\uparrow0\.52640\.53130\.9%–
At generationgg, FunL2O constructspgp\_\{g\}from the contract and the current memory, and samples the proposed populationΦ~g\\widetilde\{\\Phi\}\_\{g\}\. Candidates that pass the validator form

Φg=\{ϕ∈Φ~g:V𝒫​\(ϕ\)=1\}\.\\Phi\_\{g\}=\\left\\\{\\phi\\in\\widetilde\{\\Phi\}\_\{g\}:V\_\{\\mathcal\{P\}\}\(\\phi\)=1\\right\\\}\.\(16\)
If validation fails, the violated condition is appended to the prompt and the proposal model is asked to repair the function\. A function is submitted for training only after it passes the contract\.

Everyϕ∈Φg\\phi\\in\\Phi\_\{g\}replacesϕ0\\phi\_\{0\}in a fresh copy of the original pipeline\. The corresponding model is trained using the original data, loss, optimizer, and schedule, and its outcome is measured byμ𝒫\\mu\_\{\\mathcal\{P\}\}\. Candidates within a generation are independent given the current memory and can therefore be trained and evaluated in parallel\.

After evaluation, the search memory is updated as

ℳg\+1=ℳg∪\{\(ϕ,dϕ,𝐦𝒫​\(ϕ\),ρ𝒫\(ϕ\)\):ϕ∈Φg\}\.\\begin\{split\}\\mathcal\{M\}\_\{g\+1\}=\\mathcal\{M\}\_\{g\}\\cup\\bigl\\\{\(&\\phi,d\_\{\\phi\},\\mathbf\{m\}\_\{\\mathcal\{P\}\}\(\\phi\),\\\\ &\\rho\_\{\\mathcal\{P\}\}\(\\phi\)\):\\phi\\in\\Phi\_\{g\}\\bigr\\\}\.\\end\{split\}\(17\)
The elite functions supplied to the next prompt are selected by

ℰg\+1=TopKρ𝒫⁡\(ℳg\+1\),\\mathcal\{E\}\_\{g\+1\}=\\operatorname\{TopK\}\_\{\\rho\_\{\\mathcal\{P\}\}\}\\left\(\\mathcal\{M\}\_\{g\+1\}\\right\),\(18\)
whereTopKρ𝒫\\operatorname\{TopK\}\_\{\\rho\_\{\\mathcal\{P\}\}\}returns theKKentries with the lowest pipeline\-specific ranking keys\. The same proposal, validation, evaluation, and memory update are used for every L2O pipeline\. Only the contract𝒞𝒫\\mathcal\{C\}\_\{\\mathcal\{P\}\}, outcome measureμ𝒫\\mu\_\{\\mathcal\{P\}\}, and ranking keyρ𝒫\\rho\_\{\\mathcal\{P\}\}change\. After the final generation, FunL2O returns the lowest\-ranked valid feature function in the memory\. The selected function is ordinary executable preprocessing code\. Neither the proposal model nor the search memory is required during deployment\.

## Experiments

### Experimental Setup

Table[1](https://arxiv.org/html/2607.27389#Sx3.T1)summarizes the eight host pipelines\. The continuous\-optimization study includes IPM\-MPNN for LP solution prediction\(Qianet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib22)\); FSNet and DC3 for constrained prediction\(Dontiet al\.[2021](https://arxiv.org/html/2607.27389#bib.bib24); Nguyen and Donti[2026](https://arxiv.org/html/2607.27389#bib.bib27)\); PDHG\-Net for PageRank LP warm starts\(Liet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib23)\); and Smart Initial Basis and Learning to Pivot for set\-cover simplex decisions\(Fanet al\.[2023](https://arxiv.org/html/2607.27389#bib.bib25); Liuet al\.[2024b](https://arxiv.org/html/2607.27389#bib.bib26)\)\. The MILP study includes Predict\-and\-Search and learned backdoor branching on CA, MIS, and MVC instances\(Hanet al\.[2023](https://arxiv.org/html/2607.27389#bib.bib12); Caiet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib9)\)\. In every experiment,FunL2Omodifies only the feature function\.

For all pipelines, candidate feature functions are selected on validation instances and evaluated on a disjoint test set, with no overlap among the training, validation, and test instances\. Each search consists of eight iterations with six proposal slots\. We repeat every experiment with three different random seeds and report the mean\. For the continuous\-optimization experiments, we runFunL2Owith four frontier LLMs: Claude\-Opus\-4\.8\(Anthropic[2026](https://arxiv.org/html/2607.27389#bib.bib3)\), GPT\-5\.5\(OpenAI[2026](https://arxiv.org/html/2607.27389#bib.bib5)\), Gemini\-3\.1\-Pro Preview\(Google DeepMind[2026](https://arxiv.org/html/2607.27389#bib.bib6)\), and DeepSeek\-Math\-V2\(Shaoet al\.[2025](https://arxiv.org/html/2607.27389#bib.bib7)\)\. For the MILP experiments, we use Claude\-Opus\-4\.8 and GPT\-5\.5\. Predict\-and\-Search runs CPLEX\(Manual[1987](https://arxiv.org/html/2607.27389#bib.bib4)\)with a 1000\-second time limit, whereas the backdoor\-branching experiments use a 600\-second CPLEX time limit\.

Table 3:Results for mixed\-integer search\. For Predict\-and\-Search, final primal gap and primal integral are lower\-is\-better\. For learned backdoors, mean wall time is lower\-is\-better and win rate is higher\-is\-better\.Boldindicates the better evolved feature function in each row\.#### Metrics\.

Solution\-prediction hosts retain their original objective gap and feasibility criteria\. IPM\-MPNN reports the relative objective gap, while FSNet and DC3 report the gap in percent and measure feasibility using a constraint\-violation threshold of10−410^\{\-4\}\.

For solver\-facing pipelines, iteration saving is100​C−HC,100\\frac\{C\-H\}\{C\},whereCCandHHare the cold\-start and learned\-start iteration counts; time saving is defined analogously\. Learning to Pivot reports the geometric mean pivot count\. For a lower\-is\-better metric with handcrafted resultBBandFunL2OresultFF, the improvement rate isIR=100​B−F\|B\|\.\\operatorname\{IR\}=100\\frac\{B\-F\}\{\|B\|\}\.For higher\-is\-better metrics, the numerator isF−BF\-B, so positive IR always denotes improvement\.

For P&S, final primal gap and primal integral are lower\-is\-better, with the primal integral additionally rewarding methods that find strong incumbents earlier\. Learned backdoors are evaluated by mean wall time\. We also report the percentage of instances on which each method is the fastest\.

### Results Across L2O Pipelines

#### Continuous optimization\.

Table[L2O\-in\-the\-Loop Search](https://arxiv.org/html/2607.27389#Sx3.SSx4)summarizes all continuous\-optimization results\. On IPM\-MPNN,FunL2Oreduces the median objective gap on all five LP distributions\. The reductions range from16\.4%16\.4\\%on independent set to49\.4%49\.4\\%on set cover, demonstrating consistent gains across structurally different LP families\.

The improvements extend to self\-supervised constrained prediction\. On the nine 1000\-variable FSNet tasks,FunL2Oimproves six displayed medians and matches the remaining three\. The largest gains occur on SOCP\-constrained problems, where the gap decreases by46\.8%46\.8\\%,84\.9%84\.9\\%, and52\.0%52\.0\\%across the convex, nonconvex, and nonsmooth nonconvex objectives\. On nonconvex SOCP, the gap falls from933\.85%933\.85\\%to141\.28%141\.28\\%, while feasibility increases from0\.250\.25to1\.001\.00\. DC3 also improves across all three QP variants, with reductions of2\.0%2\.0\\%–4\.3%4\.3\\%\.

The evolved features also improve solver\-facing learned roles\. For PDHG\-Net, iteration saving increases from42\.44%42\.44\\%to51\.05%51\.05\\%, and time saving increases from34\.03%34\.03\\%to39\.28%39\.28\\%\. Learning to Pivot reduces the geometric mean pivot count from1162\.861162\.86to857\.78857\.78, a26\.2%26\.2\\%improvement\. Its top\-1 accuracy changes only modestly, indicating that downstream solver progress can improve substantially even when local prediction accuracy changes little\. Smart Initial Basis also increases iteration saving, from59\.28%59\.28\\%to60\.24%60\.24\\%\.

#### Mixed\-integer Prediction\.

Table[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1)evaluates evolved feature functions for two learned MILP search strategies\. For Predict\-and\-Search, the bestFunL2Ovariant outperforms the handcrafted features on both metrics across all three domains\. The final primal gap decreases from0\.423%0\.423\\%to0\.350%0\.350\\%on CA, from0\.699%0\.699\\%to0\.095%0\.095\\%on MIS, and from0\.018%0\.018\\%to0\.013%0\.013\\%on MVC\. The corresponding primal integrals decrease from10\.6110\.61to9\.009\.00, from20\.8920\.89to12\.7812\.78, and from13\.8513\.85to3\.693\.69, respectively\. These results indicate that the evolved representations help Predict\-and\-Search find stronger incumbents earlier\.

For learned backdoor branching, bothFunL2O\-Claude andFunL2O\-GPT reduce mean wall time relative to the handcrafted features on all three domains\. The best evolved functions achieve mean wall times of147\.6147\.6seconds on CA,46\.046\.0seconds on MIS, and18\.918\.9seconds on MVC\. Relative to the default solver, these results correspond to reductions of11\.7%11\.7\\%,55\.2%55\.2\\%, and23\.5%23\.5\\%, respectively\. The evolved functions also achieve the highest win rates on every domain, reaching41%41\\%on CA,49%49\\%on MIS, and46%46\\%on MVC\. Together, these results show that feature evolution extends beyond continuous optimization to learned partial assignments and branching priorities for MILP search\.

### Analysis of the Evolved Features

#### Semantic structure versus feature width\.

We compare evolved functions with matched\-width random projections wherever the host interface supports an append\-only control\. On PDHG\-Net, iteration saving rises from35\.59%35\.59\\%with handcrafted features to45\.19%45\.19\\%with random features and47\.99%47\.99\\%withFunL2O\. Evolved functions also outperform matched\-width random controls on four of the five IPM\-MPNN distributions\. These comparisons indicate that the gains often reflect optimization\-relevant structure beyond the effect of increasing the input dimension\.

#### Inspecting the evolved programs\.

The returned programs expose recognizable optimization relationships\. For set cover, they normalize variable costs by column coverage and constraint signals by row coverage\. For PageRank LPs, they compare row and column mass and degree, exposing asymmetry in the directed graph\. For nonlinear problems, they construct multiscale representations using magnitude, sign, logarithmic, squared, normalized, and local\-difference transformations\. The MILP programs combine variable types and objective coefficients with LP\-relaxation values, fractionality, reduced costs, row and column statistics, and normalized constraint signals\. Because the selected functions are ordinary preprocessing code, these transformations can be inspected directly and deployed without an LLM\.

#### Proposal\-model dependence\.

Across the continuous experiments shared by all four LLMs, Claude\-Opus\-4\.8 , GPT\-5\.5, Gemini\-3\.1\-Pro Preview, and DeepSeek\-Math\-V2 improve 15, 17, 17, and 6 cells, respectively\. In the MILP experiment, Claude\-Opus\-4\.8 and GPT\-5\.5 both produce feature functions that reduce backdoor\-branching wall time across CA, MIS, and MVC, while the best proposal\-model variant also improves both Predict\-and\-Search metrics on all three domains\. These results show thatFunL2Ois not tied to a single LLM provider, although stronger proposal models generate successful feature functions more reliably\.

## Related Work

#### Learning to optimize\.

Learning\-to\-optimize replaces or augments part of an optimization procedure with a model trained on related instances\(Andrychowiczet al\.[2016](https://arxiv.org/html/2607.27389#bib.bib2); Chenet al\.[2022](https://arxiv.org/html/2607.27389#bib.bib1)\)\. Prior work learns update rules, predicts solutions, or guides classical solvers through warm starts, basis selection, pivoting, partial assignments, and branching decisions\(Wichrowskaet al\.[2017](https://arxiv.org/html/2607.27389#bib.bib8); Chenet al\.[2020](https://arxiv.org/html/2607.27389#bib.bib15); Harrisonet al\.[2022](https://arxiv.org/html/2607.27389#bib.bib16); Yanget al\.[2023](https://arxiv.org/html/2607.27389#bib.bib21); Dontiet al\.[2021](https://arxiv.org/html/2607.27389#bib.bib24); Nguyen and Donti[2026](https://arxiv.org/html/2607.27389#bib.bib27); Qianet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib22); Sambharyaet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib18); Liet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib23); Fanet al\.[2023](https://arxiv.org/html/2607.27389#bib.bib25); Liuet al\.[2024b](https://arxiv.org/html/2607.27389#bib.bib26); Hanet al\.[2023](https://arxiv.org/html/2607.27389#bib.bib12); Caiet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib9)\)\. These methods typically keep the input representation fixed\. In contrast,FunL2Osearches over executable feature functions while leaving the model, training procedure, prediction target, and solver interface unchanged\.

#### Representations for optimization\.

Optimization problems are often represented as variable–constraint bipartite graphs\(Gasseet al\.[2019](https://arxiv.org/html/2607.27389#bib.bib28)\)\. Their node and edge features determine what information is available to the model and can limit expressiveness even when the architecture is fixed\(Chenet al\.[2025](https://arxiv.org/html/2607.27389#bib.bib30)\)\. Prior work has proposed specialized or manually designed features for particular problem classes and applications\(Qianet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib22); Caiet al\.[2025c](https://arxiv.org/html/2607.27389#bib.bib13),[b](https://arxiv.org/html/2607.27389#bib.bib10)\)\.FunL2Oinstead automates representation design by evaluating executable feature functions through the complete L2O pipeline\.

#### LLM\-guided program and feature search\.

FunSearch and related systems use LLMs to generate executable programs that are selected by an external evaluator\(Romera\-Paredeset al\.[2024](https://arxiv.org/html/2607.27389#bib.bib33); Liuet al\.[2024a](https://arxiv.org/html/2607.27389#bib.bib34); Yeet al\.[2024](https://arxiv.org/html/2607.27389#bib.bib35); Van Stein and Bäck[2024](https://arxiv.org/html/2607.27389#bib.bib36); Novikovet al\.[2025](https://arxiv.org/html/2607.27389#bib.bib37)\)\. LLMs have also been used to construct features for tabular prediction\(Hollmannet al\.[2023](https://arxiv.org/html/2607.27389#bib.bib31); Abhyankaret al\.[2025](https://arxiv.org/html/2607.27389#bib.bib32)\)\.FunL2Oapplies this idea to structured optimization data and solver states: candidates must satisfy a pipeline\-specific semantic contract and are evaluated only after retraining the unchanged L2O model and measuring downstream optimization performance\.

## Discussion

FunL2Oautomates a previously fixed component of the L2O pipeline: the feature function\. Whereas existing methods learn solver actions from hand\-designed representations,FunL2Ooptimizes the representation itself using downstream optimization performance\. This enables a unified framework to improve pipelines with different model architectures, training objectives, and learned outputs\.

The evolved programs are especially effective when handcrafted features omit useful structural relationships\. They often combine problem\-specific quantities, such as cost and coverage, PageRank imbalance, nonlinear transformations, or LP\-relaxation statistics\. Because the output is executable code, each evolved program serves both as a deployable feature function and as an interpretable hypothesis about what information the original representation lacked\. We intentionally use a simple evolutionary loop so that performance gains can be attributed primarily to feature quality rather than to search\-engineering choices\. More sophisticated selection, mutation, and crossover mechanisms are complementary to our contribution and could be incorporated in future work\.

A natural concern is that program search may overfit the finite set of instances used during evolution\. To reduce this risk, all candidate selection is performed on validation instances, while final performance is reported only on disjoint test sets\. The consistent test\-set gains suggest that the evolved features generalize beyond the instances used for candidate ranking\. The main limitation is the offline search cost, since each valid candidate requires retraining the host model and evaluating downstream performance\. However, the LLM is not needed after the search, so deployment cost remains unchanged\. In repeated\-optimization settings, this offline cost can be amortized over many future instances\(Chenet al\.[2022](https://arxiv.org/html/2607.27389#bib.bib1)\)\.

## Conclusion

L2O methods learn how to act on optimization problems, but typically inherit a handcrafted representation of those problems\.FunL2Omakes this representation searchable\. Within a semantic contract, an LLM proposes executable feature functions whose value is determined by retraining the unchanged L2O pipeline and measuring downstream optimization performance\. Across LP, QP, constrained nonlinear, and mixed\-integer optimization, evolved features improve solution prediction, warm starting, simplex decisions, and MILP search without changing the underlying models or solvers\. The selected programs require no LLM at deployment and expose interpretable relations that the original representations omitted\. These results establish feature\-function design as a distinct and automatable component of learning to optimize\.

## Acknowledgment

This paper reports on research conducted while Bingheng Li, Junyang Cai, and Yupeng Zhang interned at IBM Research, Yorktown Heights, New York, US\.

## References

- N\. Abhyankar, P\. Shojaee, and C\. K\. Reddy \(2025\)Llm\-fe: automated feature engineering for tabular data with llms as evolutionary optimizers\.arXiv preprint arXiv:2503\.14434\.Cited by:[LLM\-guided program and feature search\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px3.p1.1)\.
- M\. Andrychowicz, M\. Denil, S\. Gomez, M\. W\. Hoffman, D\. Pfau, T\. Schaul, B\. Shillingford, and N\. De Freitas \(2016\)Learning to learn by gradient descent by gradient descent\.Advances in neural information processing systems29\.Cited by:[Introduction](https://arxiv.org/html/2607.27389#Sx1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx2.SS0.SSS0.Px1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- Anthropic \(2026\)Introducing claude opus 4\.8\.Note:https://www\.anthropic\.com/news/claude\-opus\-4\-8Cited by:[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p2.1)\.
- J\. Cai, T\. Huang, and B\. Dilkina \(2024\)Learning backdoors for mixed integer linear programs with contrastive learning\.InECAI 2024,pp\. 2418–2425\.Cited by:[Mixed\-integer Prediction\.](https://arxiv.org/html/2607.27389#Sx2.SS0.SSS0.Px4.p1.1),[Table 1](https://arxiv.org/html/2607.27389#Sx3.T1.18.18.18.3),[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- J\. Cai, T\. Huang, and B\. Dilkina \(2025a\)Multi\-task representation learning for mixed integer linear programming\.InInternational Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research,pp\. 134–151\.Cited by:[Introduction](https://arxiv.org/html/2607.27389#Sx1.p1.1)\.
- J\. Cai, W\. Huang, B\. Long, M\. Cleaveland, J\. V\. Deshmukh, L\. Lindemann, and B\. Dilkina \(2025b\)Neuro\-symbolic acceleration of milp motion planning with temporal logic and chance constraints\.arXiv preprint arXiv:2508\.07515\.Cited by:[Representations for optimization\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px2.p1.1)\.
- J\. Cai, E\. M\. E\. Raqabi, P\. Van Hentenryck, and B\. Dilkina \(2025c\)ID\-pas: identity\-aware predict\-and\-search for general mixed\-integer linear programs\.arXiv preprint arXiv:2512\.10211\.Cited by:[Introduction](https://arxiv.org/html/2607.27389#Sx1.p2.1),[Representations for optimization\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px2.p1.1)\.
- Q\. Chen, L\. Li, Q\. Li, J\. Wu, A\. Wang, R\. Sun, X\. Luo, T\. Chang, and Q\. Shi \(2025\)When gnns meet symmetry in ilps: an orbit\-based feature augmentation approach\.InThe Thirteenth International Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2607.27389#Sx1.p2.1),[Representations for optimization\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px2.p1.1)\.
- T\. Chen, X\. Chen, W\. Chen, H\. Heaton, J\. Liu, Z\. Wang, and W\. Yin \(2022\)Learning to optimize: a primer and a benchmark\.Journal of Machine Learning Research23\(189\),pp\. 1–59\.Cited by:[Introduction](https://arxiv.org/html/2607.27389#Sx1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx2.SS0.SSS0.Px1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1),[Discussion](https://arxiv.org/html/2607.27389#Sx6.p3.1)\.
- T\. Chen, W\. Zhang, Z\. Jingyang, S\. Chang, S\. Liu, L\. Amini, and Z\. Wang \(2020\)Training stronger baselines for learning to optimize\.Advances in Neural Information Processing Systems33,pp\. 7332–7343\.Cited by:[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- P\. L\. Donti, D\. Rolnick, and J\. Z\. Kolter \(2021\)DC3: a learning method for optimization with hard constraints\.InInternational Conference on Learning Representations,Cited by:[Table 1](https://arxiv.org/html/2607.27389#Sx3.T1.8.8.8.4),[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- Z\. Fan, X\. Wang, O\. Yakovenko, A\. A\. Sivas, O\. Ren, Y\. Zhang, and Z\. Zhou \(2023\)Smart initial basis selection for linear programs\.InInternational conference on machine learning,pp\. 9650–9664\.Cited by:[Introduction](https://arxiv.org/html/2607.27389#Sx1.p1.1),[Table 1](https://arxiv.org/html/2607.27389#Sx3.T1.12.12.12.3),[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- M\. Gasse, D\. Chételat, N\. Ferroni, L\. Charlin, and A\. Lodi \(2019\)Exact combinatorial optimization with graph convolutional neural networks\.Advances in neural information processing systems32\.Cited by:[Feature functions\.](https://arxiv.org/html/2607.27389#Sx2.SS0.SSS0.Px2.p1.1),[Representations for optimization\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px2.p1.1)\.
- Google DeepMind \(2026\)Gemini 3\.1 pro\.Note:https://deepmind\.google/models/model\-cards/gemini\-3\-1\-pro/Cited by:[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p2.1)\.
- Q\. Han, L\. Yang, Q\. Chen, X\. Zhou, D\. Zhang, A\. Wang, R\. Sun, and X\. Luo \(2023\)A gnn\-guided predict\-and\-search framework for mixed\-integer linear programming\.InThe Eleventh International Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2607.27389#Sx1.p1.1),[Mixed\-integer Prediction\.](https://arxiv.org/html/2607.27389#Sx2.SS0.SSS0.Px4.p1.1),[Table 1](https://arxiv.org/html/2607.27389#Sx3.T1.16.16.16.3),[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- J\. Harrison, L\. Metz, and J\. Sohl\-Dickstein \(2022\)A closer look at learned optimization: stability, robustness, and inductive biases\.Advances in neural information processing systems35,pp\. 3758–3773\.Cited by:[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- N\. Hollmann, S\. Müller, and F\. Hutter \(2023\)Large language models for automated data science: introducing caafe for context\-aware automated feature engineering\.Advances in Neural Information Processing Systems36,pp\. 44753–44775\.Cited by:[LLM\-guided program and feature search\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px3.p1.1)\.
- T\. Huang, A\. M\. Ferber, A\. Zharmagambetov, Y\. Tian, and B\. Dilkina \(2024\)Contrastive predict\-and\-search for mixed integer linear programs\.InForty\-first International Conference on Machine Learning,Cited by:[Mixed\-integer Prediction\.](https://arxiv.org/html/2607.27389#Sx2.SS0.SSS0.Px4.p1.1)\.
- B\. Li, L\. Yang, Y\. Chen, S\. Wang, H\. Mao, Q\. Chen, Y\. Ma, A\. Wang, T\. Ding, J\. Tang,et al\.\(2024\)PDHG\-unrolled learning\-to\-optimize method for large\-scale linear programming\.InProceedings of the 41st International Conference on Machine Learning,pp\. 29164–29180\.Cited by:[Introduction](https://arxiv.org/html/2607.27389#Sx1.p1.1),[Table 1](https://arxiv.org/html/2607.27389#Sx3.T1.10.10.10.3),[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- F\. Liu, T\. Xialiang, M\. Yuan, X\. Lin, F\. Luo, Z\. Wang, Z\. Lu, and Q\. Zhang \(2024a\)Evolution of heuristics: towards efficient automatic algorithm design using large language model\.InForty\-first International Conference on Machine Learning,Cited by:[LLM\-guided program and feature search\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px3.p1.1)\.
- T\. Liu, S\. Pu, D\. Ge, and Y\. Ye \(2024b\)Learning to pivot as a smart expert\.InProceedings of the aaai conference on artificial intelligence,Vol\.38,pp\. 8073–8081\.Cited by:[Introduction](https://arxiv.org/html/2607.27389#Sx1.p1.1),[Table 1](https://arxiv.org/html/2607.27389#Sx3.T1.14.14.14.3),[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- C\. U\. Manual \(1987\)Ibm ilog cplex optimization studio\.Version12\(1987\-2018\),pp\. 1\.Cited by:[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p2.1)\.
- H\. Nguyen and P\. Donti \(2026\)Fsnet: feasibility\-seeking neural network for constrained optimization with guarantees\.Advances in neural information processing systems38,pp\. 39670–39708\.Cited by:[Table 1](https://arxiv.org/html/2607.27389#Sx3.T1.5.5.5.4),[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- A\. Novikov, N\. Vũ, M\. Eisenberger, E\. Dupont, P\. Huang, A\. Z\. Wagner, S\. Shirobokov, B\. Kozlovskii, F\. J\. Ruiz, A\. Mehrabian,et al\.\(2025\)Alphaevolve: a coding agent for scientific and algorithmic discovery\.arXiv preprint arXiv:2506\.13131\.Cited by:[LLM\-guided program and feature search\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px3.p1.1)\.
- OpenAI \(2026\)Introducing gpt\-5\.5\.Note:https://openai\.com/index/introducing\-gpt\-5\-5/Cited by:[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p2.1)\.
- C\. Qian, D\. Chételat, and C\. Morris \(2024\)Exploring the power of graph neural networks in solving linear optimization problems\.InInternational conference on artificial intelligence and statistics,pp\. 1432–1440\.Cited by:[Introduction](https://arxiv.org/html/2607.27389#Sx1.p1.1),[Table 1](https://arxiv.org/html/2607.27389#Sx3.T1.2.2.2.3),[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p1.1),[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1),[Representations for optimization\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px2.p1.1)\.
- B\. Romera\-Paredes, M\. Barekatain, A\. Novikov, M\. Balog, M\. P\. Kumar, E\. Dupont, F\. J\. Ruiz, J\. S\. Ellenberg, P\. Wang, O\. Fawzi,et al\.\(2024\)Mathematical discoveries from program search with large language models\.Nature625\(7995\),pp\. 468–475\.Cited by:[LLM\-guided program and feature search\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px3.p1.1)\.
- R\. Sambharya, G\. Hall, B\. Amos, and B\. Stellato \(2024\)Learning to warm\-start fixed\-point optimization algorithms\.Journal of Machine Learning Research25\(166\),pp\. 1–46\.Cited by:[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- Z\. Shao, Y\. Luo, C\. Lu, Z\. Ren, J\. Hu, T\. Ye, Z\. Gou, S\. Ma, and X\. Zhang \(2025\)Deepseekmath\-v2: towards self\-verifiable mathematical reasoning\.arXiv preprint arXiv:2511\.22570\.Cited by:[Experimental Setup](https://arxiv.org/html/2607.27389#Sx4.SSx1.p2.1)\.
- J\. Sun, L\. Zhang, G\. Chen, P\. Xu, K\. Zhang, and Y\. Yang \(2023\)Feature expansion for graph neural networks\.InInternational Conference on Machine Learning,pp\. 33156–33176\.Cited by:[Introduction](https://arxiv.org/html/2607.27389#Sx1.p2.1)\.
- N\. Van Stein and T\. Bäck \(2024\)Llamea: a large language model evolutionary algorithm for automatically generating metaheuristics\.IEEE Transactions on Evolutionary Computation29\(2\),pp\. 331–345\.Cited by:[LLM\-guided program and feature search\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px3.p1.1)\.
- O\. Wichrowska, N\. Maheswaranathan, M\. W\. Hoffman, S\. G\. Colmenarejo, M\. Denil, N\. Freitas, and J\. Sohl\-Dickstein \(2017\)Learned optimizers that scale and generalize\.InInternational conference on machine learning,pp\. 3751–3760\.Cited by:[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- J\. Yang, X\. Chen, T\. Chen, Z\. Wang, and Y\. Liang \(2023\)M\-l2o: towards generalizable learning\-to\-optimize by test\-time fast self\-adaptation\.arXiv preprint arXiv:2303\.00039\.Cited by:[Learning to optimize\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px1.p1.1)\.
- H\. Ye, J\. Wang, Z\. Cao, F\. Berto, C\. Hua, H\. Kim, J\. Park, and G\. Song \(2024\)Reevo: large language models as hyper\-heuristics with reflective evolution\.Advances in neural information processing systems37,pp\. 43571–43608\.Cited by:[LLM\-guided program and feature search\.](https://arxiv.org/html/2607.27389#Sx5.SSx3.SSS0.Px3.p1.1)\.

## Appendix AExperimental and Reproducibility Details

This appendix documents the final experimental protocol used throughout the paper\. The same principle is applied to every host pipeline:FunL2Oreplaces the executable feature function while retaining the host model, learning objective, training schedule, data generator or instance lists, learned output, and downstream solver interface\. Candidate quality is determined by the host evaluator rather than by the proposal model, and the returned feature function is used as ordinary deterministic preprocessing code at deployment\. Unless stated otherwise, every reported handcrafted–selected comparison is the arithmetic mean of three independent, seed\-paired repetitions of the corresponding host training and evaluation protocol\.

#### Computational cost\.

EachFunL2Osearch evaluates 48 candidate feature functions\. The average LLM API cost is approximately $4 per proposal, resulting in a total LLM cost of 48×$4=$192 per search\. Retraining each candidate takes approximately 15 GPU\-minutes, for a total of 48×15=720 GPU\-minutes, or 12 GPU\-hours\. Downstream CPU\-based evaluation takes approximately 1000 seconds per candidate, resulting in 48,000 CPU\-seconds, or 13\.3 CPU\-hours, per search\. With one GPU and one CPU worker and no parallelization, a complete search requires approximately 25\.3 hours of computation, excluding LLM response latency and pipeline overhead\. These costs scale linearly with the number of pipeline–task–proposal\-model combinations, although candidate evaluations can be parallelized in practice\. Once the search is complete, the selected feature function incurs no additional LLM cost and only standard preprocessing overhead at deployment\.

### Host Pipelines and Training Configurations

Table[4](https://arxiv.org/html/2607.27389#A1.T4)summarizes the eight pipeline instantiations\. The continuous study covers solution prediction, primal–dual warm starts, and simplex decisions\. The mixed\-integer study applies the same feature\-search interface to Predict\-and\-Search and learned backdoor branching\.

Table 4:Host configurations\. Only the feature function is searched\. The learned output, network template, training objective, and solver interface remain those of the corresponding host implementation\.
### Data and Evaluation Splits

Every host follows the same separation principle\. A candidate feature program is trained using the host training split and ranked exclusively by its validation outcome\. After the search terminates, the selected program is frozen and evaluated on a disjoint test split that is never exposed to the search loop or the proposal model\. Training, validation, and test instances are disjoint\. Different proposal\-model runs use the same split definition, so their paired outcomes are directly comparable\. Within each reported comparison, the handcrafted feature and the validation\-selected feature are evaluated with the same three training seeds, initialization seeds, data\-order seeds, instance splits, and solver settings\. This pairing removes training\-seed variation from the method difference before the three paired outcomes are averaged\.

Table 5:Data organization used by the host evaluators\. “Selection” denotes the data used to rank candidate feature programs; “reporting” denotes the data used for the displayed outcome\.

## Appendix BSearch Procedure

### Budget and Elitism

The handcrafted feature function is evaluated once before program generation and remains in the search memory as the fixed baseline\. Each search then runs eight proposal generations with six proposal slots per generation, giving a budget of 48 LLM\-proposed candidate programs for every pipeline, task, and proposal model\. The two highest\-ranked programs form the elite context for the next generation\. The same eight\-generation, six\-proposal, two\-elite protocol is used for the continuous and mixed\-integer hosts\.

Each proposal is compiled and checked against the host contract\. A rejected proposal receives the localized validation error and may be repaired up to three times\. Valid candidates within a generation are independent conditioned on the current elite memory and are therefore trained and evaluated in parallel\.

memory=\[train\_and\_validate\(handcrafted\_seed\)\]

forgenerationinrange\(8\):

elites=top\_two\_by\_validation\(memory\)

context=contract\+elites\+validation\_scores\(elites\)

forprograminpropose\(context,count=6\):

program=validate\_or\_repair\(program,max\_repairs=3\)

ifprogramisvalid:

val=train\_and\_validate\(program\)

memory\.append\(\(program,val\)\)

selected=best\_by\_validation\(memory\)

test\_result=evaluate\_once\_on\_disjoint\_test\(selected\)

returnselected,test\_result

The proposal model receives the semantic contract, complete source of the elite feature programs, and aggregate validation outcomes\. It never receives validation instances, targets, or test outcomes, and it does not assign fitness\. The final\-test evaluator is called only after validation\-based program selection\. After search, neither the proposal model nor the evolutionary memory is needed for inference\.

### Ranking

For prediction hosts with feasibility requirements, ranking is lexicographic: feasibility acceptance, degree of violation, and then objective quality\. This prevents an objective\-only improvement from displacing a feasible program\. For solver\-facing hosts, the native host metric determines the ordering\. The handcrafted seed remains in the memory throughout search and is returned whenever no proposal ranks above it\.

## Appendix CSemantic Feature Contracts

A semantic feature contract defines the callable interface, permitted deployment\-time information, tensor orientation, feature\-width limit, and required handcrafted prefix\. Table[6](https://arxiv.org/html/2607.27389#A3.T6)lists the principal interfaces\. The contracts permit deterministic transformations of information already exposed by the host and exclude training labels, reference solutions, evaluation outcomes, file or network access, and additional solver calls inside the feature function\.

Table 6:Feature\-function interfaces and output requirements\.mmandnndenote the numbers of constraints and variables, respectively;BBdenotes batch size\.#### Static validation\.

Candidate source is restricted to the requested function and a safe execution environment\. Imports are sanitized, and the validator prevents external I/O, dynamic code execution, mutable global state, and unavailable information sources\.

#### Dynamic validation\.

A host\-specific probe checks the function signature, return structure, row or batch alignment, feature width, numerical finiteness, and required seed channels\. Passing the probe establishes interface validity; utility is determined only by retraining and downstream evaluation\.

## Appendix DPrompt and Proposal\-Model Configuration

The same run\-independent prompt structure is instantiated with a host\-specific function signature, metric, and contract\. The complete prompt files are included in the accompanying source artifact\. The central template is:

YoudesigninputfeaturesforaFIXEDlearning\-to\-optimize

pipeline\.Themodel,data,trainingprocedure,solver,and

seedbaselinearefixed\.Changeonly\{function\_name\}\.

Useonlyfieldslistedinthesemanticfeaturecontract\.

Returndeterministic,finitefeatureswiththerequired

shapeandpreservedseedchannels\.Donotaccesstargets,

referencesolutions,files,networks,oradditionalsolvers\.

Metricanddirection:\{host\_metric\}

Functionsignatureandtypedoutput:\{contract\}

Eliteprogramsandmeasuredoutcomes:\{elite\_memory\}

ReturnexactlyonePythoncodeblockdefiningthefunction\.

The recorded API model identifiers areclaude\-opus\-4\-8,azure/gpt\-5\.5,gcp/gemini\-3\.1\-pro\-preview, andrits/deepseek\-ai/DeepSeek\-Math\-V2\. The continuous study uses all four proposal models; the displayed MILP study reports Claude and GPT\-5\.5 runs separately\. The maximum completion length is 16,000 tokens\. Temperature and top\-ppare not explicitly overridden in the search client and therefore follow the provider defaults associated with the recorded endpoints\.

## Appendix EMetrics and Aggregation

#### Prediction metrics\.

IPM\-MPNN reports relative objective gap\. FSNet and DC3 report the host optimality gap in percent and feasibility at maximum constraint violation10−410^\{\-4\}\. Candidate selection for constrained prediction is feasibility\-first\.

#### Solver metrics\.

For cold\-start workCCand learned\-start workHH, iteration saving is

100​C−HC,100\\frac\{C\-H\}\{C\},and time saving is defined analogously\. Learning to Pivot reports the geometric mean of pivot counts\. Predict\-and\-Search reports final primal gap and primal integral; learned backdoors report mean wall time and the percentage of instances on which a method is fastest\.

#### Improvement rate\.

For a lower\-is\-better metric with handcrafted valueBBand searched valueFF,

IR=100​B−F\|B\|\.\\operatorname\{IR\}=100\\frac\{B\-F\}\{\|B\|\}\.For a higher\-is\-better metric, the numerator isF−BF\-B\. Positive values therefore always denote improvement\.

#### Selection and reporting\.

All search\-time ranking fields are validation metrics\. Final\-test metrics are produced only for the validation\-selected program and are not available to candidate ranking, elite updates, prompt construction, or proposal\-model comparison\.

#### Three\-repeat evaluation protocol\.

Each reported handcrafted–selected pair is obtained from three independent repetitions\. In repetitionr∈\{1,2,3\}r\\in\\\{1,2,3\\\}, the handcrafted baseline and the selected feature program use the same host initialization, minibatch or data\-order seed, train/validation/test split, solver parameters, and stopping rule\. LetmrHm\_\{r\}^\{\\mathrm\{H\}\}andmrFm\_\{r\}^\{\\mathrm\{F\}\}denote the resulting test metrics\. The reported values are

m¯H=13​∑r=13mrH,m¯F=13​∑r=13mrF\.\\bar\{m\}^\{\\mathrm\{H\}\}=\\frac\{1\}\{3\}\\sum\_\{r=1\}^\{3\}m\_\{r\}^\{\\mathrm\{H\}\},\\qquad\\bar\{m\}^\{\\mathrm\{F\}\}=\\frac\{1\}\{3\}\\sum\_\{r=1\}^\{3\}m\_\{r\}^\{\\mathrm\{F\}\}\.Thus, the displayed difference is the mean of three paired feature effects rather than a difference between unrelated training runs\. The four proposal models are distinct search configurations and are not treated as statistical repetitions\.

For the MILP hosts, wall time is first averaged over the 40 test instances within each repetition and then averaged over the three repetitions\. Win rate is computed analogously from the fraction of the 40 instances on which a method is fastest; displayed percentages are rounded to the nearest percentage point\. Consequently, the reported MILP values summarize3×403\\times 40paired instance evaluations per domain while preserving the original 40\-instance benchmark in each repetition\.

#### Across\-model summaries\.

The continuous headline table reports the median across the four independently executed proposal\-model configurations after each configuration has itself been averaged over three paired repetitions\. This median is a robustness summary across proposal mechanisms; it is not used as a substitute for the three training repetitions and does not select a proposal model using test performance\. The mixed\-integer table retains Claude and GPT\-5\.5 as separate columns; boldface is descriptive only and is not a row\-wise oracle used by the method\.

## Appendix FComplete Results

Tables[7](https://arxiv.org/html/2607.27389#A6.T7)–[11](https://arxiv.org/html/2607.27389#A6.T11)provide the proposal\-model\-level continuous results\. Every displayed value is the arithmetic mean of three independent, seed\-paired repetitions\. Each cell is written as handcrafted→\\toreturned program\. A check mark denotes improvement on the reported primary metric, while an equality sign denotes retention of the seed value\.

Table 7:IPM\-MPNN final\-test objective gap by LP distribution and proposal model \(lower is better\)\. Values are means over three independent, seed\-paired repetitions\.Table 8:FSNet final\-test optimality gap for all 12 evaluated tasks \(lower is better\)\. Values are means over three independent, seed\-paired repetitions\. The 1,000\-variable tasks form the main\-paper grid; the 100\-variable tasks provide the corresponding smaller\-scale evaluation\.Table 9:DC3 final\-test optimality gap by proposal model \(lower is better\)\. Values are means over three independent, seed\-paired repetitions\.Table 10:PDHG\-Net iteration saving by proposal model \(higher is better\)\. Values are means over three independent, seed\-paired repetitions\.Table 11:Simplex\-oriented host results, averaged over three independent, seed\-paired repetitions\. Learning to Pivot reports pivot\-count geometric mean \(lower is better\); Smart Initial Basis reports iteration saving \(higher is better\)\.Learning to Pivot

Smart Initial Basis

Table 12:Mixed\-integer final\-test outcomes for the two independently reported proposal\-model runs\. Each value averages three independent repetitions of the 40\-instance domain benchmark\. Bold marks the stronger displayed value in each row for readability; no row\-wise proposal\-model selection is used by the method\.## Appendix GControlled Analyses

### What the Equal\-Budget Controls Establish

The purpose of the controls is to distinguish the paper’s mechanism\-level claims from the generic benefit of evaluating many candidate representations\. The paper does not claim thatFunL2Odominates every possible AutoML, genetic\-programming, or fixed\-library system\. It claims that LLM\-proposed executable features, selected by the unchanged L2O evaluator and refined with measured feedback, provide an effective and general way to improve heterogeneous L2O representations\. The available controls test the two principal alternative explanations for the reported gains: best\-of\-many sampling and feature\-width expansion\.

Table 13:Equal\-budget and matched\-capacity evidence\. Each comparison retains the host evaluator, validation rule, split, and training protocol\.These controls provide convergent evidence\. The equal\-budget PDHG comparison isolates evolutionary feedback from the number of LLM calls; the matched\-width experiments isolate searched semantic structure from simple feature expansion; and the cross\-model results show that successful programs are not an artifact of one proposal provider\. Together with disjoint test reporting and three seed\-paired repetitions, the current results support the paper\-level conclusion that LLM\-guided feature evolution is a meaningful contributor to performance, rather than merely a consequence of selecting the best among many wider inputs\. The experiments are not intended to establish universal dominance over every conceivable program\-search baseline\.

### Matched\-Width Features

When a feature program changes the input width, only the corresponding input projection is resized mechanically; the hidden architecture, learning objective, optimizer, schedule, and solver remain fixed\. The append\-only control replaces the searched additional channels with a fixed random projection of the same width while retaining the host training setup\. Across the six supported LP and PDHG settings, the searched programs attain the best value in five\. Because the random control matches the added width, input projection size, paired training seeds, and evaluation protocol, this comparison rules out simple width expansion as a sufficient explanation in these controlled settings\.

Table 14:Matched\-width controls\. Lower is better for LP objective gap; higher is better for PDHG iteration saving\.
### Measured Evolutionary Feedback

On PDHG\-Net, the independent\-sampling baseline uses the same proposal model, the same 48 LLM calls, the same 48 candidate retraining budget, the same semantic contract, and the same validation evaluator, but every proposal is conditioned only on the handcrafted seed and receives no elite programs or measured outcome feedback\. Independent sampling increases iteration saving from 35\.89% to 44\.42%\. Retaining high\-performing programs and returning their measured outcomes to later generations reaches 52\.23%\. The additional 7\.81 percentage points under an equal budget isolate the contribution of measured evolutionary feedback from the generic best\-of\-48 effect\.

### Seed\-Preserving Selection

The handcrafted function remains an eligible program throughout search\. This safeguard makes replacement evidence\-based: a proposed program is returned only when it improves the host ranking criterion\. It also makes the displayed equality cases interpretable as successful preservation of a strong original representation rather than as a forced modification\.

## Appendix HRepresentative Feature Programs

The source artifact contains the complete executable programs collected for every continuous host–task–proposal\-model cell\. The following excerpts illustrate three recurring structures discovered by the search\.

### Set Cover: Coverage\-Aware Scaling

row\_deg=\(A\.abs\(\)\>eps\)\.float\(\)\.sum\(1,keepdim=True\)

col\_deg=\(A\.abs\(\)\>eps\)\.float\(\)\.sum\(0,keepdim=True\)\.T

row\_norm=torch\.sqrt\(\(A\*A\)\.sum\(1,keepdim=True\)\+eps\)

col\_norm=torch\.sqrt\(\(A\*A\)\.sum\(0,keepdim=True\)\+eps\)\.T

cover\_ratio=torch\.tanh\(b\.reshape\(\-1,1\)/\(A\.sum\(1,keepdim=True\)\+eps\)\)

obj\_eff=torch\.tanh\(c\.reshape\(\-1,1\)/\(col\_deg\+eps\)\)

cons=torch\.cat\(\[cons\_seed,torch\.log1p\(row\_deg\),

torch\.log1p\(row\_norm\),cover\_ratio\],dim=1\)

vals=torch\.cat\(\[vals\_seed,torch\.log1p\(col\_deg\),

torch\.log1p\(col\_norm\),obj\_eff\],dim=1\)

The program exposes column coverage, row load, coefficient scale, and objective cost relative to coverage while preserving the handcrafted mean and standard\-deviation channels\.

### PageRank: Directed Mass and Degree

absA=torch\.abs\(A\.to\_dense\(\)ifA\.is\_sparseelseA\)

col\_l1,row\_l1=absA\.sum\(0\),absA\.sum\(1\)

col\_deg=\(absA\>eps\)\.float\(\)\.sum\(0\)

row\_deg=\(absA\>eps\)\.float\(\)\.sum\(1\)

col\_share=torch\.tanh\(col\_l1/\(col\_l1\.mean\(\)\+eps\)\)

row\_share=torch\.tanh\(row\_l1/\(row\_l1\.mean\(\)\+eps\)\)

var=torch\.stack\(\[lb,ub,c,torch\.log1p\(col\_l1\),

torch\.log1p\(col\_deg\),col\_share\],dim=1\)

con=torch\.stack\(\[sense\_le,sense\_eq,sense\_ge,b,

torch\.log1p\(row\_l1\),torch\.log1p\(row\_deg\),

row\_share\],dim=1\)

These channels distinguish row and column mass, exposing the directed asymmetry relevant to PageRank while preserving the original primal–dual inputs\.

### SOCP: Normalized Dense Parameters

X=fs\_state\["X"\]

row\_norm=torch\.sqrt\(torch\.sum\(X\*X,dim=\-1,

keepdim=True\)\+1e\-6\)

x\_scaled=X/row\_norm

features=torch\.cat\(\[X,x\_scaled,row\_norm\],dim=\-1\)

The returned representation retains the raw dense parameter vector and augments it with a scale\-invariant view and its norm\.

Table 15:Principal software environments\. CUDA versions correspond to the PyTorch builds used by the GPU jobs\.