Evolutionary Feature Engineering for Structured Data

# Evolutionary Feature Engineering for Structured Data
Source: [https://arxiv.org/html/2607.01548](https://arxiv.org/html/2607.01548)
Ege Onur Taga1Yilin Zhuang1M\. Emrullah Ildiz1Petros Mol2Abhimanyu Das2Karthik Duraisamy1Samet Oymak1,21University of Michigan2Google Research

###### Abstract

Large language models are increasingly used as open\-ended search operators in evolutionary optimization\. We introduce Evolutionary Feature Engineering \(EFE\), a framework for using LLM\-based evolution to discover preprocessing transformations for structured data\. EFE represents transformations as Python programs with a standardizedfit/transforminterface, allowing them to be inserted directly into existing machine learning pipelines\. During evolution, candidate programs are refined using dataset context, summary statistics, and downstream performance feedback on validation set\. We instantiate EFE in two settings\. For time\-series forecasting, EFE\-Time learns invertible, dataset\-specific normalizations that improve off\-the\-shelf time\-series foundation models\. It reduces forecasting errors \(MASE, WQL, MAE\) 3% or more when averaged across datasets and improvements are as much as 19% on the COVID\-Deaths dataset\. Notably, these improvements occur with recent TSFMs such as Chronos\-2\. For tabular prediction, EFE\-Tab evolves compact feature programs that add useful interpretable features and remove redundant ones, improving or matching existing LLM\-based feature\-engineering methods\. We found EFE\-Tab to be particularly effective on classical decision trees, where small sets of evolved features yield competitive accuracy while preserving interpretability\. Overall, EFE demonstrates that LLM\-based evolution can improve both accuracy and interpretability when automatically tackling structured data\.

## 1Introduction

Feature engineering has a simple premise: a model need not change if the data can be presented in a better form\. In forecasting, normalization or detrending can make a sequence easier to extrapolate; in tabular prediction, a ratio, interaction, threshold, or aggregate can make a simple classifier substantially more expressive\. Such transformations are often obvious in hindsight to human experts but difficult to specify in advance, since they depend on the dataset, the downstream model, and structure not captured by fixed preprocessing libraries\.

Large language models make it possible to search over a richer open\-ended space of executable feature\-engineering programs\. Recent systems such as AlphaEvolve\(Novikov et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib21)\)show that LLMs can guide evolutionary code discovery, while LLM\-based feature\-engineering methods such as CAAFE and LLM\-FE show that dataset context can help construct useful tabular features\(Hollmann et al\.,[2023](https://arxiv.org/html/2607.01548#bib.bib11); Abhyankar et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib1)\)\. These results suggest a broader question: can LLM\-based evolution discover useful preprocessing programs for structured data, specifically time series and tables, where temporal order and column structure constrain and guide LLM’s action space?

We propose*Evolutionary Feature Engineering*\(EFE\) as a framework for evolving feature transformations as Python programs\. Each candidate follows a standardfit/transforminterface and is evaluated by inserting it before a fixed downstream model\. An LLM proposes transformations using dataset metadata, summary statistics, and outcomes of previous trials; validation performance is then returned to the evolutionary loop as feedback\. Thus, search is driven directly by downstream task performance and dataset\-specific context which captures domain expertise\.

We instantiate EFE in two settings as depicted in Figure[1](https://arxiv.org/html/2607.01548#S1.F1)\. For time\-series forecasting,*EFE\-Time*evolves dataset\-specific, invertible normalization programs withfit,transform, andinverse\_transformmethods\. This is motivated by the importance of normalization for time\-series foundation models: widely used transforms such as RevIN\(Kim et al\.,[2022](https://arxiv.org/html/2607.01548#bib.bib15)\)or arcsinh\-style scaling\(Ansari et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib4)\)are useful but necessarily limited, since no single rule can handle all trends, scales, outliers, and seasonal patterns\. EFE\-Time instead searches for transformations tailored to each dataset\. The forecaster operates in the transformed space, and predictions are mapped back before evaluation \(Figure[1](https://arxiv.org/html/2607.01548#S1.F1), top right\)\.

For tabular prediction, EFE\-Tab searches for small, high\-value feature programs as illustrated in Figure[1](https://arxiv.org/html/2607.01548#S1.F1)\. Rather than generating large feature sets, it optimizes validation AUC improvement over the raw\-feature baseline while penalizing added or dropped features\. This encourages transformations that improve prediction enough to justify their complexity, making the approach especially suitable when we intend the downstream model to remain simple and interpretable\.

![Refer to caption](https://arxiv.org/html/2607.01548v1/x1.png)Figure 1:Given a dataset and optional context, EFE uses metadata, statistics, and past feedback to prompt an LLM to proposefit/transformprograms\. Candidates are evaluated against the identity baseline, and scores are fed back into an evolutionary loop\.Experiments show that EFE improves performance in both domains\. On synthetic exponential\-trend forecasting tasks, EFE\-Time discovers transformations that make series easier for foundation models to extrapolate\. On real time\-series datasets from GIFT\-Eval\(Aksu et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib3)\), EFE\-Time improves Chronos\-2\(Ansari et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib4)\)on several datasets and transfers to other forecasters, including TimesFM 2\.5\(Das et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib7)\), Moirai 2\.0\(Liu et al\.,[2026a](https://arxiv.org/html/2607.01548#bib.bib16)\), and Reverso\(Fu et al\.,[2026](https://arxiv.org/html/2607.01548#bib.bib9)\)\. It is also complementary to model adaptation: applying the evolved transform before fine\-tuning yields larger average gains than either component alone \(Figure[4](https://arxiv.org/html/2607.01548#S3.F4)\)\. On tabular datasets, EFE\-Tab achieves the best mean rank among compared feature\-engineering methods across TabPFN, LightGBM, and decision trees, with particularly strong gains for decision trees \(see Table[2](https://arxiv.org/html/2607.01548#S3.T2), Figure[5](https://arxiv.org/html/2607.01548#S3.F5)\)\.

Overall, our contributions are threefold:First, we formulate feature engineering for structured data as evolutionary search over stateful preprocessing programs\.Second, we introduce EFE\-Time, which evolves invertible dataset\-specific transformations for time\-series foundation models\.Finally, we introduce EFE\-Tab, a parsimonious feature\-engineering method that balances predictive improvement against feature complexity\.

Related Work\.Recent work has used LLMs to automate feature engineering for tabular data, including CAAFE, which proposes semantic feature interactions from dataset descriptions\(Hollmann et al\.,[2023](https://arxiv.org/html/2607.01548#bib.bib11)\), and OCTree, which combines LLM reasoning with feedback from shallow decision trees\(Nam et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib19)\)\. Closest to our work, LLM\-FE applies FunSearch\-style evolutionary search to optimize tabular feature\-transformation programs using dataset context\(Abhyankar et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib1); Romera\-Paredes et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib22)\)\. In contrast, we also evolve full invertible time\-series normalization programs with train\-fitted state and explicit inverse transformations, following the broader full\-program optimization paradigm of AlphaEvolve\(Novikov et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib21)\)\. While ELATE also uses evolutionary LLM search for time\-series covariate generation\(Murray et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib18)\), it does not target invertible normalizations\. Since no codebase is available for ELATE, we omit comparisons against its covariate\-generation method\. We provide further discussion of related work in Appendix[A](https://arxiv.org/html/2607.01548#A1)\.

![Refer to caption](https://arxiv.org/html/2607.01548v1/x2.png)\(a\)
![Refer to caption](https://arxiv.org/html/2607.01548v1/x3.png)\(b\)

Figure 2:\(a\) Effect of the evolved transform on forecasting: Top: Chronos\-2 on a steep exponential series under\-extrapolates vs\. ground truth; Middle: after the transform the series is near\-stationary and forecasts better match the transformed truth; Bottom: inverse\-transforming the forecast yields a much closer fit\. \(b\) Aggregate forecast metrics on a synthetic exponential\-growth benchmark \(100 series, length 5000, 128\-step horizon\): NO FE uses raw inputs, EFE\-Time forecasts in transformed space and inverts back\.
## 2Evolving Feature Programs

Evolutionary Feature Engineering \(EFE\) searches over executable preprocessing programs rather than over model architectures\. Each candidate program follows a standardfit/transforminterface, so the selected transformation can be inserted directly before an existing forecasting or prediction pipeline\. The downstream predictive routine is fixed: EFE does not change model architecture, hyperparameters, or training protocol, but only the representation passed into that routine\.

Starting from the identity transformation, EFE runs an evolutionary loop\. At each iteration, a prompt generator provides an LLM with dataset context, metadata, summary statistics, previous evaluation feedback, and examples of high\-performing programs\. The LLM proposes a modified transformation program\. The evaluator checks that the program is executable and leakage\-free, inserts it before the fixed downstream model, and scores it against the identity baseline\. The resulting score and feedback are returned to the prompt generator, allowing later candidates to build on previous successes and failures\. Thus, the LLM acts as a variation operator, while selection is driven by downstream validation performance\.

### 2\.1General Setup

Let𝒟\\mathcal\{D\}be a structured dataset with optional contextcc\. A validation protocol producesKKevaluation instancesΠ​\(𝒟\)=\{\(Djfit,Djin,Yj\)\}j=1K,\\Pi\(\\mathcal\{D\}\)=\\\{\(D^\{\\mathrm\{fit\}\}\_\{j\},D^\{\\mathrm\{in\}\}\_\{j\},Y\_\{j\}\)\\\}\_\{j=1\}^\{K\},whereDjfitD^\{\\mathrm\{fit\}\}\_\{j\}is the data available to a preprocessing program when fitting its state,DjinD^\{\\mathrm\{in\}\}\_\{j\}is the input passed to the downstream routine, andYjY\_\{j\}is the held\-out target used only for evaluation\. A candidate programppconsists of three operations:fitp\\mathrm\{fit\}\_\{p\},transformp\\mathrm\{transform\}\_\{p\}, andpostp\\mathrm\{post\}\_\{p\}\. On validation instancejj, it is applied as

σj,p=fitp​\(Djfit;c\),D~j,p=transformp​\(Djin;σj,p\)Y^j,p=postp​\(ℬj​\(D~j,p\);σj,p\),\\sigma\_\{j,p\}=\\mathrm\{fit\}\_\{p\}\(D^\{\\mathrm\{fit\}\}\_\{j\};c\),\\quad\\widetilde\{D\}\_\{j,p\}=\\mathrm\{transform\}\_\{p\}\(D^\{\\mathrm\{in\}\}\_\{j\};\\sigma\_\{j,p\}\)\\quad\\widehat\{Y\}\_\{j,p\}=\\mathrm\{post\}\_\{p\}\\\!\\left\(\\mathcal\{B\}\_\{j\}\(\\widetilde\{D\}\_\{j,p\}\);\\sigma\_\{j,p\}\\right\),
whereℬj\\mathcal\{B\}\_\{j\}is a fixed downstream predictive routine\. EFE does not modifyℬj\\mathcal\{B\}\_\{j\}; it searches only over preprocessing programspp\. For a lower\-is\-better lossℓ\\ell, we defineL^​\(p\)=1K​∑j=1Kℓ​\(Yj,Y^j,p\)\.\\widehat\{L\}\(p\)=\\frac\{1\}\{K\}\\sum\_\{j=1\}^\{K\}\\ell\(Y\_\{j\},\\widehat\{Y\}\_\{j,p\}\)\.Letpidp\_\{\\mathrm\{id\}\}denote the identity preprocessing program\. For loss\-based tasks, we measure relative improvement asΔ​\(p\)=1−L^​\(p\)/L^​\(pid\)\.\\Delta\(p\)=1\-\\widehat\{L\}\(p\)/\\widehat\{L\}\(p\_\{\\mathrm\{id\}\}\)\.For utility\-based metrics such as AUC, we instead use additive improvement overpidp\_\{\\mathrm\{id\}\}\. The final scores​\(p\)s\(p\)combines validation improvement with reliability, runtime, and complexity penalties\. Exact final scores for EFE\-Time and EFE\-Tab are provided in Sections[2\.3](https://arxiv.org/html/2607.01548#S2.SS3)and[2\.4](https://arxiv.org/html/2607.01548#S2.SS4)The valid program class𝒫valid\\mathcal\{P\}\_\{\\mathrm\{valid\}\}enforces executability, deterministic behavior, preservation of the required input structure, and leakage prevention:fitp\\mathrm\{fit\}\_\{p\}may use onlyDjfitD^\{\\mathrm\{fit\}\}\_\{j\}and contextcc, whileYjY\_\{j\}is accessible only to the evaluator\.

### 2\.2LLM\-Driven Evolutionary Optimization

FollowingLiu et al\. \([2026b](https://arxiv.org/html/2607.01548#bib.bib17)\), we formalize the evolutionary optimization where EFE performs sequential search over𝒫valid\\mathcal\{P\}\_\{\\mathrm\{valid\}\}\. Afterttevaluations, the optimizer maintains the historyℋt=\{\(pi,si,ai\)\}i=1t,\\mathcal\{H\}\_\{t\}=\\\{\(p\_\{i\},s\_\{i\},a\_\{i\}\)\\\}\_\{i=1\}^\{t\},wheresi=S​\(pi\)s\_\{i\}=S\(p\_\{i\}\)andaia\_\{i\}contains auxiliary feedback such as logs, diagnostics, or evaluator feedback\. At steptt, a search strategyStS\_\{t\}selects parent candidates, a prompt\-level variation operator, and optional inspiration examples:\(p​ct,πt,It\)∼CSt​\(ℋt\)\.\(pc\_\{t\},\\pi\_\{t\},I\_\{t\}\)\\sim C\_\{S\_\{t\}\}\(\\mathcal\{H\}\_\{t\}\)\.An LLM\-based generator proposes a new candidate,pt\+1∼Gsol\(⋅∣pct,πt,It\),p\_\{t\+1\}\\sim G\_\{\\mathrm\{sol\}\}\(\\cdot\\mid pc\_\{t\},\\pi\_\{t\},I\_\{t\}\),which is evaluated and appended to the history:

\(st\+1,at\+1\)=E​\(pt\+1\),ℋt\+1=ℋt∪\{\(pt\+1,st\+1,at\+1\)\}\.\(s\_\{t\+1\},a\_\{t\+1\}\)=E\(p\_\{t\+1\}\),\\qquad\\mathcal\{H\}\_\{t\+1\}=\\mathcal\{H\}\_\{t\}\\cup\\\{\(p\_\{t\+1\},s\_\{t\+1\},a\_\{t\+1\}\)\\\}\.Under budgetTT, EFE returns the best program in the final population:

p⋆∈arg⁡max\(p,s,a\)∈ℋT⁡s\.p^\{\\star\}\\in\\arg\\max\_\{\(p,s,a\)\\in\\mathcal\{H\}\_\{T\}\}s\.Thus, the search strategy determines which candidates are reused, how they are varied, and which prior examples are shown to the generator\.

### 2\.3Specialization to EFE\-Time

For time\-series forecasting,𝒟=\{y1:Ti\(i\)\}i=1M\\mathcal\{D\}=\\\{y^\{\(i\)\}\_\{1:T\_\{i\}\}\\\}\_\{i=1\}^\{M\}and each validation instance is a rolling forecast windowj=\(i,t\)j=\(i,t\)with historyy1:t\(i\)y^\{\(i\)\}\_\{1:t\}and future targetyt\+1:t\+H\(i\)y^\{\(i\)\}\_\{t\+1:t\+H\}:

Djfit=y1:t\(i\),Djin=\(y1:t\(i\),H\),Yj=yt\+1:t\+H\(i\)\.D^\{\\mathrm\{fit\}\}\_\{j\}=y^\{\(i\)\}\_\{1:t\},\\qquad D^\{\\mathrm\{in\}\}\_\{j\}=\(y^\{\(i\)\}\_\{1:t\},H\),\\qquad Y\_\{j\}=y^\{\(i\)\}\_\{t\+1:t\+H\}\.The fixed downstream routine is a forecasterFθF\_\{\\theta\}\. A candidate program transforms the history, the forecaster predicts in the transformed space, and the program maps the forecast back:

z1:t\(i\)=transformp​\(y1:t\(i\);σj,p\),z^t\+1:t\+H\(i\)=Fθ​\(z1:t\(i\),H,c\(i\)\)y^t\+1:t\+H\(i\)=inverse​\_​transformp​\(z^t\+1:t\+H\(i\);σj,p\),z^\{\(i\)\}\_\{1:t\}=\\mathrm\{transform\}\_\{p\}\(y^\{\(i\)\}\_\{1:t\};\\sigma\_\{j,p\}\),\\quad\\widehat\{z\}^\{\(i\)\}\_\{t\+1:t\+H\}=F\_\{\\theta\}\(z^\{\(i\)\}\_\{1:t\},H,c^\{\(i\)\}\)\\quad\\widehat\{y\}^\{\(i\)\}\_\{t\+1:t\+H\}=\\mathrm\{inverse\\\_transform\}\_\{p\}\\left\(\\widehat\{z\}^\{\(i\)\}\_\{t\+1:t\+H\};\\sigma\_\{j,p\}\\right\),
Thus,postp≡inverse​\_​transformp\\mathrm\{post\}\_\{p\}\\equiv\\mathrm\{inverse\\\_transform\}\_\{p\}\. Because predictions are made in the transformed space, EFE\-Time restricts search to approximately invertible programs\. For example, a candidate may fit a recent medianmmand robust scaleqq, transformyyto\(y−m\)/q\(y\-m\)/q, let the forecaster predict in that normalized space, and invert predictions asq​z\+mqz\+m\. Using MASE as the selection loss,

stime​\(p\)=\(1−MASE​\(p\)MASE​\(pid\)−λτtime​τ​\(p\)\)​𝟏​\{p∈𝒫inv\}\.s\_\{\\mathrm\{time\}\}\(p\)=\\left\(1\-\\frac\{\\mathrm\{MASE\}\(p\)\}\{\\mathrm\{MASE\}\(p\_\{\\mathrm\{id\}\}\)\}\-\\lambda^\{\\mathrm\{time\}\}\_\{\\tau\}\\tau\(p\)\\right\)\\mathbf\{1\}\\\{p\\in\\mathcal\{P\}\_\{\\mathrm\{inv\}\}\\\}\.Typical EFE\-Time normalizations include robust scaling, variance\-stabilizations, history\-only detrending, seasonal adjustment, and changepoint\-aware rescaling computed from the observed history\.

### 2\.4Specialization to EFE\-Tab

For tabular prediction,𝒟=\{\(xi,yi\)\}i=1n\\mathcal\{D\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{n\}, and each validation instance is a fold\(Ijtr,Ijval\)\(I^\{\\mathrm\{tr\}\}\_\{j\},I^\{\\mathrm\{val\}\}\_\{j\}\):

Djfit=\(XIjtr,yIjtr\),Djin=\(XIjtr,yIjtr,XIjval\),Yj=yIjval\.D^\{\\mathrm\{fit\}\}\_\{j\}=\(X\_\{I^\{\\mathrm\{tr\}\}\_\{j\}\},y\_\{I^\{\\mathrm\{tr\}\}\_\{j\}\}\),\\qquad D^\{\\mathrm\{in\}\}\_\{j\}=\(X\_\{I^\{\\mathrm\{tr\}\}\_\{j\}\},y\_\{I^\{\\mathrm\{tr\}\}\_\{j\}\},X\_\{I^\{\\mathrm\{val\}\}\_\{j\}\}\),\\qquad Y\_\{j\}=y\_\{I^\{\\mathrm\{val\}\}\_\{j\}\}\.A candidate program fits its state on the training fold and applies the same state to both training and validation features:

σj,p=fitp​\(XIjtr,yIjtr;c\),X~Ijtr,p=transformp​\(XIjtr;σj,p\),X~Ijval,p=transformp​\(XIjval;σj,p\)\.\\sigma\_\{j,p\}=\\mathrm\{fit\}\_\{p\}\(X\_\{I^\{\\mathrm\{tr\}\}\_\{j\}\},y\_\{I^\{\\mathrm\{tr\}\}\_\{j\}\};c\),\\qquad\\widetilde\{X\}\_\{\{I\_\{j\}^\{\\mathrm\{tr\}\}\},p\}=\\mathrm\{transform\}\_\{p\}\(X\_\{I\_\{j\}^\{\\mathrm\{tr\}\}\};\\sigma\_\{j,p\}\),\\qquad\\widetilde\{X\}\_\{\{I\_\{j\}^\{\\mathrm\{val\}\}\},p\}=\\mathrm\{transform\}\_\{p\}\(X\_\{I\_\{j\}^\{\\mathrm\{val\}\}\};\\sigma\_\{j,p\}\)\.The fixed learner𝒜\\mathcal\{A\}is trained on the transformed training fold and evaluated on the transformed validation fold:

fj,p=𝒜​\(X~Ijtr,p,yIjtr\),y^Ijval,p=fj,p​\(X~Ijval,p\)\.f\_\{j,p\}=\\mathcal\{A\}\\left\(\\widetilde\{X\}\_\{I^\{\\mathrm\{tr\}\}\_\{j\},p\},y\_\{I^\{\\mathrm\{tr\}\}\_\{j\}\}\\right\),\\qquad\\widehat\{y\}\_\{I^\{\\mathrm\{val\}\}\_\{j\},p\}=f\_\{j,p\}\(\\widetilde\{X\}\_\{I^\{\\mathrm\{val\}\}\_\{j\},p\}\)\.Herepostp≡id\\mathrm\{post\}\_\{p\}\\equiv\\mathrm\{id\}because predictions are already in the original target space\.

For binary classification, EFE\-Tab uses mean AUC improvement over the identity program:

δ¯​\(p\)=1K​∑j=1K\[AUCj​\(p\)−AUCj​\(pid\)\]\.\\bar\{\\delta\}\(p\)=\\frac\{1\}\{K\}\\sum\_\{j=1\}^\{K\}\\left\[\\mathrm\{AUC\}\_\{j\}\(p\)\-\\mathrm\{AUC\}\_\{j\}\(p\_\{\\mathrm\{id\}\}\)\\right\]\.The final score is

stab​\(p\)=δ¯​\(p\)−λf​k​\(p\)n−λτtab​τ​\(p\),s\_\{\\mathrm\{tab\}\}\(p\)=\\bar\{\\delta\}\(p\)\-\\lambda\_\{f\}\\sqrt\{\\frac\{k\(p\)\}\{n\}\}\-\\lambda^\{\\mathrm\{tab\}\}\_\{\\tau\}\\tau\(p\),wherek​\(p\)k\(p\)counts generated and dropped features, andτ​\(p\)\\tau\(p\)is runtime\. This favors feature\-engineering programs whose performance gains justify their added complexity\.

## 3Experiments

We evaluate the selected programsp⋆p^\{\\star\}returned by EFE in two settings: time\-series forecasting and tabular prediction\. EFE\-Time evolves approximately invertible programsppfor fixed time\-series foundation models, while EFE\-Tab evolves compact feature programs for tabular classifiers\. Since LLM\-based tabular feature generation has already been studied extensively\(Hollmann et al\.,[2023](https://arxiv.org/html/2607.01548#bib.bib11); Nam et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib19); Abhyankar et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib1)\), our tabular experiments focus on whether EFE can find small, useful programs that improve validation performance enough to justify their complexity\.111Code is available at[https://github\.com/egetaga/EFE](https://github.com/egetaga/EFE)\.

### 3\.1Experimental Setup

Evolutionary Optimization Interface\.Both EFE\-Time and EFE\-Tab are implemented on top of OpenEvolve\(Sharma,[2025](https://arxiv.org/html/2607.01548#bib.bib23)\), an open\-source evolutionary program\-search system inspired by AlphaEvolve\(Novikov et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib21)\)\. We use its code\-diversity coordinate as the MAP\-Elites feature and useClaude\-Opus\-4\.6as the LLM backbone in all main experiments\. All runs start frompidp\_\{\\mathrm\{id\}\}\. At each iteration, the LLM receives parent code, selected inspiration programs, score metadata, dataset context, and aggregate evaluator feedback\. The proposed programpt\+1p\_\{t\+1\}is executed, checked for validity and leakage, inserted before a fixed downstream model, and scored againstpidp\_\{\\mathrm\{id\}\}\. Thus, evolution changes only the preprocessing program, not the downstream model\.

Table 1:Comparison of EFE\-Time against identity transformation across time\-series datasets for Chronos\-2\.EFE\-Time implementation details\.For EFE\-Time, each dataset has a fixed dataset context computed from the training split\. It summarizes metadata such as frequency, prediction length, seasonal period, number of evaluation series, and aggregate series statistics\. At each iteration, valid candidatesppreceive anevaluation\_summarycontaining aggregate validation metrics for the identity baseline and candidate, metric ratios, error counts, timing information, and the fraction of evaluated series helped or harmed\. Failed candidates receive capped transformation or inverse transformation errors\. The prompt never includes raw time\-series values, forecasts, per\-series errors, or sampled evaluation subsets\.

Each candidateppis aTransformProgramwithfit,transform, andinverse\_transform\. At execution time,fitreceives only one historical seriesDjfitD^\{\\mathrm\{fit\}\}\_\{j\}and compact summary metadata, returningσj,p\\sigma\_\{j,p\}for the forward and inverse transformations\. The evaluator rejects candidates that change the series length, introduce invalid values, or collapse nonconstant series\. Candidates receive no future targetsYjY\_\{j\}, no data from other series, no baseline forecasts, no scoring feedback, no evolution step, and no dataset identity at execution time\.

EFE\-Tab implementation details\.For EFE\-Tab, each iteration forms a validation instancejjby splitting the training data into a fitting pool \(Pool A\) and held\-out scoring pool \(Pool B\)\. The dataset context is computed from Pool A and includes the dataset name, shape, target column, problem type, class balance, task description, column statistics, correlations, and a small number of sample rows\. Because Pool A rotates across iterations, these summaries may vary slightly during evolution\.

EFE\-Tab evolves thefit/transformlogic of aFeatureProgram\. Programs learnσj,p\\sigma\_\{j,p\}only from Pool A and apply the same fitted state to both Pool A and Pool B\. Candidates may create interactions, ratios, nonlinear transforms, binned features, target encodings, group aggregates, or threshold indicators, and may also drop weak or redundant original columns\.

Candidates are first checked for required entry points, valid outputs, schema consistency, and absence of in\-place dataframe mutation\. Valid programs are then scored on Pool B with𝒜\\mathcal\{A\}set to TabPFN: the model is fit on raw Pool A features to obtain thepidp\_\{\\mathrm\{id\}\}ROC\-AUC on Pool B, and then refit on the candidate\-engineered Pool A features to obtain the candidate ROC\-AUC on transformed Pool B\. The evaluator reports to the LLM the baseline and candidate AUCs,stab​\(p\)s\_\{\\mathrm\{tab\}\}\(p\), generated columns, dropped columns, output\-column counts, and validation status\. The evaluator also returns interpretable feedback to the LLM, computed only from Pool A: permutation\-importance labels for surviving features, correlations between features in the table, a readable decision tree trained on the candidate representation, and an original\-column importance report\.

Benchmarks and evaluation\.We evaluate EFE\-Time on ten GIFT\-Eval datasets\(Aksu et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib3)\)spanning healthcare, energy, finance and business, weather and nature, transportation, and web/cloud operations\. Programs are evolved against Chronos\-2\(Ansari et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib4)\)using validation MASE improvement overpidp\_\{\\mathrm\{id\}\}as the selection signal\. For each dataset and seed, we freezep⋆p^\{\\star\}and evaluate it on the held\-out test split\. We also test cross\-model transfer by applying the same frozenp⋆p^\{\\star\}, without re\-evolution, to TimesFM 2\.5, Moirai 2\-Small, and Reverso\-Nano\(Das et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib7); Liu et al\.,[2026a](https://arxiv.org/html/2607.01548#bib.bib16); Fu et al\.,[2026](https://arxiv.org/html/2607.01548#bib.bib9)\)\. We run three independent seeds for 100 iterations and report mean MASE, wQL, and MAE with standard deviations\.

![Refer to caption](https://arxiv.org/html/2607.01548v1/x4.png)\(a\)Over three datasets
![Refer to caption](https://arxiv.org/html/2607.01548v1/x5.png)\(b\)Over entire benchmark set

Figure 3:Cross\-model transfer of EFE\-Time\. The programs were evolved against Chronos\-2 only and reused without modification on TimesFM\-2\.5, Moirai\-2\-Small, and Reverso\-Nano\. \(a\) Average percent improvement across Covid Deaths, M4\-yearly, and solar hourly\. \(b\) Same setup averaged across all 10 GIFT\-Eval datasets in Table[1](https://arxiv.org/html/2607.01548#S3.T1)\. We report average scores across three replicates\.We evaluate EFE\-Tab on nine binary\-classification datasets from TabArena\(Erickson et al\.,[2026](https://arxiv.org/html/2607.01548#bib.bib8)\), spanning telecommunication churn, employment, e\-commerce, banking, healthcare, and sports\. During evolution, TabPFN\-v2\(Grinsztajn et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib10)\)is the scoring model\. After evolution, the selected programs are frozen and evaluated with TabPFN\-v2\(Grinsztajn et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib10)\), LightGBM, and single decision trees\. To control LLM API cost, we use the three folds from the first official repetition, run EFE\-Tab for each fold for 100 iterations, and report averages and standard deviation over folds\. We ran CAAFE and LLM\-FE in the same way for 100 iterations for each fold\.

As API costs prevented us from conducting experiments on the entirety of Gift\-Eval and TabArena, we used selected datasets from these benchmarks, chosen to ensure domain diversity\.

### 3\.2EFE\-Time Experiments

We begin by demonstrating that time\-series foundation models struggle with synthetic data containing exponential trends, and that the learned EFE\-Time programs help mitigate this issue\. We then show that EFE\-Time improves Chronos\-2 across a range of benchmarks\. Next, we demonstrate that the same learned normalizations, optimized using Chronos\-2, transfer to other time\-series foundation models\. Finally, we show that EFE\-Time provides benefits comparable to fine\-tuning Chronos\-2, and in some datasets even exceeds them\. More importantly, EFE\-Time and fine\-tuning provide additive gains: combining the two improves performance beyond either approach alone\. Overall, these results show that learning data\-specific normalizations can significantly improve TSFM performance\.

#### 3\.2\.1TSFMs Struggle with Multiplicative Growth Patterns

We generate synthetic time series that exhibit smooth exponential growth without spikes or abrupt events, the generation process detailed in Appendix[C\.6](https://arxiv.org/html/2607.01548#A3.SS6)\. Each sequence is sampled by choosing a random initial value and total growth factor, then constructing a smoothly varying monotone growth trajectory with smooth multiplicative observation noise\. This produces a clean toy exponential\-growth dataset in which the latent trend is monotone, while the observed sequence contains only smooth multiplicative deviations around that trend\. We sample 100 examples of length 5,000, reserving the final 128 time steps of each series for testing\.

As shown in Figure[2\(b\)](https://arxiv.org/html/2607.01548#S1.F2.sf2), Chronos\-2, TimesFM\-2\.5, and Moirai\-2\.0 struggle with this type of multiplicative\-growth data\. Despite the smoothness of the observed sequences and the monotonicity of the latent trend, all three models obtain poor MASE scores\. By contrast, Figure[2\(b\)](https://arxiv.org/html/2607.01548#S1.F2.sf2)shows that the EFE\-Time programp⋆p^\{\\star\}, optimized using Chronos\-2, substantially improves evaluation performance across all TSFMs considered\. Figure[2\(a\)](https://arxiv.org/html/2607.01548#S1.F2.sf1)illustrates why this happens: the program first transforms the original series into an approximately stationary space, where Chronos\-2 can produce a more accurate forecast\. Applying the inverse transformation then maps this forecast back to the original scale, closely matching the ground\-truth trajectory\.

![Refer to caption](https://arxiv.org/html/2607.01548v1/x6.png)\(a\)Over three datasets
![Refer to caption](https://arxiv.org/html/2607.01548v1/x7.png)\(b\)Over entire benchmark set

Figure 4:EFE\-Time, Fine\-tuning, and EFE\-Time\+Fine\-tuning gains over the Chronos\-2 zero\-shot forecast\. \(a\) Average percent improvement across Covid Deaths, M4\-yearly, and solar hourly\. \(b\) Same metrics averaged across 10 GIFT\-Eval datasets\. In both cases, combining the evolved time\-series transform with fine\-tuning \(EFE\-T\. \+ F\.T\.\) yields the largest and additive gains on every metric, outperforming either component alone\.
#### 3\.2\.2Dataset Specific Normalizations Improve TSFMs

Learning Normalization Programs for Chronos 2\.We now evaluate EFE\-Time on real datasets from the Gift\-Eval benchmark\(Aksu et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib3)\)\. We begin by comparing the selected programs with Chronos\-2 as the downstream forecaster\. As shown in Table[1](https://arxiv.org/html/2607.01548#S3.T1), EFE\-Time yields average improvements of3\.0%3\.0\\%,3\.6%3\.6\\%, and3\.7%3\.7\\%in MASE, WQL, and MAE, respectively, compared to the raw Chronos\-2 forecasts, i\.e\.,pidp\_\{\\mathrm\{id\}\}\. Another observation is that the magnitude of improvement varies substantially across datasets: for Covid Deaths, Solar \(hourly\), and M4\-Yearly, the improvement is as high as19%19\\%, whereas forkdd\_cup\_2018\(hourly\), we observe an improvement of approximately3%3\\%\. For several other datasets, the gains are more modest, close to0%0\\%, and statistically insignificant\. This is expected, as not all datasets benefit from normalization, especially when they are already relatively regular or when normalization does not substantially reduce their forecasting difficulty\. We visualize the raw Chronos\-2 forecasts, the forecasts in the transformed space, and their inversion back to the original space in Figures[10](https://arxiv.org/html/2607.01548#A5.F10),[11](https://arxiv.org/html/2607.01548#A5.F11),[12](https://arxiv.org/html/2607.01548#A5.F12),[13](https://arxiv.org/html/2607.01548#A5.F13), and[14](https://arxiv.org/html/2607.01548#A5.F14)\.

Cross\-Model Transfer of Learned Normalization Programs\.We investigate whether the learned dataset\-specific normalizations transfer to other time\-series foundation models\. To this end, we evaluate TimesFM\-2\.5, Moirai\-2\-small \(the other Moirai\-2 variants are not open\-sourced\), and Reverso\-Nano\(Fu et al\.,[2026](https://arxiv.org/html/2607.01548#bib.bib9)\)\. We chose Reverso\-Nano because it is extremely lightweight compared to the other TSFMs considered, allowing us to investigate whether the improvements persist for smaller TSFMs as well\. We use the same programs learned using Chronos\-2, without modification\.

As shown in Figure[3](https://arxiv.org/html/2607.01548#S3.F3), the benefits of learned normalization programs transfer to other TSFMs\. Specifically, on Covid Deaths, M4\-Yearly, and Solar \(hourly\), shown in Figure[3\(a\)](https://arxiv.org/html/2607.01548#S3.F3.sf1), we observe an average MASE improvement of more than6%6\\%across all TSFMs, reaching7%7\\%for TimesFM\. Over the entire benchmark set, shown in Figure[3\(b\)](https://arxiv.org/html/2607.01548#S3.F3.sf2), we again observe consistent MASE improvements of3\.06%3\.06\\%,2\.25%2\.25\\%, and2\.42%2\.42\\%for TimesFM, Moirai\-2\-small, and Reverso\-Nano, respectively\. Overall, the consistent improvements achieved by these TSFMs on the same datasets, using programs learned with Chronos\-2, suggest that the forecasting difficulty of these datasets is not specific to a particular model architecture\. Rather, it arises from the underlying time\-series dynamics, which EFE\-Time makes more amenable to forecasting\.

Fine\-tuning and Normalizations are Additive\.Both fine\-tuning and learning EFE\-Time programs use the training portion of the data\. In Figure[4](https://arxiv.org/html/2607.01548#S3.F4), we investigate how they compare and whether their benefits are additive; that is, whether first learning normalizations and then fine\-tuning the model in the normalized space yields substantial gains\. On datasets where normalization is highly effective, Figure[4\(a\)](https://arxiv.org/html/2607.01548#S3.F4.sf1)demonstrates that EFE\-Time outperforms fine\-tuning on every metric\. On the entire benchmark set, shown in Figure[4\(b\)](https://arxiv.org/html/2607.01548#S3.F4.sf2), we observe that EFE\-Time provides larger gains than fine\-tuning for MASE, but slightly lower gains for wQL and MAE\. Importantly, as shown in Figure[4](https://arxiv.org/html/2607.01548#S3.F4), EFE\-Time\+fine\-tuning yields additive benefits in both settings, outperforming either approach\.

Table 2:Comparison of feature engineering methods for single decision trees across tabular datasets over ROC\-AUC\. The reported results are 3 outer\-fold averages from the TabArena benchmark\.Learning EFE\-Time Programs Requires Strong LLMs\.Appendix[C\.8](https://arxiv.org/html/2607.01548#A3.SS8)evaluates EFE\-Time with the Qwen3\.5 family\(Yang et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib24)\)on Covid\-Deaths and Solar Hourly\. Since these models are substantially weaker than Opus\-4\.6, we ran each experiment for 300 iterations\. Qwen models perform reasonably well on Solar Hourly but are much less stable on Covid\-Deaths\. Given the difficulty of generating full transform and inverse\-transform pipelines, this performance drop is expected\. We believe that adapting EFE\-Time to smaller, non\-frontier LLMs remains a promising direction\.

### 3\.3EFE\-Tab Experiments

Learning Parsimonious Feature Programs\.Learning feature\-engineering programs for tabular data with LLMs has been widely studied\(Hollmann et al\.,[2023](https://arxiv.org/html/2607.01548#bib.bib11); Abhyankar et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib1); Nam et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib19)\)\. Thus, our main goal here is to show that the same EFE paradigm can learn parsimonious programsp⋆p^\{\\star\}, where each added or removed feature must yield a benefit greater than its associated penalty\. We show that this approach improves decision trees while providing a significant interpretability advantage in real\-world use\.

We evaluate the EFE\-Tab programsp⋆p^\{\\star\}in Table[2](https://arxiv.org/html/2607.01548#S3.T2)and compare it with no feature engineering, CAAFE, and LLM\-FE, on a single decision tree\. We see that EFE\-Tab yields mean rank of1\.391\.39among 4 approaches when used with a single decision tree with the closest method after it achieving a mean rank of2\.442\.44\. We see in Figure[5](https://arxiv.org/html/2607.01548#S3.F5)that EFE\-Tab on our tabular benchmark also outperforms LLM\-FE, and CAAFE on LightGBM and TabPFN downstream models\. The separation on LightGBM is high, whereas on TabPFN, each method achieves a similar mean rank\. During experiments, we have observed that due to its explicit parsimony penalty, EFE\-Tab yields compact programs, whereas LLM\-FE and CAAFE usually generate larger feature sets\. We hypothesize that TabPFN appears more robust to redundant features, yet a single decision tree, and also LightGBM has less so\. Thus, we see a lower separation between parsimonious feature engineering programs of EFE\-Tab and competing approaches in TabPFN\.

The Regimes where EFE\-Tab is Useful\.In Figure[6\(a\)](https://arxiv.org/html/2607.01548#S3.F6.sf1), we vary the decision tree depth and observe that EFE\-Tab programs have particularly strong benefit on shallower decision trees\. Yet, the separation exists in higher depths too, and more importantly only with depth 3, EFE\-Tab achieves better performance than the no feature engineered baseline with higher depths\. This signals that EFE\-Tab is particularly good for learning simple but effective decision rules, which are particularly important for real\-world applications requiring an explanation of the final decision\.

![Refer to caption](https://arxiv.org/html/2607.01548v1/x8.png)Figure 5:Mean rank of methods across our tabular benchmark\.In Figure[6\(b\)](https://arxiv.org/html/2607.01548#S3.F6.sf2), we quantify the effect of EFE\-Tab programs in low\-data regimes\. We use TabPFN here because it performs well in low\-data regimes and does not require hyperparameter tuning, which we observed to be unstable for decision trees and LightGBM when only small amounts of training data are available\. We observe that, with less training data, feature engineering significantly boosts the performance of TabPFN\. As the amount of data increases, this separation diminishes, demonstrating that generating meaningful features is particularly helpful for models in low\-data scenarios\.

Characterizing the Fraction of Training Data Used\.As described in Section[1](https://arxiv.org/html/2607.01548#S3.T1), at each iteration we instantiatejjby randomly splitting the training data into Pool A and Pool B\. Pool A summary is provided to the LLM and used as input to the downstream model, while Pool B is used for scoring\. This split naturally introduces diversity across iterations\. Figure[6\(c\)](https://arxiv.org/html/2607.01548#S3.F6.sf3)shows that EFE\-Tab improvements follow an inverted\-U trend as the fraction assigned to pool A increases\. Large pool A fractions reduce diversity across iterations, whereas very small fractions make pool A less representative of the underlying dataset\.

![Refer to caption](https://arxiv.org/html/2607.01548v1/x9.png)\(a\)Test AUC vs\. tree depth\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x10.png)\(b\)Test AUC vs\. training rows\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x11.png)\(c\)Δ\\DeltaAUC vs\. used training fraction\.

Figure 6:\(a\) Single decision tree over 3 folds×\\times5 seeds \(15 runs per cell\)\. \(b\) Test AUC vs\. training\-set size for TabPFN\. EFE\-Tab features deliver a larger lift in the low\-data regime, with the gap shrinking as more training data becomes available\. \(c\) AverageΔ\\DeltaAUC \(EFE\-Tab \- baseline\) over the held\-out test and validation sets vs\. the fraction of training data exposed to the evolution loop, averaged over TabPFN/LightGBM/decision\-tree and 3 seeds onchurnandin\_vehicle\_coupon\_recommendation\.

## 4Conclusion and Limitations

We introduced EFE, an evolutionary framework for feature engineering that iteratively refines feature programs\. EFE\-Time improves the forecasting performance of TSFMs across diverse downstream applications and achieves performance competitive with fine\-tuning\. More importantly, combining EFE\-Time with fine\-tuning yields additive gains, suggesting that learned normalizations and model adaptation provide complementary benefits\. EFE\-Tab, in turn, learns compact and parsimonious feature\-engineering programs that substantially improve simple decision trees while preserving interpretability\. Our paper also points to several promising directions for future work\. First, the evolutionary process currently uses fixed hyperparameters, such as the exploration–exploitation ratio or the number of evolutionary islands\. This design keeps the experimental protocol simple and controlled, but future work could study how these choices affect performance\. Second, the quality and structure of feedback play an important role in guiding program evolution\. A detailed ablation of which feedback components are most useful, and when they help or hinder refinement, would provide a clearer understanding of the mechanisms driving EFE’s improvements\.

## Acknowledgments

We thank Halil Alperen Gozeten of the University of Michigan for discussions on LLM\-based evolutionary discovery\. This work was supported by the National Science Foundation under grants CCF\-2046816, CCF\-2403075, and CCF\-2212426; by the Office of Naval Research under grant N000142412289; and by Los Alamos National Laboratory under grant AWD026741 at the University of Michigan\. The computational aspects of this research were generously supported by resources provided through the Amazon Research Award on Foundation Model Development\.

## References

- Abhyankar et al\. \[2025\]Nikhil Abhyankar, Parshin Shojaee, and Chandan K Reddy\.Llm\-fe: Automated feature engineering for tabular data with llms as evolutionary optimizers\.*arXiv preprint arXiv:2503\.14434*, 2025\.
- Agrawal et al\. \[2025\]Lakshya A Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl\-Ong, Arnav Singhvi, Herumb Shandilya, Michael J Ryan, Meng Jiang, et al\.Gepa: Reflective prompt evolution can outperform reinforcement learning\.*arXiv preprint arXiv:2507\.19457*, 2025\.
- Aksu et al\. \[2024\]Taha Aksu, Gerald Woo, Juncheng Liu, Xu Liu, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo\.Gift\-eval: A benchmark for general time series forecasting model evaluation\.*arxiv preprint arxiv:2410\.10393*, 2024\.
- Ansari et al\. \[2025\]Abdul Fatir Ansari, Oleksandr Shchur, Jaris Küken, Andreas Auer, Boran Han, Pedro Mercado, Syama Sundar Rangapuram, Huibin Shen, Lorenzo Stella, Xiyuan Zhang, et al\.Chronos\-2: From univariate to universal forecasting\.*arXiv preprint arXiv:2510\.15821*, 2025\.
- Aygün et al\. \[2025\]Eser Aygün, Anastasiya Belyaeva, Gheorghe Comanici, Marc Coram, Hao Cui, Jake Garrison, Renee Johnston Anton Kast, Cory Y McLean, Peter Norgaard, Zahra Shamsi, et al\.An ai system to help scientists write expert\-level empirical software\.*arXiv preprint arXiv:2509\.06503*, 2025\.
- Cemri et al\. \[2026\]Mert Cemri, Shubham Agrawal, Akshat Gupta, Shu Liu, Audrey Cheng, Qiuyang Mang, Ashwin Naren, Lutfi Eren Erdogan, Koushik Sen, Matei Zaharia, et al\.Adaevolve: Adaptive llm driven zeroth\-order optimization\.*arXiv preprint arXiv:2602\.20133*, 2026\.
- Das et al\. \[2024\]Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou\.A decoder\-only foundation model for time\-series forecasting\.In*International Conference on Machine Learning*, pages 10148–10167\. PMLR, 2024\.
- Erickson et al\. \[2026\]Nick Erickson, Lennart Purucker, Andrej Tschalzev, David Holzmüller, Prateek Desai, David Salinas, and Frank Hutter\.Tabarena: A living benchmark for machine learning on tabular data\.*Advances in Neural Information Processing Systems*, 38, 2026\.
- Fu et al\. \[2026\]Xinghong Fu, Yanhong Li, Georgios Papaioannou, and Yoon Kim\.Reverso: Efficient time series foundation models for zero\-shot forecasting, 2026\.URL[https://arxiv\.org/abs/2602\.17634](https://arxiv.org/abs/2602.17634)\.
- Grinsztajn et al\. \[2025\]Léo Grinsztajn, Klemens Flöge, Oscar Key, Felix Birkel, Philipp Jund, Brendan Roof, Benjamin Jäger, Dominik Safaric, Simone Alessi, Adrian Hayler, et al\.Tabpfn\-2\.5: Advancing the state of the art in tabular foundation models\.*arXiv preprint arXiv:2511\.08667*, 2025\.
- Hollmann et al\. \[2023\]Noah Hollmann, Samuel Müller, and Frank Hutter\.Large language models for automated data science: Introducing caafe for context\-aware automated feature engineering\.In A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine, editors,*Advances in Neural Information Processing Systems*, volume 36, pages 44753–44775\. Curran Associates, Inc\., 2023\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2023/file/8c2df4c35cdbee764ebb9e9d0acd5197\-Paper\-Conference\.pdf](https://proceedings.neurips.cc/paper_files/paper/2023/file/8c2df4c35cdbee764ebb9e9d0acd5197-Paper-Conference.pdf)\.
- Horn et al\. \[2019\]Franziska Horn, Robert Pack, and Michael Rieger\.The autofeat python library for automated feature engineering and selection\.In*Joint European Conference on Machine Learning and Knowledge Discovery in Databases*, pages 111–120\. Springer, 2019\.
- Kanter and Veeramachaneni \[2015\]James Max Kanter and Kalyan Veeramachaneni\.Deep feature synthesis: Towards automating data science endeavors\.In*2015 IEEE international conference on data science and advanced analytics \(DSAA\)*, pages 1–10\. IEEE, 2015\.
- Khurana et al\. \[2016\]Udayan Khurana, Deepak Turaga, Horst Samulowitz, and Srinivasan Parthasrathy\.Cognito: Automated feature engineering for supervised learning\.In*2016 IEEE 16th international conference on data mining workshops \(ICDMW\)*, pages 1304–1307\. IEEE, 2016\.
- Kim et al\. \[2022\]Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang\-Ho Choi, and Jaegul Choo\.Reversible instance normalization for accurate time\-series forecasting against distribution shift\.In*International Conference on Learning Representations*, 2022\.URL[https://openreview\.net/forum?id=cGDAkQo1C0p](https://openreview.net/forum?id=cGDAkQo1C0p)\.
- Liu et al\. \[2026a\]Chenghao Liu, Taha Aksu, Juncheng Liu, Xu Liu, Hanshu Yan, Quang Pham, Silvio Savarese, Doyen Sahoo, Caiming Xiong, and Junnan Li\.Moirai 2\.0: When less is more for time series forecasting, 2026a\.URL[https://arxiv\.org/abs/2511\.11698](https://arxiv.org/abs/2511.11698)\.
- Liu et al\. \[2026b\]Shu Liu, Shubham Agarwal, Monishwaran Maheswaran, Mert Cemri, Zhifei Li, Qiuyang Mang, Ashwin Naren, Ethan Boneh, Audrey Cheng, Melissa Z Pan, et al\.Evox: Meta\-evolution for automated discovery\.*arXiv preprint arXiv:2602\.23413*, 2026b\.
- Murray et al\. \[2025\]Andrew Murray, Danial Dervovic, and Michael Cashmore\.Elate: Evolutionary language model for automated time\-series engineering, 2025\.URL[https://arxiv\.org/abs/2508\.14667](https://arxiv.org/abs/2508.14667)\.
- Nam et al\. \[2024\]Jaehyun Nam, Kyuyoung Kim, Seunghyuk Oh, Jihoon Tack, Jaehyung Kim, and Jinwoo Shin\.Optimized feature generation for tabular data via llms with decision tree reasoning\.*Advances in neural information processing systems*, 37:92352–92380, 2024\.
- Nargesian et al\. \[2017\]Fatemeh Nargesian, Horst Samulowitz, Udayan Khurana, Elias B Khalil, and Deepak S Turaga\.Learning feature engineering for classification\.In*Ijcai*, volume 17, pages 2529–2535, 2017\.
- Novikov et al\. \[2025\]Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po\-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al\.Alphaevolve: A coding agent for scientific and algorithmic discovery\.*arXiv preprint arXiv:2506\.13131*, 2025\.
- Romera\-Paredes et al\. \[2024\]Bernardino Romera\-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M\. Pawan Kumar, Emilien Dupont, Francisco J\. R\. Ruiz, Jordan S\. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi\.Mathematical discoveries from program search with large language models\.*Nature*, 625\(7995\):468–475, 2024\.doi:10\.1038/s41586\-023\-06924\-6\.URL[https://doi\.org/10\.1038/s41586\-023\-06924\-6](https://doi.org/10.1038/s41586-023-06924-6)\.
- Sharma \[2025\]Asankhaya Sharma\.Openevolve: an open\-source evolutionary coding agent, 2025\.URL[https://github\.com/algorithmicsuperintelligence/openevolve](https://github.com/algorithmicsuperintelligence/openevolve)\.
- Yang et al\. \[2025\]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu\.Qwen3 technical report\.*arXiv preprint arXiv:2505\.09388*, 2025\.
- Zhang et al\. \[2019\]Jianyu Zhang, Jianye Hao, Françoise Fogelman\-Soulié, and Zan Wang\.Automatic feature engineering by deep reinforcement learning\.In*Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems*, pages 2312–2314, 2019\.

## Appendix AExtended Related Work

Classical Automated Feature Engineering Methods\.Early work on automated feature engineering for tabular data has largely relied on predefined transformation operators combined with search or selection strategies\. Deep Feature Synthesis generates features from relational data by applying aggregation and transformation primitives\[Kanter and Veeramachaneni,[2015](https://arxiv.org/html/2607.01548#bib.bib13)\], whereas Cognito uses a greedy search procedure to explore feature transformations that improve supervised learning performance\[Khurana et al\.,[2016](https://arxiv.org/html/2607.01548#bib.bib14)\]\. AutoFeat constructs and selects nonlinear feature combinations\[Horn et al\.,[2019](https://arxiv.org/html/2607.01548#bib.bib12)\], while learning\-based methods, including LFE and reinforcement\-learning approaches, aim to automatically identify useful transformations\[Nargesian et al\.,[2017](https://arxiv.org/html/2607.01548#bib.bib20), Zhang et al\.,[2019](https://arxiv.org/html/2607.01548#bib.bib25)\]\. Despite their effectiveness, these methods typically make limited use of rich dataset\-level context, such as column semantics or task descriptions, and are instead primarily guided by fixed operators and heuristic search procedures\.

Evolutionary Optimization With LLMs\.Recent studies have investigated the use of LLMs to guide evolutionary optimization for discovering programs, prompts, and algorithms\. FunSearch evolves candidate functions by combining LLM\-generated proposals with evaluator feedback\[Romera\-Paredes et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib22)\], while AlphaEvolve extends this paradigm to full\-program optimization through iterative code modification and evaluation\[Novikov et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib21)\]\. OpenEvolve offers an open\-source framework for this code search\[Sharma,[2025](https://arxiv.org/html/2607.01548#bib.bib23)\]\. Other approaches further adapt the search process itself: EvoX evolves both candidate solutions and search strategies\[Liu et al\.,[2026b](https://arxiv.org/html/2607.01548#bib.bib17)\], and AdaEvolve leverages LLM feedback for adaptive zeroth\-order optimization\[Cemri et al\.,[2026](https://arxiv.org/html/2607.01548#bib.bib6)\]\. LLM\-based search has also been applied to scientific software generation\[Aygün et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib5)\]and reflective prompt evolution\[Agrawal et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib2)\]\. However, these works primarily focus on general\-purpose program, prompt, or algorithm discovery, rather than feature engineering for structured data\.

Evolutionary Feature Engineering for Structured Data with LLMs\.More recent work has explored the use of LLMs for automated feature engineering on structured data\. CAAFE iteratively prompts an LLM with dataset descriptions to propose semantically meaningful feature interactions and selects features one at a time based on changes in cross\-validation performance\[Hollmann et al\.,[2023](https://arxiv.org/html/2607.01548#bib.bib11)\]\. OCTree combines LLM reasoning with feedback from shallow decision trees fitted to the data, a direction that also informs our use of tree\-based dataset reasoning\[Nam et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib19)\]\. Most closely related to our work, LLM\-FE adopts a FunSearch\-style evolutionary search framework that incorporates dataset context to optimize tabular feature\-transformation programs\[Abhyankar et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib1), Romera\-Paredes et al\.,[2024](https://arxiv.org/html/2607.01548#bib.bib22)\]\. However, LLM\-FE evolves a single transformation function, rather than an AlphaEvolve\-style full program that includes state fitted on the training data and an explicit inverse transformation\[Novikov et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib21)\]\. In the time\-series setting, ELATE uses evolutionary LLM search to generate predictive covariates, but it does not focus on invertible normalizations, and no open\-source implementation is available for direct comparison\[Murray et al\.,[2025](https://arxiv.org/html/2607.01548#bib.bib18)\]\. To the best of our knowledge, our work is the first to evolve invertible time\-series normalization modules\.

## Appendix BBroader Impact

We develop a methodology for LLM\-based evolutionary feature engineering\. A potential risk of using LLMs for automated feature engineering is that they may introduce bias into the preprocessed data representations\. This risk is relatively limited for EFE\-Time, which learns invertible time\-series normalizations, but is more pronounced for EFE\-Tab, which operates directly on tabular features\. In particular, semantic cues from column names may lead the LLM to generate spurious or discriminatory features\. Therefore, EFE\-Tab should be supervised carefully, and generated preprocessing programs should only be used after human review\.

## Appendix CAdditional Experimental Details

We give reproducibility details that are not explicit in the main text in Section[3\.1](https://arxiv.org/html/2607.01548#S3.SS1)\.

### C\.1Evolutionary optimization interface

The OpenEvolve\-specific configuration uses its island population and MAP\-Elites archive, with code\-diversity coordinates as the archive descriptor\. Candidate proposals are complete program rewrites rather than parameter updates\. The optimizer records both valid and invalid attempts so that later prompts can use failures as debugging feedback, but only valid programs are eligible for final selection\. After the evolution budget is exhausted, the selected source code is frozen and reloaded by a separate held\-out evaluator\.

Prompts contain the parent program, a small set of previous programs or summaries selected by the optimizer, dataset\-level context, and aggregate evaluator feedback\. They do not contain evaluator source code, private scoring logic, held\-out examples, or test\-set results\.

### C\.2EFE\-Time Evolution

Evolution Design\.Each time\-series candidate implementsfit,transform, andinverse\_transform\. At a forecast origin,fit\(y\_hist, meta\)receives a single observed history and a compact metadata dictionary\. The metadata keys are: prediction length, seasonal period, history length, number of valid observations, mean, standard deviation, median, minimum, maximum, range, positive fraction, zero fraction, skewness, coefficient of variation, trend strength, recent mean, and recent standard deviation\. The returned state dictionary must includehist\_len\. This is required becauseinverse\_transformis called on forecast\-horizon arrays rather than on the original history; any detrending, time\-varying scaling, or seasonal\-phase operation must therefore align forecast positions using the stored history length\.

The validity gate checks the forward and inverse transforms before forecasting\. A valid transform must preserve length, order, and NaN locations; avoid Inf values or new NaNs; remain numerically stable on forecast medians and quantiles; and avoid mapping nonconstant histories to constants\. The inverse must be well defined on arrays of arbitrary forecast length, not only on arrays of lengthhist\_len\.

The prompt\-visibledataset\_contextis fixed once per dataset from the training split\. It contains only aggregate information such as frequency, horizon, seasonal period, number of evaluation series, and distributional summaries of histories\. The per\-iterationevaluation\_summaryis also aggregate: valid candidates receive baseline and candidate metrics, metric ratios, valid\-series counts, error counts, timing, and help/harm rates; invalid candidates receive capped transformation or inverse\-transformation errors\. The LLM never sees raw series values, future values, sampled evaluation windows, model forecasts, per\-series errors, baseline forecasts, scoring code, or test\-set information\.

Hyperparameters\.All experiments use fixed Openevolve hyperparameters\. We used Openevolve evolution with 3 islands, with exploitation ratio of 0\.70, exploration ratio of 0\.10, and elite selection ratio of 0\.20\. We give the system prompt we have used at Section[G\.1](https://arxiv.org/html/2607.01548#A7.SS1)\. We have a code diversity axis with 8 bins\. We have used Claude Opus\-4\.6 through AWS API access with temperature of 0\.7\. While calculating the combined score, to let invalid programs live so that they could be fixed, we added an offset of 0\.5 to the combined score so that bad performing programs won’t get the same score as failing ones through validity checks\. Therefore, an identity program corresponds to a combined score of 0\.5\. For each dataset, we extracted 3 validation portions from the training portion of the data with non\-overlapping windows, using the given forecast length of the data\. At each Openevolve iteration, we use 1/3 of total extracted samples, matching the original dataset size\. For fine\-tuning experiments with Chronos\-2, we have used the official Chronos script from their github\. We used Chronos in its univariate mode, forecasting each time series independently\.

### C\.3EFE\-Tab Evolution

Evolution Design\.Beyond the Pool A/Pool B protocol described in Section[3\.1](https://arxiv.org/html/2607.01548#S3.SS1), the tabular evaluator exposes a structureddataset\_contextcomputed only from Pool A\. It includes the dataset name, shape, target column, problem type, class balance, task description, column\-level statistics, target correlations, top inter\-feature correlations, and a small number of Pool A rows\. For numerical columns, the context reports ranges, quartiles, skewness, cardinality, and missingness, while for non\-numerical columns, it reports cardinality, top values, and missingness\. Since Pool A rotates across iterations, these summaries vary slightly during evolution\.

The stage\-one validity gate checks that the program has valid function signatures, returns DataFrames for both pools, preserves row alignment, avoids in\-place mutation of the input frames, and produces a schema that can be applied consistently to Pool A and Pool B\. Target\-dependent operations such as target encodings or group aggregates are permitted only through state fitted on Pool A and then reused unchanged on Pool B\.

For valid candidates,stage2\_summaryreports the baseline AUC, candidate AUC, AUC difference, combined fitness score, pool sizes, iteration seed, newly created columns, dropped original columns, output\-column counts, and validation status\. The evaluator may also return Pool\-A\-only interpretability feedback, including permutation\-importance labels, an original\-column importance report, and a readable decision tree trained on the candidate representation\. Pool B is visible to the LLM only through aggregate AUC values and their difference\. Raw or transformed Pool B rows, per\-row predictions, per\-row losses are not exposed\.

The complexity term counts both added columns and dropped original columns as feature modifications\. This makes the objective prefer feature programs whose held\-out AUC gain is large enough to justify their additional representation cost and runtime\.

Hyperparameters\.Just like EFE\-Time, all experiments use fixed Openevolve hyperparameters\. We used Openevolve evolution with 3 islands, with exploitation ratio of 0\.70, exploration ratio of 0\.10, and elite selection ratio of 0\.20\. Thus, all the hyperparameters with respect to Openevolve are the same with EFE\-Time\. During evolution, we used 70% of the training data for Pool A, and the rest 30% of the data for pool B\. At each Openevolve iteration, pool A and pool B are resampled again\.

### C\.4Benchmarks and Reporting Conventions

For EFE\-Time, each dataset is evolved under three random seeds; reported MASE, wQL, and MAE values are means and standard deviations across those runs\. Transfer experiments reuse the frozen programs learned with Chronos\-2 without re\-evolving or tuning them for the target forecaster\.

For EFE\-Tab, TabArena provides three repetitions of three\-fold splits for the medium\-sized datasets\. Using all repetitions with three EFE seeds would require2727evolution/evaluation runs per dataset, so the main benchmark uses the three folds from the first official repetition\. Baseline comparisons against CAAFE and LLM\-FE use the same 100\-iteration LLM\-query budget as EFE\-Tab\. Frozen feature programs are then evaluated with TabPFN, LightGBM, and single decision trees\.

### C\.5The List of Datasets Used

Table 3:EFE\-Time datasets used in the evaluation\.Table 4:The TabArena datasets used in the EFE\-Tab evaluation\.
### C\.6Synthetic Smooth Exponential Time\-Series Generator

We generate synthetic exponential\-growth time series designed to isolate smooth multiplicative growth from transient spike effects\.

For each series, let the sequence length beLLand defineT=L−1T=L\-1, with time index

t∈\{0,1,…,T\}\.t\\in\\\{0,1,\\ldots,T\\\}\.We first sample a starting valuessand an overall growth ratiorrlog\-uniformly:

s∼LogUniform​\(smin,smax\),s\\sim\\mathrm\{LogUniform\}\(s\_\{\\min\},s\_\{\\max\}\),r∼LogUniform​\(rmin,rmax\)\.r\\sim\\mathrm\{LogUniform\}\(r\_\{\\min\},r\_\{\\max\}\)\.Equivalently,

log⁡s∼Uniform​\(log⁡smin,log⁡smax\),\\log s\\sim\\mathrm\{Uniform\}\(\\log s\_\{\\min\},\\log s\_\{\\max\}\),log⁡r∼Uniform​\(log⁡rmin,log⁡rmax\)\.\\log r\\sim\\mathrm\{Uniform\}\(\\log r\_\{\\min\},\\log r\_\{\\max\}\)\.The total log\-growth over the full sequence is therefore

The generator constructs a monotone trajectory in log space\. Let

ηi=SmoothNoise​\(T,σ\),i=1,…,T,\\eta\_\{i\}=\\mathrm\{SmoothNoise\}\(T,\\sigma\),\\qquad i=1,\\ldots,T,whereSmoothNoise\\mathrm\{SmoothNoise\}denotes Gaussian white noise smoothed by convolution with a one\-dimensional Gaussian kernel of width controlled byσ\\sigma, then normalized to have approximately zero mean and unit root\-mean\-square magnitude\. We convert this smooth noise into positive growth weights:

wi=exp⁡\(γ​ηi\),w\_\{i\}=\\exp\(\\gamma\\eta\_\{i\}\),whereγ\\gammacontrols the volatility of the growth rate\. These weights are normalized so that the total log\-growth is exactlyGG:

Δ​ℓi=wi∑j=1Twj​G\.\\Delta\\ell\_\{i\}=\\frac\{w\_\{i\}\}\{\\sum\_\{j=1\}^\{T\}w\_\{j\}\}G\.The latent log\-trend is then

ℓ0=log⁡s,\\ell\_\{0\}=\\log s,ℓt=log⁡s\+∑i=1tΔ​ℓi,t=1,…,T\.\\ell\_\{t\}=\\log s\+\\sum\_\{i=1\}^\{t\}\\Delta\\ell\_\{i\},\\qquad t=1,\\ldots,T\.The corresponding latent exponential trend is

mt=exp⁡\(ℓt\)\.m\_\{t\}=\\exp\(\\ell\_\{t\}\)\.Because eachΔ​ℓi\\Delta\\ell\_\{i\}is positive, the latent trend is monotone increasing\. In addition, the normalization enforces exact endpoint control:

For reference, we also define the clean log\-linear exponential baseline

ct=s​rt/T,c\_\{t\}=sr^\{t/T\},which is the trajectory obtained by distributing the total log\-growth uniformly over time\.

Finally, the observable time series is obtained by applying smooth multiplicative observation noise\. Let

εt=α​SmoothNoise​\(L,σ/2\),\\varepsilon\_\{t\}=\\alpha\\,\\mathrm\{SmoothNoise\}\(L,\\sigma/2\),whereα\\alphacontrols the observation noise scale\. The final observed sequence is

xt=mt​exp⁡\(εt\)\.x\_\{t\}=m\_\{t\}\\exp\(\\varepsilon\_\{t\}\)\.Equivalently,

xt=exp⁡\(ℓt\+εt\)\.x\_\{t\}=\\exp\(\\ell\_\{t\}\+\\varepsilon\_\{t\}\)\.
Combining the above definitions, the generated series can be written as

xt=exp⁡\(log⁡s\+∑i=1texp⁡\(γ​ηi\)∑j=1Texp⁡\(γ​ηj\)​log⁡r\+εt\)x\_\{t\}=\\exp\\left\(\\log s\+\\sum\_\{i=1\}^\{t\}\\frac\{\\exp\(\\gamma\\eta\_\{i\}\)\}\{\\sum\_\{j=1\}^\{T\}\\exp\(\\gamma\\eta\_\{j\}\)\}\\log r\+\\varepsilon\_\{t\}\\right\)fort=0,…,Tt=0,\\ldots,T, with the convention that the summation term is zero whent=0t=0\.

In the experiments, the default parameter ranges are

s∼LogUniform​\(0\.5,2\.0\),s\\sim\\mathrm\{LogUniform\}\(0\.5,2\.0\),r∼LogUniform​\(5000,8000\),r\\sim\\mathrm\{LogUniform\}\(5000,8000\),γ∼Uniform​\(0\.2,1\.2\),\\gamma\\sim\\mathrm\{Uniform\}\(0\.2,1\.2\),α∼Uniform​\(0\.0,0\.08\),\\alpha\\sim\\mathrm\{Uniform\}\(0\.0,0\.08\),σ∼Uniform​\(8\.0,32\.0\)\.\\sigma\\sim\\mathrm\{Uniform\}\(8\.0,32\.0\)\.Each time series draws its own independent values ofss,rr,γ\\gamma,α\\alpha, andσ\\sigma\.

### C\.7Compute

Except for running local LLMs detailed in Section[C\.8](https://arxiv.org/html/2607.01548#A3.SS8), we have only used 4 L40S GPUs throughout experiments, with 8 CPU cores and 128 GB of memory\.

### C\.8Local\-model evolution

The main paper used a hosted frontier LLM to drive evolution\. To test whether the same loop also works with smaller open\-weight models, we repeated the experiment on𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}and𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}using four Qwen3\.5 sizes: 4B, 9B, 27B, and 35B\-A3B\. All non\-model settings were held fixed\. The 4B and 9B models were served locally with vLLM on one RTX A6000, while 27B and 35B\-A3B used two A6000s or a hosted Qwen endpoint\. Each setting was run for 300 iterations with three random seeds\.

Table[5](https://arxiv.org/html/2607.01548#A3.T5)reports the final optimizer\-selected program for each setting\. On𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}, every model size found useful MASE improvements, but larger models were not consistently better\. The 9B model had the largest mean MASE gain \(\+5\.3±3\.3%\+5\.3\\pm 3\.3\\%\), while 4B also improved and matched the larger models on its best seed\. However, some configurations produced poorly calibrated quantiles: 9B and 35B\-A3B both showed large WQL regressions, even when their median forecasts improved\. The 27B model was more stable in WQL but had the smallest MASE gain\. Overall, performance did not scale monotonically with model size\.

The𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}results were cleaner\. All four sizes improved MASE, WQL, and MAE on average, with no calibration failures like those seen on𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}\. The 9B and 35B\-A3B models performed best, reaching about\+11%\+11\\%mean MASE improvement, while 4B still achieved a strong mean gain of\+7\.8±6\.9%\+7\.8\\pm 6\.9\\%\. The 27B model was less strong but more consistent\. Much of the variance came from whether a seed discovered a high\-performing program: successful seeds reached a∼\\sim15%15\\%improvement regime, while weaker seeds stayed close to baseline\. As before, gains did not follow a smooth model\-size scaling trend\.

Yet, in the end, our results demonstrate that end\-to\-end program evolution, with our existing framework and large context sizes provided to LLMs, is not stable for non\-frontier, small\-scale models\.

Table 5:Model\-size scan of the EFE\-Time evolution loop on𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}and𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}\. All runs use 300 evolution iterations and three seeds; the reported program per seed is the optimizer’s combined\-score winner \(frozen atiteration\_found, evaluated on the full held\-out test split\)\. Each Chronos\-2 zero\-shot baseline is constant across sizes within the dataset block and is therefore reported once\. The 4B model is served locally via vLLM on a single RTX A6000 \(two A6000s for 27B and 35B\-A3B\); larger sizes are served through a hosted Qwen API endpoint\.LLMBaselineEFE\-Time% ImprovementMASEWQLMAEMASEWQLMAEMASEWQLMAE*Dataset:𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}*𝚀𝚠𝚎𝚗𝟹​\.5​\-​𝟺​𝙱\\mathtt\{Qwen3\.5\\text\{\-\}4B\}35\.46135\.4610\.0390\.039123\.880123\.88034\.116±2\.33434\.116\\pm 2\.3340\.035±0\.0070\.035\\pm 0\.007111\.051±22\.272111\.051\\pm 22\.2723\.8±6\.63\.8\\pm 6\.610\.4±17\.810\.4\\pm 17\.810\.4±18\.010\.4\\pm 18\.0𝚀𝚠𝚎𝚗𝟹​\.5​\-​𝟿​𝙱\\mathtt\{Qwen3\.5\\text\{\-\}9B\}33\.573±1\.17033\.573\\pm 1\.1700\.159±0\.1020\.159\\pm 0\.102133\.536±13\.124133\.536\\pm 13\.1245\.3±3\.35\.3\\pm 3\.3−301\.8±258\.2\-301\.8\\pm 258\.2−7\.8±10\.6\-7\.8\\pm 10\.6𝚀𝚠𝚎𝚗𝟹​\.5​\-​𝟸𝟽​𝙱\\mathtt\{Qwen3\.5\\text\{\-\}27B\}34\.849±0\.52734\.849\\pm 0\.5270\.040±0\.0020\.040\\pm 0\.002128\.647±9\.975128\.647\\pm 9\.9751\.7±1\.51\.7\\pm 1\.5−1\.9±5\.1\-1\.9\\pm 5\.1−3\.8±8\.1\-3\.8\\pm 8\.1𝚀𝚠𝚎𝚗𝟹​\.5​\-​𝟹𝟻​𝙱​\-​𝙰𝟹𝙱\\mathtt\{Qwen3\.5\\text\{\-\}35B\\text\{\-\}A3B\}34\.994±0\.48434\.994\\pm 0\.4840\.153±0\.1050\.153\\pm 0\.105116\.952±12\.132116\.952\\pm 12\.1321\.3±1\.41\.3\\pm 1\.4−288\.9±267\.0\-288\.9\\pm 267\.05\.6±9\.85\.6\\pm 9\.8*Dataset:𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}*𝚀𝚠𝚎𝚗𝟹​\.5​\-​𝟺​𝙱\\mathtt\{Qwen3\.5\\text\{\-\}4B\}0\.9960\.9960\.3520\.35212\.93612\.9360\.919±0\.0690\.919\\pm 0\.0690\.334±0\.0110\.334\\pm 0\.01111\.912±0\.91111\.912\\pm 0\.9117\.8±6\.97\.8\\pm 6\.95\.1±3\.05\.1\\pm 3\.07\.9±7\.07\.9\\pm 7\.0𝚀𝚠𝚎𝚗𝟹​\.5​\-​𝟿​𝙱\\mathtt\{Qwen3\.5\\text\{\-\}9B\}0\.885±0\.0730\.885\\pm 0\.0730\.330±0\.0110\.330\\pm 0\.01111\.473±0\.97511\.473\\pm 0\.97511\.2±7\.411\.2\\pm 7\.46\.1±3\.06\.1\\pm 3\.011\.3±7\.511\.3\\pm 7\.5𝚀𝚠𝚎𝚗𝟹​\.5​\-​𝟸𝟽​𝙱\\mathtt\{Qwen3\.5\\text\{\-\}27B\}0\.962±0\.0240\.962\\pm 0\.0240\.340±0\.0070\.340\\pm 0\.00712\.496±0\.31512\.496\\pm 0\.3153\.5±2\.43\.5\\pm 2\.43\.2±2\.13\.2\\pm 2\.13\.4±2\.43\.4\\pm 2\.4𝚀𝚠𝚎𝚗𝟹​\.5​\-​𝟹𝟻​𝙱​\-​𝙰𝟹𝙱\\mathtt\{Qwen3\.5\\text\{\-\}35B\\text\{\-\}A3B\}0\.887±0\.0720\.887\\pm 0\.0720\.333±0\.0080\.333\\pm 0\.00811\.489±0\.96111\.489\\pm 0\.96111\.0±7\.211\.0\\pm 7\.25\.3±2\.45\.3\\pm 2\.411\.2±7\.411\.2\\pm 7\.4

![Refer to caption](https://arxiv.org/html/2607.01548v1/x12.png)Figure 7:Per\-checkpoint validation and test MASE ratios for the𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}and𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}model\-size scans\. Faint traces are individual seeds; solid lines are per\-size means;×\\times\-marks denote the iteration at which each run’s optimizer\-final program was first found\. Values below the dashed line \(ratio=1\\mathrm\{ratio\}\{=\}1\) improve over the Chronos\-2 identity\-transform baseline\.

## Appendix DExtended Results

We present here our raw results in Table[6](https://arxiv.org/html/2607.01548#A4.T6)and in Table[7](https://arxiv.org/html/2607.01548#A4.T7)\. We also demonstrate EFE\-Time evolutions in Figure[8](https://arxiv.org/html/2607.01548#A4.F8)and Figure[9](https://arxiv.org/html/2607.01548#A4.F9)across 3 repetitions\.

Table 6:Performance comparison of EFE\-Time against identity transformation across time\-series datasets\. The EFE\-Time transformations are learned for Chronos\-2, and other models reuse them\. As Reverso does not output quantiles, we take theWQLreverso;=;∑t\|actualt−mediant\|∑t\|actualt\|\\text\{WQL\}\_\{\\text\{reverso\}\};=;\\frac\{\\sum\_\{t\}\|\\text\{actual\}\_\{t\}\-\\text\{median\}\_\{t\}\|\}\{\\sum\_\{t\}\|\\text\{actual\}\_\{t\}\|\}Table 7:Performance comparison of feature engineering methods across tabular datasets over ROC\-AUC\. The reported results are 3 outer\-fold averages from the TabArena benchmark\.![Refer to caption](https://arxiv.org/html/2607.01548v1/x13.png)Figure 8:The change of validation and test MASE during EFE\-Time evolution for Covid\-Deaths\.![Refer to caption](https://arxiv.org/html/2607.01548v1/x14.png)Figure 9:The change of validation and test MASE during EFE\-Time evolution for Solar\-hourly\.
## Appendix EPer\-series forecast trajectories under the evolved transform

This section visualizes the per\-series effect of the evolved transform on a small selection of test instances drawn from each dataset in the EFE\-Time evaluation suite\. Each three\-panel column shows, top to bottom, Chronos\-2 on the raw input window, Chronos\-2 in the transformed space \(the forecaster seesz=transform​\(y\)z=\\texttt\{transform\}\(y\)and predicts the transformed continuation\), and the inverse\-transformed forecast plotted back on the original scale\. The four picks per dataset include both series where the transform helps and series where it has little or no effect, so the panels should be read as qualitative inspection rather than as a ranking\. On𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}and𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}the transformed input window often resembles a near\-periodic shape that Chronos\-2 extrapolates more cleanly\. We point this out in the per\-dataset paragraphs where it applies\. A recurring theme across the harder cases is the role of exogenous, perhaps unpredictable drivers with existing information: when the dynamics that govern the held\-out window are not visible in the input history, no input\-only transform can recover what is missing\.

### E\.1𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}

The transformed input on Fig\.[10\(a\)](https://arxiv.org/html/2607.01548#A5.F10.sf1)and Fig\.[10\(b\)](https://arxiv.org/html/2607.01548#A5.F10.sf2)is closer to a smooth periodic signal than the raw counter is, and Chronos\-2’s median forecast in the transformed space follows the gentle wave the model is now seeing\. The inverse step then restores the original level and brings the forecast median into the neighbourhood of the held\-out trajectory\. Fig\.[10\(c\)](https://arxiv.org/html/2607.01548#A5.F10.sf3)and Fig\.[10\(d\)](https://arxiv.org/html/2607.01548#A5.F10.sf4)display sudden spikes in the held\-out window that have no precedent in the input history: a death surge of this kind is typically triggered by an exogenous event \(a regional outbreak, a hospital admission wave, a reporting catch\-up\)\. The dynamics governing such jumps are non\-Markovian with respect to the model’s input, which places a fundamental limit on what an input\-only transform can recover\. The transform absorbs some of the level shift and reduces the inverted\-forecast MAE on both panels, but the residual on Fig\.[10\(d\)](https://arxiv.org/html/2607.01548#A5.F10.sf4)widens the quantile cone enough that wQL regresses even as the median improves\.

### E\.2𝚖𝟺​\_​𝚢𝚎𝚊𝚛𝚕𝚢\\mathtt\{m4\\\_yearly\}

The M4\-yearly horizon is short and the input windows are short as well\. The transformed window typically rescales the trajectory’s growth so that Chronos\-2 in the transformed space sits closer to the ground truth’s slope, and the inverse step then amplifies that corrected slope back to the original units as seen in Figure[11](https://arxiv.org/html/2607.01548#A5.F11)\. We do not see the periodic transform\-space shape that appears on𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}and𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}; the effect here is closer to a learned per\-series scale and trend correction\. The same caveat as on𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}applies: the M4\-yearly aggregates contain a substantial financial and macroeconomic component, and multi\-year drift in those series is often driven by exogenous events \(policy regimes, market shocks, demographic transitions\) that the input window does not encode\. Fig\.[11\(d\)](https://arxiv.org/html/2607.01548#A5.F11.sf4)is an example where the transform tightens the slope but the held\-out trajectory continues a regime change that no input\-only mapping could anticipate, leaving a residual MASE above1616\.

### E\.3𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}

The hourly solar series are dominated by their diurnal cycle\. In the transformed space as seen in Figure[12](https://arxiv.org/html/2607.01548#A5.F12)the daily oscillation is regularized into a smoother periodic shape, and Chronos\-2 follows the periodic continuation more closely than it does on the raw window; the inverse\-transformed forecast then sits within a tight band around the held\-out trajectory\. Fig\.[12\(d\)](https://arxiv.org/html/2607.01548#A5.F12.sf4)shows the opposite case: a short, near\-flat segment where the transform shifts Chronos\-2 off the true level by a small amount and every metric regresses\. Solar irradiance is largely driven by deterministic astronomy plus weather, so the transform’s leverage here is geometric rather than informational, and the failure case on Fig\.[12\(d\)](https://arxiv.org/html/2607.01548#A5.F12.sf4)is a small one\.

### E\.4𝚛𝚎𝚜𝚝𝚊𝚞𝚛𝚊𝚗𝚝\\mathtt\{restaurant\}

The transformed input window resembles the raw input on each of these picks, and the top and middle panel forecasts overlap closely as seen in Figure[13](https://arxiv.org/html/2607.01548#A5.F13)\. The inverse\-transformed median is essentially indistinguishable from the raw median, and the per\-series gains in MASE, wQL, and MAE are small\. The corpus is built from daily visitor counts at Japanese restaurants drawn from two reservation platforms, and the official held\-out window spans the Golden Week holiday; days on which a restaurant was closed are excluded from scoring\. The drivers of variation are therefore primarily exogenous: the holiday calendar, day\-of\-week effects modulated by closure decisions, and platform\-level demand\. None of these are encoded in the input history at the level of detail needed to anticipate the held\-out trajectory, and an input\-only transform cannot manufacture the missing signal\. The aggregate near\-identity behaviour reported in the main paper is consistent with this: when the relevant covariates live outside the history window, the optimizer settles close to the identity rather than overfit a spurious mapping\.

### E\.5𝚋𝚒𝚝𝚋𝚛𝚊𝚒𝚗𝚜​\_​𝚛𝚗𝚍/𝙷\\mathtt\{bitbrains\\\_rnd/H\}

The corpus is a workload trace from a virtualized datacenter hosting business\-critical applications\. The channels record requested and used resource demand on a heterogeneous shared cluster\. Demand on such a cluster is driven mostly by external user activity and operational events \(deployments, batch windows, traffic incidents\), which makes the dynamics highly non\-Markovian with respect to the channel\-only input window we feed Chronos\-2\. The per\-series picture is correspondingly heavy\-tailed: Fig\.[14\(c\)](https://arxiv.org/html/2607.01548#A5.F14.sf3)is a series where the transform compresses a high\-magnitude spike into a smoother envelope and the inverse\-transformed forecast tracks the held\-out trajectory inside that envelope, dropping per\-series MASE by roughly an order of magnitude\. The other three picks retain the raw window’s spiky character through the transform, and the inverse\-transformed median sits close to the raw median\. The aggregate near\-identity behaviour reported in the main paper is therefore a mean over a heavy\-tailed distribution of per\-series effects, not a uniform absence of effect, and we do not extend the periodic transform\-space observation from𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}and𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}to this dataset\.

![Refer to caption](https://arxiv.org/html/2607.01548v1/x15.png)\(a\)MASE10\.68→1\.6310\.68\\rightarrow 1\.63, wQL0\.038→0\.0060\.038\\rightarrow 0\.006, MAE4810→7354810\\rightarrow 735\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x16.png)\(b\)MASE116\.63→34\.50116\.63\\rightarrow 34\.50, wQL0\.279→0\.1070\.279\\rightarrow 0\.107, MAE382\.1→113\.0382\.1\\rightarrow 113\.0\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x17.png)\(c\)MASE37\.39→12\.0937\.39\\rightarrow 12\.09, wQL0\.117→0\.0570\.117\\rightarrow 0\.057, MAE222\.9→72\.1222\.9\\rightarrow 72\.1\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x18.png)\(d\)MASE346\.8→268\.9346\.8\\rightarrow 268\.9, wQL0\.539→0\.6970\.539\\rightarrow 0\.697, MAE1\.92→1\.491\.92\\rightarrow 1\.49\.

Figure 10:Forecast trajectories on four𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}test instances\. Each three\-panel column shows, top to bottom, Chronos\-2 on the raw input, Chronos\-2 in the transformed space, and the inverse\-transformed forecast back on the original scale\. Panels \(a\) and \(b\) show series where the transform substantially reduces the inverted\-forecast error\. Panels \(c\) and \(d\) show series whose held\-out windows contain sudden spikes; the transform partially recovers the level on \(c\) and improves the median MAE but not the quantile loss on \(d\)\.![Refer to caption](https://arxiv.org/html/2607.01548v1/x19.png)\(a\)MASE4\.74→0\.554\.74\\rightarrow 0\.55, wQL0\.242→0\.0520\.242\\rightarrow 0\.052, MAE3120→3593120\\rightarrow 359\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x20.png)\(b\)MASE2\.81→1\.772\.81\\rightarrow 1\.77, wQL0\.352→0\.2560\.352\\rightarrow 0\.256, MAE3359→21123359\\rightarrow 2112\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x21.png)\(c\)MASE7\.79→3\.737\.79\\rightarrow 3\.73, wQL0\.420→0\.1970\.420\\rightarrow 0\.197, MAE9804→46999804\\rightarrow 4699\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x22.png)\(d\)MASE19\.59→16\.4619\.59\\rightarrow 16\.46, wQL0\.561→0\.4900\.561\\rightarrow 0\.490, MAE5690→47845690\\rightarrow 4784\.

Figure 11:Forecast trajectories on four𝚖𝟺​\_​𝚢𝚎𝚊𝚛𝚕𝚢\\mathtt\{m4\\\_yearly\}test instances\. The yearly horizon is six steps; the panels read as a per\-series scale and trend correction rather than the periodic\-shape rewriting visible on𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}and𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}\. The forecast\-horizon zoom inset is omitted because the six\-step horizon is short enough to read directly from the panels\.![Refer to caption](https://arxiv.org/html/2607.01548v1/x23.png)\(a\)MASE1\.20→0\.161\.20\\rightarrow 0\.16, wQL0\.333→0\.0840\.333\\rightarrow 0\.084, MAE19\.69→2\.6719\.69\\rightarrow 2\.67\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x24.png)\(b\)MASE1\.46→0\.281\.46\\rightarrow 0\.28, wQL0\.363→0\.0970\.363\\rightarrow 0\.097, MAE18\.55→3\.5318\.55\\rightarrow 3\.53\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x25.png)\(c\)MASE2\.11→0\.322\.11\\rightarrow 0\.32, wQL0\.473→0\.1040\.473\\rightarrow 0\.104, MAE45\.06→6\.7545\.06\\rightarrow 6\.75\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x26.png)\(d\)MASE0\.55→0\.600\.55\\rightarrow 0\.60, wQL0\.195→0\.2190\.195\\rightarrow 0\.219, MAE6\.96→7\.536\.96\\rightarrow 7\.53\.

Figure 12:Forecast trajectories on four𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}test instances\. Hourly solar irradiance is strongly diurnal; on panels \(a\)\-\(c\) the transform folds the daily cycle into a smoother oscillation that Chronos\-2 extrapolates more cleanly\. Panel \(d\) is included as a series where the transform leaves Chronos\-2 slightly worse on every metric\. The history line is rendered with reduced opacity so the forecast traces and the forecast\-horizon inset stay legible against the long, densely diurnal context\.![Refer to caption](https://arxiv.org/html/2607.01548v1/x27.png)\(a\)MASE1\.09→1\.081\.09\\rightarrow 1\.08, wQL0\.348→0\.3460\.348\\rightarrow 0\.346, MAE15\.62→15\.4015\.62\\rightarrow 15\.40\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x28.png)\(b\)MASE0\.71→0\.700\.71\\rightarrow 0\.70, wQL0\.214→0\.2140\.214\\rightarrow 0\.214, MAE6\.79→6\.686\.79\\rightarrow 6\.68\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x29.png)\(c\)MASE0\.50→0\.490\.50\\rightarrow 0\.49, wQL0\.292→0\.2800\.292\\rightarrow 0\.280, MAE3\.56→3\.433\.56\\rightarrow 3\.43\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x30.png)\(d\)MASE0\.63→0\.610\.63\\rightarrow 0\.61, wQL0\.495→0\.4640\.495\\rightarrow 0\.464, MAE3\.29→3\.173\.29\\rightarrow 3\.17\.

Figure 13:Forecast trajectories on four𝚛𝚎𝚜𝚝𝚊𝚞𝚛𝚊𝚗𝚝\\mathtt\{restaurant\}test instances\. The dataset\-aggregate test MASE shifts by roughly0\.6%0\.6\\%; the per\-series traces match this near\-identity regime, with the transformed window resembling the raw window and the inverse\-transformed median essentially indistinguishable from the raw median\.![Refer to caption](https://arxiv.org/html/2607.01548v1/x31.png)\(a\)MASE0\.63→0\.550\.63\\rightarrow 0\.55, wQL0\.080→0\.0760\.080\\rightarrow 0\.076, MAE41\.94→36\.9141\.94\\rightarrow 36\.91\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x32.png)\(b\)MASE1\.37→1\.181\.37\\rightarrow 1\.18, wQL0\.122→0\.1110\.122\\rightarrow 0\.111, MAE269\.2→232\.0269\.2\\rightarrow 232\.0\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x33.png)\(c\)MASE1\.47→0\.181\.47\\rightarrow 0\.18, wQL24\.23→6\.5924\.23\\rightarrow 6\.59, MAE14296→175814296\\rightarrow 1758\.
![Refer to caption](https://arxiv.org/html/2607.01548v1/x34.png)\(d\)MASE3\.13→2\.003\.13\\rightarrow 2\.00, wQL0\.031→0\.0230\.031\\rightarrow 0\.023, MAE24\.58→15\.7024\.58\\rightarrow 15\.70\.

Figure 14:Forecast trajectories on four𝚋𝚒𝚝𝚋𝚛𝚊𝚒𝚗𝚜​\_​𝚛𝚗𝚍/𝙷\\mathtt\{bitbrains\\\_rnd/H\}test instances; all four are channel0of the bivariate input\. The dataset\-aggregate test MASE shifts by about0\.1%0\.1\\%, but the per\-series effect varies sharply: panel \(c\) shows an order\-of\-magnitude reduction in MASE on a single series, while panels \(a\), \(b\), and \(d\) show only modest gains and the transformed window retains the spiky character of the raw input\.

## Appendix FEvolution\-tree visualization of the search process

This section visualizes the OpenEvolve search itself across the EFE\-Time evaluation suite\. Each evolution run is rendered as a single panel; the three random seeds for a given dataset are laid out horizontally so that within\-dataset variability is visible at a glance\. The intent is qualitative\.

Every run is initialized from an identity\-transform seed program at iteration0and proceeds for100100iterations\. At each iteration, the controller samples a parent program from the in\-memory MAP\-Elites database \(three islands with periodic migration\), prompts an LLM with the parent code together with a small set of inspiration and diversity exemplars, and asks for a complete rewrite\. The new candidate is evaluated on the per\-dataset evolution pool by running Chronos\-2 zero\-shot on the transformed history, inverse\-transforming the forecast, and scoring the result against the identity baseline\.

In every panel, each program is plotted at its\(combined score,iteration\)\(\\text\{combined score\},\\text\{iteration\}\)coordinates, coloured by the island the program was placed in\. Faint grey segments are parent\-to\-child edges read from the per\-program𝚙𝚊𝚛𝚎𝚗𝚝​\_​𝚒𝚍\\mathtt\{parent\\\_id\}pointer in the run’s saved checkpoints\. The violet trace highlights the lineage of the run’s final selection from root to leaf, and the violet star marks that program\. The dashed grey line is the running best\-so\-far\. The y\-axis grows downward so the seed program sits at the top of every panel\. A few panels show the violet trace truncated, or only the final star with no incoming line\. This reflects the bounded MAP\-Elites store rather than the search itself: as new candidates arrive, weaker programs are evicted from their feature cells, and a saved checkpoint persists only the active population at the moment of saving\. A program that lives transiently in the database and then evicts before the next checkpoint write is never serialized to disk, so the parent JSON needed to draw an edge is missing\. The lineage line therefore stops at the last ancestor that remained in the population\.

The five panels reproduce, at the search\-trajectory level, the qualitative pattern visible at the per\-series level\. On𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}\(Fig\.[15](https://arxiv.org/html/2607.01548#A6.F15)\) all three seeds locate strongly\-improving programs that cross the10%10\\%\-improvement line; the seed\-22panel shows a long\-lived ancestor whose lineage walks back to the seed, whereas the other two panels render only the final star because the winners’ direct parents were evicted before the next checkpoint write\. On𝚖𝟺​\_​𝚢𝚎𝚊𝚛𝚕𝚢\\mathtt\{m4\\\_yearly\}\(Fig\.[16](https://arxiv.org/html/2607.01548#A6.F16)\) the seed\-to\-seed variance is visibly wider, and only one seed crosses the10%10\\%\-improvement line; the slowest run flattens its best\-so\-far frontier in the first half of the budget and does not escape the corresponding plateau, which we read as the search getting stuck in a basin reachable by the LLM’s rewrites rather than as evidence that no better program exists\. On𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}\(Fig\.[17](https://arxiv.org/html/2607.01548#A6.F17)\) the three runs converge to similar combined scores in a narrow band; the populations crowd a tight vertical strip and the best\-so\-far frontier moves in small increments, consistent with the per\-series picture from Sec\.[E\.3](https://arxiv.org/html/2607.01548#A5.SS3), where the diurnal regularity of hourly solar irradiance gives Chronos\-2 a strong baseline that bounds the headroom for any single transform\. On𝚛𝚎𝚜𝚝𝚊𝚞𝚛𝚊𝚗𝚝\\mathtt\{restaurant\}\(Fig\.[18](https://arxiv.org/html/2607.01548#A6.F18)\) and𝚋𝚒𝚝𝚋𝚛𝚊𝚒𝚗𝚜​\_​𝚛𝚗𝚍/𝙷\\mathtt\{bitbrains\\\_rnd/H\}\(Fig\.[19](https://arxiv.org/html/2607.01548#A6.F19)\) the optimizer\-selected scores sit just above the identity baseline; the populations crowd a near\-vertical band around the identity score, the best\-so\-far frontiers inch rather than jump, and on𝚛𝚎𝚜𝚝𝚊𝚞𝚛𝚊𝚗𝚝\\mathtt\{restaurant\}one seed locks in its final selection in the early iterations and does not move thereafter\. Read together with Sec\.[E\.4](https://arxiv.org/html/2607.01548#A5.SS4)and Sec\.[E\.5](https://arxiv.org/html/2607.01548#A5.SS5), this matches a search that does not find a transform with a meaningful effect on these corpora: the population drifts around identity across all three seeds, ruling out single\-run failure as the explanation\.

![Refer to caption](https://arxiv.org/html/2607.01548v1/x35.png)Figure 15:Evolution trees for the three𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}runs \(seeds11,22,33from left to right\)\. All three seeds locate programs that cross the10%10\\%\-improvement line\.![Refer to caption](https://arxiv.org/html/2607.01548v1/x36.png)Figure 16:Evolution trees for the three𝚖𝟺​\_​𝚢𝚎𝚊𝚛𝚕𝚢\\mathtt\{m4\\\_yearly\}runs\. The seed\-to\-seed score variability is visibly wider than on𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}\.![Refer to caption](https://arxiv.org/html/2607.01548v1/x37.png)Figure 17:Evolution trees for the three𝚜𝚘𝚕𝚊𝚛​\_​𝙷\\mathtt\{solar\\\_H\}runs\. The three panels are visually similar\.![Refer to caption](https://arxiv.org/html/2607.01548v1/x38.png)Figure 18:Evolution trees for the three𝚛𝚎𝚜𝚝𝚊𝚞𝚛𝚊𝚗𝚝\\mathtt\{restaurant\}runs\. The optimizer\-selected scores sit only marginally above the identity baseline\.![Refer to caption](https://arxiv.org/html/2607.01548v1/x39.png)Figure 19:Evolution trees for the three𝚋𝚒𝚝𝚋𝚛𝚊𝚒𝚗𝚜​\_​𝚛𝚗𝚍/𝙷\\mathtt\{bitbrains\\\_rnd/H\}runs\. As on𝚛𝚎𝚜𝚝𝚊𝚞𝚛𝚊𝚗𝚝\\mathtt\{restaurant\}, the populations crowd a vertical band around the identity baseline\.
## Appendix GPrompt example and reasoning traces

This section demonstrates a working example of EFE\-Time on𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}\. We also provide the EFE\-Tab system prompt and its example final program on𝚌𝚑𝚞𝚛𝚗\\mathtt\{churn\}and𝚒𝚗​\_​𝚟𝚎𝚑𝚒𝚌𝚕𝚎​\_​𝚌𝚘𝚞𝚙𝚘𝚗​\_​𝚛𝚎𝚌𝚘𝚖𝚖𝚎𝚗𝚍𝚊𝚝𝚒𝚘𝚗\\mathtt\{in\\\_vehicle\\\_coupon\\\_recommendation\}\.

### G\.1Time\-series example:𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}

We pick run22of𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}and feature the step from iteration66to iteration99in its lineage \(middle panel of Fig\.[15](https://arxiv.org/html/2607.01548#A6.F15), Sec\.[F](https://arxiv.org/html/2607.01548#A6); Table[8](https://arxiv.org/html/2607.01548#A7.T8)\)\. The interest of this case is that the search recovers from two consecutive failed mutations \(iterations33and66, both with combined score0\) and lands on a robust affine normalization that becomes the structural core of every later program in the lineage\. We show the rendered prompt sent to the LLM, the LLM’s reasoning, and the resulting program\.

Table 8:Six\-node lineage of the displayed final program for𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}run22\. The iteration\-99candidate \(the worked example in this section\) takes the search from a failure regime back to identity\-class performance, after which two further candidates bring the score above the10%10\\%\-improvement line\.The system prompt establishes the transform\-program API, lists transform families, and is held constant across iterations of a run\.

The user prompt carries the per\-iteration content: a JSON evaluation summary of the parent program, the dataset context for𝚌𝚘𝚟𝚒𝚍​\_​𝚍𝚎𝚊𝚝𝚑𝚜\\mathtt\{covid\\\_deaths\}, and the evolution history of previously evaluated programs\. To keep the section readable, embedded Python code blocks for previously evaluated programs are elided in place; each elision marker reports the line count of the elided body\. Section headers, the evaluation\-summary JSON, the dataset\-context block, and the response template at the end are kept verbatim\.

The LLM’s response opens with a diagnostic paragraph identifying why the parent failed, states a strategy, and emits the candidate program\. We treat the prose preamble as the model’s reasoning trace and present the assistant message as a single transcript\.

The takeaway from this step is structural: the recent\-window affine normalization introduced at iteration99is the ancestor of every later program in the lineage\. Subsequent iterations refine it \(branching for short and constant series, a winsorisation step, a numerical guard\) without revisiting the core\. For completeness, we list the displayed final program at iteration3030also below\.

System prompt for EFE\-TimeYou are evolving a target transformation program for time series forecasting\.The program defines a TransformProgram class with three methods:\- fit\(y\_hist, meta\) \-\> state dict of fitted parameters\- transform\(y, state\) \-\> transformed values\- inverse\_transform\(z, state\) \-\> original\-scale valuesThe downstream forecasting model \(Chronos\-2\) is fixed\.Your job is to find a transform that makes the series easier to forecast\.HOW IT WORKS:At each forecast origin, the evaluator:1\. Passes the historical target window to fit\(\) along with metadata2\. Transforms the history with transform\(\)3\. Feeds the transformed history to Chronos\-24\. Inverse\-transforms the forecast back to original scale5\. Scores MASE against the identity\-transform baselineMETA DICT AVAILABLE IN fit\(\):The meta parameter is a dict with these keys:\- prediction\_length: forecast horizon \(int\)\- seasonal\_period: e\.g\. 12 for monthly \(int\)\- length: total history length\- n\_valid: non\-NaN count\- mean, std, median: basic location/scale stats\- min\_val, max\_val, range: value range\- positive\_frac: fraction of values \> 0\- zero\_frac: fraction of values == 0\- skewness: skew of the series\- cv: coefficient of variation \(std/\|mean\|\)\- trend\_strength: Rˆ2 of linear fit \(0 = no trend, 1 = perfect trend\)\- recent\_mean, recent\_std: stats of the last seasonal\_period observationsUse these to decide WHICH transform family to use and HOW to parameterize it\.For example:\- High positive\_frac \+ high skewness \-\> shifted log1p\- Mixed positive/negative \+ high skewness \-\> asinh\- Low skewness \+ low cv \-\> identity might be best\- Large range \+ high cv \-\> scaled transformTRANSFORM FAMILIES TO EXPLORE:\- identity: no transform \(the baseline you must beat\)\- shifted log1p: shift = \-min \+ eps, z = log1p\(y \+ shift\)\- signed log: z = sign\(y\) \* log1p\(\|y\|\)\- asinh: z = asinh\(y / scale\), with scale fitted from data\- scaled asinh: z = asinh\(\(y \- center\) / scale\)\- bounded power: Box\-Cox\-like with clamped lambda\- piecewise: different transforms for different value ranges\- affine normalization: \(y \- center\) / scale, e\.g\. mean/std, median/IQR,median/MAD \-\- robust variants often help when outliers are present\- recent\-window normalization: use stats from the last seasonal\_periodobservations instead of the full history \(useful for non\-stationary series\)\- detrending: z\[i\] = y\[i\] \- \(slope \* i \+ intercept\), fit slope/interceptvia linear regression\. Inverse for the FORECAST must usestate\["hist\_len"\] \+ np\.arange\(len\(z\)\) for the time index, NOTnp\.arange\(len\(z\)\) \-\- see the CRITICAL hist\_len section below\.\- seasonal phase centering: subtract the mean of each phase i % seasonal\_periodfrom y\. Inverse forecast must use \(state\["hist\_len"\] \+ np\.arange\(len\(z\)\)\) % sfor the phase index, NOT np\.arange\(len\(z\)\) % s\.\- composites: combine any of the above \(e\.g\. detrend \-\> asinh, or seasonalcentering \-\> affine\)\. Each step must be exactly invertible in reverse order\.WHAT TO VARY:\- Family choice based on meta dict\- Shift formula \(min\-based, percentile\-based, etc\.\)\- Scale formula \(std\-based, IQR\-based, MAD\-based\)\- Power parameter bounds\- Recent\-window vs full\-window statistics\- Branching logic \(if/else based on meta values\)CRITICAL \-\- hist\_len AND THE FORECAST WORKFLOW:fit\(\) MUST store "hist\_len": len\(y\_hist\) in the returned state dict\.This is critical because transform\(\) and inverse\_transform\(\) are calledon DIFFERENT\-LENGTH arrays:\- transform\(\) receives the full history \(length T\)\- inverse\_transform\(\) receives the forecast \(length prediction\_length\)For ANY position\-dependent operation \(detrending, time\-varying scaling\),you MUST use state\["hist\_len"\] to compute the correct time offset forthe forecast positions, NOT len\(z\)\. Example for detrending:transform: z\[i\] = y\[i\] \- slope \* iinverse: y\[i\] = z\[i\] \+ slope \* \(state\["hist\_len"\] \+ i\)If you use np\.arange\(len\(z\)\) in inverse\_transform, it will produceindices 0\.\.11 instead of the correct 72\.\.83, DESTROYING the forecast\.RULES \-\- YOUR PROGRAM MUST:\- fit parameters using ONLY the history passed to fit\(\)\- ALWAYS include "hist\_len": len\(y\_hist\) in the state dict\- NOT use future data\- NOT mutate input arrays \(copy first\)\- Preserve length, order, and NaN positions\- Return finite values \(no Inf\); NaN only where input had NaN\- Provide an exact inverse: \|inverse\(transform\(y\)\) \- y\| < 1e\-4\- Keep inverse stable slightly beyond observed transformed range\- inverse\_transform MUST work on arrays of any length \(not just hist\_len\)\- Be deterministic given the same input\- Return a dict from fit\(\) \(the state\)COMMON MISTAKES TO AVOID:\- Using np\.arange\(len\(z\)\) in inverse\_transform for position\-dependent ops\(MUST use state\["hist\_len"\] \+ np\.arange\(len\(z\)\) for forecast positions\)\- Dividing by zero \(always add eps to denominators\)\- log of negative numbers \(check and shift first\)\- Inverse that doesn’t match forward \(verify the math\)\- Using np\.log instead of np\.log1p \(log\(0\) = \-inf\)\- Forgetting to copy arrays before modifying\- State values that are NaN or InfFEEDBACK YOU WILL RECEIVE:The evaluator provides an "evaluation\_summary" artifact with:\- candidate\_mase / baseline\_mase: your MASE vs identity baseline\- mase\_ratio: < 1\.0 means you’re better than baseline\- help\_rate: fraction of series where you’re betterUse this to understand WHERE your transform helps and hurts\.STRATEGY:1\. Start by understanding the meta dict values2\. Choose an appropriate transform family3\. Fit parameters carefully from the history using meta dict statistics4\. Verify your inverse math is correct5\. Add branching if a single transform family cannot cover all series

Openevolve user prompt for Covid\-Deaths \(parent code blocks omitted\)\# Current Program Information\- Fitness: 0\.0000\- Feature coordinates:\- Focus areas: \- Fitness unchanged at 0\.0000\- No feature coordinates\- Consider simplifying \- code length exceeds 500 characters\#\# Last Execution Output\#\#\# evaluation\_summary‘‘‘\{"dataset": "covid\_deaths","prediction\_length": 30,"seasonal\_period": 1,"score": 0\.0,"baseline\_mase": 38\.91405167517269,"baseline\_wql": 0\.07616885181861571,"baseline\_mae": 139\.53250150509274,"candidate\_mase": 51\.3413728390549,"candidate\_wql": 0\.16410550044604347,"candidate\_mae": 337\.09089491275,"mase\_ratio": 1\.3193530518902734,"wql\_ratio": 2\.154496182203655,"n\_valid\_series": 223,"n\_transform\_errors": 0,"n\_inverse\_errors": 0,"transform\_time\_sec": 0\.16,"forecast\_time\_sec": 0\.13,"harm\_rate": 0\.5336322869955157,"help\_rate": 0\.2556053811659193,"median\_mase\_ratio": 1\.020853901536389,"best\_series\_ratio": 0\.0,"worst\_series\_ratio": 28\.91401322388652,"complexity": \{"n\_literals": 8,"n\_branches": 7,"n\_fitted\_params": 2,"penalty": 0\.0\}\}‘‘‘\#\#\# dataset\_context‘‘‘Dataset: covid\_deathsFrequency: DPrediction length: 30Seasonal period: 1Eval series: ?Sample series statistics \(across training samples\):history length: median=152, range=\[152, 152\]mean: median=23\.8, range=\[0\.987, 4\.03e\+03\]std: median=29, range=\[0\.987, 4\.17e\+03\]min: median=0, range=\[0, 0\]max: median=101, range=\[2, 9\.7e\+03\]positive\_frac: median=0\.609zero\_frac: median=0\.391skewness: median=0\.234, range=\[\-0\.040, 1\.815\]Description: 266 daily COVID death counts, extreme spikes \+ regime changes‘‘‘\# Program Evolution History\#\# Previous Attempts\#\#\# Attempt 2\- Changes: Full rewrite\- Metrics: combined\_score: 0\.0000, mase: 49\.9797, wql: 0\.1596, mae: 285\.3394\- Outcome: Mixed results\#\#\# Attempt 1\- Changes: Full rewrite\- Metrics: combined\_score: 0\.0000, mase: 51\.3414, wql: 0\.1641, mae: 337\.0909\- Outcome: Mixed results\#\# Top Performing Programs\#\#\# Program 1 \(Score: 0\.0000\)‘‘‘python\# \[\.\.\. 79 lines of program code elided \.\.\.\]‘‘‘Key features: Performs well on combined\_score \(0\.0000\), Performs well on mase \(51\.3414\), Performs well on wql \(0\.1641\), Performs well on mae \(337\.0909\)\#\#\# Program 2 \(Score: 0\.0000\)‘‘‘python\# \[\.\.\. 88 lines of program code elided \.\.\.\]‘‘‘Key features: Performs well on combined\_score \(0\.0000\), Performs well on mase \(49\.9797\), Performs well on wql \(0\.1596\), Performs well on mae \(285\.3394\)\#\# Inspiration ProgramsThese programs represent diverse approaches and creative solutions that may inspire new ideas:\#\#\# Inspiration 1 \(Score: 0\.0000, Type: Exploratory\)‘‘‘python\# \[\.\.\. 88 lines of program code elided \.\.\.\]‘‘‘Unique approach: Modification:, \[Fragment formatting error: ’metric\_name’\], \[Fragment formatting error: ’metric\_name’\]\# Current Program‘‘‘python\# \[\.\.\. 79 lines of program code elided \.\.\.\]‘‘‘\# TaskRewrite the program to improve its FITNESS SCORE\.The system maintains diversity across these dimensions: diversityDifferent solutions with similar fitness but different features are valuable\.Provide the complete new program code\.IMPORTANT: Make sure your rewritten program maintains the same inputs and outputsas the original program, but with improved internal implementation\.‘‘‘python\# Your rewritten program here‘‘‘

LLM response for Covid\-Deaths \(reasoning followed by the candidate program\)Looking at the evaluation results, my current transforms are performing worse than the identity baseline \(mase\_ratio = 1\.32\)\. The key insight is that this is COVID death data \- daily counts with extreme spikes and regime changes\. The median skewness is only 0\.234, so most series have moderate skewness\.The problem with my previous approaches:1\. log1p/sqrt transforms compress the recent high values, making it harder for Chronos to extrapolate the current level2\. The scaling in log1p\_scaled was distorting the signal3\. The baseline MASE is already 38\.9 which is high, meaning this is hard to forecastLet me try a different strategy: \*\*identity for most series\*\* but with a simple \*\*robust affine normalization\*\* that helps Chronos by centering/scaling the data to a more "standard" range without distorting the shape\. I’ll use recent\-window statistics to capture the current regime\.‘‘‘pythonimport numpy as npclass TransformProgram:def \_\_init\_\_\(self, seed=42\):self\.seed = seed\# EVOLVE\-BLOCK\-STARTdef fit\(self, y\_hist, meta=None\):y = y\_hist\.copy\(\)valid = y\[˜np\.isnan\(y\)\]state = \{"hist\_len": len\(y\_hist\)\}if meta is None:meta = \{\}\# For COVID deaths: regime changes mean recent stats matter most\# Use a simple affine normalization based on recent window\# This preserves shape \(no distortion\) but helps Chronos with scalen = len\(valid\)if n < 5:state\["family"\] = "identity"return state\# Use recent window \(last ˜30 days\) to capture current regimerecent\_n = min\(30, n\)recent = valid\[\-recent\_n:\]center = float\(np\.median\(recent\)\)\# Use IQR\-based scale for robustness to spikesq75 = float\(np\.percentile\(recent, 75\)\)q25 = float\(np\.percentile\(recent, 25\)\)iqr = q75 \- q25\# MAD as fallbackmad = float\(np\.median\(np\.abs\(recent \- center\)\)\)\# Pick best scale measureif iqr \> 1e\-6:scale = iqr / 1\.35 \# normalize IQR to std\-equivalentelif mad \> 1e\-6:scale = mad \* 1\.4826else:\# Nearly constant recent values \- use full history scale or identityfull\_std = meta\.get\("std", 1\.0\)if full\_std \> 1e\-6:scale = float\(full\_std\)else:state\["family"\] = "identity"return statestate\["family"\] = "affine"state\["center"\] = centerstate\["scale"\] = float\(scale\)return statedef transform\(self, y, state\):z = y\.copy\(\)family = state\["family"\]if family == "affine":mask = ˜np\.isnan\(z\)z\[mask\] = \(z\[mask\] \- state\["center"\]\) / state\["scale"\]return zdef inverse\_transform\(self, z, state\):y = z\.copy\(\)family = state\["family"\]if family == "affine":mask = ˜np\.isnan\(y\)y\[mask\] = y\[mask\] \* state\["scale"\] \+ state\["center"\]return y\# EVOLVE\-BLOCK\-ENDdef build\_transform\_program\(seed=42\):return TransformProgram\(seed=seed\)def get\_transform\_program\(seed=42\):return build\_transform\_program\(seed=seed\)‘‘‘

Example Final Program for Covid Deaths \(reached at iteration 30\)[⬇](data:text/plain;base64,aW1wb3J0IG51bXB5IGFzIG5wCgoKY2xhc3MgVHJhbnNmb3JtUHJvZ3JhbToKICAgIGRlZiBfX2luaXRfXyhzZWxmLCBzZWVkPTQyKToKICAgICAgICBzZWxmLnNlZWQgPSBzZWVkCgogICAgIyBFVk9MVkUtQkxPQ0stU1RBUlQKICAgIGRlZiBmaXQoc2VsZiwgeV9oaXN0LCBtZXRhPU5vbmUpOgogICAgICAgIHkgPSB5X2hpc3QuY29weSgpCiAgICAgICAgdmFsaWQgPSB5W35ucC5pc25hbih5KV0KICAgICAgICBzdGF0ZSA9IHsiaGlzdF9sZW4iOiBsZW4oeV9oaXN0KX0KICAgICAgICBpZiBtZXRhIGlzIE5vbmU6CiAgICAgICAgICAgIG1ldGEgPSB7fQogICAgICAgIG4gPSBsZW4odmFsaWQpCiAgICAgICAgaWYgbiA8IDEwOgogICAgICAgICAgICBzdGF0ZVsiZmFtaWx5Il0gPSAiaWRlbnRpdHkiCiAgICAgICAgICAgIHJldHVybiBzdGF0ZQoKICAgICAgICBzdGRfdmFsID0gbWV0YS5nZXQoInN0ZCIsIGZsb2F0KG5wLnN0ZCh2YWxpZCkpKQogICAgICAgIG1pbl92YWwgPSBtZXRhLmdldCgibWluX3ZhbCIsIGZsb2F0KG5wLm5hbm1pbih2YWxpZCkpKQogICAgICAgIHNrZXduZXNzID0gbWV0YS5nZXQoInNrZXduZXNzIiwgMC4wKQogICAgICAgIGN2ID0gbWV0YS5nZXQoImN2IiwgMS4wKQoKICAgICAgICAjIERldHJlbmQgdXNpbmcgbGFzdCA0NSBkYXlzIC0gY29uc2VydmF0aXZlCiAgICAgICAgdHJlbmRfbiA9IG1pbig0NSwgbikKICAgICAgICB0dyA9IHZhbGlkWy10cmVuZF9uOl0KICAgICAgICB0X2lkeCA9IG5wLmFyYW5nZSh0cmVuZF9uLCBkdHlwZT1mbG9hdCkKICAgICAgICB0X21lYW4gPSBucC5tZWFuKHRfaWR4KQogICAgICAgIHlfbWVhbiA9IG5wLm1lYW4odHcpCiAgICAgICAgZGVub20gPSBucC5zdW0oKHRfaWR4IC0gdF9tZWFuKSoqMikKCiAgICAgICAgc2xvcGUgPSAwLjAKICAgICAgICBpbnRlcmNlcHQgPSAwLjAKICAgICAgICBpZiBkZW5vbSA+IDFlLTEwOgogICAgICAgICAgICBzID0gZmxvYXQobnAuc3VtKCh0X2lkeCAtIHRfbWVhbikgKiAodHcgLSB5X21lYW4pKSAvIGRlbm9tKQogICAgICAgICAgICB0cmVuZF9yYW5nZSA9IGFicyhzKSAqIDMwCiAgICAgICAgICAgIHJlY2VudF9yYW5nZSA9IGZsb2F0KG5wLm1heCh0dykgLSBucC5taW4odHcpKSArIDFlLTYKICAgICAgICAgICAgIyBWZXJ5IGNvbnNlcnZhdGl2ZTogb25seSBkZXRyZW5kIGlmIGV4dHJhcG9sYXRpb24gaXMgc2FmZQogICAgICAgICAgICBpZiBhYnMocykgKiBuID4gc3RkX3ZhbCAqIDAuMTUgYW5kIHRyZW5kX3JhbmdlIDwgcmVjZW50X3JhbmdlICogMS4wOgogICAgICAgICAgICAgICAgc2xvcGUgPSBzCiAgICAgICAgICAgICAgICBvZmZzZXQgPSBuIC0gdHJlbmRfbgogICAgICAgICAgICAgICAgaW50ZXJjZXB0ID0gZmxvYXQoeV9tZWFuIC0gc2xvcGUgKiAodF9tZWFuICsgb2Zmc2V0KSkKCiAgICAgICAgc3RhdGVbInNsb3BlIl0gPSBzbG9wZQogICAgICAgIHN0YXRlWyJpbnRlcmNlcHQiXSA9IGludGVyY2VwdAoKICAgICAgICB2X2lkeCA9IG5wLmFyYW5nZShuLCBkdHlwZT1mbG9hdCkKICAgICAgICBkZXRyZW5kZWQgPSB2YWxpZCAtIChzbG9wZSAqIHZfaWR4ICsgaW50ZXJjZXB0KQoKICAgICAgICByZWNlbnRfbiA9IG1pbigzMCwgbGVuKGRldHJlbmRlZCkpCiAgICAgICAgZHRfbWluID0gZmxvYXQobnAubWluKGRldHJlbmRlZCkpCgogICAgICAgICMgVXNlIGxvZzFwIGZvciBub24tbmVnYXRpdmUgc2tld2VkIGRhdGEgKHNhZmVyIGludmVyc2UgdGhhbiBzcXJ0KQogICAgICAgIGlmIGR0X21pbiA+PSAtMWUtNiBhbmQgbWluX3ZhbCA+PSAwIGFuZCBza2V3bmVzcyA+IDAuNiBhbmQgY3YgPiAwLjU6CiAgICAgICAgICAgIHN0YXRlWyJub25saW5lYXIiXSA9ICJsb2cxcCIKICAgICAgICAgICAgbG9nX2R0ID0gbnAubG9nMXAobnAubWF4aW11bShkZXRyZW5kZWQsIDAuMCkpCiAgICAgICAgICAgIHJlY2VudF9sb2cgPSBsb2dfZHRbLXJlY2VudF9uOl0KICAgICAgICAgICAgY2VudGVyID0gZmxvYXQobnAubWVkaWFuKHJlY2VudF9sb2cpKQogICAgICAgICAgICBtYWQgPSBmbG9hdChucC5tZWRpYW4obnAuYWJzKHJlY2VudF9sb2cgLSBjZW50ZXIpKSkKICAgICAgICAgICAgc2NhbGUgPSBtYWQgKiAxLjQ4MjYgaWYgbWFkID4gMWUtNiBlbHNlIG1heChmbG9hdChucC5zdGQocmVjZW50X2xvZykpLCAxZS02KQogICAgICAgIGVsc2U6CiAgICAgICAgICAgIHN0YXRlWyJub25saW5lYXIiXSA9ICJub25lIgogICAgICAgICAgICByZWNlbnRfZHQgPSBkZXRyZW5kZWRbLXJlY2VudF9uOl0KICAgICAgICAgICAgY2VudGVyID0gZmxvYXQobnAubWVkaWFuKHJlY2VudF9kdCkpCiAgICAgICAgICAgIG1hZCA9IGZsb2F0KG5wLm1lZGlhbihucC5hYnMocmVjZW50X2R0IC0gY2VudGVyKSkpCiAgICAgICAgICAgIHNjYWxlID0gbWFkICogMS40ODI2IGlmIG1hZCA+IDFlLTYgZWxzZSBtYXgoZmxvYXQobnAuc3RkKHJlY2VudF9kdCkpLCAxZS02KQoKICAgICAgICBzdGF0ZVsiZmFtaWx5Il0gPSAiZGV0cmVuZF9hZmZpbmUiCiAgICAgICAgc3RhdGVbImNlbnRlciJdID0gY2VudGVyCiAgICAgICAgc3RhdGVbInNjYWxlIl0gPSBtYXgoZmxvYXQoc2NhbGUpLCAxZS02KQogICAgICAgIHJldHVybiBzdGF0ZQoKICAgIGRlZiB0cmFuc2Zvcm0oc2VsZiwgeSwgc3RhdGUpOgogICAgICAgIHogPSB5LmNvcHkoKQogICAgICAgIGlmIHN0YXRlWyJmYW1pbHkiXSA9PSAiaWRlbnRpdHkiOgogICAgICAgICAgICByZXR1cm4gegogICAgICAgIG1hc2sgPSB+bnAuaXNuYW4oeikKICAgICAgICBpZHggPSBucC5hcmFuZ2UobGVuKHopLCBkdHlwZT1mbG9hdCkKICAgICAgICB6W21hc2tdID0gelttYXNrXSAtIChzdGF0ZVsic2xvcGUiXSAqIGlkeFttYXNrXSArIHN0YXRlWyJpbnRlcmNlcHQiXSkKICAgICAgICBpZiBzdGF0ZVsibm9ubGluZWFyIl0gPT0gImxvZzFwIjoKICAgICAgICAgICAgelttYXNrXSA9IG5wLmxvZzFwKG5wLm1heGltdW0oelttYXNrXSwgMC4wKSkKICAgICAgICB6W21hc2tdID0gKHpbbWFza10gLSBzdGF0ZVsiY2VudGVyIl0pIC8gc3RhdGVbInNjYWxlIl0KICAgICAgICByZXR1cm4gegoKICAgIGRlZiBpbnZlcnNlX3RyYW5zZm9ybShzZWxmLCB6LCBzdGF0ZSk6CiAgICAgICAgeSA9IHouY29weSgpCiAgICAgICAgaWYgc3RhdGVbImZhbWlseSJdID09ICJpZGVudGl0eSI6CiAgICAgICAgICAgIHJldHVybiB5CiAgICAgICAgbWFzayA9IH5ucC5pc25hbih5KQogICAgICAgIHlbbWFza10gPSB5W21hc2tdICogc3RhdGVbInNjYWxlIl0gKyBzdGF0ZVsiY2VudGVyIl0KICAgICAgICBpZiBzdGF0ZVsibm9ubGluZWFyIl0gPT0gImxvZzFwIjoKICAgICAgICAgICAgIyBDbGFtcCB0byBwcmV2ZW50IGV4cCBvdmVyZmxvdwogICAgICAgICAgICB5W21hc2tdID0gbnAuY2xpcCh5W21hc2tdLCAtMTAuMCwgMjAuMCkKICAgICAgICAgICAgeVttYXNrXSA9IG5wLmV4cG0xKHlbbWFza10pCiAgICAgICAgICAgIHlbbWFza10gPSBucC5tYXhpbXVtKHlbbWFza10sIDAuMCkKICAgICAgICBpZHggPSBzdGF0ZVsiaGlzdF9sZW4iXSArIG5wLmFyYW5nZShsZW4oeSksIGR0eXBlPWZsb2F0KQogICAgICAgIHlbbWFza10gPSB5W21hc2tdICsgKHN0YXRlWyJzbG9wZSJdICogaWR4W21hc2tdICsgc3RhdGVbImludGVyY2VwdCJdKQogICAgICAgIHJldHVybiB5CiAgICAjIEVWT0xWRS1CTE9DSy1FTkQKCgpkZWYgYnVpbGRfdHJhbnNmb3JtX3Byb2dyYW0oc2VlZD00Mik6CiAgICByZXR1cm4gVHJhbnNmb3JtUHJvZ3JhbShzZWVkPXNlZWQpCgoKZGVmIGdldF90cmFuc2Zvcm1fcHJvZ3JhbShzZWVkPTQyKToKICAgIHJldHVybiBidWlsZF90cmFuc2Zvcm1fcHJvZ3JhbShzZWVkPXNlZWQp)importnumpyasnpclassTransformProgram:def\_\_init\_\_\(self,seed=42\):self\.seed=seed\#EVOLVE\-BLOCK\-STARTdeffit\(self,y\_hist,meta=None\):y=y\_hist\.copy\(\)valid=y\[~np\.isnan\(y\)\]state=\{"hist\_len":len\(y\_hist\)\}ifmetaisNone:meta=\{\}n=len\(valid\)ifn<10:state\["family"\]="identity"returnstatestd\_val=meta\.get\("std",float\(np\.std\(valid\)\)\)min\_val=meta\.get\("min\_val",float\(np\.nanmin\(valid\)\)\)skewness=meta\.get\("skewness",0\.0\)cv=meta\.get\("cv",1\.0\)\#Detrendusinglast45days\-conservativetrend\_n=min\(45,n\)tw=valid\[\-trend\_n:\]t\_idx=np\.arange\(trend\_n,dtype=float\)t\_mean=np\.mean\(t\_idx\)y\_mean=np\.mean\(tw\)denom=np\.sum\(\(t\_idx\-t\_mean\)\*\*2\)slope=0\.0intercept=0\.0ifdenom\>1e\-10:s=float\(np\.sum\(\(t\_idx\-t\_mean\)\*\(tw\-y\_mean\)\)/denom\)trend\_range=abs\(s\)\*30recent\_range=float\(np\.max\(tw\)\-np\.min\(tw\)\)\+1e\-6\#Veryconservative:onlydetrendifextrapolationissafeifabs\(s\)\*n\>std\_val\*0\.15andtrend\_range<recent\_range\*1\.0:slope=soffset=n\-trend\_nintercept=float\(y\_mean\-slope\*\(t\_mean\+offset\)\)state\["slope"\]=slopestate\["intercept"\]=interceptv\_idx=np\.arange\(n,dtype=float\)detrended=valid\-\(slope\*v\_idx\+intercept\)recent\_n=min\(30,len\(detrended\)\)dt\_min=float\(np\.min\(detrended\)\)\#Uselog1pfornon\-negativeskeweddata\(saferinversethansqrt\)ifdt\_min\>=\-1e\-6andmin\_val\>=0andskewness\>0\.6andcv\>0\.5:state\["nonlinear"\]="log1p"log\_dt=np\.log1p\(np\.maximum\(detrended,0\.0\)\)recent\_log=log\_dt\[\-recent\_n:\]center=float\(np\.median\(recent\_log\)\)mad=float\(np\.median\(np\.abs\(recent\_log\-center\)\)\)scale=mad\*1\.4826ifmad\>1e\-6elsemax\(float\(np\.std\(recent\_log\)\),1e\-6\)else:state\["nonlinear"\]="none"recent\_dt=detrended\[\-recent\_n:\]center=float\(np\.median\(recent\_dt\)\)mad=float\(np\.median\(np\.abs\(recent\_dt\-center\)\)\)scale=mad\*1\.4826ifmad\>1e\-6elsemax\(float\(np\.std\(recent\_dt\)\),1e\-6\)state\["family"\]="detrend\_affine"state\["center"\]=centerstate\["scale"\]=max\(float\(scale\),1e\-6\)returnstatedeftransform\(self,y,state\):z=y\.copy\(\)ifstate\["family"\]=="identity":returnzmask=~np\.isnan\(z\)idx=np\.arange\(len\(z\),dtype=float\)z\[mask\]=z\[mask\]\-\(state\["slope"\]\*idx\[mask\]\+state\["intercept"\]\)ifstate\["nonlinear"\]=="log1p":z\[mask\]=np\.log1p\(np\.maximum\(z\[mask\],0\.0\)\)z\[mask\]=\(z\[mask\]\-state\["center"\]\)/state\["scale"\]returnzdefinverse\_transform\(self,z,state\):y=z\.copy\(\)ifstate\["family"\]=="identity":returnymask=~np\.isnan\(y\)y\[mask\]=y\[mask\]\*state\["scale"\]\+state\["center"\]ifstate\["nonlinear"\]=="log1p":\#Clamptopreventexpoverflowy\[mask\]=np\.clip\(y\[mask\],\-10\.0,20\.0\)y\[mask\]=np\.expm1\(y\[mask\]\)y\[mask\]=np\.maximum\(y\[mask\],0\.0\)idx=state\["hist\_len"\]\+np\.arange\(len\(y\),dtype=float\)y\[mask\]=y\[mask\]\+\(state\["slope"\]\*idx\[mask\]\+state\["intercept"\]\)returny\#EVOLVE\-BLOCK\-ENDdefbuild\_transform\_program\(seed=42\):returnTransformProgram\(seed=seed\)defget\_transform\_program\(seed=42\):returnbuild\_transform\_program\(seed=seed\)

### G\.2Tabular example:𝚌𝚑𝚞𝚛𝚗\\mathtt\{churn\}

We provide the EFE\-Tab system prompt, as well as an example final program learned with EFE\-Tab here\. The Openevolve prompt is similar to the one for EFE\-Time\. So we omit this for brevity\.

System prompt for EFE\-TabYou are an expert data scientist\. You are designing a feature engineeringpipeline for tabular binary classification\. Your goal is to improve the AUCof the model\. Think carefully and step\-by\-step before responding\.You are allowed to create new features from the originals, and to drop the originals\.You are not allowed to train any machine learning model inside fit\(\),store a model in the state dict, or mutate the input DataFrame in place\.Do not standardize the data \-\- it is unnecessary for the models you are using\.Pay attention to the domain context \(column names and descriptions\), thedecision tree, and the permutation importance reports\.Comment your code to explain your decisions\. You should state your reasoning for each decision\.The program you are designing has two methods:1\. fit\(X, y\) \-\> state dict \-\- compute statistics from TRAINING data only2\. transform\(X, state\) \-\> DataFrame \-\- generate features using fitted stateIn fit\(\), compute and store any statistics you need\.In transform\(\), use the state to build / mutate features\.Allowed operations:\- Arithmetic combining TWO OR MORE columns: sums, differences, products, ratios\- Centered/z\-scored interactions using training\-fitted statistics from state\- Rank\-based interactions using training quantiles from state\- Row\-wise aggregates: NaN count, zero count, sum across columns\- Target\-rate mapping: per\-category target rate from fit\(\) applied in transform\(\)\- Residual features: col\_A minus its predicted value from col\_B \(fitted in fit\(\)\)\- String/categorical: length, token count, pattern match, category crossing\- Datetime: year, month, day, weekday, elapsed differences\- Parsing string\-encoded ordinals into integers \(e\.g\. "<1" / "\>20" \-\> 0 / 21\)\- Dropping an original column you have strong evidence is irrelevant\(id\-like, constant, or flagged as noise/HARMFUL by perm importance\)\- Make sure you are not dividing by zero\.FORBIDDEN:\- Training any ML model inside fit\(\) \(no LightGBM, no LogReg, no sklearnclassifiers / regressors, etc\.\)\- Storing fitted ML models in the state dict \(state must remain a plainlightweight dict \-\- see STATE DICT RULES below\)\- Modifying an original column in place \-\- keep originals byte\-identical\.If you want a cleaned/encoded version, add it as a NEW column with adifferent name; drop the original only if you don’t need it\.\- Mutating the input DataFrame in place \-\- always copy firstSTATE DICT RULES:\- Must be a plain dict \(returned from fit\(\)\)\- Values should be simple types: int, float, str, list, dict, numpy arrays\- Do NOT store DataFrames, models, or non\-serializable objects in stateUSE THE CONTEXT:The artifacts include a "dataset\_context" with column names, data types,value ranges, and a dataset description\. READ IT CAREFULLY:\- ONLY use column names that appear in the dataset\_context\- Read column names to understand what each represents to createdomain\-relevant interactions\- Read the dataset description for domain reasoning, including domainquirks\- Check data types before applying operations\- Use the decision tree to understand which features drive the splits\.Features near the root are the most important; features that appearanywhere in the tree carry signal\.\- Use the permutation importance over ORIGINAL features to judge whichraw columns matter on their own\. Consider dropping or replacing anyflagged as HARMFUL\.\- Use the permutation importance over NEW features to judge which ofyour engineered columns earned their cost\. Drop new columns flagged as noise or HARMFUL\- Think like an expert data scientist: use the context, domain knowledge, statistics, andfeedback artifacts to reason about what drives the predictions, thenengineer accordingly\.FEEDBACK YOU WILL RECEIVE:1\. Dataset context: Column names, dtypes, value ranges, description2\. Stage 2 summary: baseline\_cv\_auc, candidate\_cv\_auc, delta\_vs\_baseline,combined\_score, AND new\_cols / dropped\_original\_cols lists so you cansee exactly which original columns were dropped vs kept and whichfresh columns were added\.3\. Decision tree: depth\-3 tree on all output features\. Features that are closest to the root of the tree are the mostimportant\. Features that exist in the tree are also important\. Use this to understand whichfeatures are important and which are not\. Use this to understand how to combine featuresto create new features that are important\.4\. Permutation importance over ORIGINAL features: AUC drop per rawcolumn when shuffled\. Computed once at the start of the run\.USEFUL \(\>0\.005\), marginal \(0\.001\-0\.005\), noise \(˜0\), HARMFUL \(<\-0\.001\)\.Drop or replace features labeled HARMFUL\. Do not drop features labeled USEFUL, marginal or noise\.5\. Permutation importance over NEW features: AUC drop per engineeredcolumn when shuffled\. Computed every evaluation\.USEFUL \(\>0\.005\), marginal \(0\.001\-0\.005\), noise \(˜0\), HARMFUL \(<\-0\.001\)\.Drop NEW features that are noise or HARMFUL\.SCORING:total\_changes = n\_drops \+ n\_new\_colsYour score = 0\.5 \+ \(CV AUC improvement over baseline\) x 5 \- 0\.10 \* sqrt\(total\_changes / n\_rows\)

Example final program with EFE\-Tab for churn \(reached at iteration 33\)[⬇](data:text/plain;base64,IiIiClRhYnVsYXIgRmVhdHVyZSBFdm9sdXRpb24gLS0gRXZvbHZlZCBGZWF0dXJlIFByb2dyYW0gKGZpdC90cmFuc2Zvcm0pCiIiIgoKaW1wb3J0IG51bXB5IGFzIG5wCmltcG9ydCBwYW5kYXMgYXMgcGQKCgpjbGFzcyBGZWF0dXJlUHJvZ3JhbToKICAgIGRlZiBfX2luaXRfXyhzZWxmLCBzZWVkPTQyKToKICAgICAgICBzZWxmLnNlZWQgPSBzZWVkCgogICAgIyBFVk9MVkUtQkxPQ0stU1RBUlQKICAgIGRlZiBmaXQoc2VsZiwgWCwgeSk6CiAgICAgICAgc3RhdGUgPSB7fQogICAgICAgIGdsb2JhbF9tZWFuID0gZmxvYXQoeS5tZWFuKCkpCiAgICAgICAgc3RhdGVbJ2dsb2JhbF9tZWFuJ10gPSBnbG9iYWxfbWVhbgoKICAgICAgICAjIFNtb290aGVkIHRhcmdldCBlbmNvZGluZyBmb3Igc3RhdGUgKDUxIGNhdGVnb3JpZXMpCiAgICAgICAgIyBzbW9vdGhfZmFjdG9yPTMwIHBlcmZvcm1lZCBiZXN0IGhpc3RvcmljYWxseSB0byBhdm9pZCBvdmVyZml0CiAgICAgICAgc21vb3RoX2ZhY3RvciA9IDMwCiAgICAgICAgc3RhdGVfdGFyZ2V0ID0ge30KICAgICAgICBzdGF0ZV9jb2wgPSBYWydzdGF0ZSddLmFzdHlwZShzdHIpCiAgICAgICAgZm9yIGNhdCBpbiBzdGF0ZV9jb2wudW5pcXVlKCk6CiAgICAgICAgICAgIG1hc2sgPSBzdGF0ZV9jb2wgPT0gY2F0CiAgICAgICAgICAgIG4gPSBtYXNrLnN1bSgpCiAgICAgICAgICAgIGNhdF9tZWFuID0gZmxvYXQoeVttYXNrXS5tZWFuKCkpIGlmIG4gPiAwIGVsc2UgZ2xvYmFsX21lYW4KICAgICAgICAgICAgc21vb3RoZWQgPSAobiAqIGNhdF9tZWFuICsgc21vb3RoX2ZhY3RvciAqIGdsb2JhbF9tZWFuKSAvIChuICsgc21vb3RoX2ZhY3RvcikKICAgICAgICAgICAgc3RhdGVfdGFyZ2V0W2NhdF0gPSBzbW9vdGhlZAogICAgICAgIHN0YXRlWydzdGF0ZV90YXJnZXRfZW5jJ10gPSBzdGF0ZV90YXJnZXQKCiAgICAgICAgcmV0dXJuIHN0YXRlCgogICAgZGVmIHRyYW5zZm9ybShzZWxmLCBYLCBzdGF0ZSk6CiAgICAgICAgZGYgPSBYLmNvcHkoKQoKICAgICAgICAjIERyb3AgSEFSTUZVTCBvcmlnaW5hbDogdG90YWxfZXZlX2NoYXJnZSAocGVybSBpbXAgPSAtMC4wMDE3OSkKICAgICAgICBkZiA9IGRmLmRyb3AoY29sdW1ucz1bJ3RvdGFsX2V2ZV9jaGFyZ2UnXSwgZXJyb3JzPSdpZ25vcmUnKQogICAgICAgICMgRHJvcCByZWR1bmRhbnQgY2hhcmdlcyAocGVyZmVjdGx5IGNvcnJlbGF0ZWQgd2l0aCBtaW51dGVzKQogICAgICAgIGRmID0gZGYuZHJvcChjb2x1bW5zPVsndG90YWxfbmlnaHRfY2hhcmdlJywgJ3RvdGFsX2ludGxfY2hhcmdlJ10sIGVycm9ycz0naWdub3JlJykKCiAgICAgICAgaW50bF9wbGFuX251bSA9IGRmWydpbnRlcm5hdGlvbmFsX3BsYW4nXS5hc3R5cGUoc3RyKS5tYXAoeycxJzogMSwgJzAnOiAwfSkuZmlsbG5hKDApCgogICAgICAgICMgPT09IFBST1ZFTiBVU0VGVUwgKCswLjEwMik6IFRvdGFsIGNoYXJnZSAtLSByb290IHNwbGl0IGluIGRlY2lzaW9uIHRyZWUgPT09CiAgICAgICAgIyBSZWNvbnN0cnVjdHMgdG90YWwgYmlsbGluZyB1c2luZyBrbm93biBwZXItbWludXRlIHJhdGVzCiAgICAgICAgZGZbJ3RvdGFsX2NoYXJnZSddID0gKGRmWyd0b3RhbF9kYXlfY2hhcmdlJ10gKwogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkZlsndG90YWxfZXZlX21pbnV0ZXMnXSAqIDAuMDg1ICsKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZGZbJ3RvdGFsX25pZ2h0X21pbnV0ZXMnXSAqIDAuMDQ1ICsKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZGZbJ3RvdGFsX2ludGxfbWludXRlcyddICogMC4yNykKCiAgICAgICAgIyA9PT0gUFJPVkVOIFVTRUZVTCAoKzAuMDI1KTogSGlnaCBzZXJ2aWNlIGNhbGxzIGJpbmFyeSAodHJlZSBzcGxpdCBhdCAzLjUpID09PQogICAgICAgIGRmWydoaWdoX3NlcnZpY2VfY2FsbHMnXSA9IChkZlsnbnVtYmVyX2N1c3RvbWVyX3NlcnZpY2VfY2FsbHMnXSA+IDMpLmFzdHlwZShpbnQpCgogICAgICAgICMgPT09IFBST1ZFTiBVU0VGVUwgKCswLjAxMCk6IGludGxfcGxhbiB4IGludGxfY2FsbHMgPT09CiAgICAgICAgIyBMb3cgaW50bCBjYWxscyB3aXRoIGludGwgcGxhbiA9IGZydXN0cmF0ZWQgdXNlciBsaWtlbHkgdG8gY2h1cm4KICAgICAgICBkZlsnaW50bF9wbGFuX3hfaW50bF9jYWxscyddID0gaW50bF9wbGFuX251bSAqIGRmWyd0b3RhbF9pbnRsX2NhbGxzJ10KCiAgICAgICAgIyA9PT0gRnJvbSBiZXN0IHByb2dyYW0gKDAuNTY4NSk6IGludGxfcGxhbiB4IHNlcnZpY2VfY2FsbHMgPT09CiAgICAgICAgIyBDb21iaW5lcyB0aGUgdHdvIHN0cm9uZ2VzdCBvcmlnaW5hbCBwcmVkaWN0b3JzCiAgICAgICAgZGZbJ2ludGxfcGxhbl94X3N2Y19jYWxscyddID0gaW50bF9wbGFuX251bSAqIGRmWydudW1iZXJfY3VzdG9tZXJfc2VydmljZV9jYWxscyddCgogICAgICAgICMgPT09IFN0YXRlIHRhcmdldCBlbmNvZGluZyB3aXRoIGhlYXZ5IHNtb290aGluZyA9PT0KICAgICAgICBlbmMgPSBzdGF0ZVsnc3RhdGVfdGFyZ2V0X2VuYyddCiAgICAgICAgZ20gPSBzdGF0ZVsnZ2xvYmFsX21lYW4nXQogICAgICAgIGRmWydzdGF0ZV9jaHVybl9yYXRlJ10gPSBkZlsnc3RhdGUnXS5hc3R5cGUoc3RyKS5tYXAoZW5jKS5maWxsbmEoZ20pCgogICAgICAgICMgVG90YWw6IDMgZHJvcHMgKyA1IG5ldyA9IDggY2hhbmdlcwogICAgICAgICMgUGVuYWx0eTogMC4xMCAqIHNxcnQoOC8yMzMzKSA9IDAuMDA1OQogICAgICAgICMgUmVtb3ZlZCBzdmNfY2FsbHNfeF9jaGFyZ2UgKEhBUk1GVUwgLTAuMDA0KSBhbmQgaW50bF9wbGFuX3hfZGF5X21pbiAobm9pc2UpCgogICAgICAgIHJldHVybiBkZgogICAgIyBFVk9MVkUtQkxPQ0stRU5ECgoKZGVmIGJ1aWxkX2ZlYXR1cmVfcHJvZ3JhbShzZWVkPTQyKToKICAgIHJldHVybiBGZWF0dXJlUHJvZ3JhbShzZWVkPXNlZWQpCgoKZGVmIGdldF9mZWF0dXJlX3Byb2dyYW0oc2VlZD00Mik6CiAgICByZXR1cm4gYnVpbGRfZmVhdHVyZV9wcm9ncmFtKHNlZWQ9c2VlZCk=)"""TabularFeatureEvolution–EvolvedFeatureProgram\(fit/transform\)"""importnumpyasnpimportpandasaspdclassFeatureProgram:def\_\_init\_\_\(self,seed=42\):self\.seed=seed\#EVOLVE\-BLOCK\-STARTdeffit\(self,X,y\):state=\{\}global\_mean=float\(y\.mean\(\)\)state\[’global\_mean’\]=global\_mean\#Smoothedtargetencodingforstate\(51categories\)\#smooth\_factor=30performedbesthistoricallytoavoidoverfitsmooth\_factor=30state\_target=\{\}state\_col=X\[’state’\]\.astype\(str\)forcatinstate\_col\.unique\(\):mask=state\_col==catn=mask\.sum\(\)cat\_mean=float\(y\[mask\]\.mean\(\)\)ifn\>0elseglobal\_meansmoothed=\(n\*cat\_mean\+smooth\_factor\*global\_mean\)/\(n\+smooth\_factor\)state\_target\[cat\]=smoothedstate\[’state\_target\_enc’\]=state\_targetreturnstatedeftransform\(self,X,state\):df=X\.copy\(\)\#DropHARMFULoriginal:total\_eve\_charge\(permimp=\-0\.00179\)df=df\.drop\(columns=\[’total\_eve\_charge’\],errors=’ignore’\)\#Dropredundantcharges\(perfectlycorrelatedwithminutes\)df=df\.drop\(columns=\[’total\_night\_charge’,’total\_intl\_charge’\],errors=’ignore’\)intl\_plan\_num=df\[’international\_plan’\]\.astype\(str\)\.map\(\{’1’:1,’0’:0\}\)\.fillna\(0\)\#===PROVENUSEFUL\(\+0\.102\):Totalcharge–rootsplitindecisiontree===\#Reconstructstotalbillingusingknownper\-minuteratesdf\[’total\_charge’\]=\(df\[’total\_day\_charge’\]\+df\[’total\_eve\_minutes’\]\*0\.085\+df\[’total\_night\_minutes’\]\*0\.045\+df\[’total\_intl\_minutes’\]\*0\.27\)\#===PROVENUSEFUL\(\+0\.025\):Highservicecallsbinary\(treesplitat3\.5\)===df\[’high\_service\_calls’\]=\(df\[’number\_customer\_service\_calls’\]\>3\)\.astype\(int\)\#===PROVENUSEFUL\(\+0\.010\):intl\_planxintl\_calls===\#Lowintlcallswithintlplan=frustrateduserlikelytochurndf\[’intl\_plan\_x\_intl\_calls’\]=intl\_plan\_num\*df\[’total\_intl\_calls’\]\#===Frombestprogram\(0\.5685\):intl\_planxservice\_calls===\#Combinesthetwostrongestoriginalpredictorsdf\[’intl\_plan\_x\_svc\_calls’\]=intl\_plan\_num\*df\[’number\_customer\_service\_calls’\]\#===Statetargetencodingwithheavysmoothing===enc=state\[’state\_target\_enc’\]gm=state\[’global\_mean’\]df\[’state\_churn\_rate’\]=df\[’state’\]\.astype\(str\)\.map\(enc\)\.fillna\(gm\)\#Total:3drops\+5new=8changes\#Penalty:0\.10\*sqrt\(8/2333\)=0\.0059\#Removedsvc\_calls\_x\_charge\(HARMFUL\-0\.004\)andintl\_plan\_x\_day\_min\(noise\)returndf\#EVOLVE\-BLOCK\-ENDdefbuild\_feature\_program\(seed=42\):returnFeatureProgram\(seed=seed\)defget\_feature\_program\(seed=42\):returnbuild\_feature\_program\(seed=seed\)