py-evoFE is an open-source Python library that uses genetic algorithms to automate and optimize feature engineering for tabular machine learning datasets, with scikit-learn compatibility and Polars for performance.
Hey everyone! I’m excited to announce the release of py-evoFE (v0.3.0) — an open-source Python library that uses genetic algorithms to automatically discover, combine, and optimize feature transformations for tabular datasets. GitHub: https://github.com/tanopereira/py-evoFE PyPI: pip install py-evoFE License: MIT The Problem It Solves Feature engineering is still where most tabular ML competitions and production models are won or lost. While GBDTs like LightGBM and XGBoost excel on raw tabular data, they struggle to discover complex ratios, nested group-by aggregations, nonlinear dimensional projections, and interaction graphs on their own. Manual feature engineering is either tedious or constrained by human intuition, while brute-force feature generation explodes the feature space exponentially with colinear noise and high memory usage. What py-evoFE Does py-evoFE searches the space of possible feature recipes using genetic programming: 1. Hierarchical Chaining: Evolved features become building blocks for future generations (e.g., log(ratio(groupby_mean(x1, by=x2), x3))). 2. 40+ Built-in Transformers: - Non-linear arithmetic & log-ratios - Target encoding (multiclass, pooled, WoE, quantile target encodings) - String similarity (MinHash, Gap encodings) - Manifold & Dimensionality Reduction (PCA, UMAP, MCA, FAMD, Between-Group PCA) - Graph & Density Clustering (Genie, Lumbermark, MST anomaly scoring) 3. Performance & Speed: - Vectorized computation powered by Polars and PyArrow. - Matrix Hashing & Nearest-Neighbor Caching: Stateful projections (like UMAP and $K$-NN lookups) are cached via byte-hashing to eliminate redundant computation across CV folds. - Multi-Fidelity Screening: Fast low-fidelity CV screens initial populations; only promising candidates proceed to full-fidelity evaluation. 4. Island Model & Caruana Ensembling: - Multi-population parallel search across Ring, Torus, Grid, Hypercube, and Tiered topologies with Gibbs migration. - Post-search greedy Caruana ensembling over island winners' out-of-fold predictions. 5. Interactive Replay Viewer: - Run view(evo.get_recipe()) to generate a self-contained, zero-dependency HTML dashboard replaying the evolutionary search over time. 6. 100% Scikit-Learn Compatible: - Implements fit, transform, predict, and predict_proba. Plugs directly into standard sklearn.pipeline.Pipeline and GridSearchCV. Quick Example ```python import polars as pl from sklearn.datasets import load_breast_cancer from sklearn.model_selection import train_test_split from evofe import EvoFE Load data bc = load_breast_cancer(as_frame=True) df = pl.from_pandas(bc.frame) X, y = df.drop("target"), df["target"].to_numpy() X_train, X_test, y_train, y_test = train_test_split( X.to_numpy(), y, test_size=0.2, random_state=42, stratify=y ) X_train_df = pl.DataFrame(X_train, schema=X.columns) X_test_df = pl.DataFrame(X_test, schema=X.columns) 1. Initialize EvoFE evo = EvoFE( task="classification", evaluator="lightgbm", # "lightgbm" | "xgboost" pop_size=15, n_generations=10, cv_folds=3, verbose=True, random_state=42 ) 2. Fit: Runs evolutionary search evo.fit(X_train_df, y_train) 3. Inspect evolved recipe recipe = evo.get_recipe() print(f"Discovered {len(recipe.genes)} high-impact features:") for gene in recipe.genes: print(f" • {gene.to_formula()} -> {gene.output_col}") 4. Transform & Predict preds = evo.predict(X_test_df) proba = evo.predict_proba(X_test_df) ``` Why not just brute-force feature generation? Brute-force libraries generate thousands of features upfront, leading to severe overfitting, massive memory usage, and colinear noise that degrades tree-based models. py-evoFE uses evolutionary selection pressures with complexity penalties to discover compact, parsimonious recipes that actually improve generalization. I’d love for the community to try it out on your datasets or Kaggle benchmarks! Feedback, issues, and feature requests are very welcome on GitHub.
Introduces Evolutionary Feature Engineering (EFE), a framework that uses LLM-based evolution to automatically discover preprocessing transformations for structured data, improving time-series forecasting and tabular prediction accuracy while preserving interpretability.
The paper introduces PACEvolve++, a reinforcement learning framework that improves test-time policy adaptation for evolutionary search agents by decoupling hypothesis generation from execution.
Evolver is a GEP-powered self-evolution engine for AI agents that automates prompt optimization and creates auditable, reusable evolution assets. The project is transitioning from fully open source to source-available while maintaining backward compatibility with existing MIT and GPL-3.0 releases.
This paper proposes a method that uses large language models to generate new features from tabular data via an evolutionary algorithm, demonstrating improved classification results across multiple datasets.
Introduces an evolutionary neural architecture search framework (EvoTS) for discovering task-adaptive Transformer-like models for multivariate time-series forecasting. The approach uses a modular genome representation and achieves competitive performance on ETT benchmark datasets.