TabFM-Auto: Self-Evolving Pipelines for Tabular Foundation Models

Hugging Face Daily Papers Papers
tabular-foundation-models self-evolving-pipelines autoML machine-learning data-pipeline

Summary

TabFM-Auto introduces a self-evolving pipeline system that pairs a tabular foundation model with a language model agent to iteratively refine data processing, achieving top performance on TabArena and MLE-Bench benchmarks with transferability to other models.

Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, task descriptions, and auxiliary files that carry dataset semantics. Meanwhile, self-evolving machine learning engineering (MLE) agents train models from scratch on each dataset, yet jointly searching over features, architectures, and hyperparameters is noisy and prone to overfitting. We introduce TabFM-Auto, which pairs a tabular foundation model, TabFM, with a language model agent that evolves the data pipeline around it. Guided by dataset metadata and validation feedback, TabFM-Auto iteratively refines data cleaning, feature engineering, context selection, and post-processing to reduce TabFM's error. Across all 51 datasets of the TabArena benchmark, five TabFM-Auto configurations with different agents and language models take the top five overall positions, and the best raises TabFM from 1785 to 2013 Elo. The discovered pipelines also transfer to other frozen tabular foundation models (+69 to +143 Elo) with no further search. On the 8 tabular competitions of MLE-Bench, TabFM-Auto ranks first overall among MLE agents.
Original Article
View Cached Full Text

Cached at: 09/30/26, 04:14 AM

Paper page - TabFM-Auto: Self-Evolving Pipelines for Tabular Foundation Models

Source: https://huggingface.co/papers/2609.37989

Abstract

Tabularfoundationmodelsachievestrongzero-shotaccuracyonstructureddatabypretrainingonsynthetictables,buttheyignorethecolumnnames,taskdescriptions,andauxiliaryfilesthatcarrydatasetsemantics.Meanwhile,self-evolvingmachinelearningengineering(MLE)agentstrainmodelsfromscratchoneachdataset,yetjointlysearchingoverfeatures,architectures,andhyperparametersisnoisyandpronetooverfitting.WeintroduceTabFM-Auto,whichpairsatabularfoundationmodel,TabFM,withalanguagemodelagentthatevolvesthedatapipelinearoundit.Guidedbydatasetmetadataandvalidationfeedback,TabFM-Autoiterativelyrefinesdatacleaning,featureengineering,contextselection,andpost-processingtoreduceTabFM’serror.Acrossall51datasetsoftheTabArenabenchmark,fiveTabFM-Autoconfigurationswithdifferentagentsandlanguagemodelstakethetopfiveoverallpositions,andthebestraisesTabFMfrom1785to2013Elo.Thediscoveredpipelinesalsotransfertootherfrozentabularfoundationmodels(+69to+143Elo)withnofurthersearch.Onthe8tabularcompetitionsofMLE-Bench,TabFM-AutoranksfirstoverallamongMLEagents.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2609\.37989

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.37989 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.37989 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.37989 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

TabFM: A zero-shot foundation model for tabular data

Hacker News Top

Google Research introduces TabFM, a zero-shot foundation model for tabular data that uses in-context learning to perform classification and regression without requiring manual model training or hyperparameter tuning.

TabPFN-3.5: Technical Report

arXiv cs.LG

TabPFN-3.5 is a new tabular foundation model that sets state-of-the-art performance across multiple benchmarks, with improvements in inference speed and multimodal capabilities.

Training Fair Tabular Foundation Models

arXiv cs.LG

This paper proposes FairTFM, a training strategy to incorporate fairness constraints into tabular foundation models, enabling fair predictions via in-context learning without task-specific retraining.

Do Tabular Foundation Models Agree with Themselves?

arXiv cs.LG

This paper investigates whether tabular foundation models (TFMs) like TabPFN, TabICL, TabDPT, and TabFM produce predictions consistent with any joint distribution. It demonstrates that all evaluated TFMs violate both marginalization and factorization consistency for classification and regression, questioning their Bayesian inference claims.