Beyond IID: How General Are Tabular Foundation Models, Really?

Hugging Face Daily Papers Papers
tabular-data foundation-models benchmarking iid non-iid machine-learning deep-learning

Summary

This paper introduces BeyondArena, a unified holistic benchmark for tabular data, and finds that existing tabular foundation models excel only on small to medium-sized IID data, while traditional tree-based and deep learning models still dominate on non-IID, large, and high-dimensional datasets.

Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry. Research communities across disciplines are increasingly evaluating tabular foundation models on diverse datasets and tasks. However, these task- and discipline-specific evaluations remain largely inaccessible to model researchers because benchmark software and evaluation protocols are fragmented. As a result, model researchers rely on standard benchmarks, which are mostly defined for tasks where tabular foundation models already excel. The most challenging scenarios are excluded, limiting meaningful progress in the field by focusing on marginal improvements on IID data rather than on broader, more demanding challenges. To overcome this, we introduce BeyondArena, the first unified holistic benchmark for tabular data that supports diverse task types (IID, temporal, grouped), across sample size and feature dimensionality scales, with diverse feature types (with text, with high cardinality) from a broad range of disciplines. To enable unified benchmarking beyond standard benchmarks, we introduce Data Foundry, a Python framework and metadata schema for curating tabular datasets for predictive machine learning. Our results across 11 models and 142 curated datasets show that existing tabular foundation models excel on tiny- to medium-sized IID data, while traditional tree-based and deep learning models still dominate on non-IID, large, and high-dimensional datasets. BeyondArena guides model research for the most demanding challenges in tabular data, enabling progress towards truly foundational tabular models.
Original Article
View Cached Full Text

Cached at: 06/30/26, 11:35 AM

Paper page - Beyond IID: How General Are Tabular Foundation Models, Really?

Source: https://huggingface.co/papers/2606.30410

Abstract

Tabular foundation models show varying performance across different data conditions, with traditional methods still outperforming newer approaches on complex, large-scale datasets.

Foundation models forpredictive machine learningon tabular data have recently gained significant traction in academia and industry. Research communities across disciplines are increasingly evaluatingtabular foundation modelson diverse datasets and tasks. However, these task- and discipline-specific evaluations remain largely inaccessible to model researchers because benchmark software and evaluation protocols are fragmented. As a result, model researchers rely on standard benchmarks, which are mostly defined for tasks wheretabular foundation modelsalready excel. The most challenging scenarios are excluded, limiting meaningful progress in the field by focusing on marginal improvements onIID datarather than on broader, more demanding challenges. To overcome this, we introduce BeyondArena, the first unified holistic benchmark for tabular data that supports diverse task types (IID, temporal, grouped), across sample size and feature dimensionality scales, with diverse feature types (with text, with high cardinality) from a broad range of disciplines. To enable unifiedbenchmarkingbeyond standard benchmarks, we introduceData Foundry, a Python framework and metadata schema for curating tabular datasets forpredictive machine learning. Our results across 11 models and 142 curated datasets show that existingtabular foundation modelsexcel on tiny- to medium-sizedIID data, while traditional tree-based anddeep learning modelsstill dominate on non-IID, large, andhigh-dimensional datasets. BeyondArena guides model research for the most demanding challenges in tabular data, enabling progress towards truly foundational tabular models.

View arXiv pageView PDFProject pageAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.30410 in a model README.md to link it from this page.

Datasets citing this paper1

#### TabArena/BeyondArena Viewer• Updatedabout 4 hours ago • 19.7M • 2.16k • 2

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.30410 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

When Tabular Foundation Models Meet Strategic Tabular Data: A Prior Alignment Approach

arXiv cs.AI

This paper studies whether tabular foundation models based on pretrained prior-data fitted networks (PFNs) can generalize to strategic tabular data where individuals modify features after deployment. It proposes Strategic Prior-data Fitted Network (SPN), an inference-time framework that aligns PFN predictions with the post-manipulation distribution without retraining.

TabPFN-3: Technical Report

arXiv cs.LG

TabPFN-3 is a new foundation model for tabular data, pretrained on synthetic data, that scales to 1M training rows while reducing training and inference time, achieving state-of-the-art performance on tabular prediction, time series, and relational data.