Tag
论文提出 GIST（Generator-state-Informed Supervision for Tabular Foundation Models），利用生成器中间状态的条件标签分布替代硬标签监督，以降低表格基础模型预训练中的采样方差，在 TabArena 和 TALENT 基准上取得更好的平均排名。
论文提出 MotherTree，一个通过合成先验数据进行元学习的表格 Transformer，可在单次前向传播中输出可解释的硬决策树，在小样本任务上表现优于从零训练的梯度方法，并可作为强初始化器进一步提升决策树学习。
This paper proposes using controlled synthetic pretraining with the O'PRIOR task generator to make training-data attribution experimentally verifiable for tabular foundation models, showing that top-attributed task removal measurably degrades held-out performance and revealing that provenance association and interventional faithfulness can diverge.
This paper proposes activation alignment, a lightweight linear transformation trained on synthetic unlabeled data to map partial-context student model activations toward full-context teacher activations, enabling faster tabular in-context learning with little performance loss. Evaluated on 38 TabArena datasets with TabPFN-3 and TabFM, the method recovers nearly half the teacher's advantage in low-data regimes while running on commodity hardware without a GPU.
LoopICL decouples parameter count from depth by using a single weight-tied transformer block with dual cell/row streams and a learned exit gate, matching TabICL v2 on TabArena/TALENT at the same FLOPs while using ~90% fewer parameters.
TabFM-Auto introduces a self-evolving pipeline system that pairs a tabular foundation model with a language model agent to iteratively refine data processing, achieving top performance on TabArena and MLE-Bench benchmarks with transferability to other models.
This paper proposes a training recipe for tabular foundation models using a Joint-Embedding Predictive Architecture (JEPA) to address latent term collapse, and compares its performance to a value-only approach across 147 real datasets.
A controlled study finds that feature engineering gains diminish for stronger tabular foundation models, while adding in-context information from related datasets still improves performance.
LoGIC introduces budgeted context construction for node-level graph in-context learning, optimizing labeled and unlabeled node selection to reduce memory usage and enable scalable inference on large graphs.
This paper investigates whether tabular foundation models (TFMs) like TabPFN, TabICL, TabDPT, and TabFM produce predictions consistent with any joint distribution. It demonstrates that all evaluated TFMs violate both marginalization and factorization consistency for classification and regression, questioning their Bayesian inference claims.
This paper systematically compares tabular foundation models (TabPFN) with classical regression approaches across 85 soil spectroscopy tasks, finding that TabPFN combined with PLS-derived features achieves the best predictive performance from field-scale to global spectral libraries.
This paper investigates memory requirements for tabular foundation models like TabPFN and shows that model compression (e.g., INT4 quantization) can reduce memory footprint up to 7.6x with minimal accuracy loss, improving practical deployment efficiency.
This paper proposes a reformulation to apply tabular foundation models (TFMs) to discrete choice estimation, addressing the structural gap of row-independent assumptions. The best reformulation outperforms hierarchical Bayesian estimation by 8% in holdout log-likelihood and 3.6% in hit rate while running 16 times faster.
This paper investigates whether tabular in-context learning models, pretrained on synthetic causal tables, can generalize to predict biomolecular properties from limited labeled data. The authors find that these models are competitive for protein fitness regression but that representation choice is crucial for small-molecule classification.
This paper presents a formal impossibility result showing that tabular foundation models cannot distinguish legal from rule-violating database states without access to operational rules, verified through the Operational Turing Test (OTT) and empirical experiments.
This paper evaluates the robustness of tabular foundation models to biologically inspired distribution shifts in microbiome data, finding that protecting discriminative features is insufficient and zero-imputation is the most harmful perturbation.
This paper proposes CRUMB, a three-stage inference wrapper that clusters test queries and selects a distributionally matched training subset via MMD minimization to enable efficient Prior-Fitted Network inference on large datasets, achieving state-of-the-art context selection on 51 TabArena datasets.
This paper introduces GOTabPFN, a method that combines Graph-guided Ordering with Local Refinement (GO-LR) and Neuro-Inspired Subunit Compression (NSC) to make small tabular foundation models effective for high-dimensional, low-sample-size prediction without retraining large backbones.
This paper proposes a domain-aware coreset construction pipeline that enables a tabular foundation model to predict flood depth with only 0.7% of the training data, achieving 98.5% of the supervised reference accuracy and allowing transfer across watersheds without retraining.
This paper evaluates the use of tabular foundation models, particularly TabPFN, for calibrating near-infrared spectroscopy data. The model shows strong performance on regression and classification tasks compared to traditional chemometric methods.