The Importance of Encoder Choice:A Tabular-Image Study

arXiv cs.LG Papers
multimodal-learning tabular-data encoder-choice deep-learning in-context-learning arxiv

Summary

This paper evaluates state-of-the-art tabular models as encoders in image-tabular multimodal learning, addressing the challenge of using In-Context Learning models that require labels for embedding training and test instances.

arXiv:2607.07756v1 Announce Type: new Abstract: Multimodal learning usually requires a dedicated encoder per modality. When a tabular modality is involved, prior work has been mostly using a \emph{plain MLP} as the encoder. Yet if it were a strong encoder, the tabular domain would not be ``the last unconquered castle for deep learning''. This study evaluates state-of-the-art tabular models as encoders in the image-tabular setting for the first time. An obstacle stands out. In-Context Learning models, among the best performing methods in the tabular domain, require labels to process instances, making it non-trivial to embed training and test instances the same way. We addressed this problem across multiple models of this family. With this study, we would like to highlight the importance of encoder factor in the multimodal learning.
Original Article
View Cached Full Text

Cached at: 07/10/26, 06:14 AM 

# The Importance of Encoder Choice:A Tabular-Image Study
Source: [https://arxiv.org/abs/2607.07756](https://arxiv.org/abs/2607.07756)
[View PDF](https://arxiv.org/pdf/2607.07756)

> Abstract:Multimodal learning usually requires a dedicated encoder per modality\. When a tabular modality is involved, prior work has been mostly using a \\emph\{plain MLP\} as the encoder\. Yet if it were a strong encoder, the tabular domain would not be \`\`the last unconquered castle for deep learning''\. This study evaluates state\-of\-the\-art tabular models as encoders in the image\-tabular setting for the first time\. An obstacle stands out\. In\-Context Learning models, among the best performing methods in the tabular domain, require labels to process instances, making it non\-trivial to embed training and test instances the same way\. We addressed this problem across multiple models of this family\. With this study, we would like to highlight the importance of encoder factor in the multimodal learning\.

## Submission history

From: Ilia Koloiarov \[[view email](https://arxiv.org/show-email/f67b19e0/2607.07756)\] **\[v1\]**Wed, 8 Jul 2026 14:01:16 UTC \(2,746 KB\)

Similar Articles

Stateful Visual Encoders for Vision-Language Models

Hugging Face Daily Papers

This paper introduces a stateful visual encoder for vision-language models that conditions visual representations on prior features, enabling better visual comparison in multi-image and agentic settings. The method shows consistent improvements across tasks such as cross-image spatial aggregation and longitudinal radiology.

MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image

Hugging Face Daily Papers

Introduces MulTaBench, a benchmark of 40 datasets for multimodal tabular learning with text and image modalities, demonstrating that task-specific embedding tuning improves performance over frozen pretrained embeddings, particularly when modalities provide complementary predictive signals.

Probing Memorization of Tabular In-Context Learning

arXiv cs.LG

This paper investigates parametric memorization in tabular foundation models that use in-context learning, introducing a probing framework (IclMem) to separate context-based predictions from memorization. It finds moderate memorization signals under specific conditions but notes they largely vanish under realistic training scenarios.