表格数据上的类型化决策模型

# Typed Decision Models on Tabular Data
Source: [https://arxiv.org/html/2610.19536](https://arxiv.org/html/2610.19536)
Hangting Ye††thanks:Corresponding author\.Affiliation:School of Artificial Intelligence, Jilin UniversityEmail:[yeht22@mails\.jlu\.edu\.cn](mailto:[email protected])Hongyuan ZhaAffiliation:School of Data Science, Chinese University of Hong Kong, ShenzhenEmail:[zhahy@cuhk\.edu\.cn](mailto:[email protected])

###### Abstract

Typed decision models are language models that answer a question with a fixed set of possible answers by returning a probability for each answer instead of generating text\. We study them on tabular data and compare four decision models with three large language models \(LLMs\) on row classification, fact verification and anomaly detection\. Every request is sent in its original form and under 21 types of rewrite that keep its content, such as renaming the options or writing the table in another format\. The decision models perform as well as the LLMs at a lower cost\. Three of them are also far more robust to how the question is formulated\. No model we test is robust to how the table is organized\. For every model, at least a third of the row decisions change under some rewrite of the table or the question\. The decisions to which a decision model gives low probability are the ones most likely to change\. Decision models are a practical choice for tabular decisions, but their robustness remains an open problem\.

## 1Introduction

Tabular data, in which each row describes a sample and each column one of its heterogeneous features, is one of the most common forms of data in fields such as healthcare, finance and network security\([Borisov et al\., 2022](https://arxiv.org/html/2610.19536#bib.bib40);[Fang et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib8)\)\. On such data, gradient\-boosted trees\([Chen and Guestrin, 2016](https://arxiv.org/html/2610.19536#bib.bib41);[Ke and others, 2017](https://arxiv.org/html/2610.19536#bib.bib42);[Prokhorenkova and others, 2018](https://arxiv.org/html/2610.19536#bib.bib43)\)still outperform most deep networks designed for tables\([Grinsztajn et al\., 2022](https://arxiv.org/html/2610.19536#bib.bib37);[Borisov et al\., 2022](https://arxiv.org/html/2610.19536#bib.bib40)\), and tabular foundation models do best on small tables\([Hollmann et al\., 2025](https://arxiv.org/html/2610.19536#bib.bib35);[Prior Labs, 2026](https://arxiv.org/html/2610.19536#bib.bib36)\)\.

Table 1:Our four research questions and their answers for the decision models and the LLMs\.Large language models \(LLMs\) have opened a new route for tabular data\([Fang et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib8)\)\. LLM\-based methods first serialize a row or a table into text, for example as feature–value pairs or as a Markdown table\([Hegselmann et al\., 2023](https://arxiv.org/html/2610.19536#bib.bib2);[Sui et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib13)\)\. The serialized row is then classified by prompting an LLM with a description of the task and a few in\-context examples, or by fine\-tuning it\([Dinh et al\., 2022](https://arxiv.org/html/2610.19536#bib.bib14);[Hegselmann et al\., 2023](https://arxiv.org/html/2610.19536#bib.bib2)\)\. The LLM can also turn the labelled rows into features and rules for a conventional model\([Han et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib1);[Ye et al\., 2025](https://arxiv.org/html/2610.19536#bib.bib39)\)\. This works well even when only a few labelled rows are available\([Hegselmann et al\., 2023](https://arxiv.org/html/2610.19536#bib.bib2)\)\. However, LLMs are not built for decisions\. The LLM\-generated output has to be mapped to one of the options, and every decision needs a full decoding process, which can be expensive when a large number of decisions need to be made\. Their answers can also change with the order of the columns, the format of the rows or the wording of the question\([Sui et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib13);[Liu et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib18);[Sclar et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib19)\)\.

Recently, a new class of language models has been built for such decisions:*typed decision models*\(decision models for short\)\. TypeSafe released the first of them, Jev\([TypeSafe, 2026](https://arxiv.org/html/2610.19536#bib.bib9)\)\. Jev takes a sample or a table together with one or more questions, each with a fixed set of possible answers such as the classes of a task\. Instead of generating text, it returns a probability for every answer of every question in a single request\. TypeSafe trains it with Reinforcement Learning for Calibrated Decisions \(RLCD\) so that these probabilities match outcomes, and reports that its confidence is calibrated and that similar inputs receive similar answers\([TypeSafe, 2026](https://arxiv.org/html/2610.19536#bib.bib9)\)\. Open\-weight decision models such as Clef and Kev\-27B followed, trained from an open\-weight LLM with proper scoring rules\([Cloudflare, 2026](https://arxiv.org/html/2610.19536#bib.bib15);[Palmer, 2026](https://arxiv.org/html/2610.19536#bib.bib17)\)\. OpenAI now also offers GPT\-6 Luna Decisions, a decision model built on its LLM GPT\-6 Luna, through its Decisions API\([OpenAI, 2026a](https://arxiv.org/html/2610.19536#bib.bib11)\)\. Decision models thus avoid generation and decoding, since every decision comes with a probability in a single request\. Whether they are also robust to how the table and the question are written is open, since a decision model also reads the table as text\.

Of the 112 arXiv papers on decision models published in the three weeks after the release of Jev, none studies them as a method for tabular data \(Table[11](https://arxiv.org/html/2610.19536#A3.T11)\)\. For LLMs on tables, decisions are known to depend on the serialization\([Sui et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib13);[Hegselmann et al\., 2023](https://arxiv.org/html/2610.19536#bib.bib2)\), on the order of rows and columns\([Liu et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib18);[Zhou et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib34)\)and on the in\-context examples\([Hegselmann et al\., 2023](https://arxiv.org/html/2610.19536#bib.bib2)\)\. These questions have not been asked of decision models\.

To study them in depth, we change how each request is written without changing the information the decision needs, for example by renaming the options or writing the table in another format\. We call such a change a*rewrite*\. We send every request in its original form and in each rewritten form to decision models and LLMs\.

Our research questions are as follows:

1. RQ1Are decision models practical for tabular decisions?
2. RQ2Are they robust to the formulation of the question, such as the names and order of the options?
3. RQ3Are they robust to how the table is organized, such as its format and the order of its rows and columns?
4. RQ4Does their confidence indicate which decisions are sensitive to rewrites?

We evaluate seven models—four decision models \(Jev, Clef, Kev\-27B and GPT\-6 Luna Decisions\) and three LLMs \(GPT\-6 Luna, Qwen3\.8\-27B and Llama\-3\.3\-70B\)—across three tabular tasks covering row classification, fact verification and anomaly detection, under 21 types of rewrite\. Clef and Kev\-27B are trained from Qwen3\.8\-27B, which lets us compare the two ways of deciding on the same backbone \(Section[2](https://arxiv.org/html/2610.19536#S2)\)\.

Our main findings are as follows:

Decision models perform as well as LLMs on these tasks and are cheaper to run \(Table[1](https://arxiv.org/html/2610.19536#S1.T1)\)\. They are also far less affected by how the question is formulated\. In contrast, GPT\-6 Luna Decisions is as sensitive to the order of the options as the LLMs\. Writing the table differently, however, changes many decisions of every model, even though accuracy stays almost the same\. The decisions that a decision model gives low probability are the ones that change most often\.

## 2Experimental setup

### 2\.1Decision interfaces

We refer to a model’s output for one item as a*decision*: the class of a sample, the verdict on a claim about a table, or whether a sample is anomalous\. A*request*is one call to a model and may contain several items\. A decision model receives a request as a*state*, such as a group of samples or a table, with one question per item whose possible answers are fixed in advance\. It returns a probability for every option of every question in a single request\([TypeSafe, 2026](https://arxiv.org/html/2610.19536#bib.bib9)\)\. An LLM receives a prompt built from the same request and generates text, which we map to one of the options\. Its confidence has to be read from that text, from its token probabilities or from repeated draws\([Li and Wagle, 2026](https://arxiv.org/html/2610.19536#bib.bib26);[Deußer et al\., 2026](https://arxiv.org/html/2610.19536#bib.bib25)\)\.

### 2\.2Tasks and models

Table 2:The three tasks: row classification on the ten datasets of TabLLM and FeatLLM\([Hegselmann et al\., 2023](https://arxiv.org/html/2610.19536#bib.bib2);[Han et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib1)\), fact verification on TabFact\([Chen et al\., 2020](https://arxiv.org/html/2610.19536#bib.bib3)\)and anomaly detection on ReTabAD\([Yoon et al\., 2026](https://arxiv.org/html/2610.19536#bib.bib4)\)\. Metrics: Macro\-F1, accuracy and AUROC, averaged over datasets or sources\.Figure 1:One row\-classification request \(blood donation\) under the rewrites of Figure[3](https://arxiv.org/html/2610.19536#S3.F3), shortened to the part that each rewrite changes\. Shuffling or reversing the columns permutes their names, descriptions and values together\.#### Tasks\.

We use three tasks \(Table[2](https://arxiv.org/html/2610.19536#S2.T2)\)\. Row classification assigns a class to a sample from the ten datasets of TabLLM and FeatLLM\. Fact verification decides whether a table from TabFact supports a claim\. Anomaly detection decides whether a sample from one of 20 ReTabAD sources is anomalous\.

#### Models\.

The four decision models are Jev 1\.13\([TypeSafe, 2026](https://arxiv.org/html/2610.19536#bib.bib9)\), the open\-weight Clef\([Cloudflare, 2026](https://arxiv.org/html/2610.19536#bib.bib15)\)and Kev\-27B\([Palmer, 2026](https://arxiv.org/html/2610.19536#bib.bib17)\), and GPT\-6 Luna Decisions\([OpenAI, 2026a](https://arxiv.org/html/2610.19536#bib.bib11)\)\. Jev is available only through TypeSafe’s API\. Clef and Kev\-27B are both trained from Qwen3\.8\-27B\([Qwen Team, 2026](https://arxiv.org/html/2610.19536#bib.bib16)\)\. GPT\-6 Luna Decisions is built on the LLM GPT\-6 Luna and served through OpenAI’s Decisions API\. It accepts requests in the same format as Jev\. OpenAI does not document how the API computes its probabilities, and we use its public\-beta snapshot of 6 October 2026\. The LLMs are GPT\-6 Luna\([OpenAI, 2026b](https://arxiv.org/html/2610.19536#bib.bib10)\)and the open\-weight Qwen3\.8\-27B and Llama\-3\.3\-70B\([Llama Team, AI @ Meta, 2024](https://arxiv.org/html/2610.19536#bib.bib38)\)\. We run GPT\-6 Luna at temperature 1 and decode the open\-weight LLMs greedily\. Appendix[A](https://arxiv.org/html/2610.19536#A1)lists all versions\.

In row classification, a decision\-model request contains the 16 rows of a group as separate typed questions over a shared state, whereas an LLM prompt contains one row, as in TabLLM\. In fact verification and anomaly detection, every model receives the same claim or the same batch of 15 samples\. All decision models receive identical requests, and the LLM prompts are built from them\.

Figure 2:In\-context examples \(RQ1\)\. Grey lines show the best of eight classifiers or five anomaly detectors trained on the same rows, and dotted lines the best trained on all training rows\.*max*stands for as many rows as fit in the context\. In \(c\),*stats*is the original request of ReTabAD, which summarizes the normal training samples by statistics\.*none*leaves these statistics out, and*\+16*and*\+max*add raw normal samples\.

### 2\.3Evaluation

We evaluate row classification by Macro\-F1, fact verification by accuracy and anomaly detection by AUROC, each averaged over datasets or sources \(Table[2](https://arxiv.org/html/2610.19536#S2.T2)\)\. We also report the latency and price of each decision, and for RQ1 we vary the number of in\-context examples from none to as many as fit\.

Besides its original form, we send every request under rewrites, which change how it is written but keep all the information the decision needs \(Figure[1](https://arxiv.org/html/2610.19536#S2.F1)\)\. Rewrites of the question serve RQ2, rewrites of the table serve RQ3, and each task uses those of the 21 types of rewrite that apply to it \(Table[12](https://arxiv.org/html/2610.19536#A3.T12)\)\. We checked that each rewrite keeps every cell and header by recovering the original request from it\. The one exception is the order of rows on TabFact, which can change the truth of claims about positions such as “first”\. We leave these claims out of RQ3 and RQ4\.

A decision*flips*when it differs from the decision under the original request, and an invalid output counts as a flip\. Even an identical request can receive a different decision\. We therefore report each model’s*re\-send rate*, the share of its decisions that flip when the identical request is sent again, and compare the flips under rewrites with it\. Section[6](https://arxiv.org/html/2610.19536#S6)defines confidence, and Appendix[C](https://arxiv.org/html/2610.19536#A3)gives every result with intervals\.

## 3RQ1: Are decision models practical for tabular decisions?

Figure 3:Row classification, per rewrite: decisions that differ from the model’s original decision \(left\) and the change of Macro\-F1 \(right\)\. The upper block shows RQ2 and the lower block RQ3\. Filled markers are decision models and hollow markers LLMs\. Clef’s point for option order is hollow because Clef sorts the options\.performancegenerated tokensModelCl\.TFADCl\.TFADJev52\.189\.774\.6none: one passbClef58\.485\.074\.7none: one passKev\-27B55\.288\.881\.2none: one passGPT\-6 Luna Decisions61\.278\.780\.5nonecGPT\-6 Luna62\.579\.077\.35\.611\.887\.5Qwen3\.8\-27B52\.282\.571\.02\.69\.1111\.7Llama\-3\.3\-70B44\.080\.561\.02\.58\.044\.4traineda80\.0n/a78\.8noneTable 3:RQ1: performance under the original request and the tokens each model generates per decision \(an anomaly batch carries 15 decisions\)\. Cl\. = row classification \(Macro\-F1\), TF = TabFact \(accuracy\), AD = anomaly detection \(AUROC\)\.aBest of eight classifiers trained on the whole training split, LightGBM \(Cl\.\), and of five anomaly detectors,kk\-nearest neighbours \(AD, 19 of 20 sources\)\. We train no model for TabFact\.bAs stated by its provider\. Jev’s architecture is not public\.cNo output tokens are billed\.#### Performance\.

Decision models perform as well as LLMs and better on two of three tasks\. Jev verifies TabFact claims best and Kev\-27B detects anomalies best, while GPT\-6 Luna leads on row classification \(Table[3](https://arxiv.org/html/2610.19536#S3.T3)\)\. Clef and Kev\-27B are also more accurate than Qwen3\.8\-27B, which shares their backbone, on every task\. GPT\-6 Luna Decisions comes within 1\.3 points of GPT\-6 Luna on row classification and TabFact and is 3\.2 points above it on anomaly detection\. In\-context examples help the decision models as they help the LLMs\. With as many rows as fit in the context, Clef and Kev\-27B beat every LLM on row classification and every LLM and detector fitted to the same samples on anomaly detection \(Figure[2](https://arxiv.org/html/2610.19536#S2.F2)\)\. From about 64 rows on, TabPFN\-3\.5\([Prior Labs, 2026](https://arxiv.org/html/2610.19536#bib.bib36)\)and methods trained on the same rows match or beat every in\-context model on row classification\.

#### Latency and price\.

The LLMs generate 2\.5 to 112 tokens per decision, and the other models none \(Table[3](https://arxiv.org/html/2610.19536#S3.T3)\)\. Of the closed models, Jev and GPT\-6 Luna Decisions take 0\.08–0\.11 seconds per row or sample, against 0\.73–1\.62 for GPT\-6 Luna \(Table[4](https://arxiv.org/html/2610.19536#S4.T4)\)\. Per decision, GPT\-6 Luna costs 1\.6 to 8 times as much as Jev\. Served through the Decisions API, the same model costs 3\.4 times less on row classification and 1\.8 times less on anomaly detection\. On TabFact, where every request holds a single claim, the two interfaces cost the same per decision\. We time only the closed models, since the open\-weight models run on our own servers\.

Takeaway\.Decision models are practical for tabular decisions\. They perform as well as LLMs or better, and are generally faster and cheaper per decision\. Where 64 or more labelled rows exist, a tabular foundation model or a trained model is better\.

## 4RQ2: Are decision models robust to the formulation of the question?

#### Decision models are robust to the names and order of the options\.

Renaming or reordering the options changes 2–8% of the classifications of Jev, Clef and Kev\-27B, at most 5\.5 percentage points above their re\-send rates \(Figure[3](https://arxiv.org/html/2610.19536#S3.F3), upper block\)\. Their Macro\-F1 changes by at most 1\.3 points\. The same rewrites change 14–30% of the open\-weight LLMs’ classifications and cost them up to 11\.5 points\. Posing the question as yes/no costs these three models at most 2 points but Qwen3\.8\-27B 14, and Llama\-3\.3\-70B answers it with “Yes\.” so often that half of its outputs are unusable\. Only paraphrasing the instructions affects both kinds of model alike\. On TabFact these rewrites change under 5% of their verdicts and under 7% of the open\-weight LLMs’\. Clef and Kev\-27B change far fewer decisions under these rewrites than Qwen3\.8\-27B, which shares their backbone\. For option order, part of this comes from Clef’s release code, which sorts the options\.

Table 4:Latency and price per decision of the closed models: mean wall\-clock seconds, and US$ per million decisions as billed, on the original requests\. A Jev or GPT\-6 Luna Decisions request answers 16 rows or a batch of 15 samples, and a GPT\-6 Luna request one row, one claim or one batch\.
#### GPT\-6 Luna Decisions is not robust to the order of the options\.

GPT\-6 Luna Decisions repeats its decision whenever the identical request is re\-sent\. Yet reversing the options changes 23% of its classifications, 12% of its TabFact verdicts and 22% of its anomaly decisions, and renaming them changes 8–17%\. Few of these flips come from ties of its rounded probabilities\. Even among its classifications with a probability of at least 0\.8, 14% flip, and 92–100% of all flips move to the option now listed first\. Robustness to the question thus does not come with the decision interface itself\.

Takeaway\.Decisions of Jev, Clef and Kev\-27B remain largely unchanged when the options are renamed or reordered\. However, LLMs and GPT\-6 Luna Decisions can change up to 30% of their decisions\.

## 5RQ3: Are decision models robust to how the table is organized?

Table 5:Items \(%\) whose decision changes under at least one information\-preserving rewrite of the table \(6, 10 and 8 types of rewrite, Table[12](https://arxiv.org/html/2610.19536#A3.T12)\)\. GPT\-6 Luna is run at temperature 1, so its shares include its re\-send rate\.#### No language model is fully robust to how the table is organized\.

Writing each row as a list, as sentences, as a JSON object or as a Markdown table instead of an array of values changes 12–21% of the row classifications of every model\. This is about ten times the re\-send rate of the open\-weight models\. Shuffling or reversing the columns changes 15–23% \(Figure[3](https://arxiv.org/html/2610.19536#S3.F3), lower block\)\. On TabFact, other table formats and row or column orders change 3–13% of the decision models’ verdicts and 4–12% of the open\-weight LLMs’\. Here decision models and LLMs do not differ\. Clef and Kev\-27B change about as many decisions as Qwen3\.8\-27B, which shares their backbone\.

#### Nor is any model fully robust to how the in\-context examples are presented\.

We present the same 16 in\-context examples with their classes interleaved instead of grouped, or as text lines instead of arrays\. This changes 7–12% of the decision models’ classifications, 7–8% of Qwen3\.8\-27B’s and 15% of Llama\-3\.3\-70B’s, far above their re\-send rates\. Macro\-F1 changes by at most 3 points\. GPT\-6 Luna’s 14% is close to its re\-send rate\. The presentation of the in\-context examples is thus part of the organization of the table as well\.

Table 6:Of each model’s 25% most confident items, the share \(%\) changed by at least one rewrite of the table\. Cl\. = row classification, TF = TabFact, AD = anomaly detection\.
#### Individual decisions change while overall accuracy stays the same\.

For every model, some rewrite of the table changes the decision on about a third of the row classifications and on a fifth or more of the TabFact claims and anomaly\-detection samples \(Table[5](https://arxiv.org/html/2610.19536#S5.T5)\)\. Re\-sending the request changes only 2–9% of the open\-weight models’ rows\. Yet Macro\-F1 changes by less than a point for some models and by up to 7 for others, because the flips fall on borderline items that are right about as often as wrong \(Section[6](https://arxiv.org/html/2610.19536#S6)\)\. This is the arbitrariness that the multiplicity literature studies across retrained models and prompts\([Marx et al\., 2020](https://arxiv.org/html/2610.19536#bib.bib31);[Ganesh, 2025](https://arxiv.org/html/2610.19536#bib.bib32)\), here caused by the table\.

Takeaway\.Neither decision models nor LLMs are fully robust to how the table is organized\. About a third of every model’s row\-classification decisions change under at least one rewrite of the table\. But accuracy largely hides these changes\.

## 6RQ4: Does their confidence indicate which decisions are sensitive to rewrites?

Every model we test makes some decisions that a rewrite of the table changes \(Section[5](https://arxiv.org/html/2610.19536#S5)\)\. We call such a decision*fragile*, meaning that at least one rewrite of the table flips it, and any other decision*stable*\. We ask whether a model’s confidence indicates which of its decisions are fragile\. Since a table can hold many rows, such a signal is most useful if it comes with each decision rather than from repeated requests\.

#### Confidence\.

Letddbe a model’s original decision on an item\. A decision model returns a probabilityp⁡\(o\)p\(o\)for every optionoo, and its confidence is the probability of the chosen option,

c=maxo⁡p⁡\(o\)\.c=\\max\_\{o\}p\(o\)\.\(1\)This is not theconfidencefield that some of these APIs return, which is the gap between the two highest probabilities\. The closed LLM returns no probabilities, so for all three LLMs we draw ten answersa1,…,a10a\_\{1\},\\dots,a\_\{10\}at temperature 1 and use the share that agrees withdd,

c=110​\|\{k:ak=d\}\|,c=\\tfrac\{1\}\{10\}\\,\\bigl\|\\\{k:a\_\{k\}=d\\\}\\bigr\|,\(2\)where an invalid draw counts as different\. We also compare two other LLM readouts, the stated confidence divided by 100 and the probability of the first token of the generated answer \(Table[10](https://arxiv.org/html/2610.19536#A3.T10)\)\.

#### Measure\.

The*flip AUROC*measures how well low confidence ranks the decisions that a rewrite flips\. It is the AUROC of−c\-cfor a flip over all pairs of an item and a rewrite\. For comparison, the*error AUROC*measures how well−c\-cranks wrong decisions\. For both, 0\.5 is chance\.

#### A decision model’s confidence is a strong indicator of fragile decisions\.

Of the 25% of items with its highest confidence, 0\.1–5% change under some rewrite of the table, against about a third of all items \(Table[6](https://arxiv.org/html/2610.19536#S5.T6)\)\. The exception is GPT\-6 Luna Decisions on TabFact, with 15\.5%\. Pooled over the rewrites both kinds of model receive, the flip AUROC is 0\.82–0\.87 for the decision models on row classification, 0\.81–0\.90 on TabFact and 0\.84–0\.93 on anomaly detection, with GPT\-6 Luna Decisions lowest on every task\. It is similar for each rewrite on its own \(Figure[6](https://arxiv.org/html/2610.19536#A3.F6)\)\. Their mean confidence on a dataset also ranks the ten classification datasets by flip rate \(Spearman−\-0\.62 to−\-0\.90, Figure[7](https://arxiv.org/html/2610.19536#A3.F7)\), as does an LLM’s ten\-draw agreement\.

#### An LLM’s confidence is a weaker indicator of fragile decisions\.

Read as the share of ten draws that repeat the original decision, it ranks the flips less well than the probability of Jev, Clef and Kev\-27B on every task, and the best LLM ranks them about as well as GPT\-6 Luna Decisions \(Table[9](https://arxiv.org/html/2610.19536#A3.T9)\)\. The LLMs’ most confident decisions are also less stable \(Table[6](https://arxiv.org/html/2610.19536#S5.T6)\)\. The confidence an LLM states is the least informative of all\. With the best of its three readouts, the best LLM comes within 0\.02 of the weakest of Jev, Clef and Kev\-27B on row classification and TabFact and stays 0\.05 behind on anomaly detection \(Table[10](https://arxiv.org/html/2610.19536#A3.T10)\), but these readouts need ten decoding passes or token probabilities\.

Figure 4:Re\-querying the least confident rows \(row classification\)\. For the x% of rows with the lowest confidence, the answer in the original format is replaced by the combination of the answers in the original and the four other formats\. The y\-axis gives the share of the Macro\-F1 gain obtained when every row is re\-queried, and the legend gives that gain in points\.
#### Re\-querying the least confident rows\.

Since low confidence marks the decisions that a rewrite is likely to change, it can also tell which rows are worth asking again\. On row classification, we send a share of the rows again in the four other formats of Table[12](https://arxiv.org/html/2610.19536#A3.T12)and combine the five answers, by the highest mean probability for a decision model and by majority vote for an LLM\. Re\-querying every row raises Macro\-F1 by 2\.0 to 6\.4 points for six of the models but takes four more requests per row\. Re\-querying only the 25% least confident rows takes one more request per row on average and recovers 79–89% of the gain for the decision models and 64–86% for the LLMs, against 23–44% for random rows \(Figure[4](https://arxiv.org/html/2610.19536#S6.F4)\)\. Most of this gain comes from leaving the original array of values, the weakest format for six of the seven models, and the best other format alone comes within 0\.2 points of the combination or exceeds it\. The rows whose decisions change with the format are thus mostly the least confident ones, and re\-querying them undoes most of the effect of a weak format\. Kev\-27B, whose weakest format is not the array, loses 0\.7 points \(not shown\)\.

#### Confidence follows stability more than correctness\.

Confident decisions are more often right\. Among the 25% of decisions with the highest confidence, the decision models are right 67–99% of the time, against 43–73% among the 25% with the lowest\. Yet their confidence separates stable from fragile decisions far better than right from wrong ones \(Table[19](https://arxiv.org/html/2610.19536#A3.T19)\)\. On row classification and anomaly detection, stable but wrong decisions carry almost as much confidence as stable and right ones \(0\.67–0\.86 against 0\.76–0\.90\) and far more than fragile ones \(0\.54–0\.66\)\. On TabFact the order is the same, with smaller gaps\. Such stable errors make up 17–23% of the decisions in row classification and anomaly detection and 4–5% on TabFact, and no signal of instability can reveal them\([Ganesh et al\., 2026](https://arxiv.org/html/2610.19536#bib.bib21), cf\.\)\. Llama\-3\.3\-70B’s confidence is high in all four groups \(0\.85–1\.00\)\.

#### Why confidence predicts flips\.

A decision flips when a rewrite moves its*margin*, the gap between the log\-probabilities of the chosen option and the best other option, below zero\([Cohen et al\., 2019](https://arxiv.org/html/2610.19536#bib.bib23);[Bhojanapalli et al\., 2021](https://arxiv.org/html/2610.19536#bib.bib22), cf\.\)\. For the decision models, the flip rates predicted from the margin alone, assuming a small shift that is nearly independent of the item, match the observed ones decile by decile \(Figure[5](https://arxiv.org/html/2610.19536#A3.F5)\)\. Such a shift flips only borderline decisions, which would explain why the flips cancel in accuracy and why the confidence predicts them\.

Takeaway\.A decision model’s probability indicates which of its decisions a rewrite is likely to change\. The confidence of an LLM is a weaker indicator\. Re\-querying only the least confident rows recovers most of the gain of re\-querying every row\.

## 7Related work

#### LLMs on tables\.

LLMs can classify rows serialized as text from a few labelled examples, by prompting or fine\-tuning\([Dinh et al\., 2022](https://arxiv.org/html/2610.19536#bib.bib14);[Hegselmann et al\., 2023](https://arxiv.org/html/2610.19536#bib.bib2)\)\. They can also build features and rules for a conventional model\([Han et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib1);[Ye et al\., 2025](https://arxiv.org/html/2610.19536#bib.bib39)\)\. LLMs also detect anomalous samples\([Tsai et al\., 2025](https://arxiv.org/html/2610.19536#bib.bib7);[Yoon et al\., 2026](https://arxiv.org/html/2610.19536#bib.bib4);[Ye et al\., 2026](https://arxiv.org/html/2610.19536#bib.bib44)\)and verify claims about tables on TabFact\([Chen et al\., 2020](https://arxiv.org/html/2610.19536#bib.bib3)\), and[Fang et al\. \(2024\)](https://arxiv.org/html/2610.19536#bib.bib8)survey the field\. Each of these methods addresses one kind of task and reports accuracy\. We compare LLMs with decision models on three kinds and look at single decisions\.

#### Robustness of LLMs\.

LLM answers change with the order, names and format of the options and with the wording of the prompt\([Zhao et al\., 2021](https://arxiv.org/html/2610.19536#bib.bib5);[Zheng et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib12);[Sclar et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib19);[Mizrahi et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib24)\)\. On tables, the serialization changes results\([Sui et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib13);[Fang et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib8)\), and reordering or transposing a table lowers accuracy on table question answering\([Liu et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib18);[Zhao and others, 2023](https://arxiv.org/html/2610.19536#bib.bib33);[Zhou et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib34)\)\. For single predictions,[Hamman et al\. \(2025\)](https://arxiv.org/html/2610.19536#bib.bib30)find that the probability of tabular LLMs does not reveal their inconsistency across fine\-tuning seeds, and[Ganesh et al\. \(2026\)](https://arxiv.org/html/2610.19536#bib.bib21)find that an LLM’s uncertainty identifies prompt\-sensitive items but not wrong ones\. We measure which individual decisions rewrites of the table change, for LLMs and for models that return a probability by design\.

#### Decision models\.

Early studies measure Jev’s accuracy and calibration on text\([Deußer et al\., 2026](https://arxiv.org/html/2610.19536#bib.bib25)\), its consistency under benign and injected edits\([Hu et al\., 2026](https://arxiv.org/html/2610.19536#bib.bib20)\)and the readout stability of open\-weight decision models\([Li and Chen, 2026](https://arxiv.org/html/2610.19536#bib.bib27)\)\. Others apply decision models to single tabular applications\([Severo et al\., 2026](https://arxiv.org/html/2610.19536#bib.bib29);[Jung, 2026](https://arxiv.org/html/2610.19536#bib.bib28)\), and an audit lists the missing tests\([Tang and Zheng, 2026](https://arxiv.org/html/2610.19536#bib.bib6)\)\. We compare decision models, including one built on an LLM, with LLMs across tabular tasks\.

## 8Discussion

#### When to use decision models?

Decision models suit tabular decisions whose answers form a fixed set, such as classifying rows, verifying claims or flagging anomalies\. In tasks with few labelled rows, they perform as well as LLMs, cost less per decision, respond faster and return a useful confidence with each decision \(Section[3](https://arxiv.org/html/2610.19536#S3)\)\. They are most useful when many rows need a decision, when the wording of the question may vary and when uncertain decisions need to be identified \(Sections[4](https://arxiv.org/html/2610.19536#S4)and[6](https://arxiv.org/html/2610.19536#S6)\)\. They are less suited when many labelled rows are available\. From around 64 labelled rows on, tabular foundation models and trained models perform better \(Section[3](https://arxiv.org/html/2610.19536#S3)\), and decision models have shorter context windows than LLMs, which limits how many labelled rows they can read\. By design they return only one of a fixed set of choices, so they do not cover tasks that need free text, such as an explanation or a computed value\.

#### When to use LLMs?

LLMs suit the task better when the answer has to be written in tokens rather than chosen from a fixed set, for example when a decision must be explained or an open question about a table answered\. A strong LLM is also a reasonable starting point when almost no labelled rows exist \(Section[3](https://arxiv.org/html/2610.19536#S3)\)\. When an LLM is used for decisions with a fixed set of answers, a single question prompt may not be relied on, since presenting the options in several orders and aggregating the answers can reduce its dependence\([Zheng et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib12)\)\.

#### Future directions\.

No model we test is robust to how the table is organized, and this is the main limitation of decision models on tables\. A natural next step is to train decision models on rewrites of the table\. In previous work, LLMs have been trained to predict on unseen tables in few shots more accurately than XGBoost and TabPFN\([Gardner et al\., 2024](https://arxiv.org/html/2610.19536#bib.bib45)\)\. With RLCD, decision models can also follow this path and be trained to improve their decision accuracy and robustness on tabular tasks\. LLMs and decision models can also work together: a decision model can make most of the decisions, and an LLM can explain or review the few it is least confident about\. Systems that combine the two kinds of model in this way are another direction for future work\.

## 9Conclusion

We studied typed decision models as a method for decisions on tabular data\. On row classification, fact verification and anomaly detection, they perform as well as LLMs at a fraction of the cost\. Except for GPT\-6 Luna Decisions, renaming or reordering the options barely changes their decisions\. How the table is organized, however, affects decision models and LLMs alike, and about a third of the row decisions change under some rewrite of the table\. The decisions to which a decision model gives low probability are the ones most likely to change\. Whether decision models can be made robust to how the table is organized remains open\.

## Limitations

First, each of the seven models is tested in one version\. Jev and GPT\-6 Luna Decisions are closed services whose internal processing is not visible to us and may change across versions, and the Decisions API is still in public beta\. The two open\-weight decision models reproduce the findings we rely on, but other decision models, and later versions of the ones we test, may behave differently\.

Second, we cover three kinds of tabular decisions and use subsets of each benchmark, since every request is sent under every rewrite to seven models\. The 21 types of rewrite cover the format of the table, the order of its rows and columns and the formulation of the question\. They do not cover other changes that keep the content, such as renaming columns or converting units, so our results say little about robustness to those changes\.

## References

- Bhojanapalliet al\.\(2021\)S\. Bhojanapalli, K\. Wilber, A\. Veit, A\. S\. Rawat, S\. Kim, A\. Menon, and S\. KumarOn the reproducibility of neural network predictions\.Note:arXiv:2102\.03349Cited by:[§6](https://arxiv.org/html/2610.19536#S6.SS0.SSS0.Px7.p1.1)\.
- Borisovet al\.\(2022\)V\. Borisov, T\. Leemann, K\. Seßler, J\. Haug, M\. Pawelczyk, and G\. KasneciDeep neural networks and tabular data: a survey\.IEEE Transactions on Neural Networks and Learning Systems\.Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p1.1)\.
- Chen and Guestrin \(2016\)T\. Chen and C\. GuestrinXGBoost: a scalable tree boosting system\.InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p1.1)\.
- Chenet al\.\(2020\)W\. Chen, H\. Wang, J\. Chen, Y\. Zhang, H\. Wang, S\. Li, X\. Zhou, and W\. Y\. WangTabFact: a large\-scale dataset for table\-based fact verification\.InInternational Conference on Learning Representations,Cited by:[Table 2](https://arxiv.org/html/2610.19536#S2.T2),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px1.p1.1)\.
- Cloudflare \(2026\)CloudflareIntroducing Clef: our open\-source decision models, and new RL fine\-tuning platform\.Note:Blog post by M\. Chen,[https://blog\.cloudflare\.com/clef\-decision\-models/](https://blog.cloudflare.com/clef-decision-models/)Weights:[https://huggingface\.co/Cloudflare/clef](https://huggingface.co/Cloudflare/clef), Apache\-2\.0Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p3.1),[§2\.2](https://arxiv.org/html/2610.19536#S2.SS2.SSS0.Px2.p1.1)\.
- Cohenet al\.\(2019\)J\. Cohen, E\. Rosenfeld, and Z\. KolterCertified adversarial robustness via randomized smoothing\.InProceedings of the 36th International Conference on Machine Learning,Cited by:[§6](https://arxiv.org/html/2610.19536#S6.SS0.SSS0.Px7.p1.1)\.
- Deußeret al\.\(2026\)T\. Deußer, L\. Sparrenberg, and R\. SifaEvaluating and benchmarking the system one model Jev\.External Links:2609\.37647Cited by:[§2\.1](https://arxiv.org/html/2610.19536#S2.SS1.p1.1),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px3.p1.1)\.
- Dinhet al\.\(2022\)T\. Dinh, Y\. Zeng, R\. Zhang, Z\. Lin, M\. Gira, S\. Rajput, J\. Sohn, D\. Papailiopoulos, and K\. LeeLIFT: language\-interfaced fine\-tuning for non\-language machine learning tasks\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p2.1),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px1.p1.1)\.
- Fanget al\.\(2024\)X\. Fang, W\. Xu, F\. A\. Tan, J\. Zhang, Z\. Hu, Y\. Qi, S\. Nickleach, D\. Socolinsky, S\. Sengamedu, and C\. FaloutsosLarge language models \(LLMs\) on tabular data: prediction, generation, and understanding – a survey\.Transactions on Machine Learning Research\.Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p1.1),[§1](https://arxiv.org/html/2610.19536#S1.p2.1),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px1.p1.1),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px2.p1.1)\.
- Ganeshet al\.\(2026\)P\. Ganesh, R\. Shokri, and G\. FarnadiRethinking hallucinations: correctness, consistency, and prompt multiplicity\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(EACL\),Cited by:[§6](https://arxiv.org/html/2610.19536#S6.SS0.SSS0.Px6.p1.1),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px2.p1.1)\.
- Ganesh \(2025\)P\. GaneshFrom model multiplicity to prompt multiplicity: emerging arbitrariness concerns in the age of generative AI\.InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society,Cited by:[§5](https://arxiv.org/html/2610.19536#S5.SS0.SSS0.Px3.p1.1)\.
- Gardneret al\.\(2024\)J\. Gardner, J\. C\. Perdomo, and L\. SchmidtLarge scale transfer learning for tabular data via language modeling\.InAdvances in Neural Information Processing Systems,Cited by:[§8](https://arxiv.org/html/2610.19536#S8.SS0.SSS0.Px3.p1.1)\.
- Grinsztajnet al\.\(2022\)L\. Grinsztajn, E\. Oyallon, and G\. VaroquauxWhy do tree\-based models still outperform deep learning on typical tabular data?\.InAdvances in Neural Information Processing Systems: Datasets and Benchmarks Track,Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p1.1)\.
- Hammanet al\.\(2025\)F\. Hamman, P\. Dissanayake, S\. Mishra, F\. Lecue, and S\. DuttaQuantifying prediction consistency under fine\-tuning multiplicity in tabular LLMs\.InProceedings of the 42nd International Conference on Machine Learning,Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px2.p1.1)\.
- Hanet al\.\(2024\)S\. Han, J\. Yoon, S\. Ö\. Arik, and T\. PfisterLarge language models can automatically engineer features for few\-shot tabular learning\.InProceedings of the 41st International Conference on Machine Learning,PMLR, Vol\.235,pp\. 17454–17479\.Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p2.1),[Table 2](https://arxiv.org/html/2610.19536#S2.T2),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px1.p1.1)\.
- Hegselmannet al\.\(2023\)S\. Hegselmann, A\. Buendia, H\. Lang, M\. Agrawal, X\. Jiang, and D\. SontagTabLLM: few\-shot classification of tabular data with large language models\.InProceedings of The 26th International Conference on Artificial Intelligence and Statistics,PMLR, Vol\.206,pp\. 5549–5581\.Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p2.1),[§1](https://arxiv.org/html/2610.19536#S1.p4.1),[Table 2](https://arxiv.org/html/2610.19536#S2.T2),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px1.p1.1)\.
- Hollmannet al\.\(2025\)N\. Hollmann, S\. Müller, L\. Purucker, A\. Krishnakumar, M\. Körfer, S\. B\. Hoo, R\. T\. Schirrmeister, and F\. HutterAccurate predictions on small data with a tabular foundation model\.Nature637,pp\. 319–326\.Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p1.1)\.
- Huet al\.\(2026\)J\. Hu, H\. Zhang, Y\. Liu, Y\. Zeng, L\. Zeng, X\. Wang, R\. Wang, and L\. Y\. ZhangJevAdvBench: a benchmark and black\-box attacks for reinforcement learning for calibrated decisions models\.External Links:2609\.31142Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px3.p1.1)\.
- Jung \(2026\)W\. JungWhere can a decision model diagnose HVAC faults? reasoning demand, physical representation, and robustness under shift\.External Links:2610\.09937Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px3.p1.1)\.
- Keet al\.\(2017\)G\. Keet al\.LightGBM: a highly efficient gradient boosting decision tree\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p1.1)\.
- Li and Chen \(2026\)R\. Li and L\. ChenReadout stability in prefill\-only decision models: zero\-label prediction and inference\-time compute allocation\.External Links:2610\.07716Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px3.p1.1)\.
- Li and Wagle \(2026\)Y\. Li and J\. WagleLLM\-as\-Jev: LLMs are already Jev\-style decision models – when and how to fine\-tune them\.External Links:2610\.02076Cited by:[§2\.1](https://arxiv.org/html/2610.19536#S2.SS1.p1.1)\.
- Liuet al\.\(2024\)T\. Liu, F\. Wang, and M\. ChenRethinking tabular data understanding with large language models\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 450–482\.Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p2.1),[§1](https://arxiv.org/html/2610.19536#S1.p4.1),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px2.p1.1)\.
- Llama Team, AI @ Meta \(2024\)Llama Team, AI @ MetaThe Llama 3 herd of models\.External Links:2407\.21783Cited by:[§2\.2](https://arxiv.org/html/2610.19536#S2.SS2.SSS0.Px2.p1.1)\.
- Marxet al\.\(2020\)C\. Marx, F\. Calmon, and B\. UstunPredictive multiplicity in classification\.InProceedings of the 37th International Conference on Machine Learning,Cited by:[§5](https://arxiv.org/html/2610.19536#S5.SS0.SSS0.Px3.p1.1)\.
- Mizrahiet al\.\(2024\)M\. Mizrahi, G\. Kaplan, D\. Malkin, R\. Dror, D\. Shahaf, and G\. StanovskyState of what art? a call for multi\-prompt LLM evaluation\.Transactions of the Association for Computational Linguistics12,pp\. 933–949\.Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px2.p1.1)\.
- OpenAI \(2026a\)OpenAIDecisions API\.Note:[https://developers\.openai\.com/api/docs/guides/decisions](https://developers.openai.com/api/docs/guides/decisions)Public beta since 6 October 2026; modelopenai/gpt\-6\-luna\-decisions, snapshotgpt\-6\-luna\-decisions\-20261006Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p3.1),[§2\.2](https://arxiv.org/html/2610.19536#S2.SS2.SSS0.Px2.p1.1)\.
- OpenAI \(2026b\)OpenAIGPT\-6 Luna\.Note:[https://openrouter\.ai/openai/gpt\-6\-luna](https://openrouter.ai/openai/gpt-6-luna)Snapshotgpt\-6\-luna\-20260922Cited by:[§2\.2](https://arxiv.org/html/2610.19536#S2.SS2.SSS0.Px2.p1.1)\.
- Palmer \(2026\)J\. PalmerKev\.Note:[https://github\.com/jaredpalmer/kev](https://github.com/jaredpalmer/kev)Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p3.1),[§2\.2](https://arxiv.org/html/2610.19536#S2.SS2.SSS0.Px2.p1.1)\.
- Prior Labs \(2026\)Prior LabsTabPFN\-3\.5\.Note:[https://huggingface\.co/Prior\-Labs/tabpfn\_3\_5](https://huggingface.co/Prior-Labs/tabpfn_3_5)Model card and weights, checkpoint tabpfn\-v3\.5\-20260909; TabPFN\-3\.5 Non\-Commercial LicenseCited by:[§1](https://arxiv.org/html/2610.19536#S1.p1.1),[§3](https://arxiv.org/html/2610.19536#S3.SS0.SSS0.Px1.p1.1)\.
- Prokhorenkovaet al\.\(2018\)L\. Prokhorenkovaet al\.CatBoost: unbiased boosting with categorical features\.InAdvances in Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p1.1)\.
- Qwen Team \(2026\)Qwen TeamQwen3\.8\-27B\.Note:[https://huggingface\.co/Qwen/Qwen3\.8\-27B](https://huggingface.co/Qwen/Qwen3.8-27B)Cited by:[§2\.2](https://arxiv.org/html/2610.19536#S2.SS2.SSS0.Px2.p1.1)\.
- Sclaret al\.\(2024\)M\. Sclar, Y\. Choi, Y\. Tsvetkov, and A\. SuhrQuantifying language models’ sensitivity to spurious features in prompt design or: how i learned to start worrying about prompt formatting\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p2.1),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px2.p1.1)\.
- Severoet al\.\(2026\)P\. Severo, S\. E\. Quincozes, and A\. DiasJev\-IDS: a system one model for network intrusion detection in label\-scarce and zero\-day scenarios\.External Links:2610\.01079Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px3.p1.1)\.
- Suiet al\.\(2024\)Y\. Sui, M\. Zhou, M\. Zhou, S\. Han, and D\. ZhangTable meets LLM: can large language models understand structured table data? a benchmark and empirical study\.InProceedings of the 17th ACM International Conference on Web Search and Data Mining,Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p2.1),[§1](https://arxiv.org/html/2610.19536#S1.p4.1),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px2.p1.1)\.
- Tang and Zheng \(2026\)L\. Tang and Y\. ZhengTyped decision models: an early evidence audit and evaluation checklist\.External Links:2609\.32160Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px3.p1.1)\.
- Tsaiet al\.\(2025\)C\. Tsai, G\. Teng, P\. Wallis, and W\. DingAnoLLM: large language models for tabular anomaly detection\.InInternational Conference on Learning Representations,Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px1.p1.1)\.
- TypeSafe \(2026\)TypeSafeJev\.Note:[https://docs\.typesafe\.ai](https://docs.typesafe.ai/)Modeltypesafe/jev\-1\.13, revision 20260917Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p3.1),[§2\.1](https://arxiv.org/html/2610.19536#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2610.19536#S2.SS2.SSS0.Px2.p1.1)\.
- Yeet al\.\(2025\)H\. Ye, J\. Li, H\. Zhao, D\. Guo, and Y\. ChangLLM meeting decision trees on tabular data\.External Links:2505\.17918Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p2.1),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px1.p1.1)\.
- Yeet al\.\(2026\)H\. Ye, J\. Liu, Y\. Yao, J\. Li, P\. Wang, Y\. Cao, H\. Zhao, D\. Guo, Y\. Chang, and H\. ZhaA comprehensive survey of tabular anomaly detection\.Note:Preprints\.org, 202607\.1125[https://www\.preprints\.org/manuscript/202607\.1125](https://www.preprints.org/manuscript/202607.1125)Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px1.p1.1)\.
- Yoonet al\.\(2026\)S\. Yoon, D\. Kim, S\. Yoon, Y\. S\. Sim, S\. Yoa, H\. Cho, S\. Lee, H\. Lee, and W\. LimReTabAD: a benchmark for restoring semantic context in tabular anomaly detection\.InInternational Conference on Learning Representations,Note:arXiv:2510\.02060Cited by:[Table 2](https://arxiv.org/html/2610.19536#S2.T2),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px1.p1.1)\.
- Zhaoet al\.\(2023\)Y\. Zhaoet al\.RobuT: a systematic study of table QA robustness against human\-annotated adversarial perturbations\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics,Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2021\)Z\. Zhao, E\. Wallace, S\. Feng, D\. Klein, and S\. SinghCalibrate before use: improving few\-shot performance of language models\.InProceedings of the 38th International Conference on Machine Learning,Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px2.p1.1)\.
- Zhenget al\.\(2024\)C\. Zheng, H\. Zhou, F\. Meng, J\. Zhou, and M\. HuangLarge language models are not robust multiple choice selectors\.InInternational Conference on Learning Representations,Cited by:[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px2.p1.1),[§8](https://arxiv.org/html/2610.19536#S8.SS0.SSS0.Px2.p1.1)\.
- Zhouet al\.\(2024\)W\. Zhou, M\. Mesgar, H\. Adel, and A\. FriedrichFREB\-TQA: a fine\-grained robustness evaluation benchmark for table question answering\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics,Cited by:[§1](https://arxiv.org/html/2610.19536#S1.p4.1),[§7](https://arxiv.org/html/2610.19536#S7.SS0.SSS0.Px2.p1.1)\.

## Appendix AModels

Table[7](https://arxiv.org/html/2610.19536#A1.T7)lists the models we evaluate, the exact versions and how we run them\.

Table 7:The models we evaluate\. Section[6](https://arxiv.org/html/2610.19536#S6)defines the confidence\. For every LLM, the ten draws are taken at temperature 1\.
## Appendix BCompute

Table[8](https://arxiv.org/html/2610.19536#A2.T8)lists the hardware and the dates of all runs\. The open\-weight models run on servers with four or six NVIDIA A800 80GB PCIe GPUs and on a server with four NVIDIA RTX 6000D GPUs \(84 GB each\)\. The trained methods and TabPFN\-3\.5 run on CPUs\.

Table 8:Hardware and dates of the runs \(2026, UTC\)\. The closed models are called through their providers’ APIs\.

## Appendix CFurther results

Figure 5:Decision models: observed flip rate against the rate predicted when a rewrite adds item\-independent noise to the margin, per margin decile and task\.Table 9:How well a model’s confidence in its original decision predicts \(left\) that an information\-preserving rewrite changes the decision and \(right\) that the original decision is wrong \(flip AUROC and error AUROC; 0\.5 is chance\)\. Left: pooled over the rewrites of Figure[3](https://arxiv.org/html/2610.19536#S3.F3)and the table formats, with 95% cluster\-bootstrap intervals\. Confidence: for the decision models, the returned probability of the chosen option; for the LLMs, the share of ten draws at temperature 1 that give the original decision\.†Strict parse: an answer such as “Yes\.” to the yes/no question counts as invalid, hence as changed\. Ignoring case and a final full stop: Llama\-3\.3\-70B on row classif\. 0\.66 \[0\.65, 0\.67\]\.Table 10:Every readout of an LLM’s confidence, on the same rewrites: flip AUROC \(rewrites shared by both kinds of model\)\.*Ten draws*: share of ten draws at temperature 1 that give the original decision, the readout of the main text\.*Stated*: the confidence the model gives when asked, 0–100 \(anomaly detection: the confidence in its answer\)\.*Answer prob\.*: probability of the first token of the generated label, or the stated anomaly score’s distance from 0\.5; for GPT\-6 Luna, agreement of three draws\. Bold: the best readout of each model and task\. –: not available\.Table 11:Of the 112 arXiv papers on decision models posted between 15 September and 8 October 2026, the fourteen whose main input is a table or table rows\.TaskCodeRewriteSourceRow classificationCOLSColumns shuffled: one fixed permutation per dataset; field descriptions, names and values permuted togetherorderCOLRColumns in reverse orderorderLISTOne line per field, “\- name: value” \(TabLLM list\)formatTEXTOne sentence per field, “The name is value\.” \(TabLLM text\)formatJSONOBJJSON object \{name: value\}formatMDOne\-row Markdown table \(header and value row\)formatTabFactCOLSColumns shuffled: one fixed permutation per table; header and rows permuted togetherorderCOLRColumns in reverse orderorderROWSData rows shuffled; header kept firstborderROWRData rows in reverse orderborderTRANSTable transposed; the format sentence says that each line starts with a column name followed by that column’s valuesorderMDMarkdown tableformatCSVCSV with standard quotingformatJSONRECList of JSON records, one object per row keyed by the headerformatHTMLHTML tableformatTAPEXTAPEX linearization, “col : … row 1 : …”formatAnomaly detectionCOLSFields shuffled within each sampleorderCOLRFields in reverse orderorderROWSSamples of a batch shuffled; sample identifiers unchangedorderROWRSamples of a batch in reverse orderorderJSONOBJEach sample as a JSON objectformatLISTEach sample as “\- name: value” linesformatTEXTEach sample as sentences, “The name is value\.”formatMDThe batch as one Markdown table, sample identifier in the first columnformatDEC3Numeric fields rendered with three decimals instead of twoaformatTable 12:Rewrites of the table that keep the information needed for the decision\. The original format is, for row classification, field descriptions followed by one array of values; for TabFact, the official format with a header line and \# between cells; for anomaly detection, the serializer of the ReTabAD authors with 15 samples per batch\. Source: order of rows and columns, or format of the table\.aAdds a decimal of precision, so it is left out of Tables[5](https://arxiv.org/html/2610.19536#S5.T5)and[6](https://arxiv.org/html/2610.19536#S5.T6)\.bCan change the truth of claims that refer to positions \(*first*,*last*, …\); such claims are left out of Tables[5](https://arxiv.org/html/2610.19536#S5.T5)and[6](https://arxiv.org/html/2610.19536#S5.T6)\.Table 13:How the question is asked, raw results\.*Flip*: share of items whose decision differs from the model’s own decision under the original request \(%; invalid outputs count as flips\)\.*Score*: the task metric×\\times100 \(row classification: Macro\-F1 averaged over the 10 datasets; TabFact: accuracy; anomaly detection: AUROC averaged over the 20 sources\)\.*Identical re\-send*: the original request sent again, which gives the re\-send rate\. Conditions below the line remove or mislead information\. Clef sorts the options, so reversing them cannot reach it\. On row classification the LLMs receive no option descriptions, and on anomaly detection no options\. Llama\-3\.3\-70B answers the yes/no question mostly with “Yes” or “Yes\.”, which the strict parse counts as invalid; with a parse that ignores case and a final full stop its flip rate there is 13\.0%\. Invalid outputs above 5% \(%\): Llama\-3\.3\-70B on row classification, yes/no question: 54\.8; GPT\-6 Luna on TabFact, option names A, B, …: 9\.8; GPT\-6 Luna on TabFact, option names replaced by synonyms: 43\.7; GPT\-6 Luna on TabFact, no task description: 6\.6; GPT\-6 Luna on ReTabAD, paraphrased instructions: 7\.0; Llama\-3\.3\-70B on ReTabAD, paraphrased instructions: 9\.5; GPT\-6 Luna on ReTabAD, no task description: 5\.2\. –: does not apply\.Table 14:Order of rows and columns, raw results\.*Flip*: share of items whose decision differs from the model’s own decision under the original request \(%; invalid outputs count as flips\)\.*Score*: the task metric×\\times100 \(row classification: Macro\-F1 averaged over the 10 datasets; TabFact: accuracy; anomaly detection: AUROC averaged over the 20 sources\)\.*Identical re\-send*: the original request sent again, which gives the re\-send rate\. Shuffling the questions of a request applies only to the decision models, which answer 16 questions per request; the LLMs receive one row per prompt\. Reordering rows can change the truth of TabFact claims with position words\. Invalid outputs above 5% \(%\): GPT\-6 Luna on TabFact, columns shuffled: 6\.7; GPT\-6 Luna on TabFact, columns reversed: 7\.2; GPT\-6 Luna on TabFact, rows shuffled: 5\.9; GPT\-6 Luna on TabFact, rows reversed: 6\.8; GPT\-6 Luna on TabFact, table transposed: 8\.2; GPT\-6 Luna on ReTabAD, columns shuffled: 7\.3; GPT\-6 Luna on ReTabAD, columns reversed: 7\.1; GPT\-6 Luna on ReTabAD, rows shuffled: 8\.8; GPT\-6 Luna on ReTabAD, rows reversed: 8\.0\. –: does not apply\.Figure 6:Row classification, by rewrite\. Left: flip AUROC\. Middle: among flips that change correctness, the share that turn a right answer wrong\. Right: standard deviation of the change of the decision models’ margin\. Clef is omitted for option order, since it sorts the options\.Table 15:Format of the table, raw results\.*Flip*: share of items whose decision differs from the model’s own decision under the original request \(%; invalid outputs count as flips\)\.*Score*: the task metric×\\times100 \(row classification: Macro\-F1 averaged over the 10 datasets; TabFact: accuracy; anomaly detection: AUROC averaged over the 20 sources\)\.*Identical re\-send*: the original request sent again, which gives the re\-send rate\. Conditions below the line remove or mislead information\. Rendering numbers with three decimals adds precision\. Invalid outputs above 5% \(%\): GPT\-6 Luna on TabFact, CSV: 5\.1; GPT\-6 Luna on TabFact, JSON records: 7\.3; GPT\-6 Luna on TabFact, TAPEX linearization: 8\.9; GPT\-6 Luna on ReTabAD, JSON object: 9\.7; GPT\-6 Luna on ReTabAD, list \(“\- name: value”\): 7\.7; GPT\-6 Luna on ReTabAD, text \(“The name is value\.”\): 9\.2; GPT\-6 Luna on ReTabAD, Markdown table: 5\.6; GPT\-6 Luna on ReTabAD, numbers with three decimals: 8\.2; GPT\-6 Luna on ReTabAD, values without names: 10\.9\. –: does not apply\.JevClefKev\-27BGPT\-6 LunaDecisionstrained on the same examplesRow classification \(Macro\-F1\)ScoreΔ\\DeltaScoreΔ\\DeltaScoreΔ\\DeltaScoreΔ\\DeltaLogRegLightGBMBestTabPFNno examples \(zero\-shot\)52\.1–58\.4–55\.1–61\.2–––––4 examples60\.2\+8\.160\.0\+1\.659\.0\+3\.859\.6−\-1\.651\.845\.355\.251\.516 examples \(the original ones\)60\.0\+7\.961\.7\+3\.262\.7\+7\.660\.4−\-0\.858\.651\.858\.759\.564 examplesb65\.3\+12\.268\.2\+9\.069\.3\+12\.265\.4\+3\.369\.269\.169\.470\.8as many as fit \(30–681\)66\.7\+14\.670\.6\+12\.173\.0\+17\.669\.9\+8\.772\.276\.076\.377\.8whole training split––––––––74\.780\.080\.0–16 examples, classes interleaved62\.0\+9\.961\.9\+3\.562\.8\+7\.763\.6\+2\.4––––16 examples, written as text57\.1\+5\.062\.7\+4\.361\.1\+5\.963\.7\+2\.5––––TabFact \(accuracy\)ScoreΔ\\DeltaScoreΔ\\DeltaScoreΔ\\DeltaScoreΔ\\Deltano examples \(zero\-shot\)89\.7–85\.0–88\.8–78\.7–––––2 examples88\.4−\-1\.386\.2\+1\.189\.8\+1\.077\.9−\-0\.8––––4 examples88\.0−\-1\.786\.4\+1\.389\.8\+0\.978\.1−\-0\.6––––8 examples88\.1−\-1\.686\.6\+1\.589\.6\+0\.877\.5−\-1\.3––––as many as fit \(68\)88\.8−\-1\.087\.2\+2\.189\.8\+1\.076\.1−\-2\.6––––Anomaly detection \(AUROC\)ScoreΔ\\DeltaScoreΔ\\DeltaScoreΔ\\DeltaScoreΔ\\DeltaIForestOCSVMkkNNBestno normal\-sample statistics65\.4−\-9\.167\.3−\-7\.373\.8−\-7\.474\.6−\-6\.4––––statistics only \(the original request\)74\.6–74\.7–81\.2–80\.6–––––statistics and 16 normal samples79\.0\+4\.480\.5\+5\.984\.2\+3\.083\.6\+3\.068\.172\.272\.372\.3statistics and as many normal samples as fit80\.8\+6\.384\.1\+9\.586\.5\+5\.385\.7\+5\.275\.676\.178\.578\.5all training normals––––––––75\.775\.778\.878\.8Table 16:In\-context examples, raw results\. Examples are labelled training rows placed in the request \(for TabFact, other tables with a claim and its label; for anomaly detection, normal training records after the normal\-sample statistics\); the sets are nested\.*Score*: task metric×\\times100\.*Δ\\Delta*: change of the score from no examples \(anomaly detection: from the original request\), on the same rows\. The other methods are trained on exactly the same examples: logistic regression with cross\-validated regularization, LightGBM with few\-row settings, and the best of eight trained classifiers \(logistic regression, default and tuned; random forest;kk\-nearest neighbours; histogram gradient boosting, LightGBM, CatBoost and XGBoost\), the line of Figure[2](https://arxiv.org/html/2610.19536#S2.F2); on the normal samples, isolation forest, one\-class SVM,kk\-nearest neighbours and the best of five detectors\.*TabPFN*: TabPFN\-3\.5, a tabular foundation model that reads the same examples in its context without training \(not run on the whole split\); TabPFN v2, run the same way, scores 52\.3, 57\.3, 69\.3, 76\.4 with 4, 16, 64 and as many examples as fit\.bWithout communities \(about 100 features: 64 examples exceed the budget\); the trained methods are averaged over the same nine datasets\. The trained anomaly methods cover 19 of the 20 sources \(covertype left out\)\.GPT\-6 LunaQwen3\.8\-27BLlama\-3\.3\-70BRow classification \(Macro\-F1\)ScoreΔ\\DeltaScoreΔ\\DeltaScoreΔ\\Deltano examples \(zero\-shot\)62\.5–52\.2–44\.0–4 examples64\.9\+2\.459\.2\+7\.057\.6\+13\.516 examples \(the original ones\)64\.5\+2\.059\.5\+7\.356\.7\+12\.664 examplesb66\.2\+3\.065\.4\+10\.457\.4\+12\.5as many as fit \(30–681\)69\.3\+6\.867\.1\+14\.960\.7\+16\.6whole training split––––––16 examples, classes interleaved65\.6\+3\.260\.9\+8\.756\.6\+12\.616 examples, written as text64\.6\+2\.159\.6\+7\.455\.5\+11\.4TabFact \(accuracy\)ScoreΔ\\DeltaScoreΔ\\DeltaScoreΔ\\Deltano examples \(zero\-shot\)79\.0–82\.5–80\.5–2 examples79\.3\+0\.482\.6\+0\.179\.9−\-0\.54 examples79\.5\+0\.683\.5\+1\.180\.7\+0\.28 examples80\.2\+1\.283\.8\+1\.379\.8−\-0\.7as many as fit \(68\)81\.5\+2\.581\.0−\-1\.577\.5−\-2\.9Anomaly detection \(AUROC\)ScoreΔ\\DeltaScoreΔ\\DeltaScoreΔ\\Deltano normal\-sample statistics70\.8−\-6\.558\.9−\-2\.255\.4−\-5\.6statistics only \(the original request\)77\.3–63\.8–61\.0–statistics and 16 normal samples80\.9\+3\.673\.6\+9\.868\.6\+7\.6statistics and as many normal samples as fit82\.3\+5\.077\.3\+13\.472\.3\+11\.4all training normals––––––Table 17:In\-context examples, raw results of the LLMs; setup,*Score*and*Δ\\Delta*as in Table[16](https://arxiv.org/html/2610.19536#A3.T16), where the methods trained on the same examples are listed\.bWithout communities \(about 100 features: 64 examples exceed the budget\)\.Figure 7:Left: flip AUROC within each dataset or source; bars are medians\. Right: for each row classification dataset, the mean confidence of the original decisions against the share of decisions changed by the information\-preserving rewrites\.Table 18:Items whose decision depends on how the table is organized \(%\)\.*All*: share of items whose decision changes under at least one of thekkinformation\-preserving table rewrites of Table[12](https://arxiv.org/html/2610.19536#A3.T12);*1–4*: the same for the items sorted by the model’s confidence in its original decision and split into four bands of 25%, 1 the lowest \(ties broken at random with a fixed seed\)\.*Re\-send*: share of decisions changed by an identical re\-send of the original request \(pp\)\.*Bound*:1−\(1−p\)k1\-\(1\-p\)^\{k\}, the share that would change at least once if every rewrite flipped independently at ratepp; an upper bound on what noise alone explains, since re\-send flips concentrate on the same low\-margin items\.*Question*: share of items changed by at least one rewrite of the question \(options reordered or renamed, yes/no question, option descriptions removed, paraphrased instructions\)\. Brackets \(shown for all items, the outer bands and the question\): 95% cluster\-bootstrap intervals \(Poisson\(1\) weights, 500 replicates; clusters are classification rows, anomaly\-detection samples and TabFact tables\)\. Invalid outputs count as changes\. Numbers with three decimals and TabFact claims with position words are left out\. Jev’s original TabFact decisions carry no probabilities\. Confidence: for the decision models, the returned probability of the chosen option; for the LLMs, the share of ten draws at temperature 1 that give the original decision\. On anomaly detection the LLMs receive no options, so only one question rewrite \(paraphrase\) applies to them\.aIts decisions are themselves draws at temperature 1, so its counts are dominated by temperature noise\.bMostly answers such as “Yes\.” to the yes/no question, which the strict parse counts as invalid\.Figure 8:Items whose decision changes under at least one information\-preserving rewrite of the table\. Items are sorted by the model’s confidence in its original decision and split into four bands of 25%; 95% cluster\-bootstrap intervals\.Figure 9:Share of decisions changed by a rewrite, for items ranked by the model’s confidence in its original decision and split into bands of 10% \(information\-preserving rewrites pooled\)\. Confidence as in the main text: the decision models’ returned probability and the LLMs’ share of ten draws that give the original decision; tied values are split at random\.Table 19:Confidence against correctness and fragility \(RQ4\)\. Second column: share of right decisions among the 25% of decisions with the highest and the lowest confidence\. Other columns: share of decisions \(%\) and their mean confidence in each group; a decision is fragile when at least one rewrite of the table flips it\. Confidence as in §[6](https://arxiv.org/html/2610.19536#S6.SS0.SSS0.Px1)\.