Interpretable vs Learned Encoders for High-Cardinality Fraud Detection

# Interpretable versus Learned Encoders for High-Cardinality Fraud Detection
Source: [https://arxiv.org/html/2607.00477](https://arxiv.org/html/2607.00477)
###### Abstract

A total of seven categorical encoding methods were tested on the IEEE\-CIS fraud benchmark dataset \(590,540 records, 3\.5% positives, 8 high\-cardinality columns\)\. The encoders were evaluated using a stratified 5\-fold cross\-validation \(CV\) with three repetitions\. Five of the encoders had identical frozen LightGBM learners in the downstream phase, allowing for controlled comparisons of their performance to each other\. CatBoost and TabNet were included as comparisons across paradigms using different learners\. The entity embeddings produced the highest AUC\-ROC \(0\.9612\), with a statistically significant tie with that of CatBoost \(0\.9602\) and statistically superior to tier group encoding \(0\.9548\), whereas target encoding was only 0\.0023 worse than tier group encoding and the auditor\-friendly tier boundaries were maintained\. Off\-the\-shelf TabNet did not outperform tree\-based pipelines and collapsed under data scarcity\. On AUC\-PR, CatBoost leads \(0\.822 vs\. 0\.793\); no encoder dominated both metrics\. Per\-column analysis confirmed the embedding advantage arises from joint multi\-column representation\.

## IIntroduction

Tabular transaction records contain many high\-cardinality categorical variables \(card identifiers, billing addresses, email domains, device fingerprints\) with cardinalities from four to tens of thousands\. How to represent such categories is an important modeling decision, but evidence is inconsistent: encoder benchmarks stay within the classical family\[[21](https://arxiv.org/html/2607.00477#bib.bib5),[6](https://arxiv.org/html/2607.00477#bib.bib3)\], while tabular deep\-learning studies fix the encoding and vary the model\[[13](https://arxiv.org/html/2607.00477#bib.bib16),[25](https://arxiv.org/html/2607.00477#bib.bib17),[2](https://arxiv.org/html/2607.00477#bib.bib18)\]\. A banking practitioner cannot determine whether neural entity embeddings\[[14](https://arxiv.org/html/2607.00477#bib.bib9)\]or a leakage\-safe target encoder\[[18](https://arxiv.org/html/2607.00477#bib.bib1),[23](https://arxiv.org/html/2607.00477#bib.bib6)\]will serve best in a fraud model when model\-transparency requirements \(such as SR 11\-7 in the US, or analogous audit standards elsewhere\) demand that scores be explainable\.

The question spans three axes\.*Accuracy*margins are narrow, so isolating the encoder from the learner matters\.*Compute*varies widely: end\-to\-end deep models can be an order of magnitude slower than tree\-plus\-embedding pipelines\.*Interpretability*is critical under SR 11\-7, which requires model\-risk owners to justify scores, so a slightly less accurate but more interpretable encoder may be preferable\. We report all three together\.

We address:*what are the accuracy, interpretability, and computational trade\-offs of seven encoders on a large fraud dataset with a fixed downstream learner, and where does lightweight statistical grouping sit on this continuum?*Our contributions:

1. 1\.Cross\-paradigm controlled experiment\.We compared five encoders under a single fixed LightGBM learner on the IEEE\-CIS dataset with stratified CV, plus CatBoost and TabNet as cross\-paradigm reference points\. Previous studies\[[13](https://arxiv.org/html/2607.00477#bib.bib16),[25](https://arxiv.org/html/2607.00477#bib.bib17)\]varied the model family with a default encoder; we varied only the encoder, finding≈\\approx0\.01 AUC\-ROC attributable to encoding alone\.
2. 2\.Auditor\-readable encoding operationalized and benchmarked\.We operationalized and benchmarked target\-aware tier grouping \(Bayesian\-smoothed fraud rates binned intoKKordinal tiers\[[18](https://arxiv.org/html/2607.00477#bib.bib1),[26](https://arxiv.org/html/2607.00477#bib.bib25)\]\) and evaluated it against learned representations\.
3. 3\.Interpretability\-first framing for regulated finance\.We scored interpretability, compute, and metric robustness as primary outcome dimensions alongside SR 11\-7 model\-risk\-management expectations\.

## IIRelated Work

Categorical encoding\.Target encoding with Bayesian smoothing\[[18](https://arxiv.org/html/2607.00477#bib.bib1)\]underpins leakage\-safe variants in CatBoost\[[23](https://arxiv.org/html/2607.00477#bib.bib6)\]\. Other methods include feature hashing\[[29](https://arxiv.org/html/2607.00477#bib.bib4)\], similarity encoding\[[5](https://arxiv.org/html/2607.00477#bib.bib2),[6](https://arxiv.org/html/2607.00477#bib.bib3)\], and conjugate Bayesian encoders\[[26](https://arxiv.org/html/2607.00477#bib.bib25)\]\. Pargent et al\.\[[21](https://arxiv.org/html/2607.00477#bib.bib5)\]find regularized target encoding best among classical encoders; Guo and Berkhahn\[[14](https://arxiv.org/html/2607.00477#bib.bib9)\]established entity embeddings as the deep\-learning default\. Our tier grouping applies the smoothing principles of\[[18](https://arxiv.org/html/2607.00477#bib.bib1),[26](https://arxiv.org/html/2607.00477#bib.bib25)\]and bins smoothed rates into ordinal tiers; we evaluate it as an auditor\-readable encoder\.

Trees versus deep learning on tabular data\.Recent deep models include TabNet\[[1](https://arxiv.org/html/2607.00477#bib.bib10)\], NODE\[[22](https://arxiv.org/html/2607.00477#bib.bib11)\], and FT\-Transformer\[[12](https://arxiv.org/html/2607.00477#bib.bib12)\]\. Grinsztajn et al\.\[[13](https://arxiv.org/html/2607.00477#bib.bib16)\]and Shwartz\-Ziv and Armon\[[25](https://arxiv.org/html/2607.00477#bib.bib17)\]show GBDTs \(XGBoost\[[7](https://arxiv.org/html/2607.00477#bib.bib8)\], LightGBM\[[16](https://arxiv.org/html/2607.00477#bib.bib7)\]\) still match or exceed them; Borisov et al\.\[[2](https://arxiv.org/html/2607.00477#bib.bib18)\]identify interpretability and compute as unresolved trade\-offs\. None of these studies compare encoders across both paradigms\.

Closing the divide\.Encoder benchmarks use classical learners\[[21](https://arxiv.org/html/2607.00477#bib.bib5),[6](https://arxiv.org/html/2607.00477#bib.bib3)\]; tree\-vs\-deep studies fix the encoding\[[13](https://arxiv.org/html/2607.00477#bib.bib16),[25](https://arxiv.org/html/2607.00477#bib.bib17)\]\. No study has treated the encoder as the controlled variable across both\. We close this gap on IEEE\-CIS\[[15](https://arxiv.org/html/2607.00477#bib.bib21)\], a 590K\-transaction fraud task \(≈\\approx3\.5% base rate\)\[[20](https://arxiv.org/html/2607.00477#bib.bib22),[24](https://arxiv.org/html/2607.00477#bib.bib23),[11](https://arxiv.org/html/2607.00477#bib.bib24),[27](https://arxiv.org/html/2607.00477#bib.bib31)\], keeping the learner, folds, and protocol constant\[[4](https://arxiv.org/html/2607.00477#bib.bib26)\]\.

## IIIMethods

We defined seven encoders on the IEEE\-CIS fraud dataset \(590,540 records, 3\.50% positive, eight high\-cardinality columns, cardinalities 4–13,553\)\. E1–E4 and E6 feed the same downstream LightGBM\[[16](https://arxiv.org/html/2607.00477#bib.bib7)\]with frozen hyperparameters; E5 uses CatBoost\[[23](https://arxiv.org/html/2607.00477#bib.bib6)\]as both encoder and learner; E7 feeds target\-encoded input to TabNet\[[1](https://arxiv.org/html/2607.00477#bib.bib10)\]\. All encoders receive identical raw features \(eight high\-card columns, eleven low\-cardinality one\-hot columns, 395 numerical columns\)\. We partition results into*controlled comparisons*\(E1–E4, E6: same learner\) and*cross\-paradigm observations*\(E5, E7: different learner\)\.

The eight high\-cardinality target columns \(with cardinalities\) are:card1\(13,553\),DeviceInfo\(1,786\),addr1\(332\),addr2\(74\),R\_emaildomain\(60\),P\_emaildomain\(59\),card4\(4\),card6\(4\)\.

E1: One\-hot\.Each value becomes an indicator; values that occurred fewer than fifty times in the training fold are dropped \(≈\\approx2,912 features per fold\)\.

E2: Target encoding\.Each value is replaced with its out\-of\-fold mean target via five\-fold inner cross\-fitting with Bayesian smoothing \(m=30m\{=\}30\); a permutation test confirms leakage invariance\.

E3: Frequency\.Each value maps to its relative frequency in the training fold\.

E4: Tier grouping \(interpretable\)\.Per high\-cardinality column on each fold’s training partition: \(1\) compute Bayesian\-smoothed fraud rater^v=\(nv​rv\+m​r¯\)/\(nv\+m\)\\hat\{r\}\_\{v\}=\(n\_\{v\}r\_\{v\}\+m\\bar\{r\}\)/\(n\_\{v\}\+m\),m=30m\{=\}30; \(2\) sort byr^v\\hat\{r\}\_\{v\}and bin intoK∈\{3,5,7\}K\\in\\\{3,5,7\\\}ordinal tiers via equal\-frequency, KS\-greedy, or chi\-square binning; \(3\) replace raw values with tier IDs, monotone in fraud rate\. Unseen test values map to the tier closest tor¯\\bar\{r\}\. Tiers are auditor\-readable: an inspector sees “tier 5==outlook\.com at 16\.5%, tier 1==comcast at 1\.2%” from bin boundaries\. Because several columns are skewed \(addr2is 88% one value\),KKis an upper bound on populated tiers\.

E5: CatBoost\.CatBoost\[[23](https://arxiv.org/html/2607.00477#bib.bib6)\]consumes raw categoricals natively via a permutation\-based ordered target statistic, avoiding leakage without explicit inner CV\. Hyperparameters mirror LightGBM \(iterations=500,learning\_rate=0\.05,depth=7,early\_stopping\_rounds=50\)\.

E6: Entity embeddings→\\toLightGBM\.Following Guo and Berkhahn\[[14](https://arxiv.org/html/2607.00477#bib.bib9)\], a shallow network learns dense embeddings per high\-card column; trained embeddings are extracted and fed to the same frozen LightGBM\. Per\-column dimension ismin\(50,\(nunique\+1\)//2\)\\min\(50,\(n\_\{\\text\{unique\}\}\+1\)//2\)\(255 total\)\. Classifier head: two Linear–ReLU–Dropout\(0\.3\) blocks \(256, 128\), Adam \(l​r=10−3lr\{=\}10^\{\-3\}\), batch 4096, 20 epochs, patience 3\. The extracted matrix averages≈\\approx1,824 columns\. For each outer fold, the embedding network is fitonlyon the training partition; validation labels enter solely via early\-stopping\. TabNet \(E7\) follows the same rule\.

E7: Target encoding→\\toTabNet\.TabNet\[[1](https://arxiv.org/html/2607.00477#bib.bib10)\]with published defaults \(n\_steps=3,n\_a=n\_d=64,γ=1\.5\\gamma\{=\}1\.5, batch 4096, 50 epochs, patience 5\)\. E7 receives the same pre\-encoded numerical feature matrix as E2 \(low\-cardinality columns one\-hot, high\-cardinality columns target\-encoded\) and does not consume raw categorical indices; it differs from E2 only in the downstream learner \(TabNet vs\. LightGBM\), so the E7–E2 contrast isolates the learner with encoding held fixed\. No architecture or hyperparameter search was performed\.

## IVExperimental Setup

Dataset\.IEEE\-CIS Fraud Detection\[[15](https://arxiv.org/html/2607.00477#bib.bib21)\]: 590,540 transactions, 3\.50% positive, 432 features\. Eight high\-cardinality columns \(card1,DeviceInfo,addr1,addr2,R\_emaildomain,P\_emaildomain,card4,card6\) are the encoding targets; eleven low\-cardinality columns are always one\-hot encoded identically\.

Protocol\.Stratified 5\-fold CV×\\times3 repeats==15 runs per encoding\. LightGBM hyperparameters fixed across E1–E4, E6 \(n\_estimators=500,learning\_rate=0\.05,max\_depth=7,early\_stopping\_rounds=50\), with no per\-encoding tuning\. The robustness experiment retrains all encoders on 50/25/10% subsamples \(5 folds per cell, 105 runs\); the per\-column experiment trains E2, E4, E6 on one high\-card column at a time \(5 folds per cell, 120 runs\)\.

Metrics\.We fixedAUC\-ROC as the primary metricin advance of running the experiments and reportAUC\-PR as a secondary, imbalance\-aware metric\. Fixing the primary metric beforehand blocks ex\-post metric\-shopping; we keep AUC\-ROC primary precisely because the two metrics disagree on the winner \(Section V\-D\)\.

Statistical tests\.OurR=15R\{=\}15resamplings have correlated folds, so ordinary paired tests understate variance\. We take theNadeau–Bengio corrected resampledtt\-test\[[19](https://arxiv.org/html/2607.00477#bib.bib27)\]\(ntest/ntrain=0\.25n\_\{\\text\{test\}\}/n\_\{\\text\{train\}\}\{=\}0\.25, df=14\{=\}14\) as primary pairwise inference\. For an illustrative omnibus ranking we run Friedman \+ Nemenyi\[[10](https://arxiv.org/html/2607.00477#bib.bib32)\]\(χ2=80\.9\\chi^\{2\}\{=\}80\.9,p=2\.3×10−15p\{=\}2\.3\\times 10^\{\-15\}, CD=0\.052\.325\{\}\_\{0\.05\}\{=\}2\.325\); the CD diagram serves as a ranking aid alongside the NB\-corrected test\.

Hardware and reproducibility\.All encoders ran on the same Apple\-Silicon machine \(neural models on MPS GPU, tree models on CPU\)\. MPS is low\-throughput relative to NVIDIA, so absolute wall\-clock times overstate DL cost; the cost*ranking*E4<<E6<<E5<<E7 is architectural\. Seeds: fold splits \{42,43,44\}, models 42\. Software: LightGBM 4\.6, CatBoost 1\.2\.10, PyTorch 2\.12\.1, pytorch\-tabnet 4\.1\.0\. The IEEE\-CIS dataset is public\[[15](https://arxiv.org/html/2607.00477#bib.bib21)\]; all hyperparameters and splits are in Sections III–IV\. Reproducibility of execution assumptions is an active concern in financial ML\[[30](https://arxiv.org/html/2607.00477#bib.bib28)\]\.

## VResults

### V\-AMain results and significance

Table[I](https://arxiv.org/html/2607.00477#S5.T1)reports mean±\\pmstd over 15 runs, sorted by AUC\-ROC\. Scores span only 0\.9515–0\.9612 \(≈\\approx0\.97 pp\), with entity embeddings \(E6\) leading at0\.9612, narrowly ahead of CatBoost \(E5, 0\.9602\)\.

TABLE I:Main results, 15 runs \(stratified 5\-fold CV×\\times3 repeats\)\. Sorted by AUC\-ROC\. Type: D==Deep, C==Classical, ML==ML\-native, ip==interpretable\.Table[II](https://arxiv.org/html/2607.00477#S5.T2)provides the NB\-corrected test results that settle the anchor contrasts\. E6 significantly outperforms both E4 \(\+0\.0064\+0\.0064,t=11\.74t\{=\}11\.74,p<0\.001p\{<\}0\.001\) and E7 \(\+0\.0091\+0\.0091,t=6\.95t\{=\}6\.95,p<0\.001p\{<\}0\.001\)\. Conversely, E4 vs\. E7 \(p=0\.089p\{=\}0\.089\) and E6 vs\. E5 \(p=0\.23p\{=\}0\.23\) are statistical ties\. It is also noteworthy thatE2 outperforms E4\(−0\.0023\-0\.0023,p=0\.0003p\{=\}0\.0003\), meaning tier grouping is competitive with but slightly below the classical champion\.

TABLE II:Nadeau–Bengio corrected resampledtt\-test \(correlated CV folds,R=15R\{=\}15,ntest/ntrain=0\.25n\_\{\\text\{test\}\}/n\_\{\\text\{train\}\}\{=\}0\.25, df=14\{=\}14\)\. PositiveΔ\\Delta==first encoder ahead\.The Friedman/Nemenyi CD diagram \(Figure[1](https://arxiv.org/html/2607.00477#S5.F1)\) is illustrative only, since CV folds violate the cross\-dataset independence the test assumes\. The fold\-level CD \(N=5N\{=\}5,k=7k\{=\}7\) does not account for repeat\-level pairing, limiting its power; it reads E6–E4 as a tie \(Δ\\Deltarank 3\.00 vs\. CD=4\.03\{=\}4\.03\) while the properly powered NB\-corrected test finds significance\. The cliques are consistent with Table[II](https://arxiv.org/html/2607.00477#S5.T2): E6/E5/E2 in the top band, E4 just below \(E2 statistically ahead atp=0\.0003p\{=\}0\.0003\), and E7/E3/E1 in the bottom band with E7 indistinguishable from E4 \(p=0\.089p\{=\}0\.089\)\.

![Refer to caption](https://arxiv.org/html/2607.00477v1/x1.png)Figure 1:Per\-run \(N=15N\{=\}15\) Nemenyi critical\-difference diagram \(illustrative; CV folds are correlated\)\. Encoders not joined by a bold bar differ atα=0\.05\\alpha\{=\}0\.05\(CD==2\.325\)\. Primary inference is the NB\-corrected test \(Table[II](https://arxiv.org/html/2607.00477#S5.T2)\)\.
### V\-BAccuracy–compute frontier

Compute cost differences are much larger than accuracy differences\. With tier grouping \(E4\) as the cheapest encoder \(24\.5 s/fold, clearing 0\.954 AUC\), embeddings \(E6\) gain\+0\.0064\+0\.0064AUC at4\.3×\\timesthe time \(104\.6 s\)\. TabNet \(E7\) costs approximately67×\\timesmore \(1654\.3 s/fold\) yet scores 0\.0091 below E6\. E7’s fit time varies \(1263–2255 s\) due to the sample\-size sensitivity of the sparse\-attention architecture; our comparisons use default published parameters with no architecture or hyperparameter search\. Given the total AUC\-ROC spread is≈\\approx0\.97 pp, practical encoder selection depends on compute cost, interpretability, and metric choice\.

### V\-CInterpretability and what the embeddings encode

Figure[2](https://arxiv.org/html/2607.00477#S5.F2)projects E6’sP\_emaildomainembeddings \(30\-dim\) to 2\-D via t\-SNE, colored by E4’s tiers\. K\-means on the 2\-D projection vs\. E4’s tier assignment gives ARI0\.051and NMI0\.175; because t\-SNE does not preserve distances, these values are illustrative of weak visual alignment only and do not quantify representational similarity\. E4’s tiers encode single\-column fraud\-rate ordering; the per\-column probe \(Section V\-E\) provides the sound evidence that E6’s embeddings capture cross\-column structure the tiers discard\.

![Refer to caption](https://arxiv.org/html/2607.00477v1/x2.png)Figure 2:t\-SNE of E6’sP\_emaildomainembeddings, colored by E4 tier\. The weak spatial alignment \(ARI==0\.051\) shows embeddings capture structure beyond single\-column fraud\-rate ordering\.
### V\-DAUC\-ROC vs AUC\-PR: the metrics disagree

Because fraud detection is highly imbalanced \(3\.5% positive\), AUC\-PR is the more deployment\-sensitive metric\. HereCatBoost \(E5\) leads at 0\.8216, well above E6 \(0\.7928\), E7 \(0\.7828\), and E4 \(0\.7704\); the E5–E6 AUC\-PR gap \(0\.029\) is an order of magnitude larger than their AUC\-ROC gap \(0\.001\)\. None of the five LightGBM\-routed encoders approach E5’s precision\-recall performance, and no encoder dominates both metrics\. We fixed AUC\-ROC as the primary metric in advance to prevent metric\-shopping\.

### V\-ERobustness and sensitivity

Data scarcity\.We conducted subsampling at 50/25/10% \(105 runs\) for the six tree\-routed encoders \(E1–E6\)\. All six exhibited nearly parallel degradation from 50% to 10%, with−0\.031\-0\.031to−0\.043\-0\.043AUC\-ROC loss over that interval, andE6 remaining the highest performer at every fraction\(0\.9527/0\.9391/0\.9157 at 50/25/10%\)\. Over the same 50%\-to\-10% interval, E4 lost slightly more than E6 \(−0\.043\-0\.043vs\.−0\.037\-0\.037\), sothe hypothesis that learned embeddings degrade faster than tier grouping is not supported\. The contrast is driven by E7: off\-the\-shelfTabNet collapsedto 0\.8108 at 10%, a loss roughly 3×\\timesany tree\-routed encoder’s, with variance exploding \(std==0\.061 vs\.≤\\leq0\.004\)\. This finding is scoped to stock TabNet; tuning or regularization may improve results\.

Per\-column localization\.E2, E4, and E6 were evaluated \(120 runs\) with training on one high\-cardinality column at a time\. Results indicateE6 underperformed classical encoders on all eight columns\(−0\.0009\-0\.0009to−0\.0045\-0\.0045AUC\-ROC\), confirming that E6’s edge is attributable to*joint multi\-column embedding*: latent co\-occurrence acrosscard1,addr1, andDeviceInfo, rather than any single column\.

Tier\-grouping sensitivity\.As shown in Table[III](https://arxiv.org/html/2607.00477#S5.T3), E4 is largely insensitive to its hyperparameters: AUC\-ROC moves less than 0\.0019 acrossK=3→7K\{=\}3\\\!\\to\\\!7for equal\-frequency binning, the best sub\-variant at everyKK\. TheK=5K\{=\}5headline config \(0\.9548\) is virtually equal to the best \(K=7K\{=\}7, 0\.9556\) within one standard deviation\.

TABLE III:E4 tier\-grouping sensitivity \(mean AUC\-ROC±\\pmstd,n=15n\{=\}15per cell\)\.

## VIDiscussion

Encoding as a confounder\.The confounding role of encoding is evident in the work of Grinsztajn et al\.\[[13](https://arxiv.org/html/2607.00477#bib.bib16)\]and Shwartz\-Ziv and Armon\[[25](https://arxiv.org/html/2607.00477#bib.bib17)\], who varied the model family along with default encodings\. Our approach fixes the learner and varies only the encoder, finding a≈\\approx0\.01 AUC\-ROC swing attributable to encoding alone\. We do not claim this matches the tuned, cross\-dataset GBDT\-vs\-DL gaps those authors report, but our structural point stands: encoder choice is an independent degree of freedom that the trees\-versus\-deep comparisons controlled for implicitly and could not observe\.

Extracted embeddings versus TabNet\.The two deep\-network conditions sit at opposite rank extremes: E7 \(target\-encoded input to TabNet\) is worst and E6 \(extracted embeddings to LightGBM\) is best\. Because E7 shares E2’s target encoding, the E7–E2 gap \(−0\.0049\-0\.0049, 1579 features each\) isolates the downstream learner, showing untuned TabNet trails LightGBM on identical input\. Therefore, this E6–E7 gap illustrates a richer encoding of the data, as well as a more capable learner, and whether tuned TabNet, FT\-Transformer\[[12](https://arxiv.org/html/2607.00477#bib.bib12)\], or other deep tabular architectures will help close this gap remains open\. In addition, task\-specific evaluations have the potential to reverse the expected complexity rank order of conditions in other domains\. For example, Cao et al\.\[[3](https://arxiv.org/html/2607.00477#bib.bib29)\]found that smaller language models perform better than larger models on targeted tasks, which also reflects the result in this study where the most complex pipeline did not outperform all others\. A parallel observation holds for representation learning: Lai et al\.\[[17](https://arxiv.org/html/2607.00477#bib.bib33)\]report that transformer\-based embeddings do not consistently outperform simpler semantic encoders on short\-text classification, echoing our finding that a more elaborate encoder is not universally superior\. Using the controlled LightGBM comparisons, we can cleanly isolate E6 compared to E4 \(p<0\.001p\{<\}0\.001\) and E4 compared to E2 \(p=0\.0003p\{=\}0\.0003\)\.

Transparency costs\.E4’s transparency costs\+0\.0064\+0\.0064AUC\-ROC versus E6 and 0\.0023 versus E2; both gaps are significant but relatively small\. E4 ties E7 \(p=0\.089p\{=\}0\.089\) at 1/67 the cost and trails E2/E5/E6 by at most 0\.0064\. Under SR 11\-7, an auditor can directly inspect tier boundaries \(e\.g\.,card6: debit\-or\-credit at 2\.0% in tier 1, credit at 6\.6% top tier\); a 30\-dim embedding vector offers no such reading\. The interpretability gap that Borisov et al\.\[[2](https://arxiv.org/html/2607.00477#bib.bib18)\]identify can, for the encoding axis, be addressed by tier grouping\.

Metric sensitivity\.AUC\-ROC and AUC\-PR disagree on the top performer: E6 leads AUC\-ROC \(narrowly, tying CatBoost\), while E5 is far ahead on AUC\-PR \(0\.8216 vs\. 0\.7928\)\. This lead reflects CatBoost’s bundled encoder\+learner, not encoding alone, since E5 is a cross\-paradigm observation \(Section III\)\. Sun et al\.\[[28](https://arxiv.org/html/2607.00477#bib.bib30)\]similarly find, in bank\-account fraud, that training objective and class\-imbalance handling can matter more than architecture\. Our result shows that the encoding decision reshapes rankings independently of the learner\. The same pattern recurs across financial modeling, from cross\-market volatility forecasting\[[8](https://arxiv.org/html/2607.00477#bib.bib35)\]to robustness–precision trade\-offs in financial retrieval\[[9](https://arxiv.org/html/2607.00477#bib.bib34)\]\. The two metrics thus produce different winners, and the choice depends on deployment objectives\.

Practical recommendation \(on IEEE\-CIS\)\.Table[IV](https://arxiv.org/html/2607.00477#S6.T4)maps deployment priorities to encoder choice\. Maximum accuracy is achieved with E6 and E5; E4 is the preferred choice under model\-risk constraints\. Off\-the\-shelf TabNet provides no advantage under this protocol, though a tuned TabNet or alternative deep tabular architecture \(e\.g\., FT\-Transformer\[[12](https://arxiv.org/html/2607.00477#bib.bib12)\]\) could produce different results\.

TABLE IV:Practical recommendation by deployment priority \(IEEE\-CIS\)\.
## VIIConclusion

On the IEEE\-CIS fraud benchmark, a controlled comparison of seven categorical encoding strategies yielded measurable differences in accuracy, computational cost, and interpretability\. Learned entity embeddings fed to LightGBM produced the highest AUC\-ROC \(0\.9612\), slightly ahead of CatBoost and\+0\.0064\+0\.0064above tier grouping, an edge limited to joint multi\-column embedding\. The tier grouping method proved competitive and fully auditable, trailing target encoding by only 0\.0023 while offering tier boundaries suitable for SR 11\-7 review\.

The off\-the\-shelf version of TabNet did not outperform tree\-based pipelines and collapsed under data scarcity\. On AUC\-PR, the more deployment\-sensitive metric, CatBoost leads by a wide margin; no single encoder dominated both metrics\. Whether tuned deep tabular architectures close the gap remains to be tested\.

Limitations\.\(1\)*Single dataset\.*We used IEEE\-CIS only; we chose depth over breadth and scope all claims accordingly\. \(2\)*TabNet not exhaustively tuned\.*E7 uses default published parameters; a tuned TabNet, FT\-Transformer\[[12](https://arxiv.org/html/2607.00477#bib.bib12)\], or NODE\[[22](https://arxiv.org/html/2607.00477#bib.bib11)\]could produce different results\. \(3\)*No temporal split\.*We applied stratified random CV; a time\-ordered split could change rankings, particularly for target\-derived encoders \(E2, E4\) whose smoothed rates may be inflated by future\-leaking folds\. \(4\)*Anonymized features\.*V1–V339 are anonymized, so E4’s tier boundaries can be inspected numerically but not mapped to business semantics\. \(5\)*Fixed downstream model\.*LightGBM hyperparameters remain unchanged across E1–E4/E6; per\-encoding tuning might reorder classical encoders\. \(6\)*Small AUC deltas*\(≈\\approx0\.97 pp spread\); the practical signal lies in the compute, interpretability, and data\-scarcity axes\.

## References

- \[1\]S\. Ö\. Arık and T\. Pfister\(2021\)TabNet: attentive interpretable tabular learning\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 6679–6687\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v35i8.16826)Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p2.1),[§III](https://arxiv.org/html/2607.00477#S3.p1.1),[§III](https://arxiv.org/html/2607.00477#S3.p9.2)\.
- \[2\]V\. Borisov, T\. Leemann, K\. Seßler, J\. Haug, M\. Pawelczyk, and G\. Kasneci\(2022\)Deep neural networks and tabular data: a survey\.IEEE Transactions on Neural Networks and Learning Systems34\(10\),pp\. 7499–7519\.External Links:[Document](https://dx.doi.org/10.1109/TNNLS.2022.3229161)Cited by:[§I](https://arxiv.org/html/2607.00477#S1.p1.1),[§II](https://arxiv.org/html/2607.00477#S2.p2.1),[§VI](https://arxiv.org/html/2607.00477#S6.p3.2)\.
- \[3\]J\. Cao, Y\. Ma, X\. Li, Q\. Ren, and X\. Chen\(2026\)Task\-specific efficiency analysis: when small language models outperform large language models\.External Links:2603\.21389,[Link](https://arxiv.org/abs/2603.21389)Cited by:[§VI](https://arxiv.org/html/2607.00477#S6.p2.3)\.
- \[4\]R\. Caruana and A\. Niculescu\-Mizil\(2006\)An empirical comparison of supervised learning algorithms\.InProceedings of the 23rd International Conference on Machine Learning \(ICML\),pp\. 161–168\.External Links:[Document](https://dx.doi.org/10.1145/1143844.1143865)Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p3.1)\.
- \[5\]P\. Cerda, G\. Varoquaux, and B\. Kégl\(2018\)Similarity encoding for learning with dirty categorical variables\.Machine Learning107\(8–10\),pp\. 1477–1494\.External Links:[Document](https://dx.doi.org/10.1007/s10994-018-5724-2)Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p1.1)\.
- \[6\]P\. Cerda and G\. Varoquaux\(2022\)Encoding high\-cardinality string categorical variables\.IEEE Transactions on Knowledge and Data Engineering34\(3\),pp\. 1164–1176\.External Links:[Document](https://dx.doi.org/10.1109/TKDE.2020.2992529)Cited by:[§I](https://arxiv.org/html/2607.00477#S1.p1.1),[§II](https://arxiv.org/html/2607.00477#S2.p1.1),[§II](https://arxiv.org/html/2607.00477#S2.p3.1)\.
- \[7\]T\. Chen and C\. Guestrin\(2016\)XGBoost: a scalable tree boosting system\.InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,pp\. 785–794\.External Links:[Document](https://dx.doi.org/10.1145/2939672.2939785)Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p2.1)\.
- \[8\]K\. Cheng, X\. Qi, Z\. Cheng, and L\. Lai\(2026\)Volatility persistence and model choice in cross\-market volatility forecasting\.Available at SSRN 6610278\.Cited by:[§VI](https://arxiv.org/html/2607.00477#S6.p4.1)\.
- \[9\]Z\. Cheng, L\. Lai, and Y\. Liu\(2026\)Sustainable hybrid document\-routed retrieval for financial RAG: resolving the robustness\-precision trade\-off\.External Links:2603\.26815,[Link](https://arxiv.org/abs/2603.26815)Cited by:[§VI](https://arxiv.org/html/2607.00477#S6.p4.1)\.
- \[10\]J\. Demšar\(2006\)Statistical comparisons of classifiers over multiple data sets\.Journal of Machine Learning Research7,pp\. 1–30\.Cited by:[§IV](https://arxiv.org/html/2607.00477#S4.p4.7)\.
- \[11\]U\. Fiore, A\. D\. Santis, F\. Perla, P\. Zanetti, and F\. Palmieri\(2019\)Using generative adversarial networks for improving classification effectiveness in credit card fraud detection\.Information Sciences479,pp\. 448–455\.External Links:[Document](https://dx.doi.org/10.1016/j.ins.2017.12.030)Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p3.1)\.
- \[12\]Y\. Gorishniy, I\. Rubachev, V\. Khrulkov, and A\. Babenko\(2021\)Revisiting deep learning models for tabular data\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.34,pp\. 18932–18943\.Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p2.1),[§VI](https://arxiv.org/html/2607.00477#S6.p2.3),[§VI](https://arxiv.org/html/2607.00477#S6.p5.1),[§VII](https://arxiv.org/html/2607.00477#S7.p3.1)\.
- \[13\]L\. Grinsztajn, E\. Oyallon, and G\. Varoquaux\(2022\)Why do tree\-based models still outperform deep learning on typical tabular data?\.InAdvances in Neural Information Processing Systems \(NeurIPS\), Datasets and Benchmarks Track,Vol\.35,pp\. 507–520\.Cited by:[item 1](https://arxiv.org/html/2607.00477#S1.I1.i1.p1.1),[§I](https://arxiv.org/html/2607.00477#S1.p1.1),[§II](https://arxiv.org/html/2607.00477#S2.p2.1),[§II](https://arxiv.org/html/2607.00477#S2.p3.1),[§VI](https://arxiv.org/html/2607.00477#S6.p1.1)\.
- \[14\]C\. Guo and F\. Berkhahn\(2016\)Entity embeddings of categorical variables\.arXiv preprint arXiv:1604\.06737\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1604.06737)Cited by:[§I](https://arxiv.org/html/2607.00477#S1.p1.1),[§II](https://arxiv.org/html/2607.00477#S2.p1.1),[§III](https://arxiv.org/html/2607.00477#S3.p8.4)\.
- \[15\]IEEE Computational Intelligence Society and Vesta Corporation\(2019\)IEEE\-CIS fraud detection dataset\.Note:Kaggle Competitionhttps://kaggle\.com/competitions/ieee\-fraud\-detectionCited by:[§II](https://arxiv.org/html/2607.00477#S2.p3.1),[§IV](https://arxiv.org/html/2607.00477#S4.p1.1),[§IV](https://arxiv.org/html/2607.00477#S4.p5.3)\.
- \[16\]G\. Ke, Q\. Meng, T\. Finley, T\. Wang, W\. Chen, W\. Ma, Q\. Ye, and T\. Liu\(2017\)LightGBM: a highly efficient gradient boosting decision tree\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.30,pp\. 3149–3157\.Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p2.1),[§III](https://arxiv.org/html/2607.00477#S3.p1.1)\.
- \[17\]L\. Lai, Z\. Cheng, K\. Cheng, and X\. Qi\(2026\)Do transformers always win? an empirical study of semantic embeddings for short\-text e\-commerce reviews\.In2026 9th International Symposium on Big Data and Applied Statistics \(ISBDAS\),pp\. 525–529\.External Links:[Document](https://dx.doi.org/10.1109/ISBDAS69350.2026.11484350)Cited by:[§VI](https://arxiv.org/html/2607.00477#S6.p2.3)\.
- \[18\]D\. Micci\-Barreca\(2001\)A preprocessing scheme for high\-cardinality categorical attributes in classification and prediction problems\.ACM SIGKDD Explorations Newsletter3\(1\),pp\. 27–32\.External Links:[Document](https://dx.doi.org/10.1145/507533.507538)Cited by:[item 2](https://arxiv.org/html/2607.00477#S1.I1.i2.p1.1),[§I](https://arxiv.org/html/2607.00477#S1.p1.1),[§II](https://arxiv.org/html/2607.00477#S2.p1.1)\.
- \[19\]C\. Nadeau and Y\. Bengio\(2003\)Inference for the generalization error\.Machine Learning52\(3\),pp\. 239–281\.External Links:[Document](https://dx.doi.org/10.1023/A%3A1024068626366)Cited by:[§IV](https://arxiv.org/html/2607.00477#S4.p4.7)\.
- \[20\]T\. T\. Nguyen, H\. Tahir, M\. Abdelrazek, and A\. Babar\(2020\)Deep learning methods for credit card fraud detection\.arXiv preprint arXiv:2012\.03754\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2012.03754)Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p3.1)\.
- \[21\]F\. Pargent, F\. Pfisterer, J\. Thomas, and B\. Bischl\(2022\)Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features\.Computational Statistics37\(5\),pp\. 2671–2692\.External Links:[Document](https://dx.doi.org/10.1007/s00180-022-01207-6)Cited by:[§I](https://arxiv.org/html/2607.00477#S1.p1.1),[§II](https://arxiv.org/html/2607.00477#S2.p1.1),[§II](https://arxiv.org/html/2607.00477#S2.p3.1)\.
- \[22\]S\. Popov, S\. Morozov, and A\. Babenko\(2020\)Neural oblivious decision ensembles for deep learning on tabular data\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p2.1),[§VII](https://arxiv.org/html/2607.00477#S7.p3.1)\.
- \[23\]L\. Prokhorenkova, G\. Gusev, A\. Vorobev, A\. V\. Dorogush, and A\. Gulin\(2018\)CatBoost: unbiased boosting with categorical features\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.31,pp\. 6639–6649\.Cited by:[§I](https://arxiv.org/html/2607.00477#S1.p1.1),[§II](https://arxiv.org/html/2607.00477#S2.p1.1),[§III](https://arxiv.org/html/2607.00477#S3.p1.1),[§III](https://arxiv.org/html/2607.00477#S3.p7.1)\.
- \[24\]K\. Randhawa, C\. K\. Loo, M\. Seera, C\. P\. Lim, and A\. K\. Nandi\(2018\)Credit card fraud detection using AdaBoost and majority voting\.IEEE Access6,pp\. 14277–14284\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2018.2806420)Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p3.1)\.
- \[25\]R\. Shwartz\-Ziv and A\. Armon\(2022\)Tabular data: deep learning is not all you need\.Information Fusion81,pp\. 84–90\.External Links:[Document](https://dx.doi.org/10.1016/j.inffus.2021.11.011)Cited by:[item 1](https://arxiv.org/html/2607.00477#S1.I1.i1.p1.1),[§I](https://arxiv.org/html/2607.00477#S1.p1.1),[§II](https://arxiv.org/html/2607.00477#S2.p2.1),[§II](https://arxiv.org/html/2607.00477#S2.p3.1),[§VI](https://arxiv.org/html/2607.00477#S6.p1.1)\.
- \[26\]A\. Slakey, D\. Salas, and Y\. Schamroth\(2019\)Encoding categorical variables with conjugate Bayesian models for WeWork lead scoring engine\.arXiv preprint arXiv:1904\.13001\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1904.13001)Cited by:[item 2](https://arxiv.org/html/2607.00477#S1.I1.i2.p1.1),[§II](https://arxiv.org/html/2607.00477#S2.p1.1)\.
- \[27\]W\. Sun, Z\. Qi, and Q\. Shen\(2025\)High\-recall deep learning: a gated recurrent unit approach to bank account fraud detection on imbalanced data\.In2025 5th International Conference on Digital Society and Intelligent Systems \(DSInS\),pp\. 207–212\.External Links:[Document](https://dx.doi.org/10.1109/DSInS68311.2025.11329905)Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p3.1)\.
- \[28\]W\. Sun, Q\. Shen, Y\. Gao, Q\. Mao, T\. Qi, and S\. Xu\(2025\)Objective over architecture: fraud detection under extreme imbalance in bank account opening\.Computation13\(12\),pp\. 290\.External Links:[Document](https://dx.doi.org/10.3390/computation13120290)Cited by:[§VI](https://arxiv.org/html/2607.00477#S6.p4.1)\.
- \[29\]K\. Weinberger, A\. Dasgupta, J\. Langford, A\. Smola, and J\. Attenberg\(2009\)Feature hashing for large scale multitask learning\.InProceedings of the 26th International Conference on Machine Learning \(ICML\),pp\. 1113–1120\.External Links:[Document](https://dx.doi.org/10.1145/1553374.1553516)Cited by:[§II](https://arxiv.org/html/2607.00477#S2.p1.1)\.
- \[30\]J\. Yao and Z\. Zheng\(2026\)Beyond agent architecture: execution assumptions and reproducibility in LLM\-based trading systems\.External Links:2606\.08285,[Link](https://arxiv.org/abs/2606.08285)Cited by:[§IV](https://arxiv.org/html/2607.00477#S4.p5.3)\.