Modeling Decisions in Blockchain Analytics: A Leakage-Aware Evaluation of Tree-Based vs. Sequential Models

# Modeling Decisions in Blockchain Analytics: A Leakage-Aware Evaluation of Tree-Based vs. Sequential Models
Source: [https://arxiv.org/html/2607.27350](https://arxiv.org/html/2607.27350)
11institutetext:Faculty of Electronics and Information Technology
Warsaw University of Technology, Warsaw, Poland
11email:\{michal\.bartnicki\.stud, jaroslaw\.chudziak\}@pw\.edu\.pl###### Abstract

Sybil bots are Ethereum actors that imitate legitimate users to extract airdrop rewards or influence governance\. Recent Sybil detection methods increasingly use deep learning and treat blockchain activity as a quasi\-linguistic sequence\. However, complex sequence models are computationally expensive for real\-time monitoring, and their reported performance may be inflated by label leakage from high\-signal smart contracts\. We ask whether and how organic users, Sybil bots, and MEV bots differ in the structural complexity of their transaction histories; whether sequential models outperform tree\-based tabular models once leakage is reduced; whether transaction order or timing provides the stronger behavioral signal; and whether the resulting models are practical for low\-latency deployment\. Our approach to leakage\-aware Sybil bot detection consists of a Blind\-Spot protocol and a Transaction Grammar representation of wallet behavior\. The former eliminates shortcuts associated with high\-signal contracts, whereas the latter models wallets using rhythm, EVM execution structure, and intent\. We evaluate this approach on Ethereum actor classification by comparing Transformer and BiLSTM sequence models against XGBoost and SVM baselines\. We contribute a framework for leakage\-aware Ethereum actor classification and a Transaction Grammar representation of wallet behavior\. Our results demonstrate that, under leakage\-aware evaluation, XGBoost outperforms Transformer\-based sequence models while providing lower latency and estimated energy use\.

## 1Introduction

Blockchain systems increasingly rely on automated actors to provide liquidity and arbitrage\. While Maximal Extractable Value \(MEV\) bots, known as arbitrage bots, are generally an expected component of these ecosystems\[[14](https://arxiv.org/html/2607.27350#bib.bib1)\], an emerging concern is the growing number of automated actors masquerading as legitimate users – Sybil bots\. This challenges fairness and incentive design, as Sybil farmers undermine airdrops and governance mechanisms intended for legitimate participants\[[15](https://arxiv.org/html/2607.27350#bib.bib3)\]\. With the saturation of legitimate MEV activity, distinguishing between Sybil behavior and essential MEV infrastructure has become critical\.

Recent trends in blockchain behavioral modeling emphasize complex deep learning, particularly Natural Language Processing \(NLP\)\-inspired sequence models\[[22](https://arxiv.org/html/2607.27350#bib.bib5)\]\. These approaches operate on the assumption that transaction history resembles a language exhibiting meaningful long\-range dependencies\[[13](https://arxiv.org/html/2607.27350#bib.bib6)\]\. However, this assumption lacks rigorous evaluation\. Unlike natural language, transaction history is sparse, highly structured, and often dominated by aggregate statistics rather than compositional semantics\. This aligns with recent findings suggesting that in highly volatile financial domains, a strategic subset of features, or a partial\-multivariate approach, often offers superior predictive power over noise\-prone full\-multivariate models\[[23](https://arxiv.org/html/2607.27350#bib.bib27)\]\. Notably, many state\-of\-the\-art results rely on superficial interactions with specific contracts, introducing label leakage that inflates reported performance\.

In this work, we revisit blockchain actor classification under a leakage\-aware setting\. To this end, we introduce the*Blind\-Spot*protocol, which mitigates the influence of high\-signal smart contracts\. To better capture intrinsic patterns, we propose a hierarchical*transaction grammar*that parses transaction sequences using the internal*Ethereum Virtual Machine \(EVM\) execution trace*of every call\. We leverage this framework to compare sequential and tabular inductive biases under identical information conditions\. We investigate whether organic and automated actors diverge grammatically and whether sequential models retain their advantage once shortcut features are removed\.

We find that while organic users exhibit higher structural entropy and complexity than automated actors, attention\-based models exploit this structure only weakly\. In contrast, tree\-based models outperform deep learning architectures at a fraction of the computational cost, highlighting a critical efficiency\-performance trade\-off for sustainable, real\-time blockchain analytics\. This finding aligns with broader literature demonstrating that tree ensembles often retain state\-of\-the\-art performance on tabular domains where deep learning inductive biases are ill\-suited\[[2](https://arxiv.org/html/2607.27350#bib.bib8)\]\. We illustrate our end\-to\-end workflow in Fig\.[1](https://arxiv.org/html/2607.27350#S1.F1)\.

Labeled WalletsInternal EVM TracesBlind\-Spot Protocol \(Leakage Removal\)Transaction Grammar ExtractionClassifier Benchmark \(Sequential vs\. Tabular\)Figure 1:End\-to\-end workflow for leakage\-aware Ethereum actor classification\.
## 2Related Work

#### Blockchain actor and security classification

Researchers have explored neural networks for blockchain analytics, including sequence encoder models for transaction histories\[[12](https://arxiv.org/html/2607.27350#bib.bib10)\]and graph neural networks for interaction graph structures\[[16](https://arxiv.org/html/2607.27350#bib.bib9)\], while traditional gradient boosting decision trees remain effective baselines due to their strong performance on structured features and moderate\-sized datasets\[[17](https://arxiv.org/html/2607.27350#bib.bib12)\]\. However, reported performance is often inflated because models exploit class\-defining proxies rather than generalizable patterns\[[7](https://arxiv.org/html/2607.27350#bib.bib7)\]\.

#### Deep learning vs\. trees on tabular data

Apart from blockchain, existing research confirms the lack of dominance of deep learning approaches over tree\-based models in tabular data domains, where gradient boosting decision trees have been found to obtain higher accuracy with less tuning complexity in large\-scale benchmarks\[[19](https://arxiv.org/html/2607.27350#bib.bib15)\]\. To address these differences, several transformer\-based models have been proposed for tabular learning, such as FT\-Transformer\[[6](https://arxiv.org/html/2607.27350#bib.bib17)\]and SAINT\[[20](https://arxiv.org/html/2607.27350#bib.bib16)\], where self\-attention is utilized for tabular data learning\. While these models have been found to be effective, research confirms limited advantages for medium\-scale data sets, where the presence of uninformative features is critical for model performance\. While pretraining\-based models like TabPFN\[[9](https://arxiv.org/html/2607.27350#bib.bib18)\]excel on small datasets, their complexity and scalability issues make comparing sequential and tabular inductive biases critical for Ethereum modeling\. Furthermore, while Transformer models show promise for financial time series, their performance is heavily contingent on the choice of training objectives and loss functions, as demonstrated in stock ranking tasks\[[11](https://arxiv.org/html/2607.27350#bib.bib28)\]\. Efficient long\-sequence variants such as Informer address the quadratic attention bottleneck through sparse attention, and have also been applied in financial option pricing, which further motivates verifying whether long\-range sequence modeling is actually beneficial in blockchain analytics\[[25](https://arxiv.org/html/2607.27350#bib.bib29),[1](https://arxiv.org/html/2607.27350#bib.bib30)\]\.

#### Leakage\-aware evaluation

Literature emphasizes the need for a robust evaluation and leakage mitigation process, where even benign feature selection can reveal target labels\[[5](https://arxiv.org/html/2607.27350#bib.bib20)\]\. We define label leakage as the presence of domain\-specific ’shortcuts’ \(e\.g\., protocol and counterparty addresses\) that trivially expose an actor’s class\. Consequently, explicit controls are necessary to accurately measure generalization behaviors\[[4](https://arxiv.org/html/2607.27350#bib.bib21)\]\. As a result of these considerations, we propose a leakage\-oriented benchmark for Ethereum actor classification that directly compares sequential and tabular inductive bias under a common evaluation methodology, as well as providing efficiency metrics of interest for real\-time deployment\.

## 3Proposed Approach

The central methodological question is how the same leakage\-aware transaction history should be consumed by competing inductive biases\. Accordingly, this section first defines a common Transaction Grammar representation, derived from EVM traces and expressed through rhythm, structure, and intent tokens\. Sequential architectures consume the grammar as an ordered transaction sequence, whereas tree\-based and linear baselines consume aggregate statistics derived from the same grammar\. This makes the title comparison explicit: we test whether preserving order adds predictive value once obvious leakage is removed\.

### 3\.1Transaction Grammar as a Sequence Representation

We represent each wallet as a sequence of transactionsSW=\{x1,…,xL\}S\_\{W\}=\\\{x\_\{1\},\\dots,x\_\{L\}\\\}\. Each transactionxtx\_\{t\}is encoded as a tripletxt=\(τt,γt,νt\)x\_\{t\}=\(\\tau\_\{t\},\\gamma\_\{t\},\\nu\_\{t\}\)capturing*Rhythm*,*Structure*, and*Intent*\.

Rhythm \(τt\\tau\_\{t\}\):We compute the inter\-arrival timeΔ​t\\Delta tsincext−1x\_\{t\-1\}and discretize it into 16 bins, ranging from same\-block activity \(Δ​t=0\\Delta t=0\) to long\-term dormancy \(Δ​t\>30\\Delta t\>30days\), with a specialSTARTtoken for sequence start\.

Structure \(γt\\gamma\_\{t\}\):We describe each transaction using nine EVM trace properties \(e\.g\., depth and error indicators\)\. Each property is discretized and concatenated into a single composite token \(e\.g\.,Cnt:2\_Dp:4\_Err:0\.\.\.\), representing the execution\-trace shape\.

Intent \(νt\\nu\_\{t\}\):We encode semantic intent using the 4\-byte function selector\. To control vocabulary size and reduce class imbalance, we merge the top\-kkselectors from each class into a global vocabulary of size\|V\|=100\|V\|=100, mapping rare selectors to\[UNK\]\.

### 3\.2Sequential vs\. Tree\-Based Model Families

The Transaction Grammar is shared across both model families\. For sequential models, the grammar preserves transaction order and allows attention or recurrence to exploit temporal dependencies\. Tree\-based and linear baselines aggregate the same grammar into distributional features\. We also compare token\-level and transaction\-level sequence architectures\.

Flattened Transformer \(Token\-Level\)\.We utilize a standard Transformer\[[24](https://arxiv.org/html/2607.27350#bib.bib22)\]encoder on the flattened list of all tokens\. The wallet transaction history is transformed from a tensor withLLtransactions andKKtokens in each transaction into a single list withN=L×KN=L\\times Ktokens\. Each token is encoded and added with learnable positional embeddings\. We use 4 layers of the Transformer encoder with dimensionsdm​o​d​e​l=256d\_\{model\}=256andnh​e​a​d=4n\_\{head\}=4with causal masking disabled\. We also add a\[CLS\]token to aggregate global context for the final linear classifier\. This approach enables the attention mechanism to attend to any two tokens, regardless of the transaction they belong to\. We also evaluate a Hierarchical Transformer, which aggregates tokens into transaction vectors before sequence encoding\.

Hierarchical BiLSTM \(Transaction\-Level\)\.For the Recurrent Baseline, a hierarchical approach is adopted to overcome the vanishing gradient problem associated with the processing of long token streams\. First, the embeddings are summed within each transaction to form a dense feature vectorhth\_\{t\}\. The set of transaction vectors is then input into a Bidirectional Long Short\-Term Memory \(BiLSTM\)\[[8](https://arxiv.org/html/2607.27350#bib.bib23)\]architecture \(2\-layers, 256 hidden units\)\. The hidden states across all time steps are aggregated via max\-pooling and projected to class probability outputs\. Unlike the Transformer architecture, sequential dependencies are explicitly modeled at the transactional cadence\.

Tabular Baselines \(Feature Engineering\)\.To validate the necessity of sequential dependencies in the model, a Gradient Boosted Decision Tree \(XGBoost\)\[[3](https://arxiv.org/html/2607.27350#bib.bib24)\]is trained on global aggregate statistics\. The feature set is a “Bag\-of\-Grammar“ consisting of:

- •Token Counts: Counting the frequency of all unique Rhythm, Structure, and Intent tokens\. This approximates the distribution of time and call types\.
- •Complexity Metrics: Shannon Entropy and Lempel\-Ziv \(LZ\) compression ratios are computed on the token streams to quantify the diversity in behavior\.

A Linear Support Vector Machine \(SVM\) is also used as a feature engineering baseline on Term Frequency\-Inverse Document Frequency \(TF\-IDF\) vector representations and Bigram counts to evaluate the importance of local transition statistics\.

### 3\.3Common Training and Evaluation Protocol

To ensure a rigorous comparison between inductive biases, we enforced a common evaluation protocol across all models\. Every experiment utilized the exact same dataset splits \(Nt​r​a​i​n=12,650N\_\{train\}=12,650,Nv​a​l=3,163N\_\{val\}=3,163\) generated with a fixed seed\. We report results averaged over five distinct random seeds to rule out initialization luck\.

Neural models were trained on a single NVIDIA A100 GPU, while XGBoost and SVM baselines were trained on CPU under identical data and split conditions\. To handle class imbalance \(Organic≈4×\\approx 4\\timesSybil\), we applied inverse\-frequency weighting across all architectures: the Cross\-Entropy loss was weighted for neural networks, and equivalent sample weights were computed for the XGBoost and SVM objectives\.

Table[1](https://arxiv.org/html/2607.27350#S3.T1)details the final hyperparameter configurations for the top\-performing variants selected based on validation Matthews Correlation Coefficient \(MCC\)\.

Table 1:Hyperparameter Configuration

## 4Experiments

The experiments evaluate modeling decisions under leakage\-aware Ethereum actor classification\. We test whether sequential architectures exploit Transaction Grammar better than tree\-based models once high\-signal shortcuts are removed\. We structure the evaluation around four questions: how the dataset is constructed after leakage mitigation, whether actor classes differ in grammatical complexity, how sequential and tabular models compare in classification performance, and whether the resulting models are viable for low\-latency deployment\.

### 4\.1Experimental Setup and Baselines

We benchmarked five architectures on the same leakage\-aware80/2080/20stratified split\. The comparison includes:

- •Sequential Models:aFlattened Transformer, aHierarchical Transformer, and aHierarchical BiLSTMto capture temporal dependencies at different input resolutions\.
- •Tabular Baseline:XGBoost, using aggregated feature engineering \(entropy, counts, timing statistics\) to test whether global statistics suffice\.
- •Feature\-Based Baseline:Linear SVMon TF\-IDF and bigram representations to evaluate the importance of local transition statistics\.

We use MCC as the primary metric, alongside macro\-F1 and accuracy, averaged over five runs\.

### 4\.2Dataset and Preprocessing

We constructed the dataset by collecting labeled addresses from two sources\. Labels for Organic \(verified legitimate human users\) and Sybil \(coordinated bot rings eliminated for reward farming\) were sourced from the Hop Protocol’s official investigation list\[[10](https://arxiv.org/html/2607.27350#bib.bib26)\], while MEV Arbitrage Bots were identified via Dune Analytics\. We obtained addresses marked as Arbitrage Bots\. For transaction histories that include internal execution traces, we utilized Google BigQuery with a hard cutoff on May 13th, 2022, when Hop Protocol published their snapshot\.

We filtered out wallets with fewer thanLmin=10L\_\{\\min\}=10transactions to exclude them from the dataset\. To mitigate label leakage—defined here as spurious correlations where high\-signal proxy contracts trivially reveal the target class rather than underlying behavior—our Blind\-Spot protocol removes interactions with entities like OpenSea or Uniswap Router, reducing total volume by 21\.8% \(Table[2](https://arxiv.org/html/2607.27350#S4.T2)\)\. Our dataset now comprises 15,813 sequences \(Table[3](https://arxiv.org/html/2607.27350#S4.T3)\), with 80/20 splits for training and validation sets respectively\.

Table 2:Examples of Top Leakage Sources Removed by the “Blind\-Spot” ProtocolAfter removing high\-signal contracts, the remaining dataset preserves all three actor categories while reducing shortcut\-based separability\. Table[3](https://arxiv.org/html/2607.27350#S4.T3)summarizes the resulting class distribution used for the common train/validation split\.

Table 3:Dataset Statistics Post\-Filtering
### 4\.3Behavior Grammar Analysis \(Q1\)

We investigated whether organic users exhibit higher structural entropy \(HGH\_\{G\}\), complexity \(CL​ZGC\_\{LZ\}^\{G\}\), and timing entropy \(HτH\_\{\\tau\}\) than automated bots\.

Results show that organic users possess significantly higherHGH\_\{G\}\(3\.50\) than MEV \(3\.12\) and Sybil \(2\.66\) actors, with a statistically significant difference \(Kruskal\-Wallis testp<0\.01p<0\.01and Cliff’sδ\>0\.3\\delta\>0\.3\)\. Hence, organic users exhibit greater structural complexity \(Table[4](https://arxiv.org/html/2607.27350#S4.T4)\)\.

Table 4:Median Entropy and Normalized LZ Complexity by ClassOrganic users also exhibited the highest timing entropy\. However, the difference between MEV and Sybil was negligible \(Cliff’sδ≈−0\.02\\delta\\approx\-0\.02\), suggesting that bot timing is shaped by factors beyond actor\-specific behavior\.

### 4\.4Classification Performance \(Q2\)

We addressed the following question: do we really need sophisticated sequence modeling to achieve strong performance, or is a tabular inductive bias sufficient? We tested the hypothesis that XGBoost would outperform sequence models under leakage\-aware evaluation, because transaction history is more structured than text\-like\.

Empirical results strongly support this hypothesis\. As shown in Table[5](https://arxiv.org/html/2607.27350#S4.T5), XGBoost outperforms all other models across metrics \(MCC:0\.75350\.7535, Macro\-F1:0\.81410\.8141\), surpassing the Flattened Transformer \(MCC:0\.66020\.6602\) and the Hierarchical BiLSTM \(MCC:0\.61870\.6187\) by a substantial margin\. Linear SVM also outperforms the BiLSTM\. This suggests that local recurrence provides minimal predictive benefit\.

The Flattened Transformer achieved the best recall on bot types MEV and Sybil \(MEV0\.900\.90, Sybil0\.590\.59\), outperforming XGBoost on those two types\. But this came at a heavy cost on the ability to generalize well on Organic users:0\.820\.82recall compared to XGBoost’s0\.930\.93\.

Table 5:Classification Performance \(Blind\-Spot Dataset\)
### 4\.5Order vs\. Timing Ablation \(Q3\)

To investigate the performance gap between sequence and tabular models, we performed an ablation study by masking specific information in the validation set\. We applied three perturbations to the Hierarchical Transformer baseline \(M​C​C=0\.5733MCC=0\.5733\):

- •Timing Scramble \(τ\\tau\-Ablation\):Timestamp tokens are randomly permuted while maintaining the transaction body sequence\.
- •Grammar Shuffle \(γ\\gamma\-Ablation\):Transaction bodies are permuted while maintaining the original temporal rhythm\.
- •Full Shuffle:The entire sequence \(Rhythm, Structure, and Intent\) is scrambled, destroying all sequential dependencies\.

The results \(see Fig\.[2](https://arxiv.org/html/2607.27350#S4.F2)\) show that the model is surprisingly resilient to structural destruction\. The Full Shuffle \(simulating a Bag\-of\-Transactions\) resulted in an MCC of0\.54850\.5485, a drop of onlyΔ​MCC=−0\.0248\\Delta\\text\{MCC\}=\-0\.0248\. The Grammar Shuffle \(0\.55120\.5512\) had a significantly larger impact than the Timing Scramble \(0\.56570\.5657\), indicating that the model derives more signal from the ordering of execution traces than from inter\-arrival times\.

Critically, the model maintains over 95% of its predictive power even when the sequence is fully scrambled\. This confirms that the Transformer is primarily utilizing local token distributions, effectively acting as a high\-dimensional density estimator, rather than capturing long\-range temporal dependencies\. This explains why XGBoost, which explicitly optimizes for these distribution\-based “Bag\-of\-Grammar” features, achieves superior performance with significantly lower complexity\.

![Refer to caption](https://arxiv.org/html/2607.27350v1/shuffling.png)Figure 2:Impact of sequence ablations on Transformer performance \(MCC\)\.
### 4\.6Deployment Viability and Efficiency

Fig\.[3](https://arxiv.org/html/2607.27350#S4.F3)shows that the tabular approach is superior in terms of efficiency\. This evaluation follows the broader Green AI argument that model quality should be considered together with computational cost and accessibility\[[18](https://arxiv.org/html/2607.27350#bib.bib31),[21](https://arxiv.org/html/2607.27350#bib.bib32)\]\. XGBoost is100×100\\timesfaster in our experiments, running in microseconds on regular hardware\. The Transformer, on the other hand, requires specialized hardware to remain competitive\. From the perspective ofGreen AI, the complexity of the Transformer’s self\-attention atO​\(L2\)O\(L^\{2\}\)\[[24](https://arxiv.org/html/2607.27350#bib.bib22)\]implies that its energy costs are more than30×30\\timeshigher than those of the linear complexity of XGBoost\. XGBoost dominates the efficiency\-accuracy curve because it achieves better predictive performance \(Table[5](https://arxiv.org/html/2607.27350#S4.T5)\) while remaining competitive on latency and sustainability\.

![Refer to caption](https://arxiv.org/html/2607.27350v1/fig7_efficiency_accuracy_final.png)Figure 3:Efficiency vs\. Accuracy\. XGBoost \(top left\) achieves the most optimal trade\-off on the efficiency\-accuracy curve\.

## 5Discussion and Future Work

Our results show a mismatch between attention\-based inductive biases and blockchain transactions\. Because transaction histories lack the compositional semantics of natural language, global sequencing provides less signal than local frequency and execution patterns\. Thus, Ethereum behavior is closer to aBag\-of\-Transactionsthan to a formal language\.

This aligns with tabular\-learning evidence that tree\-based models often outperform deep models on structured datasets\. Structured datasets favor axis\-aligned splits, robustness to uninformative features, and feature selection, all naturally handled by gradient\-boosted trees\. Our results extend this argument to leakage\-aware Web3 behavioral analytics\.

TheBlind\-Spotapproach was essential to this finding\. By removing interactions with class\-defining contracts, we effectively removed the shortcut features and forced the models to rely on the authentic behavioral signals\. With this approach, the tree\-based models were able to leverage the global grammatical statistics—entropy, frequency, and dispersion—whereas the sequence models did not capitalize on the long\-range structure\.

The results also matter for sustainable and decentralized security\. XGBoost improved performance while reducing latency and energy use\. Such efficiency enables deployment by validators, light clients, and wallet extensions\. Transformer\-based models instead favor centralized infrastructure\. In Web3, simplicity supports decentralization\.

Looking ahead, Graph Neural Networks are promising because Ethereum activity forms a heterogeneous interaction graph\. Future work should develop scalable sampling strategies that preserve low latency\. Second, Transaction Grammar relies on discrete binning\. Continuous embeddings from self\-supervised pretraining may test whether richer representations close the gap\. Finally, it is essential to be resilient to adversarial camouflage\. Clever actors will eventually find ways to adapt their strategies—for example, injecting artificial entropy or random delays—to evade detection\. It will be crucial to test models against active evasion attacks to determine whether behavioral classifiers can continue to be effective in the long term\.

## 6Conclusion

This paper evaluated Ethereum actor classification under a leakage\-aware protocol designed to remove high\-signal contract shortcuts before modeling\. Using the proposed Transaction Grammar, we showed that organic wallets exhibit higher structural entropy, intent diversity, and grammar complexity than automated actors\. However, the classification experiments demonstrated that this behavioral structure is better exploited by aggregate tabular features than by sequence models: XGBoost achieved the strongest MCC, macro\-F1, and accuracy, while the ablation study showed that Transformers lose only limited predictive power when transaction order is disrupted\. These results support the empirical conclusion that, in this setting, blockchain activity behaves more like a Bag\-of\-Grammar than a natural\-language\-like sequence\.

The main contribution of the work is therefore a leakage\-aware comparison of tree\-based and sequential inductive biases for blockchain analytics\. The Blind\-Spot protocol, Transaction Grammar, order\-versus\-timing ablation, and efficiency analysis together show that simpler tabular models can be both more accurate and more deployable than heavier neural alternatives, with substantially lower latency and estimated energy use\. Future work should extend the same evaluation logic to graph\-based representations, self\-supervised transaction embeddings, and adversarial camouflage scenarios, while preserving the emphasis on leakage control, computational efficiency, and real\-time deployment suitability established in this study\.

## References

- \[1\]F\. Bańka and J\. Chudziak\(2025\)Applying informer for option pricing: a transformer\-based approach\.InProc\. ICAART,pp\. 1270–1277\.External Links:[Document](https://dx.doi.org/10.5220/0013320900003890)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px2.p1.1)\.
- \[2\]V\. Borisovet al\.\(2024\)Deep neural networks and tabular data: a survey\.IEEE Trans\. Neural Netw\. Learn\. Syst\.35,pp\. 7499–7519\.External Links:[Document](https://dx.doi.org/10.1109/tnnls.2022.3229161)Cited by:[§1](https://arxiv.org/html/2607.27350#S1.p4.1)\.
- \[3\]T\. Chen and C\. Guestrin\(2016\)XGBoost: a scalable tree boosting system\.InProc\. 22nd KDD,pp\. 785–794\.External Links:[Document](https://dx.doi.org/10.1145/2939672.2939785)Cited by:[§3\.2](https://arxiv.org/html/2607.27350#S3.SS2.p4.1)\.
- \[4\]S\. Fanet al\.\(2021\)Al\-spsd: anti\-leakage smart ponzi schemes detection in blockchain\.Inf\. Process\. Manage\.58,pp\. 102587\.External Links:[Document](https://dx.doi.org/10.1016/j.ipm.2021.102587)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px3.p1.1)\.
- \[5\]R\. Geirhoset al\.\(2020\)Shortcut learning in deep neural networks\.Nat\. Mach\. Intell\.2,pp\. 665–673\.External Links:[Document](https://dx.doi.org/10.1038/s42256-020-00257-z)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px3.p1.1)\.
- \[6\]Y\. Gorishniy, I\. Rubachev, V\. Khrulkov, and A\. Babenko\(2021\)Revisiting deep learning models for tabular data\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2106.11959)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px2.p1.1)\.
- \[7\]L\. Grinsztajn, E\. Oyallon, and G\. Varoquaux\(2022\)Why do tree\-based models still outperform deep learning on typical tabular data?\.InProc\. 36th NeurIPS,External Links:[Document](https://dx.doi.org/10.48550/arXiv.2207.08815)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px1.p1.1)\.
- \[8\]S\. Hochreiter and J\. Schmidhuber\(1997\)Long short\-term memory\.Neural Comput\.9,pp\. 1735–1780\.External Links:[Document](https://dx.doi.org/10.1162/neco.1997.9.8.1735)Cited by:[§3\.2](https://arxiv.org/html/2607.27350#S3.SS2.p3.1)\.
- \[9\]N\. Hollmannet al\.\(2022\)TabPFN: a transformer that solves small tabular classification problems in a second\.InNeurIPS 2022 1st Table Rep\. Workshop,External Links:[Link](https://openreview.net/forum?id=eu9fVjVasr4)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px2.p1.1)\.
- \[10\]Hop Protocol\(2022\)Hop Protocol: Hop Airdrop distribution\.Note:[https://github\.com/hop\-protocol/hop\-airdrop](https://github.com/hop-protocol/hop-airdrop)Cited by:[§4\.2](https://arxiv.org/html/2607.27350#S4.SS2.p1.1)\.
- \[11\]J\. Kwiatkowski and J\. A\. Chudziak\(2025\)On evaluating loss functions for stock ranking: an empirical analysis with transformer model\.InProc\. 34th ACM CIKM,pp\. 4910–4914\.External Links:[Document](https://dx.doi.org/10.1145/3746252.3760812)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px2.p1.1)\.
- \[12\]Q\. Liuet al\.\(2025\)Detecting sybil addresses in blockchain airdrops\.In2025 IEEE ICBC,pp\. 1–7\.External Links:[Document](https://dx.doi.org/10.1109/ICBC64466.2025.11185061)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px1.p1.1)\.
- \[13\]T\. Liuet al\.\(2026\)The role of transformer models in advancing blockchain technology: a systematic survey\.Eng\. Appl\. Artif\. Intell\.163,pp\. 112968\.External Links:[Document](https://dx.doi.org/10.1016/j.engappai.2025.112968)Cited by:[§1](https://arxiv.org/html/2607.27350#S1.p2.1)\.
- \[14\]H\. Materwalaet al\.\(2025\)Maximal extractable value in decentralized finance: taxonomy, detection, and mitigation\.IEEE Trans\. Serv\. Comput\.18,pp\. 4386–4407\.External Links:[Document](https://dx.doi.org/10.1109/tsc.2025.3620604)Cited by:[§1](https://arxiv.org/html/2607.27350#S1.p1.1)\.
- \[15\]J\. Messias, A\. Yaish, and B\. Livshits\(2023\)Airdrops: giving money away is harder than it seems\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2312.02752)Cited by:[§1](https://arxiv.org/html/2607.27350#S1.p1.1)\.
- \[16\]T\. Niedermayer, P\. Saggese, and B\. Haslhofer\(2024\)Detecting financial bots on the ethereum blockchain\.InCompanion Proc\. Web Conf\. 2024,pp\. 1742–1751\.External Links:[Document](https://dx.doi.org/10.1145/3589335.3651959)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px1.p1.1)\.
- \[17\]G\. Palaiokrassas, S\. Bouraga, and L\. Tassiulas\(2024\)Machine learning on blockchain data: a systematic mapping study\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2403.17081)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px1.p1.1)\.
- \[18\]R\. Schwartz, J\. Dodge, N\. A\. Smith, and O\. Etzioni\(2020\-11\)Green ai\.Commun\. ACM63\(12\),pp\. 54–63\.External Links:ISSN 0001\-0782,[Document](https://dx.doi.org/10.1145/3381831)Cited by:[§4\.6](https://arxiv.org/html/2607.27350#S4.SS6.p1.3)\.
- \[19\]R\. Shwartz\-Ziv and A\. Armon\(2022\)Tabular data: deep learning is not all you need\.Inf\. Fusion81,pp\. 84–90\.External Links:[Document](https://dx.doi.org/10.1016/j.inffus.2021.11.011)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px2.p1.1)\.
- \[20\]G\. Somepalliet al\.\(2021\)SAINT: improved neural networks for tabular data via row attention and contrastive pre\-training\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2106.01342)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px2.p1.1)\.
- \[21\]E\. Strubell, A\. Ganesh, and A\. Mccallum\(2019\-01\)Energy and policy considerations for deep learning in nlp\.pp\. 3645–3650\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1355)Cited by:[§4\.6](https://arxiv.org/html/2607.27350#S4.SS6.p1.3)\.
- \[22\]J\. Sunet al\.\(2025\)Ethereum fraud detection via joint transaction language model and graph representation learning\.Inf\. Fusion120,pp\. 103074\.External Links:[Document](https://dx.doi.org/10.1016/j.inffus.2025.103074)Cited by:[§1](https://arxiv.org/html/2607.27350#S1.p2.1)\.
- \[23\]A\. Tokajuk and J\. Chudziak\(2025\)Partial multivariate transformer as a tool for cryptocurrencies time series prediction\.External Links:[Document](https://dx.doi.org/10.1109/ICTAI66417.2025.00147)Cited by:[§1](https://arxiv.org/html/2607.27350#S1.p2.1)\.
- \[24\]A\. Vaswaniet al\.\(2017\)Attention is all you need\.InAdv\. Neural Inf\. Process\. Syst\.,Vol\.30\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1706.03762)Cited by:[§3\.2](https://arxiv.org/html/2607.27350#S3.SS2.p2.5),[§4\.6](https://arxiv.org/html/2607.27350#S4.SS6.p1.3)\.
- \[25\]H\. Zhou, S\. Zhang, J\. Peng, S\. Zhang, J\. Li, H\. Xiong, and W\. Zhang\(2021\)Informer: beyond efficient transformer for long sequence time\-series forecasting\.InProc\. AAAI,Vol\.35,pp\. 11106–11115\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v35i12.17325)Cited by:[§2](https://arxiv.org/html/2607.27350#S2.SS0.SSS0.Px2.p1.1)\.