ChurnNet: A Optimized Modern AI for Churn Prediction

# ChurnNet: A Optimized Modern AI for Churn Prediction
Source: [https://arxiv.org/html/2606.00169](https://arxiv.org/html/2606.00169)
11institutetext:Department of Computer, Control, and Management Engineering, Sapienza University of Rome, Rome, Italy22institutetext:Department of Law and Economics, UnitelmaSapienza University of Rome, Rome, Italy33institutetext:R&D Center, Token Financial Technologies, Istanbul, Turkey44institutetext:Department of Civil, Computer Science and Aeronautical Technologies Engineering,
Roma Tre University, Rome, Italy###### Abstract

Increased competition and the growing similarity of products and services offered by retailers have lowered the barriers for customers to switch to competitors\. Accurate churn prediction can be a valuable tool for driving effective personalized marketing campaigns and helping to reduce customer attrition\. This study evaluates the performance of traditional machine learning techniques, namely, Random Forests, XGBoost, and Support Vector Machines, and compares them with the Unified Multi\-Task Time Series Model for churn prediction, a binary time\-series classification task\. Despite the strong capacity of the latter to model complex temporal dynamics and inter\-variable relationships, our results indicate that for churn prediction, conventional methods can still outperform it in terms of predictive performance, data efficiency, and computational resource requirements for training and deployment\. These findings are consistent across multiple datasets and various churn labeling techniques\.

## 1Introduction

In recent years, the retail industry has undergone a profound transformation\. The adoption of online as well as omni\-channel sales has allowed customers to have convenient access to multiple retailers through digital platforms, unlike traditional retail, which relied primarily on physical touch\-points and local customer bases\. This evolution has intensified competition and dramatically lowered switching barriers\[[27](https://arxiv.org/html/2606.00169#bib.bib4),[4](https://arxiv.org/html/2606.00169#bib.bib5)\]making customer retention a strategic priority for retailers of all sizes\.

Customer churn, the process by which consumers reduce or cease their purchasing activity, has emerged as a critical challenge in this context\. In non\-contractual settings, such as e\-commerce and grocery retail, churn does not occur through formal cancellations but through behavioral changes, such as decreased purchase frequency or volume\[[8](https://arxiv.org/html/2606.00169#bib.bib8)\]\. Detecting these gradual forms of defection is particularly challenging yet crucial, as subtle declines in customer engagement often precede full attrition\[[31](https://arxiv.org/html/2606.00169#bib.bib10),[17](https://arxiv.org/html/2606.00169#bib.bib11)\]\.

At the same time, the increasing availability of customer data, particularly through loyalty programs, digital receipts, browsing histories, and CRM systems has enabled more sophisticated approaches to customer analytics\[[35](https://arxiv.org/html/2606.00169#bib.bib9),[18](https://arxiv.org/html/2606.00169#bib.bib6)\]\. This abundance of data has catalyzed the application of machine learning \(ML\) and artificial intelligence \(AI\) methods for churn prediction, which in turn can help businesses to identify early warning signals and implement targeted retention strategies\[[17](https://arxiv.org/html/2606.00169#bib.bib11),[20](https://arxiv.org/html/2606.00169#bib.bib12)\]\.

AI‑based churn prediction has been extensively explored incontractual industries, such as telecommunications, banking, and insurance, with several recent surveys reviewing both traditional ML and deep learning approaches \(e\.g\.,\[[16](https://arxiv.org/html/2606.00169#bib.bib16),[22](https://arxiv.org/html/2606.00169#bib.bib17),[3](https://arxiv.org/html/2606.00169#bib.bib18)\]\)\. However, thenon‑contractual retail sectorremains relatively under investigated\. Differently from contractual contexts where churn is defined by discrete events such as service cancellations, in non\-contractual retail the absence of explicit termination complicates both model design and the operationalization of churn labels\[[9](https://arxiv.org/html/2606.00169#bib.bib13),Rodríguez2023\]\.

In order to develop a churn prediction system, the question regarding the most suitable machine learning approach naturally arises\. On one hand, recent techniques, such as UniTS\[[15](https://arxiv.org/html/2606.00169#bib.bib19)\], TimesNet\[[33](https://arxiv.org/html/2606.00169#bib.bib1)\], and FlowFormer\[[34](https://arxiv.org/html/2606.00169#bib.bib2)\], show impressive performance for binary time\-series classification tasks on datasets of varying domains and levels of difficulty\. To achieve this performance, these models adopt a deep and complex architecture and, therefore, are computationally expensive and prone to overfitting\[[5](https://arxiv.org/html/2606.00169#bib.bib38)\]\. To mitigate this issue, they are required to be trained on large datasets\. In contrast, conventional techniques have also been successfully applied to achieve good performance in churn prediction\. These models, thanks to their relatively simple architectures, are more computationally and data\-efficient\. In this paper, we examine whether the comparatively simple nature of churn prediction as a time\-series binary classification task necessitates the use of deep, modern, but complex architectures, or if traditional, simpler models are sufficient to deliver strong predictive performance\.

For this analysis, we compared traditional machine learning models, namely, Random Forests, XGBoost and Support Vector Machines \(SVM\), with the Unified Multi\-Task Time Series Model \(UniTS\)\[[15](https://arxiv.org/html/2606.00169#bib.bib19)\], a transformer\-based model specifically designed to handle temporal data\. We chose the UniTS model to represent modern deep learning architectures, as it delivers top performance on the time\-series binary classification task\[[15](https://arxiv.org/html/2606.00169#bib.bib19)\]\. The study spans multiple real\-world retail datasets and employs a family of churn labeling definitions based on relative drops in average weekly spending over time windows\. Our goal is to assess not only the raw predictive performance \(precision, recall, F1\-score\) but also to analyze it across datasets with varying characteristics, including size and levels of imbalance\. Such an analysis is important when considering systems intended for real\-world deployment\. In fact, the results of this study will inform the development of a commercial churn prediction system by Infocube111[https://www\.infocube\.it](https://www.infocube.it/), a digital marketing solutions provider, which supplied us with two of the datasets included in the evaluation suite of this research\.

The remainder of this paper is structured as follows\. Section[2](https://arxiv.org/html/2606.00169#S2)reviews relevant studies on AI\-based churn prediction in e\-commerce and retail, with a focus on applied models, recent innovations, and the use of advanced AI techniques\. Section[3](https://arxiv.org/html/2606.00169#S3)provides an overview of customer churn predictive models used in this study, discussing their theoretical suitability for the churn prediction task\. Section[4](https://arxiv.org/html/2606.00169#S4)details our methodology, including the preparation of the datasets and the churn labeling strategy\. Section[5](https://arxiv.org/html/2606.00169#S5)describes the implementation of the models\. Section[6](https://arxiv.org/html/2606.00169#S6)presents the empirical results, offering a discussion on the performance and interpretation of the results, as well as comments on the implications of each method in practical churn prediction scenarios\. Section[7](https://arxiv.org/html/2606.00169#S7)discusses the limitations and threats to the validity of this study\. Finally, Section[8](https://arxiv.org/html/2606.00169#S8)concludes the paper and outlines potential directions for future research\.

## 2Related Works

Although research on churn prediction has predominantly focused on contractual environments, a number of studies have also examined this task in non\-contractual settings\. These studies employ a wide range of AI techniques\. While the literature is largely centered on traditional methods, such as Decision Trees \(DT\), Random Forests \(RF\), XGBoost \(XGB\), SVM, K\-Nearest Neighbors \(KNN\), and Logistic Regression \(LR\), more recent approaches, including Artificial Neural Networks \(ANN\), Convolutional Neural Networks \(CNN\), and Long Short\-Term Memory \(LSTM\) networks, have been applied with varying degrees of success\[[30](https://arxiv.org/html/2606.00169#bib.bib39),[22](https://arxiv.org/html/2606.00169#bib.bib17),[11](https://arxiv.org/html/2606.00169#bib.bib20)\]\.

Rahib et al\.\[[26](https://arxiv.org/html/2606.00169#bib.bib30)\]applied several traditional machine learning methods such as SVM, RF and DT\. Sunarya et al\.\[[32](https://arxiv.org/html/2606.00169#bib.bib33)\]implemented LR and RF\. Agbemadon et al\.\[[1](https://arxiv.org/html/2606.00169#bib.bib35)\]applied both conventional AI techniques, like LR and XGB, as well as modern approaches such as ANN and LSTM\. Boukrouh et al\.\[[6](https://arxiv.org/html/2606.00169#bib.bib31)\]conducted a comprehensive analysis of a broad suite of AI techniques that included DT, RF, SVM, LR, NB, KNN, and ANN\. In addition to evaluating predictive performance this study also and utilized explainable artificial intelligence \(XAI\) methods such as Shapley Additive Explanations \(SHAP\) and Local Interpretable Model\-agnostic Explanations \(LIME\) to develop interpretations of these models\.

Beyond the application of standard techniques, some studies have focused on methodological innovations aimed at enhancing AI model performance\. Gan et al\.\[[14](https://arxiv.org/html/2606.00169#bib.bib34)\]integrated social behavior and influence dynamics into the XGBoost framework to capture the impact of customers on their social networks\. Pondel et al\.\[[24](https://arxiv.org/html/2606.00169#bib.bib36)\]focused on e\-commerce data, incorporating non\-time\-series departmental information exprimenting with variations of ANN and Recurrent Neural Networks \(RNN\) to identify the best\-performing option\. Miguéis et al\.\[S35\-ID783\-Miguéis\]explored the implementation of Multivariate Adaptive Regression Splines \(MARS\) and benchmarked its performance against LR\. Notably, Seema et al\.\[[29](https://arxiv.org/html/2606.00169#bib.bib32)\]introduced a novel approach involving a fading channel patch\-based heat map for training CNN, aiming to improve deep learning model effectiveness in the churn prediction context\.

While previous studies have explored a range of AI techniques for churn prediction, most focus on either traditional or complex models in isolation\. Our work stands out by comparing traditional machine learning models, such as Decision Trees, Random Forests, and XGBoost, with UniTS, a transformer\-based model specifically designed for time\-series data\. This unique comparison highlights the effectiveness of simpler models versus those tailored for temporal tasks, filling a gap in the existing literature\.

## 3Background

### 3\.1Definition of Churn

In the retail sector, customer churn refers to the phenomenon in which a previously active customer ceases or substantially reduces engagement in purchasing behavior over a defined period\. Unlike contractual settings, where churn is explicitly observed through contract terminations, churn in retail is typically inferred from behavioral inactivity, which makes its detection more challenging\.

Labeling customers as churners in retail typically involves defining a period of inactivity or significant drop in engagement, known as the churn window or churn horizon, beyond which a customer is considered lost\. Common approaches include fixed thresholds, such as no purchases within 3, 6, or 12 months\[[19](https://arxiv.org/html/2606.00169#bib.bib24),[2](https://arxiv.org/html/2606.00169#bib.bib25)\], or dynamic thresholds based on individual purchase patterns, such as multiples of the customer’s average inter\-purchase time or spend\[[25](https://arxiv.org/html/2606.00169#bib.bib26),[23](https://arxiv.org/html/2606.00169#bib.bib27),[28](https://arxiv.org/html/2606.00169#bib.bib40)\]\. Some studies also apply customer segmentation and clustering approaches, probability\-based scoring, or survival analysis to estimate churn probabilities\[[21](https://arxiv.org/html/2606.00169#bib.bib28),[12](https://arxiv.org/html/2606.00169#bib.bib29)\]\.

Accurate churn labeling is critical, as it influences both model training and the business interpretation of churn risk\. However, definitions vary widely across studies and industries\.

### 3\.2Utilized AI Techniques

As highlighted by several literature reviews\[[30](https://arxiv.org/html/2606.00169#bib.bib39),[22](https://arxiv.org/html/2606.00169#bib.bib17)\]and our previous study\[[11](https://arxiv.org/html/2606.00169#bib.bib20)\], the traditional machine learning models selected in this work are among the most widely used and well\-established AI techniques in the churn prediction literature\. In contrast, UniTS represents a modern deep learning model specifically designed for time\-series data that has been successfully applied to a variety of domains\[[15](https://arxiv.org/html/2606.00169#bib.bib19)\]\.

Below, we briefly describe the AI techniques used in this study, including both traditional machine learning methods and UniTS, highlighting their key characteristics and relevance to churn prediction\.

#### 3\.2\.1Random Forests

RF, introduced by Breiman in 2001, addresses overfitting in decision trees by aggregating predictions from multiple trees built on bootstrapped samples and random subsets of features\[[7](https://arxiv.org/html/2606.00169#bib.bib21)\]\. Each tree in the forest votes on the outcome, and the majority vote determines the final prediction\. This ensemble approach enhances stability and accuracy compared to a single tree\. Random Forests are particularly effective in handling high\-dimensional, nonlinear data and offer valuable insights into feature importance\. In churn prediction, Random Forests can capture complex patterns in customer behavior; however, it may be less effective at modeling time dependencies compared to sequential models\.

#### 3\.2\.2XGBoost

XGBoost, released by Chen in 2016, is an efficient gradient boosting framework designed to improve speed and performance through parallelization and regularization\[[10](https://arxiv.org/html/2606.00169#bib.bib23)\]\. It builds additive decision trees sequentially, where each new tree corrects the errors made by the previous ensemble using gradient information from a loss function\. XGBoost integrates techniques such as shrinkage, column subsampling, and tree pruning to prevent overfitting and enhance performance\. It achieves high accuracy on tabular data, handles missing values, and manages class imbalance effectively\. However, to extract its best performance, careful fine\-tuning is required\. In churn prediction, XGBoost has proven to be particularly effective in capturing complex patterns in customer behavior and handling imbalanced datasets\.

#### 3\.2\.3Support Vector Machines

SVM, introduced in the early 1990s, aim to identify optimal separating hyperplanes for classification tasks, using kernel functions to model nonlinear relationships\[[13](https://arxiv.org/html/2606.00169#bib.bib22)\]\. The core idea is to find the hyperplane that maximizes the margin between different classes, while support vectors \(data points closest to the boundary\) define this separation\. Kernel tricks transform data into higher\-dimensional spaces to make it linearly separable when needed\. SVMs are powerful in high\-dimensional spaces but may encounter scalability issues with large datasets and are particularly sensitive to class imbalance, a common challenge in churn prediction\.

#### 3\.2\.4UniTS

UniTS, introduced in 2024 by Gao et al\.\[[15](https://arxiv.org/html/2606.00169#bib.bib19)\], is a unified multi\-task time\-series model designed to handle forecasting, classification, anomaly detection, and imputation within a single architecture\. Inspired by foundation models in Natural Language Processing \(NLP\) and computer vision, it incorporates task\-tokenization, attention mechanisms, and dynamic linear operators to enable parameter sharing and adaptation across tasks\. UniTS processes input sequences through transformers equipped with specialized tokens that signal different tasks, allowing the same network to adapt flexibly to diverse objectives\. UniTS shows strong performance across diverse datasets but requires significant computational resources and large\-scale pre\-training\. Its capacity to capture complex temporal patterns makes it theoretically well\-suited for modeling customer behavior over time, such as in churn prediction\.

## 4Methodology

Our methodology follows a structured pipeline designed to prepare transactional data, generate churn labels, and evaluate multiple machine learning models\. An overview of the entire training and evaluation process is presented in Figure[4\.2](https://arxiv.org/html/2606.00169#S4.SS2)\.

The process begins with dataset pre\-processing and filtering, where raw transactional data is cleaned and transformed into time series for each customer\. Next, customers are labeled based on different churn definitions\. For model training, temporal windows are selected\. Various balancing and sub\-sampling strategies are subsequently applied\. This produces multiple representative training datasets, which are then used to train machine learning models and evaluate their performance\. The following subsections provide a detailed explanation each step\.

### 4\.1Dataset Pre\-processing

The goal of this step is to clean the data, remove anomalies, and establish a standardized and consistent starting point for analysis\. During pre\-processing, transactional data is aggregated by customer ID and filtered to exclude non\-positive values\. Additionally, transaction amounts are aggregated on a weekly basis to ensure temporal consistency across the dataset\. The dataset is also converted into time series data by zero\-padding the weeks where no transactions occurred for a customer\.

### 4\.2Dataset Filtering

To ensure the reliability of predictions, we focus on customers with a sufficient transactional history\. Customers with limited activity provide insufficient behavioral patterns for meaningful analysis and are typically of lower commercial interest\. Therefore, in our study, we exclude customers with fewer than 20 transactional weeks i\.e\., weeks during which the customer made at least one transaction\.

![[Uncaptioned image]](https://arxiv.org/html/2606.00169v1/x1.png)Figure 1:Data processing, training and evaluation pipeline of the proposed methodology\.Table 1:Churn Labeling Definitions\.

![Refer to caption](https://arxiv.org/html/2606.00169v1/x2.png)Figure 2:Churn labeling strategy\.

![Refer to caption](https://arxiv.org/html/2606.00169v1/x3.png)

\(a\) Churn customer\.

![Refer to caption](https://arxiv.org/html/2606.00169v1/x4.png)

\(b\) Non\-churn customer\.

Figure 3:Feature windowing strategies\.

### 4\.3Churn Labeling

As discussed in the background section[3](https://arxiv.org/html/2606.00169#S3), churn labeling in non\-contractual settings is a non\-trivial task, with various approaches available depending on the use case and data characteristics\. In this work, we adopt a dynamic churn definition, formulated in collaboration with Infocube and tailored to the needs of their retail clients\. This approach aims to provide a more accurate and individualized churn label by accounting for each customer’s historical shopping behavior\.

We define two key timeframes: anobservation windowof lengthA=40A=40weeks and a subsequentlabeling windowof lengthB=20B=20weeks\. The two windows are contiguous and are aligned with the end date of their respective dataset\. The observation window is used to calculate personalized thresholds which are then applied to the labeling window to determine each customer’s churn status\.

The labeling logic is based on two behavioral parameters, which are elaborated below:

- •Monetary value: The average weekly spending in the observation window is denoted byμA\\mu\_\{A\}, and in the labeling window byμB\\mu\_\{B\}\.
- •Frequency of activity: The average number of transactions per week in the observation window is denoted byfAf\_\{A\}, and byfBf\_\{B\}in the labeling window\.

A customer is labeled as achurner\(1\) if both of the following conditions are met:

α⋅μA\>μBandβ⋅fA\>fB\\alpha\\cdot\\mu\_\{A\}\>\\mu\_\{B\}\\quad\\text\{and\}\\quad\\beta\\cdot f\_\{A\}\>f\_\{B\}\(1\)
Otherwise, the customer is labeled as anon\-churner\(0\)\. Here,α\\alphaandβ\\betaare configurable threshold parameters, which control the sensitivity of the churn detection logic\. By evaluating changes relative to each customer’s historical behavior, this method reduces noise and provides a more robust and personalized definition of churn\.

A visual illustration of this labeling method, showing the positioning of the observation and labeling windows, is provided in Figure[4\.2](https://arxiv.org/html/2606.00169#S4.SS2)\.

We employed three different churn labeling definitions by adjusting the sensitivity parametersα\\alphaandβ\\beta, as summarized in Table[4\.2](https://arxiv.org/html/2606.00169#S4.SS2)\. These variants enable us to evaluate the robustness of our models under different churn detection thresholds\.

### 4\.4Feature Windowing

The feature window refers to the time period over which input features \(XX\) are passed to our models for training\. In our study, this window is set to a fixed length of 20 weeks\. The selection of the feature window depends on the churn label assigned to the customer:

- •For churners, the feature window ends immediately before the customer’s final period of sustained inactivity\.
- •For non\-churners, a random valid segment of 20 weeks is selected from the customer’s available transaction history\.

This strategy is motivated by the assumption that, for churners, churn\-related behavioral patterns are likely to be more pronounced in the period leading up to the churn event\. Therefore, capturing this window enables the model to learn relevant dynamics associated with customer disengagement\. In contrast, for non\-churners, there is no a priori knowledge about when, or if, such patterns might occur, so a random 20\-week segment is selected to provide a representative sample of normal activity\.

An illustrative example of the feature windowing process for both churners and non\-churners is provided in Figure[4\.2](https://arxiv.org/html/2606.00169#S4.SS2)\.

### 4\.5Balancing and Subsampling

To assess the impact of dataset size and class imbalance on model performance, we employed the following dataset balancing and sub\-sampling strategies:

- •Balanced Dataset \(1:1\): These datasets include all churners remaining after filtering\. An equal number of non\-churners are then randomly sampled to create a balanced dataset\. This setting is designed to neutralize the impact of class imbalance on model performance\. Although this reduces the total amount of data used for training, it is less of a concern for large\-scale datasets\.
- •Limited Unbalanced Dataset \(1:4\): In this setting, all filtered churners are retained, and four times as many non\-churners are randomly sampled\. This controlled imbalance allows us to evaluate model performance under moderately imbalanced conditions while still utilizing more training data\. It provides insight into how well models handle class imbalance in a realistic yet constrained scenario\.
- •Balanced Subsampled Dataset \(B\-SS\): To investigate the data efficiency of different models, especially on smaller datasets, we created downsampled versions of the larger datasets\. Forsingle\-star \(\*\)datasets, we randomly select 2,500 customers after filtering; all churners are included, and an equal number of non\-churners are sampled\. Fordouble\-star \(\*\*\)datasets, the same process is applied with 5,000 customers\. This setting allows us to isolate the impact of dataset size on performance while maintaining balance\.

Table 2:Dataset Characteristics\.NormalizedTenureRatioNumber of CustomerBefore BalancingAfter BalancingCat\.DataTotalCHN\-CHCHCHN\-CHCHN\-CHA1:1IC10\.920\.690\.940\.0812,425143,45412,42512,425IC20\.890\.710\.900\.0935,309360,58535,30935,309CJ0\.900\.750\.900\.03722,0607272LGSR0\.940\.760\.940\.011,500283,8801,5001,5001:4IC10\.920\.690\.940\.0812,425143,45412,42549,700IC20\.890\.710\.900\.0935,309360,58535,309141,236CJ0\.900\.750\.900\.03722,06072288LGSR0\.940\.760\.940\.011,500283,8801,5006,000B\-SSIC1\*0\.920\.690\.940\.0812,425143,454390390IC2\*0\.890\.710\.900\.0935,309360,585416416IC1\*\*0\.920\.690\.940\.0812,425143,454200200IC2\*\*0\.890\.710\.900\.0935,309360,585220220B1:1IC10\.920\.740\.940\.1116,808139,07116,80816,808IC20\.890\.740\.900\.1143,291352,60343,29143,291CJ0\.900\.790\.900\.061242,008124124LGSR0\.940\.780\.940\.013,338282,0423,3383,3381:4IC10\.920\.740\.940\.1116,808139,07116,80867,232IC20\.890\.740\.900\.1143,291352,60343,291173,164CJ0\.900\.790\.900\.061242,008124496LGSR0\.940\.780\.940\.013,338282,0423,33813,352B\-SSIC1\*0\.920\.740\.940\.1116,808139,071540540IC2\*0\.890\.740\.900\.1143,291352,603509509IC1\*\*0\.920\.740\.940\.1116,808139,071275275IC2\*\*0\.890\.740\.900\.1143,291352,603270270C1:1IC10\.910\.780\.940\.1421,858134,02121,85821,858IC20\.890\.760\.900\.1352,989342,90552,98952,989CJ0\.890\.800\.900\.071581,974158158LGSR0\.930\.790\.940\.026,537278,8436,5376,5371:4IC10\.910\.780\.940\.1421,858134,02121,85887,432IC20\.890\.760\.900\.1352,989342,90552,989211,956CJ0\.890\.800\.900\.071581,974158632LGSR0\.930\.790\.940\.026,537278,8436,53726,148B\-SSIC1\*0\.910\.780\.940\.1421,858134,021690690IC2\*0\.890\.760\.900\.1352,989342,905636636IC1\*\*0\.910\.780\.940\.1421,858134,021356356IC2\*\*0\.890\.760\.900\.1352,989342,905326326

Table 3:Dataset statistics\.AverageTransactionalWeeksper CustomerNormalizedAverageTransactionalWeeksper CustomerIC1788,484,586490,13817\.310\.22IC211525,097,111937,50726\.770\.23CJ102123,8882,50049\.560\.49LGSR11720,527,165494,88041\.480\.35

\(a\)Before filtering\.
AverageTransactionalWeeksper CustomerNormalizedAverageTransactionalWeeksper CustomerIC1786,507,930155,87941\.750\.54IC211521,642,961395,89454\.670\.48CJ102119,6422,13256\.120\.55LGSR11719,318,828285,38067\.700\.58

\(b\)After filtering\.

### 4\.6Model Training and Evaluation

In this step, we train multiple machine\-learning models on the prepared datasets and evaluate their performance using the selected classification metrics\. Section[5](https://arxiv.org/html/2606.00169#S5)provides implementation details, covering model architectures, hyperparameters, the experimental setup, libraries, and computational environment\.

### Dataset Information

In total, four dataset sources were used: two proprietary datasets provided by Infocube \(denoted as IC1 and IC2\), and two publicly available datasets,The Complete JourneyandLet’s Get Sort of Real\(denoted as CJ and LGSR, respectively\)222[https://www\.dunnhumby\.com/source\-files/](https://www.dunnhumby.com/source-files/)\. The main characteristics of these sources are summarized in Table[3a](https://arxiv.org/html/2606.00169#S4.T3.sf1)for the unfiltered datasets and in Table[3b](https://arxiv.org/html/2606.00169#S4.T3.sf2)after filtering was applied\. In these tables, theTransactional Weeksfield for each dataset source is calculated by counting, for each customer, the weeks in which they made at least one transaction, and then summing these counts across all customers\.Average Transactional Weeks per Customergives the average number of transactional weeks per customer\.Normalized Average Transactional Weeks per Customerscales this value by the total period, showing the average transaction frequency per week per customer\.

From the four dataset sources, we constructed 36 distinct datasets by applying the three churn\-labeling definitions introduced in Subsection[4\.3](https://arxiv.org/html/2606.00169#S4.SS3), together with the three balancing and subsampling strategies described in Subsection[4\.5](https://arxiv.org/html/2606.00169#S4.SS5)\. Table[2](https://arxiv.org/html/2606.00169#S4.T2)summarizes the resulting dataset families and their characteristics: theCat\.column indicates whether a dataset is subsampled or balanced/imbalanced; N\-CH and CH denote non\-churners and churners, respectively\. TheNormalized Tenurerepresents each customer’s active span divided by the total number of weeks\. The churn ratio, along with the number of customers both before and after the balancing step, is also reported\.

## 5Implementation

This section outlines the implementation details, experimental environment, and model\-specific hyperparameters used in the study\. All experiments were conducted using Python 3\.10\.9, with core data processing handled using pandas 1\.5\.3 and numpy 1\.24\.4\.

Sktime 0\.16\.1 was used for handling and converting time series data structures\. Scikit\-learn 1\.7\.0 was employed for evaluating model performance using metrics such asaccuracy,precision,recall,F1\-score, andROC\-AUC, and also for implementing models such as Support Vector Machines and Random Forests\. The implementation details and hyperparameter configurations for each model are described below\.

### 5\.1UniTS

The UniTS models were implemented using the official UniTS framework, publicly available on GitHub333[https://github\.com/mims\-harvard/UniTS](https://github.com/mims-harvard/UniTS)\. All experiments were conducted using PyTorch 2\.0\.0, with GPU acceleration to enhance training efficiency\.

The UniTS framework supports a variety of training paradigms, including prompt learning, few\-shot learning, zero\-shot inference, and supervised learning\.

- •Prompt learningenables the model to adapt to new tasks using learned task\-specific prompt tokens, without modifying the underlying model weights\.
- •Few\-shot learningallows the model to generalize to new tasks using only a small number of labeled examples\.
- •Zero\-shot inferenceleverages the pre\-trained model’s representations to perform tasks for which it has not been explicitly trained\.
- •Supervised learningrefers to traditional fine\-tuning using fully labeled datasets for a specific downstream task\.

In the original UniTS paper, the supervised learning approach yielded the best overall performance\. Consequently, we adopt this approach in our study to showcase the full potential of the model\. Rather than training from randomly initialized weights, we leveraged transfer learning by initializing the model with the publicly available checkpointunits\_x64\_supervised\_checkpoint\.pth444[https://github\.com/mims\-harvard/UniTS/releases/tag/ckpt](https://github.com/mims-harvard/UniTS/releases/tag/ckpt), obtained from the official UniTS release repository\.

It is worth noting that we did not perform extensive hyperparameter tuning for UniTS\. This decision aligns with the framework’s design philosophy, which emphasizes plug\-and\-play usability over complex configuration\.

The key hyperparameters and training settings used in our experiments are summarized in Table[4a](https://arxiv.org/html/2606.00169#S5.T4.sf1)\. As discussed in Section[7](https://arxiv.org/html/2606.00169#S7.SSx2), more targeted tuning of UniTS\-specific hyperparameters or the use of different transformer\-based architectures may yield different outcomes and insights\. Therefore, the performance results presented here for UniTS should be taken as reflective of the potential performance of transformer\-based architectures in general\.

### 5\.2XGBoost

Experiments with XGBoost were conducted using theXGBClassifierinterface from version3\.0\.2\. The model was trained using the binary logistic objective for churn classification\. Feature values were used in their raw form without normalization, as the model is robust to unscaled input\. The hyperparameters used are summarized in Table[4c](https://arxiv.org/html/2606.00169#S5.T4.sf3)\.

### 5\.3Support Vector Machine

Support Vector Machine models were implemented using theSVCclass from the scikit\-learn library, version1\.7\.0\. Before training, the input features were standardized usingStandardScaler, which centers each feature around a zero mean and unit variance\. This scaling step is essential for distance\-based models like SVM to function properly\.

The classifier was trained using a radial basis function \(RBF\) kernel\. Table[4d](https://arxiv.org/html/2606.00169#S5.T4.sf4)lists the hyperparameters used\.

### 5\.4Random Forest

Random Forest models were implemented using theRandomForestClassifierclass from scikit\-learn version 1\.7\.0\. No feature scaling was applied prior to training, as tree\-based models are invariant to the scale of input features\.

Table[4b](https://arxiv.org/html/2606.00169#S5.T4.sf2)shows the configuration used for the Random Forest model\.

Table 4:Model Hyperparameters\.\(a\)UniTS\.
\(b\)Random Forest\.

ParameterValueDescriptionobjectivebinary:logisticFor binary classificationn\_estimators100Number of boosting roundsmax\_depth4Limits tree complexity to reduce overfittinglearning\_rate0\.1Moderate rate for stable convergencesubsample0\.8Random subset of data per treecolsample\_bytree0\.8Subset of features per treerandom\_state42For reproducibilityeval\_metricloglossLog\-loss used during training

\(c\)XGBoost\.
ParameterValueDescriptionkernelrbfRadial basis function kernelC1\.0Regularization parametergammascaleKernel coefficient based on number of featuresprobabilityTrueEnables probability estimates \(used for ROC\-AUC\)random\_state42For reproducibility

\(d\)SVM\.

All experiments were conducted on a system with an AMD Ryzen 9 7950X CPU, 64 GB DDR5 RAM, two NVIDIA RTX 4090 GPUs, a 2 TB NVME SSD, running Ubuntu LTS 22\.04\.

## 6Results

In this section, we evaluate the models using standard performance metrics: Accuracy, Precision, Recall, F1\-score, and AUC\-ROC\. These metrics are particularly relevant for churn prediction, where datasets are often imbalanced\. In such cases, AUC\-ROC provides a more reliable basis for performance comparison across models\. Moreover, both false positives and false negatives carry significant business implications: excessive false positives may lead to unnecessary spending on customer retention, while false negatives can result in missed opportunities to intervene with actual churners\. Consequently, Precision and Recall serve as critical complementary measures, providing deeper insights into these trade\-offs\.

The evaluation is conducted on 36 datasets derived from four raw data sources, as detailed in Section[4](https://arxiv.org/html/2606.00169#S4)\. To ensure a robust comparison, the datasets were systematically constructed with controlled variations in size, class imbalance, and churn definitions \(A, B, and C\)\. The corresponding results are reported in Table[5](https://arxiv.org/html/2606.00169#S6.T5)\.

We observe that the tree\-based models,XGBoostandRandom Forest, consistently outperform both the transformer\-basedUniTSarchitecture and the more conventionalSupport Vector Machine\(SVM\) under all the conditions mentioned above\.

We first contrast the performance under standard conditions with sufficient dataset size, namely the datasets generated from the data sourcesIC1,IC2, andLGSRunder balanced \(1:1\) and limited unbalanced \(1:4\) conditions\. The datasets generated from theCJdata source and all datasets under theB\-SScategories are considered data\-scarce, i\.e\., due to the limited number of training examples, model performance is, in some cases, severely limited\. We discuss and contrast model performance under these conditions separately in Subsection[6\.2](https://arxiv.org/html/2606.00169#S6.SS2)\.

When applyingChurn Definition\-Ato balanced datasets \(1:1\),XGBoostachieves the highest accuracy of0\.770\.77onIC1, closely followed byRandom Forestat0\.760\.76, whileUniTSandSVMremain lower at0\.720\.72\. The superiority of tree\-based models holds acrossIC2andLGSRas well\. Accuracy is reported here, as it serves as a sufficient metric to evaluate performance under balanced conditions\.

When shifting to the imbalanced datasets \(1:4\), we contrast precision and recall metrics to more effectively highlight performance differences under this imbalance\. OnIC1, the precision ranking is as follows:Random Forest\(0\.760\.76\),XGBoost\(0\.750\.75\),SVM\(0\.700\.70\), andUniTS\(0\.690\.69\)\. For recall,XGBoostachieves the highest value of0\.470\.47, followed byRandom Forestat0\.430\.43,UniTSat0\.350\.35, andSVMat only0\.090\.09\. Similar patterns are observed onIC2andLGSR, confirming the superior performance of tree\-based methods also under class imbalance\.

Comparable trends hold under the alternative churn definitions B and C as well\.

Below, we further analyze how model performance varies with class imbalance, dataset size, and different churn definitions, providing detailed insights into model\-specific behavior\.

### 6\.1Performance by Churn Definition

Increasing the churn threshold \(moving fromChurn Definition AtoChurn Definition C\) generally improves performance across all models\. For example, on the 1:1 balanced IC1 dataset, UniTS achieves an accuracy of0\.720\.72underChurn Definition A, which rises to0\.740\.74underChurn Definition B, and further improves to0\.750\.75under the most lenient threshold,Churn Definition C\.

This increased performance could be due to an increase in the number of dataset samples used for training\.

Note that since we use sub\-sampling to limit dataset imbalance, increasing the size of the minority class, which is the churn class in this instance, increases the total number of samples used for training\. As can be seen in Table[2](https://arxiv.org/html/2606.00169#S4.T2), by moving to a more lenient definition, fromChurn Definition AtoChurn Definition C, the number of customers classified as churners increases\. For example, for theIC2data source, the churn rate increases from 8% underChurn Definition Ato 11% underChurn Definition B, and to 14% underChurn Definition C\. This corresponds to an increase in the total number of customers in the dataset from 24,850 to 43,716 betweenChurn Definitions AandChurn Definition\-C\.

Another reason for the improved performance could be that churn prediction under a more lenient definition is an easier task, as the classes are more easily separable\. ForChurn Definition A, both the monetary value and frequency of a customer must drop by 70% between the observation and labeling window\. Customers exhibiting such an aggressive drop may represent involuntary churners, e\.g\., those who have churned due to relocation or unexpected life events, where the weeks leading up to the churn event might not show strong behavioral signals indicative of churn\.

Table 5:Model results\.Cat\.DataMetricsUniTSXGBRFSVM1:1IC1Accuracy0\.720\.770\.760\.72Precision0\.720\.770\.750\.74Recall0\.740\.780\.780\.67F1\-score0\.730\.770\.760\.70ROC\-AUC0\.800\.870\.870\.77IC2Accuracy0\.780\.810\.810\.77Precision0\.780\.810\.810\.82Recall0\.770\.820\.800\.69F1\-score0\.770\.820\.810\.75ROC\-AUC0\.870\.910\.910\.83CJAccuracy0\.520\.720\.720\.62Precision0\.570\.850\.850\.69Recall0\.760\.650\.650\.65F1\-score0\.650\.730\.730\.67ROC\-AUC0\.500\.760\.760\.65LGSRAccuracy0\.800\.940\.930\.83Precision0\.800\.930\.930\.82Recall0\.840\.950\.950\.90F1\-score0\.820\.940\.940\.86ROC\-AUC0\.880\.980\.980\.921:4IC1Accuracy0\.840\.860\.860\.81Precision0\.690\.750\.760\.70Recall0\.350\.470\.430\.09F1\-score0\.460\.570\.550\.17ROC\-AUC0\.810\.860\.860\.78IC2Accuracy0\.870\.870\.870\.83Precision0\.730\.740\.760\.71Recall0\.520\.570\.510\.24F1\-score0\.600\.640\.610\.35ROC\-AUC0\.880\.900\.900\.81CJAccuracy0\.280\.940\.960\.86Precision0\.110\.800\.890\.00Recall0\.600\.800\.800\.00F1\-score0\.190\.800\.840\.00ROC\-AUC0\.450\.910\.930\.86LGSRAccuracy0\.850\.940\.940\.85Precision0\.680\.860\.880\.83Recall0\.520\.860\.810\.39F1\-score0\.590\.860\.850\.53ROC\-AUC0\.860\.970\.980\.90B\-SSIC1\*Accuracy0\.490\.730\.720\.67Precision0\.520\.750\.760\.75Recall0\.670\.750\.720\.59F1\-score0\.590\.750\.740\.66ROC\-AUC0\.470\.840\.840\.72IC2\*Accuracy0\.470\.780\.780\.68Precision0\.510\.780\.780\.70Recall0\.640\.830\.830\.69F1\-score0\.570\.810\.810\.70ROC\-AUC0\.480\.870\.870\.74IC1\*\*Accuracy0\.490\.800\.780\.57Precision0\.510\.770\.760\.62Recall0\.740\.880\.830\.50F1\-score0\.600\.820\.800\.55ROC\-AUC0\.500\.850\.860\.69IC2\*\*Accuracy0\.490\.750\.770\.73Precision0\.540\.800\.860\.82Recall0\.760\.740\.720\.66F1\-score0\.630\.770\.780\.73ROC\-AUC0\.460\.820\.850\.72

\(a\) A \(α\\alpha= 0\.3,β\\beta= 0\.3\)

Cat\.DataMetricsUniTSXGBRFSVM1:1IC1Accuracy0\.740\.790\.780\.73Precision0\.740\.800\.810\.76Recall0\.740\.770\.750\.68F1\-score0\.740\.780\.780\.72ROC\-AUC0\.820\.890\.880\.79IC2Accuracy0\.800\.830\.820\.74Precision0\.810\.820\.820\.76Recall0\.780\.830\.820\.71F1\-score0\.790\.830\.820\.73ROC\-AUC0\.880\.910\.910\.82CJAccuracy0\.540\.920\.920\.70Precision0\.490\.910\.880\.65Recall0\.770\.910\.950\.68F1\-score0\.600\.910\.910\.67ROC\-AUC0\.620\.960\.970\.77LGSRAccuracy0\.800\.950\.940\.84Precision0\.800\.920\.920\.79Recall0\.790\.970\.970\.91F1\-score0\.790\.950\.950\.85ROC\-AUC0\.880\.980\.980\.921:4IC1Accuracy0\.850\.870\.870\.83Precision0\.710\.760\.770\.75Recall0\.460\.530\.490\.22F1\-score0\.560\.620\.600\.34ROC\-AUC0\.840\.880\.880\.81IC2Accuracy0\.870\.880\.870\.83Precision0\.730\.730\.750\.71Recall0\.560\.610\.550\.23F1\-score0\.640\.660\.640\.35ROC\-AUC0\.890\.910\.910\.83CJAccuracy0\.350\.930\.920\.85Precision0\.170\.931\.001\.00Recall0\.710\.620\.520\.14F1\-score0\.270\.740\.690\.25ROC\-AUC0\.510\.980\.970\.85LGSRAccuracy0\.870\.940\.940\.86Precision0\.710\.840\.850\.81Recall0\.590\.870\.840\.42F1\-score0\.650\.860\.850\.56ROC\-AUC0\.900\.980\.980\.92B\-SSIC1\*Accuracy0\.470\.770\.750\.69Precision0\.460\.730\.730\.65Recall0\.730\.800\.760\.72F1\-score0\.560\.760\.740\.69ROC\-AUC0\.550\.860\.860\.75IC2\*Accuracy0\.490\.770\.780\.73Precision0\.500\.750\.760\.75Recall0\.740\.820\.830\.70F1\-score0\.600\.790\.790\.73ROC\-AUC0\.490\.860\.870\.77IC1\*\*Accuracy0\.550\.780\.850\.76Precision0\.580\.820\.870\.80Recall0\.780\.790\.870\.78F1\-score0\.660\.810\.870\.79ROC\-AUC0\.500\.880\.900\.83IC2\*\*Accuracy0\.540\.810\.810\.66Precision0\.570\.840\.880\.71Recall0\.720\.840\.750\.66F1\-score0\.640\.840\.810\.68ROC\-AUC0\.610\.910\.910\.78

\(b\) B \(α\\alpha= 0\.4,β\\beta= 0\.45\)

Cat\.DataMetricsUniTSXGBRFSVM1:1IC1Accuracy0\.750\.800\.800\.75Precision0\.740\.810\.810\.77Recall0\.770\.790\.780\.71F1\-score0\.760\.800\.800\.74ROC\-AUC0\.820\.890\.890\.81IC2Accuracy0\.800\.840\.830\.79Precision0\.800\.830\.830\.83Recall0\.800\.850\.830\.72F1\-score0\.800\.840\.830\.77ROC\-AUC0\.890\.920\.920\.85CJAccuracy0\.560\.810\.810\.75Precision0\.540\.810\.810\.70Recall0\.840\.810\.810\.88F1\-score0\.660\.810\.810\.78ROC\-AUC0\.610\.850\.860\.79LGSRAccuracy0\.790\.930\.930\.83Precision0\.760\.910\.910\.78Recall0\.870\.960\.960\.92F1\-score0\.810\.940\.930\.84ROC\-AUC0\.890\.970\.970\.911:4IC1Accuracy0\.860\.880\.870\.84Precision0\.700\.740\.760\.73Recall0\.490\.570\.530\.27F1\-score0\.580\.650\.630\.40ROC\-AUC0\.860\.890\.900\.82IC2Accuracy0\.870\.880\.880\.83Precision0\.720\.740\.750\.70Recall0\.590\.640\.590\.30F1\-score0\.650\.690\.660\.42ROC\-AUC0\.900\.920\.920\.84CJAccuracy0\.390\.900\.850\.80Precision0\.240\.880\.830\.80Recall0\.800\.630\.430\.11F1\-score0\.370\.730\.570\.20ROC\-AUC0\.610\.940\.940\.83LGSRAccuracy0\.870\.940\.930\.86Precision0\.710\.820\.830\.76Recall0\.540\.870\.820\.43F1\-score0\.610\.840\.830\.55ROC\-AUC0\.900\.970\.970\.92B\-SSIC1\*Accuracy0\.730\.760\.780\.71Precision0\.790\.770\.800\.71Recall0\.630\.760\.760\.74F1\-score0\.700\.760\.780\.72ROC\-AUC0\.790\.880\.880\.78IC2\*Accuracy0\.730\.850\.850\.75Precision0\.780\.870\.860\.73Recall0\.640\.840\.830\.81F1\-score0\.700\.850\.850\.77ROC\-AUC0\.800\.930\.930\.84IC1\*\*Accuracy0\.470\.750\.710\.66Precision0\.470\.770\.720\.63Recall0\.720\.680\.670\.71F1\-score0\.570\.720\.690\.67ROC\-AUC0\.490\.820\.830\.71IC2\*\*Accuracy0\.500\.820\.830\.69Precision0\.530\.830\.840\.72Recall0\.680\.830\.860\.74F1\-score0\.600\.830\.850\.73ROC\-AUC0\.540\.860\.860\.78

\(c\) C \(α\\alpha= 0\.5,β\\beta= 0\.5\)

### 6\.2Impact of Class Imbalance

In imbalanced classification settings, accuracy becomes a misleading metric, as any apparent gain is largely driven by the dominance of the non\-churner class\. To properly capture the models’ ability to detect churners, we instead evaluate performance using precision and recall\. Precision measures how many of the predicted churners are truly churners, while recall measures how many of the actual churners the model successfully identifies\. These metrics are more informative under imbalance, as they directly reflect performance on the minority class, which is the primary focus in churn prediction for real\-world scenarios\.

When transitioning from balanced \(1:1\) to imbalanced \(1:4\) datasets, model performance generally decreases as the class ratio becomes more skewed\. The observed trend shows that this decline is most pronounced for models that are highly sensitive to class distribution, such as SVM, while tree\-based methods \(XGB, RF\) and the transformer\-based UniTS demonstrate greater robustness\.

For example, under the IC2 dataset withChurn Definition A, the following results are observed when moving from a balanced to an imbalanced setting\. For UniTS, precision decreases from0\.780\.78to0\.730\.73and recall drops from0\.770\.77to0\.520\.52\. For XGB, precision declines from0\.810\.81to0\.740\.74and recall decreases from0\.820\.82to0\.570\.57\. For RF, precision goes from0\.810\.81to0\.760\.76and recall falls from0\.800\.80to0\.510\.51\. Finally, for SVM, precision decreases from0\.820\.82to0\.710\.71, but recall collapses dramatically from0\.690\.69to0\.240\.24\.

These results reveal a clear pattern: while precision remains relatively stable across all models, recall suffers a substantial decline under imbalance\. This indicates that models become less effective at identifying churners as the minority class becomes underrepresented\. Among the evaluated methods, tree\-based models \(XGB and RF\) and the transformer\-based UniTS demonstrate better resilience, maintaining moderate recall values even when the class distribution is skewed\.

SVM experiences the most severe performance degradation under imbalance because, in the soft\-margin setting, the separating hyperplane becomes skewed toward the minority class\. As imbalance grows, the ratio of positive to negative support vectors becomes more imbalanced and samples near the margin are more likely to be classified as negative\. While its precision decreases moderately \(0\.82→0\.710\.82\\to 0\.71\), recall collapses to0\.240\.24because the model fails to predict churners effectively\. This consistent trend demonstrates SVM’s sensitivity to class imbalance and it’s worse performance compared to XGB, RF, and UniTS, which handle skewed distributions more robustly\.

### 6\.3Small Dataset and Data Efficiency Analysis

Since the datasets considered in this section are balanced \(1:1\), the accuracy metric is sufficient for comparing performance\. As the dataset size decreases, the performance of all models generally declines, although the degree of degradation under data scarcity varies significantly across models\.

As a representative example, we examine the IC2 dataset underChurn Definition C\(see Table[2](https://arxiv.org/html/2606.00169#S4.T2)\), where IC2 represents the full dataset, IC2\* is a considerably smaller subset, and IC2\*\* is approximately half the size of IC2\*\. Under these settings, UniTS achieves an accuracy of0\.800\.80on the full IC2 dataset but drops to0\.730\.73under IC2\*, and further collapses to0\.500\.50on IC2\*\*\. XGB achieves0\.840\.84accuracy on IC2 and maintains a strong0\.850\.85on IC2\*, before slightly decreasing to0\.820\.82under IC2\*\*\. Similarly, RF obtains0\.830\.83accuracy on IC2, remains stable at0\.850\.85on IC2\*, and retains0\.830\.83on IC2\*\*\. In contrast, SVM performs reasonably well on the full IC2 dataset with0\.790\.79accuracy and drops slightly to0\.750\.75under IC2\*, but declines further to0\.690\.69on IC2\*\*\. While these numbers are reported for IC2 as an example, similar patterns are consistently observed across other datasets\.

These results reveal a clear and consistent pattern: tree\-based models \(XGB and RF\) exhibit strong robustness under limited data availability, retaining high accuracy even when datasets are significantly reduced\. SVM shows moderate resilience but suffers more noticeable degradation in extremely low\-resource scenarios\. UniTS, on the other hand, demonstrates a consistent trend of severe performance deterioration when data becomes scarce, confirming that it struggles to generalize effectively without sufficient training samples\.

Transformer\-based models like UniTS require larger datasets due to their higher complexity and large parameter count\. Unlike tree\-based methods, which generalize well with limited samples, transformers need abundant data to effectively capture temporal patterns and learn stable representations\. In scarce\-resource scenarios, the limited number of examples reduces the diversity of behavioral signals, making it harder for the model to distinguish churners from non\-churners\.

## 7Threats to Validity

This section discusses potential limitations and factors that may affect the validity and generalizability of the findings presented in this study\.

### Limitations of Dataset, Applied Churn Definition, and Filtering Techniques

Although the experiments were conducted on multiple datasets and under different churn definitions, the scope of this study remains limited, and the results may not fully generalize to all churn prediction tasks in retail contexts\. In domains where consumer behavior is more complex, the UniTS model could have greater opportunities to capture intricate interdependencies, potentially leading to improved performance\.

Moreover, while several churn labeling strategies were explored based on Infocube’s requirements, the study does not cover the full spectrum of possible churn definitions\. Alternative labeling schemes may substantially change the framing of the classification task and, in turn, affect model performance\.

Finally, the dataset filtering procedure also introduces limitations to the applicability of our results, as customers with fewer than 20 transactional weeks were excluded\. While this threshold was justified in our case, ensuring that the analysis focused on customers with sufficient transactional history to produce more reliable predictions, it can restrict the transferability of findings to other domains\. For instance, in e\-commerce or other retail environments where shorter customer life\-cycles are common and business objectives may differ, models could produce different outcomes when applied to customers with limited histories\.

### Considerations Regarding Transformer\-Based Methods

This work does not attempt to exhaustively optimize transformer\-based models for churn prediction\. Since UniTS is designed as a user\-friendly, plug\-and\-play platform without the need for extensive model tuning, the reported results are valid for performance comparisons under default configurations\. However, it remains possible that transformer\-based approaches with architecture, optimizers, and hyperparameters specifically fine\-tuned for churn prediction tasks could achieve superior results\.

## 8Conclusion

This study presents a comprehensive comparison between conventional machine learning models and the transformer\-based UniTS model for the task of churn prediction in non\-contractual retail environments\. By conducting a rigorous evaluation of the models across 36 datasets with varying levels of imbalance, size, and churn definitions, we shed light on the performance trade\-offs between classical and modern AI techniques in real\-world retail applications\.

Our findings demonstrate that tree\-based models, particularly XGBoost and Random Forest, consistently outperform UniTS and SVM in most experimental settings\. These models offer high predictive accuracy, robustness to class imbalance, and remarkable data efficiency, achieving strong performance even with as few as 200–600 churners\. In contrast, the UniTS model, while theoretically well\-suited for modeling temporal dynamics, underperforms on small to medium\-sized tabular datasets, likely due to its high data requirements\.

From a practical perspective, the results suggest that traditional ensemble methods remain a reliable and efficient choice for churn prediction in typical retail environments where data is structured and often limited\. Although transformer\-based models may hold promise, particularly in scenarios involving large\-scale behavioral data and rich temporal histories, their advantages are not fully realized under constrained data settings\.

Looking ahead, future research could explore how UniTS and similar architectures perform when additional behavioral and contextual features \(e\.g\., browsing activity, promotions, seasonality\) are included\. Additionally, a systematic analysis of training time, energy efficiency, and deployment constraints would help assess the suitability of different model types for modern AI\-accelerated infrastructures\. By advancing our understanding of the conditions under which advanced architectures can provide tangible benefits, we hope to guide more effective and scalable applications of AI in customer retention strategies\.

\{credits\}

#### 8\.0\.1Acknowledgements

This work was carried out while Syed Saad Saif was enrolled in the Italian National Doctorate on Artificial Intelligence run by Sapienza University of Rome in collaboration with UnitelmaSapienza University of Rome, with his doctoral fellowship co\-funded by Infocube SRL\.

#### 8\.0\.2\\discintname

The authors declare that they have no competing interests\.

## References

- \[1\]K\. B\. Agbemadon, R\. Couturier, and D\. Laiymani\(2022\-03\)Churn detection using machine learning in the retail industry\.In2022 2nd International Conference on Computer, Control and Robotics \(ICCCR\),Vol\.,pp\. 172–178\.Note:External Links:[Document](https://dx.doi.org/10.1109/ICCCR54399.2022.9790213),ISSNCited by:[§2](https://arxiv.org/html/2606.00169#S2.p2.1)\.
- \[2\]S\. Baghla and G\. Gupta\(2022\)Performance evaluation of various classification techniques for customer churn prediction in e\-commerce\.Microprocessors and MicrosystemsInternational Journal of Electrical and Computer Engineering \(IJECE\)IEEE Open Journal of the Computer SocietyBulletin of Electrical Engineering and InformaticsIAES International Journal of Artificial Intelligence \(IJ\-AI\)International Journal of System Assurance Engineering and ManagementJournal of Applied Data SciencesComputational Intelligence and NeuroscienceEcological InformaticsJournal of Information and Telecommunication94,pp\. 104680\.External Links:ISSN 0141\-9331,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.micpro.2022.104680),[Link](https://www.sciencedirect.com/science/article/pii/S0141933122002101)Cited by:[§3\.1](https://arxiv.org/html/2606.00169#S3.SS1.p2.1)\.
- \[3\]A\. Barsotti, G\. Gianini, C\. Mio, J\. Lin, H\. Babbar, A\. Singh, F\. Taher, and E\. Damiani\(2024\)A decade of churn prediction techniques in the telco domain: a survey\.SN Computer Science5\(4\),pp\. 404\.External Links:[Document](https://dx.doi.org/10.1007/s42979-024-02722-7)Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p4.1)\.
- \[4\]E\. Basker\(2012\)Supersize it: the growth of retail chains and the rise of the big\-box store\.The Journal of Economic Perspectives26\(2\),pp\. 177–198\.Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p1.1)\.
- \[5\]D\. J\. Benkendorf and C\. P\. Hawkins\(2020\)Effects of sample size and network depth on a deep learning approach to species distribution modeling\.60,pp\. 101137\.Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p5.1)\.
- \[6\]I\. Boukrouh and A\. Azmani\(2025\)Explainable machine learning models applied to predicting customer churn for e\-commerce\.14\(1\)\.Note:Cited by:[§2](https://arxiv.org/html/2606.00169#S2.p2.1)\.
- \[7\]L\. Breiman\(2001\)Random forests\.Machine learning45\(1\),pp\. 5–32\.Cited by:[§3\.2\.1](https://arxiv.org/html/2606.00169#S3.SS2.SSS1.p1.1)\.
- \[8\]W\. Buckinx and D\. Van den Poel\(2005\)Customer retention in the non\-contractual retail setting: the role of purchase behavior\.Expert Systems with Applications28\(3\),pp\. 479–486\.Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p2.1)\.
- \[9\]L\. Chen and Y\. Wang\(2019\)B2C e\-commerce customer churn prediction based on k\-means and svm\.InInternational Conference on Data Mining,Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p4.1)\.
- \[10\]T\. Chen and C\. Guestrin\(2016\)Xgboost: a scalable tree boosting system\.InProceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining,pp\. 785–794\.Cited by:[§3\.2\.2](https://arxiv.org/html/2606.00169#S3.SS2.SSS2.p1.1)\.
- \[11\]N\. C\. Colli, S\. S\. Saif, G\. Maggiore, and D\. Distante\(2025\)Artificial intelligence for customer churn prediction in non\-contractual retail environments: a systematic literature review\.Note:Manuscript under peer reviewCited by:[§2](https://arxiv.org/html/2606.00169#S2.p1.1),[§3\.2](https://arxiv.org/html/2606.00169#S3.SS2.p1.1)\.
- \[12\]S\. Coolwijk, S\. S\. M\. Ziabari, and F\. Angileri\(2024\)Vision transformer approach to customer churn prediction radar chart image classification for non\-subscription based e\-commerce\.InInformation Integration and Web Intelligence: 26th International Conference, IiWAS 2024, Bratislava, Slovak Republic, December 2–4, 2024, Proceedings, Part II,Berlin, Heidelberg,pp\. 75–80\.External Links:ISBN 978\-3\-031\-78092\-9,[Link](https://doi.org/10.1007/978-3-031-78093-6_6),[Document](https://dx.doi.org/10.1007/978-3-031-78093-6%5F6)Cited by:[§3\.1](https://arxiv.org/html/2606.00169#S3.SS1.p2.1)\.
- \[13\]C\. Cortes and V\. Vapnik\(1995\)Support\-vector networks\.Machine learning20\(3\),pp\. 273–297\.Cited by:[§3\.2\.3](https://arxiv.org/html/2606.00169#S3.SS2.SSS3.p1.1)\.
- \[14\]L\. Gan\(2022\)XGBoost\-based e\-commerce customer loss prediction\.2022\(1\),pp\. 1858300\.Note:External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.1155/2022/1858300),[Link](https://onlinelibrary.wiley.com/doi/abs/10.1155/2022/1858300),https://onlinelibrary\.wiley\.com/doi/pdf/10\.1155/2022/1858300Cited by:[§2](https://arxiv.org/html/2606.00169#S2.p3.1)\.
- \[15\]S\. Gao, T\. Koker, O\. Queen, T\. Hartvigsen, T\. Tsiligkaridis, and M\. Zitnik\(2024\)UniTS: a unified multi\-task time series model\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=nBOdYBptWW)Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p5.1),[§1](https://arxiv.org/html/2606.00169#S1.p6.1),[§3\.2\.4](https://arxiv.org/html/2606.00169#S3.SS2.SSS4.p1.1),[§3\.2](https://arxiv.org/html/2606.00169#S3.SS2.p1.1)\.
- \[16\]S\. Geiler, S\. Affeldt, and M\. Nadif\(2022\)Machine learning techniques for customer churn prediction: a review\.IEEE Access10,pp\. 34567–34589\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2022.3141592)Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p4.1)\.
- \[17\]R\. Gupta and N\. Sharma\(2021\)Churn detection using machine learning in the retail industry\.InProceedings of ICMLC,Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p2.1),[§1](https://arxiv.org/html/2606.00169#S1.p3.1)\.
- \[18\]J\. Hagberg, M\. Sundstrom, and N\. Egels\-Zandén\(2016\)The digitalization of retailing: an exploratory framework\.International Journal of Retail & Distribution Management44\(7\),pp\. 694–712\.Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p3.1)\.
- \[19\]S\. J, Ch\. Gangadhar, R\. K\. Arora, P\.N\. Renjith, J\. Bamini, and Y\. d\. Chincholkar\(2023\)E\-commerce customer churn prevention using machine learning\-based business intelligence strategy\.Measurement: Sensors27\.External Links:[Document](https://dx.doi.org/10.1016/j.measen.2023.100728),[Link](https://www.sciencedirect.com/science/article/pii/S2665917423000648)Cited by:[§3\.1](https://arxiv.org/html/2606.00169#S3.SS1.p2.1)\.
- \[20\]H\. Lee and S\. Cho\(2020\)Machine learning\-based customer churn prediction in home appliance rental business\.Expert Systems with Applications150,pp\. 113255\.Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p3.1)\.
- \[21\]N\. Lee, H\. Lee, J\. Hsin, and S\. Fang\(2024\)Prediction of customer behavior changing via a hybrid approach\.5\(\),pp\. 27–38\.External Links:[Document](https://dx.doi.org/10.1109/OJCS.2023.3336904),ISSN 2644\-1268Cited by:[§3\.1](https://arxiv.org/html/2606.00169#S3.SS1.p2.1)\.
- \[22\]A\. Manzoor, M\. A\. Qureshi, E\. Kidney, and L\. Longo\(2024\)A review on machine learning methods for customer churn prediction and recommendations for business practitioners\.IEEE Access12,pp\. 70434–70463\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2024.3402092)Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p4.1),[§2](https://arxiv.org/html/2606.00169#S2.p1.1),[§3\.2](https://arxiv.org/html/2606.00169#S3.SS2.p1.1)\.
- \[23\]M\. Pondel, M\. Wuczynski, W\. Gryncewicz, Ł\. Łysik, M\. Hernes, A\. Rot, and A\. Kozina\(2021\)Deep learning for customer churn prediction in e\-commerce decision support\.Vol\.1,pp\. 3 – 12\.External Links:[Document](https://dx.doi.org/10.52825/bis.v1i.42),[Link](https://www.tib-op.org/ojs/index.php/bis/article/view/42)Cited by:[§3\.1](https://arxiv.org/html/2606.00169#S3.SS1.p2.1)\.
- \[24\]M\. Pondel, M\. Wuczynski, W\. Gryncewicz, Ł\. Łysik, M\. Hernes, A\. Rot, and A\. Kozina\(2021\)Deep learning for customer churn prediction in e\-commerce decision support\.Vol\.1,pp\. 3 – 12\.Note:; Cited by: 18; All Open Access, Hybrid Gold Open AccessExternal Links:[Document](https://dx.doi.org/10.52825/bis.v1i.42),[Link](https://www.tib-op.org/ojs/index.php/bis/article/view/42)Cited by:[§2](https://arxiv.org/html/2606.00169#S2.p3.1)\.
- \[25\]A\. D\. Rachid, A\. Abdellah, B\. Belaid, and L\. Rachid\(2018\)Clustering prediction techniques in defining and predicting customers defection: the case of e\-commerce context\.8\(4\),pp\. 2367–2383\.External Links:ISSN 2722\-2578,[Document](https://dx.doi.org/10.11591/ijece.v8i4.pp2367-2383),[Link](https://ijece.iaescore.com/index.php/IJECE/article/view/8531)Cited by:[§3\.1](https://arxiv.org/html/2606.00169#S3.SS1.p2.1)\.
- \[26\]M\. A\. A\. Rahib, N\. Saha, R\. Mia, and A\. Sattar\(2024\)Customer data prediction and analysis in e\-commerce using machine learning\.13\(4\)\.Note:Cited by:[§2](https://arxiv.org/html/2606.00169#S2.p2.1)\.
- \[27\]W\. Reinartz and V\. Kumar\(2003\)The impact of customer relationship characteristics on profitable lifetime duration\.Journal of Marketing67\(1\),pp\. 77–99\.Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p1.1)\.
- \[28\]D\. H\. Rudd, H\. Huo, and G\. Xu\(2021\)Causal analysis of customer churn using deep learning\.In2021 International Conference on Digital Society and Intelligent Systems \(DSInS\),Vol\.,pp\. 319–324\.External Links:[Document](https://dx.doi.org/10.1109/DSInS54396.2021.9670561)Cited by:[§3\.1](https://arxiv.org/html/2606.00169#S3.SS1.p2.1)\.
- \[29\]Seema and G\. Gupta\(2024\-01\)Development of fading channel patch based convolutional neural network models for customer churn prediction\.15\(1\),pp\. 391–411\.Note:External Links:[Document](https://dx.doi.org/10.1007/s13198-022-01759-),[Link](https://ideas.repec.org/a/spr/ijsaem/v15y2024i1d10.1007_s13198-022-01759-2.html)Cited by:[§2](https://arxiv.org/html/2606.00169#S2.p3.1)\.
- \[30\]M\. Shahabikargar, A\. Beheshti, X\. Zhang, J\. Foo, and A\. Jolfaei\(2025\)A comprehensive survey on customer churn analysis studies\.0\(0\),pp\. 1–47\.External Links:[Document](https://dx.doi.org/10.1080/24751839.2025.2528440),[Link](https://doi.org/10.1080/24751839.2025.2528440),https://doi\.org/10\.1080/24751839\.2025\.2528440Cited by:[§2](https://arxiv.org/html/2606.00169#S2.p1.1),[§3\.2](https://arxiv.org/html/2606.00169#S3.SS2.p1.1)\.
- \[31\]R\. Singh and M\. Kapoor\(2021\)Understanding customer attrition at an individual level: a new model in grocery retail context\.Journal of Retailing and Consumer Services58,pp\. 102345\.Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p2.1)\.
- \[32\]P\. Sunarya, U\. Rahardja, S\. Chen, Y\. Lic, and M\. Hardini\(2024\)Deciphering digital social dynamics: a comparative study of logistic regression and random forest in predicting e\-commerce customer behavior\.5\(1\)\.Note:Cited by:[§2](https://arxiv.org/html/2606.00169#S2.p2.1)\.
- \[33\]H\. Wu, T\. Hu, Y\. Liu, H\. Zhou, J\. Wang, and M\. Long\(2023\)TimesNet: temporal 2d\-variation modeling for general time series analysis\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p5.1)\.
- \[34\]H\. Wu, J\. Wu, J\. Xu, J\. Wang, and M\. Long\(2022\)Flowformer: linearizing transformers with conservation flows\.arXiv preprint arXiv:2202\.06258\.Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p5.1)\.
- \[35\]R\. Zhang and J\. Kim\(2022\)Customer churn in retail e\-commerce business: spatial and machine learning approach\.Journal of Retail Analytics18,pp\. 122–135\.Cited by:[§1](https://arxiv.org/html/2606.00169#S1.p3.1)\.