Performance Drift Detection in Machine Learning as a Service (MLaaS) for IoT Environments

# Performance Drift Detection in Machine Learning as a Service (MLaaS) for IoT Environments
Source: [https://arxiv.org/html/2608.18555](https://arxiv.org/html/2608.18555)
Sajib Mistry[https://orcid.org/0000-0001-7513-3789](https://orcid.org/0000-0001-7513-3789)Sheik Mohammad Mostakim Fattah[https://orcid.org/0000-0002-9103-6089](https://orcid.org/0000-0002-9103-6089)Erik Elmroth[https://orcid.org/0000-0002-2633-6798](https://orcid.org/0000-0002-2633-6798)Aneesh Krishna[https://orcid.org/0000-0001-8637-5732](https://orcid.org/0000-0001-8637-5732)and Monowar BhuyanThanks:Deepak Kanneganti, Sajib Mistry, Sheik Mohammad Mostakim Fattah, and Aneesh Krishna are with the School of Electrical Engineering, Computing and Mathematical Sciences \(EECMS\), Curtin University, Perth, WA, Australia \(e\-mail: \{s\.kanneganti, sajib\.mistry, sheik\.fattah, a\.krishna\}@curtin\.edu\.au\)\.Thanks:Erik Elmroth and Monowar Bhuyan are with the Department of Computing Science, Umeå University, Umeå, Sweden \(e\-mail: \{elmroth, monowar\}@cs\.umu\.se\)\.

###### Abstract

Machine Learning as a Service \(MLaaS\) is a powerful cloud paradigm enabling data\-driven intelligent applications in Internet of Things \(IoT\) environments, widely adopted across healthcare, smart homes, and industry due to its cost\-effectiveness\. However, the dynamic nature of IoT frequently alters data distributions, affecting MLaaS stability, while periodic MLaaS updates further introduce performance drift\. Unlike traditional ML systems, MLaaS clients operate as black\-box users without access to internal data or parameters, making drift detection particularly challenging\. To address this, we propose a novel MLaaS Performance Drift Detection framework for IoT environments\. The framework first employs an MLaaS extraction model that learns service behavior from input–output pairs and identifies prediction\-influenced features\. Building on this, the proposed MLaaS Performance Drift Detection \(MPDD\) model jointly captures variations in input data and MLaaS behavior\. We further design an Adaptive\-Temporal Performance Drift Detection Mechanism \(APDDM\) that dynamically adjusts monitoring frequency based on behavioral and data variations, enabling timely drift detection for effective service management\. Extensive experiments on real\-world datasets demonstrate that MPDD achieves up to 22–25% accuracy improvement over baseline drift detection methods\. APDDM provides an average accuracy gain of approximately 4% and reduces the miss detection rate by around 9% compared to fixed\-interval monitoring\.

###### Index Terms:

Machine Learning as a Service, IoT, Performance Drift, Drift Detection, Model Monitoring

## IIntroduction

Machine Learning as a Service \(MLaaS\) is a popular cloud\-based service that provides scalable infrastructure and tools to build, train, and deploy ML models\[[29](https://arxiv.org/html/2608.18555#bib.bib1)\]\. These services eliminate the complexity of managing hardware and manually designing ML models\. MLaaS is typically accessed through web interfaces, application programming interfaces \(APIs\), or software development kits \(SDKs\)\. Major providers such asMicrosoft Azure111[https://azure\.microsoft\.com/en\-us/products/machine\-learning/](https://azure.microsoft.com/en-us/products/machine-learning/),AWS SageMaker222[https://aws\.amazon\.com/sagemaker/](https://aws.amazon.com/sagemaker/), andOpenAI ChatGPT API333[https://platform\.openai\.com/docs/](https://platform.openai.com/docs/)offer comprehensive MLaaS solutions\. MLaaS services are generally categorized asplatform\-based MLaaS \(PMLaaS\)for centralized training and storage \(e\.g\.,AWS SageMaker††footnotemark:\), andinference\-based MLaaS \(IMLaaS\)for local prediction using pretrained models \(e\.g\.,Google Activity Recognition444[https://developers\.google\.com/location\-context/activity\-recognition](https://developers.google.com/location-context/activity-recognition)\)\.

Internet of Things \(IoT\) companies often rely on IMLaaS to add intelligent capabilities without training models in\-house\[[27](https://arxiv.org/html/2608.18555#bib.bib10)\]\. For example,Medtronic, a healthcare IoT provider, integrates IMLaaS fromIBM IQ castinto its application to predict low blood sugar events\[[22](https://arxiv.org/html/2608.18555#bib.bib18)\]\. Clients typically evaluate these services using ground truth data from their environments to assess functional attributes \(e\.g\., model and data specifications\) and quality\-of\-service metrics \(e\.g\., efficiency and scalability\)\. However, the performance of an IMLaaS model is not static and may change over time\. This occurs mainly due to two factors:1\) the dynamic nature of IoT environments, where data patterns evolve\[[12](https://arxiv.org/html/2608.18555#bib.bib23)\], and2\) MLaaS updates, where providers periodically update services beyond client control\[[22](https://arxiv.org/html/2608.18555#bib.bib18)\]\. As a result, clients may experienceperformance driftover time\.

Performance driftrefers to the gradual degradation of the predictive ability of IMLaaS over time, reflected in metrics such as accuracy, precision, or F1 score\[[2](https://arxiv.org/html/2608.18555#bib.bib28)\]\. It occurs when the underlying relationship between input dataXXand output labelsyychanges, a phenomenon known asconcept drift\[[2](https://arxiv.org/html/2608.18555#bib.bib28),[14](https://arxiv.org/html/2608.18555#bib.bib17)\]\. IoT environments are highly dynamic due to evolvinguser behavior, sensor distributions, and system requirements, making it difficult for clients to maintain the reliability of MLaaS services\. For example, an IoT environment using Google’s Activity Recognition API for healthcare monitoring may face reduced accuracy aspatient age, health condition, or mobilityalter sensor data over time, leading to performance drift\. Additionally, MLaaS providers periodically update or retrain their models, which may further contribute to performance drift\. Hence, IoT environments need to collect ground truth periodically to monitor MLaaS performance\. However, collecting ground truth data requires substantialtime and effort, making it costly and often impractical\. The key challenge is thetimely detection of performance driftwithout relying on continuous ground truth collection\. To address this, we propose anMLaaS performance drift detection modelthat operates without requiring continuous ground truth data\.

Traditional ML drift detection methods typically rely on monitoring error\-related measures \(e\.g\., accuracy\)\[[2](https://arxiv.org/html/2608.18555#bib.bib28),[38](https://arxiv.org/html/2608.18555#bib.bib22),[14](https://arxiv.org/html/2608.18555#bib.bib17),[3](https://arxiv.org/html/2608.18555#bib.bib15)\]\. These approaches assume timely access to ground\-truth labels, which is often not feasible in real\-world deployments\. Consequently, label\-free drift detection techniques focus on monitoring internal model parameters, training data characteristics, or input data distributions to identify shifts\. For example, methods such as DRIFTLENS\[[16](https://arxiv.org/html/2608.18555#bib.bib33)\]and Type\-LDD\[[40](https://arxiv.org/html/2608.18555#bib.bib25)\]analyze training data properties and internal model parameters \(e\.g\., weights and gradients\) to infer distributional changes, while other approaches rely solely on observable input data and use statistical tests or discriminative models to detect shifts in input data distributions\[[15](https://arxiv.org/html/2608.18555#bib.bib27),[18](https://arxiv.org/html/2608.18555#bib.bib29),[7](https://arxiv.org/html/2608.18555#bib.bib30)\]\. However, these techniques may not be directly applied in MLaaS settings due to the black\-box nature of MLaaS, where service providers do not expose internal parameters or training data because of security and proprietary constraints\. Moreover, approaches that rely only on changes in input data characteristics mainly capture distributional variation and cannot determine whether such changes actually alter the predictive behavior of the deployed MLaaS service under evolving IoT streams\. As a result, the relationship between changing IoT data and MLaaS output behavior remains unmodeled\. Therefore, IoT environments require an effective mechanism that explicitly captures the interaction between input data evolution and observable MLaaS outputs to accurately identify performance drift and support reliable service management\. To the best of our knowledge, performance drift detection in MLaaS remains an underexplored problem\. We identifytwokey challenges in detecting performance drift in MLaaS for IoT environments:

- •How to detect the MLaaS performance drift without ground truth?:The dynamic nature of IoT environments can cause changes in input data patterns, potentially affecting MLaaS performance\[[4](https://arxiv.org/html/2608.18555#bib.bib19),[19](https://arxiv.org/html/2608.18555#bib.bib4)\]\. IoT environments require continuous collection of ground\-truth data to evaluate these services, which is time\-consuming and often impractical\. Existing studies often analyze statistical properties of input data distributions to detect drifts but distributional changes do not necessarily imply a performance change in MLaaS\[[16](https://arxiv.org/html/2608.18555#bib.bib33)\]\. We consider this phenomenon aspseudo drift\(see Definition 3\)\. In addition, MLaaS providers periodically update or retrain their models, whereas IoT environments lack access to internal training data, feature distributions, or parameter updates\. This highlights the need for mechanisms thatjointly capture input data changes and MLaaS behaviorto distinguish performance drift without relying on ground truth\.
- •How frequently should the MLaaS be re\-evaluated?:The second critical challenge is determining the appropriate time period for monitoring MLaaS\. Since MLaaS performance may degrade due to changes in input data distribution and MLaaS behavior, using fixed or inappropriate intervals can lead tooversensitivity \(short intervals\)orunder\-sensitivity \(long intervals\)in detecting performance drift\. For example, if the MLaaS is evaluated every five minutes, the drift detection technique may continuously generate pseudo\-drift notifications, reflecting an oversensitive setting that incurs unnecessary computational cost and resource consumption\. Conversely, if evaluation occurs only every few hours, the system may fail to capture timely variations and detect real drift too late\. This highlights the need for anadaptive time\-variable mechanismthat can adjust its evaluation period to detect performance drift and support effective service management decisions in IoT environments\.

To address this challenge,we propose an MLaaS performance drift detection frameworkthat identifies performance degradation without continuously collecting ground truth\. The framework is built upon anMLaaS Performance Drift Detection Model \(MPDD\), which determines when the MLaaS should be re\-evaluated to detect potential drift\. First, we design an MLaaS Extraction Model that leverages IoT input data and MLaaS outputs to mimic MLaaS behavior and identify key input features without requiring access to black\-box MLaaS parameters or training data\. Next, we design two complementary scores, theFréchet Data Drift Score \(FDDS\)and theMLaaS\-Aware Drift Exposure Score \(MDES\), to capture changes in input data and their impact on MLaaS behavior\. We then propose theMLaaS Performance Drift Detection Model, which captures the relationship between input data and MLaaS behavior, enabling detection of both real and pseudo drift without relying on ground\-truth labels\. Finally, we propose anAdaptive\-Temporal Performance Drift Detection Mechanism \(APDDM\)that dynamically adjusts the evaluation interval to mitigate oversensitivity and undersensitivity, supporting effective service management decisions in IoT environments\. Our contributions are summarized as follows:

- •We propose an MPDD framework designed for IoT environments that identifies performance degradation in black\-box MLaaS models without relying on ground truth data\.
- •We develop an MLaaS Extraction Model to approximate MLaaS behavior using input–output pairs\.
- •We propose a novel MPDD model that jointly captures the relationship between input data and MLaaS behavior, allowing accurate identification of performance drift\.
- •We propose an APDDM that dynamically adjusts evaluation intervals to balance oversensitivity and undersensitivity\.

## IIPrior Work

### II\-AError\-Rate\-Based Drift Detection

ML services deployed in IoT environments often suffer from performance degradation over time due to changes in the underlying data\-generating process, commonly referred to as concept drift\[[35](https://arxiv.org/html/2608.18555#bib.bib21)\]\. Prior research in IoT data stream learning has proposed several drift detection techniques that mainly rely on observable error\-rate signals\[[2](https://arxiv.org/html/2608.18555#bib.bib28),[38](https://arxiv.org/html/2608.18555#bib.bib22),[13](https://arxiv.org/html/2608.18555#bib.bib3)\]\. Classical error\-monitoring detectors, such as Drift Detection Method \(DDM\) and Early Drift Detection Method \(EDDM\)\[[2](https://arxiv.org/html/2608.18555#bib.bib28)\], along with adaptive extensions including Reactive Drift Detection Method \(RDDM\) and Adaptive Windowing \(ADWIN\)\[[3](https://arxiv.org/html/2608.18555#bib.bib15),[5](https://arxiv.org/html/2608.18555#bib.bib35)\], identify drift by monitoring variations in prediction errors within sliding or adaptive windows\. More advanced variants, including Hoeffding\-based approaches such as HDDM and FHDDM\[[14](https://arxiv.org/html/2608.18555#bib.bib17),[26](https://arxiv.org/html/2608.18555#bib.bib16)\], improve sensitivity through statistical testing based on Hoeffding bounds\[[38](https://arxiv.org/html/2608.18555#bib.bib22)\]\. Recent studies have also explored drift\-aware analytics in IoT streaming environments\. Yang and Shami\[[39](https://arxiv.org/html/2608.18555#bib.bib9)\]proposed a lightweight IoT anomaly detection framework using sliding\-window monitoring and dynamic retraining for evolving sensor and network traffic streams\. Similarly, FedConD\[[8](https://arxiv.org/html/2608.18555#bib.bib2)\]introduced a federated IoT sensor learning framework that detects drift by monitoring historical prediction behavior from distributed local models\. Despite addressing IoT\-oriented streaming scenarios, these approaches still depend on observable prediction errors and continuous access to ground\-truth labels\[[2](https://arxiv.org/html/2608.18555#bib.bib28)\]\. Such assumptions are often impractical in real\-world IoT deployments due to the cost, delay, and difficulty of obtaining reliable labeled feedback from distributed sensor devices\[[30](https://arxiv.org/html/2608.18555#bib.bib31),[9](https://arxiv.org/html/2608.18555#bib.bib20)\]\.

### II\-BBlack\-Box, Label\-Independent Drift Detection Methods

Input data and distribution\-based detection methods focus on analyzing changes in the input data stream rather than output errors\. These techniques rely on statistical divergence measures such as Maximum Mean Discrepancy \(MMD\), Kolmogorov–Smirnov\[[21](https://arxiv.org/html/2608.18555#bib.bib14)\], and Fréchet \(Wasserstein\-2\) distance\[[36](https://arxiv.org/html/2608.18555#bib.bib34)\]to identify distributional shifts\. MMD\-based approaches compare data segments over time but often require large observation windows, limiting responsiveness in black\-box environments\[[31](https://arxiv.org/html/2608.18555#bib.bib36)\]\. Similarly, DRIFTLENS\[[16](https://arxiv.org/html/2608.18555#bib.bib33)\]detects drift using training data representations and internal model parameters for unsupervised latent feature analysis, yet its effectiveness depends on access to model internals and training information, which is typically unavailable in service\-oriented settings\. The Type\-LDD framework\[[40](https://arxiv.org/html/2608.18555#bib.bib25)\]formulates drift detection using a multi\-task sharing\-loss function pre\-trained on synthetic data to capture drift timing and type, but assumes availability of training data and semantic knowledge aligned with the model structure\.

Recent studies have also explored label\-independent drift handling in IoT and edge environments under limited observability\. For example,\[[24](https://arxiv.org/html/2608.18555#bib.bib6)\]detected concept drift by analyzing IoT traffic flow behavior and classification score distributions without requiring true labels, while\[[1](https://arxiv.org/html/2608.18555#bib.bib7)\]combined KSWIN\-based drift detection with unsupervised anomaly monitoring over IoT traffic streams\. Similarly, A\-Detection\[[37](https://arxiv.org/html/2608.18555#bib.bib8)\]identified drift using reliability data streams collected from edge and IoT services\. However, these approaches assume access to historical traffic distributions or reference data that are typically unavailable to clients\. Existing black\-box drift detection methods mainly focus on input\-stream distributional changes rather than behavioral performance variations of deployed services\. D3\[[15](https://arxiv.org/html/2608.18555#bib.bib27)\]detects drift by training a discriminative classifier to separate historical and recent input samples, while SCSD\[[18](https://arxiv.org/html/2608.18555#bib.bib29)\]models drift detection as a sequential classifier two\-sample test to identify covariate shift through calibrated confidence intervals between training and live inputs\. The STUDD framework\[[7](https://arxiv.org/html/2608.18555#bib.bib30)\]adopts a student–teacher paradigm, where drift is inferred by monitoring imitation loss as the student replicates teacher predictions under evolving inputs\. Although these methods operate under limited observability, they primarily capture input\-space variations rather than behavioral performance changes of deployed ML services\. In MLaaS environments, clients only observe input–output interactions without access to training data or internal model states\[[9](https://arxiv.org/html/2608.18555#bib.bib20),[3](https://arxiv.org/html/2608.18555#bib.bib15),[5](https://arxiv.org/html/2608.18555#bib.bib35)\]\. This limitation can lead to pseudo drift, where input distribution changes do not necessarily indicate actual service degradation\. This gap motivates the need for methods that directly relate observable inputs to MLaaS behavioral performance, which we address through aperformance drift detection frameworkdesigned for black\-box MLaaS settings\.

## IIIMotivation Scenario

Let us consider an IoT healthcare environment that aims to integrate Human Activity Recognition \(HAR\) capabilities to monitor and analyze patient activities such as walking, sitting, and sleeping\. However, the environment lacks the expertise and infrastructure required to develop, train, and maintain an HAR model\. Therefore, the environment leverages an inference\-based MLaaS provider to obtain HAR predictions\. Major cloud platforms such as Google Cloud offer pre\-trained activity recognition models that can be deployed without in\-house ML expertise\. In practice, the IoT environment submits collected sensor data to the provider in periodic batches through batch inference channels such asGoogle Cloud Vertex AI Batch Prediction555[https://cloud\.google\.com/vertex\-ai/docs/predictions/batch\-predictions](https://cloud.google.com/vertex-ai/docs/predictions/batch-predictions)orAzure Machine Learning batch endpoints666[https://learn\.microsoft\.com/en\-us/azure/machine\-learning/concept\-endpoints\-batch](https://learn.microsoft.com/en-us/azure/machine-learning/concept-endpoints-batch)\. Fig\.[1](https://arxiv.org/html/2608.18555#S3.F1)illustrates the motivating scenario of MLaaS integration within the IoT\.

Let us consider that the IoT environment is subject to changes influenced by user behavior, device configurations, and environmental factors\. As a result, the current IoT environment may no longer reflect the conditions assumed during the original deployment of the MLaaS service\. This phenomenon may lead to a noticeable degradation in performance reflected in metrics such as accuracy, precision, and recall\. For instance, in healthcare, variations in patient demographics \(e\.g\., pediatric to elderly\), health conditions \(e\.g\., recovery stages or mobility impairments\), or sensor placements \(e\.g\., wrist versus ankle\) can alter sensor data and reduce prediction accuracy from 93% to 68%\. Hence, the IoT environment requires a performance drift detection strategy for effective service management, enabling informed service\-level decision making such as notifying the MLaaS provider or switching to a more suitable service\.

The primary challenge is to detect performance drift in MLaaS without the ground truth\. IoT environments are dynamic and subject to changes influenced by user behavior, device configurations, and environmental factors\. However, a change in input data distribution does not necessarily indicate performance drift, as some MLaaS models may be robust enough to handle such variations, leading to pseudo drift\. Therefore, detecting input changes alone does not solve the problem of identifying real performance drift\. Moreover, MLaaS providers’ behavior must also be considered\. Due to security restrictions, clients lack access to internal parameters or training data, making MLaaS a black\-box model\. This highlights the need for an effective mechanism to capture the underlying relationship between input data and MLaaS to detect real and pseudo drift without relying on ground truth\.

Let us assume that an IoT environment designs a performance drift detection model to monitor MLaaS\. Identifying an appropriate time interval for re\-evaluation is a key challenge, as inappropriate choices can lead tooversensitivity \(short intervals\)orundersensitivity \(long intervals\)in detecting performance drift\. For example, frequent checks \(e\.g\., every minute\) may trigger repeated pseudo\-drift notifications, while infrequent checks \(e\.g\., hourly\) may delay the detection of real drift, allowing true performance degradation to persist unnoticed\. Such inappropriate monitoring frequencies also increase computational and operational costs\. Hence, IoT environments require a time\-variable mechanism that can adaptively adjust the evaluation interval to detect real performance drift accurately and promptly\.

![Refer to caption](https://arxiv.org/html/2608.18555v1/images/mlaas_motivation.png)Fig\. 1:Illustration of the Motivation Scenario
## IVKey Definitions and Problem Statement

Definition 1: MLaaS\.An MLaaS service is represented as a tuple⟨F,QoS⟩\\langle F,\\mathrm\{QoS\}\\rangle, where:

- •FFis the functional specification offered byMM, including model specifications𝒩Φ\\mathcal\{N\}\_\{\\Phi\}\(e\.g\., model typeMtM\_\{t\}\) and data specificationsDsD\_\{s\}, whereDs=\{DV,DF\}D\_\{s\}=\\\{D\_\{V\},D\_\{F\}\\\}denotes data volume \(DVD\_\{V\}\), and data features \(DFD\_\{F\}\)\.
- •QoS\\mathrm\{QoS\}is the non\-functional specification offered byMM, encompassing evaluation metrics \(EfE\_\{f\}, e\.g\., accuracy,R2R^\{2\}, task\-specific scores\), quality factor \(QQ, historical performance trends\), and response time \(RtR\_\{t\}, latency\)\.

Definition 2: Real Drift\.Consider an MLaaS serviceMM\. The stream of input–prediction pairs observed over the interval\[0,t\]\[0,t\]is defined as

St=\{\(X0,Y^0\),\(X1,Y^1\),…,\(Xt,Y^t\)\},S\_\{t\}=\\\{\(X\_\{0\},\\hat\{Y\}\_\{0\}\),\(X\_\{1\},\\hat\{Y\}\_\{1\}\),\\ldots,\(X\_\{t\},\\hat\{Y\}\_\{t\}\)\\\},\(1\)whereXiX\_\{i\}denotes the input instance at timeii, andY^i=M⁡\(Xi\)\\hat\{Y\}\_\{i\}=M\(X\_\{i\}\)represents the prediction generated by the MLaaS\. LetPtM​\(X,Y^\)P\_\{t\}^\{M\}\(X,\\hat\{Y\}\)denote the joint probability distribution of the input data and the corresponding MLaaS predictions over the interval\[0,t\]\[0,t\]\. Areal driftis said to occur at timet\+1t\+1if

PtM​\(X,Y^\)≠Pt\+1M​\(X,Y^\),P\_\{t\}^\{M\}\(X,\\hat\{Y\}\)\\neq P\_\{t\+1\}^\{M\}\(X,\\hat\{Y\}\),\(2\)indicating a fundamental change in the relationship between the input data and the MLaaS predictions, thereby reflecting a degradation in predictive behaviour\.

Definition 3 \(Pseudo Drift\)\.A*pseudo drift*occurs when the input data distribution changes while the underlying predictive mapping remains stable, such that MLaaS performance does not experience sustained degradation\.

PtM​\(X\)≠Pt\+1M​\(X\)andPtM​\(Y^∣X\)≈Pt\+1M​\(Y^∣X\),P\_\{t\}^\{M\}\(X\)\\neq P\_\{t\+1\}^\{M\}\(X\)\\quad\\text\{and\}\\quad P\_\{t\}^\{M\}\(\\hat\{Y\}\\mid X\)\\approx P\_\{t\+1\}^\{M\}\(\\hat\{Y\}\\mid X\),\(3\)wherePtM​\(X\)P\_\{t\}^\{M\}\(X\)denotes the input data distribution at timett, andPtM​\(Y^∣X\)P\_\{t\}^\{M\}\(\\hat\{Y\}\\mid X\)represents the conditional predictive distribution of the MLaaS\. A pseudo drift reflects a temporary fluctuation in input data that does not alter the predictive mapping of the service and therefore does not lead to long\-term performance degradation\.

MLaaS Performance Drift Detection Problem\. The MPDD problem concerns automatically distinguishing betweenreal driftandpseudo driftin an MLaaS service when changes in the operational environment are observed\. To this end, the drift detection model outputs a drift classification \(DC\) and is defined as the following function

MPDD⁡\(M,St\)=\{1,if​PtM​\(X,Y^\)≠Pt\+1M​\(X,Y^\),0,if​PtM​\(X\)≠Pt\+1M​\(X\)\.\\mathrm\{MPDD\}\(M,S\_\{t\}\)=\\begin\{cases\}1,&\\text\{if \}P\_\{t\}^\{M\}\(X,\\hat\{Y\}\)\\neq P\_\{t\+1\}^\{M\}\(X,\\hat\{Y\}\),\\\\ 0,&\\text\{if \}P\_\{t\}^\{M\}\(X\)\\neq P\_\{t\+1\}^\{M\}\(X\)\.\\end\{cases\}\(4\)Here,MMdenotes the MLaaS service in operation, andSt=\{\(Xi,Y^i\)\}i=0tS\_\{t\}=\\\{\(X\_\{i\},\\hat\{Y\}\_\{i\}\)\\\}\_\{i=0\}^\{t\}represents the stream of recent input–prediction pairs up to timett\. A classification ofReal Driftrequires immediate intervention \(e\.g\., model retraining or service reconfiguration\), whereasPseudo Driftindicates that the service can continue operating without corrective action\.

TABLE I:Notation Summary
## VProposed MLaaS Performance Drift Detection Framework for IoT environments

![Refer to caption](https://arxiv.org/html/2608.18555v1/images/mlaas_performance.png)Fig\. 2:Machine Learning as a Service \(MLaaS\) Performance Drift Detection Framework for IoT environmentsIn this section, we present the MLaaS Performance Drift Detection \(MPDD\) framework for identifying performance drift in MLaaS within IoT environments\. The framework comprises five key components, as illustrated in Fig\.[2](https://arxiv.org/html/2608.18555#S5.F2)\. The first component is the edge and database layer, where IoT devices such as smartwatches, fitness trackers, cameras, and smart rings generate real\-time data streams stored in a centralized database\. The second component, the MLaaS layer, integrates a pre\-trained model provided by the MLaaS provider into the IoT edge for local predictions \(see Definition 1\)\. The third component, the MLaaS extraction layer, trains an extraction model to capture latent factors influencing predictions\. The extraction modelM′M^\{\\prime\}is constructed using queried MLaaS input–output interactions collected from the IoT environment and operates within the cloud infrastructure\. In the current implementation, the extraction model is initialized through an offline training phase prior to runtime monitoring\. The performance drift detection module compares input\-data statistics with extraction models–derived features over time to distinguish real and pseudo drift\. Finally, the adaptive temporal performance drift detection mechanism dynamically adjusts the monitoring interval, ensuring reliable drift detection while reducing oversensitivity to minor fluctuations\.

### V\-AMLaaS Extraction model

We propose anMLaaS Extraction Modelto approximate MLaaS behaviour using unlabeled input instances and the corresponding predictions returned by the service, as illustrated inAlgorithm[1](https://arxiv.org/html/2608.18555#alg1)\. We assume that the IoT provider maintains a historical repository of unlabeled inputs collected during normal system operation\. Accordingly, Phase 1 \(Input–Output Query Log Collection\) constructs the input–output query log from these historical inputs for offline training of the MLaaS Extraction Model\. Once trained, the proposed framework requires no additional MLaaS queries beyond the application’s normal inference requests\. Traditional model\-extraction approaches often rely on prediction probabilities, which can be affected by protection mechanisms such as output rounding, quantization, and probability perturbation\[[33](https://arxiv.org/html/2608.18555#bib.bib39),[32](https://arxiv.org/html/2608.18555#bib.bib40)\]\.To mitigate the impact of these protection mechanisms, the proposed MLaaS Extraction Model relies exclusively on predicted class labels\. The resulting input–label pairs capture the observable decision behaviour of the MLaaS and are used to train the extraction model\. Since surrogate\-based methods \(e\.g\.,SHAP\) incur high computational overhead in MLaaS settings, Phase 2 \(Extraction Model Training\) trains a lightweight decision\-tree modelM′M^\{\\prime\}using the collected query log to approximate the observable input–output behaviour \(lines 8–9\)\. In our implementation,M′M^\{\\prime\}is aCART decision\-tree classifiertrained using the Gini impurity criterion and best\-split strategy\. The maximum tree depth is set to four, with a minimum split size of two samples and a minimum leaf size of one sample\. The model is trained using MLaaS\-predicted labels from the query log\. To capture both stable and drifted operating conditions, the query pool consists of 60% clean samples and 40% drift samples, with drift data uniformly collected from sudden, gradual, incremental, and recurrent drift scenarios\. The detailed configuration and training setup ofM′M^\{\\prime\}are summarized in Table[II](https://arxiv.org/html/2608.18555#S7.T2)\. In Phase 3 \(Feature\-Importance Extraction\), the trained model perturbs each input feature individually and measures the output deviation to compute feature\-importance scores as the average deviation across samples \(lines 10–17\)\. Finally, the algorithm returns the aggregated feature\-importance vector \(line 18\), which forms the basis for subsequent performance drift analysis\.

Lemma 1LetMMbe the black\-box MLaaS service andM′M^\{\\prime\}the extraction model\. Assume thatM′M^\{\\prime\}satisfies a bounded fidelity condition such that𝔼⁡\[\|M⁡\(X\)−M′​\(X\)\|\]≤ϵ\.\\mathbb\{E\}\[\|M\(X\)\-M^\{\\prime\}\(X\)\|\]\\leq\\epsilon\.Then, the feature importance vector𝐈M′\\mathbf\{I\}^\{M^\{\\prime\}\}extracted fromM′M^\{\\prime\}consistently approximates the true behavioural importance𝐈M\\mathbf\{I\}^\{M\}ofMM, and the deviation remains bounded as

‖𝐈M′−𝐈M‖≤f⁡\(ϵ,δ\),\\\|\\mathbf\{I\}^\{M^\{\\prime\}\}\-\\mathbf\{I\}^\{M\}\\\|\\leq f\(\\epsilon,\\delta\),whereϵ\\epsilondenotes the fidelity error andδ\\deltadenotes the perturbation sensitivity in the importance estimation process\. Proof\.Please refer toAppendix A\.

Algorithm 1MLaaS Extraction Model1:Input:Unlabeled inputs

X=\{X\(1\),…,X\(N\)\}X=\\\{X^\{\(1\)\},\\dots,X^\{\(N\)\}\\\}, black\-box MLaaS

M⁡\(⋅\)M\(\\cdot\)
2:Output:Feature\-importance vector

I=\{I⁡\(x1\),…,I⁡\(xd\)\}I=\\\{I\(x\_\{1\}\),\\dots,I\(x\_\{d\}\)\\\}
3:Phase 1: Input–Output Query Log Collection

4:Initialize

𝒟←∅\\mathcal\{D\}\\leftarrow\\emptyset
5:for

j←1j\\leftarrow 1to

NNdo

6:

y^\(j\)←M⁡\(X\(j\)\)\\hat\{y\}^\{\(j\)\}\\leftarrow M\\\!\\left\(X^\{\(j\)\}\\right\)
7:

𝒟←𝒟∪\{\(X\(j\),y^\(j\)\)\}\\mathcal\{D\}\\leftarrow\\mathcal\{D\}\\cup\\\{\(X^\{\(j\)\},\\hat\{y\}^\{\(j\)\}\)\\\}
8:endfor

9:Phase 2: Train Extraction ModelM′M^\{\\prime\}

10:Initialize

M′M^\{\\prime\}as a lightweight decision\-tree model

11:Train

M′M^\{\\prime\}on

𝒟\\mathcal\{D\}to approximate the mapping

X→y^X\\rightarrow\\hat\{y\}
12:Phase 3: Feature Importance Extraction

13:for

i←1i\\leftarrow 1to

dddo

14:

I⁡\(xi\)←0I\(x\_\{i\}\)\\leftarrow 0
15:for

j←1j\\leftarrow 1to

NNdo

16:

X\+i\(j\)←\(x1\(j\),…,xi\(j\)\+δ,…,xd\(j\)\)X^\{\(j\)\}\_\{\+i\}\\leftarrow\(x^\{\(j\)\}\_\{1\},\\dots,x^\{\(j\)\}\_\{i\}\+\\delta,\\dots,x^\{\(j\)\}\_\{d\}\)
17:

I⁡\(xi\)←I⁡\(xi\)\+‖M′​\(X\+i\(j\)\)−M′​\(X\(j\)\)‖1I\(x\_\{i\}\)\\leftarrow I\(x\_\{i\}\)\+\\left\\\|M^\{\\prime\}\(X^\{\(j\)\}\_\{\+i\}\)\-M^\{\\prime\}\(X^\{\(j\)\}\)\\right\\\|\_\{1\}
18:endfor

19:

I⁡\(xi\)←1N​I​\(xi\)I\(x\_\{i\}\)\\leftarrow\\frac\{1\}\{N\}I\(x\_\{i\}\)
20:endfor

21:return

II

### V\-BFréchet Data Drift Score \(FDDS\)

Fréchet distance\(also known as the Wasserstein\-2 distance\), has been widely used to measure the distance between distributions of models’ features in deep learning studies\[[11](https://arxiv.org/html/2608.18555#bib.bib37)\]\. In our context, we use it to measure the statistical difference between the historical \(baseline\) data used to select the MLaaS and the new data windows in the stream, and we call this metric theFréchet Data Drift Score \(FDDS\)\. The proposed framework incorporates a preprocessing and feature transformation layer prior to FDDS computation\. This layer transforms categorical and raw event\-log data into structured numerical feature representations, enabling FDDS to operate on the transformed feature space\. Given a multivariate normal baseline distributionbbcharacterized by mean vectorμb\\mu\_\{b\}and covariance matrixΣb\\Sigma\_\{b\}, and a new data windowwwwith mean vectorμw\\mu\_\{w\}and covariance matrixΣw\\Sigma\_\{w\}, the FDDS is computed as

FDDS​\(bi,wi\)=∥μb−μw∥22⏟Mean Shift\+Tr⁡\(Σb\+Σw−2​Σb​Σw\)⏟Covariance Shift,\\text\{FDDS\}\(b\_\{i\},w\_\{i\}\)=\\underbrace\{\\lVert\\mu\_\{b\}\-\\mu\_\{w\}\\rVert\_\{2\}^\{2\}\}\_\{\\text\{Mean Shift\}\}\+\\underbrace\{\\operatorname\{Tr\}\\\!\\left\(\\Sigma\_\{b\}\+\\Sigma\_\{w\}\-2\\sqrt\{\\Sigma\_\{b\}\\Sigma\_\{w\}\}\\right\)\}\_\{\\text\{Covariance Shift\}\},\(5\)where∥μb−μw∥22\\lVert\\mu\_\{b\}\-\\mu\_\{w\}\\rVert\_\{2\}^\{2\}measures the squared difference between the mean vectors \(shift in center\), and the trace term measures the difference in the covariance structure \(spread\) between the two distributions\. A higher FDDS indicates a larger divergence and a higher likelihood of data drift\. Lemma 2Letbbandwwdenote the baseline and the current data window as defined in the FDDS formulation in Eq\. \(2\)\. Since FDDS decomposes the statistical difference into mean shift and covariance shift, and the covariance shift term is non\-negative, FDDS is lower bounded by the mean shift, i\.e\.,

FDDS⁡\(bi,wi\)≥‖μb−μw‖22\.\\mathrm\{FDDS\}\(b\_\{i\},w\_\{i\}\)\\geq\\\|\\mu\_\{b\}\-\\mu\_\{w\}\\\|\_\{2\}^\{2\}\.When the covariance structures are approximately unchanged, FDDS remains stable under small mean perturbations\. Conversely, larger mean and/or covariance changes yield higher FDDS\. Thus, FDDS stays low under minor deviations and increases under significant drift, ensuring effective drift differentiation between the baseline batch and the current window\. Proof\.Please refer toAppendix A\.

### V\-CMLaaS\-Aware Drift Exposure Score \(MDES\)

MDESis designed to capture the relationship between changes in input data and MLaaS behavior\. MDES quantifies the degree of exposure of an MLaaS service to performance drift by combining theMLaaS extraction feature importancewith theFDDS\. This integration allows the framework to capture changes in input data characteristics and variations in MLaaS behavior inferred through the MLaaS extraction model\. Unlike traditional drift detectors that rely solely on distributional change, MDES considers how these changes interact with the influential features identified by the MLaaS extraction model, providing a deeper understanding of drift causes\. Formally, let𝐈=\{I⁡\(x1\),I⁡\(x2\),…,I⁡\(xd\)\}\\mathbf\{I\}=\\\{I\(x\_\{1\}\),I\(x\_\{2\}\),\\ldots,I\(x\_\{d\}\)\\\}represent the importance weights obtained from the MLaaS extraction model forddinput features, and letFDDS⁡\(b,w\)\\mathrm\{FDDS\}\(b,w\)denote the Fréchet Data Drift Score between the baseline windowbband the current windowww\. TheMDESis defined as

MDES=\{1,if​\(∑i=1dI⁡\(xi\)​FDDS⁡\(bi,wi\)\)\>θm,0,otherwise,\\text\{MDES\}=\\begin\{cases\}1,&\\text\{if \}\\left\(\\sum\_\{i=1\}^\{d\}I\(x\_\{i\}\)\\,\\operatorname\{FDDS\}\(b\_\{i\},w\_\{i\}\)\\right\)\>\\theta\_\{m\},\\\\\[2\.0pt\] 0,&\\text\{otherwise\},\\end\{cases\}\(6\)Here,θm\\theta\_\{m\}is an automated drift\-exposure threshold derived from the mean \(μ\\mu\) and standard deviation \(σ\\sigma\) of the drift\-exposure scores, defined asθm=μ\+β​σ\\theta\_\{m\}=\\mu\+\\beta\\sigma\. The sensitivity coefficientβ\\betacontrols the threshold level and is empirically determined\. When the weighted combination of feature importance and data drift exceedsθm\\theta\_\{m\}, the MLaaS service is classified as experiencingreal drift\. Changes that do not exceedθm\\theta\_\{m\}are classified aspseudo drift

Lemma 3Letbbbe the baseline window andwwthe current window, and letFDDS⁡\(b,w\)\\mathrm\{FDDS\}\(b,w\)denote the statistical shift between them\. Let𝐈=\{I⁡\(x1\),…,I⁡\(xd\)\}\\mathbf\{I\}=\\\{I\(x\_\{1\}\),\\ldots,I\(x\_\{d\}\)\\\}be the bounded feature\-importance vector obtained from the MLaaS extraction model, and let MDES be defined as in Eq\. \(3\)\. If the statistical shift captured byFDDS⁡\(b,w\)\\mathrm\{FDDS\}\(b,w\)is primarily concentrated on features with low importance weights, then the cumulative weighted exposure remains bounded and the drift is characterized as pseudo drift\. Conversely, if the shift aligns with highly influential features, the weighted exposure increases proportionally and the drift is characterized as real drift\.Proof\.Please refer toAppendix A\.

Algorithm 2MLaaS Performance Drift Detection Model1:Input:Input data

\(X\)\(X\), baseline statistics

bb
2:Output:MLaaS\-Aware Drift Exposure Score

SMDESS\_\{\\text\{MDES\}\}
3:Step 1: MLaaS Extraction Model Training

4:Train lightweight extraction model

M′M^\{\\prime\}to approximate

MM:

5:

M′​\(X\)≈M⁡\(X\)M^\{\\prime\}\(X\)\\approx M\(X\)
6:Compute feature importance scores:

7:

I⁡\(xi\)=1N​∑j=1N\|M′​\(x1\(j\),…,xi\(j\)\+δ,…,xd\(j\)\)−M′​\(X\(j\)\)\|I\(x\_\{i\}\)=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}\|M^\{\\prime\}\(x\_\{1\}^\{\(j\)\},\\dots,x\_\{i\}^\{\(j\)\}\+\\delta,\\dots,x\_\{d\}^\{\(j\)\}\)\-M^\{\\prime\}\(X^\{\(j\)\}\)\|
8:Form importance vector

𝐈=\{I⁡\(x1\),I⁡\(x2\),…,I⁡\(xd\)\}\\mathbf\{I\}=\\\{I\(x\_\{1\}\),I\(x\_\{2\}\),\\dots,I\(x\_\{d\}\)\\\}
9:Step 2: Compute Fréchet Data Drift Score

10:

μb,Σb←Stats​\(Xb\)\\mu\_\{b\},\\Sigma\_\{b\}\\leftarrow\\text\{Stats\}\(X\_\{b\}\),

μw,Σw←Stats​\(Xw\)\\mu\_\{w\},\\Sigma\_\{w\}\\leftarrow\\text\{Stats\}\(X\_\{w\}\)
11:

SFDDS=∥μb−μw∥22\+Tr⁡\(Σb\+Σw−2​Σb​Σw\)S\_\{\\text\{FDDS\}\}=\\lVert\\mu\_\{b\}\-\\mu\_\{w\}\\rVert\_\{2\}^\{2\}\+\\mathrm\{Tr\}\\\!\\bigl\(\\Sigma\_\{b\}\+\\Sigma\_\{w\}\-2\\sqrt\{\\Sigma\_\{b\}\\Sigma\_\{w\}\}\\bigr\)
12:Step 3: Compute MLaaS\-Aware Drift Exposure Score

13:Compute feature\-weighted exposure:

14:

Sweighted=∑i=1dI⁡\(xi\)×SFDDSS\_\{\\text\{weighted\}\}=\\sum\_\{i=1\}^\{d\}I\(x\_\{i\}\)\\times S\_\{\\text\{FDDS\}\}
15:Compute automated threshold:

16:

θm=μ\+β​σ\\theta\_\{m\}=\\mu\+\\beta\\sigma
17:if

Sweighted\>θmS\_\{\\text\{weighted\}\}\>\\theta\_\{m\}then

18:

SMDES←1S\_\{\\text\{MDES\}\}\\leftarrow 1⊳\\trianglerightReal drift

19:else

20:

SMDES←0S\_\{\\text\{MDES\}\}\\leftarrow 0⊳\\trianglerightPseudo drift

21:endif

### V\-DMLaaS Performance Drift Detection Model

We propose a novel MPDD model to detect real and pseudo drift, specifically designed for IoT environments\. It explicitly addresses two critical challenges: \(i\) detecting changes in dynamic input data properties, and \(ii\) monitoring concurrent changes in MLaaS behavior\.Algorithm[2](https://arxiv.org/html/2608.18555#alg2)describes the MLaaS Performance Drift Detection process\. The inputs are the input dataXwX\_\{w\}, the baseline windowbb, and the current windowww\. The output is the MLaaS\-Aware Drift Exposure ScoreSMDESS\_\{\\text\{MDES\}\}\(Algorithm[2](https://arxiv.org/html/2608.18555#alg2), Lines 1–2\)\. The algorithm trains the MLaaS extraction model \(M′M^\{\\prime\}\) on input–output pairs\(X,Y\)\(X,Y\)to approximate the underlying MLaaS model\. This enables changes in input data to be mapped to shifts in the model’s decision patterns\. It then computes feature\-importance scoresI⁡\(xi\)I\(x\_\{i\}\)for each input characteristic and collects them as𝐈=\{I⁡\(x1\),…,I⁡\(xd\)\}\\mathbf\{I\}=\\\{I\(x\_\{1\}\),\\ldots,I\(x\_\{d\}\)\\\}\(Algorithm[2](https://arxiv.org/html/2608.18555#alg2), Lines 3–6\)\. Next, the algorithm computes the Fréchet Data Drift Score \(FDDS\) to compare the current input statistics with the baseline statistics\. This produces a single score that quantifies the distributional change \(Algorithm[2](https://arxiv.org/html/2608.18555#alg2), Line 7\)\. The MLaaS\-Aware Drift Exposure Score \(MDES\) is then evaluated using Eq\.[6](https://arxiv.org/html/2608.18555#S5.E6), which combines𝐈\\mathbf\{I\}withFDDS\\mathrm\{FDDS\}to weight distributional shifts by feature influence \(Algorithm[2](https://arxiv.org/html/2608.18555#alg2), Lines 8–10\)\. Finally, the score is compared against a user\-defined thresholdθm\\theta\_\{m\}to classify the event as real drift \(SMDES=1S\_\{\\text\{MDES\}\}=1\) or pseudo drift \(SMDES=0S\_\{\\text\{MDES\}\}=0\) \(Algorithm[2](https://arxiv.org/html/2608.18555#alg2), Lines 10–15\), and the procedure returnsSMDESS\_\{\\text\{MDES\}\}\.

### V\-EAdaptive\-Temporal Performance Drift Detection Mechanism \(APDDM\)

We propose an APDDM to adaptively adjust the window sizeWsW\_\{s\}based on recent drift patterns, ensuring stable performance\-drift detection in MLaaS environments\. The algorithm takes as inputs a stream of blocks\{Xt\}\\\{X\_\{t\}\\\}, buffer lengthbb, initial window sizeWsW\_\{s\}, drift ratio thresholdθr\\theta\_\{r\}, trend lengthLL, adaptation stepΔ​W\\Delta W, and window bounds\[Wmin,Wmax\]\[W\_\{\\min\},W\_\{\\max\}\]\(Algorithm[3](https://arxiv.org/html/2608.18555#alg3), line 1\), and outputs the detected drift triggers and final adapted window size \(Algorithm[3](https://arxiv.org/html/2608.18555#alg3), line 2\)\. The procedure initializes a bufferℬ\\mathcal\{B\}to store recent drift scores along with two counterscuc\_\{u\}andcoc\_\{o\}that track consecutive drift and non\-drift windows \(Algorithm[3](https://arxiv.org/html/2608.18555#alg3), lines 3–4\)\. For each incoming blockXtX\_\{t\}, the MPDD model computes a drift scoremtm\_\{t\}and appends it to the buffer while retaining only the most recent observations \(Algorithm[3](https://arxiv.org/html/2608.18555#alg3), lines 4–6\)\. Once the buffer reaches capacity, the algorithm evaluates the proportion of scores exceeding the threshold to classify the current window as drift or stable \(Algorithm[3](https://arxiv.org/html/2608.18555#alg3), lines 7–8\)\. Accordingly,cuc\_\{u\}is incremented andcoc\_\{o\}reset under drift conditions, whereas stable states incrementcoc\_\{o\}and resetcuc\_\{u\}\(Algorithm[3](https://arxiv.org/html/2608.18555#alg3), lines 9–12\), capturing short\-term temporal trends in detection outcomes\. If drift persists for at leastLLconsecutive windows, the method interprets this as under\-sensitivity and reducesWsW\_\{s\}byΔ​W\\Delta Wwithin the allowable bounds \(Algorithm[3](https://arxiv.org/html/2608.18555#alg3), lines 14–15\)\. Conversely, prolonged stability forLLwindows indicates over\-sensitivity, leading to an increase in the window size to avoid unnecessary evaluations \(Algorithm[3](https://arxiv.org/html/2608.18555#alg3), lines 17–18\)\. After processing all blocks, the algorithm returns the final adapted window size \(Algorithm[3](https://arxiv.org/html/2608.18555#alg3), line 22\)\. Through this feedback\-driven strategy, APDDM balances responsiveness and stability, reducing false alarms while ensuring timely detection of performance drift\.

Algorithm 3Adaptive\-Temporal Performance Drift Detection Mechanism \(APDDM\)1:Input:

⟨\{Xt\},b,Ws,θr,L,Δ​W,Wm​i​n,Wm​a​x⟩\\langle\\\{X\_\{t\}\\\},b,W\_\{s\},\\theta\_\{r\},L,\\Delta W,W\_\{min\},W\_\{max\}\\rangle
2:Output:drift triggers,

WsW\_\{s\}
3:

ℬ←\[\]\\mathcal\{B\}\\leftarrow\[\\ \],

cu←0c\_\{u\}\\leftarrow 0,

co←0c\_\{o\}\\leftarrow 0
4:foreach block

XtX\_\{t\}do

5:

mt←MPDD​\(Xt,b\)m\_\{t\}\\leftarrow\\textsc\{MPDD\}\(X\_\{t\},b\)⊳\\trianglerightmt∈\{0,1\}m\_\{t\}\\in\\\{0,1\\\}, where 1 indicates real drift

6:append

mtm\_\{t\}to

ℬ\\mathcal\{B\}; if

\|ℬ\|\>Ws\|\\mathcal\{B\}\|\>W\_\{s\}, remove oldest

7:if

\|ℬ\|=Ws\|\\mathcal\{B\}\|=W\_\{s\}then

8:

r←1Ws​∑m∈ℬmr\\leftarrow\\frac\{1\}\{W\_\{s\}\}\\sum\_\{m\\in\\mathcal\{B\}\}m⊳\\trianglerightdrift ratio in recent window

9:if

r\>θrr\>\\theta\_\{r\}then

10:trigger drift;

cu←cu\+1c\_\{u\}\\leftarrow c\_\{u\}\+1;

co←0c\_\{o\}\\leftarrow 0
11:else

12:

co←co\+1c\_\{o\}\\leftarrow c\_\{o\}\+1;

cu←0c\_\{u\}\\leftarrow 0
13:endif

14:if

cu≥Lc\_\{u\}\\geq Lthen

15:

Ws←max⁡\(Wm​i​n,Ws−Δ​W\)W\_\{s\}\\leftarrow\\max\(W\_\{min\},\\,W\_\{s\}\-\\Delta W\);

cu←0c\_\{u\}\\leftarrow 0
16:endif

17:if

co≥Lc\_\{o\}\\geq Lthen

18:

Ws←min⁡\(Wm​a​x,Ws\+Δ​W\)W\_\{s\}\\leftarrow\\min\(W\_\{max\},\\,W\_\{s\}\+\\Delta W\);

co←0c\_\{o\}\\leftarrow 0
19:endif

20:endif

21:endfor

22:returndrift triggers,

WsW\_\{s\}

## VIComputational complexity analysis

The computational complexity of Algorithm 3 \(APDDM\) is primarily governed by the per\-block drift score computation and the adaptive window management operations\. For each incoming data blockXtX\_\{t\}of sizennwithddfeatures, the algorithm invokes the MPDD scoring function once and then performs lightweight buffer updates and ratio evaluation\. Assuming the extraction model and baseline statistics are precomputed offline \(i\.e\., no online retraining\), the online MPDD computation involves feature\-wise statistical comparison and weighted aggregation, requiring a single pass over the block and yielding a time complexity ofO⁡\(n​d\)O\(nd\)under a diagonal/statistical representation, or up toO⁡\(n​d2\)O\(nd^\{2\}\)if full covariance\-based distance measures are employed\. The APDDM control logic consists of inserting the drift score into a bounded buffer of sizebb, computing the drift ratio over the buffer, and updating counters and window size, resulting inO⁡\(Ws\)O\(W\_\{s\}\)per block \(orO⁡\(1\)O\(1\)if a running count is maintained\)\. Therefore, the overall per\-block complexity of the algorithm can be expressed asO⁡\(n​d\+Ws\)O\(nd\+W\_\{s\}\), and overTTprocessed blocks, the total time complexity of the proposed adaptive drift prediction mechanism is given by

𝒯⁡\(T\)=O⁡\(∑t=1T\(n​d\+Ws​\(t\)\)\)\\mathcal\{T\}\(T\)=O\\\!\\left\(\\sum\_\{t=1\}^\{T\}\\left\(nd\+W\_\{s\}\(t\)\\right\)\\right\)\(7\)which simplifies toO⁡\(T​n​d\)O\(Tnd\)in typical streaming settings whereWs≪nW\_\{s\}\\ll n\. The space complexity is modest, requiringO⁡\(Ws\)O\(W\_\{s\}\)memory to store the adaptive drift\-score buffer andO⁡\(d\)O\(d\)\(orO⁡\(d2\)O\(d^\{2\}\)for full covariance\) for maintaining baseline statistical summaries, making the algorithm computationally efficient and scalable for continuous MLaaS monitoring scenarios without incurring any retraining overhead\.

## VIIExperiments and Results

In this section, we evaluate the proposed MPDD for IoT environments\. First, we assess the effectiveness of the MPDD model in distinguishing real drift from pseudo drift by comparing it with representative baseline techniques, including ADWIN, MMD, D3, STUDD, and SCSD, using accuracy, precision, and recall\. Second, we evaluate the timeliness of the proposed APDDM across multiple datasets\. This experiment investigates the effectiveness of adaptive temporal monitoring in improving drift responsiveness compared with fixed\-interval monitoring and baseline drift detectors\. Performance is evaluated using detection accuracy, accuracy gain, miss detection ratio, and false negatives\. Third, we evaluate the MLaaS extraction model by analyzing its fidelity and predictive behavior relative to the original black\-box MLaaS models\. In addition, we assess the impact of surrogate approximation on the overall detection performance\. The MLaaS servies are extracted from the MDA data generation framework\[[20](https://arxiv.org/html/2608.18555#bib.bib5)\]\. All experiments were conducted on an Intel Core i7 machine with 16 GB RAM using Python\. The source code is publicly available in the repository777[https://anonymous\.4open\.science/r/MPDD\_IoT\-6D16/README](https://anonymous.4open.science/r/MPDD_IoT-6D16/README)\.

### VII\-AExperiment Setup and Dataset

In this section, we discuss the datasets, drift setup, and baseline techniques used to evaluate the proposed framework\.

#### VII\-A1Dataset

- •Human Activity Recognition \(HAR\)\[[28](https://arxiv.org/html/2608.18555#bib.bib24)\]:We use the PAMAP2 dataset containing 3,850,505 samples across 52 sensor channels collected from multiple daily activities\.
- •Electricity\[[34](https://arxiv.org/html/2608.18555#bib.bib13)\]:The Electricity dataset has 45,325 samples with nine features from the New South Wales electricity market, capturing price changes every five minutes\.
- •Weather\[[23](https://arxiv.org/html/2608.18555#bib.bib12)\]:The NOAA Weather dataset includes 96,454 records with nine features describing weather attributes\.
- •Airline\[[10](https://arxiv.org/html/2608.18555#bib.bib26)\]:The Airline Delay dataset contains 539,395 instances with 8 features, including arrival and departure records of U\.S\. commercial flights\.
- •Poker\[[6](https://arxiv.org/html/2608.18555#bib.bib32)\]:The Poker dataset consists of 1,000,000 samples with 11 attributes, where each instance represents a hand of five cards encoded by rank and suit\.

#### VII\-A2Drift Data Generation

Traditional studies simulate different forms of concept drift, including sudden, gradual, incremental, and recurrent changes in data streams\. To comprehensively evaluate the proposed framework, we generate representative drift scenarios covering all four drift categories using the widely adoptedSEA generator\[[25](https://arxiv.org/html/2608.18555#bib.bib11)\]\. The generated scenarios introduce controlled changes in the data distribution, allowing the effectiveness of the framework to be assessed under diverse drift behaviors without being restricted to a specific drift type\. Examples of the generated drift scenarios across datasets are provided inAppendix B\.

#### VII\-A3Baseline Techniques

Existing literature does not explicitly address the challenge of MLaaS performance drift, particularly in black\-box IoT settings\. However, several related drift detection methods partially capture distributional or model\-related changes and can serve as reasonable baselines for comparison\. To ensure a fair and consistent evaluation, we compare the proposed MPDD framework with established benchmark techniques, including ADWIN\[[5](https://arxiv.org/html/2608.18555#bib.bib35)\]and MMD\[[31](https://arxiv.org/html/2608.18555#bib.bib36)\], which represent error\-based and statistical data\-driven drift detection approaches\. In addition, we include representative black\-box unsupervised methods that align with realistic MLaaS deployment constraints, namely STUDD\[[7](https://arxiv.org/html/2608.18555#bib.bib30)\], SCSD\[[18](https://arxiv.org/html/2608.18555#bib.bib29)\], and D3\[[15](https://arxiv.org/html/2608.18555#bib.bib27)\]\. All baseline models are implemented using their recommended drift thresholds and evaluated consistently across the selected datasets\.

#### VII\-A4Configuration of the MLaaS Extraction Model

The MLaaS extraction model \(M′M^\{\\prime\}\) plays a key role in the proposed framework by approximating the behaviour of the black\-box MLaaS service\. This model enables the extraction of feature\-importance information used for performance drift detection\. Table[II](https://arxiv.org/html/2608.18555#S7.T2)summarizes the architecture and parameter configuration of the extraction model, including the decision\-tree structure, splitting criterion, training target, and model settings\. Table[III](https://arxiv.org/html/2608.18555#S7.T3)presents the training\-data configuration used to constructM′M^\{\\prime\}, including the query\-pool size, clean and drift sample composition, and train–test split across datasets\. These details provide additional transparency regarding the design and training process of the extraction model\.

TABLE II:Configuration of the MLaaS extraction modelM′M^\{\\prime\}\.ParameterConfigurationParameterConfigurationModelCART decision treeCriterionGini impuritySplit strategyBest splitMaximum depth4Min samples split2Min samples leaf1Maximum featuresAll variablesTraining targetMLaaS labelsTrain/Test split50% / 50%Training data60% clean \+ 40% drift

TABLE III:Training data configuration of the MLaaS extraction modelTABLE IV:Comparison of MLaaS Performance Drift Detection \(MPDD\) under pseudo drift and real drift across datasets\.\(a\)

\(b\)

\(c\)

\(d\)

\(e\)

\(f\)

Fig\. 3:Comparative accuracy analysis of drift detection techniques across datasets: \(a\) HAR, \(b\) Electricity, \(c\) Weather, \(d\) Poker, \(e\) Airline, and \(f\) threshold sensitivity analysis, evaluated using the proposed MPDD model\.

### VII\-BExperiment 1: Evaluation of MLaaS Performance Drift Detection approach

The proposed MPDD model aims to detect real and pseudo drift in IoT environments\. To evaluate its effectiveness, we use standard metrics including accuracy, precision, and recall\.Table[IV](https://arxiv.org/html/2608.18555#S7.T4)compares MPDD against benchmark drift detection techniques, including ADWIN, MMD, STUDD, SCSD, and D3, across multiple datasets\. Identifying an appropriate threshold to distinguish real and pseudo drift is a critical challenge in drift detection\. Since trial\-and\-error threshold selection may not generalize well across different IoT environments, we adopt an automated threshold adjustment mechanism inspired by DDM and EDDM, where the drift\-exposure threshold \(θm\\theta\_\{m\}\) is automatically determined from the observed drift\-exposure scores\. Table[IV](https://arxiv.org/html/2608.18555#S7.T4)shows that MPDD consistently achieves the highest overall detection accuracy across all datasets while maintaining balanced precision and recall for both real and pseudo drift scenarios\. For pseudo drift, MPDD achieves accuracy values of 0\.91 on HAR, 0\.90 on Electricity, 0\.89 on Weather, 0\.88 on Poker, and 0\.90 on Airline, with precision ranging from 0\.85 to 0\.89 and recall values consistently above 0\.94\. More importantly, MPDD maintains strong performance under real drift conditions, where competing approaches often degrade substantially\. Across all datasets, it achieves precision values of up to 0\.99 and recall values between 0\.69 and 0\.75, demonstrating reliable identification of genuine MLaaS performance degradation\. In contrast, traditional data\-distribution\-based methods such as ADWIN and MMD show lower and less balanced performance\. While they achieve moderate pseudo\-drift detection in some datasets, their real\-drift recall ranges from only 0\.34 to 0\.69\. This suggests that methods relying primarily on data distribution changes may fail when variations in the input data do not directly affect MLaaS prediction behaviour, indicating that data drift does not always translate to performance drift\. Black\-box unsupervised approaches such as D3, STUDD, and SCSD exhibit mixed performance\. D3 achieves competitive results on some datasets, such as Electricity with a real\-drift recall of 0\.82, but shows severe precision\-recall imbalance on others, including HAR where pseudo\-drift recall falls to 0\.10\. Similarly, STUDD achieves high pseudo\-drift recall across all datasets \(0\.95–0\.99\) but performs poorly for real drift detection, with recall values dropping to as low as 0\.04–0\.10\. SCSD achieves high precision or recall for specific drift classes, but suffers from substantial precision\-recall imbalance, resulting in inconsistent detection performance\. Overall, these results confirm the robustness and generalization of MPDD in detecting both pseudo and real drift while maintaining a more balanced performance than existing data\-based and black\-box unsupervised drift detection approaches\.

Figure[3](https://arxiv.org/html/2608.18555#S7.F3)compares the detection accuracy of MPDD and baseline methods across all datasets\. Figures[3](https://arxiv.org/html/2608.18555#S7.F3)\(a\)–\(e\) show that MPDD consistently achieves the highest accuracy on HAR, Electricity, Weather, Poker, and Airline\. The performance gap between MPDD and the strongest competing method remains substantial across all datasets, highlighting the effectiveness of incorporating both feature\-importance variations and data\-drift characteristics into the drift\-detection process\. These results further demonstrate that MPDD maintains robust and consistent performance across diverse IoT environments and drift scenarios\. Figure[3](https://arxiv.org/html/2608.18555#S7.F3)\(f\) evaluates the efficiency of the automated threshold\-selection mechanism\. The results show that the proposed approach determines suitable drift\-exposure thresholds significantly faster than the trial\-and\-error method, achieving an approximately threefold reduction in threshold\-selection time\. In addition to reducing computational overhead, the automated mechanism eliminates the need for dataset\-specific threshold tuning for new IoT environments\.

### VII\-CExperiment 2: Timeliness Evaluation of Adaptive\-Temporal Performance Drift Detection Mechanism \(APDDM\)

\(a\)

\(b\)

\(c\)

\(d\)

\(e\)

Fig\. 4:Timeliness evaluation of APDDM across datasets: \(a\) HAR, \(b\) Electricity, \(c\) Weather, \(d\) Airline, and \(e\) Poker, showing accuracy comparisons for each scenario\.Fig\. 5:False negative count comparison across the HAR, Airline, Electricity, Weather, and Poker datasets\.Figure[4](https://arxiv.org/html/2608.18555#S7.F4)presents the temporal accuracy comparison between MPDD and APDDM across multiple datasets\. Each row corresponds to a dataset and shows accuracy over evaluation chunks \(approximately 250 samples per chunk\)\. The shaded green regions \(AG\) represent the accuracy gain achieved by APDDM over MPDD, illustrating the benefit of adaptive window adjustment under changing drift conditions\. As shown inFig\.[4](https://arxiv.org/html/2608.18555#S7.F4)\(a\), the HAR dataset highlights the advantage of adaptive monitoring under sudden and recurring drift\. During major drift events, MPDD experiences substantial accuracy degradation, dropping from near\-perfect performance to approximately 0\.20, whereas APDDM maintains accuracy around 0\.75 and recovers more rapidly\. The corresponding AG regions indicate substantial accuracy improvements during drift intervals\. The baseline methods exhibit larger fluctuations and less consistent behaviour across the stream\. The Electricity results inFig\.[4](https://arxiv.org/html/2608.18555#S7.F4)\(b\)demonstrate the benefits of adaptation under mixed sudden and incremental drift\. While MPDD accuracy falls to approximately 0\.2 and later approaches 0\.0 during major drift transitions, APDDM maintains higher accuracy and returns to stable performance more quickly, resulting in larger AG regions during the most severe drift periods\. For Weather,Fig\.[4](https://arxiv.org/html/2608.18555#S7.F4)\(c\)presents a smoother but under\-sensitive drift pattern\. MPDD accuracy drops to approximately 0\.15 during major drift periods, whereas APDDM limits the degradation to around 0\.4–0\.6 and recovers earlier\. Although the performance gap is smaller than in HAR and Electricity, the adaptive strategy consistently reduces both the magnitude and duration of accuracy loss, resulting in a smaller but consistent accuracy gain\. The Airline dataset, shown inFig\.[4](https://arxiv.org/html/2608.18555#S7.F4)\(d\), contains stronger sudden\-drift behaviour and therefore represents a challenging setting for fixed monitoring\. MPDD experiences severe performance drops, reaching nearly 0\.0 during major drift events, while APDDM maintains near\-perfect accuracy throughout most of the stream\. Accordingly, the AG regions are most prominent during abrupt drift events, demonstrating the effectiveness of adaptive monitoring under severe drift conditions\. Finally,Fig\.[4](https://arxiv.org/html/2608.18555#S7.F4)\(e\)highlights the recurring and incremental drift characteristics of the Poker stream\. MPDD exhibits several accuracy fluctuations throughout the stream, with noticeable drops to approximately 0\.65 during drift intervals before recovering\. In contrast, APDDM maintains more stable performance, remaining above approximately 0\.85 for most of the evaluation period and recovering faster following drift events\.

We further analyze the number of false negatives across different drift\-detection techniques to evaluate their ability to identify real performance\-drift events\. Fig\.[5](https://arxiv.org/html/2608.18555#S7.F5)presents the comparison across datasets, where lower values indicate fewer missed drift detections\. Overall, APDDM and MPDD achieve relatively low false\-negative counts across all datasets\. Although data\-distribution\-based techniques such as ADWIN, MMD, and D3 show lower false\-negative counts for some datasets, particularly HAR and Electricity, these methods also produce significantly higher false\-positive rates\. Similarly, STUDD and SCSD achieve low false\-negative counts in a few cases but exhibit substantially larger values on Weather and Poker, reducing their overall effectiveness\. In contrast, MPDD and APDDM provide more consistent performance across different datasets\. APDDM achieves the lowest or near\-lowest false\-negative counts in most cases, demonstrating that adaptive monitoring can reduce missed drift events while maintaining a better balance between false positives and false negatives\. These results highlight the effectiveness of the proposed approaches for reliable MLaaS performance\-drift detection in dynamic IoT environments\.

TABLE V:Detection\-quality comparison across drift detection techniques\.Table[V](https://arxiv.org/html/2608.18555#S7.T5)presents the comparative performance of the proposed APDDM framework against the fixed\-interval MPDD baseline and five representative drift detection techniques, namely ADWIN, MMD, D3, STUDD, and SCSD, across five datasets\. The comparison is conducted using Accuracy↑\\uparrow, Accuracy Gain↑\\uparrow, Miss Detection Ratio \(MDR\)↓\\downarrow, and False Negatives \(FN\)↓\\downarrow\. Accuracy reflects the overall correctness of drift detection decisions, while Accuracy Gain quantifies the relative improvement achieved by APDDM over competing approaches\. MDR and FN are used to evaluate the ability of each method to identify drift events without missing significant changes in the data stream\. Lower MDR and FN values indicate better drift detection capability and improved responsiveness to evolving data distributions\. Overall, APDDM consistently achieves the highest detection accuracy across all datasets while simultaneously reducing the number of missed drift events\. For theHARdataset, APDDM achieves the highest accuracy of 0\.92, representing an improvement of 1\.25% over the MPDD baseline\. Furthermore, APDDM reduces the number of false negatives from 440 under MPDD to 380, corresponding to a reduction of approximately 13\.6%, while also lowering the MDR from 0\.28 to 0\.25\. These results indicate that adaptive monitoring improves drift\-detection accuracy while reducing missed drift events\. A similar trend is observed in theAirlinedataset, where APDDM achieves an accuracy of 0\.96 compared to 0\.90 for MPDD, representing a 6\.0% improvement\. In addition, the number of false negatives is reduced from 80 to 32, corresponding to a 60\.0% reduction, while the MDR decreases from 0\.31 to 0\.17\. This demonstrates the effectiveness of adaptive monitoring in identifying evolving drift patterns and minimizing missed detections\. For theElectricitydataset, APDDM achieves the highest accuracy of 0\.92 and reduces false negatives from 355 under MPDD to 280, corresponding to a reduction of approximately 21\.1%\. Similarly, the MDR decreases from 0\.285 to 0\.23\. Although D3 achieves a lower MDR of 0\.18 with only 75 false negatives, its overall detection accuracy remains substantially lower at 0\.72, indicating a less balanced trade\-off between drift coverage and overall detection quality\. In theWeatherdataset, APDDM again provides the highest accuracy of 0\.89 while reducing false negatives from 870 to 794 and lowering the MDR from 0\.29 to 0\.26 compared with MPDD\. The performance gap is even more pronounced when compared with traditional drift\-detection techniques, which exhibit substantially higher MDR values and considerably larger numbers of missed drift events\. Similarly, for thePokerdataset, APDDM achieves the highest accuracy of 0\.97, representing a 9\.81% improvement over MPDD\. Moreover, the number of false negatives is reduced dramatically from 640 under MPDD to only 160, corresponding to a 75\.0% reduction\. This improvement is accompanied by a substantial decrease in MDR from 0\.25 to 0\.06, demonstrating the effectiveness of adaptive monitoring in reducing missed drift detections\. In contrast, methods such as STUDD and SCSD exhibit MDR values close to 1\.0 and very large false\-negative counts, indicating that a substantial proportion of drift events remain undetected\. Across all datasets, the results demonstrate that APDDM consistently provides a more favourable balance between detection accuracy and drift coverage\.

### VII\-DExperiment 3: Comprehensive Evaluation of the MLaaS Extraction Model

The MLaaS extraction model is key in the proposed MPDD framework to approximate the behaviour of the original black\-box MLaaS service\. The effectiveness of the framework depends on the quality of this approximation\. To evaluate this, we first analyze the fidelity and prediction behavior analysis of the MLaaS extracted model\. Next, we then investigate the impact of MLaaS extraction model error on the MPDD and APDDM techniques performance\.

#### VII\-D1Fidelity and Predictive Behavior Analysis of the MLaaS extraction Model

In this experiment, we evaluate the fidelity and predictive behavior of the proposed MLaaS extraction model, which aims to approximate the decision patterns of black\-box MLaaS services\. Fidelity analysis is a standard evaluation strategy in model extraction research\. It measures the agreement between the predictions of the extracted model and those of the original inaccessible MLaaS service, rather than the accuracy with respect to the ground\-truth labels\[[17](https://arxiv.org/html/2608.18555#bib.bib38)\]\. Figure[6](https://arxiv.org/html/2608.18555#S7.F6)\(a\) compares the predictive accuracy of both the original MLaaS service and the extraction model against the ground truth across all datasets after performance drift has occurred\. As expected, the predictive accuracy of the original MLaaS service decreases under drifted conditions\. The extraction model exhibits nearly the same degradation trend, closely tracking the predictive behaviour of the original MLaaS service\. The deviation between the MLaaS service and the extraction model remains small across all datasets, at approximately 3% for HAR, 1% for Electricity, 3% for Weather, and nearly identical performance for Poker and Airline\. Consequently, the extraction model achieves an average fidelity of approximately95\.2%, indicating a high level of agreement with the predictions of the original MLaaS service\. Specifically, the extraction model achieves predictive accuracies against the ground truth of approximately 0\.61 on HAR, 0\.72 on Electricity, 0\.82 on Weather, 0\.78 on Poker, and 0\.63 on Airline, closely matching the corresponding accuracies of the original MLaaS services under the same drift conditions\. These results demonstrate that the extraction model effectively captures original MLaaS service\. To further investigate behavioral consistency, Fig\.[6](https://arxiv.org/html/2608.18555#S7.F6)\(b\) presents a predictive outcome decomposition that categorizes predictions into four cases: both MLaaS and extraction models correct, MLaaS correct but extraction incorrect, extraction correct but MLaaS incorrect, and both models incorrect\. This analysis provides a more detailed understanding of the relationship between the extracted model and the original MLaaS service beyond overall accuracy\.

\(a\)\(b\)
Fig\. 6:Fidelity and predictive behavior analysis of the MLaaS extraction model across datasets: \(a\) predictive accuracy comparison, and \(b\) predictive behavior analysisAcross datasets, the dominant portion corresponds to the both\-correct category, reaching approximately 0\.80 for Weather, 0\.78 for Poker, 0\.70 for Electricity, 0\.63 for Airline, and 0\.59 for HAR, indicating substantial agreement between the models\. The disagreement regions remain relatively small, with the MLaaS\-correct/extraction\-wrong and MLaaS\-wrong/extraction\-correct categories generally below 5% across datasets\. The both\-wrong category is primarily influenced by the inherent difficulty of each dataset\. However, the small disagreement fractions indicate that the extraction model closely follows the predictive behavior of the original MLaaS service\.

#### VII\-D2Impact of MLaaS Extraction Model Errors on MPDD and APDDM Performance

A key concern in black\-box MLaaS monitoring is that the extracted model may not perfectly reproduce the decision behaviour of the original service, particularly when the underlying model is a complex non\-linear model\. Such approximation errors may occur in disagreement regions and could potentially affect downstream drift\-detection decisions\. Therefore, we quantify how much MLaaS extraction\-model errors propagate to the final drift\-detection outcome\. Figure[7](https://arxiv.org/html/2608.18555#S7.F7)compares the drift\-detection accuracy obtained using the MLaaS extracted model and the original black\-box MLaaS model for both MPDD and APDDM\. For MPDD, the results show only small differences between the two models, with accuracy differences ranging from0\.00 to 1\.67 percentagepoints across datasets\. No difference is observed for Airline and Weather, while HAR, Electricity, and Poker exhibit only minor variations\. For APDDM, the results remain highly consistent with those obtained using the original MLaaS model\. The accuracy remains unchanged for Airline, Electricity, and Weather, with only a small difference for HAR and a moderate decrease for Poker\. Overall, APDDM is even less sensitive to extraction\-model approximation errors, with deviations ranging from\-2\.58 to \+0\.42 percentage points, and in most cases showing no measurable difference\. This robustness stems from APDDM’s adaptive parameter\-selection mechanism, which adjusts the monitoring interval based on recent drift\-score trends rather than relying on the output of any single extraction\-model prediction\. Consequently, small approximation errors have only a limited influence on the final drift\-detection decision\. These results indicate that MLaaS extraction\-model errors have limited impact on the final MPDD and APDDM decisions across the evaluated scenarios\. Hence, the extracted model provides a reliable approximation of the original MLaaS service for performance\-drift detection\.

Fig\. 7:Comparison of MPDD and APDDM accuracy when using the MLaaS extracted model and the original MLaaS

## VIIIDiscussions and Implications

Our framework enables clients to identify potential drifts in MLaaS, allowing them to notify providers, opt for alternative services, or perform service composition\. Since some drift points in the experiments were generated randomly, the results may be sensitive to their placement\. To mitigate this, we repeated the random generation multiple times and reported aggregated metrics to ensure robustness\. In this context, the proposed approach functions as an effective service management tool for MLaaS consumers by enabling proactive drift awareness, informed provider notification, and timely service switching or composition decisions\. This capability is especially important in cost\-sensitive and safety\-critical domains, such as smart healthcare, industrial IoT, and intelligent transportation, where undetected performance drift can lead to incorrect decisions, operational risks, and financial losses\.

## IXConcluding Remarks

In this paper, we proposed an MLaaS Performance Drift Detection \(MPDD\) framework for IoT environments where ground\-truth labels and internal model parameters are inaccessible\. To address this limitation, we designed an MLaaS extraction model that captures observable behavioral patterns of black\-box MLaaS services, enabling reliable monitoring of feature preference variations\. We further introduced a Fréchet\-based drift measurement mechanism to quantify distributional changes in evolving data streams and developed an MLaaS\-aware Drift Exposure Score within MPDD to predict performance drift\. Extensive experiments across multiple real\-world datasets demonstrate that the framework improves drift detection effectiveness, achieving approximately 22–25% accuracy improvement over representative baseline drift detection methods while maintaining stability under diverse drift patterns\. Additionally, the adaptive\-interval APDDM dynamically adjusts monitoring frequency based on drift intensity, yielding around 4% accuracy improvement and approximately 9% reduction in miss detection rate compared to fixed\-interval monitoring, thereby enhancing detection reliability and timeliness in non\-stationary IoT streams\. Overall, the framework provides a robust and timely solution for continuous MLaaS performance monitoring without requiring manual drift inspection or continuous ground\-truth collection\.

## References

- \[1\]V\. Agate and et al\.\(2024\)Enhancing iot network security with concept drift\-aware unsupervised threat detection\.In2024 IEEE ISCC,Cited by:[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p2.1)\.
- \[2\]M\. Baena\-Garcıa and et al\.\(2006\)Early drift detection method\.InFourth international workshop on knowledge discovery from data streams,Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p3.1),[§I](https://arxiv.org/html/2608.18555#S1.p4.1),[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1)\.
- \[3\]R\. S\. Barros and et al\(2017\)RDDM: reactive drift detection method\.ESA\.Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p4.1),[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1),[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p2.1)\.
- \[4\]F\. Bayram and et al\.\(2022\)From concept drift to model degradation: an overview on performance\-aware drift detectors\.Knowledge\-Based Systems\.Cited by:[1st item](https://arxiv.org/html/2608.18555#S1.I1.i1.p1.1)\.
- \[5\]A\. Bifet and R\. Gavalda\(2007\)Learning from time\-changing data with adaptive windowing\.InSIAM,pp\. 443–448\.Cited by:[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1),[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p2.1),[§VII\-A3](https://arxiv.org/html/2608.18555#S7.SS1.SSS3.p1.1),[TABLE IV](https://arxiv.org/html/2608.18555#S7.T4.2.1.5.1.1)\.
- \[6\]R\. Cattral and F\. Oppacher\(2002\)Poker hand dataset\.Note:UCI Machine Learning RepositoryExternal Links:[Link](https://archive.ics.uci.edu/ml/datasets/poker+hand)Cited by:[5th item](https://arxiv.org/html/2608.18555#S7.I1.i5.p1.1.1),[TABLE IV](https://arxiv.org/html/2608.18555#S7.T4.2.1.1.6.1)\.
- \[7\]V\. Cerqueira and et al\.\(2023\)STUDD: a student–teacher method for unsupervised concept drift detection\.Machine Learning\.Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p4.1),[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p2.1),[§VII\-A3](https://arxiv.org/html/2608.18555#S7.SS1.SSS3.p1.1),[TABLE IV](https://arxiv.org/html/2608.18555#S7.T4.2.1.11.1.1)\.
- \[8\]Y\. Chen and et al\.\(2021\)Asynchronous federated learning for sensor data with concept drift\.InIEEE Big Data,Cited by:[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1)\.
- \[9\]P\. Chiang and S\. Tsai\(2024\)Detection of malicious domains with concept drift using ensemble learning\.IEEE TNSM\.Cited by:[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1),[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p2.1)\.
- \[10\]\(2008\)Data Expo 2009: Airline on time data\.Harvard Dataverse\.External Links:[Document](https://dx.doi.org/10.7910/DVN/HG7NV7),[Link](https://doi.org/10.7910/DVN/HG7NV7)Cited by:[4th item](https://arxiv.org/html/2608.18555#S7.I1.i4.p1.1.1),[TABLE IV](https://arxiv.org/html/2608.18555#S7.T4.2.1.1.7.1)\.
- \[11\]D\. Dowson and B\. Landau\(1982\)The fréchet distance between multivariate normal distributions\.JMA\.Cited by:[§V\-B](https://arxiv.org/html/2608.18555#S5.SS2.p1.1)\.
- \[12\]Z\. M\. Fadlullah and et al\.\(2018\)On delay\-sensitive healthcare data analytics at the network edge based on deep learning\.In2018 14th IWCMC,Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p2.1)\.
- \[13\]S\. M\. M\. Fattah and A\. Bouguettaya\(2020\)Event\-based detection of changes in iaas performance signatures\.In2020 IEEE SCC,Cited by:[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1)\.
- \[14\]I\. Frias\-Blanco and et al\(2014\)Online and non\-parametric drift detection methods based on hoeffding’s bounds\.IEEE TKDE\.Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p3.1),[§I](https://arxiv.org/html/2608.18555#S1.p4.1),[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1)\.
- \[15\]Ö\. Gözüaçık and et al\.\(2019\)Unsupervised concept drift detection with a discriminative classifier\.InACM CIKM,Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p4.1),[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p2.1),[§VII\-A3](https://arxiv.org/html/2608.18555#S7.SS1.SSS3.p1.1),[TABLE IV](https://arxiv.org/html/2608.18555#S7.T4.2.1.9.1.1)\.
- \[16\]S\. Greco and et al\.\(2025\)Unsupervised concept drift detection from deep learning representations in real\-time\.IEEE TKDE\.Cited by:[1st item](https://arxiv.org/html/2608.18555#S1.I1.i1.p1.1),[§I](https://arxiv.org/html/2608.18555#S1.p4.1),[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p1.1)\.
- \[17\]M\. Jagielski and et al\.\(2020\)High accuracy and high fidelity extraction of neural networks\.InUSENIX Security 20,pp\. 1345–1362\.Cited by:[§VII\-D1](https://arxiv.org/html/2608.18555#S7.SS4.SSS1.p1.1)\.
- \[18\]S\. Jang and et al\.\(2022\)Sequential covariate shift detection using classifier two\-sample tests\.InICML,Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p4.1),[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p2.1),[§VII\-A3](https://arxiv.org/html/2608.18555#S7.SS1.SSS3.p1.1),[TABLE IV](https://arxiv.org/html/2608.18555#S7.T4.2.1.13.1.1)\.
- \[19\]D\. Kanneganti and et al\.\(2025\)Adaptive composition of machine learning as a service \(mlaas\) for iot environments\.In2025 IEEE ICWS,Cited by:[1st item](https://arxiv.org/html/2608.18555#S1.I1.i1.p1.1)\.
- \[20\]D\. Kanneganti, S\. Mistry, S\. Fattah, J\. Boland, and A\. Krishna\(2026\)Machine learning as a service \(mlaas\) dataset generator framework for iot environments\.InProceedings of the ACM Web Conference 2026,pp\. 8553–8556\.Cited by:[§VII](https://arxiv.org/html/2608.18555#S7.p1.1)\.
- \[21\]F\. J\. Massey Jr\(1951\)The kolmogorov\-smirnov test for goodness of fit\.JASAA\.Cited by:[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p1.1)\.
- \[22\]Medtronic plc\(2019\)New Prediction Feature for Hypoglycemia Now Available in Sugar\.IQTMPersonal Diabetes Assistant App, Developed by Medtronic and IBM Watson Health\.Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p2.1)\.
- \[23\]N\. Oceanic and A\. A\. \(NOAA\)\(2025\)NOAA global surface summary of the day\.Note:[https://www\.kaggle\.com/datasets/noaa/noaa\-global\-surface\-summary\-of\-the\-day](https://www.kaggle.com/datasets/noaa/noaa-global-surface-summary-of-the-day)Cited by:[3rd item](https://arxiv.org/html/2608.18555#S7.I1.i3.p1.1.1),[TABLE IV](https://arxiv.org/html/2608.18555#S7.T4.2.1.1.5.1)\.
- \[24\]A\. Pashamokhtari and et al\.\(2023\)Dynamic inference from iot traffic flows under concept drifts in residential isp networks\.IEEE IoTJ\.Cited by:[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p2.1)\.
- \[25\]F\. Pedregosa and et al\.\(2011\)Scikit\-learn: machine learning in python\.Journal of Machine Learning Research12,pp\. 2825–2830\.Cited by:[§B\-1](https://arxiv.org/html/2608.18555#A2.SS0.SSS1.p1.1),[§VII\-A2](https://arxiv.org/html/2608.18555#S7.SS1.SSS2.p1.1)\.
- \[26\]A\. Pesaranghader and et al\(2016\)Fast hoeffding drift detection method for evolving data streams\.InJoint European conference on machine learning and knowledge discovery in databases,Cited by:[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1)\.
- \[27\]R\. Philipp and et al\.\(2020\)Machine learning as a service: challenges in research and applications\.InIIWAS,Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p2.1)\.
- \[28\]A\. Reiss\(2012\)PAMAP2 Physical Activity Monitoring\.Note:UCI Machine Learning RepositoryDOI: https://doi\.org/10\.24432/C5NW2HCited by:[1st item](https://arxiv.org/html/2608.18555#S7.I1.i1.p1.1.1),[TABLE IV](https://arxiv.org/html/2608.18555#S7.T4.2.1.1.3.1)\.
- \[29\]M\. Ribeiro and et al\.\(2015\)Mlaas: machine learning as a service\.In2015 IEEE 14th ICMLA,pp\. 896–902\.Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p1.1)\.
- \[30\]B\. Shayesteh and et al\.\(2022\)Automated concept drift handling for fault prediction in edge clouds using reinforcement learning\.IEEE TNSM\.Cited by:[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1)\.
- \[31\]M\. Sinn, A\. Ghodsi, and K\. Keller\(2012\)Detecting change\-points in time series by maximum mean discrepancy of ordinal pattern distributions\.arXiv preprint arXiv:1210\.4903\.Cited by:[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p1.1),[§VII\-A3](https://arxiv.org/html/2608.18555#S7.SS1.SSS3.p1.1),[TABLE IV](https://arxiv.org/html/2608.18555#S7.T4.2.1.7.1.1)\.
- \[32\]M\. Tang and et al\.\(2024\)ModelGuard: information\-theoretic defense against model extraction attacks\.In33rd USENIX Security 24,Cited by:[§V\-A](https://arxiv.org/html/2608.18555#S5.SS1.p1.1)\.
- \[33\]F\. Tramèr and et al\.\(2016\)Stealing machine learning models via prediction apis\.In25th USENIX Security 16,pp\. 601–618\.Cited by:[§V\-A](https://arxiv.org/html/2608.18555#S5.SS1.p1.1)\.
- \[34\]A\. Trindade\(2015\)ElectricityLoadDiagrams20112014\.Note:UCI Machine Learning RepositoryDOI: https://doi\.org/10\.24432/C58C86Cited by:[2nd item](https://arxiv.org/html/2608.18555#S7.I1.i2.p1.1.1),[TABLE IV](https://arxiv.org/html/2608.18555#S7.T4.2.1.1.4.1)\.
- \[35\]A\. Tsymbal\(2004\)The problem of concept drift: definitions and related work\.Computer Science Department, Trinity College Dublin\.Cited by:[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1)\.
- \[36\]L\. N\. Vaserstein\(1969\)Markov processes over denumerable products of spaces, describing large systems of automata\.PPI\.Cited by:[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p1.1)\.
- \[37\]L\. Wang and et al\.\(2021\)Concept drift\-based runtime reliability anomaly detection for edge services adaptation\.IEEE TKDE\.Cited by:[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p2.1)\.
- \[38\]M\. M\. W\. Yan\(2020\)Accurate detecting concept drift in evolving data streams\.ICT Express\.Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p4.1),[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1)\.
- \[39\]L\. Yang and A\. Shami\(2021\)A lightweight concept drift detection and adaptation framework for iot data streams\.IEEE IoTM\.Cited by:[§II\-A](https://arxiv.org/html/2608.18555#S2.SS1.p1.1)\.
- \[40\]H\. Yu and et al\.\(2023\)Type\-ldd: a type\-driven lite concept drift detector for data streams\.IEEE TKDE\.Cited by:[§I](https://arxiv.org/html/2608.18555#S1.p4.1),[§II\-B](https://arxiv.org/html/2608.18555#S2.SS2.p1.1)\.

## Appendix AProof of Lemmas

### Proof of Lemma 1

LetMMbe the black\-box MLaaS andM′M^\{\\prime\}be the extraction model trained to approximateMMfrom query logs\. Assume the bounded fidelity condition holds:

𝔼⁡\[\|M⁡\(X\)−M′​\(X\)\|\]≤ϵ\.\\mathbb\{E\}\\big\[\\,\|M\(X\)\-M^\{\\prime\}\(X\)\|\\,\\big\]\\leq\\epsilon\.
Fix a featurexix\_\{i\}\. In Algorithm 1, feature importance is defined operationally \(i\.e\., from input–output behavior\) as the average change in the model output under a controlled perturbation of magnitudeδ\\deltaapplied toxix\_\{i\}\. LetX\+iX\_\{\+i\}denote the perturbed version ofXXalongxix\_\{i\}\. Consider the \(per\-sample\) importance contributions

ΔiM​\(X\)\\displaystyle\\Delta\_\{i\}^\{M\}\(X\)=\|M⁡\(X\+i\)−M⁡\(X\)\|,\\displaystyle=\\big\|M\(X\_\{\+i\}\)\-M\(X\)\\big\|,ΔiM′​\(X\)\\displaystyle\\Delta\_\{i\}^\{M^\{\\prime\}\}\(X\)=\|M′​\(X\+i\)−M′​\(X\)\|\.\\displaystyle=\\big\|M^\{\\prime\}\(X\_\{\+i\}\)\-M^\{\\prime\}\(X\)\\big\|\.
By the triangle inequality,

\|ΔiM′​\(X\)−ΔiM​\(X\)\|\\displaystyle\\big\|\\Delta\_\{i\}^\{M^\{\\prime\}\}\(X\)\-\\Delta\_\{i\}^\{M\}\(X\)\\big\|≤\|M′​\(X\+i\)−M⁡\(X\+i\)\|\\displaystyle\\leq\\big\|M^\{\\prime\}\(X\_\{\+i\}\)\-M\(X\_\{\+i\}\)\\big\|\+\|M′​\(X\)−M⁡\(X\)\|\.\\displaystyle\+\\big\|M^\{\\prime\}\(X\)\-M\(X\)\\big\|\.Taking expectation and using the fidelity assumption on bothXXandX\+iX\_\{\+i\}gives

𝔼⁡\[\|ΔiM′​\(X\)−ΔiM​\(X\)\|\]≤2​ϵ\.\\mathbb\{E\}\\big\[\\,\|\\Delta\_\{i\}^\{M^\{\\prime\}\}\(X\)\-\\Delta\_\{i\}^\{M\}\(X\)\|\\,\\big\]\\;\\leq\\;2\\epsilon\.Averaging over the samples used by Algorithm 1 yields the same bound for the estimated importance of featurexix\_\{i\}:

𝔼⁡\[\|IM′​\(xi\)−IM​\(xi\)\|\]≤2​ϵ\.\\mathbb\{E\}\\big\[\\,\|I^\{M^\{\\prime\}\}\(x\_\{i\}\)\-I^\{M\}\(x\_\{i\}\)\|\\,\\big\]\\;\\leq\\;2\\epsilon\.Therefore, the extracted importance vector𝐈M′\\mathbf\{I\}^\{M^\{\\prime\}\}deviates from the behavioural importance vector𝐈M\\mathbf\{I\}^\{M\}by a bounded amount\. Since the measured importance is induced by perturbations of magnitudeδ\\delta, the estimator sensitivity is governed byδ\\delta, yielding

‖𝐈M′−𝐈M‖≤f⁡\(ϵ,δ\),\\\|\\mathbf\{I\}^\{M^\{\\prime\}\}\-\\mathbf\{I\}^\{M\}\\\|\\;\\leq\\;f\(\\epsilon,\\delta\),for some bounded functionf⁡\(ϵ,δ\)f\(\\epsilon,\\delta\)that is monotone inϵ\\epsilonand dependent on the perturbation scaleδ\\deltaand the chosen norm\. Hence, under bounded fidelity, the extraction model preserves the feature\-importance behaviour of the black\-box MLaaS up to a bounded deviation, establishing Lemma 1\.□\\square

### Proof of Lemma 2

Recall that FDDS between the baseline windowbband current windowwwis defined as in Eq\. \(5\) of the paper:

FDDS⁡\(b,w\)=‖μb−μw‖22⏟m​e​a​n​s​h​i​f​t\+Tr⁡\(Σb\+Σw−2​Σb​Σw\)⏟c​o​v​a​r​i​a​n​c​e​s​h​i​f​t\.\\mathrm\{FDDS\}\(b,w\)=\\underbrace\{\\\|\\mu\_\{b\}\-\\mu\_\{w\}\\\|\_\{2\}^\{2\}\}\_\{meanshift\}\+\\underbrace\{\\mathrm\{Tr\}\\\!\\Big\(\\Sigma\_\{b\}\+\\Sigma\_\{w\}\-2\\sqrt\{\\Sigma\_\{b\}\\Sigma\_\{w\}\}\\Big\)\}\_\{covarianceshift\}\.
\(i\) Lower bounded by the mean shift\.Let𝒞⁡\(Σb,Σw\)\\mathcal\{C\}\(\\Sigma\_\{b\},\\Sigma\_\{w\}\)denote the covariance shift term\. SinceΣb\\Sigma\_\{b\}andΣw\\Sigma\_\{w\}are symmetric positive semi\-definite, the trace\-based covariance discrepancy is non\-negative and cannot reduce the FDDS below the mean shift component\. Hence,

FDDS⁡\(b,w\)=‖μb−μw‖22\+𝒞⁡\(Σb,Σw\)≥‖μb−μw‖22,\\mathrm\{FDDS\}\(b,w\)=\\\|\\mu\_\{b\}\-\\mu\_\{w\}\\\|\_\{2\}^\{2\}\+\\mathcal\{C\}\(\\Sigma\_\{b\},\\Sigma\_\{w\}\)\\;\\geq\\;\\\|\\mu\_\{b\}\-\\mu\_\{w\}\\\|\_\{2\}^\{2\},which shows that FDDS is lower bounded by the mean shift\.

\(ii\) Under significant covariance changeConsider that the current window undergoes a substantial covariance drift case where the current covariance differs substantially from the baseline, i\.e\.,‖Σw−Σb‖\\\|\\Sigma\_\{w\}\-\\Sigma\_\{b\}\\\|is large\. Then the productΣb​Σw\\Sigma\_\{b\}\\Sigma\_\{w\}changes accordingly, and the square\-root interaction termΣb​Σw\\sqrt\{\\Sigma\_\{b\}\\Sigma\_\{w\}\}no longer aligns withΣb\\Sigma\_\{b\}\. As a result, the gap

Σb\+Σw−2​Σb​Σw\\Sigma\_\{b\}\+\\Sigma\_\{w\}\-2\\sqrt\{\\Sigma\_\{b\}\\Sigma\_\{w\}\}becomes larger in the trace sense, so𝒞⁡\(Σb,Σw\)\\mathcal\{C\}\(\\Sigma\_\{b\},\\Sigma\_\{w\}\)increases, which directly increasesFDDS⁡\(b,w\)\\mathrm\{FDDS\}\(b,w\)even ifμb≈μw\\mu\_\{b\}\\approx\\mu\_\{w\}\.

\(iii\) Under minor distributional changeLet us consider the case where the distributional change between the baseline and current windows is minor, meaning‖μb−μw‖\\\|\\mu\_\{b\}\-\\mu\_\{w\}\\\|is small andΣw≈Σb\\Sigma\_\{w\}\\approx\\Sigma\_\{b\}, then the mean shift term is small andΣb​Σw≈Σb2=Σb\\sqrt\{\\Sigma\_\{b\}\\Sigma\_\{w\}\}\\approx\\sqrt\{\\Sigma\_\{b\}^\{2\}\}=\\Sigma\_\{b\}, making𝒞⁡\(Σb,Σw\)≈0\\mathcal\{C\}\(\\Sigma\_\{b\},\\Sigma\_\{w\}\)\\approx 0\. Therefore,FDDS⁡\(b,w\)\\mathrm\{FDDS\}\(b,w\)stays small under minor fluctuations\. Combining \(i\)–\(iv\), FDDS is lower bounded by the mean shift and increases with either substantial mean drift or substantial covariance drift, while remaining low under minor changes\. This proves Lemma 2\.□\\square

### Proof of Lemma 3

Proof\.As defined in Eq\. \(6\), Algorithm 2 flags drift when the*MDES*score exceeds the thresholdθm\\theta\_\{m\}, i\.e\.,

MDES=\{1,if​∑i=1dI⁡\(xi\)⋅FDDS⁡\(bi,wi\)\>θm,0,otherwise\.\\mathrm\{MDES\}=\\begin\{cases\}1,&\\text\{if \}\\displaystyle\\sum\_\{i=1\}^\{d\}I\(x\_\{i\}\)\\cdot\\mathrm\{FDDS\}\(b\_\{i\},w\_\{i\}\)\>\\theta\_\{m\},\\\\ 0,&\\text\{otherwise\.\}\\end\{cases\}Consider the MDES defined in Eq\. \(6\), where the cumulative exposure is given by∑i=1dI⁡\(xi\)⋅FDDS⁡\(bi,wi\)\\sum\_\{i=1\}^\{d\}I\(x\_\{i\}\)\\cdot\\mathrm\{FDDS\}\(b\_\{i\},w\_\{i\}\), which jointly captures the feature importance vector𝐈\\mathbf\{I\}and the feature\-wise statistical shift betweenbbandww\. If the statistical drift is predominantly concentrated on features with high importance weights, then the corresponding termsI⁡\(xi\)⋅FDDS⁡\(bi,wi\)I\(x\_\{i\}\)\\cdot\\mathrm\{FDDS\}\(b\_\{i\},w\_\{i\}\)become large, leading to a significant increase in the cumulative weighted exposure\. Consequently, the exposure is more likely to exceed the thresholdθm\\theta\_\{m\}, which activates the conditionMDES=1\\mathrm\{MDES\}=1and indicates that the drift is aligned with influential features, thereby characterizing it as real drift\. In contrast, if the statistical shift mainly occurs on features with low importance weights, the sameFDDS⁡\(bi,wi\)\\mathrm\{FDDS\}\(b\_\{i\},w\_\{i\}\)contributions are attenuated by smallI⁡\(xi\)I\(x\_\{i\}\)values, keeping the cumulative exposure bounded and unlikely to surpassθm\\theta\_\{m\}\. Under this condition,MDES=0\\mathrm\{MDES\}=0, and the drift is interpreted as pseudo\-drift\. Therefore, the MLaaS Drift Exposure Score distinguishes real drift from pseudo\-drift based on the alignment between drift magnitude and feature importance\. This proves Lemma 3\.□\\square

## Appendix BDrift Generation

TABLE 6:Drift generation statistics across datasets\.\(a\)\(b\)\(c\)\(d\)\(e\)
Fig\. 8:Drift generation across datasets: \(A\) HAR, \(B\) Airline, \(C\) Weather, \(D\) Electricity, and \(E\) Poker\.#### B\-1Drift Data Generation

We use the SEA generator\[[25](https://arxiv.org/html/2608.18555#bib.bib11)\]to create drifted data streams across the datasets used in our experiments, modelling four common drift categories: sudden, incremental, gradual, and recurrent drift\. For each dataset, four full\-length drifted streams are generated, one for each drift type\. Consequently, the combined generated stream contains four times the number of samples in the original dataset\. Table[6](https://arxiv.org/html/2608.18555#A2.T6)summarizes the stream sizes and drift statistics across all datasets\. Specifically, the generated streams contain 240,000 samples for HAR, 39,956 for Airline, 181,248 for Electricity, 385,812 for Weather, and 400,000 for Poker\. The generated drift regions comprise 76,400 samples for HAR, 12,724 for Airline, 62,081 for Electricity, 147,572 for Weather, and 127,336 for Poker\. These drift samples are distributed across sudden, incremental, gradual, and recurrent drift scenarios\. For each dataset, we generate one sudden drift event, one gradual drift event, one incremental drift event consisting of five progressively stronger stages, and two recurrent drift events, resulting in five drift episodes per dataset and 25 drift episodes across all datasets\. Figure[8](https://arxiv.org/html/2608.18555#A2.F8)illustrates the drift behaviour across the generated streams and highlights how different drift types evolve over time\. For evaluation, each stream is processed using chunks of 250 samples, where each chunk consists of five consecutive blocks of 50 samples\. A chunk is labelled as a real\-drift chunk when more than 50% of its samples belong to a generated drift region\. The visualizations presented in Appendix B show uniformly selected representative chunks from the complete streams and are intended to illustrate drift progression rather than indicate the total number of generated drift events\.

![[Uncaptioned image]](https://arxiv.org/html/2608.18555v1/authors/deepak_kanneganti.jpg)Deepak Kannegantiis an Associate Lecturer at Curtin University, Australia\. He received his Master’s degree in Predictive Analytics from Curtin University, Bentley, Perth, WA, Australia, and his Bachelor’s degree in Electronics and Communication Engineering\. He previously served as a Data Science Intern at the Pawsey Supercomputing Research Centre\. His research interests include distributed machine learning, federated learning, and explainable artificial intelligence\.![[Uncaptioned image]](https://arxiv.org/html/2608.18555v1/authors/sajib_mistry.png)Sajib Mistryis an Associate Professor at Curtin University, Australia\. He received his Ph\.D\. degree from RMIT University, Australia\. Prior to joining Curtin University, he was a Postdoctoral Fellow with the School of Computer Science, University of Sydney, Australia\. His research interests include edge and cloud computing, big data, and the Internet of Things\. He has published articles in international journals and conferences, including IEEE Transactions on Services Computing, IEEE Transactions on Knowledge and Data Engineering, Communications of the ACM, ICSOC, WISE, and IEEE ICWS\. He received the Best Paper Award at ICSOC 2016\.![[Uncaptioned image]](https://arxiv.org/html/2608.18555v1/authors/sheik_fattah.png)Sheik Mohammad Mostakim Fattahis a Lecturer at Curtin University, Australia\. He received his Ph\.D\. degree in Computer Science from the University of Sydney, Australia, an M\.Eng\. degree in Computer and Information Communication Engineering from Hankuk University of Foreign Studies, South Korea, and a B\.Sc\. \(Hons\.\) degree in Computer Science and Engineering from the University of Dhaka, Bangladesh\. Prior to joining Curtin University, he held academic and research positions at the University of Adelaide, Torrens University, and the University of Sydney, as well as industry positions at the Korea Electronics Technology Institute and Monist IT Ltd\. His research interests include cloud computing, services computing, edge computing, the Internet of Things, and Semantic Web technologies\. His work has appeared in leading venues such as IEEE Transactions on Services Computing, ACM Transactions on the Web, ICSOC, and IEEE ICWS\. He serves as a reviewer and program committee member for several international conferences and journals\.![[Uncaptioned image]](https://arxiv.org/html/2608.18555v1/authors/erik_elmroth.png)Erik Elmroth\(Member, IEEE\) is a Professor of Computing Science at Umeå University, Sweden\. He has served as Head and Deputy Head of the Department of Computing Science for 13 years and as Deputy Director of the National Supercomputer Centre for another 13 years\. He established Umeå University’s research activities in distributed systems\. His experience in management and executive groups of large\-scale research initiatives includes the EUR 550 million Wallenberg AI, Autonomous Systems and Software Program and the strategic research area eSSENCE\. He has developed two international research strategies for the Nordic Council of Ministers\. His international experience includes one year at NERSC, Lawrence Berkeley National Laboratory, University of California, Berkeley, USA, and one semester at the Massachusetts Institute of Technology, Cambridge, MA, USA\. He has served as a member of the Swedish Research Council’s Committee for Research Infrastructure, Chair of its expert panel on eScience, and Chair of the Board of the Swedish National Infrastructure for Computing\. He is a Lifetime Member of the Royal Swedish Academy of Engineering Sciences and has served as Vice Chair of its Division for Information Technology\.![[Uncaptioned image]](https://arxiv.org/html/2608.18555v1/authors/aneesh_krishna.jpg)Aneesh Krishnais a Full Professor in the School of Electrical Engineering, Computing and Mathematical Sciences at Curtin University, Australia\. He received his Ph\.D\. degree in Computer Science from the University of Wollongong, Australia\. He has held several academic positions, including Lecturer in Software Engineering with the School of Computer Science and Software Engineering at the University of Wollongong from February 2006 to June 2009\. His research interests include artificial intelligence for software engineering, model\-driven development and evolution, requirements engineering, agent systems, formal methods, data mining, computer vision, machine learning, bioinformatics, and renewable energy systems\. He has published more than 200 articles in reputed journals and international conference proceedings\.![[Uncaptioned image]](https://arxiv.org/html/2608.18555v1/authors/monowar_bhuyan.png)Monowar Bhuyan\(WASP Fellow and Senior Member, IEEE\) received his Ph\.D\. degree in Computer Science and Engineering from Tezpur University, Assam, India\. He is currently an Associate Professor in the Department of Computing Science at Umeå University, Sweden\. He established and leads the Cyber Analytics and Learning Group and is a Senior Member of the Autonomous Distributed Systems Lab\. Prior to this, he held academic and research positions at several institutions, including the Nara Institute of Science and Technology, Japan; Assam Kaziranga University, India; and Umeå University, Sweden, at various levels from Junior Scientist to Associate Professor between January 2009 and December 2019\. He has published more than 100 papers in leading peer\-reviewed international journals and conference proceedings and authored the advanced textbook “Network Traffic Anomaly Detection and Prevention” with Springer\. His experience in leading and co\-leading research projects has attracted more than SEK 40 million in funding from national, European Commission, and international funding agencies\. His research interests include machine learning, anomaly detection, systems and AI security, and distributed systems\.