Clustering-Based Collective Anomaly Detection in IoT Systems: A Graph Neural Network Approach

# Clustering-Based Collective Anomaly Detection in IoT Systems: A Graph Neural Network Approach
Source: [https://arxiv.org/html/2609.22166](https://arxiv.org/html/2609.22166)
###### Abstract

The rapid advancement of Internet of Things \(IoT\) technology has led to the widespread deployment of smart, interconnected devices across a range of domains\. However, this expansion has also resulted in a substantial increase in network traffic, creating more opportunities for malicious actors to launch cyberattacks and compromise sensitive information, thereby increasing the need for effective anomaly detection\. The state\-of\-the\-art in anomaly detection has predominantly focused on point anomalies\. In contrast, the detection of collective anomalies remains relatively under\-explored in the literature\.

In this paper, we introduce Unsupervised Graph Collective Anomaly Detection \(UGCAD\), a novel framework designed to identify collective anomalies in IoT network traffic\. Unlike many existing methods, UGCAD operates on graph\-structured data without any prior knowledge of group labels or membership\. It leverages a variational graph autoencoder \(VGAE\) to learn the graph representation, which is subsequently used to enhance a clustering algorithm for effective grouping of nodes\. To detect collective anomalies, clusters identified as normal are first aggregated and refined, after which anomaly scores are applied to detect collective anomalies\. Extensive experiments conducted on the CICIoT2023 and ToN\-IoT network datasets demonstrate the effectiveness of UGCAD in both clustering and collective anomaly detection \(CAD\)\. Furthermore, comparative evaluations against several traditional and state\-of\-the\-art clustering\-based CAD approaches confirm the superiority of UGCAD in accurately detecting collective anomalies\.

###### Index Terms:

Collective anomaly detection, deep learning, graph neural networks, internet of things, variational graph auto\-encoder\.

††history:Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000\.††doi:10\.1109/ACCESS\.2024\.0429000††address:University of the West of England\(UWE\) Bristol BS16 1QY, England \(e\-mail: dalila\.khettaf@uwe\.ac\.uk\)††address:University of South\-Eastern Norway, Kongsberg, 3616 Norway\(e\-mail: youcef\.djenouri@usn\.no\)††corresponding:Corresponding author: Dalila Khettaf \(e\-mail: dalila\.khettaf@uwe\.ac\.uk\)\.## IIntroduction

The integration and widespread adoption of Internet of Things \(IoT\) technology have led to the proliferation of IoT\-enabled smart devices across a wide range of applications, including smart home appliances \(e\.g\., smart TVs, thermostats, smart meters, and security systems\), healthcare devices, wearable devices, and vehicular and roadside systems\[[1](https://arxiv.org/html/2609.22166#bib.bib1)\]\. These devices enable a plethora of applications but operate on diverse technologies and communication protocols, often forming a heterogeneous network\. Combined with the large volumes of data they generate and manage, this complexity increases the vulnerability to cyberattacks, placing users’ sensitive information at risk\[[2](https://arxiv.org/html/2609.22166#bib.bib2)\]\[[3](https://arxiv.org/html/2609.22166#bib.bib3)\]\.

Anomaly detection is a fundamental approach for mitigating cyberattacks and ensuring the security of IoT networks\. It involves identifying observations or patterns in data that deviate markedly from the expected behavior of a system\[[4](https://arxiv.org/html/2609.22166#bib.bib4)\]\. This capability is critical for ensuring the dependable and trustworthy operation of complex systems across a wide range of domains, including healthcare, cybersecurity, finance, and e\-commerce\[[4](https://arxiv.org/html/2609.22166#bib.bib4)\]\. In the literature, anomalies are generally categorized into three types: point anomalies, contextual anomalies, and collective anomalies\[[5](https://arxiv.org/html/2609.22166#bib.bib5)\]\. A point anomaly refers to a single data instance that deviates significantly from the expected behavior of a system\. A contextual \(or conditional\) anomaly is a data instance that is considered anomalous only within a specific context\. In contrast, a collective \(or group\) anomaly consists of a set of data instances that together exhibit anomalous behavior, even if individual instances appear normal\[[35](https://arxiv.org/html/2609.22166#bib.bib35)\]\. A common example of a collective anomaly is the Denial of Service \(DoS\) attack, along with its distributed version \(DDoS\), in which a server or network is overwhelmed by a flood of requests\. While a single request appears legitimate, the overall pattern of traffic reveals the anomalous nature of the attack\[[6](https://arxiv.org/html/2609.22166#bib.bib6)\]\. Another example is the Mirai attack, a large\-scale cyber incident in 2016 in which malware infected insecure IoT devices, turning them into a botnet\. This botnet was then used to launch powerful DDoS attacks, overwhelming major online platforms and services\.

While point anomaly detection has been extensively investigated in the literature with methods such as Local Outlier Factor \(LOF\)\[[38](https://arxiv.org/html/2609.22166#bib.bib38)\], Isolation Forest \(IF\)\[[37](https://arxiv.org/html/2609.22166#bib.bib37)\], k\-Nearest Neighbors \(KNN\)\[[39](https://arxiv.org/html/2609.22166#bib.bib39)\], and One\-Class Support Vector Machine \(OCSVM\)\[[40](https://arxiv.org/html/2609.22166#bib.bib40)\], research on collective anomaly detection \(CAD\) has been comparatively limited\. Initial studies in this area predominantly relied on statistical and probabilistic methods\[[10](https://arxiv.org/html/2609.22166#bib.bib10)\]\[[11](https://arxiv.org/html/2609.22166#bib.bib11)\]\. More recent advancements, however, have increasingly incorporated deep learning \(DL\) techniques, with a particular emphasis on applications involving time series data\[[12](https://arxiv.org/html/2609.22166#bib.bib12)\]\[[13](https://arxiv.org/html/2609.22166#bib.bib13)\]\[[14](https://arxiv.org/html/2609.22166#bib.bib14)\]\.

Graphs effectively represent relationships between entities and can model diverse data scenarios, notably when combined with DL, i\.e\., Graph Neural Networks \(GNNs\), which perform well on both node and graph\-level tasks such as classification and anomaly detection\. However, graph data remains underexplored in CAD, with most existing approaches assuming group membership is known in advance\[[15](https://arxiv.org/html/2609.22166#bib.bib15)\]\. Motivated by these challenges, we propose a clustering\-based approach that leverages GNNs to identify collective anomalies\. In particular, our method employs a variational graph auto\-encoder \(VGAE\) to learn embeddings from IoT network traffic represented as a graph\. These embeddings are subsequently grouped using a clustering algorithm, and the resulting clusters are analyzed with an anomaly detection technique to uncover collective anomalies in the data\. The main contributions of this article are:

- •Introducing a CAD approach that leverages clustering and node embeddings to detect network attacks, without requiring prior knowledge of group membership\.
- •Evaluating the proposed method on real\-world datasets, testing its scalability and robustness with different graph sizes and different anomaly ratios, and comparing its performance against state\-of\-the\-art approaches\.
- •Demonstrating the broader applicability of CAD for network traffic analysis\.

The remainder of this paper is organized as follows\. Section[II](https://arxiv.org/html/2609.22166#S2)discusses the related work\. Section[III](https://arxiv.org/html/2609.22166#S3)formulates the problem of CAD\. Section[IV](https://arxiv.org/html/2609.22166#S4)presents the system model for UGCAD\. Section[V](https://arxiv.org/html/2609.22166#S5)elaborates the proposed solution\. Section[VI](https://arxiv.org/html/2609.22166#S6)provides the experimental details and Section[VII](https://arxiv.org/html/2609.22166#S7)presents the results\. Finally, Section[VIII](https://arxiv.org/html/2609.22166#S8)concludes the paper\.

## IIRelated Work

Earlier research on CAD primarily relied on statistical and probabilistic approaches\. Yu et al\.\[[10](https://arxiv.org/html/2609.22166#bib.bib10)\]introduced a hierarchical Bayesian model to identify group anomalies in social media data\. Xiong et al\.\[[11](https://arxiv.org/html/2609.22166#bib.bib11)\]presented an extension of LDA \(Latent Dirichlet allocation\) to detect collective anomalies in predefined groups\. Song et al\.\[[16](https://arxiv.org/html/2609.22166#bib.bib16)\]proposed a method that captures correlations between groups and built a full variational Bayesian framework that integrates a genetic algorithm for parameter optimization\. These statistical methods rely on strong prior assumptions \(e\.g\., predefined groups or distributions\) and detect anomalies based on statistical properties like variance or likelihood shifts\. As a result, they often fail when anomalies are subtle or do not significantly deviate statistically, and they struggle to capture complex relational structures in the data\.

More recent approaches to CAD increasingly incorporate advanced machine learning algorithms and extend to diverse data modalities, including images, time series, and graphs\. In particular, numerous methods have been proposed for detecting collective anomalies in time series data\. For instance, Li et al\.\[[12](https://arxiv.org/html/2609.22166#bib.bib12)\]introduced Stacked Temporal Convolutional Networks \(CPA\-TCN\) for detecting both point and collective anomalies\. Weng et al\.\[[13](https://arxiv.org/html/2609.22166#bib.bib13)\]introduced a method that leverages statistical features together with the isolation forest algorithm to detect collective anomalies in multidimensional data streams within cloud environments\. Bontemps et al\.\[[14](https://arxiv.org/html/2609.22166#bib.bib14)\]adapted a Long Short\-Term Memory \(LSTM\) model to detect collective anomalies in time series\. Shi et al\.\[[17](https://arxiv.org/html/2609.22166#bib.bib17)\]modeled time series using linear fitting functions and then applied Piecewise Integration \(PI\) to detect collective anomalies in both synthetic and real\-world datasets\. Other approaches have been developed for image data\. For example, Belhadi et al\.\[[18](https://arxiv.org/html/2609.22166#bib.bib18)\]investigated both data mining techniques and Convolutional Neural Networks \(CNNs\) for detecting collective abnormal human behavior\. Chalapathy et al\.\[[19](https://arxiv.org/html/2609.22166#bib.bib19)\]proposed to use both Adversarial Autoencoders \(AAEs\) and Variational Autoencoders \(VAEs\) for detecting collective anomalies in images, treating each image as a group of pixels\.

Relatively few studies have focused on CAD for graph data\. For instance, Feng et al\.\[[20](https://arxiv.org/html/2609.22166#bib.bib20)\]first converted multivariate time series into graphs and then proposed both a Graph Autoencoder \(GAE\) and a VGAE to detect collective anomalies in the Industrial Internet of Things \(IIoT\) systems\. D’Oro et al\.\[[21](https://arxiv.org/html/2609.22166#bib.bib21)\]presented a method that utilizes graph representation learning to detect collective anomalies\. Ai et al\.\[[22](https://arxiv.org/html/2609.22166#bib.bib22)\]proposed a framework that leverages a GAE and anchor nodes to identify anomalous groups\.

These recent DL\-based approaches remain relatively under\-developed\. Many methods still rely on prior knowledge or predefined assumptions, limiting their generalizability\. Time series\-based approaches frequently employ sliding window mechanisms, which are inherently constrained and may fail to capture anomalies that do not conform to fixed window boundaries\. Image\-based methods are also limited, as collective anomalies in visual data are not well\-established and are often evaluated on simplified or synthetic scenarios\. Similarly, graph\-based approaches often depend on predefined structures or unrealistic settings, restricting their effectiveness in modeling complex and real\-world collective anomaly patterns\.

Other approaches adopted a different strategy, identifying anomalous groups by first clustering the data\. For example, Mirsky et al\.\[[23](https://arxiv.org/html/2609.22166#bib.bib23)\]proposed a clustering approach based on Principal Component Analysis \(PCA\) to detect group anomalies in smartphone data streams\. The authors of\[[28](https://arxiv.org/html/2609.22166#bib.bib28)\]introduced MCADET, an approach that relies on X\-Means clustering and cluster variance to detect flooding\-based DoS attacks\. Belhadi et al\.\[[24](https://arxiv.org/html/2609.22166#bib.bib24)\]proposed HDM\-GAD, a hybrid framework that combines DBSCAN for micro\-clustering with a pruning strategy to detect collective anomalies in sequence data\. Ahmed et al\.\[[6](https://arxiv.org/html/2609.22166#bib.bib6)\]proposed HCADET, a method for detecting DoS attacks in network traffic data that uses two\-stage X‑Means clustering and the Hurst parameter\. Wang et al\.\[[25](https://arxiv.org/html/2609.22166#bib.bib25)\]proposed CCAD, a framework that detects collective anomalies in streaming network traffic by tracking changes in clustering structure within sliding windows\. The authors of\[[27](https://arxiv.org/html/2609.22166#bib.bib27)\]proposed CADET, a method that relies on theoretic co\-clustering to detect collective anomalies in network traffic data\.

Although clustering\-based methods provide an initial step toward grouping collective anomalies, they rely on overly simplistic assumptions to distinguish normal from anomalous clusters, typically based on basic properties such as cluster size or statistical features\.

Table[I](https://arxiv.org/html/2609.22166#S2.T1)presents a comparative analysis of the aforementioned methods based on several important characteristics\. Specifically, the table indicates whether each method: \(1\) employs DL techniques, \(2\) operates on graph\-structured data, \(3\) detects collective anomalies, \(4\) uses clustering algorithms to group data, \(5\) relies on prior knowledge of group memberships, and \(6\) follows a supervised learning paradigm\. As shown in the table, our proposed method, UGCAD, combines several desirable properties by leveraging DL on graph data for CAD while incorporating clustering mechanisms without requiring prior knowledge of group memberships\. Furthermore, UGCAD operates in an unsupervised manner, making it more suitable for realistic scenarios where labeled data and predefined groups are unavailable\.

Overall, existing methods for CAD rely on strong assumptions, prior knowledge, or simplistic statistical properties, limiting their ability to capture complex and subtle anomaly patterns\. Moreover, many approaches fail to model realistic structures and dependencies in data, resulting in poor generalization to real\-world and dynamic scenarios\. In this work, we propose UGCAD, a fully unsupervised clustering\-based CAD method for graph data that requires no prior knowledge of group membership, avoids excessive hyperparameters, and makes no assumptions about the underlying data\.

TABLE I:Comparison of Collective Anomaly Detection Methods
## IIIProblem Formulation

The problem of collective \(or group\) anomaly detection can be defined as a function that takes data as input and outputs a set of anomalous groups\. In this work, we adapt the CAD problem to graph data\.

LetG=\(V,E\)G=\(V,E\)denote an undirected graph, whereV=\{vi\}i=1NV=\\\{v\_\{i\}\\\}\_\{i=1\}^\{N\}is the set of nodes andE=\{ei,j∣\{i,j\}⊆V\}E=\\\{e\_\{i,j\}\\mid\\\{i,j\\\}\\subseteq V\\\}is the set of edges\. The relationships between nodes can be represented by an adjacency matrixA∈\{0,1\}N×NA\\in\\\{0,1\\\}^\{N\\times N\}described in Eq\.[1](https://arxiv.org/html/2609.22166#S3.E1):

Ai​j=\{1,if​ei,j∈E,0,otherwise,∀i,j∈V\.A\_\{ij\}=\\begin\{cases\}1,&\\text\{if \}e\_\{i,j\}\\in E,\\\\ 0,&\\text\{otherwise\},\\end\{cases\}\\quad\\forall i,j\\in V\.\(1\)
Each nodev∈Vv\\in Vis associated with attributes represented by a feature matrixX∈ℝN×dX\\in\\mathbb\{R\}^\{N\\times d\}, whereddis the dimensionality of node features\.

We propose anUnsupervised Graph\-based Collective Anomaly Detection\(UGCAD\) method, formalized as a functionF:\(A,X\)→𝒟F:\(A,X\)\\to\\mathcal\{D\}, where𝒞=\{c1,c2,…,cm\}\\mathcal\{C\}=\\\{c\_\{1\},c\_\{2\},\\dots,c\_\{m\}\\\}denotes the set of all candidate clusters, and𝒟⊆𝒞\\mathcal\{D\}\\subseteq\\mathcal\{C\}represents the subset of anomalous clusters or collective anomalies\. Each clusterc∈𝒞c\\in\\mathcal\{C\}is assigned an anomaly score by a scoring functionS:𝒞→ℝS:\\mathcal\{C\}\\to\\mathbb\{R\}\.

A clusterc∈𝒞c\\in\\mathcal\{C\}is considered anomalous if its score exceeds a thresholdτ\\tau, formulated in Eq\. \([2](https://arxiv.org/html/2609.22166#S3.E2)\):

c∈𝒟⇔S⁡\(c\)\>τc\\in\\mathcal\{D\}\\iff S\(c\)\>\\tau\(2\)

## IVSystem Model

The main purpose of the proposed method, UGCAD, is to detect collective anomalies in network traffic and produce a set of normal clusters and a set of anomalous clusters\. The solution can be integrated into an IoT system and works with various IoT devices, including smart home hubs, industrial sensors, smart cameras, wearables, and so on\.

UGCAD, originally designed for detecting collective anomalies in graphs, can be adapted for streaming IoT network data using clustering algorithms such as DenStream\[[33](https://arxiv.org/html/2609.22166#bib.bib33)\]\. DenStream is used here as an example, but any suitable online clustering algorithm could be applied in its place\. DenStream is an online density\-based clustering algorithm designed for data streams; it maintains compact summaries called micro\-clusters that represent dense regions \(normal patterns\) while tracking sparse or emerging regions as potential anomalies\. It continuously updates these micro\-clusters as new data arrives and applies decay to outdated patterns, making it suitable for evolving IoT environments\. To reduce the computational load on IoT devices, UGCAD is first pretrained in the cloud, where clusters representing normal and anomalous behavior are generated\. Normal clusters are converted into Core Micro\-Clusters \(CMCs\), while anomalous clusters become Outlier Micro\-Clusters \(OMCs\)\. These are then deployed to IoT edge gateways or local servers\.

As new network traffic arrives, it is matched to the nearest micro\-cluster\. If it fits within a CMC, it reinforces normal behavior; if it fits an OMC or leads to the creation of a new outlier cluster, it is flagged as potentially anomalous\. Outlier clusters that accumulate enough weight can be promoted to CMCs to capture evolving normal patterns, while existing clusters decay over time to handle concept drift\.

This architecture enables UGCAD to detect collective anomalies in real time, even when individual network packets appear normal\. Heavy computation is offloaded to the cloud, while IoT devices and edge gateways perform lightweight online processing\. Periodically, micro\-cluster summaries and detected anomalies are sent back to the cloud for monitoring, visualization, and retraining\.

## VProposed Solution

![Refer to caption](https://arxiv.org/html/2609.22166v1/figures/method_v3.png)Fig\. 1:The Proposed UGCAD Approach\.The proposed method, UGCAD, consists of two modules as illustrated in Fig\.[1](https://arxiv.org/html/2609.22166#S5.F1)\. The first is the Embedding\-Driven Clustering \(EDC\) module, which applies the Leiden algorithm to node embeddings generated by a VGAE, thereby capturing the underlying community structure of the graph\. The second is the Cluster\-Refined Anomaly Detection \(CRAD\) module, which leverages these clustering results to identify collective anomalies by distinguishing anomalous cluster formations from the refined normal ones\.

A collective anomaly is, by definition, a group of nodes that exhibit anomalous behavior only when considered together\. This motivates the first step of our method: clustering similar nodes, which captures the collective aspect of the anomaly\. The second step is to determine which clusters are anomalous\. To do this, we rely on a simple intuition grounded in reconstruction\-based anomaly detection\. In a collective anomaly, some nodes will individually appear abnormal, and a VGAE will fail to reconstruct them, yielding high reconstruction errors, a standard anomaly indicator in the literature of graph anomaly detection\. However, as discussed earlier, not all nodes within a collective anomaly are necessarily anomalous on their own\. Therefore, we compute the anomaly score of a cluster by averaging the reconstruction errors of its nodes, which helps smooth out the influence of nodes that are not individually anomalous while still capturing the collective abnormality\. The rest of this section details the two modules\.

### V\-AThe Embedding\-Driven Clustering \(EDC\) Module

EDC takes the input graphGGand produces a set of clustersCC, as summarized in Algorithm[1](https://arxiv.org/html/2609.22166#algorithm1)\. The primary objective of this module is to separate collective anomalies from normal data by grouping them into distinct clusters\. This design is motivated by our previous work\[[34](https://arxiv.org/html/2609.22166#bib.bib34)\], that proposed a community detection framework that enhances the Louvain algorithm\[[32](https://arxiv.org/html/2609.22166#bib.bib32)\]with node embeddings generated using a GNN\. Authors of\[[34](https://arxiv.org/html/2609.22166#bib.bib34)\]also introduced a merging procedure to consolidate highly similar clusters and demonstrated that using learned node embeddings rather than raw features as input to Louvain results in clusters of significantly higher quality\.

GAEs are an extension of traditional autoencoders\(AEs\) to graph\-structured data, where the goal is to learn low\-dimensional node embeddings that capture the graph’s structure\. VGAEs further extend GAEs by modeling the latent embeddings probabilistically\. VGAEs can be used to learn meaningful representations of nodes or entire graphs\.

The proposed architecture uses a Graph Convolutional Network \(GCN\) encoder\. More formally, given node feature matrixXXand adjacency matrixAA, the encoder applies two GCN layers with ReLU activations and dropout as shown in Eq\.[3](https://arxiv.org/html/2609.22166#S5.E3):

H\(1\)=R​e​L​U​\(G​C​N​C​o​n​v1​\(X,A\)\)\\displaystyle H^\{\(1\)\}=ReLU\(GCNConv\_\{1\}\(X,A\)\)\(3\)H\(2\)=R​e​L​U​\(G​C​N​C​o​n​v2​\(H\(1\),A\)\)\\displaystyle H^\{\(2\)\}=ReLU\(GCNConv\_\{2\}\(H^\{\(1\)\},A\)\)
FromH\(2\)H^\{\(2\)\}, two separate GCN layers produce the meanμ\\muand log\-variancelog⁡σ2\\log\\sigma^\{2\}for each node’s latent embedding, formulated in Eq\.[4](https://arxiv.org/html/2609.22166#S5.E4):

μ=G​C​N​C​o​n​vμ​\(H\(2\),A\)\\displaystyle\\mu=GCNConv\_\{\\mu\}\(H^\{\(2\)\},A\)\(4\)log⁡σ2=G​C​N​C​o​n​vlog⁡var​\(H\(2\),A\)\\displaystyle\\log\\sigma^\{2\}=GCNConv\_\{\\log\\text\{var\}\}\(H^\{\(2\)\},A\)
Latent embeddings are then sampled using the reparameterization trick, as described in Eq\.[5](https://arxiv.org/html/2609.22166#S5.E5):

z=μ\+σ⊙ϵ,ϵ∼𝒩⁡\(0,I\)z=\\mu\+\\sigma\\odot\\epsilon,\\quad\\epsilon\\sim\\mathcal\{N\}\(0,I\)\(5\)
The graph is reconstructed implicitly through an inner product decoderA^=σ⁡\(Z​ZT\)\\hat\{A\}=\\sigma\(ZZ^\{T\}\), whereσ\\sigmais the sigmoid function\. The reconstruction loss used is the binary cross\-entropy loss computed over the set of positive edgesE\+E^\{\+\}and a set of negatively sampled edgesE−E^\{\-\}\. The decoder models the probability of an edge between nodesiiandjjasA^i​j=σ⁡\(zi⊤​zj\)\\hat\{A\}\_\{ij\}=\\sigma\(z\_\{i\}^\{\\top\}z\_\{j\}\)and the lossℒr​e​c​o​n\\mathcal\{L\}\_\{recon\}minimizes the negative log\-likelihood of the observed edges and sampled non\-edges, formulated in Eq\.[6](https://arxiv.org/html/2609.22166#S5.E6):

ℒrecon=−1\|E\+\|∑\(i,j\)∈E\+logA^i​j−1\|E−\|∑\(i,j\)∈E−log\(1−A^i​j\)\\mathcal\{L\}\_\{\\text\{recon\}\}=\-\\frac\{1\}\{\|E^\{\+\}\|\}\\sum\_\{\(i,j\)\\in E^\{\+\}\}\\log\\hat\{A\}\_\{ij\}\-\\frac\{1\}\{\|E^\{\-\}\|\}\\sum\_\{\(i,j\)\\in E^\{\-\}\}\\log\(1\-\\hat\{A\}\_\{ij\}\)\(6\)
This objective encourages embeddings of connected nodes to yield high predicted probabilities, while embeddings of unconnected nodes are encouraged to yield low probabilities\. In this way, the learned latent space captures and preserves the connectivity patterns of the input graph, producing meaningful probabilistic node representations\.

Algorithm 1EDC: Embedding\-Driven ClusteringInput:Graph

G⁡\(A,X\)G\(A,X\)
Output:

CC,

ZZ,

ℒrecon\\mathcal\{L\}\_\{\\text\{recon\}\}
\(Z,ℒrecon\)←VGAE​\(G\)\(Z,\\mathcal\{L\}\_\{\\text\{recon\}\}\)\\leftarrow\\text\{VGAE\}\(G\)

G′←KNN​\(Z\)G^\{\\prime\}\\leftarrow\\text\{KNN\}\(Z\)

𝒞←Leiden​\(G′\)\\mathcal\{C\}\\leftarrow\\text\{Leiden\}\(G^\{\\prime\}\)

Return

𝒞,Z,ℒrec\\mathcal\{C\},Z,\\mathcal\{L\}\_\{\\text\{rec\}\}

The Louvain algorithm is an algorithm for community detection that iteratively maximizes modularity to cluster nodes into densely connected groups\. However, unlike the work\[[34](https://arxiv.org/html/2609.22166#bib.bib34)\]that relies on the Louvain algorithm, we replace it with the Leiden algorithm\[[30](https://arxiv.org/html/2609.22166#bib.bib30)\]\. The Leiden algorithm was introduced as an improvement over Louvain after it was shown that Louvain can produce poorly connected communities and suffers from scalability issues\[[30](https://arxiv.org/html/2609.22166#bib.bib30)\]\. In contrast, Leiden guarantees well\-connected communities, improves the quality of detected clusters, and is computationally faster, making it more suitable for large\-scale graphs\. To generate a graph from the embeddings, we employ the k\-nearest neighbors \(k\-NN\) method as shown in Algorithm[1](https://arxiv.org/html/2609.22166#algorithm1), which is simple to implement and requires only a few hyperparameters\. The k\-NN algorithm connects each node to its closest neighbors with edges, resulting in a graphG′G^\{\\prime\}\.G′G^\{\\prime\}is then passed to the Leiden algorithm to identify the set of clusters𝒞\\mathcal\{C\}\. Finally, the cluster set𝒞\\mathcal\{C\}, the embeddingsZZ, and the reconstruction lossℒrecon\\mathcal\{L\}\_\{\\text\{recon\}\}are provided as input to the subsequent module\.

### V\-BCluster\-Refined Anomaly Detection \(CRAD\) Module:

The CRAD module is responsible for the detection of collective anomalies\. It takes as input the set of clusters𝒞\\mathcal\{C\}, node embeddingsZZ, and the reconstruction lossℒrecon\\mathcal\{L\}\_\{\\text\{recon\}\}\. It outputs a set of collective anomalies𝒟\\mathcal\{D\}\. The steps of the algorithm are summarized in Algorithm[2](https://arxiv.org/html/2609.22166#algorithm2)\.

The first step is to assign an anomaly score to each cluster\. Formally, for each clusterck∈𝒞c\_\{k\}\\in\\mathcal\{C\}, the cluster\-level anomaly score is defined using a scoring functionSSas the average reconstruction error of its constituent nodes, shown in Eq\.[7](https://arxiv.org/html/2609.22166#S5.E7):

S⁡\(ck\)=1\|ck\|​∑i∈ckLrecon​\(zi\)S\(c\_\{k\}\)=\\frac\{1\}\{\|c\_\{k\}\|\}\\sum\_\{i\\in c\_\{k\}\}L\_\{\\text\{recon\}\}\(z\_\{i\}\)\(7\)
The second step is to separate the large group of normal clusters\. This is achieved using the merging algorithm from\[[34](https://arxiv.org/html/2609.22166#bib.bib34)\], which relies on the pairwise cosine distance between clusters in the embedding space\. Clusters that are close in terms of distance are selected, based on the intuition that the majority of normal clusters tend to be close together in the embedding space\. As a result, the clusters are partitioned into a set of normal clustersℛ⊆𝒞\\mathcal\{R\}\\subseteq\\mathcal\{C\}and a set of candidate anomalous clusters𝒜=𝒞∖ℛ\\mathcal\{A\}=\\mathcal\{C\}\\setminus\\mathcal\{R\}\. The candidate set𝒜\\mathcal\{A\}is not immediately classified as anomalous, since it may still contain some normal clusters, and there is no definitive proof that its members are truly anomalous\. This cautious approach is necessary because collective anomalies can sometimes resemble normal data, meaning that normal clusters may remain within𝒜\\mathcal\{A\}\. The third step involves refining the set of normal clustersℛ\\mathcal\{R\}\. This refinement is crucial for accurately determining the anomaly detection threshold\. Since the anomaly score is based on reconstruction loss, some normal clusters that resemble collective anomalies may exhibit relatively high reconstruction errors\. To address this, a percentile thresholdTαT\_\{\\alpha\}is used, representing a small percentile \(α\\alpha\) of the highest cluster anomaly scores in the candidate set𝒜\\mathcal\{A\}, formulated in Eq\.[8](https://arxiv.org/html/2609.22166#S5.E8):

Tα=percentileα​\(\{S⁡\(c\)∣c∈𝒜\}\)T\_\{\\alpha\}=\\text\{percentile\}\_\{\\alpha\}\\big\(\\\{S\(c\)\\mid c\\in\\mathcal\{A\}\\\}\\big\)\(8\)Using this threshold, a refined set of normal clustersR′R^\{\\prime\}is created by selecting all clusters inℛ\\mathcal\{R\}whose anomaly scores are below theTαT\_\{\\alpha\}threshold as described in Eq\.[9](https://arxiv.org/html/2609.22166#S5.E9):

ℛ′=\{c∈ℛ∣S⁡\(c\)<Tα\}\\mathcal\{R\}^\{\\prime\}=\\\{c\\in\\mathcal\{R\}\\mid S\(c\)<T\_\{\\alpha\}\\\}\(9\)This ensures thatℛ′\\mathcal\{R^\{\\prime\}\}contains only the most confidently normal clusters, which in turn allows for a more reliable calculation of the anomaly detection threshold\.

The fourth step consists of calculating the anomaly detection thresholdτ\\tauto detect collective anomalies\. Given a set of anomaly scores from normal clusters inℛ′:\{s1,s2,…,sn\}\\mathcal\{R^\{\\prime\}\}:\\\{s\_\{1\},s\_\{2\},\\dots,s\_\{n\}\\\}, the probability density function of the normal data is estimated using Kernel Density Estimation \(KDE\) in Eq\.[10](https://arxiv.org/html/2609.22166#S5.E10):

f^​\(s\)=1n​h​∑i=1nK⁡\(s−sih\)\\hat\{f\}\(s\)=\\frac\{1\}\{nh\}\\sum\_\{i=1\}^\{n\}K\\left\(\\frac\{s\-s\_\{i\}\}\{h\}\\right\)\(10\)whereKKis the kernel function, typically chosen as a Gaussian, andhhis the bandwidth controlling the smoothness of the estimate\. BothKKandhhare hyperparameters selected based on prior work\[[29](https://arxiv.org/html/2609.22166#bib.bib29)\]\. The cumulative distribution function \(CDF\) is computed from the estimated density as \(Eq\.[11](https://arxiv.org/html/2609.22166#S5.E11)\) :

F^​\(s\)=∫−∞sf^​\(t\)​𝑑t\\hat\{F\}\(s\)=\\int\_\{\-\\infty\}^\{s\}\\hat\{f\}\(t\)\\,dt\(11\)which represents the proportion of normal observations below a given scoress\. The anomaly thresholdτ\\tauis defined as the smallest value satisfyingF^​\(τ\)≥β\\hat\{F\}\(\\tau\)\\geq\\beta, whereβ\\betais the confidence level chosen according to\[[29](https://arxiv.org/html/2609.22166#bib.bib29)\]at 95%\. In practice,τ\\taucorresponds to the score below which approximately 95% of normal clusters lie, and any cluster with a score exceedingτ\\tauis considered anomalous\.

In the last step, collective anomalies are identified from𝒜\\mathcal\{A\}by evaluating the anomaly scores of each cluster\. A clusterck∈𝒜c\_\{k\}\\in\\mathcal\{A\}is considered a collective anomaly if its score exceeds the previously calculated thresholdτ\\tau:S⁡\(ck\)\>τS\(c\_\{k\}\)\>\\tau, Thus, the set of collective anomalies is formally defined as𝒟=\{c∈𝒜∣S⁡\(c\)\>τ\}\\mathcal\{D\}=\\\{c\\in\\mathcal\{A\}\\mid S\(c\)\>\\tau\\\}\. This approach ensures that clusters with scores significantly higher than the threshold, representing deviations from normal behavior, are flagged as collective anomalies\.

Algorithm 2CRAD: Cluster\-Refined Anomaly DetectionInput:

𝒞\\mathcal\{C\},

ZZ,

ℒrecon\\mathcal\{L\}\_\{\\text\{recon\}\}
Output:

𝒟\\mathcal\{D\}
Step 1: Compute cluster\-level anomaly scores

foreach*ck∈𝒞c\_\{k\}\\in\\mathcal\{C\}*do

Calculate

S⁡\(ck\)S\(c\_\{k\}\)using Eq\.[7](https://arxiv.org/html/2609.22166#S5.E7)

Step 2: Separate normal and candidate anomalous clusters

ℛ←MergeNormalClusters​\(𝒞,Z\)\\mathcal\{R\}\\leftarrow\\text\{MergeNormalClusters\}\(\\mathcal\{C\},Z\)

𝒜=𝒞∖ℛ\\mathcal\{A\}=\\mathcal\{C\}\\setminus\\mathcal\{R\}

Step 3: Refine normal clusters

Calculate

TαT\_\{\\alpha\}using Eq\.[8](https://arxiv.org/html/2609.22166#S5.E8)

Derive

ℛ′\\mathcal\{R\}^\{\\prime\}from Eq\.[9](https://arxiv.org/html/2609.22166#S5.E9)

Step 4: Compute anomaly detection threshold

Compute

F^\\hat\{F\}using Eq\.[10](https://arxiv.org/html/2609.22166#S5.E10)and Eq\.[11](https://arxiv.org/html/2609.22166#S5.E11)

τ←min⁡\{s∣F^​\(s\)≥β\}\\tau\\leftarrow\\min\\\{s\\mid\\hat\{F\}\(s\)\\geq\\beta\\\}

Step 5: Identify collective anomalies

𝒟←\{c∈𝒜∣S⁡\(c\)\>τ\}\\mathcal\{D\}\\leftarrow\\\{c\\in\\mathcal\{A\}\\mid S\(c\)\>\\tau\\\}

Return

𝒟\\mathcal\{D\}

## VIExperiments

### VI\-AExperimental Setup:

The proposed method was implemented in Python using libraries including PyTorch, Pandas, and NumPy\. Computationally intensive operations were accelerated using GPU processing\. The experiments were conducted on a MacBook Air M1 and Google Colab\.

### VI\-BDataset

To evaluate the proposed method, we utilized the CICIoT2023 dataset\[[26](https://arxiv.org/html/2609.22166#bib.bib26)\]and the network ToN\-IoT dataset\[[36](https://arxiv.org/html/2609.22166#bib.bib36)\]\. The CICIoT2023 dataset is a real\-world dataset collected by the Canadian Institute for Cybersecurity\. This dataset comprises network traffic from105105IoT devices and contains3333different types of attacks\. The attacks are categorized into the following classes: DDoS, Mirai, Reconnaissance, Brute Force, Spoofing, DoS, and Web\-based attacks\. The ToN\-IoT network dataset, developed by the Cyber Range Lab at UNSW Canberra, contains network traffic from simulated IoT and IIoT environments\. It includes both normal and malicious network flows collected in a realistic smart setting using tools like Argus and Zeek\. As part of the ToN\_IoT collection, it supports evaluating AI\-based cybersecurity methods and contains cyber\-attacks such as DoS, DDoS, and scanning\.

### VI\-CDataset Pre\-Processing

For dataset preprocessing, irrelevant features were removed, non\-numerical values were converted to numerical ones, and the resulting data were standardized using StandardScaler from scikit\-learn\. From the original CICIoT2023 dataset, we constructed a subset comprising five classes: one benign class, two DDoS attack types \(DDoS\-ICMP\_Flood and DDoS\-UDP\_Flood\), and two Mirai attack types \(Mirai\-UDPPlain and Mirai\-Greeth\_Flood\)\. The rationale for this selection is to retain only collective anomalies \(DDoS and Mirai\), as this study focuses on detecting collective anomalies\. To reflect a realistic scenario, the new dataset is unbalanced; following the work in\[[10](https://arxiv.org/html/2609.22166#bib.bib10)\], we use20%20\\%anomalous data and80%80\\%normal data\.

A subset of the ToN\-IoT network dataset is selected, including benign traffic and attack types such as DoS, DDoS, and scanning, as these represent collective anomalies\.

Since our proposed method requires a graph as input, we employ the k\-NN algorithm to reconstruct graphs from the data points, where each network packet is considered as a node in the generated graph\. To ensure sparsity, a relatively low value ofkkis used, resulting in a graph that is sparse by design\. Sparse graphs are particularly advantageous for training, as they reduce computational complexity and memory requirements\. Furthermore, sparsity enhances scalability, enabling the method to handle larger graphs efficiently\. In this study, we generate three graphs corresponding to different subset sizes from the CICIoT2023 dataset, denoted asG1G\_\{1\},G2G\_\{2\}, andG3G\_\{3\}, to evaluate the scalability of our method\. Using the ToN\-IoT network dataset, four graphsG1′G^\{\\prime\}\_\{1\},G2′G^\{\\prime\}\_\{2\},G3′G^\{\\prime\}\_\{3\}, andG4′G^\{\\prime\}\_\{4\}with different anomaly ratios were constructed to assess the robustness of the proposed method\. Detailed information regarding the number of nodes and edges in each graph is reported in Table[II](https://arxiv.org/html/2609.22166#S6.T2)\.

TABLE II:Statistics about the generated graphs\.
### VI\-DExperiment Reproducibility

For reproducibility, in this part, we provide the parameters of our approach\. Graphs were generated using the k\-NN algorithm implemented with the FAISS library\[[31](https://arxiv.org/html/2609.22166#bib.bib31)\], which efficiently scales k\-NN to large datasets\. The VGAE was not fine\-tuned; all parameters were kept constant across different input graphs\. Specifically, the embedding size was set to 32 for the CICIoT2023 dataset and 64 for ToN\-IoT network dataset, dropout to 0\.2, learning rate to 0\.001, and each model was trained for 200 epochs\. For the first graphGGgenerated from raw data, we setk=5k=5with the Euclidean distance metric\. For the second graphG′G^\{\\prime\}generated from embeddings, we used a higher value forkkwith cosine distance metric, reflecting the higher similarity of the embeddings compared to the raw data\. The confidence level was set toβ=0\.95\\beta=0\.95\. Parameters not explicitly mentioned here were varied in the experiments, and their respective values are reported in the results\.

### VI\-EComparison Methods

To ensure a fair comparison, we conduct three types of evaluation\. First, we compare our proposed method against traditional anomaly detection techniques, including LOF, IF, KNN, and OCSVM, using the CICIoT23 and ToN\-IoT network datasets\. Second, we evaluate our method against clustering\-based CAD approaches on the CICIoT23 dataset, and further assess its scalability\. More specifically, we compare with five representative clustering\-based CAD methods: CADET, HCADET, MCADET, HDM\-GAD, and CCAD\. Third, we perform a robustness study by evaluating our method under varying anomaly ratios and comparing it with CAD methods on the ToN\-IoT network dataset\.

### VI\-FEvaluation Metrics

We evaluate our proposed method on standard DL evaluation metrics such as accuracy, recall, f1\-score, area under the ROC curve \(auc\), and the area under the precision–recall curve \(PR\)\. In addition to these metrics, we used purity, which is defined in Eq\.[12](https://arxiv.org/html/2609.22166#S6.E12)as the extent to which each cluster contains nodes from a single class, in this case, either normal or collective anomaly\.

P​u​r​i​t​y=1N​∑kmaxj⁡\|ck∩tj\|Purity=\\frac\{1\}\{N\}\\sum\_\{k\}\\max\_\{j\}\|c\_\{k\}\\cap t\_\{j\}\|\(12\)
whereNNis the total number of nodes,ckc\_\{k\}is the set of nodes in clusterkk, andtjt\_\{j\}is the set of nodes belonging to classjj\. The value of purity ranges from 0 to 1, with higher values indicating that clusters are more homogeneous with respect to the ground truth\.

## VIIPerformance Evaluation

In this section, we present a series of experiments to evaluate the effectiveness of our proposed method, UGCAD, and to compare it against state\-of\-the\-art approaches\. Specifically, we address the following research questions:

- •RQ1:How does UGCAD perform compared to traditional anomaly detection methods?
- •RQ2:How effective is the EDC module in clustering network traffic data?
- •RQ3:How do hyperparameters influence the performance of the CRAD module?
- •RQ4:How does UGCAD perform in comparison to state\-of\-the\-art CAD methods?
- •RQ5:How does UGCAD perform across diverse datasets, and how robust is it to variations in anomaly ratios?

### VII\-APerformance Comparison with Classical Anomaly Detection Methods\(RQ1\)

TABLE III:The Evaluation Results of UGCAD versus Traditional Anomaly Detection Methods\.A comparative analysis of UGCAD and traditional anomaly detection methods, including KNN, OCSVM, LOF, and IF, is presented in Table[III](https://arxiv.org/html/2609.22166#S7.T3), across two datasets and more specifically on graphsG1G\_\{1\}andG′​1G^\{\\prime\}\{1\}using PR and AUC as evaluation metrics\. Since this part of the evaluation is conducted in a threshold\-independent manner, metrics that require a fixed decision threshold, such as accuracy, are not considered\.

On the CICIoT2023 dataset, the proposed method achieves near\-perfect performance, with both AUC and PR values approaching 1\. This indicates an almost complete separability between normal and anomalous instances, suggesting that the proposed approach is highly effective at capturing the underlying data distribution in this setting\. In contrast, all baseline methods exhibit substantially inferior performance\. Among them, LOF attains the highest scores \(AUC = 0\.754, PR = 0\.560\), yet remains significantly below the proposed method\. The remaining approaches yield AUC values close to random performance and consistently low PR scores, highlighting their limited capability in modeling the structure of this dataset\.

On the more challenging ToN\-IoT network dataset, a noticeable degradation in performance is observed across all methods, reflecting the increased difficulty of the dataset, where normal and anomalous samples exhibit greater overlap\. The proposed method achieves an AUC of 0\.61 and a PR of 0\.557\. Although lower than the results obtained on CICIoT2023, these scores remain the highest among all evaluated methods\. LOF again constitutes the strongest baseline \(AUC = 0\.583, PR = 0\.267\), but a substantial gap persists, particularly in terms of PR\. The generally lower performance across all methods suggests that ToN\-IoT represents a more complex and realistic anomaly detection scenario\.

Overall, these results demonstrate that the proposed method consistently outperforms traditional anomaly detection techniques across both datasets, which answers RQ1\. Furthermore, the observed performance drop on ToN\-IoT dataset underscores the impact of dataset characteristics, particularly class overlap, on detection performance\. Despite these challenges, the proposed approach maintains superior discriminative capability, indicating its robustness and effectiveness in comparison to existing methods\.

### VII\-BPerformance Evaluation of the EDC module \(RQ2\)

After evaluating the proposed method UGCAD against traditional approaches, we proceed to assess its individual components on the CICIoT2023 dataset\. We begin by evaluating the EDC module\.

The first module, EDC, is responsible for generating node embeddings and clustering them into meaningful groups\. The underlying assumption is that node embeddings enhance cluster quality and facilitate the separation of normal data from collective anomalies\. Ideally, each cluster should predominantly contain nodes of the same class, resulting in distinct clusters that represent either collective anomalies or normal data\. To assess the quality of the clustering, we employ the purity metric, which quantifies the extent to which a cluster is dominated by a single class, whether anomalous or normal\. Table[IV](https://arxiv.org/html/2609.22166#S7.T4)presents the purity scores for the three graphs\. As observed, the EDC module effectively separates nodes into clusters with a clear majority class\. This strong separation is crucial for the subsequent module, which performs anomaly detection, to operate effectively\. These results address RQ2 by demonstrating the effectiveness of the EDC module in clustering network traffic data\.

TABLE IV:Performance of the EDC Module w\.r\.t Cluster Purity\.
### VII\-CParameter Analysis \(RQ3\)

Fig\. 2:Experimental results for parameter analysis\. \(a\), \(b\), and \(c\) show the impact of parameterα\\alphaon different metrics for graphsG1G\_\{1\},G2G\_\{2\}, andG3G\_\{3\}, respectively\.
In this part, the parameterα\\alphais analyzed, which represents the percentile of the cluster score in𝒜\\mathcal\{A\}used to refine normal clusters inℛ\\mathcal\{R\}, producingℛ′\\mathcal\{R^\{\\prime\}\}\. Adjustingα\\alphaaims to select the score thresholdTαT\_\{\\alpha\}at whichℛ′\\mathcal\{R^\{\\prime\}\}most accurately reflects the anomaly scores of normal data\. Typically,α\\alphacorresponds to a low percentile, since the objective is not to refine the anomalous clusters inℛ\\mathcal\{R\}to the extent that only the very lowest anomaly scores remain inR′R^\{\\prime\}\. Thus, carefully balancingα\\alphais crucial to ensureℛ′\\mathcal\{R^\{\\prime\}\}effectively captures the distinction between normal and anomalous nodes\.

For the experiments, we varyα\\alphafrom 10% to 50%, as values outside this range are inconsistent with the intended purpose ofα\\alpha\. Fig\.[2](https://arxiv.org/html/2609.22166#S7.F2)illustrates the effect of varyingα\\alphaon accuracy, recall, F1\-score for the graphsG1G\_\{1\},G2G\_\{2\}, andG3G\_\{3\}\. The results indicate that forG1G\_\{1\}andG3G\_\{3\}, the metrics initially start at relatively low values but quickly stabilize at high values exceeding 99%\. In contrast, forG2G\_\{2\}, the metrics begin high at approximately 99% and decline whenα\\alphaexceeds 40%\. Based on these observations, we conclude thatα\\alphavalues of 25% to 40% achieve the best performance across all graphs and this answers RQ3\. We adopt this range for the remainder of the experiments\.

### VII\-DPerformance Comparison \(RQ4\)

![Refer to caption](https://arxiv.org/html/2609.22166v1/figures/cic23_comp/performance_G1.png)\(a\)\(a\)
![Refer to caption](https://arxiv.org/html/2609.22166v1/figures/cic23_comp/performance_G2.png)\(b\)\(b\)
![Refer to caption](https://arxiv.org/html/2609.22166v1/figures/cic23_comp/performance_G3.png)\(c\)\(c\)

Fig\. 3:Performance comparison of different methods on the three graphsG1G\_\{1\},G2G\_\{2\}, andG3G\_\{3\}, respectivelyThis section evaluates the proposed method UGCAD, against other clustering\-based CAD approaches to address RQ4\. Fig\.[3](https://arxiv.org/html/2609.22166#S7.F3)reports the comparison results in terms of accuracy, F1\-score, and recall across the three graphs\. From Fig\.[3](https://arxiv.org/html/2609.22166#S7.F3), we can draw the following conclusions:

1\) UGCAD achieves the best accuracy and F1\-score, all exceeding 99%, and ranks second in recall \(after CADET\) on theG1G\_\{1\}andG3G\_\{3\}graphs\. This directly answers RQ4, confirming the effectiveness of UGCAD in detecting collective anomalies while maintaining a low false alarm rate\.

2\) CADET consistently achieves the best recall across all graphs, indicating strong anomaly detection capability\. However, its F1\-score remains average despite a decent accuracy , suggesting a relatively high false alarm rate\.

3\) HCADET shows comparable performance, with solid accuracy\. On theG2G\_\{2\}dataset, it achieves the highest recall \(over 99%\) after CADET, but again, its average F1\-score indicates a high false alarm rate\.

4\) MCADET and HDM\-GAD deliver reasonable accuracy \(above 0\.75\)\. However, their recall and F1\-scores are consistently low across all graphs, pointing to both a high false positive rate and a low true positive rate, meaning the anomaly detection task was not performed effectively\.

5\) CCAD shows low accuracy and F1\-score but very high recall on all graphs\. This happens because the sliding window method flags most windows as anomalous, detects many collective anomalies \(high recall\), but also misclassifies many normal samples \(low accuracy\)\.

Overall, while most methods attain satisfactory accuracy, this is partly due to the predominance of normal nodes in the graphs\. Recall and F1\-score provide more reliable indicators of anomaly detection quality\. UGCAD not only achieves the highest accuracy but also demonstrates superior recall and F1\-scores \(exceeding 0\.99\), highlighting its scalability and effectiveness for the CAD task\.

### VII\-EGeneralization and Robustness Analysis

Fig\. 4:Performance comparison of CAD methods on the graphsG1′G^\{\\prime\}\_\{1\},G2′G^\{\\prime\}\_\{2\},G3′G^\{\\prime\}\_\{3\}, andG4′G^\{\\prime\}\_\{4\}, respectively\.
After evaluating the performance of UGCAD on the CICIoT2023 dataset with an anomaly ratio of 20% and comparing it against existing CAD methods, we extend the evaluation to the ToN\-IoT network dataset under varying anomaly ratios\. To reflect more realistic scenarios, the proportion of anomalies is varied from 20% to 5%\. Specifically, the graphs \(G1′G^\{\\prime\}\_\{1\}\), \(G2′G^\{\\prime\}\_\{2\}\), \(G3′G^\{\\prime\}\_\{3\}\), and \(G4′G^\{\\prime\}\_\{4\}\) correspond to anomaly ratios of 20%, 15%, 10%, and 5%, respectively\. The corresponding results are presented in Fig\.[4](https://arxiv.org/html/2609.22166#S7.F4)\.

At higher anomaly ratios \(G1′G^\{\\prime\}\_\{1\}, 20%\), UGCAD significantly outperforms all baseline methods in terms of accuracy \(0\.94\) and F1\-score \(0\.79\), while maintaining a high recall \(0\.9\)\. Among the baselines, MCADET achieves relatively high accuracy \(0\.797\), but its extremely low recall \(0\.017\) indicates poor anomaly detection capability\. In contrast, CCAD achieves perfect recall \(1\.0\), but suffers from low accuracy and F1\-score, suggesting a bias toward predicting anomalies\.

As the anomaly ratio decreases to 15% \(G2′G^\{\\prime\}\_\{2\}\), similar trends persist\. UGCAD maintains superior overall performance, achieving the highest accuracy \(0\.94\) and F1\-score \(0\.74\), while preserving a balanced recall \(0\.67\)\. MCADET again shows high accuracy \(0\.848\) but near\-zero recall, confirming its inability to detect rare anomalies\. HCADET provides a more balanced performance among baselines, achieving the highest F1\-score \(0\.714\) in this setting, though still below UGCAD\.

At lower anomaly ratios \(G3′G^\{\\prime\}\_\{3\}, 10% andG4′G^\{\\prime\}\_\{4\}, 5%\), the performance gap becomes more pronounced\. UGCAD remains stable, achieving the highest accuracy and F1\-score across both graphs, and maintaining strong recall \(0\.819 and 0\.938, respectively\)\. In contrast, most baseline methods degrade significantly\. MCADET continues to report high accuracy \(up to 0\.947\), but its recall drops to near zero, indicating that it largely fails to identify anomalies as they become rarer\. HDM\-GAD consistently exhibits poor performance across all metrics and graphs\. CADET shows moderate recall but suffers from very low F1\-scores, reflecting poor precision\.

HCADET remains the most stable baseline, maintaining relatively consistent F1\-scores across graphs; however, it does not match the overall performance of UGCAD\. Results for CCAD degrade as the anomaly ratio decreases on bothG3′G^\{\\prime\}\_\{3\}andG4′G^\{\\prime\}\_\{4\}, suggesting a tendency toward high Recall at the expense of precision\.

In summary, the proposed method demonstrates superior robustness to varying anomaly ratios, maintaining a strong balance between accuracy, recall, and F1\-score\. In contrast, baseline methods either overfit to majority classes or over\-predict anomalies, leading to unstable performance as the anomaly ratio decreases\.

### VII\-FResults Analysis

![Refer to caption](https://arxiv.org/html/2609.22166v1/figures/1M_bold.png)Fig\. 5:Visualization on theG3G\_\{3\}dataset using PCA\.Collective anomalies are not merely aggregations of point anomalies that can be easily separated from normal data\. Instead, they exhibit group\-level properties that make them appear similar to normal samples\. Fig\.[5](https://arxiv.org/html/2609.22166#S7.F5)presents the PCA projection of theG3G\_\{3\}graph, where benign and attack samples show some separation\. However, the overlap among anomalous and normal samples highlights the difficulty of distinguishing collective anomalies from normal data\. This challenge is particularly evident in clustering\-based CAD, which relies heavily on cluster quality\. Poorly formed clusters, lacking purity \(i\.e\., containing samples from multiple classes\), degrade detection performance and lead to weak evaluation metrics\. UGCAD addresses this issue by leveraging node embeddings rather than raw features, enabling more robust clusters\. In contrast, methods that rely purely on statistical properties such as density or variance struggle with CAD, as normal data may exhibit similar properties\. This often results in high false alarm rates, as seen in approaches like HDM\-GAD that relies on cluster density\.

In particular, compared to traditional methods such as KNN, OCSVM, LOF, and IF, UGCAD consistently achieves superior performance on both the CICIoT2023 and ToN\-IoT datasets\. UGCAD obtains significantly higher AUC and PR values, demonstrating a much stronger capability in identifying collective anomalies\. In contrast, traditional approaches exhibit limited effectiveness, especially in terms of PR, as they are primarily designed for point anomaly detection and fail to capture the structural and relational characteristics inherent in graph\-based collective anomalies\.

When compared to existing CAD approaches, several limitations become evident\. MCADET attains relatively high accuracy \(around 0\.75 on CICIoT2023 and up to 0\.85–0\.95 on ToN\-IoT as the anomaly ratio decreases\), yet its recall and F1\-score remain extremely low across all experiments\. This indicates a strong bias toward the majority class, where the method predominantly predicts normal samples and fails to detect anomalies\. This behaviour stems from its multi\-stage clustering strategy, where low\-variance clusters, more likely representing normal behaviour in imbalanced settings, are incorrectly selected as anomalies, leading to deteriorating detection performance despite increasing accuracy\.

CADET, on the other hand, achieves near\-perfect recall on CICIoT2023, but this comes at the cost of moderate accuracy and F1\-score, reflecting a high number of false positives due to its strong bias toward anomaly prediction\. Its performance remains largely invariant to graph size\. However, on ToN\-IoT, its performance degrades significantly as the anomaly ratio decreases, with unstable recall, declining F1\-score, and relatively low accuracy, highlighting its sensitivity to class imbalance\. This is mainly due to its simplistic decision rule of selecting anomalous clusters based on size, which leads to overestimation of anomalies and reduced discrimination when anomalies are sparse\.

HCADET demonstrates more stable behaviour\. On CICIoT2023, recall improves with increasing graph size \(from approximately 0\.80 to nearly 0\.99\), although accuracy and F1\-score remain moderate, indicating a continued bias toward anomaly detection\. On ToN\-IoT, the method maintains relatively stable performance across varying anomaly ratios, with accuracy around 0\.55 and moderate F1\-scores\. This improved robustness is attributed to its use of Hurst\-based statistical ranking of clusters\. However, relying on global statistical properties limits its ability to sharply distinguish anomalies, resulting in moderate yet consistent performance rather than strong detection capability\.

HDM\-GAD exhibits extremely poor performance across both datasets\. On CICIoT2023, all metrics \(accuracy, recall, and F1\-score\) remain close to zero, indicating a complete failure to detect anomalies\. Similarly, on ToN\-IoT, performance remains consistently low across all anomaly ratios, with no noticeable improvement even at higher anomaly levels\. This is due to its reliance on assumptions such as dense micro\-clusters and meaningful sequential patterns for anomalies, which do not hold in network intrusion data that is typically sparse and heterogeneous\. Consequently, its multi\-stage framework fails to isolate anomalous groups and propagates incorrect detections\.

CCAD achieves consistently very high recall \(approximately 0\.93–0\.96\) on CICIoT2023 across all graph sizes, but suffers from low accuracy and F1\-score, indicating excessive false positives due to strong anomaly prediction bias\. Its performance remains stable with respect to dataset size\. On ToN\-IoT, similar behaviour is observed at higher anomaly ratios \(20% and 15%\), where recall is nearly perfect but overall classification balance remains poor\. On lower anomaly ratios \(10% and 5%\), CCAD continues to achieve high recall but other metrics degrade significally\. This behaviour arises from its reliance on detecting clustering structure changes in sliding windows, which can also be triggered by normal variations in dynamic network traffic, leading to over\-detection of anomalies\.

Overall, existing CAD methods either suffer from strong bias toward the majority class \(MCADET\), excessive anomaly prediction \(CADET and CCAD\), limited discriminative power \(HCADET\), or fundamental design mismatches with the data characteristics \(HDM\-GAD\)\. UGCAD consistently achieves superior and balanced performance\. Its ability to effectively capture the structural properties of collective anomalies allows it to outperform both traditional and state\-of\-the\-art CAD approaches across different datasets and varying anomaly conditions\.

## VIIIConclusion and future work

We proposed in this paper UGCAD, a framework for detecting collective anomalies in IoT network traffic\. It consists of two main modules: EDC and CRAD\. EDC employs a VGAE to generate node\-level embeddings, which are then used to enhance the Leiden algorithm for high\-quality clustering of network traffic data\. The CRAD module subsequently aggregates clusters identified as normal to serve as a reference and applies reconstruction error as an anomaly score to distinguish anomalous clusters from normal ones\. The effectiveness of UGCAD was evaluated on the two datasets CICIoT2023 and ToN\-IoT network dataset, and results demonstrated strong performance in both clustering quality and anomaly detection\. Moreover, comparative experiments with several clustering\-based CAD approaches and traditional methods confirm the superior performance of UGCAD\. Despite its advantages, UGCAD has certain limitations\. Its reliance on clustering makes the method highly dependent on the quality of embeddings, and because the framework is clustering\-based, it cannot generalize naturally to unseen or streaming data, limiting its applicability for real\-time anomaly detection on its own\. Future research includes adapting this CAD strategy to naturally work on unseen data and detect collective anomalies in real\-time\.

## References

- \[1\]S\. Qin, S\. Liu, S\. Ye, X\. Fan, M\. Cheng, Y\. He, X\. Deng, and J\. H\. Park, “A Partially Labeled Anomaly Data Detection Approach Based on Prioritized Deep Reinforcement Learning for Consumer Electronics Security,”IEEE Transactions on Consumer Electronics, 2024\.
- \[2\]D\. Javeed, M\. S\. Saeed, I\. Ahmad, P\. Kumar, A\. Jolfaei, and M\. Tahir, “An intelligent intrusion detection system for smart consumer electronics network,”IEEE Transactions on Consumer Electronics, vol\. 69, no\. 4, pp\. 906–913, 2023\.
- \[3\]A\. Ożadowicz, J\. Grela, L\. Wisniewski, and K\. Smok, “Application of the internet of things \(IoT\) technology in consumer electronics\-case study,” in2018 IEEE 23rd International Conference on Emerging Technologies and Factory Automation \(ETFA\), vol\. 1, pp\. 1037–1042, 2018\.
- \[4\]G\. Pang, C\. Shen, L\. Cao, and A\. Van Den Hengel, “Deep learning for anomaly detection: A review,”ACM Computing Surveys \(CSUR\), vol\. 54, no\. 2, pp\. 1–38, 2021\.
- \[5\]V\. Chandola, A\. Banerjee, and V\. Kumar, “Anomaly detection: A survey,”ACM Computing Surveys \(CSUR\), vol\. 41, no\. 3, pp\. 1–58, 2009\.
- \[6\]M\. Ahmed, “Collective anomaly detection techniques for network traffic analysis,”Annals of Data Science, vol\. 5, no\. 4, pp\. 497–512, 2018\.
- \[7\]B\. Xu, J\. Wang, Z\. Zhao, H\. Lin, and F\. Xia, “Unsupervised anomaly detection on attributed networks with graph contrastive learning for consumer electronics security,”IEEE Transactions on Consumer Electronics, vol\. 70, no\. 1, pp\. 4062–4072, 2024\.
- \[8\]M\. Yamauchi, Y\. Ohsita, M\. Murata, K\. Ueda, and Y\. Kato, “Anomaly detection in smart home operation from user behaviors and home conditions,”IEEE Transactions on Consumer Electronics, vol\. 66, no\. 2, pp\. 183–192, 2020\.
- \[9\]M\. Baker, A\. Y\. Fard, H\. Althuwaini, and M\. B\. Shadmand, “Real\-time AI\-based anomaly detection and classification in power electronics dominated grids,”IEEE Journal of Emerging and Selected Topics in Industrial Electronics, vol\. 4, no\. 2, pp\. 549–559, 2022\.
- \[10\]R\. Yu, X\. He, and Y\. Liu, “Glad: group anomaly detection in social media analysis,”ACM Transactions on Knowledge Discovery from Data \(TKDD\), vol\. 10, no\. 2, pp\. 1–22, 2015\.
- \[11\]L\. Xiong, B\. Póczos, and J\. Schneider, “Group anomaly detection using flexible genre models,”Advances in Neural Information Processing Systems, vol\. 24, 2011\.
- \[12\]Z\. Li, Z\. Xiang, W\. Gong, and H\. Wang, “Unified model for collective and point anomaly detection using stacked temporal convolution networks,”Applied Intelligence, vol\. 52, no\. 3, pp\. 3118–3131, 2022\.
- \[13\]Y\. Weng and L\. Liu, “A collective anomaly detection approach for multidimensional streams in mobile service security,”IEEE Access, vol\. 7, pp\. 49157–49168, 2019\.
- \[14\]L\. Bontemps, V\. L\. Cao, J\. McDermott, and N\.\-A\. Le\-Khac, “Collective anomaly detection based on long short\-term memory recurrent neural networks,” inInternational Conference on Future Data and Security Engineering, pp\. 141–152, 2016\.
- \[15\]L\. Van Langendonck, I\. Castell\-Uroz, and P\. Barlet\-Ros, “Towards a graph\-based foundation model for network traffic analysis,” inProceedings of the 3rd GNNet Workshop on Graph Neural Networking Workshop, pp\. 41–45, 2024\.
- \[16\]W\. Song, W\. Dong, and L\. Kang, “Group anomaly detection based on Bayesian framework with genetic algorithm,”Information Sciences, vol\. 533, pp\. 138–149, 2020\.
- \[17\]W\. Shi, G\. Azzopardi, D\. Karastoyanova, and Y\. Huang, “Bidirectional piecewise linear representation of time series with application to collective anomaly detection,”Advanced Engineering Informatics, vol\. 58, p\. 102155, 2023\.
- \[18\]A\. Belhadi, Y\. Djenouri, G\. Srivastava, D\. Djenouri, J\. C\.\-W\. Lin, and G\. Fortino, “Deep learning for pedestrian collective behavior analysis in smart cities: A model of group trajectory outlier detection,”Information Fusion, vol\. 65, pp\. 13–20, 2021\.
- \[19\]R\. Chalapathy, E\. Toth, and S\. Chawla, “Group anomaly detection using deep generative models,” inJoint European Conference on Machine Learning and Knowledge Discovery in Databases, pp\. 173–189, 2018\.
- \[20\]Y\. Feng, J\. Chen, Z\. Liu, H\. Lv, and J\. Wang, “Full graph autoencoder for one\-class group anomaly detection of IIoT system,”IEEE Internet of Things Journal, vol\. 9, no\. 21, pp\. 21886–21898, 2022\.
- \[21\]P\. D’oro, E\. Nasca, J\. Masci, and M\. Matteucci, “Group anomaly detection via graph autoencoders,” inNIPS Workshop, vol\. 2, 2019\.
- \[22\]X\. Ai, J\. Zhou, Y\. Zhu, G\. Li, T\. P\. Michalak, X\. Luo, and K\. Zhou, “Graph anomaly detection at group level: A topology pattern enhanced unsupervised approach,” in2024 IEEE 40th International Conference on Data Engineering \(ICDE\), pp\. 1213–1227, 2024\.
- \[23\]Y\. Mirsky, A\. Shabtai, B\. Shapira, Y\. Elovici, and L\. Rokach, “Anomaly detection for smartphone data streams,”Pervasive and Mobile Computing, vol\. 35, pp\. 83–107, 2017\.
- \[24\]A\. Belhadi, Y\. Djenouri, G\. Srivastava, A\. Cano, and J\. C\.\-W\. Lin, “Hybrid group anomaly detection for sequence data: Application to trajectory data analytics,”IEEE Transactions on Intelligent Transportation Systems, vol\. 23, no\. 7, pp\. 9346–9357, 2021\.
- \[25\]C\. Wang, H\. Zhou, Z\. Hao, S\. Hu, J\. Li, X\. Zhang, B\. Jiang, and X\. Chen, “Network traffic analysis over clustering\-based collective anomaly detection,”Computer Networks, vol\. 205, p\. 108760, 2022\.
- \[26\]E\. C\. P\. Neto, S\. Dadkhah, R\. Ferreira, A\. Zohourian, R\. Lu, and A\. A\. Ghorbani, “CICIoT2023: A real\-time dataset and benchmark for large\-scale attacks in IoT environment,”Sensors, vol\. 23, no\. 13, p\. 5941, 2023\.
- \[27\]M\. Ahmed and A\. N\. Mahmood, “Network traffic pattern analysis using improved information theoretic co\-clustering based collective anomaly detection,” inInternational Conference on Security and Privacy in Communication Systems, pp\. 204–219, 2014\.
- \[28\]M\. Ahmed and A\. N\. Mahmood, “Novel approach for network traffic pattern analysis using clustering\-based collective anomaly detection,”Annals of Data Science, vol\. 2, no\. 1, pp\. 111–130, 2015\.
- \[29\]Y\. Feng, Z\. Liu, J\. Chen, H\. Lv, J\. Wang, and J\. Yuan, “Make the rocket intelligent at IoT edge: Stepwise GAN for anomaly detection of LRE with multisource fusion,”IEEE Internet of Things Journal, vol\. 9, no\. 4, pp\. 3135–3149, 2021\.
- \[30\]V\. A\. Traag, L\. Waltman, and N\. J\. Van Eck, “From Louvain to Leiden: guaranteeing well\-connected communities,”Scientific Reports, vol\. 9, no\. 1, pp\. 1–12, 2019\.
- \[31\]M\. Douze, A\. Guzhva, C\. Deng, J\. Johnson, G\. Szilvasy, P\.\-E\. Mazaré, M\. Lomeli, L\. Hosseini, and H\. Jégou, “The faiss library,”arXiv preprint arXiv:2401\.08281, 2024\.
- \[32\]V\. D\. Blondel, J\.\-L\. Guillaume, R\. Lambiotte, and E\. Lefebvre, “Fast unfolding of communities in large networks,”Journal of Statistical Mechanics: Theory and Experiment, vol\. 2008, no\. 10, p\. P10008, 2008\.
- \[33\]F\. Cao, M\. Estert, W\. Qian, and A\. Zhou, “Density\-based clustering over an evolving data stream with noise,” inProceedings of the 2006 SIAM International Conference on Data Mining, pp\. 328–339, 2006\.
- \[34\]D\. Khettaf, D\. Djenouri, Z\. Rezaeifar, and Y\. Djenouri, “Hybrid Graph Embeddings and Louvain Algorithm for Unsupervised Community Detection,” in2025 10th International Conference on Machine Learning Technologies \(ICMLT\), pp\. 433–438, 2025\.
- \[35\]D\. Khettaf, D\. Djenouri, Z\. Rezaeifar, and Y\. Djenouri, “Deep Learning for Collective Anomaly Detection,”ACM Computing Surveys, 2026, doi: 10\.1145/3788280\.
- \[36\]N\. Moustafa, “The TON\_IoT Datasets,” UNSW Canberra, Cyber Range Lab, 2019\. Available:[https://research\.unsw\.edu\.au/projects/toniot\-datasets](https://research.unsw.edu.au/projects/toniot-datasets)\. Accessed: May 4, 2026\.
- \[37\]F\. T\. Liu, K\. M\. Ting, and Z\.\-H\. Zhou, “Isolation forest,” in2008 Eighth IEEE International Conference on Data Mining, pp\. 413–422, 2008\.
- \[38\]M\. M\. Breunig, H\.\-P\. Kriegel, R\. T\. Ng, and J\. Sander, “LOF: identifying density\-based local outliers,” inProceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp\. 93–104, 2000\.
- \[39\]L\. E\. Peterson, “K\-nearest neighbor,”Scholarpedia, vol\. 4, no\. 2, p\. 1883, 2009\.
- \[40\]H\. J\. Shin, D\.\-H\. Eom, and S\.\-S\. Kim, “One\-class support vector machines—an application in machine fault detection and classification,”Computers & Industrial Engineering, vol\. 48, no\. 2, pp\. 395–408, 2005\.
- \[41\]Q\. Hu, Y\. Wang, Z\. Su, T\. H\. Luan, and R\. Li, “Conwatcher: Towards adaptive and label\-efficient online smart contract analysis in blockchains,” inIEEE INFOCOM 2025—IEEE Conference on Computer Communications, pp\. 1–10, 2025\.
- \[42\]Q\. Hu, Y\. Wang, Z\. Su, T\. H\. Luan, R\. Li, and Z\. Jiang, “Rethinking Online Smart Contract Diagnosis in Blockchains: A Diffusion Perspective,”IEEE Transactions on Networking, 2025\.
- \[43\]Y\. Wang, Q\. Hu, Z\. Li, Z\. Su, R\. Li, X\. Zou, and J\. Zhou, “Blockchain\-envisioned UAV\-aided disaster relief networks: Challenges and solutions,”IEEE Communications Magazine, vol\. 63, no\. 5, pp\. 214–221, 2024\.
- \[44\]S\. Azodolmolkyet al\., “Experimental demonstration of an impairment aware network planning and operation tool for transparent/translucent optical networks,”Journal of Lightwave Technology, vol\. 29, no\. 4, pp\. 439–448, Sep\. 2011\.

![[Uncaptioned image]](https://arxiv.org/html/2609.22166v1/figures/dalila_cropped_2.png)Dalila Khettafis currently a PhD student at the University of the West of England \(UWE\) Bristol, where her research lies at the intersection of artificial intelligence and cybersecurity, with a particular focus on collective anomaly detection\. She has authored several research papers on this topic, published in respected journals and international conferences\. In September 2024, she joined the University of the West of England as an Associate Lecturer, where she teaches a range of undergraduate and postgraduate modules\.![[Uncaptioned image]](https://arxiv.org/html/2609.22166v1/figures/DjamelBio.jpg)Djamel Djenouriis with the University of the West of England, Bristol, UK, where he is leading many funded research projects\. He obtained a Doctorate in Computer Science then habilitation from the USTHB in 2007 and 2011, respectively\. He was an ERCIM postdoctoral fellow at NTNU from 2008 to 2009, then a senior research scientist \(Director of Research\) and deputy director at the CERIST research center\. He also served as an adjunct full professor at Blida University and then at the EMP Polytechnic University\. He is being reported amongst the top 2% most cited scientists in the annual Stanford University releases and ranked in the top 0\.5% of all scholars worldwide by ScholarGPS in his research fields\. He is serving as an expert examiner, TPC member of many international conferences, e\.g\., IEEE ICC, GlobeCom, VTC, ITNAC, WCNC, WiMob, guest editor and a member of the editorial board for many journals such as IEEE Trans\. on Sys\. Man\. and Cybernetics, Future Internet, etc\. He has been invited for delivering keynotes, tutorials, and panelist in many international conferences and events\. He is a senior member of ACM, IEEE, AGYA Academy Alumni life\-member, and a fellow of the UK Higher Education Academy\.![[Uncaptioned image]](https://arxiv.org/html/2609.22166v1/figures/Zeinab-photo.JPG)Zeinab Rezaeifaris a Senior Lecturer at the University of the West of England \(UWE\), UK\. She received her Ph\.D\. in Computer Science and Engineering from Hanyang University, South Korea, in July 2018\. Following her Ph\.D\., she worked as a Postdoctoral Researcher at the Information System Security Laboratory, Korea University, focusing on network security\. Prior to joining UWE, she held a Research Associate position in the School of Computing at Ulster University, UK, where she worked on advanced persistent threat detection\. Her research interests include network security, anomaly detection, security challenges in 5G and beyond, security issues in Vehicular Ad Hoc Networks \(VANETs\), security and privacy in Named Data Networking \(NDN\), and advanced persistent attack detection and reconstruction\.![[Uncaptioned image]](https://arxiv.org/html/2609.22166v1/figures/YoucefDjenouri.jpg)Youcef Djenouriis an Associate Professor at University of South\-Eastern Norway, and a senior researcher at NORCE \(Norwegian Research Center\) from 2023\. He was a research scientist at SINTEF, and a postdoc researcher at NTNU, and SDU\. His research interests include AI, smart city applications, security and privacy\. Dr\. Youcef Djenouri published more than 200 research papers in top conferences and journals such as ICDM, ICDE, AAMAS, ACM KDD, IEEE TIST, IEEE TII, IEEE TCYB, and others\. He is also in the list of 2% most outstanding researchers according to Stanford statistics\. Dr\. Youcef Djenouri is an Associate Editor in IEEE Transactions on Computational Social Systems, Neural Processing Letters, Discover AI journal, and Editorial Board in Applied Intelligence\. He also organized workshops and special sessions in top conferences such as ICDM, KDD, DSAA, IJCNN, and PAKDD\.