Autoregressive Multimodal Anomaly Detection with Large Language Models for Microservice Systems

# Autoregressive Multimodal Anomaly Detection with Large Language Models for Microservice Systems
Source: [https://arxiv.org/html/2610.31722](https://arxiv.org/html/2610.31722)
Fagui Liu[https://orcid.org/0000-0003-1135-4982](https://orcid.org/0000-0003-1135-4982)Guoxiang Zhong[https://orcid.org/0000-0002-3998-7282](https://orcid.org/0000-0002-3998-7282)Xi Yao[https://orcid.org/0009-0006-0204-9934](https://orcid.org/0009-0006-0204-9934)C\.L\. Philip Chen[https://orcid.org/0000-0001-5451-7230](https://orcid.org/0000-0001-5451-7230)††thanks:This work was supported in part by the Guangdong Major Project of Basic and Applied Basic Research under Grant 2019B030302002, in part by the Science and Technology Major Project of Guangzhou under Grant 202007030006, in part by the Science and Technology Project of Guangdong Province under Grant 2021B1111600001, supported by the Major Key Project of PCL \(Grant No\. PCL2025AS13, PCL2024A05, PCL2025A08 and PCL2025A13, PCL2025A11\), in part by the National Natural Science Foundation of China under Grant U24B20151\. \(Corresponding authors: Fagui Liu\) Yixiang Tang, Fagui Liu, Xi Yao, and C\.L\. Philip Chen are with the School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China \(e\-mail: cstyx@mail\.scut\.edu\.cn; fgliu@scut\.edu\.cn; csyaoxi@mail\.scut\.edu\.cn; philip\.chen@ieee\.org\)\. Fagui Liu and Guoxiang Zhong are with the Department of New Networks, Peng Cheng Laboratory, Shenzhen 518000, China \(e\-mail: zhonggx01@pcl\.ac\.cn\)\.††thanks:Manuscript received April 19, 2021; revised August 16, 2021\.

###### Abstract

Microservice systems have become a dominant architecture for large\-scale cloud applications, enabling rapid deployment and elastic scaling\. However, ensuring the reliability and availability of microservice systems critically relies on effective anomaly detection, as failures and performance anomalies can quickly propagate across services and severely impact end\-to\-end system behavior\. Existing multimodal anomaly detection approaches for microservice systems often struggle with two fundamental limitations: \(i\) the neglect of modality\-specific characteristics in favor of cross\-modality fusion, and \(ii\) the “identity\-shortcut” problem that affects the feature reconstruction process\. To address these issues, we propose LLMMAD, an autoregressive multimodal anomaly detection framework for microservice systems based on Large Language Models \(LLMs\)\. Initially, to reconcile cross\-modality fusion with modality\-specific preservation, LLMMAD embeds and recombines multimodal data into a unified sequence for LLM processing to model inter\-modality relationships, while employing modality\-wise reconstruction heads to retain modality\-specific information\. Subsequently, to eliminate the identity\-shortcut, we implement a prediction\-driven autoregressive objective, under which the model predicts future states rather than copies current inputs\. Finally, we conduct extensive experiments on three real\-world datasets\. The results demonstrate that LLMMAD consistently outperforms state\-of\-the\-art methods, achieving F1\-score improvements of 3\.1%, 1\.4%, and 0\.9%, respectively, thereby providing a more reliable solution for microservice monitoring\.

###### Index Terms:

Anomaly Detection, Microservice System, Multimodal Fusion\.

## IIntroduction

Microservice architecture has emerged as a mainstream paradigm for building large\-scale enterprise applications, delivering enhanced flexibility, scalability, and fault isolation compared to monolithic systems\[[1](https://arxiv.org/html/2610.31722#bib.bib1)\]\. In microservice architecture, a complex application is decomposed into a set of loosely coupled, fine\-grained services, each running in its own process and communicating via lightweight mechanisms such as HTTP or message queues\. This design enables independent deployment, rapid iteration, and elastic scaling, enhancing its industrial applicability\.

However, the distributed and dynamic nature of microservice systems introduces significant challenges for ensuring reliability and operational safety\. Failures can propagate across services in subtle ways, making timely and accurate anomaly detection critical yet difficult\. Recognizing the critical role of service reliability, the leading cloud platforms such as eBay\[[2](https://arxiv.org/html/2610.31722#bib.bib16)\]and Microsoft Azure\[[3](https://arxiv.org/html/2610.31722#bib.bib2)\]have heavily invested in advancing anomaly detection technologies for microservice systems\.

In microservice systems, system reliability engineers \(SREs\) collect metric, log, and trace data from running systems as operational data for monitoring\[[4](https://arxiv.org/html/2610.31722#bib.bib3)\]\. Metrics are time\-series data periodically collected by monitoring systems, reflecting both application performance and resource states\. Logs are sequences of semi\-structured events recording service runtime behaviors\. Traces consist of multiple spans, each representing a service invocation and including details such as latency, status code, and operation type\.

Current research has developed numerous anomaly detection algorithms for microservice systems\. However, most existing methods rely primarily on single\-modality data\[[5](https://arxiv.org/html/2610.31722#bib.bib21),[6](https://arxiv.org/html/2610.31722#bib.bib23),[7](https://arxiv.org/html/2610.31722#bib.bib18),[2](https://arxiv.org/html/2610.31722#bib.bib16)\]\. Prior studies\[[8](https://arxiv.org/html/2610.31722#bib.bib8),[9](https://arxiv.org/html/2610.31722#bib.bib10)\]have shown that single\-modality methods fail to capture anomaly signals associated with certain types of faults\. This is because the information conveyed by a single modality is inherently limited and insufficient to represent the full system state\. To overcome the inherent limitations of single\-modal data, recent works\[[10](https://arxiv.org/html/2610.31722#bib.bib17),[11](https://arxiv.org/html/2610.31722#bib.bib4),[12](https://arxiv.org/html/2610.31722#bib.bib5),[13](https://arxiv.org/html/2610.31722#bib.bib12),[14](https://arxiv.org/html/2610.31722#bib.bib7),[8](https://arxiv.org/html/2610.31722#bib.bib8),[15](https://arxiv.org/html/2610.31722#bib.bib9),[9](https://arxiv.org/html/2610.31722#bib.bib10),[16](https://arxiv.org/html/2610.31722#bib.bib11),[17](https://arxiv.org/html/2610.31722#bib.bib6)\]have explored the use of multimodal data to achieve more accurate fault detection\. These methods are shown in Table[I](https://arxiv.org/html/2610.31722#S1.T1)\. While multimodal data provide richer information from multiple sources and offer a more comprehensive view of system health, current methods still exhibit several limitations:

TABLE I:A comprehensive comparison of various multimodal anomaly detection approaches of microservice systems\. “✓\\checkmark” and “×\\times” denote that the corresponding modality is used and not used, respectively\.- •Limitation 1: Neglect of modality\-specific information\.Current multimodal anomaly detection approaches primarily focus on modeling either cross\-modal relationships at the expense of modality\-specific information\. GNN\-based approaches\[[15](https://arxiv.org/html/2610.31722#bib.bib9),[9](https://arxiv.org/html/2610.31722#bib.bib10),[16](https://arxiv.org/html/2610.31722#bib.bib11),[8](https://arxiv.org/html/2610.31722#bib.bib8),[11](https://arxiv.org/html/2610.31722#bib.bib4)\]learn graph representations that inherently entangle modalities, resulting in tightly coupled modality interactions\. Other approaches\[[12](https://arxiv.org/html/2610.31722#bib.bib5),[13](https://arxiv.org/html/2610.31722#bib.bib12),[14](https://arxiv.org/html/2610.31722#bib.bib7),[17](https://arxiv.org/html/2610.31722#bib.bib6),[18](https://arxiv.org/html/2610.31722#bib.bib24)\]preserve separate modality embeddings but ultimately integrate them via fusion methods based on attention, wavelet or merely concatenation, still prioritizing cross\-modal relationships\. Consequently, these methods ignore the modeling of modality\-specific information, potentially overlooking unique anomaly cues that are only evident within individual modalities\.
- •Limitation 2: Susceptibility to identity\-shortcut\.Existing multimodal anomaly detection methods typically rely on temporal feature extractors \(e\.g\., temporal CNNs\[[8](https://arxiv.org/html/2610.31722#bib.bib8),[9](https://arxiv.org/html/2610.31722#bib.bib10),[16](https://arxiv.org/html/2610.31722#bib.bib11),[11](https://arxiv.org/html/2610.31722#bib.bib4),[17](https://arxiv.org/html/2610.31722#bib.bib6),[12](https://arxiv.org/html/2610.31722#bib.bib5)\], LSTMs\[[10](https://arxiv.org/html/2610.31722#bib.bib17),[18](https://arxiv.org/html/2610.31722#bib.bib24)\], GRUs\[[13](https://arxiv.org/html/2610.31722#bib.bib12),[15](https://arxiv.org/html/2610.31722#bib.bib9)\], or self\-attention\[[14](https://arxiv.org/html/2610.31722#bib.bib7)\]\) to learn temporal patterns\. Among these methods, unsupervised approaches usually employ reconstruction architectures for anomaly scoring\. However, reconstruction\-based models\[[19](https://arxiv.org/html/2610.31722#bib.bib36),[20](https://arxiv.org/html/2610.31722#bib.bib37),[21](https://arxiv.org/html/2610.31722#bib.bib43),[22](https://arxiv.org/html/2610.31722#bib.bib13)\]that utilize the aforementioned temporal feature extractors are susceptible to the “identity shortcut” problem, where the decoder reproduces the input disregarding its semantics\. This problem becomes more severe in multimodal settings, as the increased complexity of multimodal data makes it easier to exploit\. As a result, the model fails to produce discriminative representations for abnormal patterns, ultimately undermining the robustness and effectiveness of anomaly detection\.

To systematically address these identified limitations, we propose LLMMAD, an autoregressive multimodal framework that adopts Large language models \(LLMs\) for unsupervised anomaly detection in microservice systems\. More specifically, LLMMAD converts the data of three modalities into a sequence of embeddings arranged in chronological order\. After passing through a LoRA\-adapted\[[23](https://arxiv.org/html/2610.31722#bib.bib14)\]LLM with parameters frozen, embedding\-level predictions are obtained, and then they are reconstructed into the original modalities\. LLMMAD not only analyzes cross\-modality relationships through unified multimodal LLM inference, but also captures modality\-specific information via modality\-wise reconstruction\. Additionally, LLMMAD implements a prediction\-driven autoregressive learning paradigm\. By leveraging the LLM’s causal masking to infer future states from cross\-modal contexts, LLMMAD decouples inputs from target predictions\. This mechanism inherently eliminates the identity\-shortcut and enhances sensitivity to both temporal and inter\-modality anomalies\. In addition, LLMMAD unifies decisions from modalities through the OR\-logic aggregation method, providing fine\-grained anomaly detection for microservice systems\.

In this paper, we make the following contributions:

- •We propose LLMMAD, a LLM\-based multimodal anomaly detection framework for microservice systems\. LLMMAD jointly accounts for both cross\-modality relationship and modality\-specific information, addressing a key limitation of existing multimodal approaches\.
- •We design a prediction\-driven autoregressive learning architecture for LLMMAD\. This paradigm effectively prevents the “identity shortcut” problem that commonly plagues reconstruction\-based anomaly detection methods, leading to more discriminative anomaly representations\.
- •We conduct comprehensive experiments on three real\-world microservice datasets\. The results demonstrate LLMMAD’s superiority over existing anomaly detection approaches for microservice systems\.

## IIRelated Work

In this section, we provide an overview of previous works and discuss their limitations\.

### II\-ACross\-Modality Knowledge Transfer via LLMs

Cross\-modality knowledge transfer denotes the process of aligning non\-textual modalities into a language representation space\. The knowledge learned from pretraining is shared to jointly model heterogeneous modalities and their interactions\.

Recent studies have started to investigate whether LLMs can serve as a general backbone for cross\-modal modeling of non\-textual data\. One\-Fits\-All\[[24](https://arxiv.org/html/2610.31722#bib.bib30)\]extends this idea to time series data by leveraging frozen pretrained transformers to capture complex patterns in time series analysis\. OneLLM\[[25](https://arxiv.org/html/2610.31722#bib.bib46)\]further establishes a unified framework that aligns eight heterogeneous modalities into a language representation space through a universal projection module and a progressive alignment pipeline\.

To the best of our knowledge, existing studies have not yet explored transforming multiple heterogeneous modalities in microservice systems into a unified language representation through cross\-modal transfer\. More importantly, leveraging LLMs for cross\-modal transformation enables heterogeneous metrics, logs, and traces to be mapped into a shared sequential representation space with strong semantic and structural priors\. This is crucial for modeling complex cross\-modal dependencies and the downstream anomaly detection task\.

### II\-B“Identity Shortcut” in Anomaly Detection

Reconstruction\-based anomaly detection relies on the assumption that models trained only on normal data can faithfully reconstruct normal patterns but fail on anomalous regions\. However, such models often suffer from the “identity shortcut” problem, where the reconstruction network optimized under reconstruction loss degenerates into an almost identity mapping and reconstructs both normal and anomalous samples equally well\. You et al\.\[[22](https://arxiv.org/html/2610.31722#bib.bib13)\]provide a systematic analysis of the “identity shortcut” phenomenon and show that commonly used architectures \(e\.g\., MLPs , CNNs, and Transformers\) are inherently prone to converging to trivial identity solutions under reconstruction objectives\.

To mitigate the “Identity Shortcut” issue, existing studies introduce additional constraints or training paradigms to regularize the reconstruction process\. Structural and semantic instructional information\[[26](https://arxiv.org/html/2610.31722#bib.bib35),[19](https://arxiv.org/html/2610.31722#bib.bib36),[20](https://arxiv.org/html/2610.31722#bib.bib37)\]enforce high\-level regularities in reconstruction\. Memory\-based mechanisms\[[27](https://arxiv.org/html/2610.31722#bib.bib38),[28](https://arxiv.org/html/2610.31722#bib.bib39),[29](https://arxiv.org/html/2610.31722#bib.bib40)\]constrain reconstruction to learned normal prototypes\. Iterative refinement\[[30](https://arxiv.org/html/2610.31722#bib.bib41)\]progressively projects inputs toward the normal data manifold through multiple correction steps\. Masking\-based reconstruction\[[31](https://arxiv.org/html/2610.31722#bib.bib42),[32](https://arxiv.org/html/2610.31722#bib.bib49),[33](https://arxiv.org/html/2610.31722#bib.bib48)\]trains models to infer missing parts of the input\. Pseudo\-anomaly synthesis\[[21](https://arxiv.org/html/2610.31722#bib.bib43),[34](https://arxiv.org/html/2610.31722#bib.bib44),[35](https://arxiv.org/html/2610.31722#bib.bib45)\]augments normal data with synthetic anomalies and enforces recovery of clean samples\.

Nevertheless, when LLMs are deployed as reconstructors, they remain susceptible to “identity shortcut” due to their immense parameter space and powerful fitting capabilities\. In contrast to existing reconstruction\-based remedies, our work adopts a prediction\-driven autoregressive learning paradigm, which explicitly decouples the input from the reconstruction target through causal modeling, thereby mitigating “identity shortcut” in LLM\-based anomaly detection\.

### II\-CAnomaly Detection for Microservice Systems

#### II\-C1Single\-Modality Approaches

Previous anomaly detection approaches are usually based on logs\[[36](https://arxiv.org/html/2610.31722#bib.bib20),[5](https://arxiv.org/html/2610.31722#bib.bib21),[37](https://arxiv.org/html/2610.31722#bib.bib22),[38](https://arxiv.org/html/2610.31722#bib.bib51)\]or metrics\[[39](https://arxiv.org/html/2610.31722#bib.bib19),[7](https://arxiv.org/html/2610.31722#bib.bib18),[40](https://arxiv.org/html/2610.31722#bib.bib31)\], targeting traditional distributed systems without complex invocation relationships\. DeepLog\[[36](https://arxiv.org/html/2610.31722#bib.bib20)\]models system logs as sequences of log keys and parameters, learns normal patterns with an LSTM, and detects anomalies via next\-event prediction and parameter deviation\. Donut\[[41](https://arxiv.org/html/2610.31722#bib.bib47)\]models seasonal KPI time series using a variational autoencoder with missing\-data injection and detects anomalies based on reconstruction probability\. Recently, some works\[[2](https://arxiv.org/html/2610.31722#bib.bib16),[6](https://arxiv.org/html/2610.31722#bib.bib23),[42](https://arxiv.org/html/2610.31722#bib.bib34)\]have begun to use distributed traces for microservice anomaly detection\. TraceVAE\[[2](https://arxiv.org/html/2610.31722#bib.bib16)\]employs a dual\-variable graph variational autoencoder to encode trace structures and node execution times, and performs anomaly detection using entropy\-gap–aware negative log\-likelihood scoring\.

Although effective within their respective modalities, these approaches can only detect anomalies that manifest in a single modality\. Consequently, they fail to capture cross\-modal correlations and cascading behaviors across modalities, which are essential for characterizing complex failure patterns\.

#### II\-C2Multi\-Modality Approaches

Graph\-based methods\[[10](https://arxiv.org/html/2610.31722#bib.bib17),[15](https://arxiv.org/html/2610.31722#bib.bib9),[9](https://arxiv.org/html/2610.31722#bib.bib10),[16](https://arxiv.org/html/2610.31722#bib.bib11),[8](https://arxiv.org/html/2610.31722#bib.bib8),[11](https://arxiv.org/html/2610.31722#bib.bib4),[43](https://arxiv.org/html/2610.31722#bib.bib50)\]organize multimodal data into unified graph structures\. MSTGAD\[[16](https://arxiv.org/html/2610.31722#bib.bib11)\]constructs a trace\-centric graph backbone and incorporates metrics and logs as node\-level features\. AnoFusion\[[15](https://arxiv.org/html/2610.31722#bib.bib9)\]treats embedding channels from different modalities as graph nodes and estimates edge weights using mutual information\. Other studies\[[12](https://arxiv.org/html/2610.31722#bib.bib5),[13](https://arxiv.org/html/2610.31722#bib.bib12),[14](https://arxiv.org/html/2610.31722#bib.bib7),[17](https://arxiv.org/html/2610.31722#bib.bib6),[18](https://arxiv.org/html/2610.31722#bib.bib24)\]adopt feature\-level fusion strategies\. Hades\[[12](https://arxiv.org/html/2610.31722#bib.bib5)\]employs modality\-specific feature extractors, Transformers for logs and CNNs for metrics, and subsequently fuses the extracted representations with cross\-modality attention\. FUSION\[[17](https://arxiv.org/html/2610.31722#bib.bib6)\]fuses serialized multimodal data in the wavelet domain and performs probabilistic spatiotemporal modeling with a diffusion network for anomaly detection\.

However, a prevailing limitation across these methodologies is their singular focus on generating a holistic system representation for anomaly detection\. By prioritizing an integrated global state, these models fail to sufficiently scrutinize the distinct information inherent to each modality\. As a result, anomalies that manifest dominantly in a single modality or exhibit modality\-specific signatures may be insufficiently represented in the fused features, which can ultimately limit detection robustness in complex microservice systems\.

## IIIPreliminary

### III\-ASymbol Definition

TABLE II:The symbols related to multimodal data and their descriptions\.Since this paper involves three different modalities, a relatively large number of symbols are used\. To facilitate readability, the symbols related to multimodal data are summarized in Table[II](https://arxiv.org/html/2610.31722#S3.T2)\. Other symbols are explained at the point of their first appearance in the text\.

### III\-BProblem Formulation

For a sliding time windowW=\[t1,t2,⋯,tT\]W=\[t\_\{1\},t\_\{2\},\\cdots,t\_\{T\}\], wheretit\_\{i\}presents theiith timestep of the window andTTis the number of timesteps inWW, the timesteps are equally spaced along the temporal axis\. Monitoring systems continuously or periodically collect multimodal operational data\. The goal of anomaly detection is to identify anomalies occurring at each timestep by jointly leveraging information from all three types of data\.

In the following, we provide a formal description, including the mathematical formulation of each modality and the anomaly detection task within the windowWW\.

Metric\.We consider a set of instances in a microservice system, where each instance represents a container, a node, or a service\. For the metric withinWW, the metrics of all instances form a set of multivariate time series as

ℳ=\{\{𝐦ti\}t=1T\|i=1,…,N\},𝐦ti∈ℝM,\\mathcal\{M\}=\\left\\\{\\\{\\mathbf\{m\}\_\{t\}^\{i\}\\\}\_\{t=1\}^\{T\}\\;\\middle\|\\;i=1,\\ldots,N\\right\\\},\\quad\\mathbf\{m\}\_\{t\}^\{i\}\\in\\mathbb\{R\}^\{M\},\(1\)whereNNis the number of instances,MMis the number of metrics and𝐦ti\\mathbf\{m\}\_\{t\}^\{i\}is the metric vector of theiith instance at timesteptt\.

Log\.A log entry consists of three core components, namely the timestep \(tt\), severity level \(ll\), and event description \(dd\)\. A log sequence within windowWWis represented as

ℒ=\{⟨ti,li,di⟩\}i=1H,\\mathcal\{L\}=\\\{\\langle t\_\{i\},l\_\{i\},d\_\{i\}\\rangle\\\}\_\{i=1\}^\{H\},\(2\)whereHHindicates the total number of log entries inWWand⟨ti,li,di⟩\\langle t\_\{i\},l\_\{i\},d\_\{i\}\\ranglecorresponds to the components of theiith log entry\.

Trace\.Each trace consists of multiple spans\. A span represents an individual service invocation and is defined by several key attributes, including the caller service \(c​rcr\), callee service \(c​ece\), duration \(dd\), start time \(t​sts\), end time \(t​ete\), operation name \(o​pop\) and type \(t​yty\)\. For trace data collected within a time window W, the formal representation is given by

𝒮=\{⟨c​ri,c​ei,di,t​si,t​ei,o​pi,t​yi⟩\}i=1K,\\mathcal\{S\}=\\\{\\langle cr\_\{i\},ce\_\{i\},d\_\{i\},ts\_\{i\},te\_\{i\},op\_\{i\},ty\_\{i\}\\rangle\\\}\_\{i=1\}^\{K\},\(3\)whereKKrepresents the total number of spans in W\.

The task of anomaly detection is considered in a multimodal setting, where the objective is to learn a functionffthat jointly integrates information from three modalities\. This function is formulated as

\(y1,y2,…,yT\)=f⁡\(ℳ,ℒ,𝒮∣θ\),\(y\_\{1\},y\_\{2\},\\ldots,y\_\{T\}\)=f\(\\mathcal\{M\},\\mathcal\{L\},\\mathcal\{S\}\\mid\\theta\),\(4\)whereθ\\thetadenotes the model parameters, andyi∈\{0,1\}y\_\{i\}\\in\\\{0,1\\\}represents the label at theiith timestep, withyi=1y\_\{i\}=1indicating an anomaly andyi=0y\_\{i\}=0indicating a normal state\.

## IVMethod

![Refer to caption](https://arxiv.org/html/2610.31722v1/img/framework.jpg)Fig\. 1:The overall pipeline of LLMMAD\.The framework of LLMMAD is presented in Fig\.[1](https://arxiv.org/html/2610.31722#S4.F1)\. Multimodal monitoring data, including metrics, logs, and traces, are collected from the monitoring system\. The unified input matrices are subsequently fed into the LLM\-based model to obtain anomaly scores and threshold\. For each timestep, alert is triggered when anomaly score exceed the threshold\.

### IV\-ATime Alignment

For common metric collection systems, such as Prometheus111https://github\.com/prometheus/prometheus, metric data are typically sampled at fixed intervals \(e\.g\., every 1 s, 30 s, or 1 min\)\. Based on these regularly sampled metric points, a sliding time windowWWis defined\. The timesteps withinWWare aligned with the sampled data points, so that the metric sequences in each window share identical sampling intervals and have consistent lengths\. In contrast, log data is event driven and log sequences vary in length while trace data consists of spans with variable number\. To enable unified processing by the following model, log and trace sequences need to be aligned with the fixed temporal structure of metric data\.

![Refer to caption](https://arxiv.org/html/2610.31722v1/img/time_align.jpg)Fig\. 2:The time alignment process of multimodal data\.In the following, we present the time alignment operation in detail\. Additionally, Fig\.[2](https://arxiv.org/html/2610.31722#S4.F2)is presented to visually illustrate the process of different modalities\.

#### IV\-A1Metric

To aggregate the metrics from all instances, we concatenate their metric vectors along the metric dimension\. An extended metric time series𝐗metric\\mathbf\{X\}\_\{\\mathrm\{metric\}\}is constructed as

𝐗metric=\{𝐦t\}t=1T∈ℝT×\(N⋅M\),\\displaystyle\\mathbf\{X\}\_\{\\mathrm\{metric\}\}=\\\{\\mathbf\{m\}\_\{t\}\\\}\_\{t=1\}^\{T\}\\in\\mathbb\{R\}^\{T\\times\(N\\cdot M\)\},\(5\)𝐦t=\[𝐦t1;𝐦t2;⋯;𝐦tN\]∈ℝN⋅M,𝐦t\(i\)∈ℝM\\displaystyle\\mathbf\{m\}\_\{t\}=\\left\[\\mathbf\{m\}\_\{t\}^\{1\};\\mathbf\{m\}\_\{t\}^\{2\};\\cdots;\\mathbf\{m\}\_\{t\}^\{N\}\\right\]\\in\\mathbb\{R\}^\{N\\cdot M\},\\mathbf\{m\}\_\{t\}^\{\(i\)\}\\in\\mathbb\{R\}^\{M\}wheremtm\_\{t\}is the concatenated metric vector of all instance at timesteptt\.

#### IV\-A2Log

The time alignment of log data starts by parsing raw log messages into structured templates\. Based on the parsed templates, we represent their temporal occurrence using a binary log occurrence matrix as

𝐎=\{𝐨t\}t=1T∈\{0,1\}T×𝒯,\\displaystyle\\mathbf\{O\}=\\\{\\mathbf\{o\}\_\{t\}\\\}\_\{t=1\}^\{T\}\\in\\\{0,1\\\}^\{T\{\\times\}\\mathcal\{T\}\},\(6\)𝐨t=\[𝐨t,1,𝐨t,2,⋯,𝐨t,𝒯\]∈\{0,1\}𝒯,\\displaystyle\\mathbf\{o\}\_\{t\}=\[\\mathbf\{o\}\_\{t,1\},\\mathbf\{o\}\_\{t,2\},\\cdots,\\mathbf\{o\}\_\{t,\\mathcal\{T\}\}\]\\in\\\{0,1\\\}^\{\\mathcal\{T\}\},where𝒯\\mathcal\{T\}is the number of parsed log templates\. Besides,ot,τo\_\{t,\\tau\}is set to11if a log corresponding to templateτ\\tauappears at timesteptt, and00otherwise\. This representation encodes the presence of log events over time\.

To capture semantic information from log templates, each templateτ\\tauis decomposed into a sequence of words as

τ=\{w1,…,wn\},\\tau=\\\{w\_\{1\},\.\.\.,w\_\{n\}\\\},\(7\)wherewiw\_\{i\}denotes theiith word of templateτ\\tau\. A word corresponds to either an alphabetic string or a placeholder mask\.

Each word is then mapped to aDD\-dimensional vector using Word2Vec\[[44](https://arxiv.org/html/2610.31722#bib.bib15)\]\. After word embedding, the template embedding is obtained via average pooling\. This process is formulated as

𝐞τ=1\|τ\|​∑wi∈τWord2Vec⁡\(wi\)∈ℝD,\\mathbf\{e\}\_\{\\tau\}=\\frac\{1\}\{\|\\tau\|\}\\sum\_\{w\_\{i\}\\in\\tau\}\\mathrm\{Word2Vec\}\(w\_\{i\}\)\\in\\mathbb\{R\}^\{D\},\(8\)where\|τ\|\|\\tau\|is the number of words in the templateτ\\tauandWord2Vec⁡\(⋅\)\\mathrm\{Word2Vec\}\(\\cdot\)presents the Word2Vec function trained using the entire template corpus\. Stacking all template embeddings, the log template embedding matrix𝐄\\mathbf\{E\}can be obtained as

𝐄=\{𝐞τ\}τ=1𝒯∈ℝ𝒯×D,𝐞τ∈ℝD\.\\mathbf\{E\}=\\\{\\mathbf\{e\}\_\{\\tau\}\\\}\_\{\{\\tau\}=1\}^\{\\mathcal\{T\}\}\\in\\mathbb\{R\}^\{\\mathcal\{T\}\{\\times\}D\},\\mathbf\{e\}\_\{\\tau\}\\in\\mathbb\{R\}^\{D\}\.\(9\)
Finally, the semantic and occurrence information of log data are integrated by multiplying the log occurrence matrix and the template embedding matrix as

𝐗log=𝐎×𝐄∈ℝT×D\.\\mathbf\{X\}\_\{\\mathrm\{log\}\}=\\mathbf\{O\}\\times\\mathbf\{E\}\\in\\mathbb\{R\}^\{T\\times D\}\.\(10\)This operation produces an aligned log sequence, where each timestep is represented by aDD\-dimensional feature vector\. The resulting representation is independent of the size of template set and its expansion\.

#### IV\-A3Trace

The span set𝒮\\mathcal\{S\}is transformed into a temporal matrix by aggregating the duration of caller\-callee\-operation group at each timestep\. Multiple statistical characteristics of duration can be selected based on specific circumstances, such as the P99th percentile, P95th percentile, mean, median, and standard deviation\. The time\-aligned span matrix𝐗t​r​a​c​e\\mathbf\{X\}\_\{trace\}is constructed as

𝐗trace=\{𝐬t\}t=1T∈ℝT×\(L⋅G\),𝐬t∈ℝL⋅G,\\mathbf\{X\}\_\{\\mathrm\{trace\}\}=\\\{\\mathbf\{s\}\_\{t\}\\\}\_\{t=1\}^\{T\}\\in\\mathbb\{R\}^\{T\\times\(L\{\\cdot\}G\)\},\\mathbf\{s\}\_\{t\}\\in\\mathbb\{R\}^\{L\{\\cdot\}G\},\(11\)whereGGis the number of caller\-callee\-operation groups,LLis the number of selected statistical characteristics and𝐬t\\mathbf\{s\}\_\{t\}represents the feature vector of timesteptt\.

### IV\-BNormalization

Through time alignment, we obtain the temporally aligned representations of multimodal operational data as

𝐗metric\\displaystyle\\mathbf\{X\}\_\{\\mathrm\{metric\}\}∈ℝT×\(N⋅M\),\\displaystyle\\in\\mathbb\{R\}^\{T\\times\(N\\cdot M\)\},\(12\)𝐗trace\\displaystyle\\mathbf\{X\}\_\{\\mathrm\{trace\}\}∈ℝT×\(L⋅G\),\\displaystyle\\in\\mathbb\{R\}^\{T\\times\(L\\cdot G\)\},𝐗log\\displaystyle\\mathbf\{X\}\_\{\\mathrm\{log\}\}∈ℝT×D,\\displaystyle\\in\\mathbb\{R\}^\{T\\times D\},
Before being fed into the subsequent modules, these matrices must be normalized to ensure numerical stability and balanced feature contributions during model training\. In the following, we use𝐗\\mathbf\{X\}to denote the matrix from each modality\.

However, metric data in microservice systems exhibits intrinsic scale heterogeneity, making normalization non\-trivial\. In particular, metric data often suffer from severe magnitude discrepancies across different variables\. To mitigate this issue, we adopt a multi\-stage normalization strategy\. For metric variables with excessively large magnitudes, we first apply logarithmic compression as

𝐗log=log⁡\(1\+𝐗\),\\mathbf\{X\}^\{\\mathrm\{log\}\}=\\log\(1\+\\mathbf\{X\}\),\(13\)which effectively reduces scale variance while preserving relative variations\. Conversely, for metric variables with extremely small magnitudes, we rescale them by a constant factorα\\alphaas

𝐗scale=α⋅𝐗\.\\mathbf\{X\}^\{\\mathrm\{scale\}\}=\\alpha\\cdot\\mathbf\{X\}\.\(14\)
After scale adjustment, we perform variable\-wise robust normalization for all three modalities\. Specifically, each feature dimension is normalized as

𝐗norm=𝐗−median⁡\(𝐗\)IQR⁡\(𝐗\),\\mathbf\{X\}^\{\\mathrm\{norm\}\}=\\frac\{\\mathbf\{X\}\-\\mathrm\{median\}\(\\mathbf\{X\}\)\}\{\\mathrm\{IQR\}\(\\mathbf\{X\}\)\},\(15\)wheremedian⁡\(⋅\)\\mathrm\{median\}\(\\cdot\)denotes the median andIQR⁡\(⋅\)\\mathrm\{IQR\}\(\\cdot\)represents the interquartile range\. Compared to standard normalization, this robust formulation alleviates the influence of outliers and heavy\-tailed distributions\.

### IV\-CModel Architecture

![Refer to caption](https://arxiv.org/html/2610.31722v1/img/model.jpg)Fig\. 3:The model architecture of LLMMAD\. The light blue, light yellow, and light red blocks denote the intermediate feature matrices of metrics, logs, and traces, respectively\.This section presents the overall architecture of LLMMAD, and introduces each component in sequence to illustrate how the design supports effective anomaly detection \(Fig\.[3](https://arxiv.org/html/2610.31722#S4.F3)\)\.

#### IV\-C1Embedding Layers

The embedding layers in our framework consist of a 1D CNN layer, a linear projection layer, and a layer normalization module\. The 1D CNN layer is used to smooth temporal features by averaging short\-term fluctuations and reducing high\-frequency noise\. This helps capture local sequential patterns while making the temporal features more stable\. The linear layer then projects these smoothed features into a latent space to align their dimensionality with the LLM embeddings\. Finally, layer normalization is applied to stabilize the feature distribution and enhance generalization, mitigating potential issues arising from feature scale variations across different modalities\. Formally, the embedding process can be expressed as

𝐇embmetric=LayerNorm⁡\(𝐖⋅Conv1D⁡\(𝐗metric\)\)∈ℝT×E,\\displaystyle\\mathbf\{H\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{metric\}\}=\\mathrm\{LayerNorm\}\\Big\(\\mathbf\{W\}\\cdot\\mathrm\{Conv1D\}\(\\mathbf\{X\}\_\{\\mathrm\{metric\}\}\)\\Big\)\\in\\mathbb\{R\}^\{T\\times E\},\(16\)𝐇emblog=LayerNorm⁡\(𝐖⋅Conv1D⁡\(𝐗log\)\)∈ℝT×E,\\displaystyle\\mathbf\{H\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{log\}\}=\\mathrm\{LayerNorm\}\\Big\(\\mathbf\{W\}\\cdot\\mathrm\{Conv1D\}\(\\mathbf\{X\}\_\{\\mathrm\{log\}\}\)\\Big\)\\in\\mathbb\{R\}^\{T\\times E\},𝐇embtrace=LayerNorm⁡\(𝐖⋅Conv1D⁡\(𝐗trace\)\)∈ℝT×E,\\displaystyle\\mathbf\{H\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{trace\}\}=\\mathrm\{LayerNorm\}\\Big\(\\mathbf\{W\}\\cdot\\mathrm\{Conv1D\}\(\\mathbf\{X\}\_\{\\mathrm\{trace\}\}\)\\Big\)\\in\\mathbb\{R\}^\{T\\times E\},whereEEis the embedding dimension of the selected LLM,𝐖\\mathbf\{W\}is the linear projection weights,LayerNorm\\mathrm\{LayerNorm\}is the fuction of layer normalization andConv1D\\mathrm\{Conv1D\}is the fuction of 1D CNN\.

#### IV\-C2LLM w/ LoRA

In LLMMAD, the backbone LLM is employed with its original tokenizer, token embeddings, and prediction head removed while only retaining the position embeddings and transformer layers\. All parameters of the LLM are frozen to preserve the knowledge acquired during pre\-training\. LoRA modules\[[23](https://arxiv.org/html/2610.31722#bib.bib14)\]are introduced for efficient fine\-tuning to achieve the bidirectional alignment of pre\-trained semantic space with multimodal embeddings\. In addition, this deisgn significantly reduces the training costs while maintaining generalization capabilities\. Formally, the processing procedure is expressed as

𝐇^=LLMLoRA​\(𝐇\),\\displaystyle\\hat\{\\mathbf\{H\}\}=\\mathrm\{LLM\}\_\{\\text\{LoRA\}\}\(\\mathbf\{H\}\),\(17\)𝐇^∈ℝ3​T×E,𝐇∈ℝ3​T×E,\\displaystyle\\hat\{\\mathbf\{H\}\}\\in\\mathbb\{R\}^\{3T\\times E\},\\mathbf\{H\}\\in\\mathbb\{R\}^\{3T\\times E\},where𝐇^\\hat\{\\mathbf\{H\}\}is the last hidden state of the LLM\. The definition and the detailed acquisition process of input𝐇\\mathbf\{H\}is presented in Section[IV\-C3](https://arxiv.org/html/2610.31722#S4.SS3.SSS3)\. Therefore, LLMMAD simultaneously extracts cross\-modal information and temporal dependency through this component\.

#### IV\-C3Recombination and Inverse Recombination

In the pipeline of LLMMAD, we introduce two novel operations, recombination and inverse recombination\. They respectively perform temporal reorganization of the input and output embeddings of the LLM to support the fusion and separation of modal information\.

Recombination is executed after the embedding layers and the multimodal embeddings are the inputs, which have the same shapeT×ET\\times E\. Specifically, the operation rearranges the embeddings by interleaving modalities along the temporal axis while preserving the original time order\. Without loss of generality, we assume that the modalities are interleaved in the order ofm​e​t​r​i​c,l​o​g,t​r​a​c​emetric,log,trace, and this order is used consistently throughout the paper\. The recombination operation itself is independent of the specific modality order\. Formally, the recombination operation can be expressed as

𝐇=\\displaystyle\\mathbf\{H\}=Recomb⁡\(𝐇embmetric,𝐇emblog,𝐇embtrace\)\\displaystyle\\mathrm\{Recomb\}\(\\mathbf\{H\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{metric\}\},\\mathbf\{H\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{log\}\},\\mathbf\{H\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{trace\}\}\)\(18\)=\\displaystyle=\[𝐡1metric,𝐡1log,𝐡1trace,\\displaystyle\[\\mathbf\{h\}\_\{1\}^\{\\mathrm\{metric\}\},\\mathbf\{h\}\_\{1\}^\{\\mathrm\{log\}\},\\mathbf\{h\}\_\{1\}^\{\\mathrm\{trace\}\},𝐡2metric,𝐡2log,𝐡2trace,\\displaystyle\\mathbf\{h\}\_\{2\}^\{\\mathrm\{metric\}\},\\mathbf\{h\}\_\{2\}^\{\\mathrm\{log\}\},\\mathbf\{h\}\_\{2\}^\{\\mathrm\{trace\}\},…,\\displaystyle\\dots,𝐡Tmetric,𝐡Tlog,𝐡Ttrace\]∈ℝ3​T×E\.\\displaystyle\\mathbf\{h\}\_\{T\}^\{\\mathrm\{metric\}\},\\mathbf\{h\}\_\{T\}^\{\\mathrm\{log\}\},\\mathbf\{h\}\_\{T\}^\{\\mathrm\{trace\}\}\]\\in\\mathbb\{R\}^\{3T\\times E\}\.
The inverse recombination operation is executed after the LLM’s inference and the last hidden state of the LLM is the input\. As the inverse of the recombination operation, this operation disentangles the interleaved sequence by partitioning the embeddings back into their respective modality\-specific streams\. The inverse recombination is formulated as

𝐇^emblog,𝐇^embtrace,𝐇^embmetric=InverseRecomb⁡\(𝐇^\),\\hat\{\\mathbf\{H\}\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{log\}\},\\hat\{\\mathbf\{H\}\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{trace\}\},\\hat\{\\mathbf\{H\}\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{metric\}\}=\\mathrm\{InverseRecomb\}\(\\hat\{\\mathbf\{H\}\}\),\(19\)where the three output embeddings change the order tol​o​g,t​r​a​c​e,m​e​t​r​i​clog,trace,metricand share the shapeT×ET\\times Eand are formulated as

𝐇^emblog=\[𝐡^1log,𝐡^2log,⋯,𝐡^Tlog\]∈ℝT×E,\\displaystyle\\hat\{\\mathbf\{H\}\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{log\}\}=\[\\hat\{\\mathbf\{h\}\}\_\{1\}^\{\\mathrm\{log\}\},\\hat\{\\mathbf\{h\}\}\_\{2\}^\{\\mathrm\{log\}\},\\cdots,\\hat\{\\mathbf\{h\}\}\_\{T\}^\{\\mathrm\{log\}\}\]\\in\\mathbb\{R\}^\{T\\times E\},\(20\)𝐇^embtrace=\[𝐡^1trace,𝐡^2trace,⋯,𝐡^Ttrace\]∈ℝT×E,\\displaystyle\\hat\{\\mathbf\{H\}\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{trace\}\}=\[\\hat\{\\mathbf\{h\}\}\_\{1\}^\{\\mathrm\{trace\}\},\\hat\{\\mathbf\{h\}\}\_\{2\}^\{\\mathrm\{trace\}\},\\cdots,\\hat\{\\mathbf\{h\}\}\_\{T\}^\{\\mathrm\{trace\}\}\]\\in\\mathbb\{R\}^\{T\\times E\},𝐇^embmetric=\[𝐡^1metric,𝐡^2metric,⋯,𝐡^Tmetric\]∈ℝT×E\.\\displaystyle\\hat\{\\mathbf\{H\}\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{metric\}\}=\[\\hat\{\\mathbf\{h\}\}\_\{1\}^\{\\mathrm\{metric\}\},\\hat\{\\mathbf\{h\}\}\_\{2\}^\{\\mathrm\{metric\}\},\\cdots,\\hat\{\\mathbf\{h\}\}\_\{T\}^\{\\mathrm\{metric\}\}\]\\in\\mathbb\{R\}^\{T\\times E\}\.
It is worth noting that the modality order of the inverse recombination output is shifted by one position relative to the recombination input, as each output is the prediction of the next timestep\. Consequently, the order ism​e​t​r​i​c,l​o​g,t​r​a​c​emetric,log,tracein Formula \([18](https://arxiv.org/html/2610.31722#S4.E18)\) whilel​o​g,t​r​a​c​e,m​e​t​r​i​clog,trace,metricin Formula \([19](https://arxiv.org/html/2610.31722#S4.E19)\)\.

In terms of function, the recombination facilitate the fusion of multimodal data while the inverse recombination assists the separation\. In addition, these two operations also make multimodal data compatible with the autoregressive framework of LLMs\.

![Refer to caption](https://arxiv.org/html/2610.31722v1/img/train_and_infer.jpg)Fig\. 4:The timestep correspondence of autogressive prediction\. During training phase, every time step except the first one is predicted\. During detection phase, only a fixed proportion of the latter part of the sliding window is predicted\.
#### IV\-C4Modality\-Wise Reconstruction

The modality\-wise reconstruction component serves as the final stage of LLMMAD\. This component is designed to reconstruct the original data for each modality, ensuring the preservation of modality\-specific features\. By independently recovering the input data of each modality, it guarantees that the unique characteristics of each modality are maintained\.

In particular, embeddings obtained through inverse recombination are passed through three independent linear layers for the reconstruction of each modality respectively\. The modality\-wise reconstruction process is formulated as

𝐗^metric=Linear⁡\(𝐇^embmetric\)∈ℝT×\(N⋅M\),\\displaystyle\\hat\{\\mathbf\{X\}\}^\{\\mathrm\{metric\}\}=\\mathrm\{Linear\}\(\\hat\{\\mathbf\{H\}\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{metric\}\}\)\\in\\mathbb\{R\}^\{T\\times\(N\\cdot M\)\},\(21\)𝐗^log=Linear⁡\(𝐇^emblog\)∈ℝT×D,\\displaystyle\\hat\{\\mathbf\{X\}\}^\{\\mathrm\{log\}\}=\\mathrm\{Linear\}\(\\hat\{\\mathbf\{H\}\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{log\}\}\)\\in\\mathbb\{R\}^\{T\\times D\},𝐗^trace=Linear⁡\(𝐇^embtrace\)∈ℝT×\(L⋅G\)\.\\displaystyle\\hat\{\\mathbf\{X\}\}^\{\\mathrm\{trace\}\}=\\mathrm\{Linear\}\(\\hat\{\\mathbf\{H\}\}\_\{\\mathrm\{emb\}\}^\{\\mathrm\{trace\}\}\)\\in\\mathbb\{R\}^\{T\\times\(L\\cdot G\)\}\.
The reconstructed output serves as the final output of the entire model, and will be used for subsequent loss calculation and anomaly detection\.

### IV\-DOptimization Objective and Anomaly Inference

#### IV\-D1Prediction\-Driven Autoregressive Learning

During the training phase, we employ the prediction\-driven autoregressive learning strategy, as predicting future observations encourages the model to capture meaningful temporal dependencies rather than merely reconstructing inputs\.

To realize this prediction\-driven paradigm, LLMMAD performs one\-step\-ahead prediction for each modality, where the reconstructed output at timestepttis required to predict the input at timestept\+1t\+1\. The one\-step\-ahead relationship between input and output is illustrated in the upper part of Fig\.[4](https://arxiv.org/html/2610.31722#S4.F4), in which the first timestep of the sliding window does not contribute to the loss computation\. The prediction objective is implemented using one\-step\-ahead MSE loss, and the individual losses for each modality are defined as follows:

ℒmetric=1T−1​∑t=1T−1‖𝐱^tmetric−𝐱t\+1metric‖22,\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{metric\}\}=\\frac\{1\}\{T\-1\}\\sum\_\{t=1\}^\{T\-1\}\\big\\\|\\hat\{\\mathbf\{x\}\}\_\{t\}^\{\\mathrm\{metric\}\}\-\\mathbf\{x\}\_\{t\+1\}^\{\\mathrm\{metric\}\}\\big\\\|\_\{2\}^\{2\},\(22\)ℒlog=1T−1​∑t=2T‖𝐱^tlog−𝐱t\+1log‖22,\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{log\}\}=\\frac\{1\}\{T\-1\}\\sum\_\{t=2\}^\{T\}\\big\\\|\\hat\{\\mathbf\{x\}\}\_\{t\}^\{\\mathrm\{log\}\}\-\\mathbf\{x\}\_\{t\+1\}^\{\\mathrm\{log\}\}\\big\\\|\_\{2\}^\{2\},ℒtrace=1T−1​∑t=2T‖𝐱^ttrace−𝐱t\+1trace‖22\.\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{trace\}\}=\\frac\{1\}\{T\-1\}\\sum\_\{t=2\}^\{T\}\\big\\\|\\hat\{\\mathbf\{x\}\}\_\{t\}^\{\\mathrm\{trace\}\}\-\\mathbf\{x\}\_\{t\+1\}^\{\\mathrm\{trace\}\}\\big\\\|\_\{2\}^\{2\}\.
The overall training lossℒ\\mathcal\{L\}is defined as the sum of the losses from all three modalities and is formulated as

ℒ=ℒmetric\+ℒlog\+ℒtrace\.\\mathcal\{L\}=\\mathcal\{L\}\_\{\\mathrm\{metric\}\}\+\\mathcal\{L\}\_\{\\mathrm\{log\}\}\+\\mathcal\{L\}\_\{\\mathrm\{trace\}\}\.\(23\)
This method helps avoid identity\-shortcut behaviors by utilizing the causal masking mechanism in decoder\-only LLMs, which ensures that the prediction of the timestept\+1t\+1can only attend to timestepttand the timestep beforett\. This restriction forces the model to rely on preceding context, preventing the model from simply copying input embeddings\.

Moreover, this design aligns with the autoregressive training paradigm of decoder\-only LLMs\. Such consistency with the model’s learning mechanism reduces fine\-tuning difficulty and overhead, facilitating effective adaptation to downstream modality\-wise reconstruction\.

#### IV\-D2Anomaly Detection with OR\-Logic Aggregation

During the anomaly detection phase, we optimize the computation of the anomaly score\. Due to the separate reconstruction of the three modalities, we introduce OR\-logic aggregation to unify the final anomaly detection results\.

The calculation of MSE loss of the detection phase is slightly different from the training phase\. The specific difference is illustrated in Fig\.[4](https://arxiv.org/html/2610.31722#S4.F4)\. In practice, the sliding window is divided into two parts according to a fixed proportion during detection phase, with the first half providing historical context and the second half used for anomaly detection\. The original windowWWwas divided toWhistoryW\_\{\\mathrm\{history\}\}andWdetectionW\_\{\\mathrm\{detection\}\}as

Whistory=\[t1,t2,…,t⌈γ​T⌉\],\\displaystyle W\_\{\\mathrm\{history\}\}=\\left\[t\_\{1\},t\_\{2\},\\dots,t\_\{\\left\\lceil\\gamma T\\right\\rceil\}\\right\],\(24\)Wdetection=\[t⌈γ​T⌉\+1,t⌈γ​T⌉\+2,…,tT\]\.\\displaystyle W\_\{\\mathrm\{detection\}\}=\\left\[t\_\{\\left\\lceil\\gamma T\\right\\rceil\+1\},t\_\{\\left\\lceil\\gamma T\\right\\rceil\+2\},\\dots,t\_\{T\}\\right\]\.whereγ∈\(0,1\)\\gamma\\in\(0,1\)present the proportion of the length ratio ofWhistoryW\_\{\\mathrm\{history\}\}toWWand is treated as a tunable hyperparameter\.

Anomaly scores are only calculated onWdetectionW\_\{\\mathrm\{detection\}\}, whileWhistoryW\_\{\\mathrm\{history\}\}is not involved\. The input ofWhistoryW\_\{\\mathrm\{history\}\}serves to provide additional context for the model, helping it make more accurate predictions on the detection window\. The specific calculation of anomaly score is as follows:

𝒜𝒮tmetric=∥𝐱^tmetric−𝐱t\+1metric∥22,t=⌈γT⌉,…,T−1;\\displaystyle\\mathcal\{AS\}\_\{t\}^\{\\mathrm\{metric\}\}=\\big\\\|\\hat\{\\mathbf\{x\}\}\_\{t\}^\{\\mathrm\{metric\}\}\-\\mathbf\{x\}\_\{t\+1\}^\{\\mathrm\{metric\}\}\\big\\\|\_\{2\}^\{2\},\\quad t=\\lceil\\gamma T\\rceil,\\dots,T\-1;\(25\)𝒜𝒮tlog=∥𝐱^tlog−𝐱t\+1log∥22,t=⌈γT⌉\+1,…,T;\\displaystyle\\mathcal\{AS\}\_\{t\}^\{\\mathrm\{log\}\}=\\big\\\|\\hat\{\\mathbf\{x\}\}\_\{t\}^\{\\mathrm\{log\}\}\-\\mathbf\{x\}\_\{t\+1\}^\{\\mathrm\{log\}\}\\big\\\|\_\{2\}^\{2\},\\quad t=\\lceil\\gamma T\\rceil\+1,\\dots,T;𝒜𝒮ttrace=∥𝐱^ttrace−𝐱t\+1trace∥22,t=⌈γT⌉\+1,…,T\.\\displaystyle\\mathcal\{AS\}\_\{t\}^\{\\mathrm\{trace\}\}=\\big\\\|\\hat\{\\mathbf\{x\}\}\_\{t\}^\{\\mathrm\{trace\}\}\-\\mathbf\{x\}\_\{t\+1\}^\{\\mathrm\{trace\}\}\\big\\\|\_\{2\}^\{2\},\\quad t=\\lceil\\gamma T\\rceil\+1,\\dots,T\.
By selecting a threshold, the anomaly score is converted into a binary classification for each modality\. A timestep is labeled as anomalous \(1\) if the score exceeds the threshold, and normal \(0\) otherwise\. Thus, for the three modalities in each timestep, we obtain three binary anomaly judgmentsy^tmetric,y^tlog,y^ttrace\\hat\{y\}^\{\\text\{metric\}\}\_\{t\},\\hat\{y\}^\{\\text\{log\}\}\_\{t\},\\hat\{y\}^\{\\text\{trace\}\}\_\{t\}\.

To obtain an overall anomaly decision across modalities, we apply an OR\-logic aggregation strategy\. Formally, the final anomaly labely^t\\hat\{y\}\_\{t\}obtained through OR\-logic aggregation is formulated as

y^t=y^tmetric∨y^tlog∨y^ttrace,\\displaystyle\\hat\{y\}\_\{t\}=\\hat\{y\}^\{\\mathrm\{metric\}\}\_\{t\}\\;\\lor\\;\\hat\{y\}^\{\\mathrm\{log\}\}\_\{t\}\\;\\lor\\;\\hat\{y\}^\{\\mathrm\{trace\}\}\_\{t\},\(26\)y^t,y^tmetric,y^tlog,y^ttrace∈\{0,1\}\.\\displaystyle\\hat\{y\}\_\{t\},\\hat\{y\}^\{\\mathrm\{metric\}\}\_\{t\},\\hat\{y\}^\{\\mathrm\{log\}\}\_\{t\},\\hat\{y\}^\{\\mathrm\{trace\}\}\_\{t\}\\in\\\{0,1\\\}\.
In a nutshell, a timestep is considered anomalous if any of the three modalities is classified as anomalous\. This simple yet effective aggregation strategy chooses to trust the judgment of each modality\.

## VExperiments and Analysis

In this section, we conduct comprehensive experiments to address the following research questions\.

- •RQ1: How accurate is LLMMAD in detecting microservice anomalies?
- •RQ2: How does each component and modality contribute to the effectiveness of LLMMAD?
- •RQ3: How efficient is LLMMAD in terms of parameter scale and time overhead?
- •RQ4: How effectively does LLMMAD mitigate the “identity shortcut” problem?
- •RQ5: How robust is LLMMAD to different LLM backbones and hyperparameter settings?

### V\-ADatasets

We evaluate our method on three publicly available datasets collected from real\-world microservice systems\. Detailed descriptions of the datasets are as follows\.

- •The SN dataset and the TT dataset\[[8](https://arxiv.org/html/2610.31722#bib.bib8)\], are built upon two widely adopted microservice benchmarks\. Specifically, the SN dataset is collected from the SocialNetwork system\[[45](https://arxiv.org/html/2610.31722#bib.bib26)\]and the TT dataset is derived from the TrainTicket system\[[46](https://arxiv.org/html/2610.31722#bib.bib32)\]\. Three kinds of faults, including CPU overload, network delay, and network packet loss, are manually injected into each microservice\.
- •The MSDS dataset\[[47](https://arxiv.org/html/2610.31722#bib.bib25)\]originates from an OpenStack\-based microservice cloud system and consists of two parts\. The former is generated by executing workloads of sequential user requests, and the latter is generated by executing workloads of concurrent user requests\. In our experiments, we use the part generated from concurrent requests\. Three kinds of faults, including container restart faults, service API faults, and network agent disruptions, are injected into the Docker containers\.

TABLE III:Statistics of the datasets used in our experiments, including time duration, instance numbers, metric types, log numbers and span numbers\.Table[III](https://arxiv.org/html/2610.31722#S5.T3)presents the characteristics of the datasets used in our experiments\. These datasets exhibit substantial diversity in time duration, instance number, log, and span volume, covering a wide range of system behaviors and data distributions\. These variability allows us to comprehensively evaluate the robustness and generalization ability of our method under different system scales and temporal characteristics\.

### V\-BExperimental Settings

All methods, including LLMMAD and the baselines, are implemented in PyTorch and evaluated on a Linux server equipped with eight NVIDIA Tesla V100 GPUs\. Drain3\[[48](https://arxiv.org/html/2610.31722#bib.bib28)\]is used for log parsing, followed by Word2Vec trained with the skip\-gram architecture, producing 64\-dimensional embeddings\. GPT\-2 \(117M\)\[[49](https://arxiv.org/html/2610.31722#bib.bib29)\]is adopted as backbone LLM\. All results are averaged over 20 runs\. For anomaly detection, the threshold percentile is selected from\{0\.999,0\.995,0\.99,0\.98,0\.95,0\.90\}\\\{0\.999,0\.995,0\.99,0\.98,0\.95,0\.90\\\}based on validation performance\. Other default hyperparameters are set as follows\.

- •Sliding window length:T=20T=20,
- •Time interval:t=1​st=1\\,\\mathrm\{s\},
- •Batch size:B=256B=256,
- •Partition ratio:γ=0\.5\\gamma=0\.5,
- •Epoch numbers:e​p​o​c​h=10epoch=10,
- •Learning rate:η=0\.01\\eta=0\.01\.

All datasets are split into training, validation, and testing subsets\. For SN and TT datasets with distinct normal and anomalous periods, only normal periods are used for training, while anomalous periods are split into validation and testing subsets with a ratio of 1:3\. For MSDS dataset where anomalies occur throughout the entire timeline, the data are divided into training, validation, and testing sets with a ratio of 6:1:3, and anomalous samples are removed from the training subset\.

### V\-CRQ1: How accurate is LLMMAD in detecting microservice anomalies?

TABLE IV:Performance comparison between LLMMAD and baselines\. The best, second\-best, and third\-best results in each column are marked by,, and, respectively\.To comprehensively assess the anomaly detection accuracy of LLMMAD in microservice systems, we compare it against a diverse set of unimodal and multimodal baselines\. For the sake of fairness, we only select unsupervised methods as the baseline\. The unimodal baselines consist of log\-based method LogAnomaly\[[5](https://arxiv.org/html/2610.31722#bib.bib21)\], metric\-based method AnomalyTransformer\[[40](https://arxiv.org/html/2610.31722#bib.bib31)\], and trace\-based method TraceAnomaly\[[6](https://arxiv.org/html/2610.31722#bib.bib23)\]\. We also opt for several recent multimodal approaches SCWarn\[[18](https://arxiv.org/html/2610.31722#bib.bib24)\], Triple\[[9](https://arxiv.org/html/2610.31722#bib.bib10)\], Deeptralog\[[11](https://arxiv.org/html/2610.31722#bib.bib4)\], and FUSION\[[17](https://arxiv.org/html/2610.31722#bib.bib6)\]that integrate two or more modalities of data\.

Table[IV](https://arxiv.org/html/2610.31722#S5.T4)presents a comprehensive comparison across three standard metrics, including precision \(P\), recall \(R\), and F1\-score \(F1\)\. Overall, LLMMAD achieves the best F1\-score on all three datasets while maintaining a favorable balance between precision and recall, demonstrating its robust anomaly detection performance across diverse microservice environments\. Compared with the previous state\-of\-the\-art method FUSION, LLMMAD further yields F1\-score improvements of 3\.1%, 1\.4%, and 0\.9% on the selected datasets, respectively\.

In addition, a clear performance hierarchy can be observed with respect to modality utilization\. The three\-modality methods \(LLMAD, FUSION, Triple\) consistently outperform those using two modalities, while two\-modality methods \(SCWarn, DeepTralog\) in turn surpass single\-modality approaches \(LogAnomaly, TraceAnomaly, AnomalyTransformer\)\. This trend holds across all datasets, highlighting the importance of comprehensive multi\-modal information for accurately characterizing complex system behaviors and detecting anomalies\. These results further validate the effectiveness of LLMMAD in fully exploiting complementary information from multiple modalities to achieve superior detection accuracy\.

### V\-DRQ2: How does each component and modality contribute to the effectiveness of LLMMAD?

TABLE V:Ablation study on LLMMAD across data modalities and components\.We conduct an ablation study to evaluate the contribution of different components in LLMMAD\. Both modality ablations and component ablations are performed\. The experimental results are summarized in Table[V](https://arxiv.org/html/2610.31722#S5.T5)\.

For modality ablation, we analyze the contribution of each input modality and any combination of them\. Reducing a single modality leads to a noticeable decline in detection performance\. Removing two modalities results in a further degradation, indicating that each modality provides complementary information\. Moreover, across all datasets, excluding the metric modality or any modality combination that includes metric causes the most significant performance drop, highlighting the dominant role of metric data in anomaly analysis\.

For component ablation, we investigate the effect of key architectural and training components by removing or modifying each component independently\. The specific ablation settings are described as follows\.

- •w/o robust embedding: Replace the original embedding modules with a single linear projection layer\.
- •w/o LoRA: Disable LoRA modules\.
- •w/o prediction: Remove the prediction\-driven autogressive mechanism and directly align the model output with the input of the same timestep\.
- •w/o OR\-logic aggregation: Replace the or\-logic aggregation strategy with majority voting\.

The robust embedding layers are introduced to improve the model’s robustness and generalization capability\. As shown in the ablation results, removing this component slightly degrades performance across all datasets\.

Removing the prediction\-driven autoregressive mechanism leads to a noticeable performance degradation\. This confirms that prediction\-based supervision provides a stronger and more informative training signal than direct alignment, as it encourages the model to learn meaningful temporal regularities rather than trivial identity mappings\. Its specific role in alleviating the “identity shortcut” problem is further analyzed in Section[V\-F](https://arxiv.org/html/2610.31722#S5.SS6)\.

When LoRA is disabled, the backbone LLM remains fully frozen\. Although the performance degrades, the model still exhibits non\-trivial anomaly detection capability, indicating that pre\-trained LLMs possess inherent temporal modeling ability\. In this setting, precision is consistently higher than recall, suggesting that the frozen LLM can identify only a limited subset of salient anomalies\. This comparison further demonstrates that introducing a small number of trainable parameters via LoRA is sufficient and necessary to effectively adapt LLMs to multimodal anomaly detection\.

Replacing the OR\-logic aggregation strategy with majority voting causes a dramatic decrease in recall\. Although precision is slightly improved, the overall F1\-score drops significantly, indicating that the voting\-based scheme is overly conservative and leads to a large number of missed anomalies\. This observation also suggests that failures in microservice systems often manifest in only a single modality, rather than simultaneously across multiple modalities, making the OR\-logic aggregation strategy more effective in balancing false positives and false negatives\. This conclusion further demonstrates the necessity of modeling modality\-specific information\.

### V\-ERQ3: How efficient is LLMMAD in terms of parameter scale and time overhead?

Although LLMs provide strong representation capabilities, their large parameter scale raises concerns regarding efficiency in time\-sensitive scenarios\. We therefore analyze the efficiency of LLMMAD in terms of model size as well as training and inference time\.

Parameter statistics are summarized in Table[VI](https://arxiv.org/html/2610.31722#S5.T6)\. Due to different input dimensionalities, the numbers of parameters in the embedding and output layers vary slightly across datasets\. The results reported in Table[VI](https://arxiv.org/html/2610.31722#S5.T6)are based on the SN dataset as a representative example\. Overall, LLMMAD updates approximately 260K parameters, accounting for less than 0\.5% of the total model parameters\.

TABLE VI:Parameter statistics of LLMMAD, including total and trainable parameters of different model components\.![Refer to caption](https://arxiv.org/html/2610.31722v1/img/train_time.png)Fig\. 5:Average per\-sample training time \(in milliseconds\) across datasets for the compared methods\.![Refer to caption](https://arxiv.org/html/2610.31722v1/img/inference_time.png)Fig\. 6:Average per\-sample inference time \(in milliseconds\) across datasets for the compared methods\.![Refer to caption](https://arxiv.org/html/2610.31722v1/img/box.png)Fig\. 7:Boxplots of anomaly scores for LLMMAD and LLMMAD w/o prediction on normal and anomalous samples\.TABLE VII:Performance comparison of LLMMAD with different underlying LLM backbones\. Params denotes the number of model parameters, and Dim denotes the embedding dimension of the LLM\.![Refer to caption](https://arxiv.org/html/2610.31722v1/img/parameter_sensitivity.png)Fig\. 8:The influence of three experimental setups on the performance of the method\.Training and inference efficiency are evaluated using the per\-sample training and inference time shown in Fig\.[5](https://arxiv.org/html/2610.31722#S5.F5)and Fig\.[6](https://arxiv.org/html/2610.31722#S5.F6), respectively, with comparisons against the multimodal baselines Triple and FUSION\. LLMMAD achieves training efficiency comparable to Triple and is substantially faster than FUSION, while its inference time is only slightly higher than both baselines and remains sufficiently low for practical online anomaly detection\. Moreover, unlike the baselines that typically exhibit a large gap between training and inference time, LLMMAD shows only a small difference between the two, indicating that the backpropagation overhead is relatively low\. This behavior is mainly attributed to the small number of trainable parameters in LLMMAD, which reduces the cost of gradient computation\.

### V\-FRQ4: How effectively does LLMMAD mitigate the “identity shortcut” problem?

To further investigate whether the prediction\-driven mechanism helps mitigate the “identity shortcut” problem, we visualize the overall anomaly score \(computed as the average anomaly scores of the three modalities\) of LLMMAD and its ablated variant, LLMMAD w/o prediction, on both normal and anomalous samples across three datasets in Fig\.[7](https://arxiv.org/html/2610.31722#S5.F7)\. For normal samples, the anomaly scores of LLMMAD and LLMMAD w/o prediction are generally similar across all datasets, indicating that removing prediction supervision has limited impact on normal data\. In contrast, for anomalous samples, LLMMAD consistently assigns substantially higher anomaly scores compared to LLMMAD w/o prediction\. Specifically, LLMMAD not only exhibits a higher median score but also shows elevated upper quartiles and maximum values, indicating a more pronounced separation between normal and anomalous points\. Additionally, the overall spread of anomaly scores for LLMMAD on anomalous samples is larger, reflecting its enhanced sensitivity to abnormal patterns\. These results supplement empirical evidence that the prediction\-

### V\-GRQ5: How robust is LLMMAD to different LLM backbones and hyperparameter settings?

To assess the robustness of LLMMAD under different configurations, this subsection conducts a two\-fold analysis\. We first study the effect of adopting different backbone LLMs and then analyze the sensitivity of LLMMAD to variations in key hyperparameters\.

To isolate the influence of the backbone LLM, we compare LLMMAD using several LLM variants\. Besides GPT\-2 with 117M parameters, we also consider other GPT\-2 variants of different sizes, as well as multiple versions of LLaMA3\.2\[[50](https://arxiv.org/html/2610.31722#bib.bib33)\]and Qwen3\[[51](https://arxiv.org/html/2610.31722#bib.bib27)\], covering a wide range of parameter scales\. For a fair comparison, the anomaly score threshold for each model is no longer selected based on the best performance, but is uniformly set to 0\.99\.

The results of comparison between LLMs are summarized in Table[VII](https://arxiv.org/html/2610.31722#S5.T7)\. It is observed that all evaluated LLMs exhibit a certain level of anomaly detection capability\. Overall, the performance differences among LLMs of different types are marginal\. Another notable observation is that for models of the same type but with different parameter scales, increasing the model size generally leads to higher Recall while slightly reducing Precision\. This behavior is likely related to the increased embedding dimensionality associated with larger models\. Higher\-dimensional embeddings capture more fine\-grained data characteristics, but also increase the risk of overfitting, thereby introducing more false positives\. This issue is less evident on the TT dataset, which provides the largest amount of training data and thus better mitigates overfitting\.

We next investigate the sensitivity of LLMMAD to key hyperparameters and experimental settings\. The results are presented in Fig\.[8](https://arxiv.org/html/2610.31722#S5.F8)\. Specifically, we analyze the impact of the sliding window lengthTT, the partition ratioγ\\gamma, and the modality input order on performance\.

Fig\.[8](https://arxiv.org/html/2610.31722#S5.F8)\(a\) reports the impact of the sliding window length on LLMMAD\. Overall, the model shows relatively stable performance when the window length is at least 20 across all three datasets\. In contrast, a consistent and noticeable degradation is observed when the window length falls below this threshold, with the poorest performance at 10\. This indicates that overly short temporal windows hinder the model’s ability to capture sufficiently complete temporal patterns, as critical long\-range dependencies are truncated\.

Fig\.[8](https://arxiv.org/html/2610.31722#S5.F8)\(b\) shows the impact of the partition ratioγ\\gammaon detection performance\. Asγ\\gammaincreases, the F1 scores on all three datasets improve, while the gains become marginal onceγ\\gammaexceeds 50%\. A largerγ\\gammaincorporates a longer history for each detected timestep and thus better captures temporal dependencies, whereas a smallerγ\\gammaenables more timesteps to be reconstructed in parallel and yields higher practical efficiency\. Therefore,γ\\gammashould be selected by balancing detection performance and efficiency\.

Fig[8](https://arxiv.org/html/2610.31722#S5.F8)\(c\) evaluates the influence of modality input order on LLMMAD\. Across all six possible permutations, the performance variations are negligible on all datasets\. This observation demonstrates that LLMMAD is insensitive to modality input order, indicating that the recombination mechanism effectively integrates multimodal information without relying on a specific input order\.

## VIConclusion

In this paper, we propose LLMMAD, an unsupervised framework that leverages LLMs for multimodal anomaly detection in microservice systems\. To reconcile cross\-modal interaction with modality\-specific detail preservation, LLMMAD adopts a hybrid fusion\-decoupling paradigm, in which multimodal embeddings are interleaved for unified LLM processing, while modality\-wise reconstruction heads preserve modality\-specific anomaly cues\. Moreover, to address the “identity shortcut” issue in reconstruction\-based methods, LLMMAD employs a prediction\-driven autoregressive objective that forces the model to infer future states from historical context rather than trivially copying inputs, thereby yielding more discriminative anomaly representations\. Extensive experiments on three real\-world datasets demonstrate that LLMMAD consistently outperforms state\-of\-the\-art approaches\. Overall, this work not only provides a reliable solution for automated microservice anomaly detection\.

## References

- \[1\]S\. Newman\(2021\)Building microservices: designing fine\-grained systems\.” O’Reilly Media, Inc\.”\.Cited by:[§I](https://arxiv.org/html/2610.31722#S1.p1.1)\.
- \[2\]Z\. Xie, H\. Xu, W\. Chen, W\. Li, H\. Jiang, L\. Su, H\. Wang, and D\. Pei\(2023\)Unsupervised anomaly detection on microservice traces through graph vae\.InProceedings of the ACM Web Conference 2023,pp\. 2874–2884\.Cited by:[§I](https://arxiv.org/html/2610.31722#S1.p2.1),[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C1](https://arxiv.org/html/2610.31722#S2.SS3.SSS1.p1.1)\.
- \[3\]X\. Zhang, C\. Du, Y\. Li, Y\. Xu, H\. Zhang, S\. Qin, Z\. Li, Q\. Lin, Y\. Dang, A\. Zhou,et al\.\(2021\)Halo: hierarchy\-aware fault localization for cloud systems\.InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining,pp\. 3948–3958\.Cited by:[§I](https://arxiv.org/html/2610.31722#S1.p2.1)\.
- \[4\]J\. Soldani and A\. Brogi\(2022\)Anomaly detection and failure root cause analysis in \(micro\) service\-based cloud applications: a survey\.ACM Computing Surveys \(CSUR\)55\(3\),pp\. 1–39\.Cited by:[§I](https://arxiv.org/html/2610.31722#S1.p3.1)\.
- \[5\]W\. Meng, Y\. Liu, Y\. Zhu, S\. Zhang, D\. Pei, Y\. Liu, Y\. Chen, R\. Zhang, S\. Tao, P\. Sun,et al\.\(2019\)Loganomaly: unsupervised detection of sequential and quantitative anomalies in unstructured logs\.\.InIJCAI,Vol\.19,pp\. 4739–4745\.Cited by:[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C1](https://arxiv.org/html/2610.31722#S2.SS3.SSS1.p1.1),[§V\-C](https://arxiv.org/html/2610.31722#S5.SS3.p1.1)\.
- \[6\]P\. Liu, H\. Xu, Q\. Ouyang, R\. Jiao, Z\. Chen, S\. Zhang, J\. Yang, L\. Mo, J\. Zeng, W\. Xue,et al\.\(2020\)Unsupervised detection of microservice trace anomalies through service\-level deep bayesian networks\.In2020 IEEE 31st International Symposium on Software Reliability Engineering \(ISSRE\),pp\. 48–58\.Cited by:[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C1](https://arxiv.org/html/2610.31722#S2.SS3.SSS1.p1.1),[§V\-C](https://arxiv.org/html/2610.31722#S5.SS3.p1.1)\.
- \[7\]S\. Tuli, G\. Casale, and N\. R\. Jennings\(2022\)TranAD: deep transformer networks for anomaly detection in multivariate time series data\.Proceedings of the VLDB Endowment15\(6\),pp\. 1201–1214\.Cited by:[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C1](https://arxiv.org/html/2610.31722#S2.SS3.SSS1.p1.1)\.
- \[8\]C\. Lee, T\. Yang, Z\. Chen, Y\. Su, and M\. R\. Lyu\(2023\)Eadro: an end\-to\-end troubleshooting framework for microservices on multi\-source data\.In2023 IEEE/ACM 45th International Conference on Software Engineering \(ICSE\),pp\. 1750–1762\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S1.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[TABLE I](https://arxiv.org/html/2610.31722#S1.T1.2.1.9.1),[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1),[1st item](https://arxiv.org/html/2610.31722#S5.I2.i1.p1.1)\.
- \[9\]R\. Ren, Y\. Wang, F\. Liu, Z\. Li, and G\. Xie\(2023\)Triple: the interpretable deep learning anomaly detection framework based on trace\-metric\-log of microservice\.In2023 IEEE/ACM 31st International Symposium on Quality of Service \(IWQoS\),pp\. 1–10\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S1.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[TABLE I](https://arxiv.org/html/2610.31722#S1.T1.2.1.11.1),[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1),[§V\-C](https://arxiv.org/html/2610.31722#S5.SS3.p1.1)\.
- \[10\]J\. Chen, F\. Liu, J\. Jiang, G\. Zhong, D\. Xu, Z\. Tan, and S\. Shi\(2023\)TraceGra: a trace\-based anomaly detection for microservice using graph deep learning\.Computer Communications204,pp\. 109–117\.Cited by:[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[TABLE I](https://arxiv.org/html/2610.31722#S1.T1.2.1.3.1),[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1)\.
- \[11\]C\. Zhang, X\. Peng, C\. Sha, K\. Zhang, Z\. Fu, X\. Wu, Q\. Lin, and D\. Zhang\(2022\)DeepTraLog: trace\-log combined microservice anomaly detection through graph\-based deep learning\.In2022 IEEE/ACM 44th International Conference on Software Engineering \(ICSE\),pp\. 623–634\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S1.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[TABLE I](https://arxiv.org/html/2610.31722#S1.T1.2.1.4.1),[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1),[§V\-C](https://arxiv.org/html/2610.31722#S5.SS3.p1.1)\.
- \[12\]C\. Lee, T\. Yang, Z\. Chen, Y\. Su, Y\. Yang, and M\. R\. Lyu\(2023\)Heterogeneous anomaly detection for software systems via semi\-supervised cross\-modal attention\.In2023 IEEE/ACM 45th International Conference on Software Engineering \(ICSE\),pp\. 1724–1736\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S1.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[TABLE I](https://arxiv.org/html/2610.31722#S1.T1.2.1.5.1),[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1)\.
- \[13\]P\. Wang, X\. Zhang, Z\. Cao, and Z\. Chen\(2024\)MADMM: microservice system anomaly detection via multi\-modal data and multi\-feature extraction\.Neural Computing and Applications36\(25\),pp\. 15739–15757\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S1.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[TABLE I](https://arxiv.org/html/2610.31722#S1.T1.2.1.6.1),[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1)\.
- \[14\]X\. Lu, Y\. Wang, Y\. Fu, Q\. Sun, X\. Ma, X\. Zheng, and C\. Zhuo\(2024\)MISP: a multimodal\-based intelligent server failure prediction model for cloud computing systems\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 5509–5520\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S1.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[TABLE I](https://arxiv.org/html/2610.31722#S1.T1.2.1.7.1),[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1)\.
- \[15\]C\. Zhao, M\. Ma, Z\. Zhong, S\. Zhang, Z\. Tan, X\. Xiong, L\. Yu, J\. Feng, Y\. Sun, Y\. Zhang,et al\.\(2023\)Robust multimodal failure detection for microservice systems\.InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 5639–5649\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S1.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[TABLE I](https://arxiv.org/html/2610.31722#S1.T1.2.1.10.1),[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1)\.
- \[16\]J\. Huang, Y\. Yang, H\. Yu, J\. Li, and X\. Zheng\(2023\)Twin graph\-based anomaly detection via attentive multi\-modal learning for microservice system\.In2023 38th IEEE/ACM International Conference on Automated Software Engineering \(ASE\),pp\. 66–78\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S1.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[TABLE I](https://arxiv.org/html/2610.31722#S1.T1.2.1.12.1),[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1)\.
- \[17\]K\. Ding, Y\. Ma, Z\. Song, and K\. Bian\(2025\)Enhancing microservices anomaly detection via multimodal data fusion in the wavelet domain and spatiotemporal graph\-based diffusion probabilistic model\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 2,pp\. 510–520\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S1.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[TABLE I](https://arxiv.org/html/2610.31722#S1.T1.2.1.13.1),[§I](https://arxiv.org/html/2610.31722#S1.p4.1),[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1),[§V\-C](https://arxiv.org/html/2610.31722#S5.SS3.p1.1)\.
- \[18\]N\. Zhao, J\. Chen, Z\. Yu, H\. Wang, J\. Li, B\. Qiu, H\. Xu, W\. Zhang, K\. Sui, and D\. Pei\(2021\)Identifying bad software changes via multimodal anomaly detection for online service systems\.InProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering,pp\. 527–539\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S1.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[TABLE I](https://arxiv.org/html/2610.31722#S1.T1.2.1.8.1),[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1),[§V\-C](https://arxiv.org/html/2610.31722#S5.SS3.p1.1)\.
- \[19\]Y\. Shi, J\. Yang, and Z\. Qi\(2021\)Unsupervised anomaly segmentation via deep feature reconstruction\.Neurocomputing424,pp\. 9–22\.Cited by:[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[20\]Y\. Xia, Y\. Zhang, F\. Liu, W\. Shen, and A\. L\. Yuille\(2020\)Synthesize then compare: detecting failures and anomalies for semantic segmentation\.InEuropean conference on computer vision,pp\. 145–161\.Cited by:[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[21\]M\. Pourreza, B\. Mohammadi, M\. Khaki, S\. Bouindour, H\. Snoussi, and M\. Sabokrou\(2021\)G2d: generate to detect anomaly\.InProceedings of the IEEE/CVF winter conference on applications of computer vision,pp\. 2003–2012\.Cited by:[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[22\]Z\. You, L\. Cui, Y\. Shen, K\. Yang, X\. Lu, Y\. Zheng, and X\. Le\(2022\)A unified model for multi\-class anomaly detection\.Advances in Neural Information Processing Systems35,pp\. 4571–4584\.Cited by:[2nd item](https://arxiv.org/html/2610.31722#S1.I1.i2.p1.1),[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p1.1)\.
- \[23\]E\. J\. Hu, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, W\. Chen,et al\.\(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,Cited by:[§I](https://arxiv.org/html/2610.31722#S1.p6.1),[§IV\-C2](https://arxiv.org/html/2610.31722#S4.SS3.SSS2.p1.1)\.
- \[24\]T\. Zhou, P\. Niu, L\. Sun, R\. Jin,et al\.\(2023\)One fits all: power general time series analysis by pretrained lm\.Advances in neural information processing systems36,pp\. 43322–43355\.Cited by:[§II\-A](https://arxiv.org/html/2610.31722#S2.SS1.p2.1)\.
- \[25\]J\. Han, K\. Gong, Y\. Zhang, J\. Wang, K\. Zhang, D\. Lin, Y\. Qiao, P\. Gao, and X\. Yue\(2024\)Onellm: one framework to align all modalities with language\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 26584–26595\.Cited by:[§II\-A](https://arxiv.org/html/2610.31722#S2.SS1.p2.1)\.
- \[26\]K\. Zhou, Y\. Xiao, J\. Yang, J\. Cheng, W\. Liu, W\. Luo, Z\. Gu, J\. Liu, and S\. Gao\(2020\)Encoding structure\-texture relation with p\-net for anomaly detection in retinal images\.InEuropean conference on computer vision,pp\. 360–377\.Cited by:[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[27\]D\. Gong, L\. Liu, V\. Le, B\. Saha, M\. R\. Mansour, S\. Venkatesh, and A\. v\. d\. Hengel\(2019\)Memorizing normality to detect anomaly: memory\-augmented deep autoencoder for unsupervised anomaly detection\.InProceedings of the IEEE/CVF international conference on computer vision,pp\. 1705–1714\.Cited by:[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[28\]J\. Hou, Y\. Zhang, Q\. Zhong, D\. Xie, S\. Pu, and H\. Zhou\(2021\)Divide\-and\-assemble: learning block\-wise memory for unsupervised anomaly detection\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 8791–8800\.Cited by:[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[29\]H\. Park, J\. Noh, and B\. Ham\(2020\)Learning memory\-guided normality for anomaly detection\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 14372–14381\.Cited by:[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[30\]D\. Dehaene, O\. Frigo, S\. Combrexelle, and P\. Eline\(2020\)Iterative energy\-based projection on a normal data manifold for anomaly localization\.arXiv preprint arXiv:2002\.03734\.Cited by:[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[31\]X\. Yan, H\. Zhang, X\. Xu, X\. Hu, and P\. Heng\(2021\)Learning semantic context from normal samples for unsupervised anomaly detection\.InProceedings of the AAAI conference on artificial intelligence,Vol\.35,pp\. 3110–3118\.Cited by:[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[32\]Y\. Fu and F\. Xue\(2022\)MAD: self\-supervised masked anomaly detection task for multivariate time series\.In2022 International Joint Conference on Neural Networks \(IJCNN\),Vol\.,pp\. 1–8\.Cited by:[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[33\]Y\. Fang, J\. Xie, Y\. Zhao, L\. Chen, Y\. Gao, and K\. Zheng\(2024\)Temporal\-frequency masked autoencoders for time series anomaly detection\.In2024 IEEE 40th International Conference on Data Engineering \(ICDE\),Vol\.,pp\. 1228–1241\.Cited by:[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[34\]A\. Collin and C\. De Vleeschouwer\(2021\)Improved anomaly detection by training an autoencoder with skip connections on images corrupted with stain\-shaped noise\.In2020 25th International Conference on Pattern Recognition \(ICPR\),pp\. 7915–7922\.Cited by:[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[35\]V\. Zavrtanik, M\. Kristan, and D\. Skočaj\(2021\)Draem\-a discriminatively trained reconstruction embedding for surface anomaly detection\.InProceedings of the IEEE/CVF international conference on computer vision,pp\. 8330–8339\.Cited by:[§II\-B](https://arxiv.org/html/2610.31722#S2.SS2.p2.1)\.
- \[36\]M\. Du, F\. Li, G\. Zheng, and V\. Srikumar\(2017\)Deeplog: anomaly detection and diagnosis from system logs through deep learning\.InProceedings of the 2017 ACM SIGSAC conference on computer and communications security,pp\. 1285–1298\.Cited by:[§II\-C1](https://arxiv.org/html/2610.31722#S2.SS3.SSS1.p1.1)\.
- \[37\]H\. Guo, S\. Yuan, and X\. Wu\(2021\)Logbert: log anomaly detection via bert\.In2021 international joint conference on neural networks \(IJCNN\),pp\. 1–8\.Cited by:[§II\-C1](https://arxiv.org/html/2610.31722#S2.SS3.SSS1.p1.1)\.
- \[38\]Y\. Fu, K\. Liang, and J\. Xu\(2023\)Mlog: mogrifier lstm\-based log anomaly detection approach using semantic representation\.IEEE Transactions on Services Computing16\(5\),pp\. 3537–3549\.Cited by:[§II\-C1](https://arxiv.org/html/2610.31722#S2.SS3.SSS1.p1.1)\.
- \[39\]G\. Zhong, F\. Liu, J\. Jiang, and C\. P\. Chen\(2023\)CauseFormer: interpretable anomaly detection with stepwise attention for cloud service\.IEEE Transactions on Network and Service Management21\(1\),pp\. 637–652\.Cited by:[§II\-C1](https://arxiv.org/html/2610.31722#S2.SS3.SSS1.p1.1)\.
- \[40\]J\. Xu, H\. Wu, J\. Wang, and M\. Long\(2021\)Anomaly transformer: time series anomaly detection with association discrepancy\.InInternational Conference on Learning Representations,Cited by:[§II\-C1](https://arxiv.org/html/2610.31722#S2.SS3.SSS1.p1.1),[§V\-C](https://arxiv.org/html/2610.31722#S5.SS3.p1.1)\.
- \[41\]Z\. Li, W\. Chen, and D\. Pei\(2018\)Robust and unsupervised kpi anomaly detection based on conditional variational autoencoder\.In2018 IEEE 37th International Performance Computing and Communications Conference \(IPCCC\),pp\. 1–9\.Cited by:[§II\-C1](https://arxiv.org/html/2610.31722#S2.SS3.SSS1.p1.1)\.
- \[42\]K\. Zhang, C\. Zhang, X\. Peng, and C\. Sha\(2022\)Putracead: trace anomaly detection with partial labels based on gnn and pu learning\.In2022 IEEE 33rd International Symposium on Software Reliability Engineering \(ISSRE\),pp\. 239–250\.Cited by:[§II\-C1](https://arxiv.org/html/2610.31722#S2.SS3.SSS1.p1.1)\.
- \[43\]S\. He, K\. Feng, K\. Meng, K\. Xie, and X\. Zhao\(2026\)Multimodal anomaly detection for microservice systems via grassmann manifolds\-based graph fusion\.IEEE Transactions on Services Computing\.Cited by:[§II\-C2](https://arxiv.org/html/2610.31722#S2.SS3.SSS2.p1.1)\.
- \[44\]Q\. Le and T\. Mikolov\(2014\)Distributed representations of sentences and documents\.InInternational conference on machine learning,pp\. 1188–1196\.Cited by:[§IV\-A2](https://arxiv.org/html/2610.31722#S4.SS1.SSS2.p3.1)\.
- \[45\]Y\. Gan, Y\. Zhang, D\. Cheng, A\. Shetty, P\. Rathi, N\. Katarki, A\. Bruno, J\. Hu, B\. Ritchken, B\. Jackson,et al\.\(2019\)An open\-source benchmark suite for microservices and their hardware\-software implications for cloud & edge systems\.InProceedings of the Twenty\-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems,pp\. 3–18\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S5.I2.i1.p1.1)\.
- \[46\]X\. Zhou, X\. Peng, T\. Xie, J\. Sun, C\. Xu, C\. Ji, and W\. Zhao\(2018\)Benchmarking microservice systems for software engineering research\.InProceedings of the 40th International Conference on Software Engineering: Companion Proceeedings,pp\. 323–324\.Cited by:[1st item](https://arxiv.org/html/2610.31722#S5.I2.i1.p1.1)\.
- \[47\]S\. Nedelkoski, J\. Bogatinovski, A\. K\. Mandapati, S\. Becker, J\. Cardoso, and O\. Kao\(2020\)Multi\-source distributed system data for ai\-powered analytics\.InService\-Oriented and Cloud Computing: 8th IFIP WG 2\.14 European Conference, ESOCC 2020, Heraklion, Crete, Greece, September 28–30, 2020, Proceedings 8,pp\. 161–176\.Cited by:[2nd item](https://arxiv.org/html/2610.31722#S5.I2.i2.p1.1)\.
- \[48\]P\. He, J\. Zhu, Z\. Zheng, and M\. R\. Lyu\(2017\)Drain: an online log parsing approach with fixed depth tree\.In2017 IEEE international conference on web services \(ICWS\),pp\. 33–40\.Cited by:[§V\-B](https://arxiv.org/html/2610.31722#S5.SS2.p1.1)\.
- \[49\]A\. Radford, J\. Wu, R\. Child, D\. Luan, D\. Amodei, I\. Sutskever,et al\.\(2019\)Language models are unsupervised multitask learners\.OpenAI blog1\(8\),pp\. 9\.Cited by:[§V\-B](https://arxiv.org/html/2610.31722#S5.SS2.p1.1)\.
- \[50\]A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Yang, A\. Fan,et al\.\(2024\)The llama 3 herd of models\.arXiv e\-prints,pp\. arXiv–2407\.Cited by:[§V\-G](https://arxiv.org/html/2610.31722#S5.SS7.p2.1)\.
- \[51\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§V\-G](https://arxiv.org/html/2610.31722#S5.SS7.p2.1)\.