Time-Series Retrieval for Grounding Multimodal Language Models in Remaining Useful Life

# Time‑Series Retrieval for Grounding Multimodal Language Models in Remaining Useful Life Prediction
Source: [https://arxiv.org/html/2608.19218](https://arxiv.org/html/2608.19218)
Valeriu Dimidov and Raphaël FrankInterdisciplinary Centre for Security, Reliability and Trust \(SnT\) University of Luxembourg 29 Avenue J\.F\. Kennedy L\-1855, Luxembourg

###### Abstract

Large language models \(LLMs\) and agentic AI systems are increasingly being explored for domain\-specific maintenance and prognostics tasks, raising the question of whether they can effectively support prognostics and health management \(PHM\)\. In this paper, we investigate remaining useful life \(RUL\) estimation with multimodal large language models \(MLLMs\) grounded through time\-series retrieval\. We propose a framework in which historically similar degradation segments are retrieved from the training set and, together with the test trajectory, transformed into a visual comparison artifact that is processed by the MLLM through a structured multimodal prompt\. The approach is evaluated on the FD001 partition of the C\-MAPSS benchmark under repeated experiments comparing retrieval\-based inference against a non\-retrieval baseline based on random reference selection\. The results show that time\-series retrieval consistently improves MLLM\-based RUL prediction across the evaluated models, yielding lower error and more stable performance\. At the same time, the magnitude of the benefit depends on model capacity, indicating that retrieval is most effective when the underlying MLLM is able to exploit the retrieved evidence\. Overall, the study shows that time\-series RAG is a promising mechanism for improving multimodal prognostic reasoning, while also highlighting the current limitations of MLLM\-based RUL estimation in practical PHM settings\.

\\phmLicenseFootnote

FirstAuthorFirstName FirstAuthorLastName

## 1Introduction

Predictive Maintenance \(PdM\) aims to reduce downtime, maintenance costs, and operational risks by anticipating failures before they occur\. A key task in this context is Remaining Useful Life \(RUL\) estimation, which provides an estimate of the time left before a component reaches failure\. Accurate RUL estimation supports maintenance planning, resource allocation, and risk\-aware decision making in Prognostics and Health Management \(PHM\)\.

Conventional RUL estimation methods are typically based on statistical degradation models, physics\-informed approaches, or data\-driven architectures trained on sensor time series\. Although these methods can achieve strong predictive performance, they usually operate only on numerical data and provide limited support for reasoning over external contextual knowledge\.

Recent advances in large language models \(LLMs\) and multimodal large language models \(MLLMs\) create an opportunity to revisit RUL estimation from a different perspective\(?, ?, ?\)\. Instead of relying only on a learned numerical predictor, an MLLM can be prompted with visual and textual evidence describing the target trajectory\. Retrieval\-augmented generation \(RAG\) can further enrich this input by selecting historical degradation segments that are similar to the query trajectory, following the broader idea of grounding language\-model generation with retrieved external evidence\(?, ?\)\. However, in the RUL setting, the effect of such retrieval is not obvious: relevant references may ground the model on useful degradation evidence, whereas poorly matched references may introduce noise or bias the prediction\.

Therefore, this paper studies whether time\-series retrieval improves MLLM\-based RUL estimation\. We propose a framework in which similar degradation segments are retrieved from historical run\-to\-failure trajectories and combined with the query trajectory into a visual comparison artifact\. The same multimodal prompt structure is then used to compare retrieval\-based inference with a non\-retrieval baseline based on random reference selection on the FD001 partition of the C\-MAPSS benchmark\(?, ?, ?\)\.

The main contributions of this study are:

- •A multimodal framework that reformulates RUL estimation as an evidence\-grounded visual reasoning task for MLLMs\.
- •A time\-series RAG mechanism that retrieves historical degradation segments and presents them to the MLLM as trajectory\-level reference evidence\.
- •A controlled experimental comparison between retrieval\-based and random\-reference inference on the FD001 partition of the C\-MAPSS benchmark\.
- •An analysis of the benefits, limitations, and failure modes associated with retrieval augmentation in the RUL estimation setting\.

The remainder of the paper is organized as follows: Section[2](https://arxiv.org/html/2608.19218#S2)reviews related work, Section[3](https://arxiv.org/html/2608.19218#S3)presents the methodology, Section[4](https://arxiv.org/html/2608.19218#S4)reports the experimental results, Section[5](https://arxiv.org/html/2608.19218#S5)discusses the main findings and limitations, and Section[6](https://arxiv.org/html/2608.19218#S6)concludes the paper\.

![Refer to caption](https://arxiv.org/html/2608.19218v1/img/cmapss_preprocessing_pipeline.png)Figure 1:Preprocessing workflow applied to the FD001 partition of the C\-MAPSS dataset\.Table 1:Overview of the main pipeline modules according to their inputs, transformations, and outputs\.
## 2Related Work

LLMs have started to attract attention in time\-series analysis\. Recent survey studies indicate that this line of research mainly follows several directions, including direct prompting, time\-series quantization into token\-like representations, alignment of numerical sequences with language\-model spaces, the use of visual representations as an intermediate reasoning interface, and the integration of LLMs with external tools\(?, ?\)\. In parallel, the rapid development of multimodal large language models \(MLLMs\) has shown that language\-centered models can reason jointly over heterogeneous modalities such as text, images and audio, which is particularly relevant when sensor trajectories are converted into plots or textual summaries before inference\(?, ?\)\.

In the specific context of PHM and RUL estimation, the use of LLMs and MLLMs is still recent but growing\. One contribution introduced an LLM\-based regression framework for turbofan RUL prediction and reported competitive results together with promising transfer\-learning behavior\(?, ?\)\. Another study explored a pre\-trained LLM\-based approach for aircraft\-engine RUL prediction on C\-MAPSS\(?, ?\)\. More recently, a multimodal framework was proposed to jointly exploit temporal signals, frequency\-domain images, and textual domain knowledge for industrial time\-series analysis, and it was evaluated on the four standard C\-MAPSS subsets\(?, ?\)\. At a broader PHM level, another work proposed a language\-model\-based framework intended to support multiple maintenance\-related tasks within a unified setting\(?, ?\)\.

Another relevant direction concerns the integration of external knowledge into LLM\-based PHM pipelines\. RAG was introduced to combine parametric language models with explicit non\-parametric memory, thereby improving grounding and updateability\(?, ?\)\. In predictive maintenance, recent studies have argued that retrieval and knowledge augmentation can help LLM\-based systems access maintenance records, technical documentation, and operational procedures more effectively\(?, ?\)\.

Despite these advances, only a limited number of studies explicitly investigate the influence of retrieval augmentation on MLLM\-based RUL estimation in a controlled setting\. Existing studies mainly focus either on adapting LLMs and MLLMs to prognostics tasks or on using textual retrieval for broader PHM support\. Consequently, the effect of time\-series RAG on RUL prediction quality remains insufficiently studied\(?, ?, ?, ?, ?, ?, ?\)\.

## 3Methodology

This section describes the problem setup, proposed model pipeline, and experimental configuration\.

### 3\.1Problem Definition

We consider a set ofnnequipment units, denoted byU=\{U0,U1,…,Un−1\}U=\\\{U\_\{0\},U\_\{1\},\\dots,U\_\{n\-1\}\\\}, monitored over time through onboard measurements\. Each unitUiU\_\{i\}is represented by a multivariate time seriesXi∈ℝTi×MX\_\{i\}\\in\\mathbb\{R\}^\{T\_\{i\}\\times M\}, whereTi∈ℕT\_\{i\}\\in\\mathbb\{N\}denotes the number of observed cycles andMMis the number of monitored variables\. The corresponding sequence of RUL targets is denoted byRi∈ℕ0TiR\_\{i\}\\in\\mathbb\{N\}\_\{0\}^\{T\_\{i\}\}\.

Given the dataset

𝒟=\{\(Xi,Ri\)\}i=0n−1,\\mathcal\{D\}=\\\{\(X\_\{i\},R\_\{i\}\)\\\}\_\{i=0\}^\{n\-1\},\(1\)the objective is to learn a mapping function

fT​S:⋃T∈ℕℝT×M→ℕ0f\_\{TS\}:\\bigcup\_\{T\\in\\mathbb\{N\}\}\\mathbb\{R\}^\{T\\times M\}\\rightarrow\\mathbb\{N\}\_\{0\}\(2\)such that, for a given time seriesXiX\_\{i\}, the functionfT​Sf\_\{TS\}predicts the RUL of its last observation\.

In this work, the mapping functionfT​Sf\_\{TS\}is approximated using a multimodal large language model\. Since MLLMs are designed to process multimodal inputs such as images and text, each time seriesXiX\_\{i\}is transformed into a multimodal representationZiZ\_\{i\}, which may include visual depictions of sensor trajectories and textual contextual information\. The prediction function can therefore be expressed as

fM​M:𝒵→ℕ0,f\_\{MM\}:\\mathcal\{Z\}\\rightarrow\\mathbb\{N\}\_\{0\},\(3\)where𝒵\\mathcal\{Z\}denotes the space of multimodal representations derived from the original time series\. The goal remains the estimation of the RUL of the last observation of each unit\.

### 3\.2Dataset

This study uses the C\-MAPSS benchmark for the empirical evaluation of RUL estimation\. C\-MAPSS is a widely used simulated turbofan\-engine degradation dataset in which each trajectory corresponds to one equipment unit monitored over successive operating cycles through three operational setting variables and multiple sensor measurements\. The training split contains complete run\-to\-failure trajectories, whereas the test split contains truncated trajectories that end before failure\. Accordingly, the task is to estimate the RUL at the last observed cycle of each test unit\(?, ?, ?\)\.

Owing to economic, computational, and time constraints, the experimental campaign reported in this paper is restricted to the FD001 partition\. This design choice enables a focused and controlled analysis within a feasible experimental budget, while a broader evaluation on the remaining partitions is deferred to future work\.

For further details on the simulator and dataset construction, the reader is referred to\(?, ?, ?\)\.

![Refer to caption](https://arxiv.org/html/2608.19218v1/img/rag_mechanism_offline.png)\(a\)RAG builder\. Training trajectories are segmented, sampled by RUL bin, encoded, and indexed in the vector database\.
![Refer to caption](https://arxiv.org/html/2608.19218v1/img/rag_mechanism_online.png)\(b\)Query retriever\. The last segment of the test trajectory is encoded and used to retrieve the nearest indexed train segments\.

Figure 2:Overview of the time‑series RAG mechanism: \(a\) offline indexing of encoded training segments; \(b\) online retrieval using the encoded test segment\.
### 3\.3Preprocessing

The preprocessing pipeline transforms the raw C\-MAPSS trajectories into standardized fixed\-length samples suitable for both retrieval and multimodal inference, as illustrated in Figure[1](https://arxiv.org/html/2608.19218#S1.F1)\. The raw trajectories are first grouped by equipment unit and ordered by cycle\. Next, min–max normalization is applied independently to all sensor channels using statistics computed from the training set\. This transformation reduces scale discrepancies across variables and yields a more comparable representation of the degradation behavior across units\.

After normalization, feature selection based on Pearson correlation is applied to reduce redundancy and eliminate non\-informative measurements\. Only the variables selected through this process are kept for the subsequent stages\.

The cycle\-level RUL targets are then reconstructed from the distance to failure, and a truncation threshold of 125 cycles is applied\. Finally, each normalized trajectory is segmented through a sliding\-window procedure\. Given a window lengthLL, the method extracts fixed\-length temporal segments, and each segment is associated with the RUL value of its last cycle\. This step converts variable\-length degradation trajectories into standardized samples that can be used consistently by the retrieval module and by the multimodal inference pipeline\.

### 3\.4Time\-Series RAG Mechanism

The proposed time\-series RAG mechanism is composed of three main modules: the encoder module, the retrieval memory builder, and the query retriever\. Figure[2](https://arxiv.org/html/2608.19218#S3.F2)summarizes the overall process, distinguishing between the offline construction of the retrieval memory and the online retrieval of historical references for a test unit\.

#### 3\.4\.1Encoder Module

The encoder module transforms each normalized sliding\-window segment into a compact vector representation used for similarity\-based retrieval\. Its purpose is to represent the recent temporal behavior of a unit, including sensor levels and local degradation trends, in a fixed\-dimensional embedding space\.

LetSjS\_\{j\}denote a normalized segment produced by the preprocessing step introduced in Section[3\.3](https://arxiv.org/html/2608.19218#S3.SS3)\. The encoder defines a mapping

zj=ϕ​\(Sj\)∈ℝdz\_\{j\}=\\phi\(S\_\{j\}\)\\in\\mathbb\{R\}^\{d\}wherezjz\_\{j\}is the embedding of segmentSjS\_\{j\}andddis the embedding dimension\.

In this work, the encoder is obtained through an LSTM\-based autoencoding scheme trained exclusively on windows extracted from the training trajectories\. The LSTM encoder processes the multivariate sensor segment and maps it to a latent embedding\. During training, a lightweight decoder is attached to this embedding and optimized to reconstruct the final sensor state of the segment\.

After training, the decoder is discarded and only the encoder, together with the embedding projection, is retained\. The main architectural and training settings of the encoder are summarized in Table[2](https://arxiv.org/html/2608.19218#S3.T2)\.

Table 2:Configuration of the learned LSTM encoder used to generate retrieval embeddings\.
#### 3\.4\.2Retrieval Memory Builder

The retrieval memory builder constructs the non\-parametric memory used by the proposed time\-series RAG mechanism\. This module corresponds to the offline stage shown in Figure[2](https://arxiv.org/html/2608.19218#S3.F2)\(a\)\. It operates only on the training partition, in order to avoid information leakage, and transforms historical degradation data into a searchable collection of representative examples\.

##### Window\-to\-Embedding Transformation\.

The module is built from the windowed training segments produced by the preprocessing step described in Section[3\.3](https://arxiv.org/html/2608.19218#S3.SS3)\. Each segment is associated with the RUL value of its last cycle\. The trained encoder described in Section[3\.4\.1](https://arxiv.org/html/2608.19218#S3.SS4.SSS1)is then used to transform each selected segment into a fixed\-dimensional embedding\.

##### RUL\-Balanced Retrieval Memory Construction\.

To reduce redundancy and improve coverage across degradation stages, the training segments are grouped according to their RUL values, and a subset of representative segments is sampled from each RUL bin\.

The resulting retrieval memory can be represented as

𝒞=\{\(zj,rj,mj\)\}j=1N\\mathcal\{C\}=\\left\\\{\(z\_\{j\},r\_\{j\},m\_\{j\}\)\\right\\\}\_\{j=1\}^\{N\}wherezjz\_\{j\}is the segment embedding defined by the encoder module,rjr\_\{j\}is the associated RUL value, andmjm\_\{j\}contains metadata such as the source unit and cycle position\. The embeddings and metadata are then stored in a vector database, forming a searchable memory of historical degradation examples\.

![Refer to caption](https://arxiv.org/html/2608.19218v1/img/multimodal_prompt_generation.png)\(a\)Multimodal prompt\-generation workflow\.
![Refer to caption](https://arxiv.org/html/2608.19218v1/img/trajectory_comparison_example.png)\(b\)Example of the trajectory\-comparison image\.

Figure 3:Multimodal prompt generation and example comparison artifact used for MLLM inference\.

#### 3\.4\.3Query Retriever

The query retriever is the online module responsible for selecting the historical references used to ground the MLLM prediction\. This module corresponds to the online stage shown in Figure[2](https://arxiv.org/html/2608.19218#S3.F2)\(b\)\. For each test unit, only the last observed segment is used as the query, since the objective is to estimate the RUL at the most recent cycle\.

The query segment is encoded using the same encoder adopted during memory construction\. This ensures that both training and test segments are represented in the same embedding space\. Given the last observed query segmentQiQ\_\{i\}, its embedding is obtained as

zi\(q\)=ϕ​\(Qi\)z\_\{i\}^\{\(q\)\}=\\phi\(Q\_\{i\}\)
A nearest\-neighbor search is then performed in the retrieval memory to identify thekkmost similar historical segments:

𝒩k​\(Qi\)=top⁡\-​kj∈\{1,…,N\}​sim⁡\(zi\(q\),zj\)\\mathcal\{N\}\_\{k\}\(Q\_\{i\}\)=\\operatorname\{top\}\\text\{\-\}k\_\{j\\in\\\{1,\\ldots,N\\\}\}\\operatorname\{sim\}\\left\(z\_\{i\}^\{\(q\)\},z\_\{j\}\\right\)wheresim⁡\(⋅,⋅\)\\operatorname\{sim\}\(\\cdot,\\cdot\)denotes the similarity measure used by the retrieval engine\.

The retrieved neighbors provide historical references whose recent sensor behavior is close to that of the test unit\. Their metadata are used to recover the corresponding training trajectories, cycle positions, and RUL values\. These references are then used in the next stage to generate the visual comparison artifact for multimodal prompting\.

### 3\.5Multimodal Prompt Generation and MLLM Inference

The multimodal prompt\-generation stage is composed of three main modules: the trajectory comparison builder, the multimodal prompt composer, and the structured output parser\. Figure[3](https://arxiv.org/html/2608.19218#S3.F3)summarizes the overall process, showing how the query trajectory and the retrieved historical references are transformed into a visual comparison artifact, inserted into a structured multimodal prompt, and processed by the MLLM to produce an RUL estimate and a textual explanation\.

#### 3\.5\.1Trajectory Comparison Builder

The Trajectory Comparison Builder transforms the query trajectory and the retrieved historical references into a trajectory\-comparison image\. This image contrasts the recent behavior of the test unit with similar training trajectories, allowing the MLLM to inspect sensor levels, temporal trends, and degradation\-stage alignment in a compact visual form\.

As shown in Figure[3\(a\)](https://arxiv.org/html/2608.19218#S3.F3.sf1), this visual artifact is generated before prompt composition and serves as the main grounding evidence for the subsequent MLLM\-based RUL prediction\.

##### Visual encoding choices\.

The visual artifact is designed to make the degradation comparison readable within the constraints of a paper figure and an MLLM input\. The query trajectory and the retrieved references are plotted on a common temporal axis, so that their recent behavior can be compared directly\. The selected sensor channels are shown as separate trajectories, allowing the model to evaluate whether the similarity between the query and the references is consistent across multiple variables\.

The example reported in Figure[3\(b\)](https://arxiv.org/html/2608.19218#S3.F3.sf2)includes two selected sensors and the corresponding RUL profile for readability\. In the actual inference pipeline, the comparison artifact incorporates the sensor variables selected during preprocessing, as described in Section[3\.3](https://arxiv.org/html/2608.19218#S3.SS3)\. The RUL information associated with the retrieved training references is used to contextualize their degradation stage, while the RUL of the test unit remains the target to be estimated\.

These visual encoding choices are intended to provide the MLLM with evidence about three main aspects: the current degradation level of the query unit, its recent temporal trend, and its similarity to historical run\-to\-failure examples\.

#### 3\.5\.2Multimodal Prompt Composer

The Multimodal Prompt Composer combines the trajectory\-comparison image with a textual prompt template, as shown in Figure[3\(a\)](https://arxiv.org/html/2608.19218#S3.F3.sf1)\. The resulting prompt provides the MLLM with visual evidence, a description of the RUL estimation task, instructions for comparing the query trajectory with the retrieved references, and an explicit output schema\.

This structure ensures that all evaluated MLLMs receive a consistent multimodal input\. The system prompt, instruction prompt, and user prompt template used in this study are reported in the Appendix\.

#### 3\.5\.3Structured Output Parsing

For each query unit, the MLLM is required to return a structured response containing both the numerical RUL estimate and a short textual explanation\. The expected output has the form

yi=\{“rul”:r^i,“explanation”:ei\},y\_\{i\}=\\left\\\{\\begin\{array\}\[\]\{l\}\\text\{\`\`rul''\}:\\hat\{r\}\_\{i\},\\\\ \\text\{\`\`explanation''\}:e\_\{i\}\\end\{array\}\\right\\\},wherer^i∈ℕ0\\hat\{r\}\_\{i\}\\in\\mathbb\{N\}\_\{0\}denotes the predicted RUL of the last observation of the query trajectory, andeie\_\{i\}is the explanation generated by the model\.

The predicted RUL value is used for quantitative evaluation, whereas the explanation provides qualitative information about the reasoning process followed by the MLLM\. In this way, the model is used not only as a regressor, but also as a reasoning component that can justify its prediction with reference to the visible degradation evidence\. A representative explanation returned by the model is provided in Section[5](https://arxiv.org/html/2608.19218#S5)\.

![Refer to caption](https://arxiv.org/html/2608.19218v1/img/results_rag_influence.png)Figure 4:Predictive performance over 10 repetitions for the three evaluated Gemini MLLMs under the two inference settings,MLLM\+RandomandMLLM\+RAG\.Table 3:FD001 test\-set comparison between baseline methods and Gemini\-based RAG variants\.

### 3\.6Experimental Protocol

The experimental protocol compares two reference\-selection strategies for multimodal RUL estimation:*random*trajectory selection and*RAG\-based*trajectory selection\. Letℳ\\mathcal\{M\}denote the set of evaluated MLLMs\. The experiment is repeatedN=10N=10times to account for variability induced by the MLLM inference parameters and by the non\-deterministic construction of the RAG component\.

At each repetitionnn, a repetition\-specific environment

E\(n\)=\(θ\(n\),𝒞\(n\),ℐ\(n\)\)E^\{\(n\)\}=\\bigl\(\\theta^\{\(n\)\},\\mathcal\{C\}^\{\(n\)\},\\mathcal\{I\}^\{\(n\)\}\\bigr\)is instantiated, whereθ\(n\)\\theta^\{\(n\)\}denotes the sampled inference parameters,𝒞\(n\)\\mathcal\{C\}^\{\(n\)\}is the reference corpus derived from the training set, andℐ\(n\)\\mathcal\{I\}^\{\(n\)\}is the corresponding retrieval index\. For each reference\-selection strategy, the prompts are generated once under the current environment and then used to query all MLLMs inℳ\\mathcal\{M\}\. This produces a paired evaluation setting in which, within each repetition, all models are assessed using the same prepared prompts\.

Input:Train set

𝒟t​r​a​i​n\\mathcal\{D\}\_\{train\}, test set

𝒟t​e​s​t\\mathcal\{D\}\_\{test\}, window length

LL, number of references

kk, prompt template

TT, set of MLLMs

ℳ\\mathcal\{M\}, parameter search space

Θ\\Theta, repetitions

NN
Output:Predictions and scores for all strategies, MLLMs, and repetitions

for*n←1n\\leftarrow 1toNN*do

E\(n\)←BuildEnvironment​\(𝒟t​r​a​i​n,L,Θ\)E^\{\(n\)\}\\leftarrow\\textsc\{BuildEnvironment\}\(\\mathcal\{D\}\_\{train\},L,\\Theta\);

foreach*s∈\{*random*,*rag*\}s\\in\\\{\\texttt\{random\},\\texttt\{rag\}\\\}*do

𝒫\(n,s\)←Prompts​\(𝒟t​e​s​t,E\(n\),s,k,T\)\\mathcal\{P\}^\{\(n,s\)\}\\leftarrow\\textsc\{Prompts\}\(\\mathcal\{D\}\_\{test\},E^\{\(n\)\},s,k,T\);

foreach*m∈ℳm\\in\\mathcal\{M\}*do

𝒴^\(n,s,m\)←CallLLM​\(𝒫\(n,s\),m,E\(n\)\)\\hat\{\\mathcal\{Y\}\}^\{\(n,s,m\)\}\\leftarrow\\textsc\{CallLLM\}\(\\mathcal\{P\}^\{\(n,s\)\},m,E^\{\(n\)\}\);

M\(n,s,m\)←Score​\(𝒴^\(n,s,m\)\)M^\{\(n,s,m\)\}\\leftarrow\\textsc\{Score\}\(\\hat\{\\mathcal\{Y\}\}^\{\(n,s,m\)\}\);

end foreach

end foreach

end for

Algorithm 1Experimental protocol used to compare random and RAG\-based reference selection across multiple MLLMs over repeated non\-deterministic environments\.Algorithm[1](https://arxiv.org/html/2608.19218#alg1)summarizes the full evaluation procedure\.BuildEnvironmentsamples the inference parameters, constructs the reference corpus, and builds the retrieval index\.Promptsextracts the last observed test segments, selects the references according to the considered strategy, generates the comparison figures, and instantiates the prompt template\. Finally,CallLLMqueries each MLLM, andScoreevaluates the resulting RUL predictions\. The scores are aggregated across repetitions to compare random and retrieval\-based reference selection\.

##### Evaluated MLLMs\.

The experimental evaluation considers three Gemini\-based multimodal large language models, namelyGemini 3\.1 Flash\-Lite,Gemini 3 Flash, andGemini 3\.1 Pro\. These models were selected to cover different levels of model capacity and computational cost\. In particular, Flash\-Lite represents a lighter and more efficient variant, Flash provides an intermediate configuration, and Pro corresponds to a more capable model intended for more demanding reasoning tasks\. This selection allows us to analyze whether the effect of retrieval augmentation is consistent across models with different capability–efficiency trade\-offs\.

## 4Results

Figure[4](https://arxiv.org/html/2608.19218#S3.F4)reports the predictive performance of the three evaluated Gemini MLLMs under the two considered inference settings, namely random multimodal inference \(MLLM\+Random\) and retrieval\-augmented multimodal inference \(MLLM\+RAG\)\. Across all evaluated models, the inclusion of time\-series retrieval leads to a consistent improvement in predictive quality, yielding lower error and more stable behavior than random reference selection\. This trend indicates that the retrieved historical trajectories provide useful grounding evidence for the MLLM during RUL estimation\.

Among the proposed configurations, the strongest results are obtained by thePro \+ RAGsetting, whileFlash \+ RAGalso shows clear gains over its non\-retrieval counterpart\. TheFlash\-Litevariant benefits less from retrieval, suggesting that the usefulness of the retrieved information depends not only on the retrieval mechanism itself, but also on the reasoning capacity of the underlying multimodal model\. Overall, the results support the claim that similarity\-based reference selection is preferable to arbitrary reference sampling when MLLMs are used for RUL estimation\.

The comparison reported in Table[3](https://arxiv.org/html/2608.19218#S3.T3)further helps position the proposed approach with respect to the literature\. On the one hand, the proposed RAG\-based variants outperform simple baselines and improve consistently over the corresponding random\-reference MLLM setting\. On the other hand, the best proposed configuration does not yet surpass the strongest task\-specific and more sophisticated deep learning architecture included in the comparison\.

A second relevant observation concerns performance dispersion across the repeated experiments\. For theFlashandProvariants, the RAG\-based setting tends to produce more compact distributions than the non\-retrieval baseline, especially for the main error metrics\. This suggests that retrieval not only improves the central tendency of the predictions, but also reduces sensitivity to arbitrary reference selection\. In summary, the results indicate that time\-series retrieval is a beneficial component within the proposed MLLM pipeline, even though the overall architecture remains simpler than the strongest predictive models reported in the literature\.

## 5Discussion

The experimental results provide a positive answer to the main research question of this paper: in the considered FD001 setting, time\-series retrieval improves MLLM\-based RUL estimation\. The observed gains suggest that retrieved training trajectories provide useful grounding evidence, enabling the model to compare the query with historically similar degradation patterns rather than relying only on generic multimodal reasoning over the input artifact\. In this sense, the proposed time\-series RAG mechanism strengthens the link between the observed trajectory and previously seen run\-to\-failure behaviors\.

Another relevant aspect concerns explainability\. Unlike conventional regression baselines, the proposed framework can return not only a numerical RUL estimate but also a textual explanation of the rationale behind the prediction\. Although this advantage does not compensate for the current performance gap with stronger specialized models, it remains valuable in PHM settings, where analysts may benefit from inspecting the prediction together with the retrieved historical references and the model’s explanation\. This explanatory capability is one of the motivations for exploring language\-model\-based approaches in prognostics\.

To illustrate this qualitative aspect, Example 1 reports a representative explanation produced by the MLLM\. The response shows that the model is able to relate the predicted RUL to observable properties of the query trajectory, such as the absence of visible degradation and the similarity with early\-life train references\. We also manually inspected a subset of the explanations and found them broadly consistent with the visual evidence provided in the trajectory\-comparison figures\. However, explanation quality was not evaluated systematically in this study\.

> Example 1\. Representative explanation returned by the MLLM\. The TEST unit’s sensor readings across all informative channels \(such as s2, s3, s4, s7, s11, and s12\) are completely flat and show no signs of degradation\. The levels closely match the baseline values of the TRAIN references during their earliest recorded cycles, well before any degradation trends begin\. Since there is no visible progression towards failure, the unit is clearly in its early life stage\. Therefore, the estimated RUL is 125 cycles\.

At the same time, the comparison with the literature must be interpreted carefully\. The proposed approach does not yet match the predictive accuracy of the strongest architectures included in Table[3](https://arxiv.org/html/2608.19218#S3.T3)\. However, this gap should be read in light of the purpose of the study\. The main objective of this paper is not to introduce the most sophisticated possible multimodal architecture for RUL prediction, but rather to isolate and evaluate the influence of time\-series RAG within a comparatively simple and controlled pipeline\.

Several additional limitations should nevertheless be acknowledged\. First, the evaluation is restricted to FD001, which is the simplest C\-MAPSS subset\. Therefore, the findings should not yet be generalized to more challenging settings involving multiple operating conditions, multiple fault modes, or previously unseen failure mechanisms\. In particular, the proposed time\-series RAG component can only retrieve degradation patterns that are represented in the indexed historical memory\. If a failure mode has not occurred before, or is not sufficiently covered by the training trajectories, the system may retrieve only partially similar references and provide misleading grounding evidence to the MLLM\. Second, the current study relies on a specific prompt design, a specific visual comparison artifact, and a single retrieval formulation, which may influence the quality of both the numerical predictions and the generated explanations\. Third, although the experiment is repeated multiple times to account for variability, MLLM\-based inference remains inherently non\-deterministic and the proposed framework does not provide a formal guarantee of correctness\. The repeated evaluation should therefore be interpreted as an empirical robustness assessment rather than as a correctness guarantee\. Finally, the present analysis remains mainly descriptive and does not yet include systematic uncertainty estimation, novelty detection, or mechanisms for flagging cases in which the retrieved evidence is insufficient\. Altogether, these limitations support interpreting the present work as a preliminary study on the role of retrieval in MLLM\-based prognostics\.

## 6Conclusions

This paper presented a framework for studying RUL estimation with multimodal language models under retrieval augmentation\. The proposed approach combines a time\-series RAG module, which retrieves historically similar train segments, with a multimodal prompting pipeline that transforms the query and retrieved references into an input suitable for MLLM\-based inference\.

The empirical results on FD001 show that retrieval augmentation consistently improves the proposed MLLM pipeline relative to non\-retrieval multimodal inference\. Across all evaluated Gemini variants, the RAG\-based setting yields better predictive performance and more stable behavior than random reference selection\. These results support the main conclusion of the paper: time\-series RAG has a positive influence on multimodal RUL estimation\.

More broadly, the study highlights two promising aspects of language\-model\-based approaches for prognostics\. First, they can benefit from explicit access to relevant historical trajectories rather than relying only on internal model reasoning\. Second, they can naturally provide textual explanations alongside the numerical prediction, which may be valuable in human\-centered PHM decision support settings\.

Several directions can be explored in future work\. First, the evaluation should be extended beyond FD001 to the more challenging C\-MAPSS subsets and to additional predictive maintenance datasets\. Second, the retrieval component could be improved through stronger embedding models, alternative similarity measures, and re\-ranking strategies tailored to degradation trajectories\. Third, future work should investigate how time\-series RAG interacts with more advanced multimodal architectures, including richer temporal backbones and more effective cross\-modal fusion mechanisms\. Finally, it would be valuable to study jointly the relationship between retrieval quality, predictive accuracy, and explanation quality in order to better understand when MLLM\-based prognostic reasoning is most useful in practice\.

## Acknowledgment

This research was funded in whole, or in part, by the Luxembourg National Research Fund \(FNR\), grant reference BRIDGES/2022/IS/17270233\. For the purpose of open access, and in fulfillment of the obligations arising from the grant agreement, the authors have applied a Creative Commons Attribution 4\.0 International \(CC BY 4\.0\) license to any Author Accepted Manuscript version arising from this submission\.\\PHMbibliographybibliography