LithoFormer: A Robust Framework for Stratigraphic Inference via Transformers

# LithoFormer: A Robust Framework for Stratigraphic Inference via Transformers
Source: [https://arxiv.org/html/2607.22804](https://arxiv.org/html/2607.22804)
Shwetha SalimathLaboratoire Interdisciplinaire des Sciences du Numérique, Université Paris\-Saclay, SLBMontpellierFrance[shwetha\.salimath@universite\-paris\-saclay\.fr](https://arxiv.org/html/2607.22804v1/mailto:[email protected])Francesca BugiottiLaboratoire Interdisciplinaire des Sciences du Numérique, CentraleSupélec, CNRS, Université Paris\-SaclayGif\-sur\-YvetteFrance[francesca\.bugiotti@centralesupelec\.fr](https://arxiv.org/html/2607.22804v1/mailto:[email protected]),Sylvain WlodarczykSLBMontpellierFrance[swlodarczyk@slb\.com](https://arxiv.org/html/2607.22804v1/mailto:[email protected])andSohaib OuzinebSLBMontpellierFrance[souzineb@slb\.com](https://arxiv.org/html/2607.22804v1/mailto:[email protected])

###### Abstract\.

Accurate geological characterization of subsurface reservoirs from well log data is essential to support projects such as carbon capture and storage \(CCS\), geothermal development, and extraction of natural resources\. Existing automated techniques for geological characterization primarily use sliding\-window classification, which limits their ability to understand broader geological contexts, often leading to misaligned formation layers\. To overcome these limitations, we introduce LithoFormer, a robust framework for stratigraphic inference using a Seq2Seq transformer model that ingests entire multivariate well logs in a single pass\. The framework utilizes a channel\-independent PatchTST backbone enhanced with rotary positional embeddings \(RoPE\) to capture long\-range geological dependencies across entire multivariate well logs\. A decoupled multi\-task head is employed to jointly predict geological zonation and precise boundary probabilities, while a geology\-informed loss function enforces physical constraints such as the Law of Superposition\. Validated and deployed on three real\-world datasets, LithoFormer demonstrates a 90% reduction in median boundary error and eliminates stratigraphic order violations compared to traditional sliding\-window baselines\. It also achieves a 80% reduction in manual expert labor and eliminates stratigraphic inconsistencies, providing a scalable and reliable solution for large\-scale subsurface modeling\.

Time Series Segmentation, Transformer, Physics\-Informed Machine Learning, Subsurface Characterization, Multi\-Task Learning\.

††ccs:Computing methodologies Supervised learning††ccs:Applied computing Earth and atmospheric sciences††ccs:Computing methodologies Neural networks## 1\.Introduction

The mapping of stratigraphy is essential for energy projects that require subsurface characterization, as it determines resource storage capacity and geological safety\. Accurate 3D models of subsurface architecture are crucial for identifying porous reservoirs for carbon sequestration, geothermal formations, and traditional petroleum reserves\. This modeling involves reconstructing geological processes over millions of years and primarily relies on well correlation, which aligns stratigraphic boundaries or ”markers” across drilled wells\(Tsuji and Matsuoka,[2014](https://arxiv.org/html/2607.22804#bib.bib10); Alam and Mohamad\-Hussein,[2023](https://arxiv.org/html/2607.22804#bib.bib11)\)\. Well logs analysis helps pinpoint these markers\(Darling,[2005](https://arxiv.org/html/2607.22804#bib.bib3); Ellis and Singer,[2007](https://arxiv.org/html/2607.22804#bib.bib4)\), providing a continuous, high\-resolution record of subsurface layers often missed by seismic data\.

Traditionally, well correlation using well logs is a manual process in which geologists visually match patterns in well log data, such as Gamma Ray, Resistivity, and Density measurements\(Mann and Dowell Jr,[1978](https://arxiv.org/html/2607.22804#bib.bib19)\)\. This method lacks scalability\. It is confirmed by both geologists and our research that a specific marker within a region usually exhibits a consistent signal pattern\(Rider,[1990](https://arxiv.org/html/2607.22804#bib.bib7); Abdel Azim and Aljehani,[2022](https://arxiv.org/html/2607.22804#bib.bib8)\)\. Early computational approaches for matching well log patterns, such as dynamic time warping \(DTW\)\(Müller,[2007](https://arxiv.org/html/2607.22804#bib.bib18)\), provided some level of automation, but were often fragile, sensitive to noise, and struggled to generalize across geologically diverse areas\. This led to the development of deep learning models that treated the task as a sliding\-window classification problem\. However, the sliding\-window paradigm is inherently limited by a local receptive field\. These models lack global context; a sliding window has a ”short\-sighted” view, unable to see the full geological sequence\. They cannot distinguish between a valid marker pattern and a visually similar artifact appearing at an incorrect position, nor can they enforce the basic physical law that geological layers are deposited sequentially\.

![Refer to caption](https://arxiv.org/html/2607.22804v1/images/seq2seq_comp.png)Figure 1\.Workflow Comparison\. The sliding\-window approach \(left\) requiresLLlocal classifications per well, while LithoFormer \(right\), processes the entire well log in a single pass to generate a complete stratigraphic map\. M1, M2, M3, and M4 are markers denoting the stratigraphic sequence\.To address these challenges, we propose an approach shift from a local classification task to a global end\-to\-end Seq2Seq task\. We introduce LithoFormer, whose core contribution is the ability to predict a sequential order of events from long, multivariate time series data\. As illustrated in Figure[1](https://arxiv.org/html/2607.22804#S1.F1), this approach uses the power of transformers to learn both local patterns and global trends that match a valid stratigraphic succession\. By processing the well log in a single pass, LithoFormer ensures that every predicted marker is placed within its proper global context, maintaining the chronological integrity of the geological interpretation\. The contributions of the paper are as follows\.

- •LithoFormer Frameworkfor identifying a precise, ordered sequence of events within complex, multivariate time series data\.
- •A robust data\-centric pipelinethat uses resampling, filtering, and constrained data augmentation to help the model distinguish true markers from out\-of\-context patterns\.
- •Multi\-Task Transformer Architectureusing a channel\-independent PatchTST transformer\(Nie and H\. Nguyen,[2023](https://arxiv.org/html/2607.22804#bib.bib29)\), enhanced with RoPE\(Suet al\.,[2024](https://arxiv.org/html/2607.22804#bib.bib36)\)and a decoupled multi\-task head\. The dual head jointly learns coarse zonations \(geological formation intervals\) and precise edges \(interval boundaries\)\.
- •AGeology\-Informed Lossto enforce stratigraphic order \(the Law of Superposition\), while athickness\-weightedloss enables the model to detect extremely thin or rare layers accurately\.
- •Real World Validation: We deploy and test on three datasets from Colorado, Wyoming, and the Norwegian North Sea\.

LithoFormer achieves state\-of\-the\-art precision while ensuring geological consistency\. This provides a scalable and physically consistent solution for subsurface reservoir characterization\.

The paper is structured as follows\. In Section[2](https://arxiv.org/html/2607.22804#S2), we review related work\. Section[3](https://arxiv.org/html/2607.22804#S3)details our methodology, including the novel LithoFormer model architecture, the data augmentation pipeline, and the multi\-stage training curriculum\. Section[4](https://arxiv.org/html/2607.22804#S4)reports our comprehensive experimental results and ablation analyses, and concludes in Section[5](https://arxiv.org/html/2607.22804#S5)\.

## 2\.Related Work

Our work intersects three key research areas: automated geological correlation, deep learning for time series, and physics\-informed machine learning\. We review the evolution of well correlation and discuss advancements in time series transformers that support our approach\.

Automated Well Log Correlationis essential in stratigraphic analysis\. Initial methods focused on automation through signal processing, using cross\-correlation, statistical methods, and feature\-based matching\(Dashtian and Jafari,[2011](https://arxiv.org/html/2607.22804#bib.bib21)\)\. Techniques like DTW were used for aligning log sequences via optimal nonlinear warping\(Linemanet al\.,[1987](https://arxiv.org/html/2607.22804#bib.bib20)\)\. However, DTW struggles with multi\-well and multivariate correlation due to its computational demands and sensitivity to noise\. Recent approaches have integrated machine learning \(ML\) techniques for better correlation\. Recurrent Neural Networks \(RNNs\) and convolutional neural networks \(CNNs\), including hybrid models like LSTM\-CNN and LSTM\-2dCNN, have been successfully employed to process well logs as multivariate time series and identify marker signatures\(Brazellet al\.,[2019](https://arxiv.org/html/2607.22804#bib.bib23); Imamverdiyev and Sukhostat,[2019](https://arxiv.org/html/2607.22804#bib.bib24); Salimathet al\.,[2025](https://arxiv.org/html/2607.22804#bib.bib25)\)\. Nonetheless, challenges persist in managing long\-range dependencies and capturing a well’s global context, problems that transformer architectures aim to address\.

Transformers for Time Series Analysis\.The transformer architecture\(Vaswaniet al\.,[2017](https://arxiv.org/html/2607.22804#bib.bib26)\)has been adapted for time series tasks, particularly with models like Informer and Autoformer\(Zhouet al\.,[2021](https://arxiv.org/html/2607.22804#bib.bib27); Wuet al\.,[2021](https://arxiv.org/html/2607.22804#bib.bib28)\), which focus on long\-range forecasting\. However, these models treat time series as sequences of individual time steps, leading to high computational costs\. An advancement in this field is PatchTST\(Nie and H\. Nguyen,[2023](https://arxiv.org/html/2607.22804#bib.bib29)\), our architectural backbone, which segments a time series into patches used as tokens\. This approach reduces sequence length, captures local semantics, and improves learning from longer contexts\. Its has achieved state\-of\-the\-art performance in classification\(Wanget al\.,[2024](https://arxiv.org/html/2607.22804#bib.bib31)\)and forecasting benchmarks\(Huanget al\.,[2024](https://arxiv.org/html/2607.22804#bib.bib32); Goswamiet al\.,[2024](https://arxiv.org/html/2607.22804#bib.bib33)\)\. Transformers need positional information to model sequential dependencies effectively\([Kimet al\.,](https://arxiv.org/html/2607.22804#bib.bib37); Dufteret al\.,[2022](https://arxiv.org/html/2607.22804#bib.bib34)\)\. Early methods employed absolute sinusoidal embeddings, which struggle with longer sequences\. RoPE improves this by representing relative positions within the attention mechanism, enhancing translation invariance and generalization\(Suet al\.,[2024](https://arxiv.org/html/2607.22804#bib.bib36)\)\. RoPE has shown exceptional performance in large language models like LLaMA and Gpt\-oss\(Touvronet al\.,[2023](https://arxiv.org/html/2607.22804#bib.bib41); OpenAIet al\.,[2025](https://arxiv.org/html/2607.22804#bib.bib43)\)\.

Physics\-Informed and Constrained Deep Learning\.An important direction in scientific machine learning is to incorporate domain knowledge and physical principles directly into the learning process\(Cuomo and Vincenzo Schiano di Cola,[2022](https://arxiv.org/html/2607.22804#bib.bib44); Fareaet al\.,[2024](https://arxiv.org/html/2607.22804#bib.bib45)\)\. While initially introduced to solve physics problems, it was also successfully applied in finance and autonomous systems for fairness and safety\(Sivaraman and Golnoosh Farnadi,[2020](https://arxiv.org/html/2607.22804#bib.bib47); Hounieet al\.,[2024](https://arxiv.org/html/2607.22804#bib.bib48)\)\. Following this idea, our method introduces a loss term that enforces the geological law of superposition, guiding optimization toward stratigraphic sequences that are geologically consistent\.

## 3\.LithoFormer framework

This section details the LithoFormer framework\. We first explain the problem formulation followed by detail description of the data\-centric pipeline, the multi\-task architecture, and the geology\-informed training curriculum\. The complete end\-to\-end workflow, encompassing both training and inference, is illustrated in Figure[2](https://arxiv.org/html/2607.22804#S3.F2)\.

![Refer to caption](https://arxiv.org/html/2607.22804v1/images/KDD26-flowcart.png)Figure 2\.The end\-to\-end LithoFormer framework\.### 3\.1\.Problem Formulation

Stratigraphic inference is formulated as a mapping from a multivariate well logX∈ℝL×CX\\in\\mathbb\{R\}^\{L\\times C\}to an ordered sequence of geological zonesy∈\{0,…,K\}L\{y\}\\in\\\{0,\\dots,K\\\}^\{L\}with corresponding marker depthsd\{d\}∈ℝK\\in\\mathbb\{R\}^\{K\}\. Here,LLrepresents the number of depth samples andCCthe number of well logging channels\. These channels correspond to well log data of physical properties such as gamma\-ray , resistivity, and density\.KKis the number of markers\. Zone indices\{0,…,K\}\\\{0,\\dots,K\\\}correspond to markers sorted by expected stratigraphic order\. The index ”0” represents the zone before the first marker\. Given wellXX, LithoFormer is trained to simultaneously predict two outputs:

1. \(1\)Zonation ProbabilitiesP^z​o​n​e\\hat\{P\}\_\{zone\}:A set of probability curvesP^z​o​n​e∈\[0,1\]L×\(K\+1\)\\hat\{P\}\_\{zone\}\\in\[0,1\]^\{L\\times\(K\+1\)\}indicating the probability of geological zones occurrence at each depth\.
2. \(2\)Edge ProbabilitiesP^e​d​g​e\\hat\{P\}\_\{edge\}:A set of probability curves,P^e​d​g​e∈\[0,1\]L×K\\hat\{P\}\_\{edge\}\\in\[0,1\]^\{L\\times K\}, indicating the probability of marker occurrence at each depth\.

A globally consistent zone sequencey^z​o​n​e∈\{0,…,K\}L\\hat\{y\}\_\{zone\}\\in\\\{0,\\dots,K\\\}^\{L\}is derived fromP^z​o​n​e\\hat\{P\}\_\{zone\}to establish the stratigraphic order\. In parallel, precise marker depthsd^e​d​g​e∈ℝK\\hat\{d\}\_\{edge\}\\in\\mathbb\{R\}^\{K\}are computed by identifying peak probabilities withinP^e​d​g​e\\hat\{P\}\_\{edge\}\. With this notation, the fundamental geological constraints described in the paper can be stated as:

- •Monotonic layer order: For any two depth indicesi<ji<j, the predicted zone indices must be increasing:y^i≤y^j\\hat\{y\}\_\{i\}\\leq\\hat\{y\}\_\{j\}\.
- •Exactly one boundary per marker: For each markerk∈\{1,…,K\}k\\in\\\{1,\\dots,K\\\},P^e​d​g​e\\hat\{P\}\_\{edge\}should contain at thekk\-th channel a single dominant peak atd^e​d​g​e,k\\hat\{d\}\_\{edge,k\}\. This reflects the geological assumption that markerkkoccurs exactly once and in the correct order\.

### 3\.2\.Data\-Centric Pipeline

Real\-world well log data often contain sensor noise, data gaps, and inconsistent sequence lengths due to varying drilling depths\. To manage this variability, a preprocessing pipeline is used\. First, the logs are resampled to a fixed sequence length\. Then, a two\-stage filtering process is applied: a Hampel filter\(Pearson and Neuvo,[2016](https://arxiv.org/html/2607.22804#bib.bib14)\)replaces outliers with the local median, and a Savitzky\-Golay filter\(Schafer,[2011](https://arxiv.org/html/2607.22804#bib.bib15)\)smooths the signal while preserving the trend\. To combat overfitting and ensure accurate depth predictions, we propose a constrained data augmentation strategy that inserts expert\-verified marker signatures into incorrect log locations, generating realistic out\-of\-context negative samples\. Our ablation studies[4\.4](https://arxiv.org/html/2607.22804#S4.SS4)demonstrate that this method significantly improves model robustness and precision\.

From a mathematical point of view, letX∈ℝL×CX\\in\\mathbb\{R\}^\{L\\times C\}be the input well log andy∈\{0,…,K\}Ly\\in\\\{0,\\dots,K\\\}^\{L\}its target zonation sequence\. Letℳy\\mathcal\{M\}\_\{y\}be the set of true markers present inyy\. We generate a new sample\(Xa​u​g,y\)\(X\_\{aug\},y\)as follows: First, a true markerm∗∈ℳym^\{\*\}\\in\\mathcal\{M\}\_\{y\}with its corresponding depth indexd∗d^\{\*\}is selected\. Extract the multivariate signature patternXs​i​g∈ℝ2​W×CX\_\{sig\}\\in\\mathbb\{R\}^\{2W\\times C\}from a window of length2​W2Wsamples aroundd∗d^\{\*\}\. To prevent overlaps with existing markers, we define𝒵e​x​c​l​u​d​e\\mathcal\{Z\}\_\{exclude\}as the union of windows of size2​E2Esamples around all marker depth indices inℳy\\mathcal\{M\}\_\{y\}\. Sample a new depth indexdn​e​wd\_\{new\}from the set of valid indices\{1,…,L\}∖𝒵e​x​c​l​u​d​e\\\{1,\\dots,L\\\}\\setminus\\mathcal\{Z\}\_\{exclude\}\. Finally, constructXa​u​gX\_\{aug\}by copyingXXand pasting the patternXs​i​gX\_\{sig\}at the new locationdn​e​wd\_\{new\}\. The new pair\(Xa​u​g,y\)\(X\_\{aug\},y\)is then added to the training set\.

### 3\.3\.LithoFormer Transformer Backbone

LithoFormer employs a multi\-task Seq2Seq transformer backbone to process multivariate well log inputsX∈ℝL×CX\\in\\mathbb\{R\}^\{L\\times C\}through two sequential blocks: a channel\-independent feature extraction PatchTST encoder and a multi\-task prediction head\. The backbone treats each log channel as a univariate series, which is essential for two reasons\. First, it accommodates diverse statistical distributions and log patterns of different physical measurements\. Second, it ensures robustness against missing data\. While Gamma Ray \(GR\) is often available, other logs such as resistivity \(RES\) or density \(DEN\) are frequently absent\. Channel independence allows effective feature extraction from available signals without interference from missing inputs\.

![Refer to caption](https://arxiv.org/html/2607.22804v1/images/lithoformer_architecture_2.png)Figure 3\.The LithoFormer transformer backbone\. Multivariate inputs are processed via a channel\-independent PatchTST with RoPE\. Decoupled multi\-task heads then generate zone and edge probabilities from the fused dense features\.The input data is segmented intoNpN\_\{p\}overlapping patches of lengthLpL\_\{p\}along the depth dimension with patch stridess, processed per input channel due to the channel\-independent architecture\. These patches are projected onto a latent dimensionDmodelD\_\{\\text\{model\}\}, and the resulting latent embedding vectors are processed by a PatchTST encoder\. To effectively capture the relative depth dependencies crucial to stratigraphy, where sequence order and layer thickness are important, RoPE is applied\. RoPE embeds relative positional information directly into the attention mechanism, enhancing translation invariance and generalization to longer sequences\(Suet al\.,[2024](https://arxiv.org/html/2607.22804#bib.bib36)\)\.

To upsample compressed patch features of the encoder output back to a dense well log resolutionLL, linear interpolation is applied\. Following the upsampling procedure, we obtain a tensor inℝC×L×Dmodel\\mathbb\{R\}^\{C\\times L\\times D\_\{\\text\{model\}\}\}\. Subsequently, the feature vectors corresponding to allCClogs are concatenated to construct a dense unified representationZupsampled∈ℝL×\(C⋅Dmodel\)Z\_\{\\text\{upsampled\}\}\\in\\mathbb\{R\}^\{L\\times\(C\\cdot D\_\{\\text\{model\}\}\)\}\. This operation facilitates the learning of cross\-log dependencies prior to the prediction head\.

As an innovative aspect of our model, we utilize two decoupled prediction heads running in parallel to forecast zonation and edge probabilities\. The zonation head focuses on the segmentation task by projecting the latent features intoK\+1K\+1channels via a linear layer, which corresponds to the geological zone probabilitiesP^z​o​n​e\\hat\{P\}\_\{zone\}at each depth index\. This approach effectively captures the global structural state of the stratigraphy\. In parallel, the edge head processes the latent features to predict marker depth probabilitiesP^e​d​g​e\\hat\{P\}\_\{edge\}through a linear layer\. Unlike the zonation head, which predicts mutually exclusive zones, the edge head is designed to identify sparse boundary events by generating independent logits for each of theKKspecific markers\. The outputP^z​o​n​e\\hat\{P\}\_\{zone\}is as follows:

\(1\)P^z​o​n​e=Linearz​o​n​e​\(Zu​p​s​a​m​p​l​e​d\)∈ℝL×\(K\+1\)\\hat\{P\}\_\{zone\}=\\text\{Linear\}\_\{zone\}\(Z\_\{upsampled\}\)\\in\\mathbb\{R\}^\{L\\times\(K\+1\)\}The outputP^e​d​g​e\\hat\{P\}\_\{edge\}is as follows:

\(2\)P^e​d​g​e=Lineare​d​g​e​\(Zu​p​s​a​m​p​l​e​d\)∈ℝL×K\\hat\{P\}\_\{edge\}=\\text\{Linear\}\_\{edge\}\(Z\_\{upsampled\}\)\\in\\mathbb\{R\}^\{L\\times K\}
A key feature of LithoFormer is the joint optimization of both heads without gradient isolation\. This allows the model to reconstruct an ordered sequence of events while preserving spatial detail through shared gradient backpropagation\.

### 3\.4\.Geology\-Informed Loss Function

Standard cross\-entropy loss is insufficient for our task because it treats each depth index independently and ignores stratigraphic sequence order\. To obtain geologically plausible predictions, we design a composite loss that embeds domain knowledge by combining the cross\-entropy term \(ℒC​E\\mathcal\{L\}\_\{CE\}\) with regularizers\. These regularizers preserve stratigraphic order \(ℒM​o​n​o\\mathcal\{L\}\_\{Mono\}\) and sharpen edge detection boundaries \(ℒE​d​g​e\\mathcal\{L\}\_\{Edge\}\) as observed in ablation studies[4\.4](https://arxiv.org/html/2607.22804#S4.SS4)\. The total loss,ℒtotal\\mathcal\{L\}\_\{\\text\{total\}\}, is a weighted sum of these components, withλm,λe∈ℝ\\lambda\_\{m\},\\lambda\_\{e\}\\in\\mathbb\{R\}\.

\(3\)ℒt​o​t​a​l=ℒC​E\+λm​ℒM​o​n​o\+λe​ℒE​d​g​e\\mathcal\{L\}\_\{total\}=\\mathcal\{L\}\_\{CE\}\+\\lambda\_\{m\}\\mathcal\{L\}\_\{Mono\}\+\\lambda\_\{e\}\\mathcal\{L\}\_\{Edge\}
Weighted Cross\-Entropy \(ℒC​E\\mathcal\{L\}\_\{CE\}\):Geological layers vary in thickness, leading to class imbalance in the zonation sequence, and as a result, thin yet significant formations may be overlooked\. Therefore, we use the inverse thickness weighting, where the weightwkw\_\{k\}for each zone classkkisL÷NkL\\div N\_\{k\}, whereNkN\_\{k\}is the number of samples inside the corresponding zone\.

Monotonic Loss \(ℒM​o​n​o\\mathcal\{L\}\_\{Mono\}\):An important physical constraint is the enforcement of the Law of Superposition, which requires that stratigraphic sequences be arranged by geological age from younger to older strata with increasing depth\. The expected zonation class sequencek^z​o​n​e\\hat\{k\}\_\{zone\}is calculated ask^z​o​n​e,i=∑k=0Kk⋅P^z​o​n​e,i,k\\hat\{k\}\_\{zone,i\}=\\sum\_\{k=0\}^\{K\}k\\cdot\\hat\{P\}\_\{zone,i,k\}at each depth indexii\. A penalty is applied only when thek^z​o​n​e\\hat\{k\}\_\{zone\}sequence violates increasing monotonicity:

\(4\)ℒM​o​n​o​\(k^z​o​n​e\)=1L−1​∑i=2LReLU​\(k^z​o​n​e,i−1−k^z​o​n​e,i\)\\mathcal\{L\}\_\{Mono\}\(\\hat\{k\}\_\{zone\}\)=\\frac\{1\}\{L\-1\}\\sum\_\{i=2\}^\{L\}\\text\{ReLU\}\(\\hat\{k\}\_\{zone,i\-1\}\-\\hat\{k\}\_\{zone,i\}\)
Edge Focal Loss \(ℒE​d​g​e\\mathcal\{L\}\_\{Edge\}\):As the number of zonation boundaries is much fewer than the total number of depth samples, it creates a significant class imbalance\. Letye​d​g​e∈ℝL×K\{y\}\_\{edge\}\\in\\mathbb\{R\}^\{L\\times K\}be encoded by the marker channel as a Gaussian distribution centered around the true marker depth index\. To obtainℒE​d​g​e\\mathcal\{L\}\_\{Edge\}, we first define the focal lossℒFL,i,k\\mathcal\{L\}\_\{\\text\{FL,i,k\}\}for a depth indexiiand a marker classkk\. The focal loss is derived from the the Binary Cross\-Entropy lossℒBCE,i,k\\mathcal\{L\}\_\{\\text\{BCE,i,k\}\}applied to the edge head output probabilitiesP^e​d​g​e\\hat\{P\}\_\{edge\}against the targetye​d​g​e\{y\}\_\{edge\}at the depth indexiiand marker channelkksuch that:

\(5\)ℒFL,i,k=αi,k​\(1−exp⁡\(−ℒBCE,i,k\)\)γ⋅ℒBCE,i,k\\mathcal\{L\}\_\{\\text\{FL,i,k\}\}=\\alpha\_\{i,k\}\(1\-\\exp\{\(\-\\mathcal\{L\}\_\{\\text\{BCE,i,k\}\}\}\)\)^\{\\gamma\}\\cdot\\mathcal\{L\}\_\{\\text\{BCE,i,k\}\}The focusing parameterγ\\gammadecreasesℒFL,i,k\\mathcal\{L\}\_\{\\text\{FL,i,k\}\}for correctly classified examples\. Letα∈\[0\.5,1\]\\alpha\\in\[0\.5,1\], the balancing factorαi,k\\alpha\_\{i,k\}is defined asα\\alphawhenye​d​g​e,i,k\>0\.5\{y\}\_\{edge,i,k\}\>0\.5and1−α1\-\\alphaotherwise\. Higher weights are assigned to indices near true marker depths, and lower weights to background indices\.ℒE​d​g​e\\mathcal\{L\}\_\{Edge\}is the mean focal loss over all depth indices and marker classes\. This loss term is crucial for achieving high precision in sparse detection tasks\.

### 3\.5\.Stratigraphic Inference

The LithoFormer framework generates two complementary sets of predictions:P^z​o​n​e\\hat\{P\}\_\{zone\}andP^e​d​g​e\\hat\{P\}\_\{edge\}\. We getd^z​o​n​e\\hat\{d\}\_\{zone\}by employing a sequence decoder that iteratively selects for eachP^z​o​n​e\\hat\{P\}\_\{zone\}channel the minimal depth at which the corresponding probability is the highest across channels\. Concurrently, high\-resolution marker depths \(d^e​d​g​e\\hat\{d\}\_\{edge\}\) are obtained by identifying the peak probabilities within each marker channel of the outputP^e​d​g​e\\hat\{P\}\_\{edge\}\.y^z​o​n​e\\hat\{y\}\_\{zone\}andy^\\hat\{y\}is the zone sequence representation ofd^z​o​n​e\\hat\{d\}\_\{zone\}andd^e​d​g​e\\hat\{d\}\_\{edge\}\. Ultimately, the framework utilizes thed^e​d​g​e\\hat\{d\}\_\{edge\}as the final result, leveraging their superior local precision while the zonation outputy^z​o​n​e\\hat\{y\}\_\{zone\}serves to validate the global stratigraphic sequence\.

## 4\.Experiments and Results

We validated the LithoFormer framework and our backbone model by comparing its performance in precision, recall, and stratigraphic order against state\-of\-the\-art methods using an end\-to\-end Seq2Seq approach and sliding\-window models\. Ablation studies explored our data\-centric pipeline, patch stride, and loss function terms\. The following sections outline our experimental setup, results, and performance analysis\.

### 4\.1\.Datasets

To perform a robust evaluation, we use three distinct datasets from publicly available well\-log data, representing various levels of geological complexity and signal degradation\. We filter training wells, with at least 60% sequence markers present\. A quality control step removes wells with entirely missing log readings and label quality control to remove marker outliers\. We perform no imputation, smoothing, or input signal cleaning; thus, the model is evaluated on raw, noisy industrial data\. The train, validation, and test split is 70:20:10\. The test set wells were manually verified by a domain expert to address inaccuracies in public datasets\.

Table 1\.Dataset SpecificationsThe three datasets of increasing difficulty levels are used for model evaluation \(Table[1](https://arxiv.org/html/2607.22804#S4.T1)\)\. Dataset Colorado\([CECMC,](https://arxiv.org/html/2607.22804#bib.bib52)\)provides a baseline where marker signatures for Niobrara, Codel, and Forth Hayes are clearly distinguishable\. Dataset Wyoming\([WOGCC,](https://arxiv.org/html/2607.22804#bib.bib53)\)adds complexity with similar signature patterns in Badger Coal, Felix Coal, and Big George Coal markers, some lacking recognizable signatures\. Dataset Norwegian North Sea\([NPD,](https://arxiv.org/html/2607.22804#bib.bib54)\)represents a complex, multivariate, long\-sequence scenario critical for proving scalability\. The zonation sequence applied is Hordaland, Rogaland, Shetland, Viking, Vestland, and Dunling\.

### 4\.2\.Experimentation setup

To evaluate LithoFormer, we benchmarked it against four baselines from different architectural paradigms\. First, a traditional DTW\-based baseline represents the industry standard\. Second, GeoTS, our previous state\-of\-the\-art model using the LSTM\-2dCNN neural network\(Salimathet al\.,[2025](https://arxiv.org/html/2607.22804#bib.bib25)\)\. Third, a Conformer model was assessed for the combined CNN and transformer features\. Fourth, a PatchTST baseline was included to evaluate performance gains from the LithoFormer backbone’s multi\-task head\. Lastly, to validate our channel\-independent backbone choice, we implemented a channel\-mixing variant combining log channels at the embedding layer\. To ensure a fair comparison, all neural baselines—including Conformer, PatchTST, and Channel\-Mixing were integrated with LithoFormer framework’s data\-centric pipeline\. All baseline models underwent rigorous hyperparameter tuning for a fair evaluation\.

The models were implemented in PyTorch and trained on a NVIDIA L4 GPU using the fastai framework\(Howard and Gugger,[2020](https://arxiv.org/html/2607.22804#bib.bib51)\)\. The transformer backbone features six PatchTST encoder layers with four attention heads, a patch stridessof eight, and a patch lengthLpL\_\{p\}of 50\. Data augmentation window parametersWWandEEare set to 100 ft and 200 ft, and adjusted to the resampling rate of the input well log sequence\. The loss function is calibrated withλm=1\.5\\lambda\_\{m\}=1\.5\(LmL\_\{m\}\) andλe=20\.0\\lambda\_\{e\}=20\.0\(LeL\_\{e\}\)\. The code for reproducing the models and experiments will be made publicly available upon publication\.

We evaluated all models using three critical metrics\. Our primary metric for localization precision is the Median Absolute Error \(MedAE\), reported in feet for its robustness against outlier predictions in real\-world datasets\. To assess sensitivity, we used Recall@τ\\tau, defined as the proportion of markers predicted withinτ\\taufeet of the ground truth\. Lastly, for geological consistency, we report the accuracy of the predicted sequence order, reflecting the percentage of wells where all predicted markers conform to the correct stratigraphic sequence\.

### 4\.3\.Main Results

The results in varying geological complexities are summarized in Table[2](https://arxiv.org/html/2607.22804#S4.T2)\. A clear performance hierarchy emerges as data difficulty increases\. In the Colorado dataset, transformer\-based models excelled, achieving MedAE below 3 ft with perfect recall, indicating efficacy with simpler stratigraphic sequences\. In the Wyoming dataset, the sliding\-window approach showed limitations; the GeoTS baseline’s recall fell to 50\.37%, while patch transformer models retained accuracy, highlighting the advantages of a global context approach\.

Table 2\.Main comparative results across all datasets\. Recall and order accuracy values are percentages \(%\)\. MedAE is in feet\.Dataset ColoradoDataset WyomingDataset North SeaModelMedAE\(ft\)Recall@20ftOrder %MedAE\(ft\)Recall@20ftOrder %MedAE\(ft\)Recall@50ftOrder %DTW Baseline6\.6089\.25100194\.015\.1457\.8112310\.2520GeoTS \(KDD ’25\)2\.097\.428076\.250\.37100667\.9835\.2488Conformer∗15\.053\.75100322\.421\.6785410\.9238\.9268Patchtst baseline∗3\.097\.671008\.089\.629884\.6751\.5692Channel\-Mixing∗2\.0100\.001002\.091\.6210059\.9258\.5494LithoFormer \(Ours\)2\.0100\.001002\.091\.6210019\.7472\.8195∗These baselines utilize the same data\-centric pipeline as LithoFormer\.![Refer to caption](https://arxiv.org/html/2607.22804v1/images/Result_lithoformer.png)Figure 4\.Qualitative visualization of LithoFormer’s output on a North Sea well\.Tracks \(1\-3\):Ground Truth \(Geologist picks\), predicted Zone and Edge sequence\.Tracks \(4\-6\):The multivariate input logs\.Tracks \(7\+\):The internal model outputs\. TheZone Probabilitiesidentify the geological zone, while theEdge Probabilitiesprecisely pinpoint the boundary location\.The analysis of the North Sea dataset reveals critical insights\. We report recall atτ=50\\tau=50ft, considering a downsampling rate of 4 feet\. The DTW baseline and GeoTS show limitations due to local matching issues, while the Conformer model performs suboptimally\. In contrast, the channel\-mixing transformer achieves a MedAE of 59\.92 ft, but our channel\-independent LithoFormer excels at 19\.74 ft\. This indicates that early fusion harms well log data analysis, as channels represent distinct properties with different noise profiles\. LithoFormer demonstrates a significant advancement in the learning of temporal structures, achieving a new benchmark in error reduction\. It decreases errors by 95% compared to GeoTS and by 75% against the PatchTST baseline\. LithoFormer also achieves the highest order accuracy\. Figure[4](https://arxiv.org/html/2607.22804#S4.F4)illustrates the inference with distinct geological zonation and precise boundary detection\.

Failure Analysis for North SeaWhileLithoFormershows high recall in Colorado and Wyoming, the North Sea \(NS\) dataset has a27%27\\%recall gap due to geological complexity\. Key drivers for these stratigraphic residuals includeGeological Ambiguity, which arises from difficult\-to\-distinguish markers like the Hordaland Group\. Additionally,Class Imbalanceplays a significant role, as there are under\-represented formations such as the Vestland and Dunlin groups that have 75% fewer samples\. The presence ofFalse Positivesis another contributing factor, stemming from bimodal probability peaks resulting from near\-identical signatures\. Furthermore,Sensor Noisecan lead to data gaps being misinterpreted as stratigraphic edges, andCovariate Shiftresults in precision loss in deeper test wells\. To address these challenges, the model flags low\-confidence edges for manual expert review during production workflows\.

### 4\.4\.Ablation study

To analyze the key components of our framework and validate our design choices, we performed a series of component\-wise ablation studies\. They are conducted on the multivariate North Sea dataset, where the impact of each component is most pronounced\. The results shown are on the 100 test wells\.

#### Backbone Stride and Resource Efficiency

Table 3\.Ablation on Backbone Stride \(North Sea\)The trade\-off between feature resolution and resource consumption was analyzed by adjusting the stride of the PatchTST backbone \(Table[3](https://arxiv.org/html/2607.22804#S4.T3)\)\. A stride of 1 captures fine temporal details but results in a high MedAE and memory usage\. In contrast, Stride 8 achieved the best MedAE and Recall while also reducing GPU usage to 0\.84 GB due to fewer tokens\. This performance gain stems from coarser sampling, which reduces overfitting to high signal variability\. Moreover, upsampling latent features via linear interpolation offers a more stable representation for prediction heads compared to the denser stride 1\. The 95% decrease in GPU memory usage to 0\.84 GB enables deployment on edge hardware with less than 1GB of VRAM\.

#### Loss Components

Table 4\.Ablation on Loss Function Components \(North Sea\)The contribution of each loss component was evaluated by isolating its effect \(Table[4](https://arxiv.org/html/2607.22804#S4.T4)\)\. The first and second rows, lacking an edge loss term, derive metrics usingd^z​o​n​e\\hat\{d\}\_\{zone\}, while the third and fourth used^e​d​g​e\\hat\{d\}\_\{edge\}\. The baseline cross\-entropy loss \(ℒC​E\\mathcal\{L\}\_\{CE\}\) shows poor precision and order due to a lack of geological context\. Adding \(ℒM​o​n​o\\mathcal\{L\}\_\{Mono\}\) boosts the raw order to 80\.57%, reinforcing the Law of Superposition\. Row 3 \(ℒE​d​g​e\\mathcal\{L\}\_\{Edge\}\) is vital for precision, significantly enhancing MedAE and Recall\. In conclusion, LithoFormer \(ℒt​o​t​a​l\\mathcal\{L\}\_\{total\}\) achieves optimal performance by integrating structural zonation logic with edge detection\.

#### Data\-Centric pipeline

Assessments of the data pipeline \(Table[5](https://arxiv.org/html/2607.22804#S4.T5)\) showed that constrained data augmentation significantly improves model performance, achieving a 67% reduction in MedAE and a 26% increase in recall\. Showing that learning the global stratigraphic context via cut\-and\-paste negative examples is important\. However, explicit preprocessing with Hampel/Savitzky\-Golay filters negatively impacted the final framework\. While it slightly improved the baseline, it increased the MedAE of the augmented framework\. The best approach is to use raw, resampled logs, allowing the multi\-task architecture to perform internal denoising\.

Table 5\.Ablation on Data\-Centric Components \(North Sea\)

## 5\.Conclusions and Future Work

This work introduces LithoFormer, a robust framework designed for high\-precision stratigraphic inference from multivariate well\-log data\. By reframing the problem from traditional sliding\-window classification to a global seq2seq paradigm, the framework successfully captures the long\-range dependencies essential for reconstructing geologically consistent sequences\. Key innovations include a decoupled multi\-task architecture and a geology\-informed training\. Validated across three diverse datasets the LithoFormer demonstrates exceptional stability, maintaining high precision even in the presence of 54% data missingness\. Ultimately, LithoFormer transforms a traditionally slow analytical process into a scalable automatic solution, applicable beyond geology to fields like medical monitoring and industrial phase detection\.

Future work aims to develop a global foundation model for well log data using a self\-supervised approach similar to the MOMENT model\(Goswamiet al\.,[2024](https://arxiv.org/html/2607.22804#bib.bib33)\)\. This model will support task\-specific heads for data imputation, marker prediction, and petrophysical property estimation, enabling faster adaptation to new regions with fewer expert engagements\.

###### Acknowledgements\.

We thank the CESMC\([CECMC,](https://arxiv.org/html/2607.22804#bib.bib52)\), WOGCC\([WOGCC,](https://arxiv.org/html/2607.22804#bib.bib53)\), and NOD\([NPD,](https://arxiv.org/html/2607.22804#bib.bib54)\)for providing well log datasets for informational and research purposes\.

## References

- R\. Abdel Azim and A\. Aljehani \(2022\)Neural network model for permeability prediction from reservoir well logs\.Processes10\(12\),pp\. 2587\.Cited by:[§1](https://arxiv.org/html/2607.22804#S1.p2.1)\.
- M\. Alam and e\. Mohamad\-Hussein \(2023\)Carbon capture and storage \(ccs\) subsurface modeling for risk identification\.Cited by:[§1](https://arxiv.org/html/2607.22804#S1.p1.1)\.
- S\. Brazell, A\. Bayeh, and A\. et\. al \(2019\)A machine\-learning\-based approach to assistive well\-log correlation\.Petrophysics \- The SPWLA Journal60\(04\),pp\. 469–479\.External Links:ISSN 1529\-9074,[Document](https://dx.doi.org/10.30632/PJV60N4-2019a1),[Link](https://doi.org/10.30632/PJV60N4-2019a1),https://onepetro\.org/petrophysics/article\-pdf/60/04/469/2202651/spwla\-2019\-v60n4a1\.pdfCited by:[§2](https://arxiv.org/html/2607.22804#S2.p2.1)\.
- \[4\]CECMCData portal\.Colorado Energy and Carbon Management Commission\.Note:[https://ecmc\.state\.co\.us/data\.html\#\#/cogis](https://ecmc.state.co.us/data.html##/cogis)Cited by:[§4\.1](https://arxiv.org/html/2607.22804#S4.SS1.p2.1)\.
- S\. Cuomo and et\. a\. Vincenzo Schiano di Cola \(2022\)Scientific machine learning through physics\-informed neural networks: where we are and what’s next\.External Links:2201\.05624,[Link](https://arxiv.org/abs/2201.05624)Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p4.1)\.
- T\. Darling \(2005\)Well logging and formation evaluation\.Elsevier\.Cited by:[§1](https://arxiv.org/html/2607.22804#S1.p1.1)\.
- H\. Dashtian and et\. a\. Jafari \(2011\)Analysis of cross correlations between well logs of hydrocarbon reservoirs\.Transport in porous media90\(2\),pp\. 445–464\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p2.1)\.
- P\. Dufter, M\. Schmitt, and H\. Schütze \(2022\)Position information in transformers: an overview\.Computational Linguistics48\(3\),pp\. 733–763\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p3.1)\.
- D\. V\. Ellis and J\. M\. Singer \(2007\)Well logging for earth scientists\.Vol\.692,Springer\.Cited by:[§1](https://arxiv.org/html/2607.22804#S1.p1.1)\.
- A\. Farea, O\. Yli\-Harja, and F\. Emmert\-Streib \(2024\)Understanding physics\-informed neural networks: techniques, applications, trends, and challenges\.AI5\(3\),pp\. 1534–1557\.External Links:[Link](https://www.mdpi.com/2673-2688/5/3/74),ISSN 2673\-2688,[Document](https://dx.doi.org/10.3390/ai5030074)Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p4.1)\.
- M\. Goswami, K\. Szafer, A\. Choudhry, Y\. Cai, S\. Li, and A\. Dubrawski \(2024\)Moment: a family of open time\-series foundation models\.arXiv preprint arXiv:2402\.03885\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p3.1),[§5](https://arxiv.org/html/2607.22804#S5.p2.1)\.
- I\. Hounie, J\. Porras\-Valenzuela, and A\. Ribeiro \(2024\)Loss shaping constraints for long\-term time series forecasting\.External Links:2402\.09373,[Link](https://arxiv.org/abs/2402.09373)Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p4.1)\.
- J\. Howard and S\. Gugger \(2020\)Fastai: a layered api for deep learning\.Information11\(2\),pp\. 108\.External Links:ISSN 2078\-2489,[Link](http://dx.doi.org/10.3390/info11020108),[Document](https://dx.doi.org/10.3390/info11020108)Cited by:[§4\.2](https://arxiv.org/html/2607.22804#S4.SS2.p2.8)\.
- X\. Huang, J\. Tang, and Y\. Shen \(2024\)Long time series of ocean wave prediction based on patchtst model\.Ocean Engineering301,pp\. 117572\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p3.1)\.
- Y\. Imamverdiyev and L\. Sukhostat \(2019\)Lithological facies classification using deep convolutional neural network\.Journal of Petroleum Science and Engineering174,pp\. 216–228\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p2.1)\.
- \[16\]J\. Kim, K\. Park, and et\. a\. YunPPT: patch order do matters in time series pretext task\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p3.1)\.
- D\. Lineman, J\. Mendelson, and M\. N\. Toksoz \(1987\)Well to well log correlation using knowledge\-based systems and dynamic depth warping\.InSPWLA Annual Logging Symposium,pp\. SPWLA–1987\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p2.1)\.
- C\. J\. Mann and T\. P\. Dowell Jr \(1978\)Quantitative lithostratigraphic correlation of subsurface sequences\.Computers & Geosciences4\(3\),pp\. 295–306\.Cited by:[§1](https://arxiv.org/html/2607.22804#S1.p2.1)\.
- M\. Müller \(2007\)Dynamic time warping\.InInformation Retrieval for Music and Motion,pp\. 69–84\.External Links:ISBN 978\-3\-540\-74048\-3,[Document](https://dx.doi.org/10.1007/978-3-540-74048-3%5F4),[Link](https://doi.org/10.1007/978-3-540-74048-3_4)Cited by:[§1](https://arxiv.org/html/2607.22804#S1.p2.1)\.
- Y\. Nie and et\. a\. H\. Nguyen \(2023\)A time series is worth 64 words: long\-term forecasting with transformers\.InInternational Conference on Learning Representations,Cited by:[3rd item](https://arxiv.org/html/2607.22804#S1.I1.i3.p1.1),[§2](https://arxiv.org/html/2607.22804#S2.p3.1)\.
- \[21\]NPDThe norwegian offshore directory \(sodir\)\.Norwegian Petroleum Directorate\.Note:[https://www\.sodir\.no/en/](https://www.sodir.no/en/)Cited by:[§4\.1](https://arxiv.org/html/2607.22804#S4.SS1.p2.1)\.
- OpenAI, :, and S\. A\. et\. al \(2025\)Gpt\-oss\-120b & gpt\-oss\-20b model card\.External Links:2508\.10925,[Link](https://arxiv.org/abs/2508.10925)Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p3.1)\.
- R\. K\. Pearson and e\. Neuvo \(2016\)Generalized hampel filters\.EURASIP Journal on Advances in Signal Processing2016\(1\),pp\. 87\.Cited by:[§3\.2](https://arxiv.org/html/2607.22804#S3.SS2.p1.1)\.
- M\. Rider \(1990\)Gamma\-ray log shape used as a facies indicator: critical analysis of an oversimplified methodology\.Geological Society, London, Special Publications48\(1\),pp\. 27–37\.Cited by:[§1](https://arxiv.org/html/2607.22804#S1.p2.1)\.
- S\. Salimath, F\. Bugiotti, and S\. Wlodarczyk \(2025\)GeoTS: a tsc framework for estimating geological formation to model carbon storage reservoirs\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 2,pp\. 4773–4783\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p2.1),[§4\.2](https://arxiv.org/html/2607.22804#S4.SS2.p1.1)\.
- R\. W\. Schafer \(2011\)What is a savitzky\-golay filter?\[lecture notes\]\.IEEE Signal processing magazine28\(4\),pp\. 111–117\.Cited by:[§3\.2](https://arxiv.org/html/2607.22804#S3.SS2.p1.1)\.
- A\. Sivaraman and et\. a\. Golnoosh Farnadi \(2020\)Counterexample\-guided learning of monotonic neural networks\.External Links:2006\.08852,[Link](https://arxiv.org/abs/2006.08852)Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p4.1)\.
- J\. Su, M\. Ahmed, and et\. a\. Lu \(2024\)Roformer: enhanced transformer with rotary position embedding\.Neurocomputing568,pp\. 127063\.Cited by:[3rd item](https://arxiv.org/html/2607.22804#S1.I1.i3.p1.1),[§2](https://arxiv.org/html/2607.22804#S2.p3.1),[§3\.3](https://arxiv.org/html/2607.22804#S3.SS3.p2.4)\.
- H\. Touvron, T\. Lavril, and et\. a\. Izacard \(2023\)Llama: open and efficient foundation language models\.arXiv preprint arXiv:2302\.13971\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p3.1)\.
- T\. Tsuji and et\. a\. Matsuoka \(2014\)Reservoir characterization for site selection in the gundih ccs project, indonesia\.Energy Procedia63,pp\. 6335–6343\.Cited by:[§1](https://arxiv.org/html/2607.22804#S1.p1.1)\.
- A\. Vaswani, N\. Shazeer, and et\. a\. Parmar \(2017\)Attention is all you need\.Advances in neural information processing systems30\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p3.1)\.
- Y\. Wang, N\. Huang, T\. Li, Y\. Yan, and X\. Zhang \(2024\)Medformer: a multi\-granularity patching transformer for medical time\-series classification\.Advances in Neural Information Processing Systems37,pp\. 36314–36341\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p3.1)\.
- \[33\]WOGCCData pipeline\.Wyoming Oil and Gas Conservation Commission\.Note:[https://pipeline\.wyo\.gov](https://pipeline.wyo.gov/)Cited by:[§4\.1](https://arxiv.org/html/2607.22804#S4.SS1.p2.1)\.
- H\. Wu, J\. Xu, and et\. a\. Wang \(2021\)Autoformer: decomposition transformers with auto\-correlation for long\-term series forecasting\.Advances in neural information processing systems34,pp\. 22419–22430\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p3.1)\.
- H\. Zhou, S\. Zhang, and et\. a\. Peng \(2021\)Informer: beyond efficient transformer for long sequence time\-series forecasting\.InProceedings of the AAAI conference on artificial intelligence,Vol\.35,pp\. 11106–11115\.Cited by:[§2](https://arxiv.org/html/2607.22804#S2.p3.1)\.