ChronoSSM: Training for Temporally Aware Representations in Autoregressive State Space Models

# ChronoSSM: Training for Temporally Aware Representations in Autoregressive State Space Models
Source: [https://arxiv.org/html/2608.10120](https://arxiv.org/html/2608.10120)
,Nachiketa Ratnakar Patil[npatil@iitb\.ac\.in](mailto:[email protected])Centre for Machine Intelligence and Data ScienceIndian Institute of Technology BombayMumbaiIndia,Arjun Bhagoji[arjunp@iitb\.ac\.in](mailto:[email protected])Centre for Machine Intelligence and Data ScienceIndian Institute of Technology BombayMumbaiIndiaandFrancesco Bronzino[francesco\.bronzino@ens\-lyon\.fr](mailto:[email protected])ENS de Lyon, CNRS, UCBL1, LIPLyonFranceInstitut universitaire de FranceFrance

###### Abstract\.

Modern sequence models, from Transformers to State Space Models, have enabled powerful generative modeling across diverse domains, yet they are typically trained to predict*what*happens while treating*when*it happens as a secondary concern\. In data\-mining settings where events are associated with explicit timing information, this separation can limit temporal reasoning, anomaly detection, and faithful reconstruction of event chronology\. A common strategy is to treat timing as an auxiliary signal, training a separate timing model using representations learned solely for event prediction\. However, this two\-stage approach implicitly assumes that representations optimized for event prediction already contain sufficient temporal structure\.

We introduce ChronoSSM, an autoregressive State Space Model \(SSM\) that jointly models events and timestamps with a shared backbone trained using combined token and temporal generation objectives\. We compare theJointregime, where temporal supervision updates the backbone, with theTwo\-Stageregime, where timing is learned only using the frozen event representations\. Across four domains spanning dense and partial timestamp supervision,Jointtraining consistently makes inter\-arrival information more recoverable from frozen representations without any systematic degradation in content\-generation quality overall\. Our results show that temporal supervision can produce more temporally informative representations without materially degrading autoregressive event modeling\.

††copyright:none## 1\.Introduction

Modern autoregressive sequence models are now used well beyond natural language, including in domains such as business\-process traces, clinical event sequences derived from electronic health records, network traffic, and temporal knowledge graphs\(van Dongen and Borchert,[2018](https://arxiv.org/html/2608.10120#bib.bib22); Johnsonet al\.,[2023](https://arxiv.org/html/2608.10120#bib.bib24); Bronzinoet al\.,[2019](https://arxiv.org/html/2608.10120#bib.bib34); Jinet al\.,[2020](https://arxiv.org/html/2608.10120#bib.bib31)\)\. In these settings, events are associated with explicit temporal metadata, and downstream tasks depend not only on what happened but also on when it happened\(Lanvinet al\.,[2023](https://arxiv.org/html/2608.10120#bib.bib29); Paparrizoset al\.,[2025](https://arxiv.org/html/2608.10120#bib.bib30); Cüpperset al\.,[2024](https://arxiv.org/html/2608.10120#bib.bib21); Jinet al\.,[2020](https://arxiv.org/html/2608.10120#bib.bib31)\)\. This raises a basic question:*can autoregressive sequence models learn both event identity and event timing within a shared representation*? We formalize this problem in[section2](https://arxiv.org/html/2608.10120#S2)\.

A common way to incorporate timing has been to treat it as an auxiliary task\(Jianget al\.,[2024](https://arxiv.org/html/2608.10120#bib.bib37); Chuet al\.,[2026](https://arxiv.org/html/2608.10120#bib.bib18)\): first train a sequence model for next\-event prediction, then freeze its representations and fit a separate timing module on top\. This two\-stage strategy is simple and modular, but it assumes that representations optimized only for token prediction already retain enough temporal information\. That assumption is questionable: the token generation objective only rewards features that improve next\-event likelihood, not inter\-arrival modeling\. A downstream timing module must therefore recover temporal structure from representations that were never explicitly trained to encode it\.

In this paper, we introduce ChronoSSM \([section3](https://arxiv.org/html/2608.10120#S3)\), an autoregressive State Space Model that predicts both the next event and its timestamp from a shared backbone\. A lightweight temporal head is trained jointly with the token prediction head, so temporal gradients can directly update the backbone\. The resulting model learns event semantics and temporal dynamics within a unified representation\. Our code and configurations shall be released upon acceptance\.

To test whether this joint formulation offers a real advantage over the two\-stage alternative, we conduct a controlled comparison between two training regimes that share the same architecture, data, and temporal objective \([section4](https://arxiv.org/html/2608.10120#S4)\): aTwo\-Stageregime, in which timing is learned after event prediction on frozen representations, and aJointregime, in which both objectives are optimized together\. We evaluate this comparison on four key domains spanning dense and partial timestamp supervision: business\-process traces\(van Dongen and Borchert,[2018](https://arxiv.org/html/2608.10120#bib.bib22); Mannhardtet al\.,[2018](https://arxiv.org/html/2608.10120#bib.bib23)\), clinical event sequences\(Johnsonet al\.,[2023](https://arxiv.org/html/2608.10120#bib.bib24)\), network traffic\(Bronzinoet al\.,[2019](https://arxiv.org/html/2608.10120#bib.bib34); Chuet al\.,[2026](https://arxiv.org/html/2608.10120#bib.bib18)\), and temporal knowledge graphs\(Jinet al\.,[2020](https://arxiv.org/html/2608.10120#bib.bib31)\)\. In addition to these four main domains, we also experiment with symbolic\-music\(Konget al\.,[2022](https://arxiv.org/html/2608.10120#bib.bib25); Hawthorneet al\.,[2019](https://arxiv.org/html/2608.10120#bib.bib26); Huanget al\.,[2019](https://arxiv.org/html/2608.10120#bib.bib27)\)generation reported in[appendixB](https://arxiv.org/html/2608.10120#A2)\. Our two main research questions are:

RQ1 \(Temporal Recoverability\)\.DoesJointtraining yield representations from which inter\-arrival information is more recoverable than underTwo\-Stagetraining? We evaluate this using a suite of*recoverability diagnostics*\(formalized in[section5](https://arxiv.org/html/2608.10120#S5)\) that probe whether temporal structure is encoded in frozen learned representations and can be extracted by lightweight analyses\. We find thatJointtraining consistently produces representations from which timing is more recoverable across all four domains, with the magnitude of the effect varying by domain\.

RQ2 \(Quality Degradation\)\.Does injecting temporal supervision into the shared backbone systematically degrade downstream modeling quality? Stronger temporal recoverability is only useful if it does not compromise the model’s primary event\-generation function\. We therefore compareJointandTwo\-Stagetraining on domain\-specific generation metrics and find no evidence of a systematic degradation underJointtraining \([section6](https://arxiv.org/html/2608.10120#S6)\), withJointgenerating higher quality data in 2 out of 4 domains\.

Overall, these results indicate that explicit temporal supervision can improve temporal recoverability without systematically degrading the model’s primary generative function\. This paves the way for temporally\-aware generative models to be trained and deployed in relevant domains\.

## 2\.Background

This section introduces the modeling framework used throughout the paper\. We first describe the underlying sequence model and the representations it produces, and then formalize the problem setting for predicting the next event and its associated timing information\.

### 2\.1\.The State\-Space Architecture

State Space Models \(SSMs\) are sequence models with recurrent latent\-state dynamics that support linear\-time processing in sequence length\(Fichtlet al\.,[2025](https://arxiv.org/html/2608.10120#bib.bib16); Guet al\.,[2022](https://arxiv.org/html/2608.10120#bib.bib17)\)\. This is useful in long\-sequence settings, where computational efficiency becomes important as context length grows\.

The core mechanism of an SSM is a linear dynamical system that maps an input sequence to an output sequence through a hidden state\(Guet al\.,[2022](https://arxiv.org/html/2608.10120#bib.bib17)\)\. At stepkk, the model updates a latent statesks\_\{k\}from the previous state and the current inputxkx\_\{k\}, and then produces an output representationhkh\_\{k\}:

\(1\)sk\\displaystyle s\_\{k\}=Ak​sk−1\+Bk​xk,\\displaystyle=A\_\{k\}s\_\{k\-1\}\+B\_\{k\}x\_\{k\},\(2\)hk\\displaystyle h\_\{k\}=Ck​sk\+Dk​xk\.\\displaystyle=C\_\{k\}s\_\{k\}\+D\_\{k\}x\_\{k\}\.In modern selective SSMs, some of these parameters are conditioned on the current input, allowing the model to selectively propagate or discard information according to the sequence context\. This input\-dependent selection mechanism enables the model to adapt its effective memory to the content of the sequence\.

Stacking several SSM layers produces a causal sequence model that processes the input in a single forward pass and generates a hidden representationhkh\_\{k\}at each position\. These token\-wise hidden states can then be used by lightweight prediction heads for tasks such as next\-token prediction or temporal prediction\. This representation\-level property is the one exploited in this paper\.

### 2\.2\.Problem Setting

Let𝒟\\mathcal\{D\}denote a dataset of timestamped sequences\. Each sequence of lengthTTin𝒟\\mathcal\{D\}is written as\{\(xk,tk\)\}k=0T−1\\\{\(x\_\{k\},t\_\{k\}\)\\\}\_\{k=0\}^\{T\-1\}, wherexkx\_\{k\}is a discrete token andtkt\_\{k\}is its associated timestamp\. From successive timestamps, we derive the inter\-arrival times\{Δ​tk=tk−tk−1\}k=1T−1\\\{\\Delta t\_\{k\}=t\_\{k\}\-t\_\{k\-1\}\\\}\_\{k=1\}^\{T\-1\}\.

Denoting the token history up to stepkkby𝒳k=\{xi\}i=0k\\mathcal\{X\}\_\{k\}=\\\{x\_\{i\}\\\}\_\{i=0\}^\{k\}, our objective is to model the joint distribution of the next tokenxk\+1x\_\{k\+1\}and its associated inter\-arrival timeΔ​tk\+1\\Delta t\_\{k\+1\}:

\(3\)pγ​\(xk\+1,Δ​tk\+1∣𝒳k\)\.p\_\{\\gamma\}\(x\_\{k\+1\},\\Delta t\_\{k\+1\}\\mid\\mathcal\{X\}\_\{k\}\)\.
Using the chain rule, we factor this joint distribution as

\(4\)pγ​\(xk\+1,Δ​tk\+1∣𝒳k\)=pθ,α​\(xk\+1∣𝒳k\)​pθ,ϕ​\(Δ​tk\+1∣xk\+1,𝒳k\),p\_\{\\gamma\}\(x\_\{k\+1\},\\Delta t\_\{k\+1\}\\mid\\mathcal\{X\}\_\{k\}\)=p\_\{\\theta,\\alpha\}\(x\_\{k\+1\}\\mid\\mathcal\{X\}\_\{k\}\)\\,p\_\{\\theta,\\phi\}\(\\Delta t\_\{k\+1\}\\mid x\_\{k\+1\},\\mathcal\{X\}\_\{k\}\),whereγ=\{θ,α,ϕ\}\\gamma=\\\{\\theta,\\alpha,\\phi\\\}collects the parameters of the shared sequence model, token\-prediction head, and temporal\-prediction head, respectively\. LetVVdenote the vocabulary size andDDthe hidden\-state dimension\.

The shared sequence model is a causal autoregressive model parameterized byθ\\thetathat produces a token\-wise hidden representationhkh\_\{k\}from the history𝒳k\\mathcal\{X\}\_\{k\}\. In our main instantiation, we use a State Space Model, although the framework is not tied to this particular architecture\. Any causal autoregressive backbone that produces token\-wise hidden representations can be used instead; Appendix[appendixC](https://arxiv.org/html/2608.10120#A3)illustrates this with an additional Transformer\-based instantiation\. Formally, the backbone computes

\(5\)hk=Backboneθ​\(𝒳k\)\.h\_\{k\}=\\mathrm\{Backbone\}\_\{\\theta\}\(\\mathcal\{X\}\_\{k\}\)\.
The token\-prediction head is a linear mapWαtok∈ℝV×DW^\{\\mathrm\{tok\}\}\_\{\\alpha\}\\in\\mathbb\{R\}^\{V\\times D\}parameterized byα\\alpha, and it models the next\-token distribution as

\(6\)pθ,α​\(xk\+1∣𝒳k\)=softmax​\(Wαtok​hk\)\.p\_\{\\theta,\\alpha\}\(x\_\{k\+1\}\\mid\\mathcal\{X\}\_\{k\}\)=\\mathrm\{softmax\}\\left\(W^\{\\mathrm\{tok\}\}\_\{\\alpha\}h\_\{k\}\\right\)\.
The temporal\-prediction head is a functionfϕtime​\(⋅;rk\+1\)f^\{\\mathrm\{time\}\}\_\{\\phi\}\(\\cdot;r\_\{k\+1\}\)parameterized byϕ\\phiand conditioned on a transition representationrk\+1r\_\{k\+1\}\. Afterxk\+1x\_\{k\+1\}is processed, the sequence model produceshk\+1h\_\{k\+1\}according to[Eq\.5](https://arxiv.org/html/2608.10120#S2.E5)\. BecauseΔ​tk\+1\\Delta t\_\{k\+1\}describes the transition from positionkkto positionk\+1k\+1, its prediction is based on a representationrk\+1r\_\{k\+1\}derived from bothhkh\_\{k\}andhk\+1h\_\{k\+1\}\. The temporal\-prediction head models the second factor as

\(7\)pθ,ϕ​\(Δ​tk\+1∣xk\+1,𝒳k\)=fϕtime​\(Δ​tk\+1;rk\+1\)\.p\_\{\\theta,\\phi\}\(\\Delta t\_\{k\+1\}\\mid x\_\{k\+1\},\\mathcal\{X\}\_\{k\}\)=f^\{\\mathrm\{time\}\}\_\{\\phi\}\\left\(\\Delta t\_\{k\+1\};r\_\{k\+1\}\\right\)\.The construction ofrk\+1r\_\{k\+1\}is introduced in[section3](https://arxiv.org/html/2608.10120#S3)\.

## 3\.ChronoSSM

Having defined the modeling setting and notation, we now describe the design choices that instantiate ChronoSSM\. We first introduce the temporal representationrk\+1r\_\{k\+1\}supplied to the temporal\-prediction head in[Eq\.7](https://arxiv.org/html/2608.10120#S2.E7)\. We then define the training objective and compare two optimization schedules that differ in whether temporal supervision is allowed to update the shared backbone\.

### 3\.1\.Model Design

An inter\-arrival time measures the elapsed time between two consecutive tokens, rather than a property of either token considered in isolation\. We therefore represent it using the change between their corresponding hidden states:

\(8\)rk\+1=Δ​hk\+1=hk\+1−hk\.r\_\{k\+1\}=\\Delta h\_\{k\+1\}=h\_\{k\+1\}\-h\_\{k\}\.
The temporal target and the representation used to predict it are thus both defined as differences between consecutive positions:

\(9\)Δ​tk\+1=tk\+1−tk⏟temporal difference⟷Δ​hk\+1=hk\+1−hk⏟hidden\-state difference\.\\underbrace\{\\Delta t\_\{k\+1\}=t\_\{k\+1\}\-t\_\{k\}\}\_\{\\text\{temporal difference\}\}\\qquad\\longleftrightarrow\\qquad\\underbrace\{\\Delta h\_\{k\+1\}=h\_\{k\+1\}\-h\_\{k\}\}\_\{\\text\{hidden\-state difference\}\}\.This choice forrk\+1r\_\{k\+1\}is a modeling decision rather than a structural requirement of the framework\. An alternative temporal representationrk\+1′=\[hk;hk\+1\]r^\{\\prime\}\_\{k\+1\}=\[h\_\{k\};h\_\{k\+1\}\]is considered in Appendix[appendixF](https://arxiv.org/html/2608.10120#A6)\.

### 3\.2\.Training Objective

Under the factorization in[Eq\.4](https://arxiv.org/html/2608.10120#S2.E4), maximum\-likelihood training decomposes the negative log\-likelihood of each transition into a token\-prediction term and a temporal\-prediction term:

ℓk\\displaystyle\\ell\_\{k\}=−log⁡pγ​\(xk\+1,Δ​tk\+1∣𝒳k\)\\displaystyle=\-\\log p\_\{\\gamma\}\(x\_\{k\+1\},\\Delta t\_\{k\+1\}\\mid\\mathcal\{X\}\_\{k\}\)\(10\)=−log⁡pθ,α​\(xk\+1∣𝒳k\)⏟ℓktok\+−log⁡pθ,ϕ​\(Δ​tk\+1∣xk\+1,𝒳k\)⏟ℓktime\.\\displaystyle=\\underbrace\{\-\\log p\_\{\\theta,\\alpha\}\(x\_\{k\+1\}\\mid\\mathcal\{X\}\_\{k\}\)\}\_\{\\ell\_\{k\}^\{\\mathrm\{tok\}\}\}\+\\underbrace\{\-\\log p\_\{\\theta,\\phi\}\(\\Delta t\_\{k\+1\}\\mid x\_\{k\+1\},\\mathcal\{X\}\_\{k\}\)\}\_\{\\ell\_\{k\}^\{\\mathrm\{time\}\}\}\.We now instantiate these two terms using the token\-prediction and temporal\-prediction heads introduced in[section2\.2](https://arxiv.org/html/2608.10120#S2.SS2)\.

For token prediction, the factorpθ,α​\(xk\+1∣𝒳k\)p\_\{\\theta,\\alpha\}\(x\_\{k\+1\}\\mid\\mathcal\{X\}\_\{k\}\)is categorical, so the sequence\-level token loss is

\(11\)Ltok=∑k=0T−2ℓktok=∑k=0T−2CE​\(Wαtok​hk,xk\+1\),L\_\{\\mathrm\{tok\}\}=\\sum\_\{k=0\}^\{T\-2\}\\ell\_\{k\}^\{\\mathrm\{tok\}\}=\\sum\_\{k=0\}^\{T\-2\}\\mathrm\{CE\}\\left\(W^\{\\mathrm\{tok\}\}\_\{\\alpha\}h\_\{k\},x\_\{k\+1\}\\right\),whereCE\\mathrm\{CE\}denotes categorical cross\-entropy, which is appropriate because\{xk\}k=0T−1\\\{x\_\{k\}\\\}\_\{k=0\}^\{T\-1\}is a sequence of discrete tokens\(Goodfellowet al\.,[2016](https://arxiv.org/html/2608.10120#bib.bib33)\)\(see[section2\.2](https://arxiv.org/html/2608.10120#S2.SS2)\)\.

For temporal prediction, the factorpθ,ϕ​\(Δ​tk\+1∣xk\+1,𝒳k\)p\_\{\\theta,\\phi\}\(\\Delta t\_\{k\+1\}\\mid x\_\{k\+1\},\\mathcal\{X\}\_\{k\}\)is defined by a conditional density over the continuous\-valued targetΔ​tk\+1\\Delta t\_\{k\+1\}\. This definition is well posed provided that the chosen conditional density has support on the observed inter\-arrival targets, including zero when zero inter\-arrival times are present\. We therefore define the sequence\-level temporal loss as the negative log\-likelihood of the observed inter\-arrival times under that density:

\(12\)Ltime=∑k=0T−2ℓktime=∑k=0T−2−log⁡fϕtime​\(Δ​tk\+1;Δ​hk\+1\),L\_\{\\mathrm\{time\}\}=\\sum\_\{k=0\}^\{T\-2\}\\ell\_\{k\}^\{\\mathrm\{time\}\}=\\sum\_\{k=0\}^\{T\-2\}\-\\log f^\{\\mathrm\{time\}\}\_\{\\phi\}\\left\(\\Delta t\_\{k\+1\};\\Delta h\_\{k\+1\}\\right\),whereΔ​hk\+1\\Delta h\_\{k\+1\}is the hidden\-state transition defined in[Eq\.8](https://arxiv.org/html/2608.10120#S3.E8)\. The concrete form offϕtimef^\{\\mathrm\{time\}\}\_\{\\phi\}is specified for each experimental regime in[section4](https://arxiv.org/html/2608.10120#S4)\.

Summing[Eq\.10](https://arxiv.org/html/2608.10120#S3.E10)over the sequence gives the joint negative log\-likelihood

\(13\)ℒNLL=Ltok\+Ltime\.\\mathcal\{L\}\_\{\\mathrm\{NLL\}\}=L\_\{\\mathrm\{tok\}\}\+L\_\{\\mathrm\{time\}\}\.Minimizing this objective corresponds to maximum\-likelihood training of the factorized distribution over tokens and inter\-arrival times\.

In practice, the token and temporal losses may have different numerical scales\. We therefore optimize the weighted objective

\(14\)ℒ=Ltok\+λτ​Ltime,\\mathcal\{L\}=L\_\{\\mathrm\{tok\}\}\+\\lambda\_\{\\tau\}L\_\{\\mathrm\{time\}\},whereλτ≥0\\lambda\_\{\\tau\}\\geq 0controls the relative contribution of temporal prediction\.

### 3\.3\.Training Schedules

We consider two approaches for optimizing the parameter setγ=\{θ,α,ϕ\}\\gamma=\\\{\\theta,\\alpha,\\phi\\\}\. Both regimes use the same shared sequence backbone model, token\-prediction head, temporal\-prediction head, and loss components\. They differ only in whether the temporal loss is allowed to update the shared backbone\. The two regimes are illustrated in[Figure1](https://arxiv.org/html/2608.10120#S3.F1):[Figure1\(a\)](https://arxiv.org/html/2608.10120#S3.F1.sf1)shows theJointsetup, whereas[Figure1\(b\)](https://arxiv.org/html/2608.10120#S3.F1.sf2)shows theTwo\-Stageschedule\. The corresponding optimization procedures are summarized in[Algorithms1](https://arxiv.org/html/2608.10120#alg1)and[2](https://arxiv.org/html/2608.10120#alg2)\.

![Refer to caption](https://arxiv.org/html/2608.10120v1/x1.png)\(a\)Jointtraining updates the shared SSM backbone and both prediction heads using the token and temporal losses\.
![Refer to caption](https://arxiv.org/html/2608.10120v1/x2.png)\(b\)Two\-Stagetraining first optimizes the shared SSM backbone and token\-prediction head, then trains the temporal\-prediction head in a separate second stage\.

Figure 1\.Overview of theJointandTwo\-Stagetraining regimes\.
Algorithm 1Jointtraining1:Epochs

EE; backbone

Backboneθ\\mathrm\{Backbone\}\_\{\\theta\}; token head

WαtokW^\{\\mathrm\{tok\}\}\_\{\\alpha\}; temporal head

fϕtimef^\{\\mathrm\{time\}\}\_\{\\phi\}; dataset

𝒟\\mathcal\{D\}of timestamped sequences; sequence length

TT; temporal weight

λτ\\lambda\_\{\\tau\}
2:forepoch

=1,…,E=1,\\ldots,Edo

3:foreach

\{\(xk,tk\)\}k=0T−1∈𝒟\\\{\(x\_\{k\},t\_\{k\}\)\\\}\_\{k=0\}^\{T\-1\}\\in\\mathcal\{D\}do

4:

\(h0,…,hT−1\)←Backboneθ​\(x0,…,xT−1\)\(h\_\{0\},\\ldots,h\_\{T\-1\}\)\\leftarrow\\mathrm\{Backbone\}\_\{\\theta\}\(x\_\{0\},\\ldots,x\_\{T\-1\}\)
5:Compute

LtokL\_\{\\mathrm\{tok\}\}using[Eq\.11](https://arxiv.org/html/2608.10120#S3.E11)

6:Compute

LtimeL\_\{\\mathrm\{time\}\}using[Eq\.12](https://arxiv.org/html/2608.10120#S3.E12)

7:

ℒ←Ltok\+λτ​Ltime\\mathcal\{L\}\\leftarrow L\_\{\\mathrm\{tok\}\}\+\\lambda\_\{\\tau\}L\_\{\\mathrm\{time\}\}
8:Update

α\\alphausing

∇Ltok\\nabla L\_\{\\mathrm\{tok\}\}
9:Update

ϕ\\phiusing

∇Ltime\\nabla L\_\{\\mathrm\{time\}\}
10:Update

θ\\thetausing

∇ℒ\\nabla\\mathcal\{L\}
11:endfor

12:endfor

Algorithm 2Two\-Stagetraining1:Token\-training epochs

EtokE\_\{\\mathrm\{tok\}\}; temporal\-training epochs

EtimeE\_\{\\mathrm\{time\}\}; backbone

Backboneθ\\mathrm\{Backbone\}\_\{\\theta\}; token head

WαtokW^\{\\mathrm\{tok\}\}\_\{\\alpha\}; temporal head

fϕtimef^\{\\mathrm\{time\}\}\_\{\\phi\}; dataset

𝒟\\mathcal\{D\}of timestamped sequences; sequence length

TT
2:forepoch

=1,…,Etok=1,\\ldots,E\_\{\\mathrm\{tok\}\}do

3:foreach

\{\(xk,tk\)\}k=0T−1∈𝒟\\\{\(x\_\{k\},t\_\{k\}\)\\\}\_\{k=0\}^\{T\-1\}\\in\\mathcal\{D\}do

4:

\(h0,…,hT−1\)←Backboneθ​\(x0,…,xT−1\)\(h\_\{0\},\\ldots,h\_\{T\-1\}\)\\leftarrow\\mathrm\{Backbone\}\_\{\\theta\}\(x\_\{0\},\\ldots,x\_\{T\-1\}\)
5:Compute

LtokL\_\{\\mathrm\{tok\}\}using[Eq\.11](https://arxiv.org/html/2608.10120#S3.E11)

6:Update

θ\\thetaand

α\\alphausing

∇Ltok\\nabla L\_\{\\mathrm\{tok\}\}
7:endfor

8:endfor

9:Freeze

θ\\thetaand

α\\alpha
10:forepoch

=1,…,Etime=1,\\ldots,E\_\{\\mathrm\{time\}\}do

11:foreach

\{\(xk,tk\)\}k=0T−1∈𝒟\\\{\(x\_\{k\},t\_\{k\}\)\\\}\_\{k=0\}^\{T\-1\}\\in\\mathcal\{D\}do

12:

\(h0,…,hT−1\)←Backboneθ​\(x0,…,xT−1\)\(h\_\{0\},\\ldots,h\_\{T\-1\}\)\\leftarrow\\mathrm\{Backbone\}\_\{\\theta\}\(x\_\{0\},\\ldots,x\_\{T\-1\}\)
13:Compute

LtimeL\_\{\\mathrm\{time\}\}using[Eq\.12](https://arxiv.org/html/2608.10120#S3.E12)

14:Update

ϕ\\phiusing

∇Ltime\\nabla L\_\{\\mathrm\{time\}\}
15:endfor

16:endfor

JointTraining\.UnderJointtraining, the backbone and both prediction heads are optimized simultaneously using the objective in[Eq\.14](https://arxiv.org/html/2608.10120#S3.E14)\. Gradients from bothLtokL\_\{\\mathrm\{tok\}\}andLtimeL\_\{\\mathrm\{time\}\}update the backbone parametersθ\\theta, while the token\-prediction and temporal\-prediction heads are updated byLtokL\_\{\\mathrm\{tok\}\}andLtimeL\_\{\\mathrm\{time\}\}, respectively\.

In particular, the temporal loss backpropagates throughΔ​hk\+1=hk\+1−hk\\Delta h\_\{k\+1\}=h\_\{k\+1\}\-h\_\{k\}, encouraging changes between consecutive hidden states to become informative about the corresponding inter\-arrival times\. Because the token\-prediction head uses hidden states produced by the same backbone, temporal supervision can also affect the representations used for next\-token prediction\.

Two\-StageTraining\.UnderTwo\-Stagetraining, token and temporal learning are separated into two phases\. In the first phase, the backbone and token\-prediction head are optimized usingLtokL\_\{\\mathrm\{tok\}\}alone\. In the second phase, the temporal\-prediction head is trained usingLtimeL\_\{\\mathrm\{time\}\}on the hidden\-state transitions produced by the token\-trained sequence model\.

Because the backbone is not updated during the second phase, temporal gradients cannot modify its representations\. The temporal\-prediction head can therefore exploit only the temporal information already present in the token\-trained hidden\-state transitions\. This regime provides a controlled baseline for isolating the effect of allowing temporal supervision to shape the backbone\.

### 3\.4\.Inference

Generation proceeds autoregressively\. Given the token history𝒳k\\mathcal\{X\}\_\{k\}, the token head first predicts the next tokenxk\+1x\_\{k\+1\}\. After this token is processed by the backbone, the model obtainshk\+1h\_\{k\+1\}and computesΔ​hk\+1\\Delta h\_\{k\+1\}\. The temporal head then predicts the associated inter\-arrival timeΔ​t^k\+1\\widehat\{\\Delta t\}\_\{k\+1\}fromΔ​hk\+1\\Delta h\_\{k\+1\}, and the timestamp is updated astk\+1=tk\+Δ​t^k\+1t\_\{k\+1\}=t\_\{k\}\+\\widehat\{\\Delta t\}\_\{k\+1\}\.

The token\-decoding strategy and the point estimate used for temporal prediction are specified in[section4](https://arxiv.org/html/2608.10120#S4)\.

## 4\.Experimental Setup

Our experimental setup is designed to isolate the effect of allowing temporal supervision to update the shared backbone\. Our primary evaluation therefore comparesJointandTwo\-Stagetraining, which use the same model components, data, temporal objective, and generation procedure within each domain, but differ in whether the temporal loss updates the shared backbone\. This controlled comparison lets us determine whether this choice makes inter\-arrival information more recoverable from the learned representations \(RQ1in[section5](https://arxiv.org/html/2608.10120#S5)\) and whether any such improvement preserves token\-generation quality \(RQ2in[section6](https://arxiv.org/html/2608.10120#S6)\)\.

We conduct this controlled comparison on four domains exhibiting distinct sequence structures, vocabularies, timestamp densities, and temporal regimes: business\-process traces, clinical event sequences, network traffic, and temporal knowledge graphs\. Together, these four domains provide complementary tests ofRQ1andRQ2across dense and partial timestamp coverage, short and long sequences, and substantially different temporal distributions\. Appendix[appendixA](https://arxiv.org/html/2608.10120#A1)provides a detailed comparison of the structural and temporal properties of the four main evaluation domains, and Appendix[appendixB](https://arxiv.org/html/2608.10120#A2)reports additional experiments on musical event sequences\.

### 4\.1\.Experimental Instantiation

Unless stated otherwise, all experiments in the main paper instantiate ChronoSSM with a Mamba2 backbone\(Dao and Gu,[2024](https://arxiv.org/html/2608.10120#bib.bib5)\)\. To assess the robustness of the controlled comparison to the choice of causal autoregressive backbone, and to illustrate that the framework can be instantiated beyond SSMs, Appendix[appendixC](https://arxiv.org/html/2608.10120#A3)repeats the comparison with a GPT\-2 backbone\. The temporal\-head architecture is selected separately for each domain to account for differences in sequence structure and temporal distribution\.

Complete domain\-specific preprocessing settings, model architectures, temporal\-head configurations, loss weights, and training hyperparameters are reported in Appendix[appendixH](https://arxiv.org/html/2608.10120#A8)\. Appendix[appendixG](https://arxiv.org/html/2608.10120#A7)reports the corresponding training\-time and peak\-memory overhead\. Appendix[appendixE](https://arxiv.org/html/2608.10120#A5)reports an auxiliary\-loss\-weight sensitivity study for network traffic and temporal knowledge graph\.

### 4\.2\.Densely Timestamped Datasets

In this subsection, each sequence in𝒟\\mathcal\{D\}follows the canonical formulation introduced in[section2\.2](https://arxiv.org/html/2608.10120#S2.SS2): every tokenxkx\_\{k\}is associated with an observed timestamptkt\_\{k\}, so inter\-arrival times are defined for every consecutive pair of events\.

Business\-process traces\.BPI Challenge 2018\(van Dongen and Borchert,[2018](https://arxiv.org/html/2608.10120#bib.bib22); Mannhardtet al\.,[2018](https://arxiv.org/html/2608.10120#bib.bib23)\)\(BPI2018\) is a business\-process event log in which each case is represented as a sequence of timestamped activities\. Each complete case forms one process trace\. For every event, the activity label defines the tokenxkx\_\{k\}, while its occurrence time defines the associated timestamptkt\_\{k\}\.

Events are ordered chronologically within each trace\. When multiple activities share the same timestamp, they are retained as separate tokens and therefore produce valid zero\-valued inter\-arrival targets\. The original event log also contains resource identifiers, lifecycle transitions, application metadata, permit information, and other event\- and case\-level attributes, but we discard these auxiliary attributes and retain only the ordered activity labels and their timestamps to keep the modeling simple\.

Clinical event sequences\.MIMIC\-IV\(Johnsonet al\.,[2023](https://arxiv.org/html/2608.10120#bib.bib24)\)is a large\-scale electronic health record dataset containing clinical observations collected during hospital and intensive\-care stays\. We use thecharteventstable, which records timestamped bedside observations such as vital signs, ventilator settings, nursing assessments, and other charted clinical measurements\. The event defines the tokenxkx\_\{k\}, and the recording time defines the timestamptkt\_\{k\}\.

Events are ordered chronologically within each stay\. When multiple observations share the same timestamp, they are retained as separate tokens, yielding valid zero\-valued inter\-arrival times, as in BPI2018\. We discard the remaining clinical and administrative attributes and retain only the ordered event identities and their timestamps\.

Log\-transformed temporal targets\.Although the temporal regimes differ across the two dense domains, both datasets exhibit highly heterogeneous inter\-arrival times: BPI2018 contains very long delays, whereas MIMIC\-IV is bursty and strongly zero\-inflated, as summarized in Appendix[appendixA](https://arxiv.org/html/2608.10120#A1)\. We therefore predict a log\-transformed auxiliary target in both settings, preserving the monotone ordering of inter\-arrival times while compressing large gaps so that they do not dominate the temporal loss and short gaps remain well resolved\.

Because both datasets contain simultaneous events, the temporal prediction mechanism must accommodate observed inter\-arrival times that are exactly zero\. In particular, a plainlog⁡\(Δ​tk\)\\log\(\\Delta t\_\{k\}\)would be undefined whenΔ​tk=0\\Delta t\_\{k\}=0\. We therefore define

\(15\)zk=log⁡\(1\+Δ​tks\),z\_\{k\}=\\log\\left\(1\+\\frac\{\\Delta t\_\{k\}\}\{s\}\\right\),wheressis a dataset\-specific time scale\. This transformed quantityzkz\_\{k\}defines the auxiliary temporal target used for supervision\. The temporal head produces a raw scalar output, which is mapped through a softplus nonlinearity to obtain a non\-negative inter\-arrival predictionΔ​t^k\\widehat\{\\Delta t\}\_\{k\}\. This prediction is then transformed in the same way:

\(16\)z^k=log⁡\(1\+Δ​t^ks\)\.\\widehat\{z\}\_\{k\}=\\log\\left\(1\+\\frac\{\\widehat\{\\Delta t\}\_\{k\}\}\{s\}\\right\)\.
We instantiate the temporal factor as the fixed\-variance log\-normal density

\(17\)fϕtime​\(Δ​tk;Δ​hk\)=1\(s\+Δ​tk\)​2​π​σ2​exp⁡\(−\(zk−z^k\)22​σ2\),f^\{\\mathrm\{time\}\}\_\{\\phi\}\\left\(\\Delta t\_\{k\};\\Delta h\_\{k\}\\right\)=\\frac\{1\}\{\(s\+\\Delta t\_\{k\}\)\\sqrt\{2\\pi\\sigma^\{2\}\}\}\\exp\\left\(\-\\frac\{\(z\_\{k\}\-\\widehat\{z\}\_\{k\}\)^\{2\}\}\{2\\sigma^\{2\}\}\\right\),whereσ2\\sigma^\{2\}is fixed\. Its negative log\-likelihood is equivalent, up to terms independent of the model parameters, to squared error betweenz^k\\widehat\{z\}\_\{k\}andzkz\_\{k\}\. This is a modeling choice rather than a requirement of the framework\. Appendix[appendixB](https://arxiv.org/html/2608.10120#A2)reports an auxiliary GiantMIDI experiment that instead applies squared\-error regression to scaled raw inter\-arrival times, testing whether the results depend on the logarithmic target transformation\.

### 4\.3\.Partially Timestamped Datasets

Many temporally structured sequences do not provide a timestamp for every token but only for selected structural boundaries\. These settings are therefore important for evaluating whether temporal supervision can still shape useful representations when it is available only sparsely within the sequence\.

In this subsection, we consider domains in which timestamps are attached only to selected boundary tokens rather than to every token in the sequence\. Letℐtime\\mathcal\{I\}\_\{\\mathrm\{time\}\}denote the set of positions carrying valid temporal targets\. For eachk∈ℐtimek\\in\\mathcal\{I\}\_\{\\mathrm\{time\}\}, letk−k^\{\-\}denote the preceding position inℐtime\\mathcal\{I\}\_\{\\mathrm\{time\}\}\. The temporal target and its corresponding hidden\-state representation are then defined as

\(18\)Δ​tk=tk−tk−,Δ​hk=hk−hk−\.\\Delta t\_\{k\}=t\_\{k\}\-t\_\{k^\{\-\}\},\\qquad\\Delta h\_\{k\}=h\_\{k\}\-h\_\{k^\{\-\}\}\.
The temporal loss is evaluated only fork∈ℐtimek\\in\\mathcal\{I\}\_\{\\mathrm\{time\}\}\. Tokens outsideℐtime\\mathcal\{I\}\_\{\\mathrm\{time\}\}continue to contribute to the token\-generation objective but do not contribute directly to the temporal objective\. This defines a different setting from the canonical formulation introduced in[section2\.2](https://arxiv.org/html/2608.10120#S2.SS2)\.

Network traffic\.Network traffic consists of timestamped packets exchanged between hosts and recorded in packet\-capture files\. We use captures of video\-streaming traffic\(Bronzinoet al\.,[2019](https://arxiv.org/html/2608.10120#bib.bib34)\), following the preprocessing procedure of Chu et al\.\(Chuet al\.,[2026](https://arxiv.org/html/2608.10120#bib.bib18)\)\. Packet contents are serialized as sequences of byte tokens, with a dedicated<\|pkt\|\>token delimiting successive packets\. Timestamps are attached only to these<\|pkt\|\>boundary tokens, soℐtime\\mathcal\{I\}\_\{\\mathrm\{time\}\}contains the packet\-boundary positions\.

Temporal knowledge graphs\.Temporal knowledge graphs represent evolving relational information as timestamped facts\(h,r,o,τ\)\(h,r,o,\\tau\), wherehhandoodenote entities,rrdenotes a relation, andτ\\taudenotes the fact timestamp\. We use the GDELT temporal knowledge graph\(Jinet al\.,[2020](https://arxiv.org/html/2608.10120#bib.bib31)\), which contains world events extracted from news media and organized chronologically\.

Facts are mapped to a fixed vocabulary of entity and relation identifiers and represented as structured token blocks of the form<\|STM\|\>, h, r, o\. Consecutive facts are grouped into fixed\-length training sequences\. The timestamp of each fact is attached only to its<\|STM\|\>marker, soℐtime\\mathcal\{I\}\_\{\\mathrm\{time\}\}contains the<\|STM\|\>positions\.

Log\-transformed temporal targets\.To keep the temporal objective comparable across all domains, we apply the same target scaling, logarithmic transformation, softplus\-constrained prediction, and squared\-error loss introduced in[section4\.2](https://arxiv.org/html/2608.10120#S4.SS2)\. For network traffic only, packet inter\-arrival times are clipped at the 99th percentile before scaling\. This reduces the influence of rare extreme delays on the dataset\-specific scaling and training stability, beyond the compression already provided by the logarithmic transformation\.

### 4\.4\.Generation Settings

Token decoding is domain\-specific: BPI2018, MIMIC\-IV, and dynamic knowledge graphs use greedy decoding, whereas network traffic uses stochastic sampling, following Chu et al\.\(Chuet al\.,[2026](https://arxiv.org/html/2608.10120#bib.bib18)\)\. Complete decoding parameters are reported in Appendix[appendixH](https://arxiv.org/html/2608.10120#A8)\. For temporal prediction, the raw output of the temporal head is passed through a softplus nonlinearity to obtain the non\-negative point estimateΔ​t^k\\widehat\{\\Delta t\}\_\{k\}, which is added to the preceding generated timestamp\.

### 4\.5\.External Reference Baselines

For additional descriptive context, Appendix[appendixI](https://arxiv.org/html/2608.10120#A9)reports one retrained domain\-adapted baseline per dataset, including its architecture, probing adaptation, results, and interpretation\. These external models are not used to answerRQ1orRQ2, because they differ from ChronoSSM in architecture, objective, temporal representation, and generation procedure\.

## 5\.Temporal Recoverability

This section addressesRQ1 \(Temporal Recoverability\): whetherJointtraining makes inter\-arrival information more recoverable from the sequence\-backbone representations thanTwo\-Stagetraining\. To answer this question, we evaluate after training the same representation that is supplied to the temporal head\. The backbone is frozen, and temporal information is extracted either with a linear probe or through a non\-parametric analysis of representation geometry\. No probe gradients are propagated into the backbone, so the resulting measurements reflect information already present in the learned representations\.

### 5\.1\.Recoverability Metrics

In order to respond toRQ1, we define*temporal recoverability*as the extent to which the temporal target associated with an event transition can be extracted from frozen backbone representations using a simple readout or reflected in their geometry\. For each valid temporal transition, we evaluate the representation supplied to the temporal head and the transformed inter\-arrival target used during training\.

Linear timing probe\.The first recoverability metric consists of training a ridge\-regression probe to predict the temporal targetzkz\_\{k\}\(see[Eq\.15](https://arxiv.org/html/2608.10120#S4.E15)\) from the corresponding frozen temporal representationrkr\_\{k\}\(see[Eq\.8](https://arxiv.org/html/2608.10120#S3.E8)\):

\(19\)rk⟶zk\.r\_\{k\}\\longrightarrow z\_\{k\}\.Although this may resemble the second stage ofTwo\-Stagetraining, the two procedures serve different purposes\. TheTwo\-Stagetemporal head is part of the model itself and is optimized for downstream temporal prediction, whereas the probe is a post hoc diagnostic trained only after model training has finished\. It uses a simple ridge\-regression readout, does not reuse the trained temporal head, and is introduced solely to measure how easily temporal information can be extracted from the frozen representation\.

We report mean absolute error \(MAE\) and the coefficient of determination \(R2R^\{2\}\)\. A lower MAE indicates that the temporal target can be reconstructed more accurately from the representation, while a higherR2R^\{2\}indicates that the probe explains a larger fraction of its variability\. Lower MAE and higherR2R^\{2\}therefore indicate stronger temporal recoverability\.

Temporal Cohesion Score\.The second recoverability metric evaluates whether transitions associated with similar temporal targets occupy nearby regions of representation space\. For each temporal representationrkr\_\{k\}, we identify itsKKnearest neighbors under cosine similarity and compute the mean absolute difference between their associated temporal targets:

\(20\)dnn=1N​K​∑k=1N∑j∈𝒩K​\(k\)\|zk−zj\|,d\_\{\\mathrm\{nn\}\}=\\frac\{1\}\{NK\}\\sum\_\{k=1\}^\{N\}\\sum\_\{j\\in\\mathcal\{N\}\_\{K\}\(k\)\}\|z\_\{k\}\-z\_\{j\}\|,where𝒩K​\(k\)\\mathcal\{N\}\_\{K\}\(k\)denotes theKKnearest neighbors ofrkr\_\{k\}\.

We compare this quantity with the mean target difference obtained by assigningKKrandomly selected transitions to eachrkr\_\{k\}, denoteddrandd\_\{\\mathrm\{rand\}\}, and report

\(21\)TCS=drand−dnndrand\+ε,\\mathrm\{TCS\}=\\frac\{d\_\{\\mathrm\{rand\}\}\-d\_\{\\mathrm\{nn\}\}\}\{d\_\{\\mathrm\{rand\}\}\+\\varepsilon\},whereε\\varepsilonprevents division by zero\. In all experiments, we setK=5K=5and use cosine similarity\. Sensitivity to the neighborhood size and similarity measure is reported in Appendix[appendixJ](https://arxiv.org/html/2608.10120#A10)\.

A high TCS indicates that transitions with similar inter\-arrival times are substantially closer in representation space than expected under random pairing\. Temporal information is therefore more strongly reflected in the geometry of the learned representations\. Appendix[appendixK](https://arxiv.org/html/2608.10120#A11)provides a separate descriptive layer\-wise alignment analysis of the same representation geometry across domains\.

### 5\.2\.Response to RQ1

Table[1](https://arxiv.org/html/2608.10120#S5.T1)reports transition\-level temporal recoverability across the four evaluation domains\. In every domain,Jointachieves higher TCS and probeR2R^\{2\}, together with lower probe MAE, thanTwo\-Stage\. Thus, joint training improves both local temporal coherence and the linear accessibility of inter\-arrival information, under both dense and partial timestamp supervision\.

The diagnostic profile nevertheless differs by domain\. On BPI2018, TCS increases only slightly, from0\.7940\.794to0\.8030\.803, while the probe improvements are clearer:R2R^\{2\}increases from0\.4610\.461to0\.6310\.631and MAE decreases from3\.3423\.342to2\.7752\.775\. This pattern may suggest that joint training makes temporal information more linearly accessible without substantially changing local temporal neighborhoods\. MIMIC\-IV shows a similar pattern\. Network traffic, by contrast, shows large improvements in all three diagnostics, including TCS from0\.1470\.147to0\.6220\.622, probeR2R^\{2\}from0\.1900\.190to0\.3940\.394, and MAE from0\.3820\.382to0\.3080\.308\. The concurrent gains may suggest changes in both local temporal coherence and linear accessibility\. For the temporal knowledge graph, the changes are smaller across all three diagnostics than on the other domains: unlike BPI2018 and MIMIC\-IV, where the probe metrics improve more clearly than TCS, and network traffic, where all three diagnostics improve substantially, its gains remain modest but consistently favorJoint\.

Takeaways\.The controlled results provide an affirmative answer toRQ1: compared withTwo\-Stage,Jointmakes inter\-arrival information more recoverable from the representations supplied to the temporal head\.

Table 1\.Transition\-level temporal recoverability across domains\. Boldface identifies the better result within the controlledJoint–Two\-Stagecomparison\.

## 6\.Effect on Generation Quality

The previous section provided an affirmative answer toRQ1, showing thatJointtraining makes temporal information more recoverable from the representations supplied to the temporal head\. However, this improvement would be of limited value if it were obtained at the expense of the model’s primary generative capability\. We therefore turn toRQ2and test whether the additional temporal supervision introduced byJointtraining systematically degrades generation quality relative toTwo\-Stagetraining across the four evaluation domains\.

### 6\.1\.Preservation Metrics

We evaluate generated samples against held\-out data using domain\-specific protocols taken from the literature\.

Business\-process traces\.For BPI2018, we evaluate generated activity sequences using Damerau–Levenshtein similarity \(DLS\), following the suffix\-prediction protocol commonly used in predictive process monitoring\(Taxet al\.,[2017](https://arxiv.org/html/2608.10120#bib.bib46)\)\. For each evaluated prefix, similarity between the generated and reference suffixes is computed as

1−dDL​\(𝐱^,𝐱\)max⁡\(\|𝐱^\|,\|𝐱\|\),1\-\\frac\{d\_\{\\mathrm\{DL\}\}\(\\hat\{\\mathbf\{x\}\},\\mathbf\{x\}\)\}\{\\max\(\|\\hat\{\\mathbf\{x\}\}\|,\|\\mathbf\{x\}\|\)\},where \(dDLd\_\{\\mathrm\{DL\}\}\) denotes Damerau–Levenshtein distance\. We report the mean and median over all evaluated suffix comparisons\. Higher values indicate that the generated continuation more closely reproduces the activity ordering of the corresponding reference trace\.

Clinical event sequences\.For MIMIC\-IV, we compare the marginal distribution and coverage of generated clinical event types against held\-out sequences\. We report Jensen–Shannon divergence and total variation distance between the generated and reference event\-token distributions, with lower values indicating closer agreement\. We also report recall over the 50 most frequent reference event types, defined as the fraction of these event types that appear at least once in the generated data\. Higher recall indicates better coverage of frequent clinical events\.

Network traffic\.We compare marginal distributions of key packet\-header fields between generated and real traffic, following prior work\(Jianget al\.,[2024](https://arxiv.org/html/2608.10120#bib.bib37); Chuet al\.,[2026](https://arxiv.org/html/2608.10120#bib.bib18)\)\. We report Jensen–Shannon divergence for source and destination IP addresses and TCP ports\. Lower divergence indicates that generated packets more faithfully reproduce the distributional characteristics of real network traffic\.

Temporal knowledge graphs\.We use filtered link prediction following\(Jinet al\.,[2020](https://arxiv.org/html/2608.10120#bib.bib31)\)\. Given a query prefix, the model ranks candidate objects while masking facts that appear elsewhere in the dataset\. We report mean reciprocal rank, mean rank, and Hits@\(1,3,10\)\. Higher mean reciprocal rank and Hits scores, and lower mean rank, indicate that the model better captures the relational structure required to complete a fact\.

Table 2\.Content\-generation quality across domains\. Boldface identifies the better result within the controlledJoint–Two\-Stagecomparison\.
### 6\.2\.Response to RQ2

Table[2](https://arxiv.org/html/2608.10120#S6.T2)reports content\-generation quality across the four domains\. The controlled comparison provides a negative answer toRQ2:Jointtraining does not systematically degrade generative performance relative toTwo\-Stagetraining\. Its effect is nevertheless domain\-dependent, but provides no evidence of a systematic degradation in generative performance\.

On BPI2018,JointandTwo\-Stageobtain identical mean and median DLS\. The additional temporal supervision therefore changes neither the generated activity suffixes nor their similarity to the reference traces in this setting\. On MIMIC\-IV,Two\-Stageachieves slightly lower token JSD and total variation distance, while both regimes obtain the same recall over the 50 most frequent clinical event types\. Thus,Jointintroduces a small degradation in matching the marginal event\-token distribution, but does not reduce coverage of frequent clinical events\. Appendix[appendixD](https://arxiv.org/html/2608.10120#A4)reports the complementary held\-out validation\-loss diagnostics: MIMIC\-IV is also the only main domain in whichJointhas a slightly higher token loss thanTwo\-Stage\(1\.296 versus 1\.293, a difference of about 0\.2%\)\. This small teacher\-forced loss difference is consistent with the limited distributional change, but does not indicate a broad degradation in generation quality: frequent\-event coverage is unchanged\.

In contrast,Jointimproves content\-generation quality in both partially timestamped domains\. On network traffic,Jointconsistently lowers the JSD of all four evaluated packet\-header fields, with particularly large improvements for source and destination TCP ports\. On temporal knowledge graphs,Jointalso improves every filtered link\-prediction metric, yielding substantially higher MRR and Hits@\{1,3,10\}\\\{1,3,10\\\}, together with a markedly lower mean rank\. In these domains, temporal supervision not only preserves content generation but appears to improve the structural regularities captured by the token generator\. The lower held\-outLtokL\_\{\\mathrm\{tok\}\}underJointin both domains is consistent with these output\-level improvements \(see[AppendixD](https://arxiv.org/html/2608.10120#A4)\)\.

Takeaways\.The results provide a negative answer toRQ2\. AlthoughJointdoes not dominateTwo\-Stageon every individual metric, there is no evidence that the improved temporal recoverability established inRQ1is obtained at the cost of a systematic degradation in the model’s primary generative capability\.

## 7\.Related Work

The prediction problem formalized in[Eq\.4](https://arxiv.org/html/2608.10120#S2.E4)has long been studied in the point\-process literature\(Daley and Vere\-Jones,[2008](https://arxiv.org/html/2608.10120#bib.bib42)\)\. Recent neural approaches have used recurrent architectures\(Boydet al\.,[2020](https://arxiv.org/html/2608.10120#bib.bib43); Duet al\.,[2016](https://arxiv.org/html/2608.10120#bib.bib45)\)and transformers\(Panos,[2024](https://arxiv.org/html/2608.10120#bib.bib44)\)to model continuous\-time event sequences\. Our focus is on the stricter generative setting, where both event identity and timing must be produced autoregressively\. Prior work in this domain typically either decouples timing from content generation or does not model timing at all\. For example, some methods assign inter\-arrival times with separate components after content generation\(Chuet al\.,[2026](https://arxiv.org/html/2608.10120#bib.bib18); Jianget al\.,[2024](https://arxiv.org/html/2608.10120#bib.bib37)\), whereas others focus on event generation without modeling inter\-arrival times within the same autoregressive generator\(Yinet al\.,[2022](https://arxiv.org/html/2608.10120#bib.bib38); Jinet al\.,[2020](https://arxiv.org/html/2608.10120#bib.bib31)\)\. By contrast, ChronoSSM focuses on how the event and temporal factors in[Eq\.4](https://arxiv.org/html/2608.10120#S2.E4)interact when they are learned within the same autoregressive sequence model\.

A key distinction is that, even when prior work models event identity and timing jointly within a common model\(Boydet al\.,[2020](https://arxiv.org/html/2608.10120#bib.bib43); Duet al\.,[2016](https://arxiv.org/html/2608.10120#bib.bib45); Panos,[2024](https://arxiv.org/html/2608.10120#bib.bib44)\), it does not study how the temporal termpθ,ϕ​\(Δ​tk\+1∣xk\+1,𝒳k\)p\_\{\\theta,\\phi\}\(\\Delta t\_\{k\+1\}\\mid x\_\{k\+1\},\\mathcal\{X\}\_\{k\}\)should interact with the event termpθ,α​\(xk\+1∣𝒳k\)p\_\{\\theta,\\alpha\}\(x\_\{k\+1\}\\mid\\mathcal\{X\}\_\{k\}\)from a representation\-learning perspective when both are learned within the same model\. This is the specific question studied in this paper\.

## 8\.Discussion and Future Work

We introduced ChronoSSM, an autoregressive model that jointly predicts tokens and inter\-event times from a shared backbone\. Across four temporally structured domains,Jointtraining yields representations from which temporal information is more easily recoverable \(RQ1\)\. At the same time, it does not systematically degrade downstream generation quality relative toTwo\-Stagetraining \(RQ2\): generation quality is unchanged on BPI2018, modestly reduced on selected MIMIC\-IV distributional metrics, and improved on network traffic and temporal knowledge graphs\. These results indicate that allowing timing gradients to shape the shared backbone can improve temporal recoverability without imposing a systematic cost on generative performance\.

### 8\.1\.Linking Temporal Recoverability and Generation Quality

The relationship between temporal recoverability and content\-generation quality is domain\-dependent\. On BPI2018, improved temporal recoverability is accompanied by unchanged activity\-sequence quality\. On MIMIC\-IV, it coexists with a modest degradation in marginal event\-token distribution metrics, but no reduction in frequent\-event coverage\. On network traffic and temporal knowledge graphs,Jointtraining is associated with better evaluated content\-generation metrics alongside improved temporal recoverability\. These results indicate that stronger temporal organization does not necessarily impose a uniform temporal–semantic trade\-off\. Instead, its effect depends on the domain structure, the density of temporal supervision, and the content properties captured by the evaluation protocol\.

Validation loss analysis:In addition to the analysis described above, we also carry out a direct check of the effectiveness of the temporal head in predicting inter\-arrival times, given the true difference in token representations \(see[Eq\.12](https://arxiv.org/html/2608.10120#S3.E12)\)\. We compute the temporal loss on the held\-out validation data for each dataset \(which has the true inter\-arrival times\), and report it in Appendix[appendixD](https://arxiv.org/html/2608.10120#A4)\. We observe that the timing loss for most cases, is lower using theJointmethod\. However, this is a only a coarse measurement how well the time head has learned the underlying distribution of temporal data\. A more careful check would require multiple, long generation sequences, appropriately calibrated against the training data, after which the temporal distributions would be compared for real and generated data\. As we note in[section8\.3](https://arxiv.org/html/2608.10120#S8.SS3), a rigorous evaluation pipeline for this would be an interesting direction for future work\.

### 8\.2\.Towards a Mechanistic Understanding

Appendix[appendixK](https://arxiv.org/html/2608.10120#A11)compares the layer\-wise representations learned by the token\-generation model with and without explicit temporal supervision\. We use Centered Kernel Alignment \(CKA\), Centered Kernel Nearest Neighbor Alignment \(CKNNA\), and Mutualkk\-Nearest Neighbor \(M\-KNN\)\. Across domains, temporal supervision changes the learned representations, but the form of this change varies: in some cases it affects both global and local geometry, whereas in others it is primarily local\. Changes are also generally less pronounced in earlier layers, which may reflect their stronger focus on the input structure\. The patterns nevertheless differ across the four domains, so they do not yet provide a single mechanistic account of how temporal supervision reshapes representation learning in SSMs\.

### 8\.3\.Future Work

Several questions remain about the conditions under which temporal supervision is beneficial\. Building on the sensitivity analyses in Appendix[appendixE](https://arxiv.org/html/2608.10120#A5), future work should test more systematically how the gains fromJointtraining depend on temporal\-supervision strength across all domains, temporal\-head capacity, and the structure of the target domain\. This would clarify the conditions under which temporal gradients beneficially reshape the backbone\.

The practical value of improved temporal recoverability also warrants direct evaluation\. Future work should test whether it translates into better performance on downstream tasks that explicitly depend on temporal structure, such as anomaly detection, forecasting, or temporal retrieval\. This would help establish whether the representation\-level gains observed here are not only diagnostic but also practically useful across temporally structured domains\.

Finally, future work should assess the distributional fidelity of timestamps produced during free autoregressive generation, testing whether the representation\-level gains translate into faithful temporal synthesis beyond the content\-generation metrics considered in this study\. ForRQ2, we prioritized domain\-specific generation protocols established in the respective application literatures, allowing a controlled comparison of the two training regimes using metrics with existing task\-specific meaning\. These protocols primarily assess event content, however\. Developing standardized evaluation procedures for jointly generated event content and timestamps is therefore an important next step, particularly because existing generative approaches often decouple temporal prediction from content generation \([section7](https://arxiv.org/html/2608.10120#S7)\)\.

## References

- A\. Boyd, R\. Bamler, S\. Mandt, and P\. Smyth \(2020\)User\-dependent neural sequence models for continuous\-time event data\.Advances in Neural Information Processing Systems33,pp\. 21488–21499\.Cited by:[§7](https://arxiv.org/html/2608.10120#S7.p1.1),[§7](https://arxiv.org/html/2608.10120#S7.p2.2)\.
- F\. Bronzino, P\. Schmitt, S\. Ayoubi, G\. Martins, R\. Teixeira, and N\. Feamster \(2019\)Inferring streaming video quality from encrypted traffic: practical models and deployment experience\.Proc\. ACM Meas\. Anal\. Comput\. Syst\.3\(3\)\.External Links:[Link](https://doi.org/10.1145/3366704),[Document](https://dx.doi.org/10.1145/3366704)Cited by:[§1](https://arxiv.org/html/2608.10120#S1.p1.1),[§1](https://arxiv.org/html/2608.10120#S1.p4.1),[§4\.3](https://arxiv.org/html/2608.10120#S4.SS3.p4.1)\.
- A\. Chu, X\. Jiang, S\. Liu, A\. Bhagoji, F\. Bronzino, P\. Schmitt, and N\. Feamster \(2026\)NetSSM: multi\-flow and state\-aware network trace generation using state\-space models\.Proc\. ACM Netw\.4\(CoNEXT1\)\.External Links:[Link](https://doi.org/10.1145/3786289),[Document](https://dx.doi.org/10.1145/3786289)Cited by:[§I\.1](https://arxiv.org/html/2608.10120#A9.SS1.p4.1),[Table 12](https://arxiv.org/html/2608.10120#A9.T12.2.4.3.2.1.1),[§1](https://arxiv.org/html/2608.10120#S1.p2.1),[§1](https://arxiv.org/html/2608.10120#S1.p4.1),[§4\.3](https://arxiv.org/html/2608.10120#S4.SS3.p4.1),[§4\.4](https://arxiv.org/html/2608.10120#S4.SS4.p1.1),[§6\.1](https://arxiv.org/html/2608.10120#S6.SS1.p4.1),[§7](https://arxiv.org/html/2608.10120#S7.p1.1)\.
- J\. Cüppers, A\. Schoen, G\. Blanc, and P\. Gimenez \(2024\)FlowChronicle: synthetic network flow generation through pattern set mining\.Proc\. ACM Netw\.2\(CoNEXT4\)\.External Links:[Link](https://doi.org/10.1145/3696407),[Document](https://dx.doi.org/10.1145/3696407)Cited by:[§1](https://arxiv.org/html/2608.10120#S1.p1.1)\.
- D\. J\. Daley and D\. Vere\-Jones \(2008\)An introduction to the theory of point processes: volume ii: general theory and structure\.Springer\.Cited by:[§7](https://arxiv.org/html/2608.10120#S7.p1.1)\.
- T\. Dao and A\. Gu \(2024\)Transformers are ssms: generalized models and efficient algorithms through structured state space duality\.InProceedings of the 41st International Conference on Machine Learning,ICML’24\.Cited by:[§4\.1](https://arxiv.org/html/2608.10120#S4.SS1.p1.1)\.
- N\. Du, H\. Dai, R\. Trivedi, U\. Upadhyay, M\. Gomez\-Rodriguez, and L\. Song \(2016\)Recurrent marked temporal point processes: embedding event history to vector\.InProceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining,pp\. 1555–1564\.Cited by:[§7](https://arxiv.org/html/2608.10120#S7.p1.1),[§7](https://arxiv.org/html/2608.10120#S7.p2.2)\.
- A\. Fichtl, J\. Bohn, J\. Kelber, E\. Mosca, and G\. Groh \(2025\)The end of transformers? on challenging attention and the rise of sub\-quadratic architectures\.ArXivabs/2510\.05364\.External Links:[Link](https://api.semanticscholar.org/CorpusID:281886741)Cited by:[§2\.1](https://arxiv.org/html/2608.10120#S2.SS1.p1.1)\.
- I\. Goodfellow, Y\. Bengio, and A\. Courville \(2016\)Deep learning\.MIT Press\.Cited by:[§3\.2](https://arxiv.org/html/2608.10120#S3.SS2.p2.3)\.
- A\. Gretton, K\. Fukumizu, C\. Teo, L\. Song, B\. Schölkopf, and A\. Smola \(2007\)A kernel statistical test of independence\.InAdvances in Neural Information Processing Systems,J\. Platt, D\. Koller, Y\. Singer, and S\. Roweis \(Eds\.\),Vol\.20,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2007/file/d5cfead94f5350c12c322b5b664544c1-Paper.pdf)Cited by:[§K\.1](https://arxiv.org/html/2608.10120#A11.SS1.p3.5)\.
- A\. Gu, K\. Goel, and C\. Ré \(2022\)Efficiently modeling long sequences with structured state spaces\.External Links:2111\.00396,[Link](https://arxiv.org/abs/2111.00396)Cited by:[§2\.1](https://arxiv.org/html/2608.10120#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2608.10120#S2.SS1.p2.4)\.
- C\. Hawthorne, A\. Stasyuk, A\. Roberts, I\. Simon, C\. A\. Huang, S\. Dieleman, E\. Elsen, J\. Engel, and D\. Eck \(2019\)Enabling factorized piano music modeling and generation with the MAESTRO dataset\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=r1lYRjC9F7)Cited by:[Appendix B](https://arxiv.org/html/2608.10120#A2.p1.2),[§I\.1](https://arxiv.org/html/2608.10120#A9.SS1.p6.1),[§1](https://arxiv.org/html/2608.10120#S1.p4.1)\.
- C\. A\. Huang, A\. Vaswani, J\. Uszkoreit, N\. Shazeer, I\. Simon, C\. Hawthorne, A\. M\. Dai, M\. D\. Hoffman, M\. Dinculescu, and D\. Eck \(2019\)Music transformer: generating music with long\-term structure\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=rJe4ShAcF7)Cited by:[Appendix B](https://arxiv.org/html/2608.10120#A2.p2.1),[§I\.1](https://arxiv.org/html/2608.10120#A9.SS1.p6.1),[Table 12](https://arxiv.org/html/2608.10120#A9.T12.2.6.5.2.1.1),[§1](https://arxiv.org/html/2608.10120#S1.p4.1)\.
- M\. Huh, B\. Cheung, T\. Wang, and P\. Isola \(2024\)Position: the platonic representation hypothesis\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 20617–20642\.External Links:[Link](https://proceedings.mlr.press/v235/huh24a.html)Cited by:[§K\.1](https://arxiv.org/html/2608.10120#A11.SS1.p4.5)\.
- X\. Jiang, S\. Liu, A\. Gember\-Jacobson, A\. N\. Bhagoji, P\. Schmitt, F\. Bronzino, and N\. Feamster \(2024\)Netdiffusion: network data augmentation through protocol\-constrained traffic generation\.Proceedings of the ACM on Measurement and Analysis of Computing Systems8\(1\),pp\. 1–32\.Cited by:[§1](https://arxiv.org/html/2608.10120#S1.p2.1),[§6\.1](https://arxiv.org/html/2608.10120#S6.SS1.p4.1),[§7](https://arxiv.org/html/2608.10120#S7.p1.1)\.
- W\. Jin, M\. Qu, X\. Jin, and X\. Ren \(2020\)Recurrent event network: autoregressive structure inferenceover temporal knowledge graphs\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),B\. Webber, T\. Cohn, Y\. He, and Y\. Liu \(Eds\.\),Online,pp\. 6669–6683\.External Links:[Link](https://aclanthology.org/2020.emnlp-main.541/),[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.541)Cited by:[§I\.1](https://arxiv.org/html/2608.10120#A9.SS1.p5.1),[Table 12](https://arxiv.org/html/2608.10120#A9.T12.2.5.4.2.1.1),[§1](https://arxiv.org/html/2608.10120#S1.p1.1),[§1](https://arxiv.org/html/2608.10120#S1.p4.1),[§4\.3](https://arxiv.org/html/2608.10120#S4.SS3.p5.5),[§6\.1](https://arxiv.org/html/2608.10120#S6.SS1.p5.1),[§7](https://arxiv.org/html/2608.10120#S7.p1.1)\.
- A\. E\. W\. Johnson, L\. Bulgarelli, T\. J\. Pollard, S\. Horng, L\. A\. Celi, and R\. G\. Mark \(2023\)MIMIC\-IV, a freely accessible electronic health record dataset\.Scientific Data10\(1\),pp\. 1\.External Links:[Document](https://dx.doi.org/10.1038/s41597-022-01899-x)Cited by:[§1](https://arxiv.org/html/2608.10120#S1.p1.1),[§1](https://arxiv.org/html/2608.10120#S1.p4.1),[§4\.2](https://arxiv.org/html/2608.10120#S4.SS2.p4.2)\.
- Q\. Kong, B\. Li, X\. Song, Q\. Meng, M\. Barlow, and S\. Dixon \(2022\)GiantMIDI\-Piano: a large\-scale MIDI dataset for classical piano music\.Transactions of the International Society for Music Information Retrieval5\(1\),pp\. 87–102\.External Links:[Document](https://dx.doi.org/10.5334/tismir.80)Cited by:[Appendix B](https://arxiv.org/html/2608.10120#A2.p1.2),[§I\.1](https://arxiv.org/html/2608.10120#A9.SS1.p6.1),[§1](https://arxiv.org/html/2608.10120#S1.p4.1)\.
- S\. Kornblith, M\. Norouzi, H\. Lee, and G\. Hinton \(2019\)Similarity of neural network representations revisited\.InProceedings of the 36th International Conference on Machine Learning,K\. Chaudhuri and R\. Salakhutdinov \(Eds\.\),Proceedings of Machine Learning Research, Vol\.97,pp\. 3519–3529\.External Links:[Link](https://proceedings.mlr.press/v97/kornblith19a.html)Cited by:[§K\.1](https://arxiv.org/html/2608.10120#A11.SS1.p3.3)\.
- M\. Lanvin, P\. Gimenez, Y\. Han, F\. Majorczyk, L\. Mé, and E\. Totel \(2023\)Towards understanding alerts raised by unsupervised network intrusion detection systems\.InProceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses,RAID ’23,New York, NY, USA,pp\. 135–150\.External Links:ISBN 9798400707650,[Link](https://doi.org/10.1145/3607199.3607247),[Document](https://dx.doi.org/10.1145/3607199.3607247)Cited by:[§1](https://arxiv.org/html/2608.10120#S1.p1.1)\.
- I\. Loshchilov and F\. Hutter \(2019\)Decoupled weight decay regularization\.External Links:1711\.05101,[Link](https://arxiv.org/abs/1711.05101)Cited by:[Appendix H](https://arxiv.org/html/2608.10120#A8.p2.14)\.
- F\. Mannhardt, M\. de Leoni, H\. A\. Reijers, and W\. M\. P\. van der Aalst \(2018\)The BPI challenge 2018\.InProceedings of the Business Process Intelligence Challenge 2018,Cited by:[§1](https://arxiv.org/html/2608.10120#S1.p4.1),[§4\.2](https://arxiv.org/html/2608.10120#S4.SS2.p2.2)\.
- A\. Panos \(2024\)Decomposable transformer point processes\.Advances in Neural Information Processing Systems37,pp\. 88932–88955\.Cited by:[§7](https://arxiv.org/html/2608.10120#S7.p1.1),[§7](https://arxiv.org/html/2608.10120#S7.p2.2)\.
- J\. Paparrizos, P\. Boniol, Q\. Liu, and T\. Palpanas \(2025\)Advances in time\-series anomaly detection: algorithms, benchmarks, and evaluation measures\.InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\.2,KDD ’25,New York, NY, USA,pp\. 6151–6161\.External Links:ISBN 9798400714542,[Link](https://doi.org/10.1145/3711896.3736565),[Document](https://dx.doi.org/10.1145/3711896.3736565)Cited by:[§1](https://arxiv.org/html/2608.10120#S1.p1.1)\.
- N\. Tax, I\. Verenich, M\. La Rosa, and M\. Dumas \(2017\)Predictive business process monitoring with LSTM neural networks\.InAdvanced Information Systems Engineering,Lecture Notes in Computer Science, Vol\.10253,pp\. 477–492\.External Links:[Document](https://dx.doi.org/10.1007/978-3-319-59536-8%5F30)Cited by:[§I\.1](https://arxiv.org/html/2608.10120#A9.SS1.p2.1),[Table 12](https://arxiv.org/html/2608.10120#A9.T12.2.2.1.2.1.1),[§6\.1](https://arxiv.org/html/2608.10120#S6.SS1.p2.2)\.
- B\. van Dongen and F\. \(\. Borchert \(2018\)BPI challenge 2018\.Eindhoven University of Technology\(en\)\.External Links:[Document](https://dx.doi.org/10.4121/UUID%3A3301445F-95E8-4FF0-98A4-901F1F204972),[Link](https://data.4tu.nl/articles/_/12688355/1)Cited by:[§1](https://arxiv.org/html/2608.10120#S1.p1.1),[§1](https://arxiv.org/html/2608.10120#S1.p4.1),[§4\.2](https://arxiv.org/html/2608.10120#S4.SS2.p2.2)\.
- Y\. Yin, Z\. Lin, M\. Jin, G\. Fanti, and V\. Sekar \(2022\)Practical gan\-based synthetic ip header trace generation using netshare\.InProceedings of the ACM SIGCOMM 2022 Conference,pp\. 458–472\.Cited by:[§7](https://arxiv.org/html/2608.10120#S7.p1.1)\.
- S\. Zuo, H\. Jiang, Z\. Li, T\. Zhao, and H\. Zha \(2020\)Transformer hawkes process\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,pp\. 11692–11702\.External Links:[Link](https://proceedings.mlr.press/v119/zuo20a.html)Cited by:[§I\.1](https://arxiv.org/html/2608.10120#A9.SS1.p3.1),[Table 12](https://arxiv.org/html/2608.10120#A9.T12.2.3.2.2.1.1)\.

## Appendix Overview

In this appendix, we provide additional experimental details, robustness analyses, and supporting results that complement the main paper\. In particular, we \(i\) compare the experimental domains and document the model configurations, \(ii\) evaluate the robustness of the results across datasets, backbones, temporal\-loss weights, and temporal representations, \(iii\) report held\-out validation\-loss diagnostics, \(iv\) provide comparisons with domain\-specific baselines, and \(v\) further analyze the recoverability metrics and the layer\-wise geometry of the learned representations\. The appendix is organized as follows:

1. \(1\)Comparison of the experimental domains \([appendixA](https://arxiv.org/html/2608.10120#A1)\)\.
2. \(2\)Additional experiments on symbolic music \([appendixB](https://arxiv.org/html/2608.10120#A2)\)\.
3. \(3\)Application to an alternative autoregressive backbone \([appendixC](https://arxiv.org/html/2608.10120#A3)\)\.
4. \(4\)Held\-out validation\-loss diagnostics for the main experiments \([appendixD](https://arxiv.org/html/2608.10120#A4)\)\.
5. \(5\)Sensitivity to the auxiliary temporal\-loss weight \([appendixE](https://arxiv.org/html/2608.10120#A5)\)\.
6. \(6\)Comparison of alternative temporal representations \([appendixF](https://arxiv.org/html/2608.10120#A6)\)\.
7. \(7\)Computational overhead \([appendixG](https://arxiv.org/html/2608.10120#A7)\)\.
8. \(8\)Domain\-specific model configurations \([appendixH](https://arxiv.org/html/2608.10120#A8)\)\.
9. \(9\)Domain\-specific baseline setup and results \([appendixI](https://arxiv.org/html/2608.10120#A9)\)\.
10. \(10\)Sensitivity of TCS to its metric parameters \([appendixJ](https://arxiv.org/html/2608.10120#A10)\)\.
11. \(11\)Layer\-wise representation analysis \([appendixK](https://arxiv.org/html/2608.10120#A11)\)\.

## Appendix AComparison of Experimental Domains

This appendix expands on the four\-domain evaluation introduced in[section4\.2](https://arxiv.org/html/2608.10120#S4.SS2)and[section4\.3](https://arxiv.org/html/2608.10120#S4.SS3), summarizing why the evaluation spans genuinely different settings\. The main paper compares two densely timestamped domains \(BPI2018 and MIMIC\-IV\) and two partially timestamped domains \(network traffic and temporal knowledge graphs\), while the auxiliary GiantMIDI experiment adds a dense musical setting\. These datasets differ not only in application domain, but also in the granularity of the atomic event, the meaning and size of the token vocabulary, the scale of the input sequences, the positions at which timestamps are available, and the characteristic shape of the temporal distribution\.

Table 3\.High\-level comparison of the main evaluation domains and the auxiliary GiantMIDI benchmark\.In[Table3](https://arxiv.org/html/2608.10120#A1.T3), we see that the datasets differ strongly even within each timestamping regime\. Among the dense domains, BPI2018 is comparatively short, template\-like, and dominated by long waiting periods between process stages, whereas MIMIC\-IV is much longer, more heterogeneous, and strongly zero\-inflated because many clinical events share the same chart time\. GiantMIDI is also dense, but it operates at a much finer temporal scale, with sub\-second note timing and many simultaneous events due to chords\. Among the partial domains, temporal knowledge graphs use a large symbolic vocabulary and relatively coarse fact\-level timing, whereas network traffic uses a much smaller byte\-level vocabulary and extremely dense packet timing\.

These differences matter directly for the interpretation of the main results\. Because the same qualitative effect of joint token–time training appears across these settings, it is less likely to be an artifact of a single temporal scale, single supervision pattern, or single type of Two\-Stage structure\.

## Appendix BMusic Event Experiments

Dataset and event representation\.We use GiantMIDI\(Konget al\.,[2022](https://arxiv.org/html/2608.10120#bib.bib25)\), a large\-scale corpus of classical piano MIDI performances\. Each piece is converted into an ordered sequence of note events as in MAESTRO\-style symbolic music modeling\(Hawthorneet al\.,[2019](https://arxiv.org/html/2608.10120#bib.bib26)\): the tokenxkx\_\{k\}is note pitch and the timestamptkt\_\{k\}is note onset\. We retain only pitch events and onset times and discard note duration, velocity, composer and performer identifiers, YouTube metadata, and transcription metadata\. As noted in[section4\.2](https://arxiv.org/html/2608.10120#S4.SS2), this auxiliary experiment also differs from the main dense\-domain setup in its temporal supervision: instead of the log\-gap target, it regresses directly on scaled raw inter\-arrival times\. More specifically, we define the scaled target

\(22\)uk=Δ​tksmusic,u^k=softplus⁡\(gϕ​\(Δ​hk\)\),u\_\{k\}=\\frac\{\\Delta t\_\{k\}\}\{s\_\{\\mathrm\{music\}\}\},\\qquad\\widehat\{u\}\_\{k\}=\\operatorname\{softplus\}\\\!\\left\(g\_\{\\phi\}\(\\Delta h\_\{k\}\)\\right\),and instantiate the temporal factor as

\(23\)fϕtime​\(Δ​tk;Δ​hk\)=1smusic​2​π​σ2​exp⁡\(−\(uk−u^k\)22​σ2\),f^\{\\mathrm\{time\}\}\_\{\\phi\}\\left\(\\Delta t\_\{k\};\\Delta h\_\{k\}\\right\)=\\frac\{1\}\{s\_\{\\mathrm\{music\}\}\\sqrt\{2\\pi\\sigma^\{2\}\}\}\\exp\\left\(\-\\frac\{\(u\_\{k\}\-\\widehat\{u\}\_\{k\}\)^\{2\}\}\{2\\sigma^\{2\}\}\\right\),with fixed varianceσ2\\sigma^\{2\}\. Thus, its negative log\-likelihood is equivalent, up to parameter\-independent terms, to squared error on scaled raw inter\-arrival times\.

Generation\-quality evaluation\.We evaluate generated music against held\-out MIDI sequences with a window\-based overlapping\-area \(OA\) protocol inspired by the evaluation used by\(Huanget al\.,[2019](https://arxiv.org/html/2608.10120#bib.bib27)\)\. Each sequence is segmented into fixed 2\-second windows, and for every window we compute four simple symbolic music statistics: note density, pitch range, mean pitch, and pitch variance\. For each feature, we compare the empirical distribution over generated windows with the corresponding held\-out distribution through their overlapping area, where higher values indicate closer agreement\. We report both the feature\-wise OA scores and their mean\.

Table 4\.Auxiliary GiantMIDI results\. OA denotes the mean and feature\-wise overlapping\-area scores computed from 2\-second window statistics\. Boldface identifies the better result within the controlledJoint–Two\-Stagecomparison\.Response\.[Table4](https://arxiv.org/html/2608.10120#A2.T4)provides the same controlled comparison for the auxiliary musical setting\. From the perspective ofRQ1, temporal recoverability remains weak for both ChronoSSM variants: TCS is identical \(−0\.044\-0\.044for both\), but the probe scores still slightly favorJoint\(R2=0\.160R^\{2\}=0\.160vs\.0\.1420\.142, MAE248\.006248\.006vs\.261\.097261\.097\)\. Thus, even in this dense symbolic\-music setting, joint training yields a modest recoverability advantage under the probe\-based diagnostics\.

From the perspective ofRQ2, the controlled comparison shows no degradation of free\-generation quality underJointin this auxiliary setting\. At the aggregate level, mean OA increases from0\.0160\.016underTwo\-Stagetraining to0\.1720\.172underJointtraining\. The same pattern holds for all four reported symbolic\-music features\. In particular,Jointimproves mean\-pitch OA from0\.0000\.000to0\.3450\.345, note\-density OA from0\.0630\.063to0\.2300\.230, pitch\-range OA from0\.0000\.000to0\.0500\.050, and pitch\-variance OA from0\.0000\.000to0\.0630\.063\. This mirrors the pattern observed for network traffic and temporal knowledge graphs in[section6](https://arxiv.org/html/2608.10120#S6), whereJointtraining is also associated with better evaluated content\-generation metrics relative toTwo\-Stage\.

## Appendix CAlternative Autoregressive Backbone

As described in[section2\.2](https://arxiv.org/html/2608.10120#S2.SS2), ChronoSSM is not tied to a State Space Model: its shared backbone can be any causal autoregressive architecture that produces token\-wise hidden states\. This appendix therefore illustrates the framework with a GPT\-2 backbone\. We repeat the controlledJoint–Two\-Stagecomparison while keeping the datasets and evaluation protocols unchanged\.[Table5](https://arxiv.org/html/2608.10120#A3.T5)summarizes the results, so we keep the discussion brief here\.

DatasetMetricJointTwo\-StageGenerationBPI2018Mean DLS↑\\uparrow0\.4869510\.487380Med\. DLS↑\\uparrow0\.5000\.500MIMIC\-IVJSD↓\\downarrow0\.2960\.286R@50↑\\uparrow0\.8200\.780NetworkJSD\(srcIP\)↓\\downarrow0\.58820\.3902JSD\(dstIP\)↓\\downarrow0\.69310\.2342JSD\(sPort\)↓\\downarrow0\.69310\.2408JSD\(dPort\)↓\\downarrow0\.69310\.4259TKGMRR↑\\uparrow0\.13100\.1329MR↓\\downarrow510\.87534\.30Hits@1↑\\uparrow0\.07110\.0723Hits@3↑\\uparrow0\.13850\.1394Hits@10↑\\uparrow0\.24390\.2492RecoverabilityBPI2018TCS↑\\uparrow\-0\.1630\.282ProbeR2R^\{2\}↑\\uparrow0\.6610\.603Probe MAE↓\\downarrow2\.6302\.806MIMIC\-IVTCS↑\\uparrow0\.8760\.857ProbeR2R^\{2\}↑\\uparrow0\.6630\.607Probe MAE↓\\downarrow0\.5950\.627NetworkTCS↑\\uparrow0\.64650\.6951ProbeR2R^\{2\}↑\\uparrow0\.42970\.3665Probe MAE↓\\downarrow0\.09670\.1188TKGTCS↑\\uparrow0\.52420\.5156ProbeR2R^\{2\}↑\\uparrow0\.60570\.6129Probe MAE↓\\downarrow839\.75833\.01Table 5\.GPT\-2 backbone instantiation under the controlledJoint–Two\-Stagecomparison\. Boldface identifies the better result for each metric\.Response\.The GPT\-2 results are more heterogeneous than the Mamba2 results\. On MIMIC\-IV and network traffic, the recoverability picture is more favorable toJointthan the generation picture\. On MIMIC\-IV,Jointclearly improves the timing\-recovery metrics, but the generation comparison is mixed:Two\-Stageachieves slightly lower JSD, whereasJointimproves recall over the 50 most frequent event types\. On network traffic, the tension is sharper:Jointimproves the linear\-probe recoverability metrics, butTwo\-Stageis clearly better on all packet\-header distribution\-matching metrics, and even TCS favorsTwo\-Stage\. On BPI2018 and the temporal knowledge graph, GPT\-2 does not reproduce the Mamba2 recoverability pattern\. On BPI2018, generation quality is effectively unchanged under DLS\. TCS favorsTwo\-Stage, whereas both probe diagnostics favorJoint\. On the temporal knowledge graph, the two regimes are effectively tied overall, withJointimproving TCS and mean rank butTwo\-Stageremaining slightly better on the probe metrics and the other link\-prediction scores\.

Overall, the GPT\-2 appendix does not support a uniform story\. In some domains,Jointyields more recoverable temporal information only with mixed or weaker generation results; in others, it does not even provide a clear recoverability advantage\. The favorable qualitative picture in the main paper is therefore not reproduced consistently with a GPT\-2 backbone\. One plausible explanation is architectural: Mamba2’s selective recurrent state dynamics may provide a more suitable inductive bias for representing temporal transitions in long event sequences, whereas GPT\-2 distributes sequence history through attention\-based representations\. This interpretation remains tentative\. The present comparison does not isolate architectural effects from differences in model capacity, optimization, or backbone\-specific hyperparameter choices, so it does not establish that one architecture is intrinsically simpler or better suited to temporal supervision than the other\.

## Appendix DHeld\-Out Validation\-Loss Diagnostics

Table[6](https://arxiv.org/html/2608.10120#A4.T6)reports validation losses on the predefined validation partitions used in the four main experiments\. We use the dataset\-provided training and validation splits; validation samples are excluded from gradient\-based training\. Both losses are evaluated under observed validation histories and therefore characterize conditional prediction rather than free\-running generation\. Consequently, the reported losses neither compare generated sequences or timestamps with their real\-data distributions nor capture errors that accumulate when the model conditions on its own generated history\. They provide complementary optimization diagnostics, but do not by themselves establish high\-quality free\-running content generation or distributionally faithful timestamp generation\.

Table 6\.Held\-out validation losses for the four main experiments\. Boldface identifies the lower result within each dataset and loss\.For token prediction,Jointobtains lower validationLtokL\_\{\\mathrm\{tok\}\}on BPI2018, network traffic, and temporal knowledge graphs, whereasTwo\-Stageis marginally lower on MIMIC\-IV\. This pattern provides complementary context for the content\-generation results in[section6](https://arxiv.org/html/2608.10120#S6), but does not replace the free\-generation evaluation\.

For temporal point prediction,Jointobtains lowerLtimeL\_\{\\mathrm\{time\}\}on BPI2018, MIMIC\-IV, and network traffic\. On the temporal knowledge graph, however,Two\-Stageachieves lower direct temporal\-head loss\. This result should be interpreted narrowly:LtimeL\_\{\\mathrm\{time\}\}measures the conditional point\-prediction error of the particular trained temporal head, rather than a general measure of temporal generation quality\.

## Appendix EEffect of Auxiliary\-Loss Weight

This section studies the effect of the auxiliary temporal\-loss weightλτ\\lambda\_\{\\mathrm\{\\tau\}\}on the two partially timestamped domains: network traffic and temporal knowledge graphs\. The goal is to assess how increasing the strength of temporal supervision changes temporal recoverability and downstream generation quality\.

Table 7\.Effect of the auxiliary temporal\-loss weightλτ\\lambda\_\{\\mathrm\{\\tau\}\}on temporal recoverability and link prediction for the temporal knowledge graph\. Hits@3 is omited for space reasons\.Table 8\.Effect of the auxiliary temporal\-loss weightλτ\\lambda\_\{\\mathrm\{\\tau\}\}on recoverability and packet\-header distribution matching for network traffic\. JSD\(src\), JSD\(dst\), JSD\(sport\), and JSD\(dport\) denote the source/destination IP and source/destination TCP\-port marginals, respectively\.The two domains exhibit different sensitivity profiles\. On the temporal knowledge graph, increasingλτ\\lambda\_\{\\mathrm\{\\tau\}\}produces a fairly consistent improvement in both recoverability and downstream link prediction\.[Table7](https://arxiv.org/html/2608.10120#A5.T7)shows both trends\. The strongest setting,λτ=1\.00\\lambda\_\{\\mathrm\{\\tau\}\}=1\.00, yields the best TCS,R2R^\{2\}, MAE, mean rank, and Hits@10, whileλτ=0\.75\\lambda\_\{\\mathrm\{\\tau\}\}=0\.75slightly improves MRR and H@1\.

On network traffic, the dependence is less monotone, and no single value ofλτ\\lambda\_\{\\mathrm\{\\tau\}\}optimizes all recoverability diagnostics\. The highest TCS is obtained atλτ=500\\lambda\_\{\\mathrm\{\\tau\}\}=500, whereas the highest probe \(R2R^\{2\}\) occurs atλτ=10\\lambda\_\{\\mathrm\{\\tau\}\}=10and the lowest probe MAE atλτ=100\\lambda\_\{\\mathrm\{\\tau\}\}=100\.[Table8](https://arxiv.org/html/2608.10120#A5.T8)further shows thatλτ=100\\lambda\_\{\\mathrm\{\\tau\}\}=100yields the best packet\-header distribution\-matching metrics\. These results therefore do not reveal a simple trade\-off between temporal recoverability and generation quality; rather, the different recoverability diagnostics respond differently to the strength of temporal supervision, while an intermediate weight provides the best agreement with the evaluated packet\-header marginals\.

Rule of Thumb for Selecting the Auxiliary\-Loss Weight\.These results indicate that there is no universally optimal value ofλτ\\lambda\_\{\\mathrm\{\\tau\}\}\. For the main experiments, we chose the auxiliary\-loss weight so that the weighted temporal term,λτ​Ltime\\lambda\_\{\\mathrm\{\\tau\}\}L\_\{\\mathrm\{time\}\}, was approximately of the same order of magnitude as the token\-prediction lossLtokL\_\{\\mathrm\{tok\}\}during training\. This scale\-matching criterion prevents either term from numerically dominating the joint objective in[Eq\.14](https://arxiv.org/html/2608.10120#S3.E14)\.

## Appendix FAlternative Temporal Representations

The default temporal representationrkr\_\{k\}introduced in[section3\.1](https://arxiv.org/html/2608.10120#S3.SS1)\([Eq\.8](https://arxiv.org/html/2608.10120#S3.E8)\) is defined as the difference between the hidden states associated with the two endpoints of a temporally supervised transition\. We compare it with the alternative representationrk′r^\{\\prime\}\_\{k\}, formed by concatenating the same endpoint states:

\(24\)rk=Δ​hk=hk−hk−1,rk′=\[hk−1;hk\]\.r\_\{k\}=\\Delta h\_\{k\}=h\_\{k\}\-h\_\{k\-1\},\\qquad r^\{\\prime\}\_\{k\}=\[h\_\{k\-1\};h\_\{k\}\]\.The representationrkr\_\{k\}has dimensiondd, whereasrk′r^\{\\prime\}\_\{k\}has dimension2​d2dand therefore changes the input projection of the temporal head\. We evaluate both choices on one densely timestamped domain \(BPI2018\) and one partially timestamped domain \(network traffic\), retaining the same backbone, data splits, temporal\-head family, and optimization schedule within each representation\.

On network traffic, usingrk′r^\{\\prime\}\_\{k\}empirically changes the magnitude and optimization behavior of the temporal objective\. Stable training therefore requiresλτ=50\\lambda\_\{\\tau\}=50, rather than the valueλτ=300\\lambda\_\{\\tau\}=300used withrkr\_\{k\}\. Consequently, the network experiment should be interpreted as a comparison between two practical temporal\-input configurations rather than as a strictly representation\-only ablation\. The results obtained withrk′r^\{\\prime\}\_\{k\}use transition\-level recoverability diagnostics\.

DatasetMethodTemporal recoverabilityBPI2018 generationNetwork\-traffic generationTCS↑\\uparrowProbeR2R^\{2\}↑\\uparrowProbe MAE↓\\downarrowDLS mean↑\\uparrowDLS median↑\\uparrowJSD ip\.src↓\\downarrowJSD ip\.dst↓\\downarrowJSD tcp\.src↓\\downarrowJSD tcp\.dst↓\\downarrowBPI2018Joint0\.7924 \(0\.8030\)0\.8753 \(0\.631\)4,919,732 \(2\.775\)0\.487\(0\.487\)0\.500\(0\.500\)––––Two\-Stage0\.8029\(0\.7940\)0\.8789\(0\.461\)4,861,086\(3\.342\)0\.487\(0\.487\)0\.500\(0\.500\)––––Network trafficJoint0\.2987\(0\.6216\)0\.0313\(0\.3943\)0\.00610\(0\.3080\)––0\.6931\(0\.0412\)0\.6931\(0\.0311\)0\.3477 \(0\.0764\)0\.4344 \(0\.1120\)Two\-Stage0\.2538 \(0\.1468\)0\.0116 \(0\.1902\)0\.00654 \(0\.3824\)––0\.6931\(0\.0669\)0\.6931\(0\.0516\)0\.3431\(0\.1248\)0\.3926\(0\.2183\)

Table 9\.Results with concatenationrk′=\[hk−1;hk\]r^\{\\prime\}\_\{k\}=\[h\_\{k\-1\};h\_\{k\}\]as the temporal representation\. Parenthesized italic values report the results from the main paper for reference\. Boldface identifies the better concatenation result within eachJoint–Two\-Stagecomparison; “–” marks metrics not applicable to that dataset\.[Table9](https://arxiv.org/html/2608.10120#A6.T9)shows that the configuration based onrk′r^\{\\prime\}\_\{k\}does not reproduce the empirical pattern obtained with the default representationrkr\_\{k\}\.

From the perspective ofRQ1\([section5](https://arxiv.org/html/2608.10120#S5)\), BPI2018 remains effectively a tie\. Withrk′r^\{\\prime\}\_\{k\},Two\-Stageis numerically better on all three recoverability diagnostics, but the differences are small\. Usingrk′r^\{\\prime\}\_\{k\}therefore removes, rather than meaningfully reverses, the clearerJointadvantage obtained withrkr\_\{k\}\. Generation quality is unchanged, with identical mean and median DLS under both regimes\.

The network\-traffic results show a clearer departure from the default configuration\. Withrk′r^\{\\prime\}\_\{k\},Jointstill outperformsTwo\-Stageon the recoverability diagnostics, but the separation is substantially smaller, especially for TCS and probeR2R^\{2\}\. The representationrk′r^\{\\prime\}\_\{k\}therefore fails to reproduce the strong recoverability advantage observed withrkr\_\{k\}\.

From the perspective ofRQ2\([section6](https://arxiv.org/html/2608.10120#S6)\), both variants usingrk′r^\{\\prime\}\_\{k\}also match the packet\-header distributions poorly\. Source\- and destination\-IP JSD reach0\.69310\.6931under both regimes, while the TCP\-port divergences are substantially higher than withrkr\_\{k\}\. Because this degradation occurs for bothJointandTwo\-Stage, it does not indicate a Joint\-specific trade\-off, but rather that the tested configuration based onrk′r^\{\\prime\}\_\{k\}is poorly suited to network traffic\.

Overall, the main empirical pattern is sensitive to the representation supplied to the temporal head\. On BPI2018, the representationrk′r^\{\\prime\}\_\{k\}does not reproduce the clearer recoverability advantage obtained withrkr\_\{k\}, while generation quality remains near\-equivalent\. It also does not reproduce the stronger recoverability or generation quality obtained withrkr\_\{k\}on network traffic\. Because the network run withrk′r^\{\\prime\}\_\{k\}uses a different temporal\-loss weight, the experiment supports the practical choice ofrkr\_\{k\}without isolating the representation as the sole causal factor\.

## Appendix GComputational Overhead

We report the computational cost of theJointandTwo\-Stagetraining procedures defined in[section3\.3](https://arxiv.org/html/2608.10120#S3.SS3)\([Algorithms1](https://arxiv.org/html/2608.10120#alg1)and[2](https://arxiv.org/html/2608.10120#alg2)\)\. All experiments were performed on NVIDIA RTX PRO 6000 Blackwell Max\-Q GPUs \(96 GB VRAM\)\. The network traffic experiments used two GPUs with a batch size of 32 per GPU \(64 total\), while the remaining datasets were trained on a single GPU\.

Our experiments consider two training strategies\. In the proposedJointtraining approach, for a single training run of 40 epochs\. InTwo\-Stagetraining first trains the language model for 40 epochs and then performs an additional 40 epochs of training using only the auxiliary objective, resulting in a total of 80 training epochs\.

[Table10](https://arxiv.org/html/2608.10120#A7.T10)summarizes the training time\. For two\-stage training, the reported time is separated into the language\-model \(LM\) stage and the auxiliary \(Aux\) stage\.

Table 10\.Total training time and peak GPU memory usage\.Jointtraining optimizes both objectives simultaneously for 40 epochs, whereasTwo\-Stagetraining consists of 40 epochs of language\-model training followed by 40 epochs of auxiliary training\.The proposed method introduces only a lightweight auxiliary prediction head on top of the language\-model backbone\. Consequently, the backbone architecture, hidden\-state dimension, and sequence length remain unchanged, and the additional computation arises only from evaluating the auxiliary loss during training\. Although each epoch of joint training is slightly more expensive than language\-model training alone, it eliminates the need for a separate auxiliary optimization stage\. As shown in[Table10](https://arxiv.org/html/2608.10120#A7.T10), this results in comparable or lower end\-to\-end training time across the evaluated datasets while maintaining a modest memory footprint\.

## Appendix HModel Configurations

[Table11](https://arxiv.org/html/2608.10120#A8.T11)records the concrete Mamba2 configurations used in our experiments\. Following the training schedules defined in[section3\.3](https://arxiv.org/html/2608.10120#S3.SS3)\([Algorithms1](https://arxiv.org/html/2608.10120#alg1)and[2](https://arxiv.org/html/2608.10120#alg2)\), for optimization, allJointmodels are trained for 40 epochs\. AllTwo\-Stagemodels use 40 epochs of token\-only training followed by 40 epochs of temporal\-head training with the backbone frozen\. For the partially timestamped domains[section4\.3](https://arxiv.org/html/2608.10120#S4.SS3), the temporal head additionally uses four temporal lags\.

Training Hyperparameters and Setup\.All models are optimized using AdamW\(Loshchilov and Hutter,[2019](https://arxiv.org/html/2608.10120#bib.bib52)\)with weight decay0\.10\.1\. We use a base peak learning rate ofη=2×10−4\\eta=2\\times 10^\{\-4\}\(scaled to4×10−44\\times 10^\{\-4\}for multi\-GPU DDP runs on Network traffic\), scheduled with a 1\-epoch linear warmup followed by cosine annealing decay down to a floor ratio of0\.10\.1\. Next\-token modeling uses Cross\-Entropy loss with label smoothing \(ϵsmooth=0\.1\\epsilon\_\{\\mathrm\{smooth\}\}=0\.1\), while temporal regression uses Mean Squared Error over log\-transformed targetslog⁡\(1\+Δ​t/s\)\\log\(1\+\\Delta t/s\)\. Models are trained with a batch size of6464for BPI2018, MIMIC\-IV, Network traffic \(32 per GPU\), and GiantMIDI \(sequence lengths of40964096tokens with50%50\\%window stride, and256256/128128for BPI2018\), and a batch size of3232for GDELT \(sequence length50005000, stride10241024\)\.

Table 11\.Domain\-specific Mamba2 configurations\. The columnssdenotes the time scale used in the transformed temporal target, andλτ\\lambda\_\{\\tau\}denotes the temporal\-loss weight in[Eq\.14](https://arxiv.org/html/2608.10120#S3.E14)\.
## Appendix IDomain\-Specific Baselines

This appendix provides the full baseline setup, results, and interpretation summarized in[section4\.5](https://arxiv.org/html/2608.10120#S4.SS5), kept separate from the controlledJoint–Two\-Stagecomparison in the main text\. Here, a*baseline*denotes a previously proposed domain\-specific model that serves as an external reference point for the task\. Studying such baselines is useful because it helps contextualize the absolute scale of the reported metrics, shows how our models compare with domain\-adapted alternatives from prior work, and clarifies which observations appear specific to ChronoSSM versus common across different modeling choices\. In all cases, the baselines are retrained on the same train/validation/test splits as ChronoSSM and evaluated with the same domain\-specific metrics\. However, they remain descriptive reference points only\. Because they differ from ChronoSSM in architecture, objective, temporal representation, and generation procedure, they do not isolate the effect of the training regime and therefore cannot be used to answerRQ1\([section5](https://arxiv.org/html/2608.10120#S5)\) orRQ2\([section6](https://arxiv.org/html/2608.10120#S6)\)\. The evidence for those two questions comes only from the controlledJoint–Two\-Stagecomparison\.

### I\.1\.Baseline Presentation

[Table12](https://arxiv.org/html/2608.10120#A9.T12)summarizes the baseline family used for each domain, including the event\-generation procedure and the temporal mechanism\.

Table 12\.Reference baselines used for descriptive comparison across the four main domains and the auxiliary GiantMIDI experiment\.BPI2018\.For business\-process traces, we use an LSTM baseline inspired by the predictive\-process\-monitoring model of Tax et al\.\(Taxet al\.,[2017](https://arxiv.org/html/2608.10120#bib.bib46)\), which jointly predicts the next activity and its time gap from a recurrent state\. In our experiments, we use a modernized implementation rather than the original legacy training script\. The model is trained for 50 epochs with batch size 256\.

MIMIC\-IV\.For MIMIC\-IV, we use the Transformer Hawkes Process \(THP\)\(Zuoet al\.,[2020](https://arxiv.org/html/2608.10120#bib.bib47)\), a sequence model designed for event streams with explicit temporal dynamics\. We train for 20 epochs with batch size 64, model dimension 256, 4 attention heads, and 4 transformer layers\. Relative times are scaled in hours during training\.

Network traffic\.For network traffic, we use NetSSM\(Chuet al\.,[2026](https://arxiv.org/html/2608.10120#bib.bib18)\)as the content generator and then assign timing with a separately fitted Gaussian mixture model, reflecting a decoupled content/time pipeline\. NetSSM is trained on the tokenized packet\-sequence representation of the dataset for 10 epochs with batch size 8\. Packet timestamps are then assigned using a 3\-component GMM fit on training\-split inter\-arrival times\.

GDELT\.For temporal knowledge graphs, we use RE\-Net\(Jinet al\.,[2020](https://arxiv.org/html/2608.10120#bib.bib31)\), again combined with a separate Gaussian mixture model for fact\-level timing, similar to what we did with NetSSM\. We follow the standard RE\-Net training schedule, with a pretraining phase followed by full training; both phases use hidden size 200, dropout 0\.5, learning rate10−310^\{\-3\}, batch size 1024, and 20 epochs\.

GiantMIDI\.For the auxiliary GiantMIDI experiment, we use Music Transformer\(Huanget al\.,[2019](https://arxiv.org/html/2608.10120#bib.bib27)\), a strong symbolic\-music baseline in which timing is represented directly through discretetime\_shifttokens\. It is trained on PerformanceRNN\-style event tokens obtained from the MAESTRO\-style export of GiantMIDI\(Konget al\.,[2022](https://arxiv.org/html/2608.10120#bib.bib25); Hawthorneet al\.,[2019](https://arxiv.org/html/2608.10120#bib.bib26)\)\. We use discrete note andtime\_shiftevents, fix note duration to 0\.10 seconds and velocity to 64 during export for 100 epochs with batch size 2\.

TimestampingDatasetBaselineContent\-generation qualityDenseBPI2018DLS mean↑\\uparrowDLS median↑\\uparrowLSTM0\.535\(0\.487\)0\.533\(0\.500\)MIMIC\-IVToken JSD↓\\downarrowRecall@50↑\\uparrowTHP0\.111\(0\.302\)1\.000\(0\.780\)GiantMIDIOA mean↑\\uparrowMean pitch OA↑\\uparrowNote density OA↑\\uparrowPitch range OA↑\\uparrowPitch var\. OA↑\\uparrowMusic Transformer0\.577\(0\.172\)0\.458\(0\.345\)0\.270\(0\.230\)0\.852\(0\.050\)0\.730\(0\.063\)PartialNetwork trafficJSD ip\.src↓\\downarrowJSD ip\.dst↓\\downarrowJSD tcp\.src↓\\downarrowJSD tcp\.dst↓\\downarrowNetSSM0\.3076 \(0\.0412\)0\.2426 \(0\.0311\)0\.2923 \(0\.0764\)0\.2923 \(0\.1120\)Temporal KGMRR↑\\uparrowMR↓\\downarrowHits@1↑\\uparrowHits@3↑\\uparrowHits@10↑\\uparrowRE\-Net0\.4136\(0\.1461\)144\.16\(269\.06\)0\.3446\(0\.0899\)0\.4398\(0\.1490\)0\.5405\(0\.2527\)Table 13\.Content\-generation results for the reference baselines on the four main evaluation domains and the auxiliary GiantMIDI experiment\. Parenthesized values report the best ChronoSSM result acrossJointandTwo\-Stage; boldface marks the better of the baseline and ChronoSSM value\.
### I\.2\.Baseline Recoverability Protocol

The recoverability analysis requires a small adaptation for the external baselines, because they do not all expose the same temporal representation as ChronoSSM\. The controlledJoint–Two\-Stagecomparison always probes the representation supplied to the temporal head\. For the baselines, we therefore probe the frozen representation most directly tied to temporal prediction in each model, so that the diagnostic remains as comparable as possible across methods\. This is the recurrent state for the LSTM, the event\-history representation for THP, the packet\-boundary representation for NetSSM, and the fact\-history representation for RE\-Net\. Within each dataset, the baseline probe uses the same targetzkz\_\{k\}and the same evaluation protocol as the controlledJoint–Two\-Stagecomparison\. The resulting numbers remain descriptive only, since the compared models still differ in architecture and temporal parameterization\.[Table14](https://arxiv.org/html/2608.10120#A9.T14)reports the resulting descriptive recoverability scores\.

Table 14\.Temporal\-recoverability results for the reference baselines on the four main evaluation domains and the auxiliary GiantMIDI experiment\. Parenthesized values report the best ChronoSSM result acrossJointandTwo\-Stage; boldface marks the better of the baseline and best ChronoSSM value\.
### I\.3\.Baseline Results

The baseline comparisons are most informative when they reveal a separation between generation quality and temporal recoverability\. On several datasets, the domain\-specific baselines are clearly stronger than ChronoSSM on the generation task, which is expected because they were designed specifically for that domain\.[Table13](https://arxiv.org/html/2608.10120#A9.T13)shows that, on BPI2018, the LSTM yields higher suffix similarity than both controlled variants\. On MIMIC\-IV, THP also dominates the generation metrics\. The largest gap appears on the temporal knowledge graph, where RE\-Net substantially outperforms ChronoSSM on filtered link prediction\. The auxiliary GiantMIDI baseline shows the same pattern, with Music Transformer far ahead on OA generation quality\. These gaps are not a problem for the main claims of the paper; they are expected, because these baselines are purpose\-built generators for their respective domains\.

By contrast, the recoverability picture is consistently weak or at least much less convincing for the baselines\. The LSTM baseline on BPI2018 has almost no recoverability under the probe diagnostics\. NetSSM is similarly weak on network traffic, including a negative probeR2R^\{2\}\. RE\-Net is another important case: despite much better generation quality, its probe recoverability remains poor relative to both ChronoSSM variants\. Music Transformer also fits this pattern, with very poor recoverability despite being the strongest generator in that setting\.

The most revealing case is THP on MIMIC\-IV\. Its TCS is close to that ofJoint, yet its probeR2R^\{2\}is much lower\. This suggests that a seemingly favorable TCS value does not by itself imply that temporal information is cleanly or linearly accessible from the representation\. More broadly, these baselines do not learn timing through the same kind of shared representation studied in ChronoSSM: some use different temporal parameterizations, and others decouple timing from content generation entirely\. Their recoverability is therefore generally weak, and when a baseline comes close toJointon a single metric, we interpret that locally as a reminder that no single recoverability metric should be read in isolation\.

## Appendix JSensitivity to Distance Metric and Neighborhood Size

The Temporal Cohesion Score \(TCS\) depends on two design choices: \(i\) the distance metric used to compare hidden states, and \(ii\) the neighborhood sizeKKused to define local neighborhoods\.

In the main experiments \([section5](https://arxiv.org/html/2608.10120#S5)\), we fixK=5K=5and use cosine similarity\. In this section, we analyze the robustness of TCS to these choices and clarify how the resulting sensitivity differs between the network\-traffic and temporal\-knowledge\-graph domains\.

### J\.1\.Sensitivity to Distance Metric

We first examine the effect of the distance metric while keeping the neighborhood size fixed atK=5K=5\. Specifically, we recompute TCS using Euclidean \(L2L\_\{2\}\) and Manhattan \(L1L\_\{1\}\) distances in place of cosine similarity\.

[Table15](https://arxiv.org/html/2608.10120#A10.T15)reports the resulting scores for both domains\. On network traffic, theJointmodel consistently achieves a TCS of approximately0\.620\.62, while the Two\-Stage baseline remains around0\.150\.15\. The relative improvement induced by joint training is therefore large and stable across all three metrics\. This robustness indicates that the strong temporal signal observed in network traffic is not an artifact of a particular geometric choice\.

On the temporal knowledge graph, the same qualitative pattern holds, but the gains are much smaller: the Two\-Stage baseline ranges from0\.1040\.104to0\.1120\.112, whereas theJointmodel ranges from0\.1260\.126to0\.1440\.144\. Temporal cohesion is therefore somewhat sensitive to the distance metric in this domain, but the advantage ofJointtraining remains positive under all three choices\.

Table 15\.TCS across distance metrics for network traffic and temporal knowledge graphs\.
### J\.2\.Sensitivity to Neighborhood SizeKK

We next vary neighborhood sizeKKon the two partially timestamped domains, again using cosine, Euclidean, and Manhattan distances\.[Table16](https://arxiv.org/html/2608.10120#A10.T16)shows a clear contrast between them\. For network traffic, theJointmodel remains far above the Two\-Stage baseline for all tested values ofKKand all three distance metrics, even though TCS gradually decreases as neighborhoods grow\. This indicates that the temporal organization induced byJointtraining is not limited to a narrow nearest\-neighbor effect\.

For temporal knowledge graphs, the picture is different\. The advantage ofJointtraining is visible at small neighborhoods, but it rapidly shrinks toward zero asKKincreases, regardless of the distance metric\. Here, temporal information is therefore much more localized: once the neighborhood expands, static relational structure dominates similarity and temporal ordering becomes difficult to recover\. Together with the distance\-metric analysis above, this comparison shows that the main conclusion is robust to the exact geometric choice, while the spatial extent of temporal organization depends strongly on the domain\.

Table 16\.Sensitivity of TCS to neighborhood sizeKKfor network traffic and temporal knowledge graphs\.

## Appendix KLayer\-Wise Representation Analysis

This appendix examines how temporal supervision changes the internal representations learned by ChronoSSM\. It is intended to complement the recoverability results of[section5](https://arxiv.org/html/2608.10120#S5)and the generation\-quality results of[section6](https://arxiv.org/html/2608.10120#S6), rather than to provide a complete mechanistic account of all four evaluation domains\.

### K\.1\.Representation\-Similarity Metrics

This subsection provides the formal definitions of the representation\-similarity metrics used in the layer\-wise analysis below\. These metrics quantify alignment between hidden representations learned underJointandTwo\-Stagetraining at different geometric scales\.

To quantify structural differences between the two training regimes, we compare hidden states produced by corresponding backbone layers on identical input sequences\. Let𝐗∈ℝn×d\\mathbf\{X\}\\in\\mathbb\{R\}^\{n\\times d\}and𝐘∈ℝn×d\\mathbf\{Y\}\\in\\mathbb\{R\}^\{n\\times d\}denote the activations from theJointandTwo\-Stagemodels, respectively, fornnsamples and representation dimensiondd\. We measure alignment at three complementary geometric scales\.

Global structural alignment \(CKA\)\.We use linear Centered Kernel Alignment \(CKA\)\(Kornblithet al\.,[2019](https://arxiv.org/html/2608.10120#bib.bib12)\)to measure similarity between the global “clouds” formed by the representations\. Given Gram matricesK=𝐗𝐗⊤K=\\mathbf\{X\}\\mathbf\{X\}^\{\\top\}andL=𝐘𝐘⊤L=\\mathbf\{Y\}\\mathbf\{Y\}^\{\\top\}, and the centering matrixH=I−1n​𝟏𝟏⊤H=I\-\\frac\{1\}\{n\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}, linear CKA is defined as

\(25\)CKA​\(K,L\)=HSIC​\(K,L\)HSIC​\(K,K\)​HSIC​\(L,L\),\\text\{CKA\}\(K,L\)=\\frac\{\\text\{HSIC\}\(K,L\)\}\{\\sqrt\{\\text\{HSIC\}\(K,K\)\\,\\text\{HSIC\}\(L,L\)\}\},where

\(26\)HSIC​\(K,L\)=1\(n−1\)2​tr​\(K​H​L​H\)\\text\{HSIC\}\(K,L\)=\\frac\{1\}\{\(n\-1\)^\{2\}\}\\,\\mathrm\{tr\}\(KHLH\)is the Hilbert–Schmidt Independence Criterion\(Grettonet al\.,[2007](https://arxiv.org/html/2608.10120#bib.bib13)\)\. High CKA values indicate strong global alignment between representation spaces\.

Local neighborhood alignment \(CKNNA\)\.To assess alignment at a local geometric scale, we use Centered Kernel Nearest Neighbor Alignment \(CKNNA\)\(Huhet al\.,[2024](https://arxiv.org/html/2608.10120#bib.bib14)\)\. Let𝐀\\mathbf\{A\}be a binary adjacency matrix such that𝐀i​j=1\\mathbf\{A\}\_\{ij\}=1if samplejjis a mutualkk\-nearest neighbor of sampleiiin both representation spaces\. CKNNA is defined as

\(27\)CKNNA​\(K,L\)=tr​\(\(𝐀∘K\)​\(𝐀∘L\)⊤\)‖𝐀∘K‖F⋅‖𝐀∘L‖F,\\text\{CKNNA\}\(K,L\)=\\frac\{\\mathrm\{tr\}\\\!\\left\(\(\\mathbf\{A\}\\circ K\)\(\\mathbf\{A\}\\circ L\)^\{\\top\}\\right\)\}\{\\\|\\mathbf\{A\}\\circ K\\\|\_\{F\}\\cdot\\\|\\mathbf\{A\}\\circ L\\\|\_\{F\}\},where∘\\circdenotes the Hadamard product and∥⋅∥F\\\|\\cdot\\\|\_\{F\}is the Frobenius norm\. Lower values indicate greater divergence in local neighborhood structure between the two representations\.

Topological overlap \(M\-KNN\)\.Finally, we compute the Mutualkk\-Nearest Neighbor \(M\-KNN\) overlap to measure agreement in neighborhood membership\. Let𝒩k𝐗​\(i\)\\mathcal\{N\}\_\{k\}^\{\\mathbf\{X\}\}\(i\)denote the set ofkknearest neighbors of sampleiiin representation space𝐗\\mathbf\{X\}\. The M\-KNN overlap is defined as

\(28\)M\-KNN​\(𝐗,𝐘\)=1n​k​∑i=1n\|𝒩k𝐗​\(i\)∩𝒩k𝐘​\(i\)\|\.\\text\{M\-KNN\}\(\\mathbf\{X\},\\mathbf\{Y\}\)=\\frac\{1\}\{nk\}\\sum\_\{i=1\}^\{n\}\\left\|\\mathcal\{N\}\_\{k\}^\{\\mathbf\{X\}\}\(i\)\\cap\\mathcal\{N\}\_\{k\}^\{\\mathbf\{Y\}\}\(i\)\\right\|\.Lower overlap indicates that the two models induce different local topological structures over the data\.

![Refer to caption](https://arxiv.org/html/2608.10120v1/x3.png)\(a\)BPI2018\.
![Refer to caption](https://arxiv.org/html/2608.10120v1/x4.png)\(b\)MIMIC\-IV\.
![Refer to caption](https://arxiv.org/html/2608.10120v1/x5.png)\(c\)Network traffic\.
![Refer to caption](https://arxiv.org/html/2608.10120v1/x6.png)\(d\)Temporal knowledge graph\.

Figure 2\.Layer\-wise alignment between ChronoSSM models trained underJointandTwo\-Stageregimes, measured using CKA, M\-KNN, and CKNNA\. CKA captures global geometric alignment, whereas M\-KNN and CKNNA emphasize local neighborhood structure\.
### K\.2\.Per\-Domain Patterns

[Figure2](https://arxiv.org/html/2608.10120#A11.F2)collects the layer\-wise alignment plots for all four domains\.

BPI2018\.[Figure2\(a\)](https://arxiv.org/html/2608.10120#A11.F2.sf1)shows the mildest change of the four domains\. CKA remains high throughout the backbone and ends at0\.9170\.917, while M\-KNN and CKNNA decline but stay well above the network\-traffic values\. The substantial probe\-based recoverability gain despite this relatively mild geometric change is consistent with the possibility thatJointmakes temporal information more linearly accessible without a major reshaping of the representation space\. The alignment measures alone do not establish that mechanism\.

MIMIC\-IV\.[Figure2\(b\)](https://arxiv.org/html/2608.10120#A11.F2.sf2)shows a different pattern: CKA stays close to11across all layers and ends at0\.9690\.969, whereas M\-KNN and CKNNA drop to roughly0\.340\.34to0\.360\.36at the final layer\. The natural reading is thatJointchanges local neighborhood structure much more than global geometry\. This local change is compatible with the favorable probe diagnostics on MIMIC\-IV, but the plot does not establish how it produces them or explain why generation quality is slightly mixed in this domain\.

Network traffic\.[Figure2\(c\)](https://arxiv.org/html/2608.10120#A11.F2.sf3)shows the strongest reorganization\. CKA is high at the input, drops sharply through much of the backbone, and recovers only near the end, while M\-KNN and CKNNA remain low across most layers\. This pattern is consistent with the concurrent gains in TCS and probe diagnostics: compared withTwo\-Stage,Jointappears to reshape fine\-grained local geometry much more substantially in this domain than in the others\. It may therefore be compatible with changes in both local temporal coherence and linear accessibility, without establishing a causal explanation for either the recoverability or generation results\.

Temporal knowledge graphs\.[Figure2\(d\)](https://arxiv.org/html/2608.10120#A11.F2.sf4)lies between the previous two cases\. The two models stay well aligned early, diverge most clearly in the middle layers, and recover substantial similarity near the output\. This fits the smaller recoverability differences relative to the other domains, but does not provide a clear account of why those differences are small:Jointchanges the intermediate computation, while the two regimes converge again toward more similar late\-layer geometries\.

### K\.3\.Cross\-Domain Interpretation

The main cross\-domain observation is negative but informative: the four domains do not support a single geometric story for whyJointimproves temporal recoverability\. BPI2018 shows only modest change, MIMIC\-IV preserves almost the same global geometry while altering local neighborhoods, temporal knowledge graphs diverge mainly in the middle layers before recovering, and network traffic undergoes the broadest local and intermediate\-layer reorganization\.

This variation weakens two simple explanations at once\. First, it weakens a single\-mechanism interpretation in which improved recoverability would always arise from the same kind of representational change\. Second, it weakens a simple dense\-versus\-partial timestamping explanation: BPI2018 and MIMIC\-IV are both densely timestamped but behave quite differently, and the two partially timestamped domains are likewise not interchangeable\. Timestamp density alone is therefore too coarse to explain the observed patterns\.

There is no simple relationship between alignment and recoverability gains\. BPI2018 exhibits a substantial probe\-based gain despite relatively high final\-layer alignment, whereas network traffic combines broad local divergence with large gains in both TCS and the probe diagnostics\. These observations are consistent with the domain\-specific interpretations in the recoverability analysis, but the plots do not verify them or identify a common mechanism\.

The safest conclusion is therefore narrow\.Jointcan improve temporal recoverability under several different patterns of representational change, and those patterns do not map cleanly onto downstream generation quality\. The layer\-wise analysis is useful mainly because it rules out overly simple explanations; it does not provide a complete account of why the domain\-level results differ\. This interpretation is robust to the neighborhood size used for M\-KNN and CKNNA: although the absolute values change, the same qualitative patterns remain\.