评估大语言模型在时间信息抽取任务中的多维泛化能力

# Evaluating Multi-Dimensional Generalization of Large Language Models in Temporal Extraction Tasks
Source: [https://arxiv.org/html/2610.02549](https://arxiv.org/html/2610.02549)
###### Abstract

Time and event expression extraction are fundamental temporal reasoning tasks, but the problem remains difficult due to annotation ambiguity, domain sensitivity, and unstable model behavior\. Existing evaluations focus on in\-domain performance, offering limited insight into reliability under distribution shifts\. We evaluate multiple model configurations across families, architectures, and reasoning strategies over four dimensions of generalization, examining transfer from base performance, cross\-dimensional correlations, and the effects of scale, architecture, and prompting\. This provides a systematic study of how prompted LLMs generalize in time and event expression extraction tasks\. We find that strong base\-task performance generally predicts better generalization\. However, this relationship weakens under substantial distribution shifts\. Inductive prompting performs most consistently across domain shift, adversarial perturbations, compositionality, and length increase, while gains from scale, architecture, and deductive and abductive prompting strategies are uneven and dimension\-specific\. We conclude that LLM generalization in temporal extraction tasks cannot be predicted from any single dimension alone and cannot be reliably inferred from in\-domain or single\-dimension evaluations, highlighting the need for reasoning strategies that generalize across dimensions\.

††Code:![[Uncaptioned image]](https://arxiv.org/html/2610.02549v1/Figures/emojifolder/github-1.png)[GitHub](https://github.com/nlpatunt/mdg-nlp.git)Datasets:![[Uncaptioned image]](https://arxiv.org/html/2610.02549v1/Figures/emojifolder/hf.png)[HF](https://huggingface.co/mdg-nlp/datasets)Models:![[Uncaptioned image]](https://arxiv.org/html/2610.02549v1/Figures/emojifolder/hf.png)[HF](https://huggingface.co/mdg-nlp/models)## 1Introduction

Temporal reasoning in NLP aims to infer temporal structure from text, including events, time expressions, and the relations that order them over time\([Pustejovsky et al\., 2003](https://arxiv.org/html/2610.02549#bib.bib2);[Ning et al\., 2018](https://arxiv.org/html/2610.02549#bib.bib17)\)\. Time and event expression extraction is a necessary first step in that process: in the sentence “Monday the company announced a merger,” a system must identify “Monday” and “announced” before determining when the announcement occurred\.

Low inter\-annotator agreement\([Cassidy et al\., 2014](https://arxiv.org/html/2610.02549#bib.bib18);[Styler IV et al\., 2014](https://arxiv.org/html/2610.02549#bib.bib19)\)in such span\-level annotation points to the inherent difficulty of the problem\. LLMs are increasingly applied to this problem, but recent work shows they are often inconsistent and unreliable\([Qiu et al\., 2024](https://arxiv.org/html/2610.02549#bib.bib21);[Kougia et al\., 2024](https://arxiv.org/html/2610.02549#bib.bib20)\)\. Consequently, robust extraction remains a critical bottleneck for reliable temporal reasoning, and in\-domain evaluation alone is insufficient to assess reliability\. Instead, models must be tested for generalizability\.

Prior work argues for testing models in multidimensional generalization settings, covering different types of distributional shift and compositional variation\([Hupkes et al\., 2023](https://arxiv.org/html/2610.02549#bib.bib9)\)\. Such multi\-dimensional generalization is crucial in temporal extraction, where deployed models must handle domain shifts, perturbations, and compositional complexity\([Dutt et al\., 2024](https://arxiv.org/html/2610.02549#bib.bib15)\)\.

Figure 1:Generalization Dimensions\.![Refer to caption](https://arxiv.org/html/2610.02549v1/experiment_setup.png)Figure 2:Pipeline for evaluation across tasks, architectures, and reasoning strategies\.We present a systematic study of the multi\-dimensional generalization of recent LLMs in time and event expression extraction tasks \(timexandeventx\)\. Following prior work\([Hupkes et al\., 2023](https://arxiv.org/html/2610.02549#bib.bib9);[Dutt et al\., 2024](https://arxiv.org/html/2610.02549#bib.bib15)\), we consider four dimensions of generalization \(Figure[1](https://arxiv.org/html/2610.02549#S1.F1)\)\.Domain generalizationmeasures models’ ability to handle distribution shifts across domains with different linguistic characteristics \(e\.g\., general vs\. clinical text\)\.Robustness generalizationtests models’ robustness to meaning\-preserving perturbations\. ForCompositional generalization, the task is extended from extracting time and event expressions independently to extracting them jointly\. Finally, forStructural generalization, the input context varies from sentence\-level to document\-level\. These dimensions form one axis of variation, capturing how model behavior changes under different types of distribution shifts\.

In parallel, we consider reasoning strategies as another axis of variation in model behavior\. Reasoning strategies define how inputs are framed and structured under different prompting conditions, including deductive, inductive, and abductive reasoning, as well as combinations of these\([Liu et al\., 2024](https://arxiv.org/html/2610.02549#bib.bib16)\), and performance under one strategy does not necessarily transfer to others\. Together, multiple generalization dimensions and reasoning strategies define two complementary axes that enable a systematic evaluation of model configurations\.

We evaluate models from three families: LLaMA\([Touvron et al\., 2023](https://arxiv.org/html/2610.02549#bib.bib10)\), Qwen\([Bai et al\., 2023](https://arxiv.org/html/2610.02549#bib.bib51)\), and Mistral\([Jiang et al\., 2024](https://arxiv.org/html/2610.02549#bib.bib50)\), and for each, we use22architectural variants: Dense and Mixture\-of\-Experts \(MoE\)\. This, combined with the77reasoning strategies, yields a total of 42 model configurations \(ℳi\\mathcal\{M\}\_\{i\}s\)\. Eachℳi\\mathcal\{M\}\_\{i\}is evaluated on both temporal and event extraction tasks across all generalization dimensions\. With this extensive setup \(Figure[2](https://arxiv.org/html/2610.02549#S1.F2)\), we ask the following research questions:

- •RQ1:If a model configurationℳi\\mathcal\{M\}\_\{i\}performs well on base extraction tasks, does the performance consistently extend across all generalization dimensions? If not, practitioners risk selectingℳi\\mathcal\{M\}\_\{i\}s that perform well in standard evaluations but fail in practice \(§[3\.1](https://arxiv.org/html/2610.02549#S3.SS1)\)\.
- •RQ2:To what extent are different generalization dimensions correlated? Specifically, if a configuration performs well in one dimension, how likely is it to perform well in others? Unlike RQ1, which studies transfer from base tasks, RQ2 examines interdependence among generalization dimensions \(§[3\.2](https://arxiv.org/html/2610.02549#S3.SS2)\)\.
- •RQ3:How do prompting and model factors affect generalization across dimensions? We examine the effects of reasoning strategy, model architecture \(Dense vs\. MoE\), and model size\. These factors are orthogonal to the generalization dimensions studied in RQ1 and RQ2, and their effects may vary across dimensions \(§[3\.3](https://arxiv.org/html/2610.02549#S3.SS3)\)\.

We show that strong base\-task performance generally predicts multi\-dimensional generalization, except under substantial distribution shifts\. Among reasoning strategies, inductive ones perform well on base tasks as expected, but they also exhibit strong multi\-dimensional generalization and greater cross\-dimensional consistency\. Surprisingly, combining instructions \(deductive\) with demonstrations \(inductive\) does not meaningfully improve generalization, whereas prompting the model to produce explanations \(abductive\) does\. MoE models outperform their dense counterparts on base tasks and across many dimensions of generalization, but this advantage diminishes under substantial distributional shifts\. Finally, scaling model size improves adversarial and structural generalization but yields no significant gains for domain or compositional ones\. Compared to prompted LLMs, fine\-tuned PLMs used as baselines exhibit more consistent cross\-dimensional behavior, though this consistency does not always reflect strong performance\. These findings call for reasoning strategies and evaluation frameworks that explicitly target robust, cross\-dimensional generalization rather than assuming it follows from strong base\-task performance\.

## 2Experimental Setup

### 2\.1Tasks & Datasets

Temporal reasoning in natural language requires identifying what happened and when\. This involves extracting event expressions \(eventx\), which capture semantic occurrences \(e\.g\., actions, states, or reporting events\), and temporal expressions \(timex\), which capture dates, times, durations, or relative references like today or last week\. These span extraction problems are prerequisites for advanced temporal reasoning tasks such as ordering events, determining causality, or building timelines\. Therefore, we focus on them\.

Base Corpus: TimeBank\.We derive the base datasets for bothtimexandeventxtasks from the TimeBank 1\.2 corpus\([Pustejovsky et al\., 2003](https://arxiv.org/html/2610.02549#bib.bib2)\)\. The corpus is converted into sentence\-level instances by segmenting documents and retaining only sentences that contain at least one target expression, with annotations projected accordingly\. We remove all additional annotation attributes \(e\.g\., normalization for temporal expressions and linguistic features for events\)\. Each dataset consists of sentence–target pairs, with a raw sentence as input and a set of textual spans as output\.

Adversarial Datasets:Adv&Adv\_P\.We construct adversarial variants of the base datasets using a modified SeqAttack framework\([Simoncini and Spanakis, 2021](https://arxiv.org/html/2610.02549#bib.bib3)\), adopting two widely used attack strategies: TextFooler\([Jin et al\., 2020](https://arxiv.org/html/2610.02549#bib.bib6)\)and DeepWordBug\([Gao et al\., 2018](https://arxiv.org/html/2610.02549#bib.bib5)\)\. TextFooler is a word\-level attack that iteratively replaces important tokens with semantically similar synonyms while preserving the overall meaning\. In contrast, DeepWordBug operates at the character level, introducing small perturbations \(e\.g\., swaps, insertions, deletions\) that disrupt tokenization and degrade model performance\. Fortimexwe use TextFooler and DeepWordBug to generate adversarial examples for each sentence and merge the perturbations\. Foreventx, we restrict the perturbation to TextFooler, as event triggers are sensitive to local syntactic context\. These datasets are referred to asAdv\.

We also construct a prompt\-based adversarial variant \(Adv\_P\) using GPT\-4\.1\([Xu et al\., 2023](https://arxiv.org/html/2610.02549#bib.bib45)\)\. We prompt the model to introduce lexical substitutions, character\-level noise, and structural modifications, while preserving sentence meaning\. We validate that all target expressions remain unchanged\.

Domain\-Shift: Clinical Dataset \(T2C\) and Vocabulary Replacement Dataset \(Voc\)\.To evaluate cross\-domain generalization, we construct datasets \(T2C\) using the 2012 i2b2 Clinical Temporal Relations corpus\([Sun et al\., 2013](https://arxiv.org/html/2610.02549#bib.bib4)\)\. Compared to TimeBank news data, clinical narratives contain specialized medical vocabulary, abbreviations, and telegraphic sentences\. To ensure consistency, we apply the same preprocessing pipeline as in the base datasets\.

We create another domain dataset \(Voc\) by systematically replacing selected tokens \(e\.g\. entity\-like tokens, rare words, and domain\-specific terms\) with synthetic or semantically related alternatives, preserving sentence structure and label\([Ribeiro et al\., 2020](https://arxiv.org/html/2610.02549#bib.bib54)\)\. Compared toT2C,Vocrepresents a more controlled and localized shift in vocabulary rather than a full domain change\.

Structural Generalization: Document Level Dataset \(Len\)\.Structural generalization refers to models’ ability to effectively handle inputs of varying or increased length\. We revert to the original documents of the TimeBank corpus for this purpose\.

Compositional Dataset \(Comp\)\.Compositional generalization captures a model’s ability to solve more complex tasks by systematically combining solutions to smaller, simpler sub\-tasks it has learned\. To mimic this, we repurpose the TimeBank dataset so that the models must jointly extract time and event expressions\.

See §[A\.4](https://arxiv.org/html/2610.02549#A1.SS4)for a detailed discussion, and examples from all base and generalization datasets \(Figures[20](https://arxiv.org/html/2610.02549#A1.F20)and[21](https://arxiv.org/html/2610.02549#A1.F21)\)\.

### 2\.2Models

Based on prior work highlighting trade\-offs between throughput, latency, and memory efficiency during inference\([Agrawal et al\., 2024](https://arxiv.org/html/2610.02549#bib.bib8);[Fu et al\., 2024](https://arxiv.org/html/2610.02549#bib.bib7)\), we organize models into two computational tiers\.

Medium Tier \(Dense Architectures, 16–24 GB VRAM\)consists of compact dense models that run efficiently on a single GPU\. We include[Mistral 8B](https://openrouter.ai/mistralai/ministral-8b-2512),[Llama 3 8B Instruct](https://openrouter.ai/meta-llama/llama-3-8b-instruct), and[Qwen2\.5 7B Instruct](https://openrouter.ai/qwen/qwen-2.5-7b-instruct), representing small\-scale dense architectures with comparable parameter sizes\.

Heavy Tier \(Mixture\-of\-Experts Models,≥\\geq48 GB VRAM\)includes large\-scale mixture\-of\-experts \(MoE\) models with conditional routing, enabling us to study the effect of expert activation on generalization\. We evaluate[Llama 4 Maverick](https://openrouter.ai/meta-llama/llama-4-maverick)\(17B active parameters\),[Qwen3 235B A22B](https://openrouter.ai/qwen/qwen3-235b-a22b-2507)\(22B active parameters\), and[Mixtral 8x7B](https://openrouter.ai/mistralai/mixtral-8x7b-instruct)Instruct \(13B active parameters\)\.

### 2\.3Reasoning strategy

Reasoning strategy\([Liu et al\., 2024](https://arxiv.org/html/2610.02549#bib.bib16)\)forms one of the primary axes of our study, alongside the different generalization dimensions\. We evaluate each model under seven strategies: inductive \(ℐ\\mathcal\{I\}\), deductive \(𝒟\\mathcal\{D\}\), abductive \(𝒜\\mathcal\{A\}\), and their combinations, across bothtimexandeventxtasks\.

Inductivereasoning corresponds to in\-context learning, in which the task is implicitly specified by examples from which the model infers patterns\([Brown et al\., 2020](https://arxiv.org/html/2610.02549#bib.bib26)\)\. We implement this via few\-shot prompting \(§[A\.3\.1](https://arxiv.org/html/2610.02549#A1.SS3.SSS1)\), where models infer extraction patterns from annotated examples\. For consistency, examples are always drawn from the base dataset, and the best\-performing number of exemplars \(k∈3,5,8k\\in\{3,5,8\}\) is fixed across all settings\. In contrast, thedeductivereasoning does not use examples; instead, the task is explicitly specified as a set of rules or a natural language instruction, and the model applies these rules to produce outputs\. Specifically, we provide a clear definition of valid time and event expressions \(§[A\.3\.2](https://arxiv.org/html/2610.02549#A1.SS3.SSS2)\)\.Abductivereasoning involves generating intermediate hypotheses or explanations to guide prediction\([Wei et al\., 2022](https://arxiv.org/html/2610.02549#bib.bib46)\)\. In our setup, we implement this by generating candidate spans with reasoning steps and selecting those that best justify the extraction \(§[A\.3\.3](https://arxiv.org/html/2610.02549#A1.SS3.SSS3)\)\. We also consider combinations of reasoning strategies that integrate multiple paradigms within a single prompt\. For instance,𝒟\+ℐ\\mathcal\{D\+I\}incorporates both instructions and few\-shot examples, enabling the model to align explicit rules with observed patterns\. These hybrid setups capture interactions between reasoning mechanisms that are not observable in isolated settings\.

### 2\.4Evaluation Metrics

We evaluate the quality of predicted expressions using a Hungarian word\-level matching metric and report the micro F1 score as our primary metric\.

Let𝒫=\{p1,…,pm\}\\mathcal\{P\}=\\\{p\_\{1\},\\dots,p\_\{m\}\\\}denote the set of predicted expressions and𝒢=\{g1,…,gn\}\\mathcal\{G\}=\\\{g\_\{1\},\\dots,g\_\{n\}\\\}denote the set of gold expressions for a given instance\. For each pair\(pi,gj\)\(p\_\{i\},g\_\{j\}\), we compute a similarity score based on the word\-level Longest Common Subsequence \(LCS\), normalized by span length:sim​\(pi,gj\)=LCS​\(pi,gj\)max⁡\(\|pi\|,\|gj\|\)\\text\{sim\}\(p\_\{i\},g\_\{j\}\)=\\frac\{\\text\{LCS\}\(p\_\{i\},g\_\{j\}\)\}\{\\max\(\|p\_\{i\}\|,\|g\_\{j\}\|\)\}where\|pi\|\|p\_\{i\}\|and\|gj\|\|g\_\{j\}\|denote the number of tokens in the spans\.

We construct a similarity matrix𝐒∈ℝm×n\\mathbf\{S\}\\in\\mathbb\{R\}^\{m\\times n\}with entriesSi​j=sim​\(pi,gj\)S\_\{ij\}=\\text\{sim\}\(p\_\{i\},g\_\{j\}\)and compute an optimal one\-to\-one matchingℳ⊆𝒫×𝒢\\mathcal\{M\}\\subseteq\\mathcal\{P\}\\times\\mathcal\{G\}that maximizes the total similarity:ℳ=arg​maxmatching∑\(i,j\)∈ℳSi​j\\mathcal\{M\}=\\operatorname\*\{arg\\,max\}\_\{\\text\{matching\}\}\\sum\_\{\(i,j\)\\in\\mathcal\{M\}\}S\_\{ij\}using the Hungarian algorithm\([Kuhn, 1955](https://arxiv.org/html/2610.02549#bib.bib22)\)\. Precision∑\(i,j\)∈ℳSi​j\|𝒫\|\\frac\{\\sum\_\{\(i,j\)\\in\\mathcal\{M\}\}S\_\{ij\}\}\{\|\\mathcal\{P\}\|\}and Recall∑\(i,j\)∈ℳSi​j\|𝒢\|\\frac\{\\sum\_\{\(i,j\)\\in\\mathcal\{M\}\}S\_\{ij\}\}\{\|\\mathcal\{G\}\|\}are then defined based on the matched overlaps\.

This approach prevents multiple matches to the same expression, providing a more rigorous evaluation similar to span\-based protocols used in the CoNLL\-2003 Shared Task\([Sang and De Meulder, 2003](https://arxiv.org/html/2610.02549#bib.bib23)\)and SemEval\-2013 Task 9\.1\([Segura\-Bedmar et al\., 2013](https://arxiv.org/html/2610.02549#bib.bib24)\)\.

## 3Results

All prompted LLMs are accessed via the OpenRouter API111[https://openrouter\.ai/](https://openrouter.ai/)with default settings and temperature 0, ensuring a consistent and reproducible inference setup\. The detailed results are provided in §[A\.5](https://arxiv.org/html/2610.02549#A1.SS5)\.

### 3\.1RQ1: Transferability of Base Task Performance Across Dimensions

Figure 3:Ifℳ1\\mathcal\{M\}\_\{1\}is significantly better thanℳ2\\mathcal\{M\}\_\{2\}on the base case, what is the probability that the advantage will hold for different generalization dimensions?Figure 4:Generalization probabilities foreventx, split into inductive\+ \(left\) and non\-inductive \(right\) strategies\.We evaluate whether strong performance on base extraction tasks translates across all generalization dimensions\. To do so, we examine whether pairwise orderings between configurations, established on the base dataset, persist across generalization dimensions\. Specifically, consider two configurationsℳi\\mathcal\{M\}\_\{i\}andℳj\\mathcal\{M\}\_\{j\}whereiiis significantly better thanjjon thebasedataset\. What is the probability thatℳi\\mathcal\{M\}\_\{i\}will be significantly better, worse, or almost equal toℳj\\mathcal\{M\}\_\{j\}ingeneralizationdatasets? Figure[3](https://arxiv.org/html/2610.02549#S3.F3)shows the results for bothtimexandeventxdatasets\. Statistical significance \(p<0\.05\\text\{p\}<0\.05\) is determined using a paired bootstrap test\([Berg\-Kirkpatrick et al\., 2012](https://arxiv.org/html/2610.02549#bib.bib53)\)\.

First, no configuration achieves a100%100\\%probability\. Based on base\-task performance, no singleℳi\\mathcal\{M\}\_\{i\}consistently guarantees better generalization than the others\. However, fortimex, if anℳi\\mathcal\{M\}\_\{i\}performs better thanℳj\\mathcal\{M\}\_\{j\}on the base task, it is highly likely to generalize better thanℳj\\mathcal\{M\}\_\{j\}on adversarial, length, and compositional datasets\. If the domain generalization dataset has a very strong distribution shift \(e\.g\.,T2C\), then the models will have a much lower probability of generalizing; otherwise, e\.g\., with a simple vocabulary shift \(Voc\), the generalization trend will hold\.eventxresults are also similar, but theT2Cgeneralization probability reduces even further\.

Can theT2Canomaly be attributed toinductive\+\(ℐ\\mathcal\{I\},𝒜\+ℐ\\mathcal\{A\+I\},𝒟\+ℐ\\mathcal\{D\+I\},𝒜\+𝒟\+ℐ\\mathcal\{A\+D\+I\}\) reasoning strategies? These strategies generally yield better performance on the base dataset \(see Figure[12](https://arxiv.org/html/2610.02549#A1.F12), §[A\.1\.1](https://arxiv.org/html/2610.02549#A1.SS1.SSS1)\), likely because they use exemplars drawn from the same distribution\. However, this use of in\-domain examples may negatively affect generalization\. In Figure[4](https://arxiv.org/html/2610.02549#S3.F4)we plot these probabilities separately for inductive\+ and non\-inductive strategies foreventxdatasets\. Interestingly, the use of in\-domain examples boosts generalization, even for difficult domain shifts such asT2C\.

### 3\.2RQ2: Correlation Across Generalization Dimensions

![Refer to caption](https://arxiv.org/html/2610.02549v1/time_all_without_clinical_configwise_sr_rank.png)\(a\)Time expression datasets
![Refer to caption](https://arxiv.org/html/2610.02549v1/event_all_without_clinical_configwise_sr_rank.png)\(b\)Event expression datasets

Figure 5:Spearman’s Rank Correlationρ\\rhobetween the source and the target datasets for different generalization dimensions: temporal and event expression extraction\.For RQ2, we assess whether configurations \(ℳi\\mathcal\{M\}\_\{i\}\) maintain consistent relative performance across different generalization dimensions\. We first rank all configurations separately on each dataset, then compute Spearman’s rank correlation \(ρ\\rho\) between every pair of datasets to measure how similar these rankings are\. Figures[5\(a\)](https://arxiv.org/html/2610.02549#S3.F5.sf1)and[5\(b\)](https://arxiv.org/html/2610.02549#S3.F5.sf2)show the results fortimexandeventxtasks, respectively\.

Fortimexdatasets, model rankings show moderate\-to\-strong consistency across most generalization dimensions\. Adversarial perturbation datasets are well correlated with each other as expected\.222ρ=0\.75\\rho=0\.75, andρ≥0\.6\\rho\\geq 0\.6indicates strong to very\-strong correlation[Statstutor \(2004\)](https://arxiv.org/html/2610.02549#bib.bib1)\.The domain\-shift datasets \(T2CandVoc\) have lower but moderate correlations\. Interestingly, adversarial generalization \(across both datasets\) is well correlated with domain and structural generalization, but not with compositional generalization\. These trends tend to hold for event expression datasets, except forT2C, which is, in fact, negatively correlated with other dimensions\. A negative correlation implies thatℳi\\mathcal\{M\}\_\{i\}s that perform well onT2Ctend to perform worse on other dimensions, i\.e\., optimizing for this domain shift may harm generalization elsewhere\. As before, we investigate whether this is due to sample selection in the inductive strategies\. Figures[13\(a\)](https://arxiv.org/html/2610.02549#A1.F13.sf1)and[13\(b\)](https://arxiv.org/html/2610.02549#A1.F13.sf2)\(§[A\.1\.2](https://arxiv.org/html/2610.02549#A1.SS1.SSS2)\) show that this is not the case\. The correlations, although low, are much higher in the inductive\+ strategies than the non\-inductive ones\.

To further characterizeT2C’s distinct behavior, we analyze its extraction errors in Figure[6](https://arxiv.org/html/2610.02549#S3.F6)\. Fortimex, errors consist of both missing and boundary\-mismatched spans, whereaseventxerrors are dominated by missing spans, with 66\.7–79\.5% of gold event expressions missed across models\. The t\-SNE projections in Figure[7](https://arxiv.org/html/2610.02549#S3.F7), based on final\-layerroberta\-baseembeddings of the 100 most frequent tokens, further illustrate the distributional differences among Base,T2C, andVoc\. Greater separation is observed betweenT2Cand Base tokens than betweenVocand Base, suggesting a stronger lexical and contextual shift\. We hypothesize this to be a reason forT2Cbehaving differently from other generalization dimensions\. In summary, a strong base\-task performance ofℳi\\mathcal\{M\}\_\{i\}is generally a good indicator of multi\-dimensional generalization, with some exceptions such as significant distribution shifts \(e\.g\.,T2C\)\. Most importantly, inductive\+ strategies, where few\-shot examples are sampled from the base datasets, not only boostbase performance\(Figure[12](https://arxiv.org/html/2610.02549#A1.F12), §[A\.1\.1](https://arxiv.org/html/2610.02549#A1.SS1.SSS1)\) but also improve generalization\. Furthermore, they maintain significantly higher cross\-dimensional correlation than non\-inductive approaches\.

Figure 6:Extraction error analysis onT2C, pooled across all reasoning strategies\.\(a\)timexdatasets\.\(b\)eventxdatasets\.
Figure 7:t\-SNE projections of final\-layerroberta\-basetoken embeddings from the Base,T2C, andVocdatasets\.
### 3\.3RQ3: Effects of Reasoning Strategies, Model Architecture, and Scale

Reasoning strategy\.The previous results indicate that inductive\+ strategies are better for generalization\. Following up, we investigate whether, among these strategies, a particular one, say,𝒟\+ℐ\\mathcal\{D\+I\}, is better than the others\. Specifically, for any reasoning strategy pair \(ri,rjr\_\{i\},r\_\{j\}\) and a generalization dimension, say,T2C, we note the percentage of models that perform significantly \(p<0\.05\\text\{p\}<0\.05\) better, worse, or almost equal \(p≥0\.05\\text\{p\}\\geq 0\.05\) forrir\_\{i\}thanrjr\_\{j\}\. Figure[8](https://arxiv.org/html/2610.02549#S3.F8)shows the results when the results are averaged overall dimensions\. Results for individual dimensions are shown in Figures[14](https://arxiv.org/html/2610.02549#A1.F14)and[15](https://arxiv.org/html/2610.02549#A1.F15), §[A\.1\.3](https://arxiv.org/html/2610.02549#A1.SS1.SSS3), and have similar trends\.

Figure 8:Percentage of models that perform significantlybetter,worse, oralmost equalfor reasoning strategyiivs\.jj\(Hyp = i vs j\), summed over all generalization dimensions\. The upper and lower panels show the results fortimex\(T\) andeventx\(E\) tasks, respectively\.Fortimex, addition of deductive prompts, i\.e\., instructions, hampers generalization, but the addition of explanations improves it \(ℐ\\mathcal\{I\}is better thanℐ\+𝒟\\mathcal\{I\+D\}, andℐ\+𝒜\\mathcal\{I\+A\}is better thanℐ\+𝒟\\mathcal\{I\+D\}\)\. Perhaps most interestingly,ℐ\+𝒟\+𝒜\\mathcal\{I\+D\+A\}, i\.e\., “few\-shot chain\-of\-thought” is not the best reasoning strategy for generalization\. Foreventx, the base inductive strategy \(ℐ\\mathcal\{I\}\) is better than all other alternatives\.

Figure 9:Percentage of Dense models that perform significantlybetter,worse, oralmost equalfor the corresponding MoE model in a particular dimension \(dim = x\), summed over all reasoning strategies\.Model architecture\.For each generalization dimension, we report whether Dense models perform significantly better, worse, or nearly equal to the closest available MoE model from the same family\. Exact official size\-matched MoE counterparts are not available, so we use the closest comparable variants: Llama 4 Maverick, Mixtral, and Qwen MoE\. Although these MoE models have larger active parameter counts, they remain broadly comparable in scale \(Llama: 8B vs\. 17B, Mistral: 8B vs\. 13B, Qwen: 7B vs\. 22B\)\. Thus, worse MoE generalizationdespite larger active sizesuggests limitations beyond parameter count\. Figure[9](https://arxiv.org/html/2610.02549#S3.F9)shows the results\.

MoE models outperform Dense counterparts on base tasks, but their advantage weakens under larger shifts such asT2CandAdv\_P, where they perform similarly to Dense models, and oneventxtasks, where they perform worse\. This suggests that higher active parameter counts do not reliably translate into better domain generalization\.

Model size\.We investigate the effect of model size on generalization with 4 Dense Mistral models of sizes 3B to 123B and a subset of dimensions: a\)Adv, b\) Length, c\)T2C, and d\) Compositional\. The results are presented in Figure[10](https://arxiv.org/html/2610.02549#S3.F10)\.

Figure 10:Generalization performance of Mistral Dense models with varying parameter sizes\.Model size correlates well with base\-task performance and generalization, but the gains are not uniform across all dimensions\. Performance improves monotonically in base, and the same behavior is observed inAdvand Length\. However, forT2Cand compositional, the performance difference between smaller changes \(3B→\\rightarrow8B\) is non\-significant\. Surprisingly, foreventx, model size has no effect on domain generalization\. In summary, scaling alone does not guarantee consistent gains across all generalization dimensions\.

Figure 11:The proportion oftimex\(upper panel\) andeventx\(lower panel\) datasets where a fine\-tuned PLMmim\_\{i\}is significantly better, worse, or almost equal to another modelmjm\_\{j\}\(hyp =mim\_\{i\}vsmjm\_\{j\}\)\.Fine\-tuned Models\.While our main focus is prompted LLMs, we include a complementary analysis of fine\-tuned pretrained models to examineconsistency, rather than absolute generalization performance\. Following[Dutt et al\. \(2024\)](https://arxiv.org/html/2610.02549#bib.bib15), we train RoBERTa\([Liu et al\., 2019](https://arxiv.org/html/2610.02549#bib.bib13)\), T5\([Raffel et al\., 2020](https://arxiv.org/html/2610.02549#bib.bib52)\), and GPT\-2\([Radford et al\., 2019](https://arxiv.org/html/2610.02549#bib.bib14)\)on the basetimexandeventxsequence labeling tasks, and evaluate on one dataset from each domain, length and adversarial dimension\. Details are provided in Appendix §[A\.2](https://arxiv.org/html/2610.02549#A1.SS2)\. Figure[11](https://arxiv.org/html/2610.02549#S3.F11)reports the proportion of generalization datasets where modelmim\_\{i\}is significantly better, worse, or almost equal to modelmjm\_\{j\}\. Although all three PLMs perform poorly in some generalization dimensions, eg\.,Adv\_PandT2C\(Table[10](https://arxiv.org/html/2610.02549#A1.T10), §[A\.5](https://arxiv.org/html/2610.02549#A1.SS5)\), their relative behavior is stable: RoBERTa consistently outperforms T5 and GPT\-2 across all dimensions\. This contrasts with prompted LLMs, where no configuration shows comparable cross\-dimensional consistency\.

## 4Related Work

Prior work has studied NLP generalization from multiple perspectives and proposed methods to improve it across dimensions\.\([Hupkes et al\., 2023](https://arxiv.org/html/2610.02549#bib.bib9)\)provides a broad taxonomy of generalization, from which we focus on four key categories that cover most benchmark settings\. Cross\-task generalization captures a model’s ability to transfer knowledge across tasks, where instruction\-tuned LLMs trained on large\-scale annotated data have shown strong performance\([Zhang et al\., 2026](https://arxiv.org/html/2610.02549#bib.bib25)\)\. Models such as GPT\-3\([Brown et al\., 2020](https://arxiv.org/html/2610.02549#bib.bib26)\)and LLaMA2\([Touvron et al\., 2023](https://arxiv.org/html/2610.02549#bib.bib10)\)further demonstrate zero\-shot capabilities, though concerns about data contamination raise questions about whether these results reflect true generalization\([Li and Flanigan, 2024](https://arxiv.org/html/2610.02549#bib.bib27)\)\. Beyond task transfer, prior work has also explored reasoning strategies as a lens for evaluating model behavior\([Liu et al\., 2024](https://arxiv.org/html/2610.02549#bib.bib16)\)\. Within the dimensions we study, domain generalization has been widely explored\([Ramponi and Plank, 2020](https://arxiv.org/html/2610.02549#bib.bib28);[Naik et al\., 2022](https://arxiv.org/html/2610.02549#bib.bib29)\), while robustness is typically evaluated using adversarial perturbations, including both white\-box and black\-box attacks\([Ebrahimi et al\., 2018](https://arxiv.org/html/2610.02549#bib.bib12);[Jin et al\., 2020](https://arxiv.org/html/2610.02549#bib.bib6);[Goyal et al\., 2023](https://arxiv.org/html/2610.02549#bib.bib30)\)\. However, many such methods produce unrealistic inputs and do not capture shortcut learning tendencies\([Geirhos et al\., 2020](https://arxiv.org/html/2610.02549#bib.bib31)\)\. Compositional generalization has been studied across tasks such as machine translation, semantic parsing, and question answering\([Dankers et al\., 2022](https://arxiv.org/html/2610.02549#bib.bib32);[Kim and Linzen, 2020](https://arxiv.org/html/2610.02549#bib.bib33);[Keysers et al\., 2020](https://arxiv.org/html/2610.02549#bib.bib34)\), though systematic dataset construction via composition remains limited\. Structural generalization, particularly length generalization, has received comparatively less attention, despite consistent evidence that models struggle in “train\-short, test\-long” settings\([Press et al\., 2022](https://arxiv.org/html/2610.02549#bib.bib47);[Anil et al\., 2022](https://arxiv.org/html/2610.02549#bib.bib48);[Ruoss et al\., 2023](https://arxiv.org/html/2610.02549#bib.bib49)\)\.

In parallel, a large body of work has proposed methods to improve generalization\. Domain adaptation approaches include gradual fine\-tuning across domains and domain adversarial training\([Xu et al\., 2021](https://arxiv.org/html/2610.02549#bib.bib35);[Wright and Augenstein, 2020](https://arxiv.org/html/2610.02549#bib.bib36)\)\. Robustness has been addressed through adversarial augmentation, loss regularization, and generative adversarial training\([Li et al\., 2019](https://arxiv.org/html/2610.02549#bib.bib37);[Goodfellow et al\., 2015](https://arxiv.org/html/2610.02549#bib.bib11);[Kang et al\., 2018](https://arxiv.org/html/2610.02549#bib.bib38)\)\. For compositionality, earlier work relies on neuro\-symbolic methods and meta\-learning, while more recent approaches leverage structured prompting in LLMs\([Gupta et al\., 2020](https://arxiv.org/html/2610.02549#bib.bib39);[Lake, 2019](https://arxiv.org/html/2610.02549#bib.bib40);[Press et al\., 2023](https://arxiv.org/html/2610.02549#bib.bib41)\)\. Additionally, several studies explore the integration of inductive, deductive, and abductive reasoning to enhance model capabilities\([He et al\., 2025](https://arxiv.org/html/2610.02549#bib.bib42);[Hu et al\., 2025](https://arxiv.org/html/2610.02549#bib.bib43);[Cao et al\., 2026](https://arxiv.org/html/2610.02549#bib.bib44)\)\. In contrast to work that improves generalization methods or evaluates broader temporal reasoning in LLMs\([Fatemi et al\., 2025](https://arxiv.org/html/2610.02549#bib.bib55);[Bazaga et al\., 2025](https://arxiv.org/html/2610.02549#bib.bib56);[Park et al\., 2025](https://arxiv.org/html/2610.02549#bib.bib57)\), we systematically assess how existing models generalize on the foundational extraction tasks of identifying time and event expressions\.

## 5Conclusion & Future Work

We present a systematic study of multidimensional generalization in prompted LLMs for time and event expression extraction, covering domain, robustness, compositional, and structural shifts\. Our results show that base\-task performance is not a reliable proxy for generalization across dimensions\. Inductive reasoning strategies improve both base and generalization performance and yield stronger cross\-dimensional correlations, but model size and architecture are not consistently predictive of generalization\. In contrast, fine\-tuned PLMs show more stable relative behavior across dimensions, though not necessarily stronger absolute performance\. These findings suggest that reliable temporal extraction requires evaluation beyond in\-domain performance and motivate our future work on models and prompting strategies that generalize more consistently across diverse distribution shifts\.

## Acknowledgments

The authors acknowledge the Texas Advanced Computing Center \(TACC\) at The University of Texas at Austin for providing computational resources that contributed to the research results reported in this paper\. More information is available at[https://www\.tacc\.utexas\.edu](https://www.tacc.utexas.edu/)\.

## Limitations

Our study focuses on extracting time \(timex\) and event \(eventx\) expressions\. These tasks are core primitives for temporal understanding, but they do not directly evaluate higher\-level temporal reasoning tasks such as temporal relation inference, timeline construction, or temporal question answering\. A second limitation is possible contamination from pretraining corpora, especially for widely used benchmark datasets\. However, our main conclusions rely on relative degradation across distribution shifts, where models still show substantial drops, particularly under cross\-domain and robustness\-oriented settings\. Thus, even if some base datasets were partially seen during pretraining, the generalization gaps remain informative\.

Our model comparisons also have limits\. We refer to all prompted generative models as LLMs for consistency, although some smaller models in our study may also be described as small language models \(SLMs\)\. The dense and MoE models differ not only in architecture, but also in active parameter count, pretraining data, and model generation\. Therefore, our results shouldnotbe interpreted as isolating the causal effect of MoE architecture alone\. Similarly, our prompting comparison reflects end\-to\-end prompting strategies as they are commonly used\. Because reasoning style, prompt wording, and output format vary together, we cannot fully attribute performance differences to the reasoning strategy alone\.

## References

- Agrawalet al\.\(2024\)A\. Agrawal, N\. Kedia, A\. Panwar, J\. Mohan, N\. Kwatra, B\. Gulavani, A\. Tumanov, and R\. RamjeeTaming\{\\\{throughput\-latency\}\\\}tradeoff in\{\\\{llm\}\\\}inference with\{\\\{sarathi\-serve\}\\\}\.In18th USENIX symposium on operating systems design and implementation \(OSDI 24\),pp\. 117–134\.Cited by:[§2\.2](https://arxiv.org/html/2610.02549#S2.SS2.p1.1)\.
- Anilet al\.\(2022\)C\. Anil, Y\. Wu, A\. Andreassen, A\. Lewkowycz, V\. Misra, V\. Ramasesh, A\. Slone, G\. Gur\-Ari, E\. Dyer, and B\. NeyshaburExploring length generalization in large language models\.Advances in Neural Information Processing Systems35,pp\. 38546–38556\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Baiet al\.\(2023\)J\. Bai, S\. Bai, Y\. Chu, Z\. Cui, K\. Dang, X\. Deng, Y\. Fan, W\. Ge, Y\. Han, F\. Huang,et al\.Qwen technical report\.arXiv preprint arXiv:2309\.16609\.Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p6.1)\.
- Bazagaet al\.\(2025\)A\. Bazaga, R\. Blloshmi, B\. Byrne, and A\. de GispertLearning to reason over time: timeline self\-reflection for improved temporal reasoning in language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 28014–28033\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Berg\-Kirkpatricket al\.\(2012\)T\. Berg\-Kirkpatrick, D\. Burkett, and D\. KleinAn empirical investigation of statistical significance in nlp\.InProceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning,pp\. 995–1005\.Cited by:[§3\.1](https://arxiv.org/html/2610.02549#S3.SS1.p1.1)\.
- Brownet al\.\(2020\)T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.Language models are few\-shot learners\.Advances in neural information processing systems33,pp\. 1877–1901\.Cited by:[§2\.3](https://arxiv.org/html/2610.02549#S2.SS3.p2.1),[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Caoet al\.\(2026\)M\. Cao, X\. Tan, M\. E\. Akhter, M\. Valentino, M\. Liakata, X\. Wang, and N\. AletrasFundamental reasoning paradigms induce out\-of\-domain generalization in language models\.arXiv preprint arXiv:2602\.08658\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Cassidyet al\.\(2014\)T\. Cassidy, B\. McDowell, N\. Chambers, and S\. BethardAn annotation framework for dense event ordering\.InProceedings of the 52nd Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),pp\. 501–506\.Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p2.1)\.
- Dankerset al\.\(2022\)V\. Dankers, E\. Bruni, and D\. HupkesThe paradox of the compositionality of natural language: a neural machine translation case study\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 4154–4175\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Duttet al\.\(2024\)R\. Dutt, S\. R\. Choudhury, V\. V\. Rao, C\. Rose, and V\. V\. VydiswaranInvestigating the generalizability of pretrained language models across multiple dimensions: a case study of nli and mrc\.InProceedings of the 2nd GenBench Workshop on Generalisation \(Benchmarking\) in NLP,pp\. 165–182\.Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p3.1),[§1](https://arxiv.org/html/2610.02549#S1.p4.1),[§3\.3](https://arxiv.org/html/2610.02549#S3.SS3.p7.1)\.
- Ebrahimiet al\.\(2018\)J\. Ebrahimi, A\. Rao, D\. Lowd, and D\. DouHotFlip: White\-Box Adversarial Examples for Text Classification\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),pp\. 31–36\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Fatemiet al\.\(2025\)B\. Fatemi, S\. M\. Kazemi, A\. Tsitsulin, K\. Malkan, J\. Yim, J\. Palowitch, S\. Seo, J\. Halcrow, and B\. PerozziTest of time: a benchmark for evaluating llms on temporal reasoning\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 94426–94447\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Fuet al\.\(2024\)Y\. Fu, L\. Xue, Y\. Huang, A\. Brabete, D\. Ustiugov, Y\. Patel, and L\. Mai\{\\\{serverlessllm\}\\\}:\{\\\{low\-Latency\}\\\}serverless inference for large language models\.In18th USENIX Symposium on Operating Systems Design and Implementation \(OSDI 24\),pp\. 135–153\.Cited by:[§2\.2](https://arxiv.org/html/2610.02549#S2.SS2.p1.1)\.
- Gaoet al\.\(2018\)J\. Gao, J\. Lanchantin, M\. L\. Soffa, and Y\. QiBlack\-box generation of adversarial text sequences to evade deep learning classifiers\.In2018 IEEE security and privacy workshops \(SPW\),pp\. 50–56\.Cited by:[§2\.1](https://arxiv.org/html/2610.02549#S2.SS1.p3.1)\.
- Geirhoset al\.\(2020\)R\. Geirhos, J\. Jacobsen, C\. Michaelis, R\. Zemel, W\. Brendel, M\. Bethge, and F\. A\. WichmannShortcut learning in deep neural networks\.Nature Machine Intelligence2\(11\),pp\. 665–673\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Goodfellowet al\.\(2015\)I\. J\. Goodfellow, J\. Shlens, and C\. SzegedyExplaining and harnessing adversarial examples\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/1412.6572)Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Goyalet al\.\(2023\)S\. Goyal, S\. Doddapaneni, M\. M\. Khapra, and B\. RavindranA survey of adversarial defenses and robustness in nlp\.ACM Computing Surveys55\(14s\),pp\. 1–39\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Guptaet al\.\(2020\)N\. Gupta, K\. Lin, D\. Roth, S\. Singh, and M\. GardnerNeural module networks for reasoning over text\.InInternational Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Heet al\.\(2025\)K\. He, M\. Zhang, S\. Yan, P\. Wu, and Z\. ChenIdea: enhancing the rule learning ability of large language model agent through induction, deduction, and abduction\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 13563–13597\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Huet al\.\(2025\)Z\. Hu, Y\. Wang, H\. Dong, Y\. Xu, A\. Saha, C\. Xiong, B\. Hooi, and J\. LiBeyond’aha\!’: toward systematic meta\-abilities alignment in large reasoning models\.arXiv preprint arXiv:2505\.10554\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Hupkeset al\.\(2023\)D\. Hupkes, M\. Giulianelli, V\. Dankers, M\. Artetxe, Y\. Elazar, T\. Pimentel, C\. Christodoulopoulos, K\. Lasri, N\. Saphra, A\. Sinclair,et al\.A taxonomy and review of generalization research in nlp\.Nature Machine Intelligence5\(10\),pp\. 1161–1174\.Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p3.1),[§1](https://arxiv.org/html/2610.02549#S1.p4.1),[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Jianget al\.\(2024\)A\. Q\. Jiang, A\. Sablayrolles, A\. Roux, A\. Mensch, B\. Savary, C\. Bamford, D\. S\. Chaplot, D\. d\. l\. Casas, E\. B\. Hanna, F\. Bressand,et al\.Mixtral of experts\.arXiv preprint arXiv:2401\.04088\.Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p6.1)\.
- Jinet al\.\(2020\)D\. Jin, Z\. Jin, J\. T\. Zhou, and P\. SzolovitsIs bert really robust? a strong baseline for natural language attack on text classification and entailment\.InProceedings of the AAAI conference on artificial intelligence,Vol\.34,pp\. 8018–8025\.Cited by:[§2\.1](https://arxiv.org/html/2610.02549#S2.SS1.p3.1),[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Kanget al\.\(2018\)D\. Kang, T\. Khot, A\. Sabharwal, and E\. HovyAdventure: adversarial training for textual entailment with knowledge\-guided examples\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 2418–2428\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Keyserset al\.\(2020\)D\. Keysers, N\. Schärli, N\. Scales, H\. Buisman, D\. Furrer, S\. Kashubin, N\. Momchev, D\. Sinopalnikov, L\. Stafiniak, T\. Tihon,et al\.Measuring compositional generalization: a comprehensive method on realistic data\.InInternational Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Kim and Linzen \(2020\)N\. Kim and T\. LinzenCOGS: a compositional generalization challenge based on semantic interpretation\.InProceedings of the 2020 conference on empirical methods in natural language processing \(emnlp\),pp\. 9087–9105\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Kougiaet al\.\(2024\)V\. Kougia, A\. Sedova, A\. J\. Stephan, K\. Zaporojets, and B\. RothAnalysing zero\-shot temporal relation extraction on clinical notes using temporal consistency\.InProceedings of the 23rd Workshop on Biomedical Natural Language Processing,pp\. 72–84\.Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p2.1)\.
- Kuhn \(1955\)H\. W\. KuhnThe hungarian method for the assignment problem\.Naval research logistics quarterly2\(1\-2\),pp\. 83–97\.Cited by:[§2\.4](https://arxiv.org/html/2610.02549#S2.SS4.p3.1)\.
- Lake \(2019\)B\. M\. LakeCompositional generalization through meta sequence\-to\-sequence learning\.Advances in neural information processing systems32\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Li and Flanigan \(2024\)C\. Li and J\. FlaniganTask contamination: language models may not be few\-shot anymore\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 18471–18480\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Liet al\.\(2019\)J\. Li, S\. Ji, T\. Du, B\. Li, and T\. WangTEXTBUGGER: generating adversarial text against real\-world applications\.In26th Annual Network and Distributed System Security Symposium, NDSS 2019,Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Liuet al\.\(2024\)E\. Liu, G\. Neubig, and J\. AndreasAn incomplete loop: instruction inference, instruction following, and in\-context learning in language models\.arXiv preprint arXiv:2404\.03028\.Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p5.1),[§2\.3](https://arxiv.org/html/2610.02549#S2.SS3.p1.1),[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Liuet al\.\(2019\)Y\. Liu, M\. Ott, N\. Goyal, J\. Du, M\. Joshi, D\. Chen, O\. Levy, M\. Lewis, L\. Zettlemoyer, and V\. StoyanovRoBERTa: A robustly optimized BERT pretraining approach\.CoRRabs/1907\.11692\.External Links:[Link](https://arxiv.org/abs/1907.11692)Cited by:[§3\.3](https://arxiv.org/html/2610.02549#S3.SS3.p7.1)\.
- Naiket al\.\(2022\)A\. Naik, J\. Lehman, and C\. RoseAdapting to the long tail: a meta\-analysis of transfer learning research for language understanding tasks\.Transactions of the Association for Computational Linguistics10,pp\. 956–980\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Ninget al\.\(2018\)Q\. Ning, H\. Wu, and D\. RothA multi\-axis annotation scheme for event temporal relations\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1318–1328\.Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p1.1)\.
- Parket al\.\(2025\)Y\. Park, C\. Yoon, J\. Park, D\. Lee, M\. Jeong, and J\. KangChroknowledge: unveiling chronological knowledge of language models in multiple domains\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 5298–5337\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Presset al\.\(2022\)O\. Press, N\. A\. Smith, and M\. LewisTrain short, test long: attention with linear biases enables input length extrapolation, 2022\.URL https://arxiv\. org/abs/2108\.124095\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Presset al\.\(2023\)O\. Press, M\. Zhang, S\. Min, L\. Schmidt, N\. A\. Smith, and M\. LewisMeasuring and narrowing the compositionality gap in language models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 5687–5711\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Pustejovskyet al\.\(2003\)J\. Pustejovsky, J\. Castano, R\. Ingria, R\. Sauri, R\. Gaizauskas, A\. Setzer, G\. Katz, and D\. RadevTimeBank: a corpus for annotating temporal information\.InProceedings of the Corpus Linguistics Conference,Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.02549#S2.SS1.p2.1)\.
- Qiuet al\.\(2024\)Y\. Qiu, Z\. Zhao, Y\. Ziser, A\. Korhonen, E\. Ponti, and S\. B\. CohenAre large language model temporally grounded?\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 7064–7083\.Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p2.1)\.
- Radfordet al\.\(2019\)A\. Radford, J\. Wu, R\. Child, D\. Luan, D\. Amodei, and I\. SutskeverLanguage models are unsupervised multitask learners\.OpenAI Blog1\(8\),pp\. 9\.Cited by:[§3\.3](https://arxiv.org/html/2610.02549#S3.SS3.p7.1)\.
- Raffelet al\.\(2020\)C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. LiuExploring the limits of transfer learning with a unified text\-to\-text transformer\.Journal of machine learning research21\(140\),pp\. 1–67\.Cited by:[§3\.3](https://arxiv.org/html/2610.02549#S3.SS3.p7.1)\.
- Ramponi and Plank \(2020\)A\. Ramponi and B\. PlankNeural unsupervised domain adaptation in nlp—a survey\.InProceedings of the 28th international conference on computational linguistics,pp\. 6838–6855\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Ribeiroet al\.\(2020\)M\. T\. Ribeiro, T\. Wu, C\. Guestrin, and S\. SinghBeyond accuracy: behavioral testing of nlp models with checklist\.InProceedings of the 58th annual meeting of the association for computational linguistics,pp\. 4902–4912\.Cited by:[§2\.1](https://arxiv.org/html/2610.02549#S2.SS1.p6.1)\.
- Ruosset al\.\(2023\)A\. Ruoss, G\. Delétang, T\. Genewein, J\. Grau\-Moya, R\. Csordás, M\. Bennani, S\. Legg, and J\. VenessRandomized positional encodings boost length generalization of transformers\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),pp\. 1889–1903\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Sang and De Meulder \(2003\)E\. T\. K\. Sang and F\. De MeulderIntroduction to the conll\-2003 shared task: language\-independent named entity recognition\.InProceedings of the seventh conference on Natural language learning at HLT\-NAACL 2003,pp\. 142–147\.Cited by:[§2\.4](https://arxiv.org/html/2610.02549#S2.SS4.p4.1)\.
- Segura\-Bedmaret al\.\(2013\)I\. Segura\-Bedmar, P\. Martínez, and M\. Herrero\-ZazoSemeval\-2013 task 9: extraction of drug\-drug interactions from biomedical texts \(ddiextraction 2013\)\.InSecond Joint Conference on Lexical and Computational Semantics \(\* SEM\), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation \(SemEval 2013\),pp\. 341–350\.Cited by:[§2\.4](https://arxiv.org/html/2610.02549#S2.SS4.p4.1)\.
- Simoncini and Spanakis \(2021\)W\. Simoncini and G\. SpanakisSeqAttack: on adversarial attacks for named entity recognition\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,pp\. 308–318\.Cited by:[§2\.1](https://arxiv.org/html/2610.02549#S2.SS1.p3.1)\.
- Statstutor \(2004\)StatstutorSpearman’s correlation\.Note:Accessed: 2026\-03\-30External Links:[Link](https://www.statstutor.ac.uk/resources/uploaded/spearmans.pdf)Cited by:[footnote 2](https://arxiv.org/html/2610.02549#footnote2)\.
- Styler IVet al\.\(2014\)W\. F\. Styler IV, S\. Bethard, S\. Finan, M\. Palmer, S\. Pradhan, P\. C\. De Groen, B\. Erickson, T\. Miller, C\. Lin, G\. Savova,et al\.Temporal annotation in the clinical domain\.Transactions of the association for computational linguistics2,pp\. 143–154\.Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p2.1)\.
- Sunet al\.\(2013\)W\. Sun, A\. Rumshisky, and O\. UzunerAnnotating temporal information in clinical narratives\.Journal of biomedical informatics46,pp\. S5–S12\.Cited by:[§2\.1](https://arxiv.org/html/2610.02549#S2.SS1.p5.1)\.
- Touvronet al\.\(2023\)H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale,et al\.Llama 2: open foundation and fine\-tuned chat models\.arXiv preprint arXiv:2307\.09288\.Cited by:[§1](https://arxiv.org/html/2610.02549#S1.p6.1),[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§2\.3](https://arxiv.org/html/2610.02549#S2.SS3.p2.1)\.
- Wright and Augenstein \(2020\)D\. Wright and I\. AugensteinTransformer based multi\-source domain adaptation\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 7963–7974\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Xuet al\.\(2021\)H\. Xu, S\. Ebner, M\. Yarmohammadi, A\. S\. White, B\. Van Durme, and K\. MurrayGradual fine\-tuning for low\-resource domain adaptation\.InProceedings of the Second Workshop on Domain Adaptation for NLP,pp\. 214–221\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p2.1)\.
- Xuet al\.\(2023\)X\. Xu, K\. Kong, N\. Liu, L\. Cui, D\. Wang, J\. Zhang, and M\. KankanhalliAn llm can fool itself: a prompt\-based adversarial attack\.arXiv preprint arXiv:2310\.13345\.Cited by:[§2\.1](https://arxiv.org/html/2610.02549#S2.SS1.p4.1)\.
- Zhanget al\.\(2026\)S\. Zhang, L\. Dong, X\. Li, S\. Zhang, X\. Sun, S\. Wang, J\. Li, R\. Hu, T\. Zhang, G\. Wang,et al\.Instruction tuning for large language models: a survey\.ACM Computing Surveys58\(7\),pp\. 1–36\.Cited by:[§4](https://arxiv.org/html/2610.02549#S4.p1.1)\.

## Appendix AAppendix

### A\.1Additional Results

#### A\.1\.1Base Task Performance

Figure[12](https://arxiv.org/html/2610.02549#A1.F12)compares the average Base performance \(Micro\-F1\) of each model under two setups: inductive\+ and non\-inductive\. We observe that inductive\+ configurations consistently achieve higher scores across nearly all models in bothtimexandeventxtasks\. This suggests that incorporating inductive reasoning, through in\-context examples, improves extraction performance on the base dataset\.

Figure 12:Base task performance in inductive\+ vs\. non\-inductive strategies\.
#### A\.1\.2Correlation Across Generalization Dimensions

Figure[13](https://arxiv.org/html/2610.02549#A1.F13)presents the configuration\-wise rank correlation \(ρ\\rho\) across generalization dimensions for event extraction under inductive\+ and non\-inductive strategies\. In the inductive\+ setting \(Figure[13\(a\)](https://arxiv.org/html/2610.02549#A1.F13.sf1)\), we observe consistently strong correlations among Base, Adversarial, Vocabulary, and Length dimensions, indicating stable transfer of configuration rankings across these settings\. In contrast, the compositional \(Comp\) and T2C dimensions exhibit notably weaker correlations, suggesting limited alignment with other dimensions\. For non\-inductive strategies \(Figure[13\(b\)](https://arxiv.org/html/2610.02549#A1.F13.sf2)\), while Base, Adversarial, and Vocabulary remain moderately correlated, overall correlations are lower and more variable\. Notably, in non\-inductive strategies, the T2C dimension shows near\-zero or negative correlations with most other dimensions, whereas in inductive\+ setups, it shows much higher correlations\.

![Refer to caption](https://arxiv.org/html/2610.02549v1/event_inductive_without_clinical_configwise_sr_rank.png)\(a\)Inductive\+ strategies
![Refer to caption](https://arxiv.org/html/2610.02549v1/event_non-inductive_without_clinical_configwise_sr_rank.png)\(b\)Non\-inductive strategies

Figure 13:ρ\\rhoacross generalization dimensions for event expression extraction in inductive\+ and non\-inductive strategies\.
#### A\.1\.3Effect of Reasoning Strategy

Figure[14](https://arxiv.org/html/2610.02549#A1.F14)shows the percentage of configurations where one reasoning strategy outperforms, underperforms, or remains indistinguishable from another across generalization dimensions for time extraction\. Most comparisons fall into theequalcategory, suggesting similar performance across strategies\. Inductive\-based methods provide small but consistent gains in Base, Adversarial, and Vocabulary settings, while differences become more variable in compositional and cross\-domain \(T2C\) dimensions\. Moreover, combining all reasoning types \(I\+D\+AI\{\+\}D\{\+\}A\) does not consistently improve performance, indicating diminishing returns from added complexity\.

Figure[15](https://arxiv.org/html/2610.02549#A1.F15)summarizes the same comparison for event extraction\. Again, most strategy pairs perform similarly, with inductive variants remaining competitive but less consistently beneficial than in time extraction\. Variability increases in compositional and cross\-domain settings, where no single strategy consistently dominates\.

Figure 14:Percentage of models that performsignificantlybetter,worse, oralmost equalfor reasoning strategyiivs\.jj, expressed asHyp = i vs j, in each dimension \(Dim = x\) fortime expression extraction tasks\. Each column corresponds to one dimension\.Figure 15:Percentage of models that performsignificantlybetter,worse, oralmost equalfor reasoning strategyiivs\.jj, expressed asHyp = i vs j, in each dimension \(Dim = x\) forevent expression extraction tasks\. Each column corresponds to one dimension\.

### A\.2Model Training Details

##### Overview\.

We fine\-tune three pretrained transformer models—RoBERTa, T5, and GPT\-2 on two sequence labeling tasks: temporal expression extraction \(timex\) and event trigger extraction \(eventx\)\. Each model is trained independently for each task, resulting in six fine\-tuned models\. All models are adapted for token classification by adding a linear classification head over token representations\. Training is performed using the HuggingFace Transformers library with the Trainer API and the AdamW optimizer\.

##### Task Formulation\.

Both tasks are formulated as named entity recognition \(NER\) problems in CoNLL format, where each line contains a token and its corresponding label, and sentences are separated by blank lines\. We convert the original annotations into a BIO tagging format\. Temporal expressions and event mentions are mapped to token\-level labels, where each span is assigned a beginning \(B\) and inside \(I\) tag, and all other tokens are labeled as outside \(O\)\. Specifically:

- •timex:B\-T, I\-T, O
- •eventx:B\-E, I\-E, O

##### Dataset\.

We train all models on the base temporal expression and event extraction datasets\. Each task is represented in CoNLL format with predefined train and test splits\. The label encoder is fit on the union of training and test labels to ensure consistent label mappings across splits\.

##### Models\.

We evaluate three transformer architectures:

- •RoBERTa \(roberta\-base\):encoder\-only model with 12 layers and hidden size 768, implemented usingRobertaForTokenClassification\. Tokenization usesadd\_prefix\_space=True\.
- •T5 \(flan\-t5\-small\):encoder\-decoder model adapted for token classification usingT5ForTokenClassification, configured to operate as an encoder for this task\.
- •GPT\-2 \(gpt2\):decoder\-only model usingGPT2ForTokenClassification\. The EOS token is used as the padding token, and embeddings are resized usingresize\_token\_embeddings\(\)\.

##### Tokenization and Label Alignment\.

All models use subword tokenization\. Word\-level labels are aligned to subword tokens by assigning the original label to all subword pieces of a word\. Special tokens and padding tokens are assigned an ignore index of−100\-100\. All sequences use the maximum context length supported by each model\.

##### Training Configuration\.

All models are trained with identical hyperparameters:

- •Optimizer: AdamW
- •Learning rate:2×10−52\\times 10^\{\-5\}
- •Weight decay: 0\.01
- •Warmup steps: 500 \(linear schedule with decay\)
- •Epochs: 3
- •Train batch size: 16
- •Eval batch size: 32
- •Max sequence length: Full model context length

Model checkpoints are saved and evaluated every 250 steps\. Early stopping is applied with a patience of 3 evaluations\. Model selection is based on entity\-level F1 score\. All model parameters are updated during training \(full fine\-tuning\); no parameter\-efficient fine\-tuning methods \(e\.g\., LoRA or adapters\) are used\.

System Prompt:You are an expert system for time expression extraction in natural language text\.Task Definition:You are given annotated examples and a new input text\.Your task is to extract all explicit time expressions from the new input text by generalizing from the examples\. Time expressions may include dates, times, durations, frequencies, or other grounded temporal references explicitly stated in the text\.REASONING TYPE DEFINITION:You must use inductive reasoning, defined as follows:Infer which spans qualify as time expressions based on the annotated examplesLearn the typical lexical forms and boundary conventions of these expressionsApply the inferred patterns consistently to unseen input textYou should not rely on external knowledge or assumptions beyond what can be inferred from the examples\.OUTPUT FORMAT:Return ONLY the following:A valid JSON array of stringsEach string is one extracted time expressionNo duplicatesOrdered by first appearance in the textNo additional text, explanation, or formattingExamples:
\(Few\-shot annotated examples\)Input:
\(New context\)Figure 16:Inductive prompting template used for time expression extraction\.System Prompt:You are an expert system for time expression extraction in natural language text\.Task Definition:You are given a new input text\.Your task is to extract all explicit time expressions from the text by applying predefined rules\. Time expressions may include dates, times, durations, frequencies, or other grounded temporal references explicitly stated in the text\.A span qualifies as a time expression if it explicitly denotes:A calendar dateA clock timeA durationA frequency or recurrenceAnother grounded temporal referenceREASONING TYPE DEFINITION:You must use deductive reasoning, defined as follows:Apply the given definition of time expressions directly to the input textSelect only spans that strictly satisfy the stated criteriaDo not infer new rules or adapt based on the input textYou should rely only on the provided rules and not on examples or learned patterns\.OUTPUT FORMAT:Return ONLY the following:A valid JSON array of stringsEach string is one extracted time expressionNo duplicatesOrdered by first appearance in the textNo additional text, explanation, or formattingInput:
\(New context\)Figure 17:Deductive prompting template used for time expression extraction\.System Prompt:You are an expert system for time expression extraction in natural language text\.Task Definition:You are given a new input text\.Your task is to extract all explicit time expressions by using abductive reasoning\. Time expressions may include dates, times, durations, frequencies, or other grounded temporal references explicitly stated in the text\.REASONING TYPE DEFINITION:You must use abductive reasoning and think step by step, defined as follows:Generate candidate spans that could plausibly express temporal informationReason about how each candidate explains the timing, duration, or recurrence of events in the textSelect the spans that best account for the author’s intended temporal meaningPrefer the most complete and non\-overlapping spansOUTPUT FORMAT:Return ONLY a valid JSON list of dictionaries\.Each dictionary must contain the following fields:"text": the original input text"time\_expressions": a list of extracted time expression strings \(verbatim from the text\)\. If no time expressions are present,"time\_expressions"must be an empty list \(\[\]\)"reasoning\_steps": a list of strings explaining the abductive reasoning processExample Output:\[\{"text": "The conference started on April 9, 1997 and lasted for three days\.","time\_expressions": \["April 9, 1997", "three days"\],"reasoning\_steps": \["Identified a calendar date\.","Identified a duration phrase\.","Selected both spans\."\]\}\]Input:
\(New context\)Figure 18:Abductive prompting template used for time expression extraction\.System Prompt:You are an expert system for time expression extraction in natural language text\.Task Definition:You are given annotated examples and a new input text\.Your task is to extract all explicit time expressions from the new input text\. Time expressions may include dates, times, durations, frequencies, or other grounded temporal references explicitly stated in the text\.REASONING TYPE DEFINITION:You must use inductive, deductive, and abductive reasoning, and think step by step, defined as follows:Inductively infer from the annotated examples which spans qualify as time expressions, including their typical lexical forms and boundary conventions\.Deductively apply the inferred criteria to the new input text to identify candidate spans that satisfy the definition of a time expression\.Abductively select the spans that best explain the temporal meaning intended by the author, preferring the most complete and non\-overlapping spans\.You should not rely on external knowledge or assumptions beyond what can be inferred from the examples and the induced criteria\.OUTPUT FORMAT:Return ONLY a valid JSON list of dictionaries\.Each dictionary must contain the following fields:"text": the original input text"time\_expressions": a list of extracted time expression strings \(verbatim from the text\)\. If no time expressions are present,"time\_expressions"must be an empty list \(\[\]\)"reasoning\_steps": a list of strings explaining the reasoning processExample Output:\[\{"text": "The conference started on April 9, 1997 and lasted for three days\.","time\_expressions": \["April 9, 1997", "three days"\],"reasoning\_steps": \["Identified a calendar date\.","Identified a duration phrase\.","Selected both spans\."\]\}\]Input:
\(New context\)Figure 19:Inductive, deductive, and abductive prompting template used for time expression extraction\.

### A\.3Prompting Templates for Reasoning Strategies

In this section, we provide the prompting templates for the three reasoning strategies: inductive \(I\), deductive \(D\), and abductive \(A\)—for the temporal expression extraction task\. The same overall prompt structure is used for both temporal and event extraction tasks, although the task definitions and extraction criteria differ for each task\. These reasoning prompts are also used consistently across all generalization settings \(e\.g\., domain, compositional, robustness, and structural\), ensuring that differences in performance arise from the evaluation conditions rather than changes in prompt design\.

#### A\.3\.1Inductive Prompt

Inductive reasoning in our temporal extraction task is formulated as an example\-driven span identification process, where the model is provided with annotated instances of valid time expressions, as illustrated in Figure[16](https://arxiv.org/html/2610.02549#A1.F16)\. By observing these examples, the model learns the typical lexical patterns and boundary conventions associated with temporal spans, such as dates, durations, and frequencies\. It then generalizes these patterns to extract similar expressions from unseen input text\.

#### A\.3\.2Deductive Prompt

Deductive reasoning in our temporal extraction task is formulated as a rule\-based span selection process, where the definition of a valid time expression is specified explicitly in the prompt \(Figure[17](https://arxiv.org/html/2610.02549#A1.F17)\)\. The model is required to identify spans that directly correspond to grounded temporal references—such as dates, times, durations, and frequencies by strictly applying these criteria to the input text\. This setup removes reliance on example\-based learning and constrains the model to make extraction decisions based solely on the provided definition, ensuring consistent and precise identification of temporal spans\.

#### A\.3\.3Abductive Prompt

In the temporal extraction task, abductive reasoning is implemented as a hypothesis\-driven process in which the model generates candidate temporal spans and selects those that best explain the temporal structure of the text \(Figure[18](https://arxiv.org/html/2610.02549#A1.F18)\)\. The model is also required to generate explicit explanations describing its selection of the final time expressions\. This allows the model to incorporate contextual cues and resolve ambiguities in identifying time expressions\. For prompting settings involving abductive reasoning, we additionally provide one structured example to guide smaller models in following the required output format\.

#### A\.3\.4Inductive \+ Deductive \+ Abductive Prompt

Inductive, deductive, and abductive reasoning in our temporal extraction task is formulated as a combined example\-guided and hypothesis\-driven span identification process \(Figure[19](https://arxiv.org/html/2610.02549#A1.F19)\)\. The model first learns from annotated examples which kinds of spans typically function as time expressions and what their boundary conventions look like\. It then applies these inferred criteria to the new input text to identify candidate temporal spans\. Finally, it selects the spans that best explain the temporal meaning of the text, favoring complete and non\-overlapping expressions, while also providing brief reasoning steps for its decisions\.

### A\.4Datasets

The original document\-leveltimexandeventxdatasets derived from the TimeBank corpus contain 2\.12K training instances, 265 validation instances, and 266 test instances each\. From these document\-level datasets, we construct sentence\-level variants for both tasks\. The sentence\-leveltimexdataset contains 22\.7K training instances, 2\.67K validation instances, and 400 test instances\. Similarly, the sentence\-leveleventxdataset contains 23\.2K training instances, 3\.16K validation instances, and 400 test instances\.

The adversarial datasets \(AdvandAdv\_P\) are generated from the sentence\-level test sets for bothtimexandeventx\. The vocabulary\-shift datasets \(Voc\) are also constructed from the same sentence\-level test sets\.

For structural generalization \(Len\), we use 80 document\-level instances from the original TimeBank datasets for evaluation\. These instances have a minimum sequence length of 63 tokens, a maximum sequence length of 1134 tokens, and an average sequence length of 369\.98 tokens\. Among the 80 instances, 73 contain fewer than 512 tokens, which is the smallest context window among the evaluated PLM models\. This helps ensure that our structural generalization results are not affected by the documents being truncated\. For all prompted models, the context length exceeds the maximum sequence length\.

For compositional generalization \(Comp\), we select sentence\-level instances from the original document\-level corpus that contain both time and event expressions\. The resulting evaluation set contains 400 instances\.

#### A\.4\.1Label and Vocabulary Shift in T2C

For the clinical domain\-shift datasets \(T2C\) derived from the 2012 i2b2 Clinical Temporal Relations corpus, the clinicaltimexdataset contains 1\.32K training instances, 1\.35K validation instances, and 400 test instances\. The clinicaleventxdataset contains 5\.56K training instances, 1\.35K validation instances, and 400 test instances\.

The statistics in Table[1](https://arxiv.org/html/2610.02549#A1.T1)highlight substantial distributional differences between the original TimeBank labels \(news\-oriented corpus\) and the clinicalT2Clabels, which reflect the specialized language and domain\-specific characteristics of clinical narratives\.

Table 1:Top 10 most frequent time and event expressions across the Base \(Base\) and clinical \(T2C\) datasets\.
#### A\.4\.2Time Expression extraction

We evaluate time expression extraction under six different settings:Base,Adversarial,Domain,Domain\-Vocab,Compositional, andStructural\. The base setting uses sentence\-level examples from TimeBank, while the adversarial setting adds perturbations to test robustness\. The domain and domain\-vocabulary settings measure generalization to clinical and modified vocabulary data, and the compositional setting requires extracting both time expressions and event triggers together\. The structural setting uses longer document\-level inputs with multiple temporal references and more complex context\. Figure[20](https://arxiv.org/html/2610.02549#A1.F20)shows example instances from each setting\. These settings provide a comprehensive evaluation framework for analyzing temporal extraction generalization across diverse conditions\.

BaseText:MHB technicians said they started the surveillance mission Thursday to monitor the body temperature of all Macao residents and visitors\. Extracted span:Thursday

AdversarialText:MHB technicianstelltheylaunchesthemonitor remitThursday toobservertheentity warmthof all Macaoresideandtraveler\. Extracted span:Thursday

Domain\(T2C\)Text:Pregnancy was IUI assisted for di\-di twin gestation and was relatively uncomplicated until mother presented on 12\-15, with spontaneous rupture of membranes\. Extracted span:12\-15

Domain\-VocabText:MHB technicians said they started the surveillance mission Thursday to monitor the body temperature of allMyfiuresidents and visitors\. Extracted span:Thursday

CompositionalText:The longtime overheating then led to a blowout and the cold air went into the T101 aniline processing tower, which was run under negative pressure and triggered the blast at about 1:45 p\.m\., and other blasts happened afterwards\. Extracted time and event spans:1:45 p\.m\.; overheating; led; blowout; went; run; triggered; blast; blasts; happened

StructuralText:XIN\_ENG\_20061129\.0012 WASHINGTON, Nov\. 29, 2006 \(Xinhua\) Clippers, Dunleavy reportedly reach four\-year deal\. The Los Angeles Clippers and coach Mike Dunleavy have reached a four\-year contract extension worth more than 21 million U\.S\. dollars, according to the Los Angeles Times Tuesday\. The team and Dunleavy’s agent have worked out a deal that will pay him more than 5\.3 million U\.S\. dollars a season\. The Clippers have not made the deal public because bonuses tied to future playoff appearances were still being worked out, the Times said\. The Clippers improved dramatically under Dunleavy last season, posting their best win total since the franchise’s glory days as the Buffalo Braves in the mid\-1970s\. The team reached the playoffs for the first time in nine seasons and came within one win of reaching the Western Conference final\. The Times reported the offer is close to the 6 million U\.S\. a year Dunleavy’s agent initially proposed and would make him among the highest\-paid coaches in the NBA\. Dunleavy, the NBA’s 1999 coach of the year, currently earns 2\.5 million U\.S\. dollars a season, and his deal expires at the end of this season\. Extracted spans:Nov\. 29, 2006; Times Tuesday; last season; mid\-1970s; nine seasons; the year; season; the end of this season

Figure 20:Example instances from the six dataset settings used in our time expression extraction experiments: base, adversarial, domain, domain\-vocab, compositional, and structural\. In the adversarial example, perturbed tokens are highlighted in red\.
#### A\.4\.3Event Expression extraction

Unlike time expression extraction, the event extraction task focuses on identifying event trigger words that describe actions, occurrences, or state changes in text\. We evaluate the same six settings to analyze how well models generalize under perturbations, domain shifts, vocabulary changes, compositional reasoning, and longer contextual inputs\. Figure[21](https://arxiv.org/html/2610.02549#A1.F21)presents representative examples from each dataset setting, with adversarial perturbations highlighted in red\.

BaseText:The typhoon, which hit the eastern region of Bicol before dawn Thursday, packing maximum winds of 150 kilometers \(93 miles\) per hour, knocked out electrical services, telephone lines and water supply, further hampering rescue and relief efforts\. Extracted spans:hit; packing; knocked; hampering

AdversarialText:The typhoon, which hit the eastern region of Bicol before dawn Thursday, packingmaimumwinds of 150 kilometers \(93 miles\) per hour, knocked out electrical services, telephone lines andkatersupply, further hampering rescue and relief efforts\. Extracted spans:hit; packing; knocked; hampering

Domain\(T2C\)Text:The patient was reportedly alert at scene, and then became unresponsive in route to the hospital\. Extracted spans:alert; unresponsive; the hospital

Domain\-VocabText:The typhoon, which hit the eastern region ofTcofabefore dawn Thursday, packing maximum winds of 150 kilometers \(93 miles\) per hour, knocked out electrical services, telephone lines and water supply, further hampering rescue and relief efforts\. Extracted spans:hit; packing; knocked; hampering

CompositionalText:The longtime overheating then led to a blowout and the cold air went into the T101 aniline processing tower, which was run under negative pressure and triggered the blast at about 1:45 p\.m\., and other blasts happened afterwards\. Extracted time and event spans:1:45 p\.m\.; overheating; led; blowout; went; run; triggered; blast; blasts; happened

StructuralText:XIN\_ENG\_20061129\.0012 WASHINGTON, Nov\. 29, 2006 \(Xinhua\) Clippers, Dunleavy reportedly reach four\-year deal\. The Los Angeles Clippers and coach Mike Dunleavy have reached a four\-year contract extension worth more than 21 million U\.S\. dollars, according to the Los Angeles Times Tuesday\. The team and Dunleavy’s agent have worked out a deal that will pay him more than 5\.3 million U\.S\. doallars a season\. The Clippers have not made the deal public because bonuses tied to future playoff appearances were still being worked out, the Times said\. The Clippers improved dramatically under Dunleavy last season, posting their best win total since the franchise’s glory days as the Buffalo Braves in the mid\-1970s\. The team reached the playoffs for the first time in nine seasons and came within one win of reaching the Western Conference final\. The Times reported the offer is close to the 6 million U\.S\. a year Dunleavy’s agent initially proposed and would make him among the highest\-paid coaches in the NBA\. Dunleavy, the NBA’s 1999 coach of the year, currently earns 2\.5 million U\.S\. dollars a season, and his deal expires at the end of this season\. Extracted spans:reached; according; worked; deal; pay; made; tied; worked; said; improved; posting; reached; came; reaching; reported; offer; proposed; make; earns; deal; expires

Figure 21:Example instances from the six dataset settings used in our event extraction experiments: base, adversarial, domain, domain\-vocab, compositional, and structural\. In the adversarial example, perturbed tokens are highlighted in red\.

### A\.5Detailed Results

This section presents the complete experimental results across all extraction tasks, model families, reasoning strategies, and generalization settings\. We include detailed results in Tables[2](https://arxiv.org/html/2610.02549#A1.T2)–[10](https://arxiv.org/html/2610.02549#A1.T10)for both time expression extraction and event extraction using dense and MoE models, along with compositional extraction experiments that jointly evaluate temporal and event understanding\. The reported scores provide a fine\-grained view of model behavior under domain shift, adversarial perturbations, vocabulary variation, and longer contextual inputs\. We also analyze the effect of model scale to understand how performance changes with increasing parameter size across different evaluation settings\. In addition, we report results for fine\-tuned models to compare task\-specific adaptation against prompting\-based reasoning strategies\. These detailed tables complement the main analysis by exposing performance consistency, robustness trends, and setting\-specific weaknesses that are less visible from aggregated summaries alone\.

Table 2:Time expression extraction results for MoE models under different reasoning strategies\. Scores are reported using micro F1\.Table 3:Event expression extraction results for MoE models under different reasoning strategies\. Scores are reported using micro F1\.Table 4:Time expression extraction results for dense models under different reasoning strategies\. Scores are reported using micro F1\.Table 5:Event expression extraction results for dense models under different reasoning strategies\. Scores are reported using micro F1\.Table 6:Event expression extraction results for model size analysis\. Scores are reported using micro F1\.Table 7:Time expression extraction results for model size analysis\. Scores are reported using micro F1\.
Table 8:Compositional extraction results for MoE models \(time \+ event\)\. Scores are reported using micro F1\.
Table 9:Compositional extraction results for dense models \(time \+ event\)\. Scores are reported using micro F1\.

Table 10:Performance of fine\-tuned models on time and event expression extraction across multiple generalization dimensions\. Scores are reported using micro F1\.