Time as Structure: Temporal Dependency Graphs for Verifiable Deadline Computation over Legal Documents

arXiv cs.CL Papers

Summary

The paper introduces temporal dependency graphs extracted from legal documents to compute filing deadlines, showing a pipeline approach outperforms direct language model answering in accuracy.

arXiv:2608.15270v1 Announce Type: new Abstract: Miss a filing deadline by one day and the claim is barred, however strong the case. Computing that deadline is rarely simple: the period runs from a triggering event, is counted by a statutory convention, and may be suspended by a mandatory conciliation window. We ask whether a language model should answer such questions directly, or read the document and leave the arithmetic to code. We extract dated facts and their dependencies into a temporal dependency graph and compute deadlines from it with a calendar-correct engine. On UK Employment Appeal Tribunal judgments the engine reproduces six of seven timeliness rulings, and matches the judges' own dates to the day. The strongest of four language models, asked the same cases, gets the arithmetic right and the answer wrong: in six of twenty-one responses its stated verdict contradicts its own thinking, and every contradiction runs the same way, calling a late claim timely. To test the systems at scale we move the dismissal date across the statutory boundary, generating 427 cases whose answers are computed rather than annotated. On the cases both systems answer, the pipeline is right 90.2% of the time against 61.2% for direct answering. The limit is extraction: on contracts the errors are almost never in the arithmetic, but in choosing which event the period starts from.
Original Article
View Cached Full Text

Cached at: 08/18/26, 10:03 AM

# Temporal Dependency Graphs for VerifiableDeadline Computation over Legal Documents
Source: [https://arxiv.org/html/2608.15270](https://arxiv.org/html/2608.15270)
## Time as Structure: Temporal Dependency Graphs for Verifiable Deadline Computation over Legal Documents

Lifeng HanSuzan VerberneAffiliation:Leiden Institute of Advanced Computer Science \(LIACS\), Leiden UniversityEmail:[mzhirko@gmail\.com](mailto:)

###### Abstract

Miss a filing deadline by one day and the claim is barred, however strong the case\. Computing that deadline is rarely simple: the period runs from a triggering event, is counted by a statutory convention, and may be suspended by a mandatory conciliation window\. We ask whether a language model should answer such questions directly, or read the document and leave the arithmetic to code\. We extract dated facts and their dependencies into a temporal dependency graph and compute deadlines from it with a calendar\-correct engine\. On UK Employment Appeal Tribunal judgments the engine reproduces six of seven timeliness rulings, and matches the judges’ own dates to the day\. The strongest of four language models, asked the same cases, gets the arithmetic right and the answer wrong: in six of twenty\-one responses its stated verdict contradicts its own thinking, and every contradiction runs the same way, calling a late claim timely\. To test the systems at scale we move the dismissal date across the statutory boundary, generating 427 cases whose answers are computed rather than annotated\. On the cases both systems answer, the pipeline is right 90\.2% of the time against 61\.2% for direct answering\. The limit is extraction: on contracts the errors are almost never in the arithmetic, but in choosing which event the period starts from\.

††This paper is based on the author’s MSc thesis at Leiden University\. Code, rule packs and evaluation harnesses are released; see the Artifact availability section\.## 1Introduction

Legal documents contain networks of dependent dates\. A filing deadline may be defined as three months beginning with an effective date, extended by an early\-conciliation interval, and compared against a presentation date recorded elsewhere in the file\. Getting the chain wrong is not recoverable, because a limitation period is a jurisdictional boundary and a claim presented one day late is barred whatever its merits\. In the UK First\-tier Tribunal \(Immigration and Asylum Chamber\), the share of disposals recorded as invalid or out of time rose from 3% in 2023/24 to 9% in the quarter to March 2026[7](https://arxiv.org/html/2608.15270#bib.bib1)\.

Established extraction systems normalise each temporal expression on its own[11](https://arxiv.org/html/2608.15270#bib.bib2)and do not record which date is derived from which, so a correction to one date does not propagate\. Giving the whole document to a language model asks it to identify the governing event, interpret the rule, perform the arithmetic and report the result in a single pass\. Language models order legal events reasonably well while nested legal language remains a bottleneck[1](https://arxiv.org/html/2608.15270#bib.bib3), and in formal routes they can report a conclusion without executing the reasoning that supports it[12](https://arxiv.org/html/2608.15270#bib.bib4)\. An aggregate error rate is also not enough on its own\. An audit of frontier models over a contract\-review corpus finds that two systems can be wrong equally often while failing in opposite directions, and proposes a separate measure for the direction of error[14](https://arxiv.org/html/2608.15270#bib.bib5)\. The two directions are not equally dangerous here\. A tool that reports a deadline as passed when it has not invites a check, and one that reports time remaining when none is left does not\.

We investigate a division of labour in which extraction and computation are separate components\. The intermediate representation is aTemporal Dependency Graph \(TDG\), a directed graph whose nodes are dated facts and whose edges are typed temporal dependencies\. Either a rule\-based pipeline or a language model builds the graph\. A deterministic engine then applies calendar arithmetic, the statutory counting convention and theAdvisory, Conciliation and Arbitration Service \(ACAS\)conciliation pause, and returns the anchor it selected, the rule it applied, the intermediate dates and a confidence score\. When the engine cannot determine which extracted fact the statute’s anchor concept refers to, it abstains and reports the cause\.

We ask two questions\. First, whether explicit structure improves temporal reasoning compared with direct answering, and how the difference behaves as dependency chains lengthen\. Second, whether current extraction recovers enough structure from legal prose to support the computation\. Five evaluations address these: a controlled 210\-item benchmark, an audited extraction comparison over 41 contracts, an end\-to\-end evaluation on recent UK employment cases, a 427\-item counterfactual sweep with computed ground truth, and a consistency measure that uses no ground truth\. Catala verification, contradiction detection and a transfer test to pairwise ordering are in the appendix\.

Our contributions are the following\.

- •A deterministic engine over an explicit temporal graph that reproduces six of seven tribunal timeliness rulings and all three judge\-stated dates, with the statutory rules recovered from statute text\.
- •A four\-model baseline on the same cases with working\-level auditing, which identifies a failure the value\-level metrics do not detect: in six of twenty\-one GPT\-family responses the emitted verdict contradicts the working in the same response, in all six cases reporting a late claim as timely\.
- •A counterfactual harness in the style of GSM\-Symbolic[8](https://arxiv.org/html/2608.15270#bib.bib12)that perturbs anchor dates across the statutory boundary and recomputes ground truth through the engine, giving 427 items whose labels no text\-only system can recover from memorisation\.
- •An exhaustive per\-edge audit of both extractors, which locates the bottleneck in anchor selection and not in arithmetic\.
- •A consistency measure that consumes no ground truth, applied to three models in two conditions, showing that removing self\-contradiction does not improve verdict accuracy\.
- •An open\-source release of the engine, the graph representation, the statutory rule packs and the evaluation harnesses\.

## 2Related Work

Temporal information extraction identifies and normalises dates, durations and intervals\. HeidelTime[11](https://arxiv.org/html/2608.15270#bib.bib2)is a widely used rule\-based system of this kind\. It normalises expressions largely in isolation and does not record which event is derived from which\. LexTime[1](https://arxiv.org/html/2608.15270#bib.bib3)shows that language models order legal events reasonably well and that nested legal language remains a bottleneck\. The task studied here extends beyond pairwise ordering to anchor identification, statutory rule application, calendar arithmetic and a deadline that can be compared against a court’s own figure\.

The work also relates to legal formalisation\. Catala[6](https://arxiv.org/html/2608.15270#bib.bib6)is a programming language whose structure mirrors statutory text, and Monat et al\.[9](https://arxiv.org/html/2608.15270#bib.bib7)give a mechanised semantics for legal date arithmetic together with an analysis that detects counting ambiguities\. Recent studies examine language model translation of law into Catala\-like representations and document faithfulness failures in that route[5](https://arxiv.org/html/2608.15270#bib.bib8);[12](https://arxiv.org/html/2608.15270#bib.bib4)\. Those studies evaluate generated code by similarity to a reference translation\. We evaluate by execution, comparing the computed deadline against the one the court used\.

Statutory reasoning as a task is established by SARA[3](https://arxiv.org/html/2608.15270#bib.bib9), outcome prediction over the court studied here by CLC\-UKET[13](https://arxiv.org/html/2608.15270#bib.bib10), and statute\-guided numerical computation by LexNum[15](https://arxiv.org/html/2608.15270#bib.bib11)\. Our perturbation design follows GSM\-Symbolic[8](https://arxiv.org/html/2608.15270#bib.bib12)and replaces its templated answers with an oracle computed from the statute\.

## 3Data and Task

#### Contracts\.

We sample 50 seeds from theen\_contractssubset of Multi\_Legal\_Pile[10](https://arxiv.org/html/2608.15270#bib.bib13)\. This returns 45 documents, of which 41 are unique under exact hashing of the source text\. The duplicates are one document appearing three times and two appearing twice\. All extraction figures below are computed over the 41 unique documents\.

#### Statutes and cases\.

The deadline task uses point\-in\-time versions of the Employment Rights Act 1996 s\. 111, the Equality Act 2010 s\. 123, and the correspondingACASextension provisions, retrieved from legislation\.gov\.uk\. Test cases are sixEmployment Appeal Tribunal \(EAT\)judgments decided in 2025 and 2026\. One judgment is evaluated under both statutes, giving seven statute\-case rows\. All six postdate the training cutoff of every model tested\.

Table[1](https://arxiv.org/html/2608.15270#S3.T1)maps each case to its neutral citation\. Party names are personal data and are not printed here, but a neutral citation contains no names and retrieves the judgment on Find Case Law, so every value computed below remains checkable against the identified public judgment\. The mnemonics are used throughout the paper\.

CitationStatute\(s\)Mnemonic\[2026\] EAT 64ERA s\.111; EqA s\.123two statutes\[2025\] EAT 155ERA s\.111termination date\[2026\] EAT 14ERA s\.111internal appeal\[2026\] EAT 76EqA s\.123long delay\[2026\] EAT 46EqA s\.123continuing act∗\[2026\] EAT 59ERA s\.111document\-bound∗anonymised by the tribunal itself\.Table 1:The sixEATjudgments in the gold set\. The first is scored under both statutes, giving seven statute\-case rows\.
#### Gold annotation\.

Each row records the statutory anchor, the claim presentation date, theACAScertificate dates where an extension applies, the tribunal verdict, and the deadline or boundary date stated by the judge where the judgment states one\. Two of the seven rows carry a judge\-stated deadline and one carries a judge\-stated boundary date\. Every field is linked to a verbatim quotation and byte\-checked against the source text by script\.

#### Leakage control\.

For direct evaluation we remove every sentence stating the deadline, the day count or the timeliness conclusion, and a leak check confirms that no deadline value survives\. One leak cannot be removed: a party appeals what it lost, so the direction of the appeal can imply the first\-instance verdict\. We therefore treat deadline exactness and working\-level consistency as more reliable than verdict accuracy alone, and we report where deference to the appeal direction affected a result\.

## 4Method

Figure[1](https://arxiv.org/html/2608.15270#S4.F1)shows the components and how they connect\. Extraction and computation are separate, so a failure can be attributed to the layer that produced it\.

deterministic engineTribunal judgmentredacted or full textStatute sectionpoint\-in\-time textLLM extractorsingle prompt, JSON facts \+ relationsRule\-based extractorHeidelTime \+ spaCy \+ graph buildereither extractorCase TDGdated facts,typed edgesRule specificationperiod, anchor concept,counting conventionstatute passes through the same extractor,convention confirmed once by hand1\. Anchor matcherbind statutory concept to a case fact2\. Calendar arithmeticanchor\+\+period−\-1 day,real months and leap years3\. Conciliation pausesuspend Day A to Day B,one\-month floorDeadline and verdictwith anchor, rule, arithmeticand confidenceAbstentionnamed cause,confidence 0\.00anchor bindsno candidate, orcandidates conflictDirect baselinemodel reads both inputs, answers in one passDeadline and verdictno abstention availablecomparison condition

Figure 1:System architecture\. Either extractor produces a case TDG, and the statute section passes through the same pipeline to yield the rule specification\. The deterministic engine binds the statutory anchor concept to a case fact, applies calendar arithmetic and the conciliation pause, and returns either a deadline with its full working or an abstention with a named cause\. The dashed path is the direct baseline, which receives the same two inputs and answers in one pass\.### 4\.1Temporal dependency graph

Nodes in aTDGstore an entity, a semantic role in \{START,END,DURATION\}, and a date normalised to ISO 8601\. Edges are typed\. Anadditiveedge carries adelta\_daysoffset and states that the target date is computed from the source date by that offset\. Anorderingedge states sequence with no offset\. The type distinction determines what the engine can execute: only additive edges with a resolved source date support arithmetic\. Appendix[B](https://arxiv.org/html/2608.15270#A2)gives the schema and a worked example graph\.

### 4\.2Extraction

The rule\-based pipeline runs four steps\. HeidelTime extracts and normalises temporal expressions\. A role classifier walks the spaCy[4](https://arxiv.org/html/2608.15270#bib.bib15)dependency tree from the date token to the head verb of its clause and matches lemmas, so thatbegan,beginningandstartsall map toSTART, with a special case for legal genitives such asDecision of 18 March 1992\. An entity linker attaches dates to entities using named entity recognition and coreference resolution\. The graph builder collectsSTART–ENDpairs sharing an entity, compares the calendar gap against every stated duration in the document, and types the edge additive when a duration matches and ordering otherwise\.

HeidelTime resolves incomplete expressions such asby 31 Marchagainst a document creation time, which this corpus does not supply\. A fixed global value places such dates outside the document’s own period, and in one 1984 instrument produced a date in 2020\. We therefore derive a per\-document creation time from the document’s earliest explicit date, ignoring bare years because citation years are not document dates\. This resolves 43 of the 45 documents\.

TheLarge Language Model \(LLM\)pipeline replaces all four steps with a single prompt\. The model receives the raw document and returns JSON containing facts and relations\. The prompt supplies trigger phrases for additive dependencies \(within X days of,no later than X days from\), a worked mapping fromwithin 30 days of the effective dateto an additive edge withdelta\_days: 30, and counterexamples preventing monetary amounts and percentages from being typed as durations\. Post\-processing removes self\-loops, downgrades additive edges whose source sentences contain no trigger phrase, and downgrades edges whose target already carries a fixed calendar date\.

### 4\.3Deterministic deadline engine

The engine takes a statute graph and a case graph\. A matcher binds the statutory anchor concept to a fact in the case graph\. The engine then computes the deadline using real month lengths and leap years\.

For periods wordedbeginning withan anchor, the UK convention makes a three\-month period end three calendar months after the anchor, less one day\. The engine implements this as anchor\+\+3 months−\-1 day, with month\-end rounding handled explicitly\. Sections 207B ERA and 140B EqA suspend the clock between Day A, the date of contact withACAS, and Day B, the date the certificate is issued, and guarantee at least one month after Day B\. The engine implements the suspension, the one\-month floor, the re\-anchoring rule, and the precondition that Day A falls within the primary period\. All conventions are covered by unit tests\. Figure[2](https://arxiv.org/html/2608.15270#A1.F2)in Appendix[A](https://arxiv.org/html/2608.15270#A1)shows these quantities on one computation\.

The rules are not hard\-coded per case\. The statute section is passed through the same extraction pipeline as any other document, which returns the period and the anchor concept it counts from\. Only the counting convention is confirmed by hand once per statute and recorded in a declarative specification\.

When the matcher finds no candidate, or several conflicting candidates, the engine returnsINDETERMINATEwith a machine\-readable cause\. We count these abstentions separately from wrong answers throughout\.

### 4\.4Baselines and evaluation

The direct baseline supplies each model with the redacted judgment, the governing statute section and the conciliation rule, and asks for a deadline, a verdict and the arithmetic\. This is more context than a retrieval system would supply, so the baseline upper\-bounds retrieval\-based approaches on this task\. We testgemma4:e4blocally andgpt\-5\.4\-nano,gpt\-5\.4\-miniandgpt\-5\.4through the API\. Prompts are frozen across models, temperature is 0, and all raw responses are archived\. Reported numbers are produced by script and not counted by hand\.

Controlled items are scored by calendar value and not by surface form\. Real cases are scored by verdict, by exact deadline where the judgment states one, and by agreement between the working shown and the fields emitted\.

## 5Experiments and Results

### 5\.1Controlled reasoning benchmark

The controlled benchmark separates reasoning from extraction\. It contains 210 generated items in seven categories of 30, with anchors drawn from a stress list of 31 January, 29 and 30 January, 29 February 2024, and month ends including 31 August and 30 November\. Deadline items give an anchor, a period and a margin from\{−7,−4,−3,−1,1,3,4,7\}\\\{\-7,\-4,\-3,\-1,1,3,4,7\\\}days\. Cascade items give a root date and a chain of two or three offsets, then replace the root, so the system must propagate the correction\. Ground truth is computed from the calendar at generation time\.

Table 2:Percentage of items fully correct in the controlled benchmark\. A cascade item is correct only when every linked date is correct\.The structured method is exact in every category, and direct accuracy falls as the period becomes less explicit and the chain lengthens \(Table[2](https://arxiv.org/html/2608.15270#S5.T2)\)\. Gemma propagates corrections in most cases, with per\-link accuracy of 97%, 96% and 93% across the three cascade steps\. Llama restates the original dates instead of applying the correction and falls from 11% at the first link to 3% at the third\. The calendar\-naive baseline reaches 41% overall and still exceeds llama’s 20%, which indicates that applying crude arithmetic to the correct anchor outperforms fluent output that does not propagate\.

### 5\.2Extraction comparison

We ran both extractors over the same 41 contracts and audited every edge the engine could execute\. A computable edge is additive, has a resolved source date, and carries an offset\. Ordering edges claim no offset by definition, and interval edges in this corpus connect nodes with no date\.

Rule\-basedLLMFacts extracted592288resolving a date48695usable nodes18850Dependency edges1192additive933ordering240interval019Documents with an edge7/4130/41Computable edges910offset matches duration9/912/13surviving audit13Table 3:Extraction over 41 unique contracts\. A usable node has a resolved calendar date, is not a bare citation year, and carries a nameable entity\. Every computable edge on both sides was audited against its source sentences\. We report precision on produced edges\. Recall would require dependency\-level annotation the corpus does not carry\.The rule\-based pipeline resolves 486 dates against theLLM’s 95 and produces 11 edges against 92, with edges on 7 documents against 30 \(Table[3](https://arxiv.org/html/2608.15270#S5.T3)\)\. Normalising individual expressions accurately therefore does not by itself recover dependency structure\. TheLLM’s advantage also narrows once the set is restricted to edges the engine can execute\. Of its 92 edges, 31 have a resolved date at neither endpoint, so a relation is recovered but no arithmetic can be performed along it\.

The audit identifies what the surviving edges have in common\. One of the nine rule\-based edges survives, a single\-sentence clause readingextended for a period of three months from 15 March 1986\. Three of the tenLLMedges survive, and all three contain an explicit trigger phrase naming the anchor concept in the same sentence, for examplewithin one month after the entry into force of this Agreement\. A fourth carries a real offset attached to the wrong event, counting twenty\-one days from the date of a letter instead of from the entry into force the clause names\.

Offsets are internally consistent in almost every case: 9 of 9 rule\-based offsets match their normalised duration, and 12 of 13 for theLLM\. The single exception disagrees with itself twice, readingthree monthsin its expression, carrying a ninety\-day duration on its endpoint, and recording thirty in its offset field\. The remaining failures are anchor selection and entity identity\. Over half the rule\-based entities are unusable, most of them clause fragments, and the graph builder pairs a start with an end on a shared entity, so a noun phrase lifted out of context leaves nothing to pair\. Both extractors make the same error on one document, anchoringthree years after the entry into forceto a different date stated nearby, which suggests anchor selection is a property of the task and not of either implementation\.

### 5\.3Deadline computation on tribunal cases

We first test the engine on human\-verified facts, which isolates it from extraction\. It reproduces six of seven tribunal verdicts, both judge\-stated deadlines, and the judge\-stated boundary date in the continuing\-act case\. The seventh row states an in\-time conclusion without reciting the conciliation dates that produced it\. We call such rows document\-bound\. The engine computes the primary deadline for that row exactly and reports the missing input\.

Table 4:DirectLLMbaseline on redacted judgments, with mean absolute deadline error in days\.†The engine is scored on the two judge\-stated deadlines and additionally reproduces the one judge\-stated boundary date\.Deadline exactness improves with model scale and verdict accuracy does not \(Table[4](https://arxiv.org/html/2608.15270#S5.T4)\)\. The strongest model applies the minus\-one\-day convention correctly in all seven responses and produces three of six computable deadlines exactly\. In two rows its emitted verdict field contradicts its own correct working, and across the GPT family six of 21 responses contain this divergence, all in the direction where the working concludes out of time and the field says in time\. The local model shows none\. The smaller models fail the convention itself, applying it in 0 of 7, 0 of 7 and 1 of 7 responses, and none below the frontier applies the s\. 207B pause correctly anywhere\.

One alternative reading of the divergence is available\. The verdict field precedes the arithmetic in the emitted JSON, so this data does not distinguish a model that commits early and rationalises afterwards from one that computes and then misreports\. The conclusion holds under both readings: the emitted field is not usable without the working\.

### 5\.4Full unattended pipeline

We ran the full chain with no human in the loop, using two extractors over redacted and full\-text judgments with the same engine, thresholds and fallback in every condition\.

Extraction recall is high\. The strongest condition finds the gold anchor in six of seven rows, all seven presentation dates, all four availableACASDay A dates and all three Day B dates\. The engine nevertheless abstains often, because the matcher does not align a statutory concept such aseffective date of terminationwith an extracted label such asdismissal\. The bottleneck therefore moves from extraction to semantic binding\.

The matcher uses five statute\-grounded mechanisms to make this binding: anchor aliases derived from the statutes’ own vocabulary, a stoplist of procedural events, claim\-presentation cues with an earliest\-date tie\-break, a conflict gate that surfaces disagreeing anchor candidates, and an alias tier so that weak lexical similarity cannot outrank statutory vocabulary\. An ablation over the same 28 graphs, with no re\-extraction and no new model calls, shows that these mechanisms raise correct bindings from 6 of 10 to 12 of 14 across the affected tables\.

Across the 28 cells the engine answers 14 and abstains on 14\. Every abstention carries confidence 0\.00 and a named cause, and no deadline is fabricated in any cell\. Twelve of the 14 answers are correct verdicts, and both errors are the same document\-bound row\. Verdict agreement alone would overstate this: 8 of the 12 correct verdicts also reproduce the day count, and 4 are correct by margin on a mis\-selected action or anchor\. Across the 28 direct baseline responses there are no abstentions and every failure is confident\.

Two observations follow\. Anchor\-selection errors move between layers: in one condition the extractor labelled the claimant’s rejected contention as the dismissal date, which is the same error the direct baselines made, and the engine computed faithfully on that label\. The matcher’s confidence on that binding was 0\.24, so the calibration signal fired where the label did not\. And one anchor is hard for both extractors: in one case both return the grievance filed the day after the incident, because the incident date appears only inside the tribunal’s ruling, which the redacted condition removes\. Anchor identification can therefore require the legal conclusion itself\.

### 5\.5Counterfactual boundary tracking

The gold set supports a taxonomy of failures and not error rates\. To obtain computed ground truth at scale we perturb the anchor date of each real judgment and recompute the label through the engine\.

For each gold case we sweep the anchor acrossk∈\[b−30,b\+30\]k\\in\[b\-30,b\+30\]days, wherebbis the engine\-computed boundary offset for that case, so every sweep straddles its own statutory boundary\. This gives7×61=4277\\times 61=427items balanced 31/30 timely to late within each case\. A self\-check verifies that every sweep is monotone with exactly one verdict flip at the boundary the statute prescribes\.

Rewriting only the anchor date leaves its satellites in place\. A judgment that elsewhere sayshis summary dismissal in July 2020, or dates a dismissal letter the day before, becomes self\-contradictory about when the anchor occurred\. The final harness therefore shifts every date expression falling before the first frozen event, which is the first conciliation date or the presentation date where no conciliation applies, so that the anchor and its narrative move together while the litigation timeline stays fixed\. Bare year tokens are not rewritten, because citation years such as\[2018\] EATmake that unsafe, and are counted and flagged per item\. A residual\-reference detector scans every emitted text and generation enforces a zero\-residual gate\.

The engine generates the labels and is not scored against them\. This experiment measures text\-reading systems against statute\-computed truth\.

Table 5:427 perturbed items, majority class 0\.508\. A system tracking the statute flips its verdict exactly once per sweep\. Accuracy is over answered items and coverage differs between conditions, so the matched comparison in the text is the fair one\.Both direct models answer every item and sit above the majority class of 0\.508 by 14 and 9 points \(Table[5](https://arxiv.org/html/2608.15270#S5.T5)\)\. The pipeline answers fewer items and reaches 90\.2% on those it answers, and 98\.3% on the subset where the extracted anchor equals the true shifted anchor \(n=234n=234\)\. On the items both gemma conditions answered, the pipeline reaches 90\.2% against 61\.2% direct \(n=286n=286, disagreements 93 to 10, McNemarp=5\.2×10−18p=5\.2\\times 10^\{\-18\}\)\. For llama the figures are 82\.2% against 54\.6% \(n=174n=174, 62 to 14,p=2\.3×10−8p=2\.3\\times 10^\{\-8\}\)\. Direct accuracy is lower on the items the pipeline abstained from than on those it answered, so abstention concentrates on the harder items\.

Over all 427 items the pipeline does not exceed direct answering: gemma direct reaches 0\.646 against the pipeline’s 0\.604, because coverage is bounded by extraction recall\. The result is the asymmetry\. Where the anchor binds the pipeline is near\-exact, where it does not the pipeline abstains with a named cause, and on rows where the extracted anchor is wrong the pipeline sits at chance \(0\.538\)\. Flip counts show the same pattern structurally: the direct models flip 68 and 117 times across seven sweeps, against 24 and 18 for the pipeline and 7 for a system tracking the statute\.

### 5\.6Consistency without ground truth

Every measure above consumes an answer known in advance, which is what limits the evaluation to seven cases\. We therefore add a measure that consumes none\.

Six questions are put to a system about one case: the presentation datepp, the effective deadlinedd, the last dateℓ\\ellon which the claim could still have been in time, the verdictvv, the number of days lateδ\\delta, and whether presentation complied\. Four constraints hold between the answers by arithmetic or definition, independently of the case:ℓ=d\\ell=d\(K1\),δ=p−d\\delta=p\-d\(K2\),\(v=in\_time\)⇔\(δ≤0\)\(v=\\textit\{in\\\_time\}\)\\iff\(\\delta\\leq 0\)\(K3\), andcomplied⇔\(v=in\_time\)\\textit\{complied\}\\iff\(v=\\textit\{in\\\_time\}\)\(K5\)\. A system violating any of these has contradicted itself whether or not we know which answer is correct\. A fifth relation is entailed by K1 to K3 and is reported only as a redundancy check\. K5 asks the same question as K3 with a different answer type, so a violation confined to K5 indicates sensitivity to phrasing\. Each question is asked in its own call with no shared context, and because that choice is open to argument we also run a single\-prompt condition identical in every other respect\. Appendix[D](https://arxiv.org/html/2608.15270#A4)gives the constraint\-level results\.

Table 6:Consistency on the seven gold rows\. Coverage \(Cov\.\) counts cases where all six answers parsed and incoherence is scored over those\. Verdicts are scored against the tribunal’s ruling over the rows whose verdict field parsed, which is a larger set, so the two columns are not nested\.Asked the same case six times, gpt\-5\.4 contradicts itself on four of seven at full coverage \(Table[6](https://arxiv.org/html/2608.15270#S5.T6)\)\. Answering in one request removes self\-contradiction for both capable models, and verdict accuracy against the tribunal’s ruling falls in both, from five of seven to four of seven for gpt\-5\.4 and from four of six to two of seven for gemma\. The engine is not scored here\. Its six fields are projections of one computation, so it cannot violate a constraint, and we treat that as a property of the architecture\.

The violations locate the failure\. For gpt\-5\.4, K3 and K5 are 0 of 7 and all four violations are K1 and K2, so the verdict never disagrees with its own day count while the dates and the subtraction do\. This differs from the divergence in Section[5\.3](https://arxiv.org/html/2608.15270#S5.SS3), where the reported answer was unfaithful to correct working\. Here the working is unstable across askings and the reporting is exact\. For llama in the single\-prompt condition, K2 is 6 of 6 and K5 is 0 of 6: with both operands present in its own output it performed the subtraction wrongly every time while never contradicting its own conclusion\.

## 6Analysis

#### Structure and arithmetic\.

The controlled benchmark, the tribunal cases and the counterfactual sweep agree on the direction of the effect\. Where a date must be carried through dependent events, an explicit graph with an executed computation is exact by construction, and direct accuracy falls as the chain lengthens\. The effect is largest where the arithmetic is hardest, which is the cascade categories and the perturbed items near the boundary\.

#### Working\-level auditing\.

Verdict accuracy and deadline exactness each miss a failure the other detects\. Verdict\-level scoring passes a wrong computation when the margin is large enough, which occurs in four of the twelve correct pipeline verdicts\. Value\-level scoring passes a right number produced by two errors that cancel, which we observe once\. Only comparing the working against the emitted fields detects the divergence class, and that class is present in the strongest model tested and absent in the weakest\.

#### Consistency and correctness\.

Removing self\-contradiction did not improve accuracy in either model where it took effect, and verdict accuracy fell in both, so self\-consistency is not usable as a confidence signal\. What the deterministic layer provides is not higher accuracy but a located cause: when the verdict is the last line of a computation, a wrong verdict is attributable to an anchor, a rule or an input\.

#### Anchor selection\.

Continuing acts, rejected factual contentions and adjacent procedural events produce competing plausible anchors\. The direct models select the wrong one, the extractors label the wrong one, and in the continuing\-act case the appellate court described the tribunal’s own scoping as clearly open to it, which makes the contrary view arguable\. Better extraction does not resolve a case where the anchor depends on a legal conclusion\. The conflict gate presents the candidates and their computations instead of selecting silently\.

#### Boundary of the claim\.

On pairwise event ordering[1](https://arxiv.org/html/2608.15270#bib.bib3)the structured route loses\. Coverage falls to 18% because most event pairs are never connected by an extracted relation, and direct answering wins outright\. Ordering has no arithmetic to verify, so the method pays the extraction cost and gains nothing\. The same benchmark reproduces the consistency result on public data: it contains 190 pairs asked in both directions of which exactly one direction can be true, and the direct models answer both affirmatively on 32% and 52% of pairs against 0% for the structured route\. Appendix[G](https://arxiv.org/html/2608.15270#A7)reports both runs\.

## 7Conclusion

We evaluated a structure\-then\-compute approach to legal temporal reasoning across five settings\. On generated reasoning items the method is exact while direct accuracy falls with chain length\. On six tribunal judgments the engine reproduces six of seven verdicts and all three judge\-stated dates using rules recovered from statute text\. On 427 perturbed items it reaches 90\.2% where it binds an anchor and abstains with a named cause where it does not\. The extraction audit and the consistency measure locate the remaining problem in anchor selection and show that self\-consistency does not indicate correctness\.

The open problem is not extracting more dates but binding statutory anchor concepts to the correct factual events with traceable provenance\. Where that binding depends on a legal conclusion, we expect the correct system behaviour is to present the candidates and their consequences\.

## Acronyms

ACASAdvisory, Conciliation and Arbitration ServiceEATEmployment Appeal TribunalLLMLarge Language ModelTDGTemporal Dependency Graph

## Artifact availability

The engine, the graph representation, the statutory rule packs and the evaluation harnesses are released as an installable command\-line tool with an inspection interface, Timebar,111[https://github\.com/mzhirko/timebar](https://github.com/mzhirko/timebar)released under Apache\-2\.0\. Each deadline reported here can be recomputed from a case file and a rule pack, and the tool prints the anchor, the rule, the arithmetic and the confidence for every computation\. The research artifacts, including the graph representation, the statutory rule packs and the evaluation harnesses, are in a separate repository\.222[https://github\.com/mzhirko/legal\-temporal\-dependency\-graphs](https://github.com/mzhirko/legal-temporal-dependency-graphs)Neither repository contains tribunal judgments\. Case bundles ship in theTDGformat’s hash\-only mode: each document carries character offsets and asource\_text\_sha256digest in place of its text, so every quoted span stays byte\-verifiable against the published original without our redistributing it\.

## Limitations

The legal gold set has seven rows, which supports a taxonomy of failure mechanisms and not error rates, so we report counts instead of percentages\. The counterfactual sweep raises the item count to 427 with computed ground truth, but its statistical unit remains seven cases\. The statutes come from one jurisdiction and one procedural domain\. The controlled benchmark is synthetic, with prose explicit enough that extraction is trivial, so it isolates the reasoning component by design\. Published time\-limit judgments are also not a random sample of time\-limit disputes, because a case turning on the point is more likely to be reported when the claim was late, so the in\-time direction is represented by one real case and by the sweep\.

The engine, its counting conventions and the rule specifications were fixed before the tribunal evaluation\. Validation of the matcher on held\-out cases beyond the seven studied here is still required\.

On extraction we report precision on produced edges and no recall, because the contracts carry no dependency\-level annotation\. Audit verdicts are judgements by a single assessor, applying a scoring rule fixed before the audit, and are under independent review\.LLMextraction varies between runs at temperature 0, so all counts come from a single named run\. Only explicit temporal expressions are detected\.

Coverage in the consistency measure is not missing at random\. All parse failures fall on three cases, two of which carry the anchors that are hardest elsewhere in our results, so accuracy over answered items is computed on an easier subset\. We exclude llama from the single\-prompt comparison, because its two conditions differ by a lost case and a changed generation task as well as by shared context\.

## Ethical considerations

The system computes statutory deadlines and does not give legal advice\. Its outputs are intended to be checked against the source documents it cites, and the abstention behaviour is deliberate: where selecting the anchor is a legal judgement rather than a reading, the tool presents the candidates instead of choosing\. All case documents are published judgments retrieved from public sources\. Party names are personal data, and although the judgments are public we do not reprint the names, republish the judgment texts, or release them with the artifact\. What we release instead are the neutral citations \(Table[1](https://arxiv.org/html/2608.15270#S3.T1)\), which contain no names and retrieve each judgment from the official service, so every computed value stays checkable without our redistributing anything\. One case was anonymised by the tribunal itself and we preserve that order\.

## Acknowledgements

This work began as the first author’s MSc thesis at Leiden University, supervised by the second and third authors\.

## References

- Baraleet al\.\(2025\)C\. Barale, L\. Barrett, V\. S\. Bajaj, and M\. RovatsosLexTime: a benchmark for temporal ordering of legal events\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 5220–5236\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.280/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.280),ISBN 979\-8\-89176\-335\-7Cited by:[Appendix G](https://arxiv.org/html/2608.15270#A7.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2608.15270#S1.p2.1),[§2](https://arxiv.org/html/2608.15270#S2.p1.1),[§6](https://arxiv.org/html/2608.15270#S6.SS0.SSS0.Px5.p1.1)\.
- Douet al\.\(2026\)G\. Dou, L\. Brena, A\. Deo, W\. Jurayj, J\. Zhang, N\. Holzenberger, and B\. V\. DurmeDeonticBench: a benchmark for reasoning over rules\.External Links:2604\.04443,[Link](https://arxiv.org/abs/2604.04443)Cited by:[Appendix G](https://arxiv.org/html/2608.15270#A7.SS0.SSS0.Px1.p1.1)\.
- Holzenbergeret al\.\(2020\)N\. Holzenberger, A\. Blair\-Stanek, and B\. Van DurmeA dataset for statutory reasoning in tax law entailment and question answering\.InProceedings of the 3rd Workshop on Natural Legal Language Processing \(NLLP\),San Diego, CA, USA\.External Links:[Link](https://arxiv.org/abs/2005.05257)Cited by:[Appendix G](https://arxiv.org/html/2608.15270#A7.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2608.15270#S2.p3.1)\.
- Honnibalet al\.\(2020\)M\. Honnibal, I\. Montani, S\. Van Landeghem, and A\. BoydspaCy: industrial\-strength natural language processing in python\.External Links:[Document](https://dx.doi.org/10.5281/zenodo.1212303),[Link](https://spacy.io/)Cited by:[§4\.2](https://arxiv.org/html/2608.15270#S4.SS2.p1.1)\.
- Lorenzoet al\.\(2025\)G\. Lorenzo, A\. Pietromatera, and N\. HolzenbergerTranslating tax law to code with LLMs: a benchmark and evaluation framework\.InProceedings of the Natural Legal Language Processing Workshop 2025,N\. Aletras, I\. Chalkidis, L\. Barrett, C\. Goanță, D\. Preoțiuc\-Pietro, and G\. Spanakis \(Eds\.\),Suzhou, China,pp\. 31–47\.External Links:[Link](https://aclanthology.org/2025.nllp-1.4/),[Document](https://dx.doi.org/10.18653/v1/2025.nllp-1.4),ISBN 979\-8\-89176\-338\-8Cited by:[§2](https://arxiv.org/html/2608.15270#S2.p2.1)\.
- Merigouxet al\.\(2021\)D\. Merigoux, N\. Chataing, and J\. ProtzenkoCatala: a programming language for the law\.Proceedings of the ACM on Programming Languages5\(ICFP\),pp\. 1–29\.Cited by:[Appendix E](https://arxiv.org/html/2608.15270#A5.p1.1),[§2](https://arxiv.org/html/2608.15270#S2.p2.1)\.
- Ministry of Justice \(2026\)Ministry of JusticeTribunals statistics quarterly: january to march 2026\.Statistical reportUK Government\.External Links:[Link](https://www.gov.uk/government/statistics/tribunals-statistics-quarterly-january-to-march-2026/tribunal-statistics-quarterly-january-to-march-2026)Cited by:[§1](https://arxiv.org/html/2608.15270#S1.p1.1)\.
- Mirzadehet al\.\(2024\)I\. Mirzadeh, K\. Alizadeh, H\. Shahrokhi, O\. Tuzel, S\. Bengio, and M\. FarajtabarGSM\-symbolic: understanding the limitations of mathematical reasoning in large language models\.External Links:2410\.05229,[Link](https://arxiv.org/abs/2410.05229)Cited by:[3rd item](https://arxiv.org/html/2608.15270#S1.I1.i3.p1.1),[§2](https://arxiv.org/html/2608.15270#S2.p3.1)\.
- Monatet al\.\(2024\)R\. Monat, A\. Fromherz, and D\. MerigouxFormalizing date arithmetic and statically detecting ambiguities for the law\.InProgramming Languages and Systems: 33rd European Symposium on Programming, ESOP 2024, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2024, Luxembourg City, Luxembourg, April 6–11, 2024, Proceedings, Part II,Berlin, Heidelberg,pp\. 421–450\.External Links:ISBN 978\-3\-031\-57266\-1,[Link](https://doi.org/10.1007/978-3-031-57267-8_16),[Document](https://dx.doi.org/10.1007/978-3-031-57267-8%5F16)Cited by:[§2](https://arxiv.org/html/2608.15270#S2.p2.1)\.
- Niklauset al\.\(2024\)J\. Niklaus, V\. Matoshi, M\. Stürmer, I\. Chalkidis, and D\. HoMultiLegalPile: a 689GB multilingual legal corpus\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15077–15094\.External Links:[Link](https://aclanthology.org/2024.acl-long.805/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.805)Cited by:[§3](https://arxiv.org/html/2608.15270#S3.SS0.SSS0.Px1.p1.1)\.
- Strötgen and Gertz \(2010\)J\. Strötgen and M\. GertzHeidelTime: high quality rule\-based extraction and normalization of temporal expressions\.InProceedings of the 5th International Workshop on Semantic Evaluation,K\. Erk and C\. Strapparava \(Eds\.\),Uppsala, Sweden,pp\. 321–324\.External Links:[Link](https://aclanthology.org/S10-1071/)Cited by:[§1](https://arxiv.org/html/2608.15270#S1.p2.1),[§2](https://arxiv.org/html/2608.15270#S2.p1.1)\.
- Wanget al\.\(2026\)O\. P\. Wang, S\. Wong\-Toropainen, D\. Amrollahi, R\. Bai, T\. Bansal, A\. Garg, and L\. H\. GilpinKnow your limits : on the faithfulness of llms as solvers and autoformalizers in legal reasoning\.External Links:2606\.16118,[Link](https://arxiv.org/abs/2606.16118)Cited by:[§1](https://arxiv.org/html/2608.15270#S1.p2.1),[§2](https://arxiv.org/html/2608.15270#S2.p2.1)\.
- Xieet al\.\(2024\)H\. Xie, F\. Steffek, J\. De Faria, C\. Carter, and J\. RutherfordThe CLC\-UKET dataset: benchmarking case outcome prediction for the UK employment tribunal\.InProceedings of the Natural Legal Language Processing Workshop 2024,N\. Aletras, I\. Chalkidis, L\. Barrett, C\. Goanță, D\. Preoțiuc\-Pietro, and G\. Spanakis \(Eds\.\),Miami, FL, USA,pp\. 81–96\.External Links:[Link](https://aclanthology.org/2024.nllp-1.7/),[Document](https://dx.doi.org/10.18653/v1/2024.nllp-1.7)Cited by:[§2](https://arxiv.org/html/2608.15270#S2.p3.1)\.
- Yadav and Gurugubelli \(2026\)L\. Yadav and A\. GurugubelliLegalHalluLens: typed hallucination failure modes and calibrated multi\-agent debate mitigation\.InWorkshop on Failure Modes of Agentic AI at ICML 2026,External Links:[Link](https://openreview.net/forum?id=BbXsMvN52C)Cited by:[§1](https://arxiv.org/html/2608.15270#S1.p2.1)\.
- Zhanget al\.\(2025\)K\. Zhang, G\. Xie, W\. Yu, M\. Xu, X\. Tang, Y\. Li, and J\. XuLegal mathematical reasoning with LLMs: procedural alignment through two\-stage reinforcement learning\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 1586–1598\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.84/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.84),ISBN 979\-8\-89176\-335\-7Cited by:[§2](https://arxiv.org/html/2608.15270#S2.p3.1)\.

## Appendix AEngine mechanics on one computation

Figure[2](https://arxiv.org/html/2608.15270#A1.F2)shows every quantity the engine reports for a case in which the conciliation pause applies\. The primary period runs from the anchor under thebeginning withconvention\. The clock stops on Day A and restarts on Day B, so the deadline moves forward by the length of the pause\. The claimant additionally has one month after Day B whatever remained, and the pause applies only where Day A falls inside the primary period\.

With a primary deadline of 10 October, Day A on 1 September and Day B on 1 October, the clock was paused for 30 days and the effective deadline moves to 9 November\. Had the pause ended with only two weeks left, the one\-month floor would give more time than the pause did, and the deadline would fall one month after Day B instead\. The engine reports the anchor it bound, the rule pack and provision it applied, the primary deadline, the pause length, which of the two branches governed, the effective deadline, and the match confidence\.

anchor11 JulDay A1 SepDay B1 Octprimary10 Octeffective9 Novprimary limitation periodclock pauseddeadline moves by the length of the pauseone month after Day B,whichever is later

Figure 2:The early\-conciliation pause of s\. 207B ERA 1996, with every quantity the engine reports\. The label, the authority and the floor come from the rule pack; the engine implements only the shape\.The counting convention itself changes the answer by a day, and a day decides the case\. Three monthsbeginning with8 June ends on 7 September, because the anchor day counts\. Three monthsfrom8 June ends on 8 September, because counting starts the next day\. The statute’s own wording fixes which applies\. The engine never infers it: it is read once per statute and recorded in the rule pack\.

## Appendix BGraph schema and rule packs

ATDGis serialised as JSON with three top\-level parts: document metadata, a list of facts, and a list of dependencies\. Each fact carries an identifier, an entity string, a role in \{START,END,DURATION,CONTAINS\}, a normalised value, the sentence it was read from with character offsets, and an extractor confidence\. Each dependency carries a source and target identifier, aconstraint\_typein \{additive,ordering,interval\}, and, for additive edges, adelta\_daysoffset\.

A statute is data in the same format\. The rule pack for ERA 1996 s\. 111 is a graph of two facts and one dependency: aSTARTfact whose entity is the anchor concepteffective date of terminationand whose value is empty, anENDfact whose value is the ISO 8601 durationP3M, and an additive dependency between them\. The counting convention and the human\-readable authority are recorded alongside\. The engine loads this file and knows nothing about the statute otherwise, so adding a jurisdiction is adding a file, not changing code\.

## Appendix CThe released tool

The engine ships as a command\-line tool with a browser inspection interface\.buildtakes a folder of documents and produces one graph per document plus a merged timeline; the computation commands then call no model at all, so a deadline can be recomputed offline from a case file and a rule pack\.

Figure[3](https://arxiv.org/html/2608.15270#A3.F3)shows the merged timeline for a three\-document bundle\. Every row carries the documents it was read from, the values each of them gave, an extractor confidence, and a provenance status:agreedwhere two or more documents give a compatible value,disputedwhere they cannot both be true,single sourcewhere only one document speaks to it, andderivedwhere the value was computed rather than read\. The disputed row in the figure is the anchor itself, where a dismissal letter and the claim form give effective dates two days apart\. That disagreement is exactly the input on which the deadline turns, and the tool surfaces it rather than silently preferring one of the two\.

![Refer to caption](https://arxiv.org/html/2608.15270v1/viewer-timeline.png)Figure 3:The inspection interface, showing the merged timeline for a three\-document bundle\. Each row records its source documents, the competing values where they disagree, and a confidence\. Nothing in the display is produced by a model at read time: the values are computed from the stored graphs\.
## Appendix DConstraint\-level consistency results

Table[6](https://arxiv.org/html/2608.15270#S5.T6)in the main text reports how often each condition contradicts itself\. The constraints locate where\. For gpt\-5\.4 in the separate\-call condition, K3 \(verdict against day count\) and K5 \(compliance against verdict\) are violated in 0 of 7 cases, and all four incoherent cases violate K1 \(the last in\-time date against the deadline\) or K2 \(days late against presentation minus deadline\)\. The verdict therefore never disagrees with the model’s own day count; the dates and the subtraction do\. For llama3\.1:8b in the single\-prompt condition the pattern inverts: K2 is violated in 6 of 6 cases and K5 in 0 of 6, so with both operands present in its own output the model performed the subtraction wrongly every time while never contradicting its own conclusion\.

## Appendix EFormal cross\-check with Catala

As a second, independent formalisation of the same documents we generate Catala[6](https://arxiv.org/html/2608.15270#bib.bib6)programs from the contract texts and compare the values they compute against the values theTDGroute computes\. Of the 45 sampled contracts that reach the comparison stage, 40 of the generated programs compile and execute; two fail the repair loop and three fail at interpretation\.

The two formalisations rarely describe the same fields\. Across the 45 contracts, 116 fields appear only in the graph and 23 only in the Catala output, and only 19 documents yield a field both sides describe\. Of those 19, 16 yield a field where both sides give a value of the same kind\. Over that shared subset the comparator records 14 agreements, 6 value mismatches, and 5 type mismatches where one side gives a date and the other a duration, so there is nothing to compare\.

The small overlap is the finding\. The two routes are complementary views rather than redundant ones, so agreement over the shared part is a check and not a headline number\. The six value mismatches are the useful cells: in one of them the cross\-check caught theTDGside misreading a duration introduced by scanning errors in the source document\.

## Appendix FContradiction detection across documents

The detector compares theTDGsof two documents and flags an obligation asserted in both with incompatible values\. The task rests on one decision: when two extracted facts describe the same obligation\. We implement that decision two ways, holding everything else fixed\. The*lexical*setting compares tokens weighted by inverse document frequency; the*embedding*setting replaces that one comparison withnomic\-embed\-text\.

On documents that genuinely agree, two tribunal judgments applying the same s\. 111 rule and each judgment paired with the statute it cites, the detector returns zero contradictions on all three pairs and identifies three parallel applications of the sameP3Mperiod\.

\#ProbeLexicalEmbedder1paraphrase \(termination/dismissal\)missok2granularity \(employment term\.\)okok3role mismatch \(END vs CONTAINS\)okok4month vs day precisionokok5duration vs datemissmiss6three\-way disagreementokok7same date, different eventsokok8two competing candidatesokok9separate matters, declaredokok10separate matters, not declaredmissmiss11undated facts onlymissmiss12same entity, eight months apartokokTotal8/129/12

Table 7:Twelve probe bundles with known correct outcomes, run under both similarity settings\. The embedder isnomic\-embed\-textat a threshold of 0\.60; the lexical setting is token overlap at 0\.50\.On the probe suite the embedder scores 9 of 12 and the lexical setting 8 of 12 \(Table[7](https://arxiv.org/html/2608.15270#A6.T7)\)\. The difference is the case the embedder exists for:terminationanddismissalshare no tokens, so their lexical similarity is 0\.00 against 0\.71 embedded\. Every case the lexical setting gets right the embedder also gets right\. The three cases both fail are not about similarity at all: two documents from separate matters that do not say so cannot be distinguished, facts with no dates carry nothing to compare, and a duration against a date is a question about meaning\.

At corpus scale the two diverge sharply\. Over the 45 contract graphs \(325 facts\) the lexical configuration returns 1183 candidate links \(784 coreference, 232 contradiction, 167 structural analogy\) and the embedding configuration returns 11674 \(9587, 1920, and the same 167\)\. The structural link type does not use the similarity function, and its count is identical under both settings, which makes the comparison clean: the embedder multiplies candidate links roughly tenfold and adds no new structural matches\. A threshold tuned for one setting is therefore useless for the other\. The discriminating statistic is the margin between the lowest true pair and the highest false pair:\+0\.06\+0\.06fornomic\-embed\-text, against−0\.22\-0\.22for a general\-purpose chat model pressed into service as an embedder, where wrong pairs outscored right ones outright\.

## Appendix GExternal benchmarks

#### Statutory computation without structure\.

Table[8](https://arxiv.org/html/2608.15270#A7.T8)reports the two local models on SARA[3](https://arxiv.org/html/2608.15270#bib.bib9)and on the airline\-fee, housing and USCIS tasks of DeonticBench[2](https://arxiv.org/html/2608.15270#bib.bib14)\. Exact match was pinned as the metric for the numeric tasks before any run\. On those tasks both models are at or near zero\. On the classification tasks they sit close to the majority class in both directions, and the two cells above majority are ahead by 6\.7 and 3\.6 points on 30 and 28 items\.

Table 8:Local models on public statutory benchmarks, instrumented runs with zero infrastructure errors in every reported cell\. Coverage is the fraction of items answered; classification accuracy is over all items; exact match is answered\-and\-exact over items attempted\.
#### A task the method does not win\.

LexTime[1](https://arxiv.org/html/2608.15270#bib.bib3)poses pairwise ordering questions over 514 instances\. Answering directly, both models beat the majority class comfortably \(73\.9% and 62\.3% against 50\.4%\)\. Through theTDGroute, coverage collapses to 18% and 28% and accuracy over all items falls below majority \(13\.0% and 21\.0%\), because most event pairs are never connected by an extracted relation\. This is the expected result: ordering two events is a reading task with no arithmetic in it, so the method pays the cost of building a graph and gets nothing back\. The claim we make is about computing dates that depend on other dates, and LexTime marks where it stops\.

The same benchmark reproduces the consistency finding on public data\. It contains 190 pairs asked in both directions, of which exactly one direction can be true\. The direct models answer both directions affirmatively on 32% and 52% of those pairs\. The structured route does so on 0%, because a single extracted relation projects to both answers\.

Similar Articles