A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learning

# A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learning
Source: [https://arxiv.org/html/2608.03015](https://arxiv.org/html/2608.03015)
Jiajun Bao1\\NoHyper\\endNoHyper, Zihao Qi1, Toni J\.B\. Liu1, Gurbir Arora1 Raphaël Sarfati1,2, Nicolas Boullé3, Christopher J\. Earls1 1Cornell University2Goodfire AI3Imperial College London

###### Abstract

Pretrained large language models \(LLMs\) have demonstrated in\-context learning \(ICL\) capabilities for numerical inference over sequences serialized as text\. Prior work has identified and characterized this form of numerical inference primarily through output\-level evaluations such as prediction error\. However, how numerical information is organized within LLM representations remains much less understood\. To study this internal organization, we adopt a graph signal processing perspective in which attention induces a weighted graph over tokens, while token hidden states define signals on its nodes\. Quantitative graph\-spectral diagnostics and qualitative token\-graph visualizations reveal that representations become more clearly differentiated by input dynamical complexity as context length increases\. Simpler inputs produce attention\-induced token graphs with stronger global connectivity and smoother, spectrally concentrated hidden\-state signals, whereas more complex inputs produce more localized graphs and hidden\-state signals with broader spectral support and greater high\-frequency energy\. Together, these findings point to systematic, context\-dependent internal signatures associated with numerical ICL that are conserved across model families\.

A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In\-Context Learning

Jiajun Bao1\\NoHyper††thanks:Correspondence author:jb2777@cornell\.edu\\endNoHyper, Zihao Qi1, Toni J\.B\. Liu1, Gurbir Arora1Raphaël Sarfati1,2, Nicolas Boullé3, Christopher J\. Earls11Cornell University2Goodfire AI3Imperial College London

![Refer to caption](https://arxiv.org/html/2608.03015v1/x1.png)Figure 1:Overview of the graph signal processing framework for studying LLM internal representations during numerical in\-context learning\.\(a\)Numerical trajectories are quantized and serialized as comma\-delimited sequences of three\-digit numbers\.\(b\)At each layerℓ\\ell, head\-aggregated attention defines a weighted, undirected token graphW¯\(ℓ\)\\bar\{W\}^\{\(\\ell\)\}over theNNtoken nodes, hidden statesX\(ℓ\)X^\{\(\\ell\)\}definedd\-dimensional node signals, and Laplacian eigenvectors form the graph Fourier basis, enabling spectral analysis of these signals\.\(c\)Layerwise graph\-spectral diagnostics and attention\-induced token\-graph visualizations characterize these representations\. Node positions reflect attention topology, node colors encode hidden\-state signals, and circles and triangles denote numeric and separator tokens, respectively\. The examples are from the final layer of Llama\-3\.2\-3B atN=500N=500\.## 1Introduction

Large language models \(LLMs\) are increasingly studied for numerical inference over sequences serialized as text\. Without task\-specific parameter updates, they can infer and extrapolate numerical processes, a capability we refer to as numerical in\-context learning \(ICL\)\. Prior work has demonstrated this capability in time\-series forecasting\(Gruveret al\.,[2023](https://arxiv.org/html/2608.03015#bib.bib1); Jinet al\.,[2024](https://arxiv.org/html/2608.03015#bib.bib7)\), numerical regression\(Vacareanuet al\.,[2024](https://arxiv.org/html/2608.03015#bib.bib2); Requeimaet al\.,[2024](https://arxiv.org/html/2608.03015#bib.bib14)\), dynamical\-system inference\(Liuet al\.,[2024](https://arxiv.org/html/2608.03015#bib.bib13); Zhang and Gilpin,[2025](https://arxiv.org/html/2608.03015#bib.bib6)\), density estimation\(Liuet al\.,[2025](https://arxiv.org/html/2608.03015#bib.bib15)\), Markov\-model prediction\(Daiet al\.,[2025](https://arxiv.org/html/2608.03015#bib.bib16); Zekriet al\.,[2024](https://arxiv.org/html/2608.03015#bib.bib17)\), and partial differential equation extrapolation\(Baoet al\.,[2026](https://arxiv.org/html/2608.03015#bib.bib18)\)\. Depending on the task, longer contexts often yield more accurate point predictions, predictive distributions closer to the ground\-truth distribution, or more faithful recovery of latent dynamics\. Yet most existing studies characterize these improvements through model outputs, treating the model as a black box and leaving it unclear how its internal representations change as context length increases\.

Recent studies have begun to examine internal representations associated with numerical ICL, providing evidence that LLM hidden states systematically encode numerical information\. In particular,Sarfatiet al\.\([2026](https://arxiv.org/html/2608.03015#bib.bib11)\)identify curved manifolds encoding beliefs about Gaussian parameters, whilePiskorzet al\.\([2026](https://arxiv.org/html/2608.03015#bib.bib12)\)use trained probes to decode point predictions and uncertainty from LLM hidden states\. Motivated by these findings, we jointly analyze token\-level hidden representations and the structure of attention\-induced token graphs\. Specifically, we ask how their organization varies with context length and input dynamical complexity\.

To study this interplay, we adopt the graph signal processing perspective summarized in[Fig\.˜1](https://arxiv.org/html/2608.03015#S0.F1): at each transformer layer, attention induces a weighted graph over tokens, while the corresponding token hidden states define signals on its nodes\. Our framework supports two complementary analyses: 1\) graph\-spectral diagnostics, which quantify global attention\-graph connectivity, high\-frequency energy in hidden\-state signals, and the breadth of their spectral support; 2\) attention\-induced token\-graph visualizations, which provide a qualitative view of token\-level organization consistent with the quantitative trends\. Across both analyses, we find that, as context length increases, the model’s internal representations become more clearly ordered by input dynamical complexity\. The diagnostics further distinguish input families whose output\-level extrapolation errors remain tightly clustered, suggesting that representation\-level analysis can reveal structure not apparent from extrapolation error alone \(cf\.[Fig\.˜5](https://arxiv.org/html/2608.03015#A1.F5)in[Appendix˜A](https://arxiv.org/html/2608.03015#A1)\)\.

## 2Background and Related Work

#### Graph spectra and graph signals\.

Spectral graph theory studies graph structure through operators such as the Laplacian, whose spectrum captures connectivity, cuts, and global organization\(Fiedler,[1973](https://arxiv.org/html/2608.03015#bib.bib23); Chung,[1997](https://arxiv.org/html/2608.03015#bib.bib24)\)\. Graph signal processing extends this spectral viewpoint to data defined on graph nodes: for undirected graphs, Laplacian eigenvectors form a graph Fourier basis whose modes are ordered by how rapidly nodewise signals vary over the graph\(Hammondet al\.,[2011](https://arxiv.org/html/2608.03015#bib.bib29); Shumanet al\.,[2013](https://arxiv.org/html/2608.03015#bib.bib30); Ortegaet al\.,[2018](https://arxiv.org/html/2608.03015#bib.bib31)\)\. This classical framework relates the variation of graph\-supported signals to the geometry of the underlying graph\.

#### Graph\-spectral analysis of LLMs\.

Recent graph\-spectral work on LLMs has focused on hallucination detection and reasoning verification\.Binkowskiet al\.\([2025](https://arxiv.org/html/2608.03015#bib.bib19)\)use attention\-derived Laplacian spectra for hallucination detection\. Closer to our setting,Noël \([2025](https://arxiv.org/html/2608.03015#bib.bib20),[2026](https://arxiv.org/html/2608.03015#bib.bib21)\)treats attention\-induced token graphs together with token\-level hidden\-state signals, using diagnostics such as spectral entropy, high\-frequency energy ratio, and the Fiedler value for hallucination analysis and proof verification\. At the output level,Balaji \([2026](https://arxiv.org/html/2608.03015#bib.bib22)\)constructs sentence\-level graph signals from generated chain\-of\-thought traces, enabling verification without access to internal attention weights or hidden states\. We adopt closely related diagnostics for quantitative, representation\-level analysis of numerical ICL rather than for hallucination detection or reasoning verification\.

#### Applications of graph constructions in LLMs\.

Several recent works represent LLM reasoning traces or token\-level attention patterns as graphs, including sentence\-level chain\-of\-thought dependency graphs\(Bogdanet al\.,[2025](https://arxiv.org/html/2608.03015#bib.bib32)\), attention\-induced token graphs for reinforcement\-learning credit assignment\(Donget al\.,[2026](https://arxiv.org/html/2608.03015#bib.bib33)\), and attributed attention graphs used with graph neural networks for hallucination detection\(Frascaet al\.,[2026](https://arxiv.org/html/2608.03015#bib.bib39)\)\. In contrast, we use training\-free, layerwise visualizations of attention\-induced token graphs to display attention topology alongside low\-dimensional hidden\-state structure\.

Collectively, these studies leave open how the organization of LLM internal representations during numerical ICL varies with context length and input dynamical complexity\. We investigate this question through complementary quantitative and qualitative analyses within a graph signal processing framework\.

## 3Methods

### 3\.1Numerical Inputs and Tokenization

#### Controlled numerical input suite\.

Our main experiments use ten one\-dimensional sequence families spanning different levels of dynamical complexity under a fixed prompt format: a sampled constant baseline, logistic map trajectories\(May,[1976](https://arxiv.org/html/2608.03015#bib.bib35)\), and trajectories from thexx\-component of the Lorenz system\(Lorenz,[1963](https://arxiv.org/html/2608.03015#bib.bib37)\)\. The logistic map,xt\+1=r​xt​\(1−xt\)x\_\{t\+1\}=rx\_\{t\}\(1\-x\_\{t\}\), wherext∈\[0,1\]x\_\{t\}\\in\[0,1\]andr∈\[0,4\]r\\in\[0,4\]is a control parameter, serves as our primary controlled family\. Our selected values ofrrspan periodic regimes with periods2,4,8,16,32,2,4,8,16,32,and6464, followed by two chaotic regimes along the classical period\-doubling route\(Strogatz,[2024](https://arxiv.org/html/2608.03015#bib.bib36)\)\. The logistic map therefore provides a controlled setting for isolating how input dynamics affect LLM representations: varyingrrproduces trajectories with progressively longer periods and then chaotic dynamics while preserving the recurrence form\.

Throughout, we use*dynamical complexity*operationally, with increasing complexity corresponding to the progression from constant inputs through periodic inputs of increasing period to chaotic inputs\. This ordering is also supported quantitatively by a standard entropy\-based characterization of the input dynamics, as detailed in[Appendix˜B](https://arxiv.org/html/2608.03015#A2)\. The constant sequence provides a low\-complexity baseline, whereas the Lorenz trajectory provides a complementary observation from a continuous\-time chaotic system\. We evaluate an ensemble of 20 trajectory realizations per family\.[Appendix˜A](https://arxiv.org/html/2608.03015#A1)provides full input\-generation details\. To broaden the input coverage beyond the controlled dynamical\-system suite, we additionally consider structured numerical sequences with distributional shifts, smooth periodic variation, and stochastic evolution, as described in[Appendix˜I](https://arxiv.org/html/2608.03015#A9)\.

#### Quantization and serialization\.

Following prior work\(e\.g\., Gruveret al\.,[2023](https://arxiv.org/html/2608.03015#bib.bib1); Liuet al\.,[2024](https://arxiv.org/html/2608.03015#bib.bib13)\), we encode each numerical input sequence as a quantized, comma\-delimited text prompt\. For a trajectory\(x1,…,xT\)\(x\_\{1\},\\ldots,x\_\{T\}\), scalar values are linearly rescaled to the interval\[50,950\]\[50,950\]and rounded to obtain quantized integersqi∈\{50,…,950\}q\_\{i\}\\in\\\{50,\\ldots,950\\\}\. Eachqiq\_\{i\}is formatted as a zero\-padded three\-digit decimal string\. The encoded values are separated by commas, yielding strings such as “087,173,642,\.\.\.”\. To ensure comparability across context lengths, we quantize the full trajectory before truncation so that the numerical mapping remains fixed\. We choose this format because widely used tokenizers, such as those used by GPT\-4\(OpenAI,[2023](https://arxiv.org/html/2608.03015#bib.bib3)\)and Llama\-3\-family models\(Grattafioriet al\.,[2024](https://arxiv.org/html/2608.03015#bib.bib4)\), encode three\-digit integer strings \(000–999\) and commas as single tokens\. Under this serialization,TTnumerical states yieldTTnumeric andTTseparator tokens, so the analyzed context length isN=2​TN=2T\. This avoids splitting numerical states across tokens and preserves one\-to\-one state\-to\-token alignment for graph analysis\. Further quantization and serialization details are provided in[Appendix˜A](https://arxiv.org/html/2608.03015#A1)\.

### 3\.2Graph\-Spectral Diagnostics

We adapt three established spectral diagnostics employed byNoël \([2026](https://arxiv.org/html/2608.03015#bib.bib21)\)to numerical ICL: the normalized Fiedler value, high\-frequency energy ratio, and spectral entropy \(all defined below\)\. In our setting, these diagnostics characterize how global attention\-graph connectivity, hidden\-state roughness, and spectral diversity vary with input dynamical complexity and context length\.

#### Layerwise representations\.

For anNN\-token input, we record two objects at each layer\. First, for each layerℓ∈\{1,…,L\}\\ell\\in\\\{1,\\ldots,L\\\}and attention headh∈\{1,…,H\}h\\in\\\{1,\\ldots,H\\\}, letA\(ℓ,h\)∈ℝN×NA^\{\(\\ell,h\)\}\\in\\mathbb\{R\}^\{N\\times N\}denote the post\-softmax attention matrix, whereAi​j\(ℓ,h\)A^\{\(\\ell,h\)\}\_\{ij\}is the attention weight from query tokeniito key tokenjj\.111When present, we exclude non\-content special tokens introduced during model input preprocessing, such as<BOS\>, and renormalize each attention row over the remaining tokens\.Second, letX\(ℓ\)∈ℝN×dX^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{N\\times d\}denote the hidden\-state matrix entering layerℓ\\ell, whoseii\-th rowXi\(ℓ\)X\_\{i\}^\{\(\\ell\)\}is thedd\-dimensional representation of analyzed tokenii\. We pair each layer’s attention matrices with its input hidden states because those states generate the queries and keys underlying the attention weights\.

#### Graph construction and Fourier modes\.

To support standard Laplacian\-based graph\-spectral analysis, we construct one undirected weighted graph per layer by representing each analyzed token as a node, symmetrizing each head’s attention matrix, and averaging the symmetrized matrices across heads \(see Appendix[Fig\.˜10](https://arxiv.org/html/2608.03015#A5.F10)for representative final\-layer attention matrices\):

W¯\(ℓ\)=12​H​∑h=1H\(A\(ℓ,h\)\+\(A\(ℓ,h\)\)⊤\)\.\\vskip\-3\.0pt\\bar\{W\}^\{\(\\ell\)\}=\\frac\{1\}\{2H\}\\sum\_\{h=1\}^\{H\}\\left\(A^\{\(\\ell,h\)\}\+\(A^\{\(\\ell,h\)\}\)^\{\\top\}\\right\)\.By construction,W¯i​j\(ℓ\)\\bar\{W\}^\{\(\\ell\)\}\_\{ij\}is the attention\-derived edge weight between tokensiiandjj\. We define the corresponding degree matrix asD\(ℓ\)=diag⁡\(W¯\(ℓ\)​𝟏\)D^\{\(\\ell\)\}=\\operatorname\{diag\}\\\!\\left\(\\bar\{W\}^\{\(\\ell\)\}\\mathbf\{1\}\\right\)and the combinatorial graph Laplacian asL\(ℓ\)=D\(ℓ\)−W¯\(ℓ\)L^\{\(\\ell\)\}=D^\{\(\\ell\)\}\-\\bar\{W\}^\{\(\\ell\)\}\. The resultingL\(ℓ\)L^\{\(\\ell\)\}is symmetric positive semidefinite and therefore admits an orthonormal eigendecompositionL\(ℓ\)=U\(ℓ\)​Λ\(ℓ\)​\(U\(ℓ\)\)⊤L^\{\(\\ell\)\}=U^\{\(\\ell\)\}\\Lambda^\{\(\\ell\)\}\(U^\{\(\\ell\)\}\)^\{\\top\}\. The eigenvectors inU\(ℓ\)U^\{\(\\ell\)\}define graph Fourier modes ordered from low to high graph frequency\(Shumanet al\.,[2013](https://arxiv.org/html/2608.03015#bib.bib30)\)\. Treating the hidden\-state matrixX\(ℓ\)X^\{\(\\ell\)\}as a vector\-valued graph signal whose value at nodeiiisXi\(ℓ\)X\_\{i\}^\{\(\\ell\)\}, we project it onto the graph Fourier basis asX^\(ℓ\)=\(U\(ℓ\)\)⊤​X\(ℓ\)\\widehat\{X\}^\{\(\\ell\)\}=\(U^\{\(\\ell\)\}\)^\{\\top\}X^\{\(\\ell\)\}\. Low\-eigenvalue modes correspond to smoother hidden\-state variation over the attention\-induced graph, whereas high\-eigenvalue modes correspond to sharper variation across token pairs joined by larger attention weights, as follows from the Laplacian quadratic\-form identity in[Appendix˜C](https://arxiv.org/html/2608.03015#A3)\.

#### Diagnostic 1: Normalized Fiedler value\.

The Fiedler value—the second\-smallest eigenvalue of the combinatorial graph LaplacianL\(ℓ\)L^\{\(\\ell\)\}—is a classical measure of global connectivity\(Fiedler,[1973](https://arxiv.org/html/2608.03015#bib.bib23)\)\. To facilitate comparisons across graph sizes and degree scales, we use the standard normalized formulation\(Chung,[1997](https://arxiv.org/html/2608.03015#bib.bib24)\):

ℒ\(ℓ\)=\(D\(ℓ\)\)−1/2​L\(ℓ\)​\(D\(ℓ\)\)−1/2\.\\mathcal\{L\}^\{\(\\ell\)\}=\(D^\{\(\\ell\)\}\)^\{\-1/2\}L^\{\(\\ell\)\}\(D^\{\(\\ell\)\}\)^\{\-1/2\}\.The spectrum ofℒ\(ℓ\)\\mathcal\{L\}^\{\(\\ell\)\}lies in\[0,2\]\[0,2\]\. Its second\-smallest eigenvalue, denoted byλ2\(ℓ\)\\lambda\_\{2\}^\{\(\\ell\)\}, is the normalized Fiedler value\. Largerλ2\(ℓ\)\\lambda\_\{2\}^\{\(\\ell\)\}indicates a more globally integrated attention graph, whereas smallerλ2\(ℓ\)\\lambda\_\{2\}^\{\(\\ell\)\}indicates more localized graph structure\.[Section˜D\.1](https://arxiv.org/html/2608.03015#A4.SS1)examines additional low\-end eigenvalues to characterize multiway localized graph structure beyondλ2\(ℓ\)\\lambda\_\{2\}^\{\(\\ell\)\}\(Leeet al\.,[2014](https://arxiv.org/html/2608.03015#bib.bib40)\)\.

#### Diagnostic 2: High\-frequency energy ratio \(HFER\)\.

Let𝐱^m\(ℓ\)∈ℝd\\widehat\{\\mathbf\{x\}\}^\{\(\\ell\)\}\_\{m\}\\in\\mathbb\{R\}^\{d\}denote the graph Fourier coefficient vector for modemm, given by themmth row ofX^\(ℓ\)\\widehat\{X\}^\{\(\\ell\)\}, andpm\(ℓ\)=‖𝐱^m\(ℓ\)‖22/∑r=1N‖𝐱^r\(ℓ\)‖22p\_\{m\}^\{\(\\ell\)\}=\\\|\\widehat\{\\mathbf\{x\}\}^\{\(\\ell\)\}\_\{m\}\\\|\_\{2\}^\{2\}/\\sum\_\{r=1\}^\{N\}\\\|\\widehat\{\\mathbf\{x\}\}^\{\(\\ell\)\}\_\{r\}\\\|\_\{2\}^\{2\}the normalized modal energy\. The HFER is the fraction of hidden\-state energy assigned to high graph\-frequency modes,

HFER\(ℓ\)=∑m=K\+1Npm\(ℓ\)\.\\mathrm\{HFER\}^\{\(\\ell\)\}=\\sum\_\{m=K\+1\}^\{N\}p\_\{m\}^\{\(\\ell\)\}\.We useK=⌊N/2⌋K=\\lfloor N/2\\rflooras the default cutoff, so HFER measures the fraction of energy in the high\-frequency half of the spectrum\.[Section˜D\.1](https://arxiv.org/html/2608.03015#A4.SS1)shows that the qualitative behavior remains consistent across alternative cutoffs\. Larger HFER indicates that a greater fraction of hidden\-state energy lies in modes associated with sharper variation over the attention\-induced graph\.

#### Diagnostic 3: Spectral entropy \(SE\)\.

Whereas HFER measures the fraction of hidden\-state energy in high\-frequency modes, SE measures how broadly that energy is distributed across graph Fourier modes\. Using the normalized modal energiespm\(ℓ\)p\_\{m\}^\{\(\\ell\)\}, we define

SE\(ℓ\)=−∑m=1Npm\(ℓ\)​log⁡pm\(ℓ\)\.\\mathrm\{SE\}^\{\(\\ell\)\}=\-\\sum\_\{m=1\}^\{N\}p\_\{m\}^\{\(\\ell\)\}\\log p\_\{m\}^\{\(\\ell\)\}\.We reportexp⁡\(SE\(ℓ\)\)\\exp\(\\mathrm\{SE\}^\{\(\\ell\)\}\)as the effective spectral support, following the standard effective\-number interpretation of exponentiated Shannon entropy\(Jost,[2006](https://arxiv.org/html/2608.03015#bib.bib25)\)\. If energy is distributed equally across exactlykkmodes, thenexp⁡\(SE\(ℓ\)\)=k\\exp\(\\mathrm\{SE\}^\{\(\\ell\)\}\)=k\. More generally, it represents the number of equally weighted graph Fourier modes that would yield the observed spectral entropy\. Larger values indicate broader spectral support, whereas smaller values indicate concentration in fewer modes\.

#### Layer averaging\.

In the main text, we report diagnostics averaged across model layers\. The corresponding layer\-resolved results in[Section˜D\.2](https://arxiv.org/html/2608.03015#A4.SS2)broadly reproduce the same qualitative trends, supporting layer averaging as a concise and representative summary across model depth\. We useλ2\\lambda\_\{2\},HFER\\mathrm\{HFER\}, andexp⁡\(SE\)\\exp\(\\mathrm\{SE\}\)to denote the resulting layer\-averaged diagnostics\.

### 3\.3Token\-Graph Visualization Pipeline

We complement the graph\-spectral diagnostics with layerwise two\-dimensional graph visualizations in which node color summarizes token hidden\-state representations and node position reflects attention topology\. These visualizations provide a qualitative view of the organization characterized quantitatively by our spectral diagnostics\.[Figure˜2](https://arxiv.org/html/2608.03015#S3.F2)illustrates this joint encoding for three representative input families across context lengths\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x2.png)Figure 2:Illustration of attention\-induced token graphs across context lengths\. Rows show final\-layer results from Llama\-3\.2\-3B for representative constant, period\-16 logistic map, and chaotic logistic map \(r=4\.0r=4\.0\) inputs\. The first five columns show token graphs atN∈\{100,200,300,400,500\}N\\in\\\{100,200,300,400,500\\\}\. Node positions reflect input\-specific attention weights through a Fruchterman–Reingold layout, with the same initialization used across inputs at eachNN\. Edge thickness encodes attention\-derived weight\. Only edges above2\.5/N2\.5/Nare displayed to reduce visual clutter\. Node colors encode the leading three uncentered SVD coordinates of token hidden states, with theN=500N=500projections shown in the rightmost column\. Circles mark numeric tokens and triangles mark separator tokens\.#### Node\-color construction for token graphs\.

In classical graph signal processing, a scalar signal is often visualized by mapping its value at each node to color or vertical height\(Shumanet al\.,[2013](https://arxiv.org/html/2608.03015#bib.bib30)\)\. To obtain a color encoding of the vector\-valued token hidden states, we project the hidden\-state matrix at each layer onto its leading three uncentered SVD coordinates and assign them to the red, green, and blue \(RGB\) channels;[Appendix˜E](https://arxiv.org/html/2608.03015#A5)provides the full construction\. AtN=500N=500, the leading three components capture 75% of the hidden\-state energy on average across the main numerical input families and analyzed layers \(Appendix[Fig\.˜12](https://arxiv.org/html/2608.03015#A5.F12)\), supporting their use as a compact representation of the dominant hidden\-state structure\. Corresponding projections for all ten main numerical input families are provided in Appendix[Fig\.˜11](https://arxiv.org/html/2608.03015#A5.F11)\. For other inputs whose hidden\-state energy is distributed across more SVD components, the visualization can be extended with additional color\-coded views\.

#### Attention\-based node positions\.

Visualizing the attention\-induced token graph as a two\-dimensional node\-link diagram requires choosing a layout\. Layouts determined solely by the tokens’ sequential positions, such as the spiral initialization shown in Appendix[Fig\.˜13](https://arxiv.org/html/2608.03015#A5.F13), preserve input order\. However, because attention weights do not influence node placement in these layouts, the resulting geometry may obscure nonlocal interactions and community structure reflected in the LLM’s internal attention organization\. We therefore apply the Fruchterman–Reingold \(FR\) algorithm\(Fruchterman and Reingold,[1991](https://arxiv.org/html/2608.03015#bib.bib34)\)to the full weighted graph defined by the symmetrized attention matrixW¯\(ℓ\)\\bar\{W\}^\{\(\\ell\)\}\. The algorithm iteratively updates the node positions by treating nodes as mutually repelling particles, which discourages node overlap, and edges as attractive forces weighted by the corresponding off\-diagonal entries ofW¯\(ℓ\)\\bar\{W\}^\{\(\\ell\)\}, so larger attention\-derived edge weights exert stronger attraction\. At a fixedNN, all inputs use the same spiral initialization and layout settings; differences in the relaxed node positions therefore reflect their input\-specific attention weights rather than different starting coordinates\. Starting from the token\-ordered spiral, we run 100 iterations, allowing the layout to relax toward a stable configuration, and use the resulting positions for all subsequent token\-graph visualizations\. Appendix[Fig\.˜13](https://arxiv.org/html/2608.03015#A5.F13)shows that most visible layout reorganization occurs by 50 iterations, with only minor changes thereafter\.[Appendix˜E](https://arxiv.org/html/2608.03015#A5)provides implementation details and demonstrates the robustness of the relaxed layouts to initialization perturbations\.

#### Edge display threshold\.

The FR layout is computed using the full weighted attention graph, which containsN​\(N−1\)/2N\(N\-1\)/2off\-diagonal edges\. Rendering every connection would turn the visualization into a dense “hairball” in which meaningful patterns are no longer discernible\. To reduce visual clutter, we display only edges satisfyingW¯i​j\(ℓ\)\>κ/N\\bar\{W\}^\{\(\\ell\)\}\_\{ij\}\>\\kappa/N, usingκ=2\.5\\kappa=2\.5for all main visualizations\. Appendix[Fig\.˜15](https://arxiv.org/html/2608.03015#A5.F15)shows that the qualitative organization is robust to changes in the display threshold\. This threshold affects only edge rendering; the FR node positions and all graph\-spectral diagnostics are computed from the full weighted graph\.

## 4Results

We examine how the model’s internal representations vary with the dynamical complexity of the input across increasing context lengths\.[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1)quantifies this representational variation using graph\-spectral diagnostics, and[Section˜4\.2](https://arxiv.org/html/2608.03015#S4.SS2)visualizes the associated changes in the organization of attention\-induced token graphs\.

#### Choice of model\.

Our main analysis focuses on the pretrained base Llama\-3\.2\-3B model\.[Appendix˜G](https://arxiv.org/html/2608.03015#A7)extends this analysis to Llama\-3\.2\-1B and Llama\-3\.1\-8B, compares base and instruction\-tuned variants, and broadens the evaluation to Microsoft Phi\-4\(Abdinet al\.,[2024](https://arxiv.org/html/2608.03015#bib.bib5)\)and Hugging Face SmolLM3\(Bakouchet al\.,[2025](https://arxiv.org/html/2608.03015#bib.bib8)\)\. The broad context\-length\- and complexity\-dependent trends persist across these models\. Within the evaluated Llama 3 family, the 1B base model distinguishes the dynamical regimes less sharply than the 3B and 8B base models, and the instruction\-tuned variants show weaker regime separation than their base counterparts\. Within a shared architecture family, these comparisons suggest that model scale and instruction tuning both affect how distinctly dynamical regimes are expressed in the model’s internal representations\.

### 4\.1Graph\-Spectral Signatures of Dynamical Complexity Across Context Lengths

For our quantitative graph\-spectral analysis, we compute the diagnostics defined in[Section˜3\.2](https://arxiv.org/html/2608.03015#S3.SS2)for 20 realizations of each numerical input family introduced in[Section˜3\.1](https://arxiv.org/html/2608.03015#S3.SS1)at 20 context lengths ranging from 100 to 2000 tokens in increments of 100\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x3.png)Figure 3:Layer\-averaged graph\-spectral diagnostics for the main numerical input families using Llama\-3\.2\-3B\. Panels report \(a\) normalized Fiedler valueλ2\\lambda\_\{2\}, \(b\) HFER, and \(c\) effective spectral supportexp⁡\(SE\)\\exp\(\\mathrm\{SE\}\)\. At longer contexts, chaotic inputs show weaker global attention connectivity \(lowerλ2\\lambda\_\{2\}\), greater high\-frequency hidden\-state energy \(higher HFER\), and broader effective spectral support \(higherexp⁡\(SE\)\\exp\(\\mathrm\{SE\}\)\) than constant and periodic inputs; periodic families vary progressively with period\. Curves and shading show the mean and±1\\pm 1standard deviation across 20 realizations after averaging each diagnostic across transformer layers\.#### Global attention connectivity reveals context\-dependent regime separation\.

[Fig\.˜3](https://arxiv.org/html/2608.03015#S4.F3)\(a\) shows that, with sufficient context, the normalized Fiedler value separates the inputs into two broad groups: the chaotic logistic map and Lorenz inputs exhibit lowerλ2\\lambda\_\{2\}, whereas the constant and periodic inputs exhibit higherλ2\\lambda\_\{2\}\. Because lowerλ2\\lambda\_\{2\}indicates weaker global connectivity, the chaotic inputs induce more localized attention geometry, while the non\-chaotic inputs retain more globally integrated graphs\. Consistent with this interpretation, Appendix[Fig\.˜7](https://arxiv.org/html/2608.03015#A4.F7)shows that the chaotic inputs remain systematically below the non\-chaotic inputs across additional low\-end eigenvalues, indicating multiway localized graph structure beyondλ2\\lambda\_\{2\}\. Among the periodic inputs, those with higher periods exhibit a finer dependence on context length\. Periods 32 and 64 initially follow the decline of the chaotic inputs before turning upward atN=300N=300andN=400N=400, respectively, and approaching the lower\-period curves\. These ordered turning points suggest that the model needs progressively more context to resolve increasingly complex input dynamics in its internal attention organization\.

#### Graph\-signal smoothing varies with dynamical complexity\.

[Fig\.˜3](https://arxiv.org/html/2608.03015#S4.F3)\(b\) shows an increasingly clear complexity\-ordered HFER ladder as context length increases\. HFER decreases most strongly for the constant and lower\-period inputs, the decline weakens as period increases, and the chaotic inputs remain comparatively high after an initial decline\. Together, these trends indicate that the effect of additional context on how smoothly hidden states vary over the attention\-induced graph depends on input dynamical complexity: for simpler inputs, hidden\-state energy becomes increasingly concentrated in low\-frequency modes, whereas for more complex inputs, a larger fraction remains in high\-frequency modes, preserving sharper variation across the graph\.

#### Effective spectral support expands at complexity\-dependent rates\.

[Fig\.˜3](https://arxiv.org/html/2608.03015#S4.F3)\(c\) shows that the effective spectral support,exp⁡\(SE\)\\exp\(\\mathrm\{SE\}\), grows with context length for all inputs but remains well below its theoretical maximumNN\. Thus, increasing context does not simply cause the representation to spread uniformly across the larger set of available spectral modes; instead, it continues to favor a restricted spectral subset\. Over the evaluated context range, effective support expands at a complexity\-dependent rate: growth is modest for the constant and lower\-period inputs, whereas the higher\-period and chaotic inputs show stronger expansion\. These differences indicate that additional context affects the effective spectral dimensionality of internal representations differently across input families: hidden\-state energy for simpler dynamics remains concentrated within a compact, slowly expanding spectral support, whereas for more complex dynamics it is distributed across an increasingly broad set of graph Fourier modes\.

Taken together, the diagnostics reveal an ordered complexity gradient: greater input complexity corresponds to weaker global attention connectivity, higher HFER, and broader effective spectral support, with periodic inputs exhibiting graded changes as their period increases\.[Appendix˜H](https://arxiv.org/html/2608.03015#A8)shows that the broad trends also appear in attention\-only and hidden\-state\-only controls, while the graph\-spectral diagnostics more clearly separate input families\. Overall, these results show that the model’s internal representations reflect numerical input dynamics in a rich and systematically ordered manner as context length increases\.

### 4\.2Attention\-Induced Token\-Graph Organization Across Dynamical Complexity and Context Length

![Refer to caption](https://arxiv.org/html/2608.03015v1/x4.png)

![Refer to caption](https://arxiv.org/html/2608.03015v1/x5.png)

Figure 4:Final\-layer attention\-induced token graphs and trajectory mappings for Llama\-3\.2\-3B atN=500N=500\.Left: Node positions reflect the topology of the attention\-induced token graph, node colors encode hidden\-state graph signals, and circles and triangles denote numeric and separator tokens\. Simpler inputs show globally integrated layouts, whereas chaotic inputs show localized, clique\-like subnetworks\.Right: Matched colors link these clusters to trajectory intervals for period 64, the chaotic logistic map, and Lorenz inputs\. For period 64, a clique\-like cluster appears early, before sufficient context reveals the recurrence, but no comparable clusters appear later; chaotic inputs exhibit multiple such clusters throughout\.Following the construction in[Section˜3\.3](https://arxiv.org/html/2608.03015#S3.SS3),[Fig\.˜4](https://arxiv.org/html/2608.03015#S4.F4)presents representative final\-layer attention\-induced token graphs atN=500N=500\. At this context length, the layer\-averagedλ2\\lambda\_\{2\}trends in[Fig\.˜3](https://arxiv.org/html/2608.03015#S4.F3)\(a\) already show broad separation between chaotic and non\-chaotic inputs in global attention\-graph connectivity\.[Figure˜2](https://arxiv.org/html/2608.03015#S3.F2)further shows that the corresponding complexity\-dependent visual organization is visible byN=300N=300and remains qualitatively stable thereafter\. We focus on the final layer because its attention\-defined topology is closest to the representations used to compute the output logits\.[Appendix˜F](https://arxiv.org/html/2608.03015#A6)provides complementary visualizations of additional transformer layers\.

#### Token graphs become more localized with dynamical complexity\.

The layouts progress from comparatively regular, globally integrated organization for constant and lower\-period inputs to less homogeneous organization as the period increases\. The chaotic logistic map and Lorenz inputs contain multiple localized, clique\-like subnetworks, consistent with their lower normalized Fiedler values\. Beyond topology, the node colors reveal additional structure in the hidden\-state signals: numeric and separator tokens generally occupy distinct regions of the representation\. For the period\-2 input, separator tokens split into two color groups aligned with the alternating numerical states, suggesting that their hidden states encode the trajectory’s alternating phase despite their shared token identity\.

#### Long\-period periodicity becomes more distinct from chaos as context increases\.

Period 64 provides an informative intermediate case: despite its periodic dynamics, its graph contains a prominent clique\-like subnetwork resembling those of the chaotic inputs\. The right panel maps this subnetwork and those in the chaotic logistic map and Lorenz inputs to their corresponding trajectory intervals\. For period 64, this subnetwork is concentrated within roughly the first128128tokens, before the model has observed enough recurrence to distinguish its periodic structure from chaotic dynamics; later trajectory segments do not form similarly pronounced subnetworks\. In contrast, the chaotic logistic map and Lorenz graphs contain multiple localized subnetworks throughout the context, each associated primarily with a contiguous trajectory interval\. This difference suggests that additional context resolves the early ambiguity of the long\-period input, whereas temporal partitioning persists for chaotic dynamics\.

The visualizations complement the spectral diagnostics by showing a broad shift from globally integrated to more localized token\-graph organization as dynamical complexity increases\.[Appendix˜I](https://arxiv.org/html/2608.03015#A9)extends this qualitative analysis beyond the controlled dynamical\-system suite to additional structured numerical inputs, including distributional shifts accompanied by distinct token\-graph clustering\.

## 5Conclusion

We introduced a graph signal processing framework for numerical ICL, in which attention defines a token graph and hidden states form signals on its nodes\. As context grows, our graph\-spectral diagnostics and analyses of attention\-induced token graphs reveal increasingly clear differences across trajectories of varying dynamical complexity\. These findings show that the models’ emergent capacity for numerical ICL manifests not only in the autoregressive continuation of numerical sequences at the output level but also in the fine\-grained, context\-dependent organization of their internal representations\. More broadly, the framework provides a mesoscale characterization of LLM attention topology and hidden\-state organization, complementing microscale neuron\- and circuit\-level analyses and macroscale output\-based evaluations\.

In numerical ICL, an important open question is whether the observed spectral signatures and attention\-graph locality causally contribute to sequence continuation\. Reconstructions from individual graph Fourier modes could localize hidden\-state variation, while frequency\-band ablations could test the functional importance of particular bands\. Contrasting locality\-preserving with locality\-disrupting attention\-edge interventions could test the role of graph locality\.

Beyond numerical sequences, preliminary analyses in[Appendix˜J](https://arxiv.org/html/2608.03015#A10)show that, at matched context lengths, natural\-language and code inputs consistently exhibit more localized attention topology, a greater high\-frequency share of hidden\-state energy, and broader effective spectral support than numerical inputs\. Across natural languages, programming languages, and English\-language domains, it remains to determine how input properties such as syntax, semantic content, formatting, and predictability, and model\-dependent factors such as tokenization granularity and training\-data exposure contribute to both the observed separation from numerical inputs and graph\-spectral variation within these broader input classes\.

## Limitations

#### Mechanistic scope\.

Our graph\-spectral diagnostics of attention\-induced token graphs and hidden\-state signals identify structural and representational signatures associated with numerical ICL, but do not establish the causal mechanisms that produce this behavior\.

#### Model and tokenization choices\.

Our experiments cover several model families with Llama 3–style tokenization, for which each three\-digit state corresponds to a single numeric node in the attention\-induced token graph and each comma delimiter to a separator node\. Other model families use different numerical tokenization schemes; for example, Gemma 4 tokenizes each digit separately\(Gemma Team,[2026](https://arxiv.org/html/2608.03015#bib.bib9)\)\. Applying the same state\-aligned construction to such models would require aggregating digit\-level tokens into state\-level nodes, which we leave to future work\.

#### Access to internal representations\.

The proposed diagnostics and visualizations require access to layerwise attention weights and hidden\-state activations\. Hosted interfaces to frontier models, such as chat applications and APIs, typically do not expose these quantities\. The framework is currently most directly applicable to open\-weight models whose internal representations are accessible\.

## Data and Code Availability

## References

- M\. Abdin, J\. Aneja, H\. Behl, S\. Bubeck, R\. Eldan, S\. Gunasekar, M\. Harrison, R\. J\. Hewett, M\. Javaheripi, P\. Kauffmann, J\. R\. Lee, Y\. T\. Lee, Y\. Li, W\. Liu, C\. C\. T\. Mendes, A\. Nguyen, E\. Price, G\. de Rosa, O\. Saarikivi, A\. Salim, S\. Shah, X\. Wang, R\. Ward, Y\. Wu, D\. Yu, C\. Zhang, and Y\. Zhang \(2024\)Phi\-4 technical report\.arXiv preprint arXiv:2412\.08905\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2412.08905),[Link](https://arxiv.org/abs/2412.08905)Cited by:[§4](https://arxiv.org/html/2608.03015#S4.SS0.SSS0.Px1.p1.1)\.
- N\. Ahmed, T\. R\. Natarajan, and K\. R\. Rao \(1974\)Discrete cosine transform\.IEEE Trans\. Comput\.C\-23\(1\),pp\. 90–93\.External Links:[Document](https://dx.doi.org/10.1109/T-C.1974.223784)Cited by:[Appendix H](https://arxiv.org/html/2608.03015#A8.SS0.SSS0.Px2.p1.3)\.
- E\. Bakouch, L\. B\. Allal, A\. Lozhkov, N\. Tazi, L\. Tunstall, C\. M\. Patiño, E\. Beeching, A\. Roucher, A\. J\. Reedi, Q\. Gallouédec, K\. Rasul, N\. Habib, C\. Fourrier, H\. Kydlicek, G\. Penedo, H\. Larcher, M\. Morlon, V\. Srivastav, J\. Lochner, X\. Nguyen, C\. Raffel, L\. von Werra, and T\. Wolf \(2025\)SmolLM3: smol, multilingual, long\-context reasoner\.External Links:[Link](https://huggingface.co/blog/smollm3)Cited by:[§4](https://arxiv.org/html/2608.03015#S4.SS0.SSS0.Px1.p1.1)\.
- A\. Balaji \(2026\)Sparse spectral signatures of reasoning: model\-agnostic verification via sentence\-level graph signals\.InICLR 2026 Workshop on Logical Reasoning of Large Language Models,External Links:[Link](https://openreview.net/forum?id=ywWsOdQ7mi)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Bao, N\. Boullé, T\. J\.B\. Liu, R\. Sarfati, and C\. J\. Earls \(2026\)Text\-trained LLMs can zero\-shot extrapolate PDE dynamics, revealing a three\-stage in\-context learning mechanism\.InAI&PDE: ICLR 2026 Workshop on AI and Partial Differential Equations,External Links:[Link](https://openreview.net/forum?id=BN8hrHQnkC)Cited by:[Appendix G](https://arxiv.org/html/2608.03015#A7.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2608.03015#S1.p1.1)\.
- J\. Binkowski, D\. Janiak, A\. Sawczyn, B\. Gabrys, and T\. J\. Kajdanowicz \(2025\)Hallucination detection in LLMs using spectral features of attention maps\.InProc\. EMNLP,pp\. 24354–24385\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1239),[Link](https://aclanthology.org/2025.emnlp-main.1239/)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px2.p1.1)\.
- P\. C\. Bogdan, U\. Macar, N\. Nanda, and A\. Conmy \(2025\)Thought anchors: which LLM reasoning steps matter?\.InMechanistic Interpretability Workshop at NeurIPS 2025,External Links:[Link](https://openreview.net/forum?id=VnSlfeRCaU)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px3.p1.1)\.
- S\. L\. Brunton, J\. L\. Proctor, and J\. N\. Kutz \(2016\)Discovering governing equations from data by sparse identification of nonlinear dynamical systems\.Proc\. Natl\. Acad\. Sci\. USA113\(15\),pp\. 3932–3937\.External Links:[Document](https://dx.doi.org/10.1073/pnas.1517384113)Cited by:[Appendix A](https://arxiv.org/html/2608.03015#A1.SS0.SSS0.Px1.p1.33)\.
- I\. Chalkidis, A\. Jana, D\. Hartung, M\. Bommarito, I\. Androutsopoulos, D\. Katz, and N\. Aletras \(2022\)LexGLUE: a benchmark dataset for legal language understanding in English\.InProc\. ACL,pp\. 4310–4330\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.297)Cited by:[Appendix J](https://arxiv.org/html/2608.03015#A10.SS0.SSS0.Px4.p1.1)\.
- D\. Christou and G\. Tsoumakas \(2025\)Artificial relationships in fiction: a dataset for advancing NLP in literary domains\.InProceedings of the 9th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature,pp\. 130–147\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.latechclfl-1.13)Cited by:[Appendix J](https://arxiv.org/html/2608.03015#A10.SS0.SSS0.Px4.p1.1)\.
- F\. R\. K\. Chung \(1997\)Spectral graph theory\.AMS\.External Links:[Document](https://dx.doi.org/10.1090/cbms/092)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2608.03015#S3.SS2.SSS0.Px3.p1.1)\.
- A\. Cohan, F\. Dernoncourt, D\. S\. Kim, T\. Bui, S\. Kim, W\. Chang, and N\. Goharian \(2018\)A discourse\-aware attention model for abstractive summarization of long documents\.InProc\. NAACL\-HLT,pp\. 615–621\.External Links:[Document](https://dx.doi.org/10.18653/v1/N18-2097)Cited by:[Appendix J](https://arxiv.org/html/2608.03015#A10.SS0.SSS0.Px4.p1.1)\.
- Y\. Dai, Z\. Gao, Y\. Sattar, S\. Dean, and J\. J\. Sun \(2025\)Pre\-trained large language models learn to predict hidden Markov models in\-context\.InNeurIPS,External Links:[Link](https://openreview.net/forum?id=btBqWTbf6q)Cited by:[§1](https://arxiv.org/html/2608.03015#S1.p1.1)\.
- Z\. Dong, Y\. Li, Y\. Sun, W\. Wang, Y\. Luo, Z\. Peng, W\. Su, C\. Yu, B\. Zheng, and J\. Yan \(2026\)How does reasoning flow? tracing attention\-induced information flow for targeted RL in LLMs\.InICML,External Links:[Link](https://openreview.net/forum?id=Ym4KYa1n76)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Eisele and Y\. Chen \(2010\)MultiUN: a multilingual corpus from United Nation documents\.InProc\. LREC,External Links:[Link](https://aclanthology.org/L10-1473/)Cited by:[Appendix J](https://arxiv.org/html/2608.03015#A10.SS0.SSS0.Px2.p1.1)\.
- J\. D\. Farmer \(1982\)Information dimension and the probabilistic structure of chaos\.Z\. Naturforsch\. A37\(11\),pp\. 1304–1325\.External Links:[Document](https://dx.doi.org/10.1515/zna-1982-1117)Cited by:[Appendix B](https://arxiv.org/html/2608.03015#A2.p1.1)\.
- M\. Fiedler \(1973\)Algebraic connectivity of graphs\.Czechoslov\. Math\. J\.23\(2\),pp\. 298–305\.External Links:[Document](https://dx.doi.org/10.21136/CMJ.1973.101168),[Link](https://dml.cz/handle/10338.dmlcz/101168)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2608.03015#S3.SS2.SSS0.Px3.p1.1)\.
- F\. Frasca, G\. Bar\-Shalom, Y\. Ziser, and H\. Maron \(2026\)Neural message\-passing on attention graphs for hallucination detection\.InICLR,External Links:[Link](https://openreview.net/forum?id=4twbqwV4br)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px3.p1.1)\.
- T\. M\. J\. Fruchterman and E\. M\. Reingold \(1991\)Graph drawing by force\-directed placement\.Softw\. Pract\. Exp\.21\(11\),pp\. 1129–1164\.External Links:[Document](https://dx.doi.org/10.1002/spe.4380211102),[Link](https://www.reingold.co/force-directed.pdf)Cited by:[Appendix E](https://arxiv.org/html/2608.03015#A5.SS0.SSS0.Px3.p1.9),[§3\.3](https://arxiv.org/html/2608.03015#S3.SS3.SSS0.Px2.p1.3)\.
- Gemma Team \(2026\)Gemma 4 technical report\.arXiv preprint arXiv:2607\.02770\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2607.02770),[Link](https://arxiv.org/abs/2607.02770)Cited by:[Model and tokenization choices\.](https://arxiv.org/html/2608.03015#Sx1.SS0.SSS0.Px2.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, G\. \(\. Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Y\. \(\. Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. Ma \(2024\)The Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2407.21783),[Link](https://arxiv.org/abs/2407.21783)Cited by:[Appendix G](https://arxiv.org/html/2608.03015#A7.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2608.03015#S3.SS1.SSS0.Px2.p1.8)\.
- N\. Gruver, M\. A\. Finzi, S\. Qiu, and A\. G\. Wilson \(2023\)Large language models are zero\-shot time series forecasters\.InNeurIPS,External Links:[Link](https://openreview.net/forum?id=md68e8iZK1)Cited by:[Appendix G](https://arxiv.org/html/2608.03015#A7.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2608.03015#S1.p1.1),[§3\.1](https://arxiv.org/html/2608.03015#S3.SS1.SSS0.Px2.p1.8)\.
- A\. A\. Hagberg, D\. A\. Schult, and P\. J\. Swart \(2008\)Exploring network structure, dynamics, and function using NetworkX\.InProc\. SciPy,External Links:[Document](https://dx.doi.org/10.25080/TCWV9851)Cited by:[Appendix E](https://arxiv.org/html/2608.03015#A5.SS0.SSS0.Px3.p1.9)\.
- D\. K\. Hammond, P\. Vandergheynst, and R\. Gribonval \(2011\)Wavelets on graphs via spectral graph theory\.Appl\. Comput\. Harmon\. Anal\.30\(2\),pp\. 129–150\.External Links:[Document](https://dx.doi.org/10.1016/j.acha.2010.04.005),[Link](https://arxiv.org/abs/0912.3848)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px1.p1.1)\.
- K\. M\. Hermann, T\. Kočiský, E\. Grefenstette, L\. Espeholt, W\. Kay, M\. Suleyman, and P\. Blunsom \(2015\)Teaching machines to read and comprehend\.InNIPS,External Links:[Link](https://proceedings.neurips.cc/paper/5945-teaching-machines-to-read-and-comprehend)Cited by:[Appendix J](https://arxiv.org/html/2608.03015#A10.SS0.SSS0.Px4.p1.1)\.
- N\. Hurley and S\. Rickard \(2009\)Comparing measures of sparsity\.IEEE Trans\. Inf\. Theory55\(10\),pp\. 4723–4741\.External Links:[Document](https://dx.doi.org/10.1109/TIT.2009.2027527)Cited by:[Appendix H](https://arxiv.org/html/2608.03015#A8.SS0.SSS0.Px1.p1.1)\.
- M\. Jin, S\. Wang, L\. Ma, Z\. Chu, J\. Y\. Zhang, X\. Shi, P\. Chen, Y\. Liang, Y\. Li, S\. Pan, and Q\. Wen \(2024\)Time\-LLM: time series forecasting by reprogramming large language models\.InICLR,External Links:[Link](https://openreview.net/forum?id=Unb5CVPtae)Cited by:[§1](https://arxiv.org/html/2608.03015#S1.p1.1)\.
- L\. Jost \(2006\)Entropy and diversity\.Oikos113\(2\),pp\. 363–375\.External Links:[Document](https://dx.doi.org/10.1111/j.2006.0030-1299.14714.x),[Link](https://nsojournals.onlinelibrary.wiley.com/doi/abs/10.1111/j.2006.0030-1299.14714.x)Cited by:[§3\.2](https://arxiv.org/html/2608.03015#S3.SS2.SSS0.Px5.p1.4)\.
- D\. Kocetkov, R\. Li, L\. Ben Allal, J\. Li, C\. Mou, Y\. Jernite, M\. Mitchell, C\. Muñoz Ferrandis, S\. Hughes, T\. Wolf, D\. Bahdanau, L\. von Werra, and H\. de Vries \(2023\)The Stack: 3 TB of permissively licensed source code\.Trans\. Mach\. Learn\. Res\.\.External Links:[Link](https://openreview.net/forum?id=pxpbTdUEpD)Cited by:[Appendix J](https://arxiv.org/html/2608.03015#A10.SS0.SSS0.Px3.p1.1)\.
- J\. R\. Lee, S\. Oveis Gharan, and L\. Trevisan \(2014\)Multiway spectral partitioning and higher\-order Cheeger inequalities\.J\. ACM61\(6\),pp\. 37:1–37:30\.External Links:[Document](https://dx.doi.org/10.1145/2665063),[Link](https://doi.org/10.1145/2665063)Cited by:[§D\.1](https://arxiv.org/html/2608.03015#A4.SS1.SSS0.Px1.p1.10),[§3\.2](https://arxiv.org/html/2608.03015#S3.SS2.SSS0.Px3.p1.7)\.
- R\. Li, L\. B\. Allal, Y\. Zi, N\. Muennighoff, D\. Kocetkov, C\. Mou, M\. Marone, C\. Akiki, J\. Li, J\. Chim, Q\. Liu, E\. Zheltonozhskii, T\. Y\. Zhuo, T\. Wang, O\. Dehaene, M\. Davaadorj, J\. Lamy\-Poirier, J\. Monteiro, O\. Shliazhko, N\. Gontier, N\. Meade, A\. Zebaze, M\. Yee, L\. K\. Umapathi, J\. Zhu, B\. Lipkin, M\. Oblokulov, Z\. Wang, R\. Murthy, J\. Stillerman, S\. S\. Patel, D\. Abulkhanov, M\. Zocca, M\. Dey, Z\. Zhang, N\. Fahmy, U\. Bhattacharyya, W\. Yu, S\. Singh, S\. Luccioni, P\. Villegas, M\. Kunakov, F\. Zhdanov, M\. Romero, T\. Lee, N\. Timor, J\. Ding, C\. Schlesinger, H\. Schoelkopf, J\. Ebert, T\. Dao, M\. Mishra, A\. Gu, J\. Robinson, C\. J\. Anderson, B\. Dolan\-Gavitt, D\. Contractor, S\. Reddy, D\. Fried, D\. Bahdanau, Y\. Jernite, C\. M\. Ferrandis, S\. Hughes, T\. Wolf, A\. Guha, L\. von Werra, and H\. de Vries \(2023\)StarCoder: may the source be with you\!\.Trans\. Mach\. Learn\. Res\.\.External Links:[Link](https://openreview.net/forum?id=KoFOg41haE)Cited by:[Appendix J](https://arxiv.org/html/2608.03015#A10.SS0.SSS0.Px3.p1.1)\.
- Y\. Li, H\. Su, X\. Shen, W\. Li, Z\. Cao, and S\. Niu \(2017\)DailyDialog: a manually labelled multi\-turn dialogue dataset\.InProc\. IJCNLP,pp\. 986–995\.External Links:[Link](https://aclanthology.org/I17-1099/)Cited by:[Appendix J](https://arxiv.org/html/2608.03015#A10.SS0.SSS0.Px4.p1.1)\.
- T\. J\.B\. Liu, N\. Boullé, R\. Sarfati, and C\. J\. Earls \(2024\)LLMs learn governing principles of dynamical systems, revealing an in\-context neural scaling law\.InProc\. EMNLP,pp\. 15097–15117\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.842),[Link](https://aclanthology.org/2024.emnlp-main.842/)Cited by:[§1](https://arxiv.org/html/2608.03015#S1.p1.1),[§3\.1](https://arxiv.org/html/2608.03015#S3.SS1.SSS0.Px2.p1.8)\.
- T\. J\.B\. Liu, N\. Boullé, R\. Sarfati, and C\. J\. Earls \(2025\)Density estimation with LLMs: a geometric investigation of in\-context learning trajectories\.InICLR,pp\. 22163–22197\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/380afe1a245a3b2134010620eae88865-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2608.03015#S1.p1.1)\.
- E\. N\. Lorenz \(1963\)Deterministic nonperiodic flow\.J\. Atmos\. Sci\.20\(2\),pp\. 130–141\.External Links:[Document](https://dx.doi.org/10.1175/1520-0469%281963%29020%3C0130%3ADNF%3E2.0.CO%3B2)Cited by:[Appendix A](https://arxiv.org/html/2608.03015#A1.SS0.SSS0.Px1.p1.33),[§3\.1](https://arxiv.org/html/2608.03015#S3.SS1.SSS0.Px1.p1.8)\.
- R\. M\. May \(1976\)Simple mathematical models with very complicated dynamics\.Nature261\(5560\),pp\. 459–467\.External Links:[Document](https://dx.doi.org/10.1038/261459a0)Cited by:[§3\.1](https://arxiv.org/html/2608.03015#S3.SS1.SSS0.Px1.p1.8)\.
- V\. Noël \(2025\)A graph signal processing framework for hallucination detection in large language models\.arXiv preprint arXiv:2510\.19117\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2510.19117),[Link](https://arxiv.org/abs/2510.19117)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px2.p1.1)\.
- V\. Noël \(2026\)Geometry of reason: spectral signatures of valid mathematical reasoning\.InICML,External Links:[Link](https://openreview.net/forum?id=CrzJGqvrG8)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2608.03015#S3.SS2.p1.1)\.
- OpenAI \(2023\)GPT\-4 technical report\.arXiv preprint arXiv:2303\.08774\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2303.08774),[Link](https://arxiv.org/abs/2303.08774)Cited by:[§3\.1](https://arxiv.org/html/2608.03015#S3.SS1.SSS0.Px2.p1.8)\.
- A\. Ortega, P\. Frossard, J\. Kovačević, J\. M\. F\. Moura, and P\. Vandergheynst \(2018\)Graph signal processing: overview, challenges, and applications\.Proc\. IEEE106\(5\),pp\. 808–828\.External Links:[Document](https://dx.doi.org/10.1109/JPROC.2018.2820126),[Link](https://arxiv.org/abs/1712.00468)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Piskorz, K\. Kobalczyk, and M\. van der Schaar \(2026\)Eliciting numerical predictive distributions of LLMs without auto\-regression\.InICLR,External Links:[Link](https://openreview.net/forum?id=SsuBd46twl)Cited by:[§1](https://arxiv.org/html/2608.03015#S1.SS0.SSS0.Px1.p1.1)\.
- J\. Requeima, J\. Bronskill, D\. Choi, R\. E\. Turner, and D\. Duvenaud \(2024\)LLM processes: numerical predictive distributions conditioned on natural language\.InNeurIPS,pp\. 109609–109671\.External Links:[Document](https://dx.doi.org/10.52202/079017-3479),[Link](https://openreview.net/forum?id=HShs7q1Njh)Cited by:[§1](https://arxiv.org/html/2608.03015#S1.p1.1)\.
- R\. Sarfati, E\. Bigelow, D\. Wurgaft, S\. Boppana, J\. Merullo, A\. Geiger, O\. Lewis, T\. McGrath, and E\. S\. Lubana \(2026\)The shape of beliefs: geometry, dynamics, and interventions along representation manifolds of language models’ posteriors\.arXiv preprint arXiv:2602\.02315\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2602.02315),[Link](https://arxiv.org/abs/2602.02315)Cited by:[Appendix I](https://arxiv.org/html/2608.03015#A9.p2.6),[Appendix I](https://arxiv.org/html/2608.03015#A9.p5.1),[§1](https://arxiv.org/html/2608.03015#S1.SS0.SSS0.Px1.p1.1)\.
- A\. See, P\. J\. Liu, and C\. D\. Manning \(2017\)Get to the point: summarization with pointer\-generator networks\.InProc\. ACL,pp\. 1073–1083\.External Links:[Document](https://dx.doi.org/10.18653/v1/P17-1099)Cited by:[Appendix J](https://arxiv.org/html/2608.03015#A10.SS0.SSS0.Px4.p1.1)\.
- C\. E\. Shannon \(1948\)A mathematical theory of communication\.Bell Syst\. Tech\. J\.27\(3\),pp\. 379–423\.External Links:[Document](https://dx.doi.org/10.1002/j.1538-7305.1948.tb01338.x)Cited by:[Appendix B](https://arxiv.org/html/2608.03015#A2.p1.1)\.
- D\. I\. Shuman, S\. K\. Narang, P\. Frossard, A\. Ortega, and P\. Vandergheynst \(2013\)The emerging field of signal processing on graphs: extending high\-dimensional data analysis to networks and other irregular domains\.IEEE Signal Process\. Mag\.30\(3\),pp\. 83–98\.External Links:[Document](https://dx.doi.org/10.1109/MSP.2012.2235192),[Link](https://arxiv.org/abs/1211.0053)Cited by:[§2](https://arxiv.org/html/2608.03015#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2608.03015#S3.SS2.SSS0.Px2.p1.12),[§3\.3](https://arxiv.org/html/2608.03015#S3.SS3.SSS0.Px1.p1.1)\.
- S\. H\. Strogatz \(2024\)Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering\.3rd edition,Chapman and Hall/CRC\.External Links:[Document](https://dx.doi.org/10.1201/9780429398490),ISBN 9780429398490,[Link](https://www.taylorfrancis.com/books/mono/10.1201/9780429398490/nonlinear-dynamics-chaos-steven-strogatz)Cited by:[§3\.1](https://arxiv.org/html/2608.03015#S3.SS1.SSS0.Px1.p1.8)\.
- J\. Tiedemann \(2012\)Parallel data, tools and interfaces in OPUS\.InProc\. LREC,pp\. 2214–2218\.External Links:[Link](https://aclanthology.org/L12-1246/)Cited by:[Appendix J](https://arxiv.org/html/2608.03015#A10.SS0.SSS0.Px2.p1.1)\.
- R\. Vacareanu, V\. A\. Negru, V\. Suciu, and M\. Surdeanu \(2024\)From words to numbers: your large language model is secretly a capable regressor when given in\-context examples\.InCOLM,External Links:[Link](https://openreview.net/forum?id=LzpaUxcNFK)Cited by:[§1](https://arxiv.org/html/2608.03015#S1.p1.1)\.
- P\. Walters \(1982\)An introduction to ergodic theory\.Springer\.External Links:[Link](https://link.springer.com/book/9780387951522)Cited by:[Appendix B](https://arxiv.org/html/2608.03015#A2.p1.1)\.
- Wikimedia Foundation \(2023\)Wikimedia downloads\.External Links:[Link](https://dumps.wikimedia.org/)Cited by:[Appendix J](https://arxiv.org/html/2608.03015#A10.SS0.SSS0.Px4.p1.1)\.
- O\. Zekri, A\. Odonnat, A\. Benechehab, L\. Bleistein, N\. Boullé, and I\. Redko \(2024\)Large language models as Markov chains\.arXiv preprint arXiv:2410\.02724\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2410.02724),[Link](https://arxiv.org/abs/2410.02724)Cited by:[§1](https://arxiv.org/html/2608.03015#S1.p1.1)\.
- Y\. Zhang and W\. Gilpin \(2025\)Zero\-shot forecasting of chaotic systems\.InICLR,External Links:[Link](https://openreview.net/forum?id=TqYjhJrp9m)Cited by:[§1](https://arxiv.org/html/2608.03015#S1.p1.1)\.

## Appendix Contents

A\.B\.C\.D\.E\.F\.G\.H\.I\.J\.

## Appendix AMain Numerical Input Suite and Next\-State Extrapolation

#### Main numerical input suite\.

The main numerical input suite comprises ten one\-dimensional trajectory families: a constant\-sequence baseline whose value is sampled separately for each realization, six periodic logistic map families with periods2,4,8,16,32,2,4,8,16,32,and6464, two chaotic logistic map families, and a family of trajectories obtained from the Lorenzxx\-component\. For each family, we construct an ensemble of 20 trajectory realizations\. The same realizations are used in the graph\-spectral analysis presented in the main text and in the extrapolation analysis described below\. Realizationjjuses a shared initial\-state drawx0\(j\)∼Unif​\(0\.3,0\.7\)x\_\{0\}^\{\(j\)\}\\sim\\mathrm\{Unif\}\(0\.3,0\.7\)across these families\. For the constant baseline, realizationjjis assigned the quantized state

qconst\(j\)=round⁡\(50\+900​x0\(j\)\)\.q\_\{\\mathrm\{const\}\}^\{\(j\)\}=\\operatorname\{round\}\\\!\\left\(50\+900x\_\{0\}^\{\(j\)\}\\right\)\.This quantized value is repeated across all 1000 states of realizationjjand varies across realizations\. Logistic map inputs are generated fromxt\+1=r​xt​\(1−xt\)x\_\{t\+1\}=rx\_\{t\}\(1\-x\_\{t\}\)\. The periodic cases user=3\.2r=3\.2,3\.53\.5,3\.553\.55,3\.5663\.566,3\.56923\.5692, and3\.56983\.5698, corresponding to periods2,4,8,16,32,2,4,8,16,32,and6464, and the chaotic cases user=3\.9r=3\.9andr=4\.0r=4\.0\. For each realization of a periodic case with expected periodpp, we generate a 20,000\-state trajectory, identify the earliest point after which\|xt\+p−xt\|<10−10\|x\_\{t\+p\}\-x\_\{t\}\|<10^\{\-10\}holds for the remainder of the trajectory, and extract a 1000\-state window after applying a random phase offset in\{0,…,p−1\}\\\{0,\\ldots,p\-1\\\}\. We use 20 realizations primarily to characterize variation in the graph\-spectral diagnostics for families with greater across\-realization trajectory diversity, particularly the higher\-period and chaotic families\. We retain the same ensemble size for the lower\-period families to ensure consistent aggregation across inputs, although a converged period\-pporbit has onlypppossible phases, so duplicate phase selections are expected\. For the chaotic logistic families, we instead use direct 1000\-state trajectories from the realization\-specificx0\(j\)x\_\{0\}^\{\(j\)\}\. The Lorenz trajectory family is generated from

x˙=σ​\(y−x\),y˙=x​\(ρ−z\)−y,z˙=x​y−β​z,\\begin\{array\}\[\]\{rcl\}\\dot\{x\}&=&\\sigma\(y\-x\),\\\\ \\dot\{y\}&=&x\(\\rho\-z\)\-y,\\\\ \\dot\{z\}&=&xy\-\\beta z,\\end\{array\}withσ=10\\sigma=10,ρ=28\\rho=28, andβ=8/3\\beta=8/3, the classical chaotic parameter setting\(Lorenz,[1963](https://arxiv.org/html/2608.03015#bib.bib37)\)\. For realizationjj, the initial condition is\(x,y,z\)=\(−8\.0\+\(x0\(j\)−0\.5\),7\.0,27\.0\)\(x,y,z\)=\(\-8\.0\+\(x\_\{0\}^\{\(j\)\}\-0\.5\),\\,7\.0,\\,27\.0\), varying the initialxx\-coordinate around the standard\(−8,7,27\)\(\-8,7,27\)initialization\(Bruntonet al\.,[2016](https://arxiv.org/html/2608.03015#bib.bib38)\)\. We integrate overτ∈\[0,100\]\\tau\\in\[0,100\]using SciPy’ssolve\_ivpwith its default solver settings, record 1000 evenly spaced time points, and retain only thexx\-component, yielding, for each realization, a partially observed one\-dimensional trajectory from a continuous\-time chaotic system\.

#### Trajectory quantization and prompt serialization\.

For a logistic or Lorenz realization𝐮=\(u1,…,u1000\)\\mathbf\{u\}=\(u\_\{1\},\\ldots,u\_\{1000\}\), let

umin=mini⁡uiandumax=maxi⁡ui\.u\_\{\\min\}=\\min\_\{i\}u\_\{i\}\\qquad\\text\{and\}\\qquad u\_\{\\max\}=\\max\_\{i\}u\_\{i\}\.Ifumax=uminu\_\{\\max\}=u\_\{\\min\}, we setqi=500q\_\{i\}=500\. Otherwise, the quantized state at positioniiis

qi=round⁡\(50\+900​ui−uminumax−umin\)\.q\_\{i\}=\\operatorname\{round\}\\\!\\left\(50\+900\\frac\{u\_\{i\}\-u\_\{\\min\}\}\{u\_\{\\max\}\-u\_\{\\min\}\}\\right\)\.Here,round⁡\(⋅\)\\operatorname\{round\}\(\\cdot\)denotes rounding to the nearest integer, soqi∈\{50,…,950\}q\_\{i\}\\in\\\{50,\\ldots,950\\\}\. The sampled constant baseline bypasses this mapping and usesqconst\(j\)q\_\{\\mathrm\{const\}\}^\{\(j\)\}as defined above\. The scaling bounds are computed once from the full 1000\-state realization before context truncation\. Each quantized state is formatted as a zero\-padded three\-digit string and followed by a comma delimiter, including the final state\. Thus, a realization is serialized schematically as “q1,q2,…,q1000,q\_\{1\},q\_\{2\},\\ldots,q\_\{1000\},”\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x6.png)Figure 5:Context\-length scaling of next\-state extrapolation error with Llama\-3\.2\-3B\. Curves show normalized expected\-value error for predicting the next quantized state from the preceding serialized numerical context\. Longer context generally reduces extrapolation error across the main numerical input families\. Several low\-error inputs have tightly clustered output\-level curves, motivating the representation\-level analysis: graph\-spectral diagnostics reveal more ordered differences in the model’s internal token organization that are less apparent from extrapolation error alone\. The shaded bands show one log standard deviation across realizations\.
#### Next\-state extrapolation evaluation\.

[Fig\.˜5](https://arxiv.org/html/2608.03015#A1.F5)reports output\-level next\-state extrapolation error across context lengths for the main numerical input families\. We evaluate 49 context fractions from 2% through 98% in increments of two percentage points\. For each fractioncc, the prompt containstc=round⁡\(c​\(1000−1\)\)t\_\{c\}=\\operatorname\{round\}\(c\(1000\-1\)\)states, reserving the following state as the prediction target\. For input familyff, realizationjj, and state indextt, the prompt contains the firstttcomma\-delimited quantized states and the target is the\(t\+1\)\(t\+1\)\-st state, denotedyj,t\+1f∈\{50,…,950\}y^\{f\}\_\{j,t\+1\}\\in\\\{50,\\ldots,950\\\}\. At the final comma position of the prompt, the model produces next\-token logits over its full vocabulary\. For each integerv∈𝒱=\{0,…,999\}v\\in\\mathcal\{V\}=\\\{0,\\ldots,999\\\}, letzj,tf​\(v\)z^\{f\}\_\{j,t\}\(v\)denote the logit assigned to the token encoding the zero\-padded three\-digit representation ofvv\. We retain the full numeric output range000–999, so values outside the input interval050–950provide buffers that reduce boundary truncation of probability mass near the quantization boundaries\. We then evaluate a softmax distribution restricted to these numeric tokens

pj,tf​\(v\)=exp⁡zj,tf​\(v\)∑u∈𝒱exp⁡zj,tf​\(u\)\.p^\{f\}\_\{j,t\}\(v\)=\\frac\{\\exp z^\{f\}\_\{j,t\}\(v\)\}\{\\sum\_\{u\\in\\mathcal\{V\}\}\\exp z^\{f\}\_\{j,t\}\(u\)\}\.The corresponding expected next state is

y^j,t\+1f=∑v∈𝒱v​pj,tf​\(v\)\.\\hat\{y\}^\{f\}\_\{j,t\+1\}=\\sum\_\{v\\in\\mathcal\{V\}\}v\\,p^\{f\}\_\{j,t\}\(v\)\.We measure extrapolation error using the normalized absolute difference between the expected next state and the target:

ej,tf=\|y^j,t\+1f−yj,t\+1f\|950−50\.e^\{f\}\_\{j,t\}=\\frac\{\|\\hat\{y\}^\{f\}\_\{j,t\+1\}\-y^\{f\}\_\{j,t\+1\}\|\}\{950\-50\}\.The plotted curves are geometric means ofej,tfe^\{f\}\_\{j,t\}over the 20 realizations at eachtt, with shaded bands corresponding to plus or minus one standard deviation on the log\-error scale\. This behavioral check shows that extrapolation accuracy generally improves with longer context across all main numerical input families, while chaotic logistic and Lorenz inputs exhibit larger extrapolation errors than simpler periodic and constant inputs\. At the same time, several low\-error inputs have tightly clustered output\-level curves, motivating the representation\-level analysis in the main text\.

The dashed reference line marks one quantization\-bin error,1/9001/900: because the error is normalized by950−50=900950\-50=900, this corresponds to an absolute error of one quantized state unit\. Periodic inputs often lie below this line, indicating that the restricted\-softmax expected output is within one bin of the ground\-truth quantized state; their curves can still decrease with longer context, indicating further reductions in expected\-value error even after reaching this output\-level scale\.

## Appendix BEntropy\-Based Quantification of Input Dynamical Complexity

An invariant probability measureμ\\muprovides a time\-independent summary of state\-space occupancy in a dynamical system\(Walters,[1982](https://arxiv.org/html/2608.03015#bib.bib28)\)\. Coarse\-grained entropies of invariant state distributions are a classical tool for characterizing the statistical structure of dynamical systems\(Farmer,[1982](https://arxiv.org/html/2608.03015#bib.bib27)\)\. We use Shannon entropy\(Shannon,[1948](https://arxiv.org/html/2608.03015#bib.bib26)\)to summarize how broadly this distribution is spread across the visited states\. This quantity provides quantitative support for the operational ordering of dynamical complexity used in[Section˜3\.1](https://arxiv.org/html/2608.03015#S3.SS1)\.

For a period\-pporbit with distinct statesx1,…,xpx\_\{1\},\\ldots,x\_\{p\}, the invariant measure assigns probability1/p1/pto each state:

μp=1p​∑j=1pδxj,\\mu\_\{p\}=\\frac\{1\}\{p\}\\sum\_\{j=1\}^\{p\}\\delta\_\{x\_\{j\}\},whereδxj\\delta\_\{x\_\{j\}\}denotes a unit point mass atxjx\_\{j\}\. Its state\-distribution entropy is therefore

Hstate​\(μp\)=−∑j=1p1p​log2⁡1p=log2⁡p\.H\_\{\\mathrm\{state\}\}\(\\mu\_\{p\}\)=\-\\sum\_\{j=1\}^\{p\}\\frac\{1\}\{p\}\\log\_\{2\}\\frac\{1\}\{p\}=\\log\_\{2\}p\.Thus, the constant input has entropy0, while periods2,4,8,16,32,2,4,8,16,32,and6464have ideal entropies1,2,3,4,5,1,2,3,4,5,and66bits, respectively\. To compare all input families on a common basis, we estimate state\-distribution entropy directly from each finite quantized trajectoryq1,…,qTq\_\{1\},\\ldots,q\_\{T\}used as an LLM input, whereqt∈\{50,…,950\}q\_\{t\}\\in\\\{50,\\ldots,950\\\}, using 100 equal\-width bins with shared edges spanning\[49\.5,950\.5\]\[49\.5,950\.5\]across all input families and realizations\. LettingBbB\_\{b\}denote thebb\-th bin, we compute

p^T,b\\displaystyle\\widehat\{p\}\_\{T,b\}=1T​∑t=1T𝟏​\{qt∈Bb\},\\displaystyle=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbf\{1\}\\\{q\_\{t\}\\in B\_\{b\}\\\},H^T\\displaystyle\\widehat\{H\}\_\{T\}=−∑b:p^T,b\>0p^T,b​log2⁡p^T,b\.\\displaystyle=\-\\sum\_\{b:\\widehat\{p\}\_\{T,b\}\>0\}\\widehat\{p\}\_\{T,b\}\\log\_\{2\}\\widehat\{p\}\_\{T,b\}\.
Table 1:Coarse\-grained state\-distribution entropy of the quantized numerical input families introduced in[Section˜3\.1](https://arxiv.org/html/2608.03015#S3.SS1)\. Values are reported as the mean±\\pmsample standard deviation in bits across 20 realizations at trajectory lengthT=1000T=1000, estimated using 100 equal\-width bins with shared edges across all input families and realizations\.Applying this estimator to realizationjjof input familyffgivesH^j,Tf\\widehat\{H\}^\{f\}\_\{j,T\}\.[Table˜1](https://arxiv.org/html/2608.03015#A2.T1)reports the mean and sample standard deviation ofH^j,1000f\\widehat\{H\}^\{f\}\_\{j,1000\}overj=1,…,20j=1,\\ldots,20for each familyff\. The empirical entropies are consistent with the intended complexity ordering: entropy increases with logistic map period, while the chaotic logistic and Lorenz inputs exhibit the highest state\-distribution entropies\. The higher\-period values fall below the ideal valueslog2⁡p\\log\_\{2\}pbecause the orbit states are nonuniformly spaced across state space, causing multiple distinct states to fall within the same histogram bin\. The reported quantity is therefore a coarse\-grained estimate of the entropy of the invariant state distribution\.

[Figure˜6](https://arxiv.org/html/2608.03015#A2.F6)shows that empirical estimates of state\-distribution entropy stabilize with increasing trajectory length, consistent with entropy characterizing the underlying invariant state distribution\. The estimates are computed at the context lengths used in the main analysis:T=50,100,…,1000T=50,100,\\ldots,1000numerical states, corresponding toN=100,200,…,2000N=100,200,\\ldots,2000tokens under theN=2​TN=2Tserialization described in[Section˜3\.1](https://arxiv.org/html/2608.03015#S3.SS1)\. The periodic estimates are already close to theirT=1000T=1000values byT=100T=100\. The chaotic logistic and Lorenz estimates increase most sharply betweenT=50T=50andT=200T=200, then gradually level off, with only modest changes beyondT=400T=400\. Thus, the context\-dependent growth in the spectral entropy of the LLM’s internal representations cannot be explained simply by variation in the estimated state\-distribution entropy of the inputs\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x7.png)Figure 6:Empirical state\-distribution entropy as a function of trajectory lengthTT\. Periodic estimates stabilize byT=100T=100, while chaotic logistic and Lorenz estimates also stabilize relatively early, with little change beyondT=400T=400\. Entropy is estimated using 100 equal\-width bins with shared edges across all input families and realizations\. Curves and shading show the mean and±1\\pm 1sample standard deviation across the same 20 realizations used in[Section˜3\.1](https://arxiv.org/html/2608.03015#S3.SS1)\.
## Appendix CLaplacian Quadratic Forms and Graph Fourier Modes

We briefly review why the graph Laplacian measures signal variation over a weighted graph\. LetW∈ℝN×NW\\in\\mathbb\{R\}^\{N\\times N\}be a symmetric nonnegative weighted adjacency matrix, letD=diag⁡\(W​𝟏\)D=\\operatorname\{diag\}\(W\\mathbf\{1\}\), and letL=D−WL=D\-Wbe the combinatorial graph Laplacian\.

First consider a scalar graph signal𝐱=\(x1,…,xN\)⊤\\mathbf\{x\}=\(x\_\{1\},\\ldots,x\_\{N\}\)^\{\\top\}, wherexix\_\{i\}is the value assigned to nodeii\. The Laplacian quadratic form expands as

𝐱⊤​L​𝐱\\displaystyle\\mathbf\{x\}^\{\\top\}L\\mathbf\{x\}=∑iDi​i​xi2−∑i,jWi​j​xi​xj\\displaystyle=\\sum\_\{i\}D\_\{ii\}x\_\{i\}^\{2\}\-\\sum\_\{i,j\}W\_\{ij\}x\_\{i\}x\_\{j\}=12​∑i,jWi​j​\(xi−xj\)2\.\\displaystyle=\\frac\{1\}\{2\}\\sum\_\{i,j\}W\_\{ij\}\(x\_\{i\}\-x\_\{j\}\)^\{2\}\.Thus,𝐱⊤​L​𝐱\\mathbf\{x\}^\{\\top\}L\\mathbf\{x\}measures the roughness of the nodewise signal over the graph: differences across node pairs joined by larger weights contribute more strongly to the quadratic form\.

This identity also motivates the graph Fourier interpretation of Laplacian eigenvectors\. Writing

L\\displaystyle L=U​Λ​U⊤,\\displaystyle=U\\Lambda U^\{\\top\},Λ\\displaystyle\\Lambda=diag⁡\(μ1,…,μN\),𝐱^=U⊤​𝐱,\\displaystyle=\\operatorname\{diag\}\(\\mu\_\{1\},\\ldots,\\mu\_\{N\}\),\\quad\\widehat\{\\mathbf\{x\}\}=U^\{\\top\}\\mathbf\{x\},where0≤μ1≤⋯≤μN0\\leq\\mu\_\{1\}\\leq\\cdots\\leq\\mu\_\{N\}, let𝐮m\\mathbf\{u\}\_\{m\}denote themm\-th column ofUU\. These eigenvectors form the graph Fourier basis used in[Section˜3\.2](https://arxiv.org/html/2608.03015#S3.SS2), andx^m=𝐮m⊤​𝐱\\widehat\{x\}\_\{m\}=\\mathbf\{u\}\_\{m\}^\{\\top\}\\mathbf\{x\}is the graph Fourier coefficient of𝐱\\mathbf\{x\}at modemm\. The quadratic form can then be written as

𝐱⊤​L​𝐱=𝐱⊤​U​Λ​U⊤​𝐱=𝐱^⊤​Λ​𝐱^=∑m=1Nμm​x^m2\.\\mathbf\{x\}^\{\\top\}L\\mathbf\{x\}=\\mathbf\{x\}^\{\\top\}U\\Lambda U^\{\\top\}\\mathbf\{x\}=\\widehat\{\\mathbf\{x\}\}^\{\\top\}\\Lambda\\widehat\{\\mathbf\{x\}\}=\\sum\_\{m=1\}^\{N\}\\mu\_\{m\}\\widehat\{x\}\_\{m\}^\{2\}\.In particular,𝐮m⊤​L​𝐮m=μm\\mathbf\{u\}\_\{m\}^\{\\top\}L\\mathbf\{u\}\_\{m\}=\\mu\_\{m\}, soμm\\mu\_\{m\}gives the graph variation of the corresponding Fourier mode\. BecauseL​𝟏=𝟎L\\mathbf\{1\}=\\mathbf\{0\}andLLis positive semidefinite, its smallest eigenvalue isμ1=0\\mu\_\{1\}=0\. More generally, the multiplicity of the zero eigenvalue equals the number of connected components of the graph\. Thus, for a connected graph, the zero eigenvalue is simple and the second\-smallest eigenvalue satisfiesμ2\>0\\mu\_\{2\}\>0\. The full weighted attention graphs used in our spectral diagnostics are connected\. This connectedness condition applies throughout our analysis\. Ordering the eigenvectors by increasing eigenvalue therefore orders the graph Fourier modes from low frequency, corresponding to smooth variation across strongly weighted edges, to high frequency, corresponding to sharper variation\.

The same interpretation extends to vector\-valued hidden\-state signals\. Suppose each nodeiiis assigned a hidden\-state vectorXi∈ℝdX\_\{i\}\\in\\mathbb\{R\}^\{d\}, and collect these vectors as the rows of a matrixX∈ℝN×dX\\in\\mathbb\{R\}^\{N\\times d\}\. Applying the scalar identity to each hidden dimension gives

tr⁡\(X⊤​L​X\)=12​∑i,jWi​j​‖Xi−Xj‖22\.\\operatorname\{tr\}\(X^\{\\top\}LX\)=\\frac\{1\}\{2\}\\sum\_\{i,j\}W\_\{ij\}\\\|X\_\{i\}\-X\_\{j\}\\\|\_\{2\}^\{2\}\.Equivalently, ifX^=U⊤​X\\widehat\{X\}=U^\{\\top\}X, then

tr⁡\(X⊤​L​X\)=∑m=1Nμm​‖𝐱^m‖22,\\operatorname\{tr\}\(X^\{\\top\}LX\)=\\sum\_\{m=1\}^\{N\}\\mu\_\{m\}\\\|\\widehat\{\\mathbf\{x\}\}\_\{m\}\\\|\_\{2\}^\{2\},where𝐱^m\\widehat\{\\mathbf\{x\}\}\_\{m\}denotes themm\-th row ofX^\\widehat\{X\}\. In our setting,W=W¯\(ℓ\)W=\\bar\{W\}^\{\(\\ell\)\}is the attention\-derived weighted adjacency matrix andX=X\(ℓ\)X=X^\{\(\\ell\)\}is the hidden\-state matrix entering layerℓ\\ell\. Therefore, the Laplacian quadratic form quantifies how rapidly hidden\-state representations vary across attention\-weighted token interactions, and the Laplacian eigenvectors provide the graph Fourier modes used in the spectral diagnostics\.

BecauseUUis orthonormal, Parseval’s identity gives

‖X‖F2=‖X^‖F2=∑m=1N‖𝐱^m‖22\.\\\|X\\\|\_\{F\}^\{2\}=\\\|\\widehat\{X\}\\\|\_\{F\}^\{2\}=\\sum\_\{m=1\}^\{N\}\\\|\\widehat\{\\mathbf\{x\}\}\_\{m\}\\\|\_\{2\}^\{2\}\.This identity justifies treating‖𝐱^m‖22\\\|\\widehat\{\\mathbf\{x\}\}\_\{m\}\\\|\_\{2\}^\{2\}as the energy assigned to modemmin the HFER and spectral entropy diagnostics\.

## Appendix DRobustness and Layerwise Consistency of Graph\-Spectral Diagnostics

### D\.1Robustness of Graph\-Spectral Diagnostics

[Section˜3\.2](https://arxiv.org/html/2608.03015#S3.SS2)defines the normalized Fiedler value as the second\-smallest normalized Laplacian eigenvalue and HFER as the fraction of hidden\-state energy in the highest\-frequency 50% of graph Fourier modes\. Here, we test whether the observed Fiedler\-value separation extends to multiway graph structure and whether the HFER trends are robust to the choice of high\-frequency cutoff by examining additional low\-end eigenvalues and alternative HFER cutoffs\.

#### Low\-end normalized Laplacian spectrum\.

Let the ordered eigenvalues of the layer\-ℓ\\ellnormalized Laplacianℒ\(ℓ\)\\mathcal\{L\}^\{\(\\ell\)\}be

0=λ1\(ℓ\)≤λ2\(ℓ\)≤⋯≤λN\(ℓ\)≤2\.0=\\lambda\_\{1\}^\{\(\\ell\)\}\\leq\\lambda\_\{2\}^\{\(\\ell\)\}\\leq\\cdots\\leq\\lambda\_\{N\}^\{\(\\ell\)\}\\leq 2\.Writingλk\\lambda\_\{k\}without a layer superscript for the layer average, we define

λk:=1L​∑ℓ=1Lλk\(ℓ\)\.\\lambda\_\{k\}:=\\frac\{1\}\{L\}\\sum\_\{\\ell=1\}^\{L\}\\lambda\_\{k\}^\{\(\\ell\)\}\.Thus,λ2\\lambda\_\{2\}is the layer\-averaged normalized Fiedler value reported in the main analysis\. To determine whether the observed separation extends beyond the two\-way connectivity captured byλ2\\lambda\_\{2\}, we examine the first ten nontrivial eigenvalues,k=2,…,11k=2,\\ldots,11\. The additional eigenvalues at the low end of the normalized Laplacian spectrum are related to multiway sparse\-partition structure through higher\-order Cheeger inequalities\(Leeet al\.,[2014](https://arxiv.org/html/2608.03015#bib.bib40)\)\. We focus onN=2000N=2000, where[Fig\.˜3](https://arxiv.org/html/2608.03015#S4.F3)in[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1)shows a clear separation inλ2\\lambda\_\{2\}between the chaotic and non\-chaotic inputs\.[Fig\.˜7](https://arxiv.org/html/2608.03015#A4.F7)shows that this separation persists throughout the examined low end of the spectrum: the chaotic logistic map and Lorenz inputs remain systematically below the constant and periodic inputs fork=2,…,11k=2,\\ldots,11\. This systematic downward shift is consistent with a greater tendency toward multiple weakly connected graph regions under chaotic inputs, indicating that the reduced global integration observed throughλ2\\lambda\_\{2\}is not confined to a single eigenvalue\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x8.png)Figure 7:Low end of the normalized Laplacian spectrum atN=2000N=2000, averaged across layers\. For eachk=2,…,11k=2,\\ldots,11, curves show the meanλk\\lambda\_\{k\}across 20 realizations, and shading denotes±1\\pm 1standard deviation\. The chaotic logistic map and Lorenz inputs remain below the constant and periodic inputs throughout this range, showing that the separation observed in the normalized Fiedler value in[Fig\.˜3](https://arxiv.org/html/2608.03015#S4.F3)extends across the low end of the spectrum\.![Refer to caption](https://arxiv.org/html/2608.03015v1/x9.png)Figure 8:Layer\-averaged HFER cutoff sensitivity\. The panels reportHFERq\\mathrm\{HFER\}\_\{q\}using the highest\-frequency 40%, 30%, and 20% of graph Fourier modes\. Under the same evaluation setup as[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1), curves and shading show the mean and±1\\pm 1standard deviation across 20 realizations\. Across all three cutoffs, chaotic inputs exhibit higher HFER at longer contexts, while the non\-chaotic inputs retain the periodicity\-dependent ladder observed with the default 50% high\-frequency tail in[Fig\.˜3](https://arxiv.org/html/2608.03015#S4.F3)\. The qualitative separation is therefore robust to the number of high\-frequency modes included\.![Refer to caption](https://arxiv.org/html/2608.03015v1/x10.png)\(i\) Final layer across context lengths
![Refer to caption](https://arxiv.org/html/2608.03015v1/x11.png)\(ii\) All layers atN=2000N=2000

Figure 9:Layer\-specific graph\-spectral diagnostics for the main numerical input families using Llama\-3\.2\-3B\. Row \(i\) shows final\-layer diagnostics across context lengths, while row \(ii\) shows the corresponding diagnostics across all 28 model layers atN=2000N=2000\. Within each row, panels \(a\)–\(c\) report normalized Fiedler valueλ2\\lambda\_\{2\}, HFER, and effective spectral supportexp⁡\(SE\)\\exp\(\\mathrm\{SE\}\), respectively\. Curves and shading denote the mean and±1\\pm 1standard deviation across the same 20 realizations analyzed in[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1)\. The final\-layer results reproduce the broad context\- and complexity\-dependent behavior of the layer\-averaged analysis, although the single\-layer curves are less smooth\. In the fixed\-context depth profiles, most layers exhibit the same broad complexity\-dependent ordering as the layer\-averaged and final\-layer results, with only a few of the earliest layers showing weaker separation\.
#### HFER cutoff sensitivity\.

To test whether the observed HFER trends depend on the default 50% high\-frequency tail, letMq=round⁡\(q​N\)M\_\{q\}=\\operatorname\{round\}\(qN\)andKq=N−MqK\_\{q\}=N\-M\_\{q\}, whereMqM\_\{q\}is the number of highest\-frequency modes retained for tail fractionqq\. We define

HFERq\(ℓ\)\\displaystyle\\mathrm\{HFER\}^\{\(\\ell\)\}\_\{q\}=∑m=Kq\+1Npm\(ℓ\),\\displaystyle=\\sum\_\{m=K\_\{q\}\+1\}^\{N\}p\_\{m\}^\{\(\\ell\)\},HFERq\\displaystyle\\mathrm\{HFER\}\_\{q\}=1L​∑ℓ=1LHFERq\(ℓ\)\.\\displaystyle=\\frac\{1\}\{L\}\\sum\_\{\\ell=1\}^\{L\}\\mathrm\{HFER\}^\{\(\\ell\)\}\_\{q\}\.Thus,q=0\.4q=0\.4,0\.30\.3, and0\.20\.2retain the highest\-frequency 40%, 30%, and 20% of graph Fourier modes, corresponding to cutoffs atKq=0\.6​NK\_\{q\}=0\.6N,0\.7​N0\.7N, and0\.8​N0\.8N, respectively, for the evaluated context lengths\. The default diagnostic in[Section˜3\.2](https://arxiv.org/html/2608.03015#S3.SS2)isHFER0\.5\\mathrm\{HFER\}\_\{0\.5\}\. As shown in[Fig\.˜3](https://arxiv.org/html/2608.03015#S4.F3)of[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1), at longer contexts this default separates the higher\-complexity chaotic inputs from the simpler constant and periodic inputs and reveals a clear ladder among the periodic inputs\.[Fig\.˜8](https://arxiv.org/html/2608.03015#A4.F8)shows the same qualitative behavior forq=0\.4q=0\.4,0\.30\.3, and0\.20\.2: chaotic inputs retain higher HFER, while the periodic inputs preserve their period\-dependent ordering\. The persistence of these trends under increasingly selective high\-frequency tails indicates that the observed behavior is not specific to the default 50% high\-frequency tail\.

### D\.2Layerwise Graph\-Spectral Diagnostics

The main analysis in[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1)reports graph\-spectral diagnostics averaged over transformer layers, providing a compact summary of how the diagnostic patterns vary across input families and context lengths\. Here, we present two complementary views of the corresponding layerwise behavior\. We first show the final\-layer diagnostics across context lengths as an illustrative layer\-specific example, and then examine all layers atN=2000N=2000to characterize how the same broad input\-family structure is distributed across model depth\. Together, these views show that the layer\-averaged results provide a holistic summary of the recurring layerwise behavior while smoothing layer\-specific variation\.

#### Final\-layer context scaling\.

The top row of[Fig\.˜9](https://arxiv.org/html/2608.03015#A4.F9)presents the three diagnostics at the final model layer as an illustrative layer\-specific example\. At longer contexts, the final\-layer normalized Fiedler value primarily separates chaotic from non\-chaotic inputs, whereas HFER and effective spectral support also reveal a finer period\-dependent ordering among the periodic families\. These patterns are broadly consistent with the layer\-averaged analysis, although the single\-layer curves are less smooth\.

#### Consistency across model depth\.

Because presenting a separate context\-length sweep for each layer of anLL\-layer model would requireLLfigures, we complement the final\-layer sweep with a fixed\-context depth profile that compares all layers in a single view\. The bottom row of[Fig\.˜9](https://arxiv.org/html/2608.03015#A4.F9)reports the normalized Fiedler value, HFER, and effective spectral support across all model layers atN=2000N=2000\. Overall, the layerwise profiles are consistent with the layer\-averaged and final\-layer results, with most layers exhibiting the same broad complexity\-dependent ordering\.

## Appendix EAttention\-Induced Token Graphs: Construction and Visualization Robustness

#### Attention matrices\.

[Fig\.˜10](https://arxiv.org/html/2608.03015#A5.F10)shows representative final\-layer, head\-averaged post\-softmax attention matrices for the main numerical input families\. These matrices are symmetrized to form the weighted adjacency matrices used in the spectral diagnostics and graph visualizations\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x12.png)Figure 10:Last\-layer attention matrices for Llama\-3\.2\-3B across the main numerical input families atN=500N=500\. Each panel shows the head\-averaged matrix for a representative input\.![Refer to caption](https://arxiv.org/html/2608.03015v1/x13.png)Figure 11:Last\-layer hidden\-state projections for Llama\-3\.2\-3B across the main numerical input families atN=500N=500\. Each panel shows token hidden states projected onto the leading three uncentered SVD coordinates, which define the RGB node colors used in the attention\-induced graph visualizations\.
#### Hidden\-state projections\.

LetX\(ℓ\)∈ℝN×dX^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{N\\times d\}contain the token hidden states at layerℓ\\ell, with one token per row\. We compute the uncentered singular value decompositionX\(ℓ\)=UX\(ℓ\)​ΣX\(ℓ\)​VX\(ℓ\)⊤X^\{\(\\ell\)\}=U\_\{X\}^\{\(\\ell\)\}\\Sigma\_\{X\}^\{\(\\ell\)\}V\_\{X\}^\{\(\\ell\)\\top\}, where the subscriptXXdistinguishes these hidden\-state SVD quantities from the graph Laplacian eigenvectorsU\(ℓ\)U^\{\(\\ell\)\}introduced in[Section˜3\.2](https://arxiv.org/html/2608.03015#S3.SS2)\. The full matrix of token\-level SVD coordinates is

Z\(ℓ\)=UX\(ℓ\)​ΣX\(ℓ\)=X\(ℓ\)​VX\(ℓ\)\.Z^\{\(\\ell\)\}=U\_\{X\}^\{\(\\ell\)\}\\Sigma\_\{X\}^\{\(\\ell\)\}=X^\{\(\\ell\)\}V\_\{X\}^\{\(\\ell\)\}\.To construct the node colors, we assign the first three columns ofZ\(ℓ\)Z^\{\(\\ell\)\}to the red, green, and blue channels, respectively\. For eachk∈\{1,2,3\}k\\in\\\{1,2,3\\\}, letmk\(ℓ\)=minj⁡Zj​k\(ℓ\)m\_\{k\}^\{\(\\ell\)\}=\\min\_\{j\}Z\_\{jk\}^\{\(\\ell\)\}andMk\(ℓ\)=maxj⁡Zj​k\(ℓ\)M\_\{k\}^\{\(\\ell\)\}=\\max\_\{j\}Z\_\{jk\}^\{\(\\ell\)\}\. Using a numerical tolerance ofε=10−12\\varepsilon=10^\{\-12\}, we independently min–max scale each coordinate across nodes:

Ci​k\(ℓ\)=\{Zi​k\(ℓ\)−mk\(ℓ\)Mk\(ℓ\)−mk\(ℓ\),Mk\(ℓ\)−mk\(ℓ\)\>ε,1/2,otherwise\.C\_\{ik\}^\{\(\\ell\)\}=\\begin\{cases\}\\displaystyle\\frac\{Z\_\{ik\}^\{\(\\ell\)\}\-m\_\{k\}^\{\(\\ell\)\}\}\{M\_\{k\}^\{\(\\ell\)\}\-m\_\{k\}^\{\(\\ell\)\}\},&M\_\{k\}^\{\(\\ell\)\}\-m\_\{k\}^\{\(\\ell\)\}\>\\varepsilon,\\\\\[9\.0pt\] 1/2,&\\text\{otherwise\}\.\\end\{cases\}Tokeniiis then assigned the RGB color𝐜i\(ℓ\)=\(Ci​k\(ℓ\)\)k=13∈\[0,1\]3\\mathbf\{c\}\_\{i\}^\{\(\\ell\)\}=\(C\_\{ik\}^\{\(\\ell\)\}\)\_\{k=1\}^\{3\}\\in\[0,1\]^\{3\}\. The fraction of hidden\-state energy captured by the firstqqcomponents is

ηq\(ℓ\)=∑k=1q\(σX,k\(ℓ\)\)2∑k=1r\(σX,k\(ℓ\)\)2=∑k=1q\(σX,k\(ℓ\)\)2∥X\(ℓ\)∥F2\.\\eta\_\{q\}^\{\(\\ell\)\}\\,=\\,\\frac\{\\displaystyle\\sum\_\{k=1\}^\{q\}\\bigl\(\\sigma\_\{X,k\}^\{\(\\ell\)\}\\bigr\)^\{2\}\}\{\\displaystyle\\sum\_\{k=1\}^\{r\}\\bigl\(\\sigma\_\{X,k\}^\{\(\\ell\)\}\\bigr\)^\{2\}\}\\,=\\,\\frac\{\\displaystyle\\sum\_\{k=1\}^\{q\}\\bigl\(\\sigma\_\{X,k\}^\{\(\\ell\)\}\\bigr\)^\{2\}\}\{\\lVert X^\{\(\\ell\)\}\\rVert\_\{F\}^\{2\}\}\.Here,r=min⁡\(N,d\)r=\\min\(N,d\)is the number of singular values\. AtN=500N=500, averaging first across the main numerical input families within each layer and then across the analyzed layers givesη3=0\.75\\eta\_\{3\}=0\.75\.[Fig\.˜11](https://arxiv.org/html/2608.03015#A5.F11)shows the resulting final\-layer projections for all ten families, while[Fig\.˜12](https://arxiv.org/html/2608.03015#A5.F12)includes the layer\-averaged cumulative SVD\-energy diagnostic and its variation across layers\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x14.png)Figure 12:Cumulative hidden\-state SVD\-energy fraction for Llama\-3\.2\-3B atN=500N=500, averaged first across the main numerical input families within each layer and then across the analyzed layers\. The leading three components capture 75% of the hidden\-state energy on average, supporting their use as RGB coordinates for token\-node colors\.![Refer to caption](https://arxiv.org/html/2608.03015v1/x15.png)Figure 13:Evolution of attention\-based node positions for Llama\-3\.2\-3B atN=500N=500\. Representative constant and chaotic logistic map inputs are shown at initialization and after 10, 30, 50, and 100 FR iterations\. Both begin from the same token\-ordered spiral, so subsequent differences reflect their input\-specific attention weights\. Most visible reorganization occurs by 50 iterations, with only minor changes thereafter; we use the 100\-iteration layouts for subsequent token\-graph visualizations\.
#### Layout computation\.

We initialize nodeiion a token\-ordered spiral by settingti=\(i−1\)/\(N−1\)t\_\{i\}=\(i\-1\)/\(N\-1\),θi=−π/2\+2​π​KN​ti\\theta\_\{i\}=\-\\pi/2\+2\\pi K\_\{N\}t\_\{i\}, and

𝐩i=\(0\.15\+0\.85​ti\)​\[cos⁡θisin⁡θi\],\\mathbf\{p\}\_\{i\}=\(0\.15\+0\.85t\_\{i\}\)\\begin\{bmatrix\}\\cos\\theta\_\{i\}\\\\ \\sin\\theta\_\{i\}\\end\{bmatrix\},whereKN=2\.25K\_\{N\}=2\.25forN<80N<80andKN=2\.75K\_\{N\}=2\.75otherwise\. We center the resulting coordinates and rescale them to unit maximum radius\. This construction produces the initialization shown in the leftmost panels of[Fig\.˜13](https://arxiv.org/html/2608.03015#A5.F13)\. Because it depends only onNNand token order, all inputs of the same length begin from identical node positions; their input\-specific attention matricesW¯\(ℓ\)\\bar\{W\}^\{\(\\ell\)\}then determine the attractive forces that drive the subsequent reorganization\. We compute the resulting positions using the implementation of the FR algorithm provided by the NetworkX Python library\(Fruchterman and Reingold,[1991](https://arxiv.org/html/2608.03015#bib.bib34); Hagberget al\.,[2008](https://arxiv.org/html/2608.03015#bib.bib10)\), with all off\-diagonal entries ofW¯\(ℓ\)\\bar\{W\}^\{\(\\ell\)\}used as edge weights\. We run 100 iterations rather than the implementation’s default of 50 to allow further relaxation toward a stable configuration\.

#### Layout evolution and iteration count\.

[Figure˜13](https://arxiv.org/html/2608.03015#A5.F13)shows the evolution of the FR layout from the shared token\-ordered spiral for representative constant and chaotic logistic map inputs\. Most visible layout reorganization occurs by 50 iterations, with only minor changes thereafter\. We therefore use the 100\-iteration layouts throughout the token\-graph visualizations\.

#### Initialization robustness\.

To evaluate sensitivity to the exact initialization, we add independent Gaussian perturbationsϵi∼𝒩​\(𝟎,σ2​I2\)\\boldsymbol\{\\epsilon\}\_\{i\}\\sim\\mathcal\{N\}\(\\mathbf\{0\},\\sigma^\{2\}I\_\{2\}\)to the normalized spiral coordinates, reapply the same normalization, and rerun the 100\-iteration layout\.[Fig\.˜14](https://arxiv.org/html/2608.03015#A5.F14)comparesσ∈\{0\.01,0\.05,0\.10\}\\sigma\\in\\\{0\.01,0\.05,0\.10\\\}for representative constant and chaotic logistic map inputs\. Their broad organization remains stable across these perturbation magnitudes, indicating that the qualitative separation is not specific to the initial spiral coordinates\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x16.png)Figure 14:Layout\-initialization robustness of last\-layer attention\-graph visualizations for Llama\-3\.2\-3B atN=500N=500\. Constant and chaotic logistic map examples are initialized from noisy spiral coordinates with perturbation standard deviations0\.010\.01,0\.050\.05, and0\.100\.10, then compared before and after FR layout relaxation\. The qualitative organization is preserved across perturbation scales\.![Refer to caption](https://arxiv.org/html/2608.03015v1/x17.png)Figure 15:Threshold robustness of last\-layer attention\-graph visualizations for Llama\-3\.2\-3B atN=500N=500\. Representative constant and chaotic logistic map inputs are shown forκ∈\{2,4,8\}\\kappa\\in\\\{2,4,8\\\}, with display thresholdκ/N\\kappa/N\. The main visualizations useκ=2\.5\\kappa=2\.5\. The qualitative structure remains stable across thresholds\.
#### Edge\-display threshold sensitivity\.

[Fig\.˜15](https://arxiv.org/html/2608.03015#A5.F15)evaluates sensitivity to the multiplierκ\\kappain the edge\-display thresholdW¯i​j\(ℓ\)\>κ/N\\bar\{W\}^\{\(\\ell\)\}\_\{ij\}\>\\kappa/N\. Because this threshold affects only edge rendering, node positions, hidden\-state colors, and graph\-spectral diagnostics remain unchanged across panels\. For both the representative constant and chaotic logistic map inputs, the qualitative structure of the displayed edge patterns remains consistent acrossκ∈\{2,4,8\}\\kappa\\in\\\{2,4,8\\\}and with the default visualization atκ=2\.5\\kappa=2\.5\.

## Appendix FAttention\-Induced Token\-Graph Visualizations at Additional Transformer Layers

[Section˜4\.2](https://arxiv.org/html/2608.03015#S4.SS2)focuses on last\-layer attention graphs because the final layer is closest to the prediction\-facing representation used by the model\.[Fig\.˜16](https://arxiv.org/html/2608.03015#A6.F16)uses layers 1 and 12 as illustrative snapshots of early and middle depth, respectively\. In the layer\-1 example, differentiation across input families is comparatively limited: although the graphs are not identical, their layouts share broad organization associated with numeric\-versus\-separator token roles and local sequence structure\. In the layer\-12 example, input\-dependent differences are more pronounced and broadly track dynamical complexity: constant and lower\-period inputs retain relatively regular, globally integrated layouts, whereas higher\-period, chaotic logistic map, and Lorenz inputs exhibit less homogeneous organization and more pronounced mesoscale partitioning into localized subnetworks\. These two layer\-specific snapshots illustrate how cross\-input differentiation can become more pronounced with depth; they do not establish a universal transition point or imply that all early and middle layers follow the same progression\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x18.png)\(a\)Transformer layer 1
![Refer to caption](https://arxiv.org/html/2608.03015v1/x19.png)\(b\)Transformer layer 12

Figure 16:Layerwise attention\-induced token graphs atN=500N=500for the main numerical input families examined in[Section˜4\.2](https://arxiv.org/html/2608.03015#S4.SS2)\.\(a\)Layer 1 provides an illustrative early\-depth snapshot in which cross\-input differentiation in graph layout is comparatively limited and the layouts retain broadly similar local organization\.\(b\)Layer 12 provides an illustrative middle\-depth snapshot in which cross\-input differentiation is more pronounced, with graph organization tending to become less uniform as input dynamical complexity increases\.
## Appendix GGraph\-Spectral Diagnostics Across Model Families, Scales, and Instruction Tuning

We investigate whether the graph\-spectral behavior observed in the main analysis persists across model families and scales, and how instruction tuning affects this behavior\. We first examine variation across model scale and instruction tuning within the Llama 3 family\. We then extend the analysis to Phi\-4 and SmolLM3, including a comparison between the base and instruction\-tuned SmolLM3 variants\.

#### Variation across model scales within the Llama 3 family\.

Using the same experimental setup as[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1),[Fig\.˜17](https://arxiv.org/html/2608.03015#A7.F17)\(a\)–\(b\) compares the layer\-averaged diagnostics for Llama\-3\.2\-1B and Llama\-3\.1\-8B with the primary Llama\-3\.2\-3B results\. The 8B model exhibits behavior highly similar to that of the 3B model, preserving the broad trends and separation among input families across the three diagnostics\. The 1B model, however, distinguishes the input families less clearly\. In particular, itsλ2\\lambda\_\{2\}andexp⁡\(SE\)\\exp\(\\mathrm\{SE\}\)results show less clear separation between the chaotic and high\-period periodic inputs, while its HFER exhibits a less distinct complexity\-ordered progression among the lower\-period inputs\. These results suggest that the smaller 1B model retains the broad context\-dependent graph\-spectral trends but less consistently resolves fine\-grained differences between dynamical regimes\.

#### Instruction\-tuned model comparison\.

We repeat the experimental setup from[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1)with the instruction\-tuned variants of the 3B model used in the main analysis and the 8B model evaluated in the preceding scale comparison: Llama\-3\.2\-3B\-Instruct and Llama\-3\.1\-8B\-Instruct\. Llama\-3 Instruct models undergo alignment\-focused post\-training to support helpful and safe instruction following\(Grattafioriet al\.,[2024](https://arxiv.org/html/2608.03015#bib.bib4)\)\. We focus on the 3B and 8B models because the preceding scale comparison shows weaker input\-family separation for the 1B base model\. As shown in[Fig\.˜17](https://arxiv.org/html/2608.03015#A7.F17)\(c\)–\(d\), both instruction\-tuned models retain the principal context\- and complexity\-dependent behavior of their base counterparts, although some fine\-grained distinctions between high\-period periodic and chaotic inputs become less pronounced, particularly inexp⁡\(SE\)\\exp\(\\mathrm\{SE\}\)\. This weakening is consistent with output\-level evidence that alignment\-related post\-training can reduce extrapolation accuracy for time series\(Gruveret al\.,[2023](https://arxiv.org/html/2608.03015#bib.bib1)\)and spatiotemporal dynamics\(Baoet al\.,[2026](https://arxiv.org/html/2608.03015#bib.bib18)\)\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x20.png)

\(a\) Llama\-3\.1\-8B![Refer to caption](https://arxiv.org/html/2608.03015v1/x21.png)

\(b\) Llama\-3\.2\-1B![Refer to caption](https://arxiv.org/html/2608.03015v1/x22.png)

\(c\) Llama\-3\.1\-8B\-Instruct![Refer to caption](https://arxiv.org/html/2608.03015v1/x23.png)

\(d\) Llama\-3\.2\-3B\-InstructFigure 17:Layer\-averaged graph\-spectral diagnostics across base\-model scales and instruction\-tuned variants within the Llama 3 family, under the same experimental setup as[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1)\. From left to right, the columns report normalized Fiedler valueλ2\\lambda\_\{2\}, HFER, andexp⁡\(SE\)\\exp\(\\mathrm\{SE\}\)as functions of context length\. Compared with the Llama\-3\.2\-3B results in[Fig\.˜3](https://arxiv.org/html/2608.03015#S4.F3), the 8B base model exhibits highly similar behavior, whereas the 1B base model less clearly distinguishes the dynamical regimes\. Both instruction\-tuned models retain the broad behavior of their base counterparts, but some finer distinctions among input families become less pronounced with instruction tuning\.
#### Additional model families\.

Using the same experimental setup as[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1),[Fig\.˜18](https://arxiv.org/html/2608.03015#A7.F18)presents the layer\-averaged diagnostics for Phi\-4 and the base and instruction\-tuned SmolLM3\-3B variants\. At the time of our experiments, only the post\-trained, chat\-optimized Phi\-4 checkpoint was publicly available; unlike Llama 3 and SmolLM3, Phi\-4 had no corresponding publicly released base\-model checkpoint\. Across all three models, the broad context\- and complexity\-dependent behavior remains consistent with the Llama 3 results\. SmolLM3\-3B\-Base closely reproduces the ordered input\-family behavior of the 3B and 8B Llama base models, whereas Phi\-4 and instruction\-tuned SmolLM3\-3B show less clear separation between high\-period periodic and chaotic inputs, similar to the instruction\-tuned Llama variants\. The absolute diagnostic magnitudes and rates of change with context differ across models, indicating that the detailed graph\-spectral signatures are model\-dependent even though their overall ordering and trends remain broadly consistent\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x24.png)

\(a\) Phi\-4![Refer to caption](https://arxiv.org/html/2608.03015v1/x25.png)

\(b\) SmolLM3\-3B\-Base![Refer to caption](https://arxiv.org/html/2608.03015v1/x26.png)

\(c\) SmolLM3\-3B \(instruction\-tuned\)Figure 18:Layer\-averaged graph\-spectral diagnostics for additional model families under the same experimental setup as[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1)\. From left to right, the columns report normalized Fiedler valueλ2\\lambda\_\{2\}, HFER, andexp⁡\(SE\)\\exp\(\\mathrm\{SE\}\)as functions of context length\. SmolLM3\-3B\-Base reproduces the broad context\- and complexity\-dependent behavior observed in the Llama 3 models\. Phi\-4, a post\-trained chat model, and instruction\-tuned SmolLM3\-3B also retain the overall trends but exhibit less consistent fine\-grained separation among high\-period and chaotic inputs\. Diagnostic magnitudes and rates of change with context length differ across models\.

## Appendix HAttention\-Only and Hidden\-State\-Only Non\-Graph Baselines

We further show that the broad context\- and complexity\-dependent trends captured by the graph\-spectral diagnostics are not artifacts of the graph\-signal construction\. Analogous summaries derived from attention alone or hidden states alone exhibit similar behavior as context length increases and across input families, although they provide less fine\-grained separation among families\. As non\-graph controls, we consider two hidden\-state\-only summaries based on fixed frequencies over serialized token position and one attention\-only summary based on attention concentration\. Following the notation of[Section˜3\.2](https://arxiv.org/html/2608.03015#S3.SS2), letA\(ℓ,h\)∈ℝN×NA^\{\(\\ell,h\)\}\\in\\mathbb\{R\}^\{N\\times N\}denote the post\-softmax attention matrix for layerℓ\\elland headhh, and letX\(ℓ\)∈ℝN×dX^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{N\\times d\}denote the hidden\-state matrix entering layerℓ\\ell\. The positional controls use onlyX\(ℓ\)X^\{\(\\ell\)\}, whereas the attention control uses onlyA\(ℓ,h\)A^\{\(\\ell,h\)\}\.

#### Normalized attention Gini coefficient\.

The Gini coefficient measures the concentration of a nonnegative vector\(Hurley and Rickard,[2009](https://arxiv.org/html/2608.03015#bib.bib41)\)\. For a nonzero vector𝐳∈ℝ≥0m\\mathbf\{z\}\\in\\mathbb\{R\}\_\{\\geq 0\}^\{m\}, define

g​\(𝐳\)=∑r=1m∑s=1m\|zr−zs\|2​m​∑r=1mzr\.g\(\\mathbf\{z\}\)=\\frac\{\\sum\_\{r=1\}^\{m\}\\sum\_\{s=1\}^\{m\}\|z\_\{r\}\-z\_\{s\}\|\}\{2m\\sum\_\{r=1\}^\{m\}z\_\{r\}\}\.This coefficient satisfiesg​\(𝐳\)∈\[0,1−1/m\]g\(\\mathbf\{z\}\)\\in\[0,1\-1/m\]: it equals0if and only if the entries of𝐳\\mathbf\{z\}are uniform, and reaches1−1/m1\-1/mwhen all mass is concentrated on a single entry\. For query positioniiand headhh, let𝐚i\(ℓ,h\)=\(Ai​1\(ℓ,h\),…,Ai​i\(ℓ,h\)\)\\mathbf\{a\}\_\{i\}^\{\(\\ell,h\)\}=\(A\_\{i1\}^\{\(\\ell,h\)\},\\ldots,A\_\{ii\}^\{\(\\ell,h\)\}\)contain its causally available attention weights\. We normalize by the maximum attainable for each row length and average across query positions and heads:

Gini\(ℓ\)=1H​\(N−1\)​∑h=1H∑i=2Ng​\(𝐚i\(ℓ,h\)\)1−1/i\.\\mathrm\{Gini\}^\{\(\\ell\)\}=\\frac\{1\}\{H\(N\-1\)\}\\sum\_\{h=1\}^\{H\}\\sum\_\{i=2\}^\{N\}\\frac\{g\\\!\\left\(\\mathbf\{a\}\_\{i\}^\{\(\\ell,h\)\}\\right\)\}\{1\-1/i\}\.The normalization makes rows with different numbers of available keys comparable and givesGini\(ℓ\)∈\[0,1\]\\mathrm\{Gini\}^\{\(\\ell\)\}\\in\[0,1\]; the first position is omitted because it has only one available key\. A value of0means that every included query distributes attention uniformly over its available keys, whereas a value of11means that every query concentrates all attention on a single key\. Intermediate values quantify the average degree of attention concentration\. In this sense, attention Gini is an attention\-only control related to the normalized Fiedler valueλ2\\lambda\_\{2\}from our graph\-spectral diagnostics: greater rowwise concentration may accompany the more localized attention\-graph structure indicated by smallerλ2\\lambda\_\{2\}, but Gini does not measure how those attention links combine into a globally connected graph\.

#### Fixed positional\-frequency representation\.

The two hidden\-state controls measure how hidden states vary across token positions in the serialized input, without using attention\. We construct this positional\-frequency representation using an orthonormal type\-II discrete cosine transform \(DCT\-II\)\(Ahmedet al\.,[1974](https://arxiv.org/html/2608.03015#bib.bib42)\)\. Unlike a standard discrete Fourier transform, the DCT\-II does not impose periodic wraparound between the first and last token positions, while its orthonormality preserves total squared hidden\-state energy\. In implementation, we apply SciPy’sscipy\.fft\.dctwithtype=2,norm="ortho", andaxis=0, transforming the token dimension independently for each hidden\-state coordinate\. We write the resulting coefficients as

X~pos\(ℓ\)=DCTII⁡\(X\(ℓ\)\)\.\\widetilde\{X\}\_\{\\mathrm\{pos\}\}^\{\(\\ell\)\}=\\operatorname\{DCT\}\_\{\\mathrm\{II\}\}\\\!\\left\(X^\{\(\\ell\)\}\\right\)\.Let𝐱~pos,k\(ℓ\)∈ℝd\\widetilde\{\\mathbf\{x\}\}\_\{\\mathrm\{pos\},k\}^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{d\}denote the coefficient vector for positional modekk\. These modes are ordered from slow to rapid variation in token hidden states across serialized positions, rather than by temporal frequency in the underlying numerical trajectory\. The fixed DCT basis thus provides a hidden\-state\-only counterpart to the graph Fourier basis used in our main diagnostics: both decompose hidden states into frequency\-ordered modes, but the DCT basis is determined only by context length and serialized token order, whereas the graph Fourier basis is determined by the attention\-induced token geometry\.

#### Positional HFER\.

Letppos,k\(ℓ\)=‖𝐱~pos,k\(ℓ\)‖22/∑r=1N‖𝐱~pos,r\(ℓ\)‖22p\_\{\\mathrm\{pos\},k\}^\{\(\\ell\)\}=\\\|\\widetilde\{\\mathbf\{x\}\}\_\{\\mathrm\{pos\},k\}^\{\(\\ell\)\}\\\|\_\{2\}^\{2\}/\\sum\_\{r=1\}^\{N\}\\\|\\widetilde\{\\mathbf\{x\}\}\_\{\\mathrm\{pos\},r\}^\{\(\\ell\)\}\\\|\_\{2\}^\{2\}denote the normalized positional\-modal energy\. Using the same default cutoffK=⌊N/2⌋K=\\lfloor N/2\\rflooras graph HFER, we define positional HFER as the fraction of hidden\-state energy assigned to positional\-frequency modes above the cutoff:

HFERpos\(ℓ\)=∑k=K\+1Nppos,k\(ℓ\)\.\\mathrm\{HFER\}\_\{\\mathrm\{pos\}\}^\{\(\\ell\)\}=\\sum\_\{k=K\+1\}^\{N\}p\_\{\\mathrm\{pos\},k\}^\{\(\\ell\)\}\.This quantity lies in\[0,1\]\[0,1\]: it is0when all energy lies in modes at or below the cutoff and11when all energy lies in modes above the cutoff\. Larger values indicate that a greater fraction of hidden\-state energy is associated with rapid variation across serialized token positions\. By comparison, graph HFER quantifies hidden\-state variation over the attention\-induced token geometry\.

#### Positional spectral entropy \(SE\)\.

Analogously to graph SE, positional SE measures how broadly hidden\-state energy is distributed across the positional\-frequency modes\. Using the normalized positional\-modal energiesppos,k\(ℓ\)p\_\{\\mathrm\{pos\},k\}^\{\(\\ell\)\}, we define

SEpos\(ℓ\)=−∑k=1Nppos,k\(ℓ\)​log⁡ppos,k\(ℓ\)\.\\mathrm\{SE\}\_\{\\mathrm\{pos\}\}^\{\(\\ell\)\}=\-\\sum\_\{k=1\}^\{N\}p\_\{\\mathrm\{pos\},k\}^\{\(\\ell\)\}\\log p\_\{\\mathrm\{pos\},k\}^\{\(\\ell\)\}\.Following the main analysis, we reportexp⁡\(SEpos\(ℓ\)\)\\exp\(\\mathrm\{SE\}\_\{\\mathrm\{pos\}\}^\{\(\\ell\)\}\)as the effective number of contributing positional modes\. This quantity lies in\[1,N\]\[1,N\]: it equals11when all energy is concentrated in one mode andNNwhen energy is distributed uniformly across all modes\. Larger values indicate that hidden\-state energy is distributed across a broader set of positional\-frequency modes\. Thus,exp⁡\(SEpos\(ℓ\)\)\\exp\(\\mathrm\{SE\}\_\{\\mathrm\{pos\}\}^\{\(\\ell\)\}\)and graph\-spectralexp⁡\(SE\(ℓ\)\)\\exp\(\\mathrm\{SE\}^\{\(\\ell\)\}\)quantify analogous notions of spectral breadth, but in the fixed positional DCT basis and the attention\-induced graph Fourier basis, respectively\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x27.png)Figure 19:Comparison of graph\-spectral diagnostics \(left\) and corresponding non\-graph controls \(right\) across context lengths under the same evaluation setup as[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1)\. The controls reproduce the broad context\- and complexity\-dependent trends\. The top\-row control is attention\-only: increasing attention Gini indicates more concentrated attention\. The lower two controls are hidden\-state\-only and measure high\-frequency energy and effective spectral support in a fixed positional\-frequency basis\. The positional controls and corresponding graph\-spectral diagnostics capture analogous spectral properties, but the latter show clearer separation among input families on matched y\-axis scales, highlighting the added value of jointly analyzing hidden\-state variation and attention\-induced graph structure\.
#### Comparison with graph\-spectral diagnostics\.

Under the same evaluation setup as[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1),[Fig\.˜19](https://arxiv.org/html/2608.03015#A8.F19)compares the three non\-graph controls, each computed at every transformer layer and then averaged across theLLlayers, with the main graph\-spectral diagnostics across context lengths\. These controls recover the broad context\- and complexity\-dependent trends across the main numerical input families, showing that the broad qualitative findings are also visible in attention\- or hidden\-state\-only summaries and are therefore not artifacts of graph construction\. When plotted with matched y\-axis limits, positional HFER andexp⁡\(SEpos\)\\exp\(\\mathrm\{SE\}\_\{\\mathrm\{pos\}\}\)exhibit narrower dynamic ranges and less separation among several input families\. One notable difference between positional and graph HFER is that the former assigns the period\-2 family greater high\-frequency energy than periods 4–64 at every context length, reflecting the sensitivity of fixed positional modes to rapid alternation in serialized token order\. Graph HFER instead measures hidden\-state variation relative to the attention\-induced token geometry, incorporating attention structure rather than relying on token order alone\. The graph\-spectral formulation therefore provides a joint characterization of attention\-induced token connectivity and hidden\-state variation and more consistently resolves the finer family\- and context\-specific behaviors discussed in[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1)\.

## Appendix IAttention\-Induced Token Graphs for Additional Structured Numerical Inputs

Beyond the main numerical input suite introduced in[Section˜3\.1](https://arxiv.org/html/2608.03015#S3.SS1), we consider several additional structured numerical sequences representing abrupt distributional shifts, smooth periodic variation, and stochastic evolution\. Each full sequence contains 500 numeric tokens and 500 separator tokens, yielding a 1000\-token reference context\.[Fig\.˜20](https://arxiv.org/html/2608.03015#A10.F20)displays the corresponding last\-layer attention\-induced token graphs at 10%, 30%, 50%, 70%, and 90% of the reference context, using the same graph construction and visualization procedure described in[Section˜3\.3](https://arxiv.org/html/2608.03015#S3.SS3)\.

The Gaussian\-switch inputs are piecewise Gaussian sequences with i\.i\.d\. samples within each segment and abrupt changes in mean at prescribed context fractions\. This construction adapts the distribution\-switching setup used bySarfatiet al\.\([2026](https://arxiv.org/html/2608.03015#bib.bib11)\)to study belief dynamics\. In the one\-switch case, samples are drawn from𝒩​\(μ=300,σ2=10\)\\mathcal\{N\}\(\\mu=300,\\sigma^\{2\}=10\)over the first half of the sequence and from𝒩​\(μ=700,σ2=10\)\\mathcal\{N\}\(\\mu=700,\\sigma^\{2\}=10\)over the second half\. In the two\-switch case, the mean is 300 over the first 35% of the sequence, 700 over the middle 30%, and 300 over the final 35%, withσ2=10\\sigma^\{2\}=10throughout\. Each switch therefore has magnitudeΔμ=400\\Delta\_\{\\mu\}=400, which is large relative to the within\-regime standard deviationσ=10\\sigma=\\sqrt\{10\}\. The sampled values are rounded to the nearest integer and clipped to\[150,850\]\[150,850\]before serialization\.

We also include a smooth periodic trajectory generated fromsin⁡\(x\)\\sin\(x\), sampled uniformly over three full periods and rescaled to the same integer range\. Unlike the periodic logistic map orbits examined in the main text, whose successive iterates can move sharply between distinct orbit values, this sequence varies smoothly from one step to the next\.

Finally, we include two bounded stochastic processes: a random walk and a random walk with drift\. Both follow

zt\+1=R\[150,850\]​\(zt\+μ\+ϵt\),ϵt∼𝒩​\(0,182\),z\_\{t\+1\}=R\_\{\[150,850\]\}\(z\_\{t\}\+\\mu\+\\epsilon\_\{t\}\),\\quad\\epsilon\_\{t\}\\sim\\mathcal\{N\}\(0,18^\{2\}\),withμ=0\\mu=0for the random walk andμ=2\.5\\mu=2\.5for the drifted random walk\. The zero\-drift and drifted walks are initialized atz0=500z\_\{0\}=500andz0=350z\_\{0\}=350, respectively\. The reflection operatorR\[150,850\]R\_\{\[150,850\]\}maps boundary overshoots back into\[150,850\]\[150,850\], repeating the reflection if necessary\.

[Fig\.˜20](https://arxiv.org/html/2608.03015#A10.F20)suggests that attention\-induced graphs reflect not only deterministic dynamical complexity, but also distributional shifts, smooth recurrence, and stochastic evolution\. In the Gaussian\-switch examples, the switch\-defined temporal segments appear as distinct clusters in the attention\-induced graphs\. This observation complementsSarfatiet al\.\([2026](https://arxiv.org/html/2608.03015#bib.bib11)\), who show that abrupt shifts in the mean of a Gaussian input distribution produce structured belief\-update trajectories in hidden\-state space, accompanied by corresponding changes in the model’s predictive distributions over numerical outputs\. Our attention\-induced token\-graph view provides a complementary perspective by showing that change\-point structure is also reflected in attention topology as clusters aligned with the switch\-defined temporal segments\. The sine input produces a graph structure distinct from periodic logistic map orbits despite their shared periodicity\. By contrast, the random\-walk inputs produce less sharply organized graph structures, consistent with the path\-dependent nature of their generating processes\.

## Appendix JPreliminary Graph\-Spectral Comparison of Numerical, Natural\-Language, and Code Inputs

#### Shared setup\.

We construct three input suites—multilingual text, source code, and English\-domain text—each comprising six input types and 50 samples per type\. Mirroring the numerical\-input evaluation in[Section˜4\.1](https://arxiv.org/html/2608.03015#S4.SS1), we compute the same graph\-spectral diagnostics at 20 context lengths from 100 to 2000 tokens in increments of 100, using the corresponding prefixes of each 2000\-token sample\.

#### Multilingual text\.

The multilingual suite contains text in six languages—Arabic, Chinese, English, French, Russian, and Spanish—from MultiUN, a parallel corpus of translated United Nations documents\(Eisele and Chen,[2010](https://arxiv.org/html/2608.03015#bib.bib43); Tiedemann,[2012](https://arxiv.org/html/2608.03015#bib.bib44)\)\. Using English as the pivot, we align corresponding records across the six languages and concatenate consecutive aligned record sets until all six language streams contain at least 2000 tokens; we then truncate each stream to exactly 2000 tokens\. The samples therefore contain aligned source material, although differences in tokenization density mean that the cutoff may fall at different positions within the final aligned record\.

#### Code inputs\.

The programming\-language suite comprises Python, JavaScript, Java, C\+\+, Go, and Rust files from the StarCoder training dataset\(Kocetkovet al\.,[2023](https://arxiv.org/html/2608.03015#bib.bib45); Liet al\.,[2023](https://arxiv.org/html/2608.03015#bib.bib46)\)\. For each language, we sample 50 contiguous 2000\-token segments at random offsets from distinct source files\. We remove encoding artifacts and invalid control characters while preserving whitespace, line breaks, indentation, comments, and other code formatting\.

#### English\-domain text\.

The English\-domain suite comprises news from CNN/DailyMail\(Hermannet al\.,[2015](https://arxiv.org/html/2608.03015#bib.bib47); Seeet al\.,[2017](https://arxiv.org/html/2608.03015#bib.bib48)\), fiction from Project Gutenberg \(50 distinct books, one per sample\)\(Christou and Tsoumakas,[2025](https://arxiv.org/html/2608.03015#bib.bib49)\), English Wikipedia articles from a November 2023 snapshot\(Wikimedia Foundation,[2023](https://arxiv.org/html/2608.03015#bib.bib50)\), scientific text from arXiv\(Cohanet al\.,[2018](https://arxiv.org/html/2608.03015#bib.bib51)\), dialogue from DailyDialog\(Liet al\.,[2017](https://arxiv.org/html/2608.03015#bib.bib52)\), and legal text from LexGLUE\(Chalkidiset al\.,[2022](https://arxiv.org/html/2608.03015#bib.bib53)\)\. We apply light common preprocessing by decoding HTML entities, removing generic boilerplate\-only lines, flattening internal line breaks, and collapsing repeated whitespace\. When a dataset entry is shorter than 2000 tokens, we concatenate consecutive entries until reaching the target length\.

![Refer to caption](https://arxiv.org/html/2608.03015v1/x28.png)Figure 20:Last\-layer attention\-induced token graphs for additional structured numerical inputs with Llama\-3\.2\-3B\. Graphs are shown at 10%, 30%, 50%, 70%, and 90% of a 1000\-token reference context\. All examples use the same attention\-induced token\-graph construction and visualization procedure as[Section˜3\.3](https://arxiv.org/html/2608.03015#S3.SS3)\. The visualizations illustrate qualitative graph\-organization patterns beyond those observed for the main dynamical\-system trajectories\.![Refer to caption](https://arxiv.org/html/2608.03015v1/x29.png)\(a\)Normalized Fiedler valueλ2\\lambda\_\{2\}
![Refer to caption](https://arxiv.org/html/2608.03015v1/x30.png)\(b\)High\-frequency energy ratio \(HFER\)
![Refer to caption](https://arxiv.org/html/2608.03015v1/x31.png)\(c\)Effective spectral supportexp⁡\(SE\)\\exp\(\\mathrm\{SE\}\)

Figure 21:Layer\-averaged graph\-spectral diagnostics for natural\-language and code inputs using Llama\-3\.2\-3B, with the main numerical families shown as dashed matched\-context references\. Columns compare content\-aligned United Nations translations across six languages, programming\-language inputs, and English\-domain inputs\. Curves and shading denote the mean and±1\\pm 1standard deviation across 50 samples per natural\-language or code family and 20 realizations per numerical family\. Across context lengths, the natural\-language and code family means exhibit lower normalized Fiedler values, higher HFER, and broader effective spectral support than the numerical inputs; this coarse separation corresponds to weaker global attention connectivity, a larger high\-frequency share of hidden\-state energy, and a broader distribution of that energy across graph Fourier modes\.
#### Findings and open questions\.

[Fig\.˜21](https://arxiv.org/html/2608.03015#A10.F21)reveals a consistent separation of the natural\-language and code inputs from the controlled numerical inputs across all three diagnostics\. Relative to the numerical inputs, the natural\-language and code inputs exhibit lower normalized Fiedler values, indicating less globally integrated and more localized attention topology, together with higher HFER, indicating sharper hidden\-state variation over that topology\. The separation is especially pronounced in effective spectral support, withexp⁡\(SE\)\\exp\(\\mathrm\{SE\}\)remaining substantially larger for the natural\-language and code families than for the numerical families across all evaluated context lengths\. This difference indicates that their hidden\-state energy is distributed across a substantially larger effective number of graph Fourier modes\. These results support a coarse representational distinction between controlled numerical sequences and natural\-language or code inputs\. More controlled comparisons are needed to determine which input and model properties contribute to the coarse separation of natural\-language and code inputs from numerical inputs and whether meaningful graph\-spectral orderings exist within the multilingual\-text, source\-code, and English\-domain suites\.