Interpreting Latent CoT Reasoning as Dynamical Systems

# 1 Introduction
Source: [https://arxiv.org/html/2607.09698](https://arxiv.org/html/2607.09698)
ICML 2026 Workshop on Foundations of Deep Generative Models: Understanding Memorization, Generalization, and Reasoning

Interpreting Latent CoT Reasoning as Dynamical Systems

Sabari Iyyappan Duraipandian\*1Shreya Sanjay Boyane\*2Manju Nagesh3Jerome Francis†\\dagger4Archana Vaidheeswaran4Kevin Zhu4

††footnotetext:1San Jose State University\.2Worcester Polytechnic Institute\.3George Mason University\.4Algoverse AI Research\.
\*Equal contribution; order decided by random coin flip\.†\\daggerProject Lead\.
Corresponding author: Jerome Francis<jerome@algoverseairesearch\.org\>\.1Code Repository:[https://github\.com/SabariIyyappan/Latent\-CoT\-Reasoning\-as\-Dynamical\-Systems](https://github.com/SabariIyyappan/Latent-CoT-Reasoning-as-Dynamical-Systems)2Project Page:[https://sabariiyyappan\.github\.io/Latent\-CoT/](https://sabariiyyappan.github.io/Latent-CoT/)
Accepted to FoGen 2026: Foundations of Deep Generative Models: Understanding Memorization, Generalization, and Reasoning, an ICML 2026 workshop \(non\-archival\)\.###### Abstract

Recent latent reasoning methods, such as CODI and COCONUT, face a fundamental interpretability problem: they maintain multiple superimposed candidate traces in the hidden space at each step, unlike explicit\-CoT, which follows a single transparent reasoning trace\. Existing mechanistic methods show compression, shortcuts, and superposition without explaining how reasoning evolves across latent steps\. To address this gap, we model latent token sequences as trajectories in representation space and apply dynamical systems analysis to characterize the evolution of reasoning\. Using quantitative measures, such as step\-to\-step change, direction consistency, and Lyapunov sensitivity, alongside qualitative projections, such as UMAP and DMD/PHATE, we show that latent CoT exhibits structured, non\-random dynamics with two distinct stability classes\. CODI behaves as a stable attractor, while COCONUT behaves as an unstable expanding system, and SIM\-CoT supervision tightens both behaviors without changing the underlying dynamics\. This framework advances the interpretability of latent CoT reasoning dynamics and provides actionable insights for improving latent reasoning performance\. Code1and Project page2available online\.

Latent CoT paradigms such as CODI and COCONUT have consistently outperformed explicit CoT in the performance\-compute tradeoff\. However, the interpretability of Latent CoT is still an active area of research\. Existing mechanistic interpretation methods \(logit lens, attention heatmaps, activation patching\) reveal relationships between latent tokens and outputs and the role of latent steps through causality, but do not show how reasoning evolves through latent steps\.Liang and Pan \([2026](https://arxiv.org/html/2607.09698#bib.bib1)\); Goyalet al\.\([2025](https://arxiv.org/html/2607.09698#bib.bib2)\)\. Prior works demonstrate compression, shortcuts, and superposition in latent CoTs, but do not quantitatively analyze reasoning evolution or highlight quantitative behavior in latent steps\.Liang and Pan \([2026](https://arxiv.org/html/2607.09698#bib.bib1)\); Liet al\.\([2026](https://arxiv.org/html/2607.09698#bib.bib3)\)\.

Dynamical systems provide a principled approach to studying how internal representations evolve during reasoning\. Applied to explicit CoT, they help evaluate whether models genuinely reason step\-by\-step or merely memorizeYuet al\.\([2025](https://arxiv.org/html/2607.09698#bib.bib4)\); Phamet al\.\([2026](https://arxiv.org/html/2607.09698#bib.bib14)\)\. These works analyze how often the model shifts and visits different states but do not thoroughly measure the rate, direction, or stability of change—factors crucial for understanding reasoning dynamics\. The faithfulness of latent CoT—how much latent steps reflect genuine, step\-by\-step reasoning rather than opaque computation—remains underexplored and lacks rigorous verification\. Dynamical systems offer tools to address this gap\. To address this gap, we model latent CoT trajectories as dynamical systems and analyze their stability and representational geometry\. Using quantitative metrics and qualitative projections, we investigate whether latent reasoning exhibits structured dynamics and how these dynamics differ across training paradigms\.

## 2Related Works

### 2\.1Chain\-of\-Thought and Latent Reasoning

Chain\-of\-Thought \(CoT\) has become the efficient paradigm for LLM reasoningWeiet al\.\([2022](https://arxiv.org/html/2607.09698#bib.bib15)\), but explicit CoTs are computationally inefficient due to verbose reasoning traces\. Latent CoTs were introduced to tackle this problem by skipping the verbose reasoning rationales\. Recent latent CoT frameworks such as CODIShenet al\.\([2025](https://arxiv.org/html/2607.09698#bib.bib16)\)and COCONUTHaoet al\.\([2024](https://arxiv.org/html/2607.09698#bib.bib17)\)preserve the CoT reasoning in the latent space, achieving better token\-efficient performance over explicit CoT\.

COCONUT reasons in the latent space by feeding back the last hidden state as next input enabling a breadth\-first search over candidate reasoning steps\. It is trained using a multi\-stage curriculum that progressively replaces explicit CoT tokens with continuous thoughts\. CODI instead avoids curriculum learning altogether by jointly training a teacher on explicit CoT and a student on implicit CoT within a single self\-distillation stage, transferring reasoning ability via hidden\-state at the answer token\.

While COCONUT and CODI represent reasoning with latent steps, both methods suffer from the latent instability problem where we see unstable training as the number of implicit reasoning tokens scale up, leading to latent representations collapsing into homogeneous states that lose semantic diversity\. SIM\-CoTWeiet al\.\([2025](https://arxiv.org/html/2607.09698#bib.bib18)\)addresses this instability by adding auxiliary decoder during training to align each implicit latent token with its corresponding explicit reasoning step, enforcing structured semantics in the latent space\.

Mechanistic interpretability has emerged as an important direction for understanding the nature of latent CoT and its reasoning behavior, which we discuss in the following section\.

### 2\.2Mechanistic Interpretability of Latent CoT

Liang and Pan \([2026](https://arxiv.org/html/2607.09698#bib.bib1)\)depicts CODI can retrieve bridge states in a multi\-hop reasoning tasks, while relying on compression and short\-cut like pathways for longer reasoning chain tasks\.Goyalet al\.\([2025](https://arxiv.org/html/2607.09698#bib.bib2)\)shows a “scratchpad\-style” latent reasoning in CODI where latent states alternate between storage and compute steps\. While these works establish that latent reasoning is functionally meaningful, they do not explain the reasoning dynamics\.

Liet al\.\([2026](https://arxiv.org/html/2607.09698#bib.bib3)\)provides a causal analysis of CODI showing that latent reasoning is a two\-stream structure where latent states propagate information, while final inputs can bypass computation through direct copy pathways\.Liang and Pan \([2026](https://arxiv.org/html/2607.09698#bib.bib1)\)has also shown latent\-CoTs enable shortcut\-like paths to the final answer\. These results suggest that latent reasoning contains compression, shortcut routing, and superposition, and they collapse during deeper reasoning tasks, but prior works still lack explanations of “why” and “how” they emerge in Latent CoTs\.

### 2\.3Dynamical Systems for Reasoning Dynamics

Dynamical systems theory provides a mathematical framework for characterizing how states evolve over time, capturing concepts such as attractors, trajectory stability, and divergence\. These tools have been widely applied in physics and neuroscienceHolmes \([1990](https://arxiv.org/html/2607.09698#bib.bib24)\); Johnet al\.\([2022](https://arxiv.org/html/2607.09698#bib.bib23)\)to study complex temporal processes, and have recently been adapted to analyze reasoning behavior in language models\.

Viewing reasoning as trajectories in latent spaces has been proven effective for studying reasoning dynamicsYuet al\.\([2025](https://arxiv.org/html/2607.09698#bib.bib4)\)\. Recent works have shown that explicit CoT reasoning can be analyzed using state\-aware and recurrence\-based measures to understand how often the model revisits, shifts, or stabilizes in particular reasoning statesYuet al\.\([2025](https://arxiv.org/html/2607.09698#bib.bib4)\); Phamet al\.\([2026](https://arxiv.org/html/2607.09698#bib.bib14)\)\.

Spectral analysis can be used to represent and cluster reasoning steps semantically to evaluate reasoning dynamicsYuet al\.\([2025](https://arxiv.org/html/2607.09698#bib.bib4)\)\. Similarly, Pham et al\.Phamet al\.\([2026](https://arxiv.org/html/2607.09698#bib.bib14)\)apply recurrence\-based analysis to reasoning traces and show that metrics such as determinism, laminarity, and stalling reveal the reasoning process and the frequency of state visits\. These results motivate treating reasoning as a dynamical system rather than as a sequence of independent steps\.

However, these approaches have mainly been developed for explicit CoTs, which leaves the study of Latent\-CoT reasoning as a dynamical system underexplored\. In particular, they do not measure how latent hidden states evolve through representation space in terms of step size, direction consistency, and stability, nor what causes the emergence of phenomena such as compression, shortcuts, and superposition in the Latent\-CoT reasoning paradigm\.

Our work addresses this gap by treating latent CoT as a trajectory in latent space and studies its reasoning evolution using dynamical systems\.

## 3Methodology

We propose a framework \(Figure[1](https://arxiv.org/html/2607.09698#S3.F1)\) for analyzing latent chain\-of\-thought \(CoT\) reasoning as a dynamical system\. The approach extracts intermediate latent representations from reasoning models, constructs trajectories in representation space, and applies dynamical systems analysis to characterize reasoning behaviors\.

![Refer to caption](https://arxiv.org/html/2607.09698v1/x1.png)Figure 1:Dynamical System Analysis of Latent\-CoTs\.\(1\) GSM8K dataset\. \(2\) Four model configs \- CODI \(teacher–student self\-distillation\) and COCONUT \(hidden\-state feedback loop\) in Vanilla and SIM\-CoT settings; auxiliary decoder enforces structured semantics\. \(3\) T = 6 latent hidden states form a trajectory in representation space\. \(4\) Quantitative metrics and qualitative projections\. \(5\) Representative latent space and dynamics visualizations\.### 3\.1Problem Formulation

Given an inputxx, a latent CoT model produces a sequence of hidden representations across reasoning steps:

z1,z2,…,zT,zt∈ℝd,z\_\{1\},z\_\{2\},\\ldots,z\_\{T\},\\quad z\_\{t\}\\in\\mathbb\{R\}^\{d\},where eachztz\_\{t\}denotes the model’s internal state at steptt, andddis the hidden dimension of the model\.

This sequence is treated as a trajectory in representation space\.

### 3\.2Dimensionality Reduction

The trajectory tensorZ∈ℝN×T×dZ\\in\\mathbb\{R\}^\{N\\times T\\times d\}lies in a high\-dimensional space that makes direct visualization intractable\. Five complementary methods are applied to project latent states into a low\-dimensional space for analysis\. For t\-SNE, UMAP and PHATE, the tensor is first flattened to\[N⋅T,d\]\[N\{\\cdot\}T,\\,d\]so that all latent states are reduced and then reshaped to\[N,T,k\]\[N,T,k\]to preserve the temporal ordering of each trajectory\.

t\-SNEvan der Maaten and Hinton \([2008](https://arxiv.org/html/2607.09698#bib.bib19)\)preserves the local neighborhood structure through pairwise, similarity modeling, making it sensitive to fine\-grained clustering among nearby reasoning states\.

UMAPMcInneset al\.\([2018](https://arxiv.org/html/2607.09698#bib.bib20)\)approximates the underlying data to balance local and global structure, producing stable embeddings suited for continuous trajectories\.

Dimensionality reduction is used solely for visualization \(Section[4\.3\.1](https://arxiv.org/html/2607.09698#S4.SS3.SSS1)\)\. All quantitative metrics are computed directly on the original representationszt∈ℝdz\_\{t\}\\in\\mathbb\{R\}^\{d\}\.

### 3\.3Dynamical Systems Concepts

DMDTuet al\.\([2013](https://arxiv.org/html/2607.09698#bib.bib21)\)treats the latent sequence as a discrete\-time dynamical system and identifies the spatial modes and eigenvalues via an SVD of the snapshot pair\(X1,X2\)=\(z0:T−2,z1:T−1\)\(X\_\{1\},X\_\{2\}\)=\(z\_\{0:T\-2\},\\,z\_\{1:T\-1\}\)\. The decomposition is computed in a single batched operation across allNNtrajectories simultaneously\. Each trajectory is then projected onto the leading two DMD modes for visualization\. The resulting eigenvalues serve as a stability measure in the quantitative analysis \(Section[4\.3\.2](https://arxiv.org/html/2607.09698#S4.SS3.SSS2)\)\.

PHATEMoonet al\.\([2019](https://arxiv.org/html/2607.09698#bib.bib22)\)uses a multi\-scale diffusion process to capture transitions in high\-dimensional data\. It uses a diffusion operator based on an affinity graph constructed by kernel methods to calculate the potentials that represent the distances between points, which represents the geometric structure of the underlying manifold\. Such a low\-dimensional embedding captures both the local structure and global dynamics, which is also ideal for representing sequential latent paths\.

## 4Experimental Configurations

### 4\.1Datasets

GSM8KExperiments are conducted on the GSM8K benchmark\(Cobbeet al\.,[2021](https://arxiv.org/html/2607.09698#bib.bib6)\), a collection of grade\-school math word problems requiring multi\-step arithmetic reasoning\. The full GSM8K dataset \(train and test splits combined, 8,792 samples\) is used across seven math concept categories: Geometry \(210\), Rates & Speed \(675\), Percentages & Ratios \(1,266\), Money & Pricing \(2,741\), Fractions & Decimals \(1,045\), Multiplication & Division \(224\), and Arithmetic & Multi\-step \(2,631\)\. Questions are assigned to categories via priority\-ordered keyword matching, with more specific categories checked before broader ones to prevent improper labeling\. Five hundred stratified samples are taken per category\. Categories with fewer available examples contribute all remaining samples\. Ground\-truth answers are parsed from the"\#\#\#\#"delimiter in GSM8K annotations\.

### 4\.2Models

CODI and COCONUT are both GPT\-2\-small backbones \(124 M parameters, 12 layers, 12 heads, hidden size 768, vocabulary extended to 50 260 with three latent special tokens<\|start\-latent\|\>,<\|latent\|\>, and<\|end\-latent\|\>\) trained on GSM8K\(Haoet al\.,[2024](https://arxiv.org/html/2607.09698#bib.bib17); Shenet al\.,[2025](https://arxiv.org/html/2607.09698#bib.bib16)\)\. Vanilla checkpoints are taken fromModalityDance/latent\-tts\-coconut111[https://huggingface\.co/ModalityDance/latent\-tts\-coconut](https://huggingface.co/ModalityDance/latent-tts-coconut)andModalityDance/latent\-tts\-codi222[https://huggingface\.co/ModalityDance/latent\-tts\-codi](https://huggingface.co/ModalityDance/latent-tts-codi); CODI additionally loads a self\-distillation projection module \(prj\.pt\) at inference time that maps each latent hidden state back into the input\-embedding space\. SIM\-CoT variants of both methods are taken frominternlm/SIM\_COT\-GPT2\-Coconut333[https://huggingface\.co/internlm/SIM\_COT\-GPT2\-Coconut](https://huggingface.co/internlm/SIM_COT-GPT2-Coconut)andinternlm/SIM\_COT\-GPT2\-CODI444[https://huggingface\.co/internlm/SIM\_COT\-GPT2\-CODI](https://huggingface.co/internlm/SIM_COT-GPT2-CODI)\(Weiet al\.,[2025](https://arxiv.org/html/2607.09698#bib.bib18)\)\. Each model generates a fixed sequence ofT=6T=6latent reasoning steps per input\. Text generation uses greedy decoding with a maximum of 512 output tokens\. All experiments use a fixed random seed of 42\.

### 4\.3Analysis of Latent CoT trajectories

Analysis is performed at 2 levels: qualitative inspection of geometric structure and quantitative characterization of dynamics\.

#### 4\.3\.1Qualitative Analysis

Reduced representations from Section[3\.2](https://arxiv.org/html/2607.09698#S3.SS2)are visualized as sequences of points inℝ2\\mathbb\{R\}^\{2\}andℝ3\\mathbb\{R\}^\{3\}with temporal ordering preserved\. These plots expose geometric phenomena including smooth progression through state space, directional pivots \(using centroids of latent representations\), convergence, and separation between correct and incorrect trajectories\.

#### 4\.3\.2Quantitative Analysis

Trajectory dynamics are characterized by two groups of metrics:*step\-based metrics*that measure how the trajectory moves between consecutive steps, and*stability metrics*that characterize sensitivity and attraction to fixed points\.

Step\-based Metrics Step\-to\-step changemeasures the magnitude of displacement at each transition:

‖Δt‖=‖zt\+1−zt‖2\.\\\|\\Delta\_\{t\}\\\|=\\\|z\_\{t\+1\}\-z\_\{t\}\\\|\_\{2\}\.\(1\)Large values indicate substantial representational transitions, while smaller values indicate reduced movement between consecutive latent states\. Although decreasing displacement is consistent with convergence toward a stable region of representation space, it does not necessarily imply convergence toward a correct solution\.

Direction consistencymeasures whether consecutive displacements point in the same direction:

Ct=cos⁡\(Δt,Δt−1\)\.C\_\{t\}=\\cos\(\\Delta\_\{t\},\\,\\Delta\_\{t\-1\}\)\.\(2\)Values near\+1\+1indicate aligned transitions, values near0indicate directional pivots, and values near−1\-1indicate reversals\. We interpret this metric as describing trajectory geometry rather than reasoning quality, since effective reasoning may involve branching, revision, or exploration of multiple solution paths\.

Arc lengthsummarizes total path complexity as the cumulative displacement over all transitions:

L=∑t=1T−1‖zt\+1−zt‖2\.L=\\sum\_\{t=1\}^\{T\-1\}\\\|z\_\{t\+1\}\-z\_\{t\}\\\|\_\{2\}\.Arc length summarizes the cumulative displacement traversed by a latent trajectory and serves as a measure of overall trajectory complexity\.

Stability Metrics Lyapunov Sensitivity \(Surrogate\)To estimate local stability without re\-running the model, a trajectory\-based surrogate is computed from the ratio of onsecutive step magnitudes:

λ​\(t\)=log⁡‖zt\+1−zt‖‖zt−zt−1‖,\\lambda\(t\)=\\log\\frac\{\\\|z\_\{t\+1\}\-z\_\{t\}\\\|\}\{\\\|z\_\{t\}\-z\_\{t\-1\}\\\|\},\(3\)where∥⋅∥\\\|\\cdot\\\|denotes theℓ2\\ell\_\{2\}norm\. A positiveλ​\(t\)\\lambda\(t\)indicates the trajectory is locally diverging\. A negativeλ​\(t\)\\lambda\(t\)indicates convergence\.λ​\(t\)=0\\lambda\(t\)=0indicates neutral stability, where the step magnitude remains constant and the trajectory is neither converging nor diverging\. Positive values indicate local expansion in trajectory magnitude, while negative values indicate contraction\. We interpret these behaviors as potential signatures of exploratory and stabilizing dynamics respectively, although additional causal validation would be required to establish a direct correspondence with reasoning processes\.

Perturbation stabilityre\-runs inference with Gaussian noise injected into input embeddings and measures divergence from the clean trajectory at each step\. Growing divergence indicates a sensitivity to initial conditions, demonstrating the robustness of latent CoT tokens during reasoning\.

## 5Results

Results are reported for COCONUT and CODI methods for both Vanilla COT and SIM\-CoT training paradigms\. The analysis is split into two parts: \(1\)Qualitative dynamics, where we visualize latent states using DMD, PHATE, and UMAP to understand how each model moves through latent space across reasoning steps; and \(2\)Quantitative dynamics, where we measure step\-to\-step change, direction consistency, and fixed\-point distances to compare how the two models behave differently\.

\* Concept\-wise plots \(Arc Length, Perturbation Stability, Step\-to\-Step Change, Direction Consistency, Lyapunov Sensitivity\) for GSM8K are provided in Appendix[A](https://arxiv.org/html/2607.09698#A1)\.

### 5\.1Quantitative Results

##### Step\-to\-Step Change

The analysis here \(as seen in Figure[2](https://arxiv.org/html/2607.09698#S5.F2)\) measures the consecutive differences between latent reasoning steps\. For the CODI method, we see an uniform curve with variance for vanilla CoT setting\. In contrast, SIM\-CoT setting displays a monotonically decreasing curve\. This decreasing characteristic demonstrates that latent CoT tokens start to become similar, a potential sign for convergence in the end phase \(solution\)\. For the COCONUT method, the vanilla CoT setting shows a drop in the transition from2n​d2^\{nd\}to3r​d3^\{rd\}latent token, a possible premature convergence in the reasoning process\. Whereas, we see a stable behavior in SIM\-CoT setting\.

Figure 2:Step\-to\-Step change for methods \(CODI, COCONUT\) under Vanilla \+ SIM\-CoT settings: Overall, the CODI method exhibits neutral or convergent behavior , while COCONUT shows premature convergence in the vanilla setting\. Both the methods have more stable curves in SIM\-CoT setting compared to their Vanilla counterparts\.
##### Lyapunov Sensitivity

Lyapunov sensitivity \(as seen in Figure[3](https://arxiv.org/html/2607.09698#S5.F3)\) measures the local stability of the latent trajectory\. Positive values indicate the trajectory is locally diverging \(the model is exploring multiple candidate reasoning paths at that step\), while negative values indicate it is locally contracting, i\.e\. committing to a single path\. For the CODI method, we see a monotonically decreasing curve with negative values across all steps in the Vanilla CoT setting\. In the SIM\-CoT setting, the curve descends deeper into the negative range, indicating that the supervision strengthens this convergence\. For the COCONUT method, the Vanilla CoT setting exhibits a sharp positive spike att=3t=3, a localized expansion that captures the model branching across candidate reasoning paths at the mid\-chain transition before settling, a genuine exploration phase native to COCONUT’s curriculum\-trained dynamics\. Whereas, in the SIM\-CoT setting, this spike disappears and the curve flattens, showing that the supervision replaces the exploratory mid\-chain step with a deterministic transition by anchoring each latent token to its corresponding textual reasoning step\.

Figure 3:Lyapunov Sensitivity for methods \(CODI, COCONUT\) under Vanilla \+ SIM\-CoT settings: CODI exhibits stable, near\-neutral or converging behavior across both settings, while COCONUT shows a sharp divergence \(local instability\) at t=3 in the Vanilla setting\. SIM\-CoT supervision suppresses this instability in COCONUT and deepens the convergence in CODI\.
\(¿ 0: diverging — ¡ 0: converging — 0: neutral \)
##### Direction Consistency

Direction consistency measures the cosine between consecutive latent transitions,Ct=cos⁡\(Δt,Δt−1\)C\_\{t\}=\\cos\(\\Delta\_\{t\},\\Delta\_\{t\-1\}\)\. Values near\+1\+1indicate smooth forward motion, values near0indicate orthogonal pivots, and values near−1\-1indicate the trajectory reverses direction at every step\. For CODI, the Vanilla setting exhibits predominantly negative values, whereas SIM\-CoT shifts the trajectory closer to orthogonal transitions\. For COCONUT, the Vanilla setting transitions from opposing to near\-orthogonal directions before returning toward opposing transitions\. SIM\-CoT produces a more consistent directional profile across latent steps\.

Figure 4:Direction consistencyCt=cos⁡\(Δt,Δt−1\)C\_\{t\}=\\cos\(\\Delta\_\{t\},\\Delta\_\{t\-1\}\)across latent steps for methods \(CODI and COCONUT\) under Vanilla \+ SIM\-CoT settings: In the Vanilla setting, CODI shows consistently opposing transitions, while COCONUT transitions become near orthogonal at t=3 before reversing again\. Under SIM\-CoT, CODI shifts to near orthogonal transitions and COCONUT maintains consistent opposition\.\(\+1: Same direction — 0: Orthogonal pivot — \-1: Reversal\)Arc length and Perturbation stabilityArc length summarizes the total displacement traversed by the latent trajectory, reflecting overall reasoning effort\. Perturbation stability measures trajectory divergence∥ztperturbed−ztclean∥2\\lVert z\_\{t\}^\{\\text\{perturbed\}\}\-z\_\{t\}^\{\\text\{clean\}\}\\rVert\_\{2\}under Gaussian noise injected into the input embeddings, tested atσ∈\{0\.01,0\.1,1\.0\}\\sigma\\in\\\{0\.01,0\.1,1\.0\\\}\.Results and additional discussions are added in the Appendix[A\.3\.1](https://arxiv.org/html/2607.09698#A1.SS3.SSS1)and[A\.3\.2](https://arxiv.org/html/2607.09698#A1.SS3.SSS2)\.

### 5\.2Qualitative Analysis

##### DMD

\(as seen in Figure[5](https://arxiv.org/html/2607.09698#S5.F5)\) projects each latent trajectory onto its dominant modes of variation, showing how the latent representations are spatially organized across the reasoning chain\. Tightly clustered latent states indicate contraction toward a bounded region, while spreading latent states indicate expansion away from it\. For the CODI method, the vanilla CoT setting shows a two\-lobe pattern with the latent CoT tokens tightly packed into both lobes across all steps, a sign of stable attractor\-like organization where the trajectory remains bounded throughout the chain\. In the SIM\-CoT setting, the two\-lobe pattern is preserved with even tighter clustering, indicating that the supervision reinforces this bounded behavior\. For the COCONUT method, the vanilla CoT setting exhibits a butterfly pattern where the latent states start near the center att=0t=0and spread outward byt=5t=5, a sign of expanding behavior where the model explores across latent space as the chain progresses\. Whereas, in the SIM\-CoT setting, the butterfly geometry is retained but with reduced spread at later steps, showing that the supervision constrains the exploration without changing the underlying geometric structure\.

![Refer to caption](https://arxiv.org/html/2607.09698v1/x14.png)

![Refer to caption](https://arxiv.org/html/2607.09698v1/x15.png)

Figure 5:DMD trajectory projections across latent steps for methods \(CODI \(left\) and COCONUT\(right\)\): COCONUT exhibits a butterfly pattern with latent states diverging outward across steps, reflecting an unstable system exploring multiple reasoning paths simultaneously\. CODI shows tightly packed representations clustering into two dense, well\-separated regions across all latent steps, indicating a convergent and stable reasoning dynamic\.
\(N = 8,792 samples x T = 6 tokens\)
##### PHATE

\(as seen in Figure[6](https://arxiv.org/html/2607.09698#S5.F6), right\) embeds the latent trajectory using a multi\-scale diffusion process that preserves both local neighborhoods and the global manifold structure, making it suitable for showing how latent representations move through different regions of the reasoning manifold\. For the CODI method, the vanilla CoT setting shows a two\-lobe pattern with compact yet interleaved representations across latent steps, a sign of convergent organization where the trajectory remains bounded throughout the chain\. In the SIM\-CoT setting, the structure becomes more compact and the temporal separation between latent steps reduces further, indicating that the supervision tightens the convergence onto a single region\. For the COCONUT method, the vanilla CoT setting shows each latent step occupying a distinct region of the manifold, with the beginning tokens \(purple\) and end tokens \(yellow/green\) positioned in close proximity, a possible indication of shortcut pathways where the model bridges early and late representations directly\. Whereas, in the SIM\-CoT setting, the distinct\-region structure is preserved but the proximity between beginning and end tokens reduces, showing that the supervision retains the regional separation while reducing the shortcut tendency\.

![Refer to caption](https://arxiv.org/html/2607.09698v1/x16.png)

![Refer to caption](https://arxiv.org/html/2607.09698v1/x17.png)

Figure 6:PHATE trajectory projections across latent steps for methods \(CODI \(left\) and COCONUT\(right\)\): COCONUT exhibits each latent step occupying distinct regions of the manifold, with beginning and end tokens positioned in closer proximity\-suggesting the formation of shortcut pathways in the reasoning process\. CODI shows interleaved representations across steps, consistent with its convergent behavior observed in the DMD projection\. \(Number of samples = 8,792 samples, 6 tokens\)
##### UMAP

\(as seen in Figure[7](https://arxiv.org/html/2607.09698#S5.F7), right\) projects the latent trajectory while preserving local neighborhood relationships and balancing global structure, showing how latent representations cluster by reasoning step\. For the CODI method, the vanilla CoT setting shows two main clusters: middle tokens \(3r​d3^\{rd\}and4t​h4^\{th\}\) scattered across one cluster while beginning and end tokens are contained in the other, a sign of role separation where mid\-chain steps explore broadly and terminal steps remain bounded\. In the SIM\-CoT setting, the cluster boundaries tighten and the middle\-token spread reduces, indicating that the supervision concentrates the latent representations more uniformly across steps\. For the COCONUT method, the vanilla CoT setting shows each latent step occupying a distinct region of the embedding space, a sign that latent representations carry step\-specific roles in the reasoning process\. Whereas, in the SIM\-CoT setting, the step\-wise regional separation is preserved with tighter cluster boundaries per step, showing that the supervision sharpens the role separation without changing the underlying organization\.

![Refer to caption](https://arxiv.org/html/2607.09698v1/x18.png)

![Refer to caption](https://arxiv.org/html/2607.09698v1/x19.png)

Figure 7:UMAP trajectory projections across latent steps for methods \(CODI \(left\) and COCONUT\(right\)\): COCONUT shows each latent step occupying distinct regions, reinforcing step\-specific reasoning roles\. CODI forms two clusters — middle tokens diffused broadly, while beginning and end tokens remain densely contained\. \(Number of samples = 8,792 samples, 6 tokens\)We also show the t\-SNE and PCA plots for the above methods \(GSM8k \+ Vanilla CoT setting\) in the Appendix[A\.2\.2](https://arxiv.org/html/2607.09698#A1.SS2.SSS2)and[A\.2\.1](https://arxiv.org/html/2607.09698#A1.SS2.SSS1)respectively\.

## 6Discussion

We present our findings based on the above analysis as follows,

Dynamics of CODI and COCONUT exhibit different latent stability types\- Lyapunov metric analysis \(see Figure[3](https://arxiv.org/html/2607.09698#S5.F3)\) highlights CODI behaves as astable attractortype, where latent representations converge as timestep approaches infinity, which is consistent with its two lobe pattern \(see Figure[6](https://arxiv.org/html/2607.09698#S5.F6)\(left\)\)\. COCONUT, in contrast, exhibitsunstable expandingdynamics, where it starts with stable latent space and diverges as time progresses reflecting its butterfly pattern \(see Figure[5](https://arxiv.org/html/2607.09698#S5.F5)\(right\)\)\.

SIM\-CoT shows better latent stability than Vanilla Latent COT paradigm\- From a dynamical perspective, we see how SIM\-CoT version of CODI and COCONUT show more stable behavior than their vanilla CoT counterparts in Step\-to\-Step change \(Figure[2](https://arxiv.org/html/2607.09698#S5.F2)\) and Lyapunov stability \(Figure[3](https://arxiv.org/html/2607.09698#S5.F3)\)\.

Latent geometry for CODI and COCONUT mirrors the training objective\- In Figure[7](https://arxiv.org/html/2607.09698#S5.F7), COCONUT exhibits more coherent and well\-separated latent CoT structure, consistent with its curriculum training which transitions from explicit to implicit tokens\. By contrast, CODI shows more diffuse latent trajectories, consistent with its training objective centered on distillation rather than enforcing structure across individual latent CoT tokens\.

Underlying cognitive mechanisms for CODI and COCONUT are fundamentally different\- Based on the latent representation structures seen in DMD plots, we notice the following,

- •COCONUT follows acomputationstrategy where itpersistsall reasoning trajectories simultaneously \(denoted by the spread seen in later stages\) and continues to expand on it\.
- •CODI follows aclassificationstrategy where all the reasoning trajectoriesconverge to two modesand the reasoning process moves between these modes before finalizing the solution\.

Overall, the two methods solve problems through fundamentally opposite dynamics\. In COCONUT, doing more work in latent space is associated with getting the answer right\. In CODI, shorter and simpler paths are the ones that lead to correct answers\.

## 7Conclusion

This paper provides an insight into the dynamics for the latent CoT reasoning methods\. Leveraging an analytical framework grounded in dynamical systems, we demonstrate how these methods differ fundamentally in their stability profiles, training procedures, and the cognitive strategies used in problem solving\. We show that interpreting latent CoT tokens as dynamical reasoning states provides the community with another axis of interpretability for further improvements in this space\. Our experiments mainly focus on the GSM8k dataset and the GPT\-2 Small model\. We also focus mainly on the CODI and COCONUT latent CoT reasoning methods, and additionally use the SIM\-CoT training paradigm to show how are analysis is paradigm agnostic in nature\. Future works should investigate on the stability types for CODI and COCONUT on long CoT chains, and whether interventions in latent space can improve downstream performance\. Other directions include extending this framework to larger reasoning models like Llama, Deepseek\-distill\-Llama3\-8b, etc\. and other datasets like MATH and Strategy\-QA\.

## Impact Statement

Our framework establishes a trajectory\-based analytical paradigm grounded in dynamical systems theory to formalize the interpretability of latent reasoning processes\. This research facilitates the interpretability of reasoning dynamics in latent CoT reasoning technique, enabling the identification of dynamical structures and stability modes in explicit CoT\. These findings offer a robust foundation for the deployment of resource\-efficient latent reasoning architectures\.

## References

- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman \(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§4\.1](https://arxiv.org/html/2607.09698#S4.SS1.p1.1)\.
- S\. Goyal, B\. Peters, M\. E\. Granda, A\. V\. Narmadha, D\. Yugeswardeenoo, C\. S\. McDougall, S\. O’Brien, A\. Panda, K\. Zhu, and C\. Blondin \(2025\)Scratchpad thinking: alternation between storage and computation in latent reasoning models\.InNeurIPS 2025 Workshop on Regularization in Machine Learning \(RegML\),Note:Available at[https://openreview\.net/forum?id=rdmnQIAdBq](https://openreview.net/forum?id=rdmnQIAdBq)Cited by:[§1](https://arxiv.org/html/2607.09698#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.09698#S2.SS2.p1.1)\.
- Training large language models to reason in a continuous latent space\.arXiv preprint arXiv:2412\.06769\.Note:Accepted at COLM 2025Cited by:[§2\.1](https://arxiv.org/html/2607.09698#S2.SS1.p1.1),[§4\.2](https://arxiv.org/html/2607.09698#S4.SS2.p1.1)\.
- P\. Holmes \(1990\)Poincaré, celestial mechanics, dynamical\-systems theory and “chaos”\.Physics Reports193\(3\),pp\. 137–163\.External Links:[Document](https://dx.doi.org/10.1016/0370-1573%2890%2990012-Q)Cited by:[§2\.3](https://arxiv.org/html/2607.09698#S2.SS3.p1.1)\.
- Y\. J\. John, K\. S\. Sawyer, K\. Srinivasan, E\. J\. Müller, B\. R\. Munn, and J\. M\. Shine \(2022\)It’s about time: linking dynamical systems with human neuroimaging to understand the brain\.Network Neuroscience6\(4\),pp\. 960–979\.External Links:[Document](https://dx.doi.org/10.1162/netn%5Fa%5F00230)Cited by:[§2\.3](https://arxiv.org/html/2607.09698#S2.SS3.p1.1)\.
- Z\. Li, X\. Bai, K\. Chen, Y\. Li, J\. Yang, C\. Lin, and M\. Zhang \(2026\)Dynamics within latent chain\-of\-thought: an empirical study of causal structure\.arXiv preprint arXiv:2602\.08783\.Cited by:[§1](https://arxiv.org/html/2607.09698#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.09698#S2.SS2.p2.1)\.
- J\. Liang and L\. Pan \(2026\)Do latent\-cot models think step\-by\-step? a mechanistic study on sequential reasoning tasks\.arXiv preprint arXiv:2602\.00449\.Cited by:[§1](https://arxiv.org/html/2607.09698#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.09698#S2.SS2.p1.1),[§2\.2](https://arxiv.org/html/2607.09698#S2.SS2.p2.1)\.
- L\. McInnes, J\. Healy, and J\. Melville \(2018\)UMAP: uniform manifold approximation and projection for dimension reduction\.arXiv preprint arXiv:1802\.03426\.Cited by:[§3\.2](https://arxiv.org/html/2607.09698#S3.SS2.p3.1)\.
- K\. R\. Moon, D\. van Dijk, Z\. Wang, S\. Gigante, D\. B\. Burkhardt, W\. S\. Chen, K\. Yim, A\. van den Elzen, M\. J\. Hirn, R\. R\. Coifman,et al\.\(2019\)Visualizing structure and transitions in high\-dimensional biological data\.Nature Biotechnology37,pp\. 1482–1492\.Cited by:[§3\.3](https://arxiv.org/html/2607.09698#S3.SS3.p2.1)\.
- Q\. T\. Pham, M\. Jafari, and F\. Salim \(2026\)Is my model ‘mind blurting’? interpreting the dynamics of reasoning tokens with recurrence quantification analysis \(RQA\)\.arXiv preprint arXiv:2602\.06266\.Cited by:[§1](https://arxiv.org/html/2607.09698#S1.p2.1),[§2\.3](https://arxiv.org/html/2607.09698#S2.SS3.p2.1),[§2\.3](https://arxiv.org/html/2607.09698#S2.SS3.p3.1)\.
- Z\. Shen, H\. Yan, L\. Zhang, Z\. Hu, Y\. Du, and Y\. He \(2025\)CODI: compressing chain\-of\-thought into continuous space via self\-distillation\.arXiv preprint arXiv:2502\.21074\.Cited by:[§2\.1](https://arxiv.org/html/2607.09698#S2.SS1.p1.1),[§4\.2](https://arxiv.org/html/2607.09698#S4.SS2.p1.1)\.
- J\. H\. Tu, C\. W\. Rowley, D\. M\. Luchtenburg, S\. L\. Brunton, and J\. N\. Kutz \(2013\)On dynamic mode decomposition: theory and applications\.arXiv preprint arXiv:1312\.0041\.Cited by:[§3\.3](https://arxiv.org/html/2607.09698#S3.SS3.p1.2)\.
- L\. van der Maaten and G\. Hinton \(2008\)Visualizing data using t\-SNE\.Journal of Machine Learning Research9,pp\. 2579–2605\.Cited by:[§3\.2](https://arxiv.org/html/2607.09698#S3.SS2.p2.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. Chi, Q\. Le, and D\. Zhou \(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Vol\.35\.Cited by:[§2\.1](https://arxiv.org/html/2607.09698#S2.SS1.p1.1)\.
- X\. Wei, X\. Liu, Y\. Cao, J\. Wang, and D\. Lin \(2025\)SIM\-cot: supervised implicit chain\-of\-thought\.arXiv preprint arXiv:2509\.20317\.Cited by:[§2\.1](https://arxiv.org/html/2607.09698#S2.SS1.p3.1),[§4\.2](https://arxiv.org/html/2607.09698#S4.SS2.p1.1)\.
- S\. Yu, Y\. Xiong, J\. Wu, X\. Li, T\. Yu, X\. Chen, R\. Sinha, J\. Shang, and J\. McAuley \(2025\)Explainable chain\-of\-thought reasoning: an empirical analysis on state\-aware reasoning dynamics\.arXiv preprint arXiv:2509\.00190\.Cited by:[§1](https://arxiv.org/html/2607.09698#S1.p2.1),[§2\.3](https://arxiv.org/html/2607.09698#S2.SS3.p2.1),[§2\.3](https://arxiv.org/html/2607.09698#S2.SS3.p3.1)\.

## Appendix AAppendix

### A\.1Hyperparameters

#### A\.1\.1Dimensionality Reduction Hyperparameters

All reduction methods project tok=2k=2andk=3k=3dimensions for visualization\. The following hyperparameters are fixed across all runs:

- •t\-SNE:Perplexity=5\.0=5\.0\(auto\-adjusted tomin⁡\(5\.0,max⁡\(2\.0,\(N−1\)/3\)\)\\min\\\!\\bigl\(5\.0,\\,\\max\(2\.0,\\,\(N\{\-\}1\)/3\)\\bigr\)for small batches\)\.
- •UMAP:nneighbors=5n\_\{\\text\{neighbors\}\}=5,min\_dist=0\.1\\text\{min\\\_dist\}=0\.1, Euclidean metric\.
- •DMD:Full\-rank SVD \(svd\_rank=−1\\text\{svd\\\_rank\}=\-1, effective rank=min⁡\(D,T−1\)=5=\\min\(D,\\,T\{\-\}1\)=5forT=6T\{=\}6step trajectories\); batched GPU computation across allNNsamples simultaneously\. Eigenvalue magnitudes\|λ\|\|\\lambda\|classify modes as stable \(\|λ\|<1\|\\lambda\|<1\) or unstable \(\|λ\|\>1\|\\lambda\|\>1\)\.
- •PHATE:kk\-nearest neighborsk=5k=5, diffusion timet=autot=\\text\{auto\}\.

#### A\.1\.2Perturbation Stability Hyperparameters

Perturbation experiments inject Gaussian noise directly into input embeddings and re\-run inference\. Noise standard deviationσ=0\.01\\sigma=0\.01andn=3n=3independent perturbation runs per sample are used\. Divergence is measured as the meanℓ2\\ell\_\{2\}distance between clean and perturbed trajectories at each step, alongside a scale\-invariant relative divergence∥ztperturbed−zt∥/∥zt∥\\lVert z\_\{t\}^\{\\text\{perturbed\}\}\-z\_\{t\}\\rVert/\\lVert z\_\{t\}\\rVert\.

#### A\.1\.3Metric Plot Conventions

All metric plots display mean curves±\\pmone standard deviation across samples, split by prediction correctness \(correct and incorrect\)\. In trajectory plots, each point represents one sample colored by latent step indextt; prediction correctness is encoded by marker shape \(filled circle = correct, open triangle = incorrect\), with mean centroid paths overlaid\.

### A\.2Additional Trajectory Visualizations

#### A\.2\.1PCA Projections

Linear PCA projects the latent trajectory tensor onto the two directions of greatest variance\. Color encodes the latent step index fromt=0t\{=\}0\(purple\) tot=5t\{=\}5\(yellow\); filled circles mark correct outputs and open triangles mark incorrect outputs\. Solid green and dashed red lines show the correct and incorrect centroid paths respectively\.

![Refer to caption](https://arxiv.org/html/2607.09698v1/x20.png)\(a\)COCONUT · GSM8K
![Refer to caption](https://arxiv.org/html/2607.09698v1/x21.png)\(b\)CODI · GSM8K

Figure 8:PCA projections of latent trajectories for COCONUT \(left\) and CODI \(right\)\.For COCONUT, the point cloud splits bimodally along PC2 \(the vertical axis\): late stepst=4t\{=\}4and55occupy an upper lobe above PC2\>0\>0, while early stepst=0t\{=\}0and11occupy a lower lobe below PC2<−10<\{\-\}10\. Mid\-stepst=2t\{=\}2and33bridge the two lobes\. Both centroid paths trace nearly identical zig\-zag routes through this structure, sitting on top of one another throughout the chain\. PC2 is acting as a temporal axis: the principal directions of variance encode where in the chain a vector lives, not whether the chain succeeds\. Linear PCA cannot separate correct from incorrect trajectories on this model\.

For CODI, the bimodal structure is preserved but rotated: the split is along PC1 \(the horizontal axis\) rather than PC2\. Late stepst=3t\{=\}3to55form a right lobe at PC1\>30\>30, and early stepst=0t\{=\}0and11form a left lobe at PC1<−30<\{\-\}30, with a quieter central gap between them\. Centroid paths sweep horizontally from left to right\. The two\-lobe horizontal axis in CODI mirrors COCONUT’s two\-lobe vertical axis, but the orientation reflects a different dominant mode structure\. In both models, the principal directions of variance encode reasoning phase rather than problem type\.

#### A\.2\.2t\-SNE Projections

t\-SNE preserves local neighborhood structure and is used here to assess whether correct and incorrect trajectories occupy distinct regions at each step\. Unlike PCA, t\-SNE is nonlinear and does not preserve global distances\.

![Refer to caption](https://arxiv.org/html/2607.09698v1/x22.png)\(a\)COCONUT · GSM8K
![Refer to caption](https://arxiv.org/html/2607.09698v1/x23.png)\(b\)CODI · GSM8K

Figure 9:t\-SNE projections of latent trajectories for COCONUT \(left\) and CODI \(right\)\.For COCONUT, the projection produces a roughly disc\-shaped cloud spanning approximately±100\{\\pm\}100in both dimensions\. A pronounced yellow island att=5t\{=\}5sits in the upper center of the disc, indicating that late\-step representations collapse into a single localized attractor region\. Early steps \(t=0t\{=\}0\) scatter across the periphery in distributed pockets\. Both centroid paths walk overlapping zig\-zag routes through the dense middle of the disc\. The correctness signal in t\-SNE coordinates is weak\. The plot is best read as evidence that the late chain commits to a small region while the early chain explores broadly — the local\-neighborhood signature of the same compression visible in the UMAP and DMD projections in the main paper\.

For CODI, the disc shape is similar in overall envelope but the internal structure is more diffuse: sub\-clusters bleed into each other rather than forming sharp islands\. Yellowt=5t\{=\}5vectors appear in scattered local pockets without a single clean breakaway region\. A dark purple concentration of early steps sits in the upper portion of the disc; mid\-step teal forms a diffuse band through the middle\. Centroid paths run as short, heavily overlapping trajectories through the dense center\. The correctness signal is weak, as in COCONUT\. This diffuseness is consistent with CODI’s distillation procedure flattening the latent geometry: the model has many equally important local structures rather than a few dominant attractor basins\.

#### A\.2\.33D Qualitative Trajectory plots of Vanilla CODI and COCONUT with GSM8K

##### DMD

For COCONUT, the latent states start near the center fort=0t\{=\}0and spread outward byt=5t\{=\}5showing that the model starts diverging through latent space at later reasoning steps proving its ”chaotic” behaviour as described by direction consistency \(Figure[10](https://arxiv.org/html/2607.09698#A1.F10), right\)\.

Unlike COCONUT, CODI shows a unclear similar divergence in latent steps as they increase \(Figure[10](https://arxiv.org/html/2607.09698#A1.F10), left\)\.

![Refer to caption](https://arxiv.org/html/2607.09698v1/x24.png)

![Refer to caption](https://arxiv.org/html/2607.09698v1/x25.png)

Figure 10:3D DMD trajectory projections across latent steps for CODI \(left\) and COCONUT\(right\)\. \(Number of samples = 8,792 samples, 6 tokens\)
##### PHATE

We can see from the PHATE projections of COCONUT that each latent step explores different latent regions demonstrating their distinct role in the reasoning process \(Figure[11](https://arxiv.org/html/2607.09698#A1.F11), right\)\.

Unlike COCONUT, CODI shows a unclear similar clustering between latent steps \(Figure[11](https://arxiv.org/html/2607.09698#A1.F11), left\)\.

![Refer to caption](https://arxiv.org/html/2607.09698v1/x26.png)

![Refer to caption](https://arxiv.org/html/2607.09698v1/x27.png)

Figure 11:3D PHATE trajectory projections across latent steps for CODI \(left\) and COCONUT\(right\)\.\(Number of samples = 8,792 samples, 6 tokens\)

#### A\.2\.43D Qualitative Trajectory Plots of SIM\-CoT CODI and COCONUT with GSM8K

##### DMD

DMD Unlike Vanilla CODI \(Figure 10, left\), CODI under SIM\-CoT paradigm \(Project Page, Section S\.1555[https://sabariiyyappan\.github\.io/Latent\-CoT/](https://sabariiyyappan.github.io/Latent-CoT/)\) shows that the latent states start near the center fort=0t\{=\}0and spread outward byt=5t\{=\}5showing that the model starts diverging through latent space at later reasoning steps like Vanilla COCONUT \(Figure[10](https://arxiv.org/html/2607.09698#A1.F10), right\) and SIM\-CoT COCONUT \(Project Page, Section S\.1\)\.

##### PHATE

We can see from the COCONUT PHATE projections that each latent step explores different latent regions, demonstrating their distinct role in the reasoning process \(Project Page, Section S\.1666[https://sabariiyyappan\.github\.io/Latent\-CoT/](https://sabariiyyappan.github.io/Latent-CoT/)\)\.

Unlike COCONUT, CODI PHATE shows a unclear similar clustering between latent steps \(Project Page, Section S\.1\)\.

### A\.3Metric Plots

#### A\.3\.1Arc Length

Figures 12\-15 show the arc length distributions for CODI and COCONUT under Vanilla and SIM\-CoT777SIM\-CoT is described in Section S\.2 of the project page: https://sabariiyyappan\.github\.io/Latent\-CoT/settings on GSM8K \(N=8,792N=8\{,\}792each\)\.Each figure shows three panels: \(a\) the arc\-length histogram split by correct \(green\) and incorrect \(red\) predictions with mean lines, \(b\) a boxplot of arc length by correctness, and \(c\) a per\-concept strip plot with mean±1​σ\\pm 1\\sigmaacross the seven GSM8K concept categories\. For both methods and both training paradigms, correct predictions show slightly longer arc lengths than incorrect ones, with CODI’s distribution shifted to a higher absolute range than COCONUT’s; SIM\-CoT compresses CODI’s distribution while leaving COCONUT’s largely unchanged, indicating that the supervision reduces CODI’s reasoning effort without altering COCONUT’s\.

![Refer to caption](https://arxiv.org/html/2607.09698v1/x28.png)Figure 12:Arc length distribution for CODI on GSM8K \(Vanilla setting,N=8,792N=8\{,\}792\)\.![Refer to caption](https://arxiv.org/html/2607.09698v1/x29.png)Figure 13:Arc length distribution for COCONUT on GSM8K \(Vanilla setting,N=8,792N=8\{,\}792\)\.
#### A\.3\.2Perturbation stability

Figures[14](https://arxiv.org/html/2607.09698#A1.F14)and[15](https://arxiv.org/html/2607.09698#A1.F15)show the perturbation relative divergence

‖ztperturbed−ztclean‖2/‖ztclean‖2\\\|z\_\{t\}^\{\\text\{perturbed\}\}\-z\_\{t\}^\{\\text\{clean\}\}\\\|\_\{2\}/\\\|z\_\{t\}^\{\\text\{clean\}\}\\\|\_\{2\}for SIM\-CoT CODI and SIM\-CoT COCONUT on GSM8K across three Gaussian noise levels \(σ∈\{0\.01,0\.1,1\.0\}\\sigma\\in\\\{0\.01,0\.1,1\.0\\\}\)\. Each panel shows correct \(green\) and incorrect \(red\) mean curves with±1\\pm 1standard\-deviation bands acrossn=3n=3perturbation runs per sample\. The incorrect\-above\-correct ordering is preserved atσ∈\{0\.01,0\.1\}\\sigma\\in\\\{0\.01,0\.1\\\}for both methods,a sign that uncertain reasoning paths are more sensitive to input\-embedding perturbations; COCONUT’s relative divergence sits consistently higher than CODI’s at everyσ\\sigma, reflecting COCONUT’s smaller clean\-state magnitudes amplifying the proportional noise effect\.

![Refer to caption](https://arxiv.org/html/2607.09698v1/x30.png)Figure 14:Perturbation relative divergence for SIM\-CoT CODI on GSM8K atσ=0\.01\\sigma=0\.01\(left\),σ=0\.1\\sigma=0\.1\(middle\),andσ=1\.0\\sigma=1\.0\(right\)\.![Refer to caption](https://arxiv.org/html/2607.09698v1/x31.png)Figure 15:Perturbation relative divergence for SIM\-CoT COCONUT on GSM8K atσ=0\.01\\sigma=0\.01\(left\),σ=0\.1\\sigma=0\.1\(middle\),andσ=1\.0\\sigma=1\.0\(right\)\.
#### A\.3\.3Concept\-wise Metric Plots

Concept\-wise metric plots are provided in Section S\.3 of the project page888[https://sabariiyyappan\.github\.io/Latent\-CoT/\#supp\-concept](https://sabariiyyappan.github.io/Latent-CoT/#supp-concept)\. The supplementary figures include step\-to\-step change, direction consistency, and Lyapunov sensitivity analyses for both CODI and COCONUT across the seven GSM8K math concept categories\. Each panel shows correct \(green\) and incorrect \(red\) mean curves with standard deviation bands for a single concept\. The per\-concept profiles are consistent with the aggregated results presented in the main paper, indicating that the observed dynamics are primarily a property of the model architecture rather than the distribution of problem types\.