Uncovering the Representation Geometry of Minimal Cores in Overcomplete Reasoning Traces

arXiv cs.AI Papers
reasoning-traces chain-of-thought representation-geometry minimal-cores overcompleteness llm-interpretability

Summary

This paper introduces the concept of 'minimal cores' in overcomplete reasoning traces, showing that on average 46% of steps can be removed while preserving the final answer, and that minimal cores improve trace separation and reduce intrinsic dimensionality.

arXiv:2605.14358v1 Announce Type: new Abstract: Language models often generate long chain-of-thought traces, but it remains unclear how much of this reasoning is necessary for preserving the final prediction. We study this through the lens of overcomplete reasoning traces: generated traces that contain more intermediate steps than are needed to support the model's answer. We define the minimal core as the smallest subset of steps that preserves either the final answer or predictive distribution, and introduce metrics for compression ratio, redundancy mass, step necessity, and necessity concentration. Across six deliberative reasoning benchmarks spanning arithmetic, competition mathematics, expert scientific reasoning, and commonsense multi-hop QA, we find substantial overcompleteness: on average, 46% of steps are removable under greedy minimal-core extraction while preserving the original answer in 86% of cases. We also find that predictive support is concentrated: the top three steps account for 65% of measured necessity mass on average. Beyond compression, minimal cores expose a cleaner geometry of reasoning: compared with full traces, they improve correct-incorrect trace separation by 11 points, reduce estimated intrinsic dimensionality by 34%, and transfer across model families with 85% off-diagonal answer retention. Theoretically, we establish existence of minimal sufficient subsets, local irreducibility guarantees for greedy elimination, and certificates of overcompleteness and sparse necessity. Together, these results suggest that full reasoning traces are often verbose and overcomplete, while minimal cores isolate the effective support underlying language-model predictions.
Original Article
View Cached Full Text

Cached at: 05/15/26, 06:23 AM 

# Uncovering the Representation Geometry of Minimal Cores in Overcomplete Reasoning Traces
Source: [https://arxiv.org/abs/2605.14358](https://arxiv.org/abs/2605.14358)
[View PDF](https://arxiv.org/pdf/2605.14358)

> Abstract:Language models often generate long chain\-of\-thought traces, but it remains unclear how much of this reasoning is necessary for preserving the final prediction\. We study this through the lens of overcomplete reasoning traces: generated traces that contain more intermediate steps than are needed to support the model's answer\. We define the minimal core as the smallest subset of steps that preserves either the final answer or predictive distribution, and introduce metrics for compression ratio, redundancy mass, step necessity, and necessity concentration\. Across six deliberative reasoning benchmarks spanning arithmetic, competition mathematics, expert scientific reasoning, and commonsense multi\-hop QA, we find substantial overcompleteness: on average, 46% of steps are removable under greedy minimal\-core extraction while preserving the original answer in 86% of cases\. We also find that predictive support is concentrated: the top three steps account for 65% of measured necessity mass on average\. Beyond compression, minimal cores expose a cleaner geometry of reasoning: compared with full traces, they improve correct\-incorrect trace separation by 11 points, reduce estimated intrinsic dimensionality by 34%, and transfer across model families with 85% off\-diagonal answer retention\. Theoretically, we establish existence of minimal sufficient subsets, local irreducibility guarantees for greedy elimination, and certificates of overcompleteness and sparse necessity\. Together, these results suggest that full reasoning traces are often verbose and overcomplete, while minimal cores isolate the effective support underlying language\-model predictions\.

## Submission history

From: Sanjoy Chowdhury \[[view email](https://arxiv.org/show-email/dd24d0d4/2605.14358)\] **\[v1\]**Thu, 14 May 2026 04:35:45 UTC \(558 KB\)

Similar Articles

CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning

Hugging Face Daily Papers

Contrastive Reflection (CORE) is a non-parametric algorithm that generates concise, interpretable insights from comparing successful and unsuccessful reasoning traces, enabling faster and more efficient self-improvement for language models with fewer samples and rollouts than existing methods.

Learning Coherent Representations: A Topological Approach to Interpretability

arXiv cs.LG

This paper introduces coherence, a geometric constraint for neural representations inspired by grid cells and head direction cells in the brain. Coherence ensures that features respond to geometrically connected regions of the data manifold, improving interpretability; the authors propose a differentiable objective (Coh) and validate it on synthetic data, rotated MNIST, and BERT token embeddings.

ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Traces

arXiv cs.CL

Introduces ReasoningFlow, a framework to capture discourse structures of large language model reasoning traces as directed acyclic graphs, enabling fine-grained analysis of reasoning behaviors like self-reflection and backtracking. Based on manual and automatic annotation of thousands of traces, it reveals structural similarities across models and that most erroneous steps do not contribute to final answers.

Inducing Reasoning Primitives from Agent Traces

arXiv cs.AI

Introduces Reasoning Primitive Induction, a method that mines successful ReAct traces to cluster recurrent reasoning moves into typed pseudo-tools, outperforming the original agent by tens of percentage points on benchmarks.