Geometric Latent Reasoning Induces Shorter Generations in LLMs
Summary
Geometric Latent Reasoning (GLR) introduces a geometric path-approximation method for latent reasoning in LLMs, enabling shorter generations while maintaining accuracy across mathematical reasoning benchmarks.
View Cached Full Text
Cached at: 06/02/26, 03:36 PM
Paper page - Geometric Latent Reasoning Induces Shorter Generations in LLMs
Source: https://huggingface.co/papers/2606.02248
Abstract
Geometric Latent Reasoning formulates latent reasoning as a geometric path-approximation problem in pretrained token-embedding space, enabling continuous intermediate reasoning states that reduce generation length while maintaining accuracy.
Large language models solve complex problems by generating lengthy chains of explicit reasoning tokens. While effective, this makes reasoning expensive, length-sensitive, and constrained to (discrete) natural language. Whilelatent reasoningoffers a continuous alternative, determining useful structures for intermediate latent states is an open challenge. In this paper, we formulatelatent reasoningas ageometric path-approximation problemwithin the model’spretrained token-embedding space. We introduce GeometricLatent Reasoning(GLR), which uses a lightweighttransition headto predict iterative direction updates in embedding space. Using textualchain-of-thought tracesas anchors, GLR learns to approximatediscrete reasoning trajectorieswhile permittingcontinuous deviationsfrom exact token embeddings. Evaluations onmathematical reasoning benchmarksusingQwen3 modelsreveal anemergent phenomenon: geometriclatent reasoninginduces substantially shorter generations without an explicit length objective. By replacing early explicit reasoning with continuous latent steps, models often reach correct answers using substantially fewer total generation steps. These findings suggest that continuous trajectories act as compact intermediate reasoning states, exposing a new tradeoff betweenlatent computation budget,output length, andaccuracy.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2606\.02248
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.02248 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.02248 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.02248 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning
Proposes Chunk-Level Guided Generation, a training-free method using off-the-shelf LLMs as process scorers to select fixed-length candidate chunks during small model generation, significantly improving mathematical reasoning accuracy over majority voting and PRM guided search.
LGMT: Logic-Grounded Metamorphic Testing for Evaluating the Reasoning Reliability of LLMs
This paper introduces LGMT, a framework that uses first-order logic to generate semantically invariant test cases for evaluating LLM reasoning reliability. Experiments on six LLMs show that LGMT exposes hidden defects missed by static benchmarks, suggesting evaluation should focus on robustness under logical invariance.
Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key
This paper introduces ScaleLogic, a framework demonstrating that RL training compute scales as a power law with reasoning depth in LLMs. It highlights that logical expressiveness is key to improving downstream transfer and training efficiency.
HyperGuide: Hyperbolic Guidance for Efficient Multi-Step Reasoning in Large Language Models
This paper proposes HyperGuide, a method that distills reasoning progress into a hyperbolic geometric signal to guide step-by-step generation in LLMs, improving multi-step reasoning efficiency without explicit tree search.
SGR: A Stepwise Reasoning Framework for LLMs with External Subgraph Generation
Introduces SGR, a stepwise reasoning framework that enhances LLM reasoning by generating query-specific subgraphs from external knowledge bases, improving accuracy and factual reliability.