LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models
Summary
LoopUS is a post-training framework that converts pretrained LLMs into looped architectures for improved reasoning performance via latent-refinement and adaptive early exiting. It addresses computational costs and capability preservation issues found in existing looped computation methods.
View Cached Full Text
Cached at: 05/13/26, 04:13 AM
Paper page - LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models
Source: https://huggingface.co/papers/2605.11011
Abstract
LoopUS is a post-training framework that transforms pretrained LLMs into looped architectures for improved reasoning performance through latent-refinement and adaptive early exiting mechanisms.
Looped computationshows promise in improving the reasoning-oriented performance ofLLMsby scaling test-time compute. However, existing approaches typically require either training recurrent models from scratch or applying disruptive retrofits, which involve substantial computational costs and may compromise pretrained capabilities. To address these limitations, we introduce Looped Depth Up-Scaling (LoopUS), apost-training frameworkthat converts a standard pretrained LLM into a looped architecture. As a key technical contribution, LoopUS recasts the pretrained LLM into anencoder, alooped reasoning block, and adecoder. It operationalizes thislatent-refinement architecturethrough four core components: (1)block decomposition, guided bystaged representation dynamics; (2) aninput-dependent selective gateto mitigatehidden-state drift; (3)random deep supervisionformemory-efficient learningover long recursive horizons; and (4) aconfidence headforadaptive early exiting. Collectively, these mechanisms transform a standard non-looped model into a looped form while stabilizing it against both computational bottlenecks andrepresentation collapse. Through stable latent looping, LoopUS improves reasoning-oriented performance without extending the generated traces or requiring recurrent training from scratch. For more details, see https://thrillcrazyer.github.io/LoopUS
View arXiv pageView PDFProject pageGitHub1Add to collection
Get this paper in your agent:
hf papers read 2605\.11011
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper6
#### Thrillcrazyer/Qwen3_1.7B_LoopUS 2B• Updatedabout 2 hours ago • 18 • 1
#### Thrillcrazyer/Qwen3-4B_LoopUS 4B• Updatedabout 2 hours ago • 83 • 1
#### Thrillcrazyer/Qwen3_1.7B_LoopUS_SFT 2B• Updatedabout 2 hours ago • 7
#### Thrillcrazyer/Phi4_LoopUS 15B• Updatedabout 2 hours ago • 46
Browse 6 models citing this paper## Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.11011 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.11011 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Learning to Refine Hidden States for Reliable LLM Reasoning
Proposes ReLAR, a reinforcement-guided latent refinement framework that iteratively updates hidden representations in LLMs before decoding, improving reasoning reliability and efficiency compared to chain-of-thought methods.
Skip a Layer or Loop It? Learning Program-of-Layers in LLMs
This paper introduces Program-of-Layers (PoLar), a method that allows LLMs to dynamically skip or loop pretrained layers per input, improving accuracy and efficiency over fixed-depth inference.
Looped World Models
Looped World Models introduce iterative latent state refinement through shared transformer blocks, achieving 100x parameter efficiency while adapting computational depth to prediction complexity.
@Raman_bansal_: If you’ve trained an LLM, you’ve may have seen doom loop, in which a LLM endlessly repeats the same token or sentence, …
A Substack article explains the 'doom loop' problem in LLMs where models repeat tokens endlessly, and introduces Final Token Preference Optimization (FTPO) from Liquid AI as a method to detect and fix such loops during fine-tuning.
@DorothyDDU: LoopCoder-v2 is out Loop Transformers reuse the same block for recurrent hidden-state refinement — letting models “thin…
This paper introduces LoopCoder-v2, a family of 7B parameter parallel loop transformers for code generation, and studies the optimal number of loops, finding that two loops yield significant gains while more loops cause degradation.