@yingfan_bot: New paper on Looped Transformers! Latent reasoning is fast, but struggles to match CoT-level accuracy at scale. Can loo…
Summary
A new paper on Looped Transformers finds that a looped padded backbone provides a parallel workspace for latent reasoning, enabling supervision similar to explicit chain-of-thought (CoT) and achieving both speed and accuracy.
View Cached Full Text
Cached at: 07/16/26, 12:17 PM
New paper on Looped Transformers!
Latent reasoning is fast, but struggles to match CoT-level accuracy at scale. Can looped Transformers give us both?
We find: yes! A looped padded backbone turns out to be a surprisingly simple recipe that works – It gives latent thoughts a parallel workspace that can be supervised similarly to explicit CoT.
Similar Articles
@ZhihuFrontier: Half a year ago, a Zhihu contributor predicted that the next Transformer would absorb loops, recurrent state, sparse ro…
A Zhihu contributor's half-year-old prediction that the next Transformer would absorb loops, recurrent state, sparse routing, and latent reasoning is gaining relevance as Loop Engineering advances. The article explores how future Transformer architectures may evolve into hybrid models blending linear-complexity layers for background context with attention for precise reasoning, plus finer-grained sparsity and native System 2 reasoning.
@askalphaxiv: Another cool research on Looped Transformers They ask the question: "Can we loop a frozen, off-the-shelf checkpoint dir…
This research introduces a technique to loop frozen, off-the-shelf transformer checkpoints at inference time by using damped Runge-Kutta substeps, treating transformer layers as Euler steps in a residual ODE. This allows extra latent compute without fine-tuning, architecture changes, or new weights, showing gains on knowledge tasks like MMLU-Pro, GPQA, and ARC.
Decoding Looped Transformers Better for (Almost) Free
The paper introduces LoopCD, a training-free contrastive decoding framework for looped transformers that uses intermediate recurrent states as weak/strong prediction pairs, delivering substantial gains (e.g., Ouro-2.6B-Thinking's AIME 2024 pass@1 rising from 61.88% to 73.33%) while enabling halving of recurrent loops and 22.5%-48.2% reductions in forward FLOPs.
@machinestein: ICML 2026: Latent Reasoning in TRMs is Secretly a Policy Improvement Operator Why does recursive reasoning, especially …
The paper reveals that latent reasoning in transformer-based reasoning models (TRMs) functions as a policy improvement operator, and proposes an algorithm that enhances learning and inference efficiency by up to 18x.
Scheduling Recursive Reasoning in Looped Transformers
The paper proposes TAPS (Trajectory Adaptive Progress–Fluctuation Scheduler), which adaptively adjusts the step size of recurrent updates in looped transformers by decomposing terminal loss sensitivity into persistent progress and centered fluctuation, improving accuracy without retraining and delivering up to 1.56× wall-clock speedup when applied during training.