looped-transformers

Tag

Cards List
#looped-transformers

On the Chain-of-Thought Monitorability of Looped Language Models

arXiv cs.AI ↗ · 2026-10-05 Cached

This paper presents the first systematic evaluation of chain-of-thought monitorability in looped language models (LoopLMs), finding that deeper loop depths can reduce CoT monitorability on some tasks under stress tests, while looped architectures are not systematically less monitorable than size/depth-matched non-looped models.

0 favorites 0 likes
#looped-transformers

LiFT: Loop Flow Transformers

Hugging Face Daily Papers ↗ · 2026-10-04 Cached

LiFT introduces a family of looped generative models that scale computation by repeatedly applying a shared Diffusion Transformer (DiT) core, training each loop step on a continuous-depth flow-matching regression target that allows inference far beyond training depth. On ImageNet 256x256, LiFT-L/2 beats a dense DiT-XL/2 baseline by 3.34 FID points while using ~60% fewer parameters and 52% fewer inference FLOPs.

0 favorites 0 likes
#looped-transformers

Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts

Hugging Face Daily Papers ↗ · 2026-10-01 Cached

The paper proposes LOOM, a scalable recipe for Looped Mixture-of-Experts models that stabilizes recurrence via residual scaling and input re-injection while diversifying expert usage with per-loop routers, achieving stable scaling to 9-12 loops and improved perplexity and zero-shot accuracy on 100M-1.7B models.

0 favorites 0 likes
#looped-transformers

Decoding Looped Transformers Better for (Almost) Free

Hugging Face Daily Papers ↗ · 2026-10-01 Cached

The paper introduces LoopCD, a training-free contrastive decoding framework for looped transformers that uses intermediate recurrent states as weak/strong prediction pairs, delivering substantial gains (e.g., Ouro-2.6B-Thinking's AIME 2024 pass@1 rising from 61.88% to 73.33%) while enabling halving of recurrent loops and 22.5%-48.2% reductions in forward FLOPs.

0 favorites 0 likes
#looped-transformers

Scaling Laws for Looped Mixture of Experts

Hugging Face Daily Papers ↗ · 2026-09-30 Cached

Introduces Loop Scaling Laws, the first scaling law jointly modeling recurrence (looped transformers) and sparsity (MoE) alongside model size and data, showing looped MoEs match models ~2x larger on reasoning benchmarks at matched compute.

0 favorites 0 likes
#looped-transformers

Scheduling Recursive Reasoning in Looped Transformers

Hugging Face Daily Papers ↗ · 2026-09-29 Cached

The paper proposes TAPS (Trajectory Adaptive Progress–Fluctuation Scheduler), which adaptively adjusts the step size of recurrent updates in looped transformers by decomposing terminal loss sensitivity into persistent progress and centered fluctuation, improving accuracy without retraining and delivering up to 1.56× wall-clock speedup when applied during training.

0 favorites 0 likes
#looped-transformers

How to Loop MoE: Flatten the Experts, Untie the Attention

Hugging Face Daily Papers ↗ · 2026-09-28 Cached

A preprint introducing Foil, which flattens MoE experts into fewer layers with more experts per layer and more recurrent passes, plus untied attention per pass, achieving lower pretraining loss at matched parameters and compute.

0 favorites 0 likes
#looped-transformers

Improving Test-Time Scaling with Adaptive Looped Transformers

Hugging Face Daily Papers ↗ · 2026-09-28 Cached

This paper introduces TaH2, an adaptive looped transformer that improves test-time scaling by dynamically allocating extra iterations to beneficial tokens, achieving a 53% improvement in accuracy-compute slope over baselines on benchmarks like AIME.

0 favorites 0 likes
#looped-transformers

GPT-6 Astra, looped transformers, and hidden reasoning

Hacker News Top ↗ · 2026-09-09 Cached

The article discusses OpenAI's GPT-6 Astra model release, its exceptional performance in benchmarks like ARC-AGI-3, and explores looped transformers and hidden reasoning concepts.

0 favorites 0 likes
#looped-transformers

@rasbt: I put together a mega write-up on GPT-6 Astra & looped transformers. How looped transformers / recurrent depth works, c…

X AI KOLs Timeline ↗ · 2026-09-09 Cached

A comprehensive write-up explaining looped transformers and recurrent depth in AI, including cost-tradeoffs and whether they hide reasoning traces, with figures and a review of recent research.

0 favorites 0 likes
#looped-transformers

Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?

arXiv cs.AI ↗ · 2026-09-03 Cached

This paper investigates whether the global workspace functionality in transformers persists under recurrence by applying Jacobian lens analysis to looped and recurrent models, finding that recurrence alters workspace accessibility and content transport.

0 favorites 0 likes
#looped-transformers

OpenAI Astra and Looped Transformers (2 minute read)

TLDR AI ↗ · 2026-09-03 Cached

The article debunks the hype around OpenAI's Astra model, explaining that the 'looped transformer' concept is a minor architectural tweak involving layer reuse for increased capacity without added parameters, as seen in models like Nanbeige 4.2.

0 favorites 0 likes
#looped-transformers

Even AI 2027 co-authors are shocked at how fast AI is progressing

Reddit r/ArtificialInteligence ↗ · 2026-09-02

OpenAI's upcoming Astra model reportedly uses looped transformers without an interpretable chain of thought, indicating faster AI progress than predicted by AI 2027 forecasts.

0 favorites 0 likes
#looped-transformers

Looped Transformers with Source-Centered State Evolution

arXiv cs.LG ↗ · 2026-07-31 Cached

The paper proposes Source-Centered State Evolution (SCSE), a method for looped Transformers that reconciles input conditioning with reference-preserving shared recurrence, improving recurrent quality frontiers across multiple benchmarks.

0 favorites 0 likes
#looped-transformers

What Are Looped Transformers? Explained Clearly (8 minute read)

TLDR AI ↗ · 2026-07-28 Cached

Looped transformers reuse the same layers across multiple passes to trade parameter count for compute, achieving better reasoning with fewer weights. The article traces the idea back to the Universal Transformer (2018) and explains why it initially failed due to scaling laws and timing.

0 favorites 0 likes
#looped-transformers

When Does Recurrence Become an Algorithm? Convergence Selection in Weight-Tied Looped Transformers

arXiv cs.LG ↗ · 2026-07-24 Cached

This paper investigates when weight-tied looped transformers implement actual algorithms, introducing the budget law and showing mechanisms are portable across training budgets, with implications for adaptive computation and interpretability.

0 favorites 0 likes
#looped-transformers

Adaptive Depth in Looped Transformers: Diagnosing Learned Halting Gates and Trajectory Readouts

arXiv cs.LG ↗ · 2026-07-24 Cached

This paper studies adaptive depth in looped Transformers by separating trajectory formation from exit readout, showing that fixed-prior depth supervision often outperforms jointly trained gates and that poor adaptive-compute performance stems more from the induced trajectory than from gate expressivity.

0 favorites 0 likes
#looped-transformers

DeepLoop: Depth Scaling for Looped Transformers

arXiv cs.LG ↗ · 2026-07-16 Cached

DeepLoop introduces a residual scaling method for looped Transformers that adjusts for parameter visits, improving stability and performance when physical blocks are reused across multiple rounds.

0 favorites 0 likes
#looped-transformers

@yingfan_bot: New paper on Looped Transformers! Latent reasoning is fast, but struggles to match CoT-level accuracy at scale. Can loo…

X AI KOLs Timeline ↗ · 2026-07-15 Cached

A new paper on Looped Transformers finds that a looped padded backbone provides a parallel workspace for latent reasoning, enabling supervision similar to explicit chain-of-thought (CoT) and achieving both speed and accuracy.

0 favorites 0 likes
#looped-transformers

@gklambauer: G-RRM: Guiding Symbolic Solvers with Recurrent Reasoning Models Symbolic solver have to branch to check different choic…

X AI KOLs Timeline ↗ · 2026-07-03 Cached

This paper introduces G-RRM, a neuro-symbolic approach that uses recurrent reasoning models to guide symbolic solvers for constraint satisfaction problems, showing significant speedups in certain conditions.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback