Loop the Loopies!

arXiv cs.CL Papers
looped-transformer mixture-of-experts reasoning math-ai science-ai transformer-models post-training

Summary

Loopie is a new looped Transformer model that achieves gold-medal performance at the 2025 IMO and IPhO without external tools, using a novel post-training pipeline. It outperforms vanilla Transformers trained with the same compute budget.

arXiv:2607.16051v1 Announce Type: new Abstract: We present Loopie, the most powerful looped Transformer to date. The Loopie series consists of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active parameters and a 6Bparameter model with 0.6B active parameters. Looped Transformers have long faced a challenge: given an N-fold increase in pre-training compute, increasing the parameter count by a factor of N usually outperforms looping a model N times. Loopie addresses this challenge. Extensive ablation studies, including comparisons with a vanilla 30B-A3B model, show that Loopie substantially outperforms vanilla Transformer baselines trained with the same compute budget. Our novel post-training pipeline equips Loopie with strong reasoning abilities. At the 2025 IMO and IPhO, Loopie achieves gold-medal performance without tools.
Original Article
View Cached Full Text

Cached at: 07/20/26, 09:36 AM 

# Loop the Loopies!
Source: [https://arxiv.org/abs/2607.16051](https://arxiv.org/abs/2607.16051)
[View PDF](https://arxiv.org/pdf/2607.16051)

> Abstract:We present Loopie, the most powerful looped Transformer to date\. The Loopie series consists of two Mixture\-of\-Experts \(MoE\) models: a 20B\-parameter model with 2B active parameters and a 6Bparameter model with 0\.6B active parameters\. Looped Transformers have long faced a challenge: given an N\-fold increase in pre\-training compute, increasing the parameter count by a factor of N usually outperforms looping a model N times\. Loopie addresses this challenge\. Extensive ablation studies, including comparisons with a vanilla 30B\-A3B model, show that Loopie substantially outperforms vanilla Transformer baselines trained with the same compute budget\. Our novel post\-training pipeline equips Loopie with strong reasoning abilities\. At the 2025 IMO and IPhO, Loopie achieves gold\-medal performance without tools\.

## Submission history

From: Zitian Gao \[[view email](https://arxiv.org/show-email/52340bf5/2607.16051)\] **\[v1\]**Fri, 17 Jul 2026 15:28:43 UTC \(829 KB\)

Similar Articles

Loop the Loopies!

Hugging Face Daily Papers

Loopie introduces looped Mixture-of-Experts Transformers that outperform vanilla transformers under the same compute budget, achieving gold-medal performance at the 2025 IMO and IPhO without tools.

@askalphaxiv: Another cool research on Looped Transformers They ask the question: "Can we loop a frozen, off-the-shelf checkpoint dir…

X AI KOLs Timeline

This research introduces a technique to loop frozen, off-the-shelf transformer checkpoints at inference time by using damped Runge-Kutta substeps, treating transformer layers as Euler steps in a residual ODE. This allows extra latent compute without fine-tuning, architecture changes, or new weights, showing gains on knowledge tasks like MMLU-Pro, GPQA, and ARC.

Multilingual-Multimodal-NLP/LoopCoder-V2 · Hugging Face

Reddit r/LocalLLaMA

LoopCoder-V2 is a 7B instruction-tuned code model built on the Parallel Loop Transformer (PLT), demonstrating non-monotonic test-time scaling with two loops providing the best gain-cost trade-off and significant improvements over baselines on code generation and reasoning benchmarks.

DeepLoop: Depth Scaling for Looped Transformers

arXiv cs.LG

DeepLoop introduces a residual scaling method for looped Transformers that adjusts for parameter visits, improving stability and performance when physical blocks are reused across multiple rounds.