Tag
This paper evaluates small recurrent-depth language models (Ouro-1.4B/2.6B-Thinking, HRM-Text-1B) fine-tuned jointly across 13 clinical NLP datasets, finding they outperform or match similarly sized Qwen3 transformers on most tasks, especially inference and question answering.
这篇论文提出 SanSi 方法，通过让小型决策模型（如 Ouro-1.4B）循环复用相同层数次推理，在多项决策任务上显著超越同规模单次前向模型，甚至超过 3 倍大的单遍模型，展示了循环深度小模型在内存受限场景下的潜力。
讨论 Universal Reasoning Model (URM) 与 Universal Transformer (UT) 通过深度上的循环计算等增强，能否在小规模从零训练下超越标准 Transformer 大模型，并追问前沿实验室是否已将其整合进前沿模型。
The paper proposes LOOM, a scalable recipe for Looped Mixture-of-Experts models that stabilizes recurrence via residual scaling and input re-injection while diversifying expert usage with per-loop routers, achieving stable scaling to 9-12 loops and improved perplexity and zero-shot accuracy on 100M-1.7B models.
A comprehensive write-up explaining looped transformers and recurrent depth in AI, including cost-tradeoffs and whether they hide reasoning traces, with figures and a review of recent research.
OpenAI's new Astra model uses a reasoning technique called opaque recurrence, which complicates chain-of-thought monitoring and raises concerns among AI safety experts about potential misalignment risks.
OpenAI's Astra model reportedly uses a looped transformer architecture for enhanced performance, though this may reduce the readability of internal reasoning. It is noted for reaching a critical cybersecurity capability threshold.
OpenAI introduces Astra, an AI model that utilizes recurrent depth for silent internal reasoning.
This paper explores surgically retrofitting a pretrained language model (e.g., Qwen2.5-0.5B) with recurrent depth, demonstrating that the resulting model can perform deeper latent reasoning, extrapolate past supervised depth, and outperform dense models fine-tuned to reason in tokens, while also revealing catastrophic interference limits.
This paper introduces CHERRY, a set of techniques for compute-efficient language models including selective token supervision, depth compression via recurrent unrolling, and a mixture of compressed experts, achieving significant efficiency gains on a Korean foundation model.