loop-transformers

The tweet from SemiAnalysis reports on GPT-6 Astra, an AI model that uses Loop Transformers, suggesting a new architectural approach in AI development.

This paper introduces LoopCoder-v2, a family of 7B parameter parallel loop transformers for code generation, and studies the optimal number of loops, finding that two loops yield significant gains while more loops cause degradation.