OpenAI's major results over the last 3 months came from scaling test-time compute with agent swarms. I've been building an open-source equivalent for the past year, and I'd love for you guys to try it.

Reddit r/LocalLLaMA Tools
test-time-compute agent-swarm open-source llm reasoning scaling

Summary

An open-source project implementing test-time compute scaling with agent swarms, reporting large accuracy gains on math and coding benchmarks when run on mid-size open models like Gemma and Qwen, spending up to hundreds of millions of tokens per problem.

I have been working on test-time scaling for quite some time now. Posted some of my previous work on this sub-reddit before. Recently, I made significant simplifications to the core system-design and it finally felt like now anybody can understand and directly use this on their hardest problems. Just on a side-note though, depending on the depth you set & your LLM's inference speed, it can take from 50 Minutes to 30 hours to give you back the results. For benchmarks I couldn't test this on frontier models or even large open models because it will be way too expensive. I directly only tested this on Gemma4-35B & Qwen-3.8-27B. They spent around 200M tokens / problem and there scores went from 14% to 65% on BrokenArxiv (Matharena) 15 problems or from 9% to 52% on 8 DeepSWE problems. I know that's almost unbelievable, but you can try and see that yourself. Repo Link: https://github.com/ryoiki-tokuiten/Deepthink
Original Article

Similar Articles

Swarm Scaling (13 minute read)

TLDR AI

This article analyzes the capabilities and scaling dynamics of large AI agent swarms, citing OpenAI's recent examples, and discusses their potential as a new form of inference scaling.

Sam Altman reveals OpenAI barely understood how to scale AI agent swarms even as they tackled some of mathematics' hardest problems. He expects major progress in swarm coordination within 3 months and believes the same approach could automate AI research itself.

Reddit r/singularity

Sam Altman revealed OpenAI barely understood how to scale AI agent swarms while tackling hard mathematics problems, forecasting major swarm-coordination progress within three months and suggesting the approach could automate AI research itself.