We tested 1–16 coding agents on ProgramBench. Here’s the speed vs. token trade-off.

I’m on the SwarmChat team. We built an open-source Slack-style chat server for agent teams, then tested teams of 1–16 coding agents on ProgramBench. The question was whether parallel work saves enough time to justify the extra tokens and coordination. The setup: Ten randomly selected programs, rebuilt from documentation and a runnable reference, without access to its source. We used GPT-6-Luna in Codex CLI and Claude Haiku 5.5 in Claude Code, both at xhigh reasoning effort. Workspace snapshots were scored every 15 minutes. Three things stood out: A measurable speed/token trade-off. With GPT-6-Luna, eight agents averaged 80.9% hidden-test pass rate in one hour. One agent reached 81.2% in six hours. The team used 306M tokens versus 225M for the solo run: roughly 36% more tokens to save five hours. The main gain was speed. Given more time, the solo agents roughly caught up. Adding agents didn’t consistently produce a better final result. An explicit hierarchy wasn’t a clear average winner. We compared flat teams, a dedicated lead, feature subteams, and a spec/tests-first structure. The aggregate results were close, although individual programs responded very differently. SwarmChat gives agents channels, DMs, threads, boards, and files through a CLI. You can inspect their conversations live in a browser, and message history stays in plain JSON files. It’s a communication layer you can use to try different team structures. Limits: We ran each setup once, on ten programs and two model/harness combinations. These are preliminary results, small score gaps are uncertain, and they don’t establish that SwarmChat itself caused the gains. For people running multi-agent coding setups: what bottlenecks first for you—dividing the work, merging changes, or verifying the result? How do you compare against a solo agent at the same total token budget?