We asked our coding agent to use at least 100 agents to update its own docs. It didn't need 100. The harness held anyway.

Reddit r/AI_Agents Tools

Summary

An open-source coding agent harness was stress-tested by using at least 100 AI agents to update its own documentation, demonstrating efficient parallel task handling and identifying real issues in the docs.

We open-sourced our coding agent harness this week and wanted a stress test that also did real work. So we gave it one prompt: update the docs for the release, and use at least 100 agents to do it. Here's what the orchestrator (GPT-6 Astra) built from that: one planner split our 24 doc pages into 100 review jobs, four per page (claims vs the Rust source, what the release changed, whether examples and flags still match the code, links and limits), plus four site-wide checks 100 reviewers on GPT-6 Luna, all running at once, told to read and report, not edit 25 editors on GPT-6 Sol, one file each, so none of them could step on another two more Sol agents checking the edits against the code at the end Honestly, 24 pages don't need 100 agents. A dozen would've done it. We wanted to know whether the harness holds at that width, and it did: all 100 came back with a report, and the slowest took just over two minutes. The fan-out took 4m 55s, and the agents' time added up to 1h 57m. Prompt to commit was just under 10 minutes: 29 files, +276 / -90. The catches were real. The SDK error example in our docs didn't compile (it returned a String where the tool wanted a ToolError). A page claimed a subagent depth limit of 5 that the code doesn't have. Three supported providers were missing from the provider page. The final check found two pages disagreeing about an MCP timeout. It also documented that our TypeScript REPL tool is broken on recent bun, which is true and not fun to read in your own docs. What it cost We ran this one through OpenRouter to get a clean per-model breakdown: $2.94 for Astra, $2.08 for Sol, and $0.73 for all 100 Luna reviewers, about $5.75 in total. All three also run on a ChatGPT subscription, so if you already pay for one, the same run adds nothing to your bill. Paying per token instead, the cheap tier carries the load: 100 reviewers at under a cent each. Either way, the cost stays under control by design: Each reviewer gets one page, one angle, at most 6 rounds of looking and at most 4 findings. No dumping the repo into context, no compiling. Findings go to a file, and the reviewer replies to the orchestrator in under 100 words. After the whole fan-out Astra's context was at 75K of 1.05M, 7%. The model list is enforced, not suggested. You set which models each tier may use in /subagents, and a worker call naming anything else is refused before it starts, so the planner can't quietly reach for the big model. Workers are bounded by default (24 steps or 5 minutes), so a stuck one ends instead of spending. It all runs as tasks inside one 6.5 MB Rust binary rather than a hundred copies of a CLI. If you run fan-outs like this, where do you put the line between reviewer and editor? Or do your workers edit directly?
Original Article

Similar Articles