I built an open-source multi-agent SDLC harness that beats a cold Claude Code run on large repos, by learning the repo once. Real benchmarks (incl. where it loses) inside. [P]

Reddit r/MachineLearning Tools

Summary

AutoDev Studio is an open-source multi-agent AI coding harness that reduces costs by building a persistent repository knowledge base, outperforming cold-start Claude Code runs on large repos in most tasks.

Built an open-source AI coding agent that was 7%–75% cheaper than a cold "claude -p" run on 6/6 well-localized tasks across repositories up to ~82k LOC. The biggest difference: Cold agent: $6.83, 207 turns AutoDev Studio: ~$1.70 for the same bug The full benchmark (including cases where it loses) is in the README. So what's different? Most AI coding agents re-explore a repository from scratch on every task just to figure out where the change belongs. AutoDev Studio pays that localization cost once. It ingests a repository and builds a persistent knowledge base using static analysis and a local embedding index. Every future task reuses that knowledge, turning localization into a lookup instead of another cold search. What it does: PM agent asks clarifying questions and drafts tickets Dev agent writes code on an isolated branch QA runs tests A different model family reviews the diff (author ≠ reviewer) If needed, it goes through a bounded revise loop Opens a real GitHub PR It also includes a live Kanban board and tracks token usage and cost per ticket/agent. Where it doesn't win: Tiny, easy-to-find edits can be cheaper with a single-shot agent because of the pipeline overhead. On one complex cross-cutting bug, it produced a cheaper but narrower fix than the baseline. Other features: Provider agnostic (Anthropic, Claude Code, OpenAI-compatible APIs, Groq, Gemini, xAI, OpenRouter, Ollama, etc.) Runs completely free/offline by default using Groq's free tier + local embeddings FastAPI + SQLite Hand-rolled UI Tests + CI MIT licensed Repo (screenshots + full benchmark): https://github.com/krishagarwal314/autodev-studio I'd love any feedback, criticism, or contributions. Happy to answer questions about the architecture or benchmarking.
Original Article

Similar Articles

Claude Code improved my agent harness by 40% overnight

Reddit r/AI_Agents

The author introduces 'Autoharness', a tool that uses Claude Code to autonomously optimize agent harnesses by iterating on prompts and hyperparameters. This resulted in a 40% performance increase on the tau2-airline benchmark.