Solution to scaling tools without route failures: Create Playbooks

Reddit r/AI_Agents Tools

Summary

An author describes how they solved scaling issues in a 200+ tool AI system by implementing predefined playbooks for tool sequences, improving reliability by avoiding model-improvised routes in multi-step executions.

I've spent two years building a creative operating system for agencies and brands. It's now past 200 tools and it runs production work for paying users every day across over 500 APIs. What made it reliable wasn't a router model, a better system prompt, or cutting the tool count. It was accepting that the model should never compose its own route through the tools at all. Watch an agent fail at scale and it's almost never one tool call that's wrong. It's the sequence. The model picks a plausible tool, gets a plausible result, picks the next plausible tool, and four plausible steps later it's somewhere confidently wrong. Selection uncertainty compounds. The tools were fine. The route was improvised. So we stopped asking for routes and started asking for outcomes. Every repeatable job became a written playbook: the steps, the order, the tools each step uses, what done looks like. About 150 of them now, each one running under the same one-line contract: execute the body as the user's instructions, faithfully and in order. The playbooks live in a library that's organized the way the work is, not the way the tools are. Three top level swim lanes: creative work, intelligence work, account work. So the agent's first decision isn't which of 200 tools, it's which lane does this job live in. Then which playbook in that lane. Then the parameters. Every decision is small, and the funnel narrows as it goes. Once a playbook is running, every step names its tools, so there's nothing left to improvise with. The lane does something else too, it concentrates context. Everything in front of the agent at that point is about producing one class of outcome, not the whole platform. Think drive-through menu. Nobody orders ingredients, you order the combo, and the kitchen asks which drink. Two honest limits. A playbook can't fix a tool that reports success when it silently did nothing, and that class of bug got worse for us, not better, because the route always looks like it worked. And discovery is a real second job. A written index the model might consult gets skipped. The index has to be in the path, not beside it. We set up our system so both the agent and our customers can drive playbooks. The customer just says what they want (i.e. Run playbook 12, or Run the playbook: Create a new brand kit). And before someone says just use fewer tools: fewer tools means more improvising per step with less precision. That's the disease, not the cure. The tool count was never the problem. The improvisation was. For context, I spent 40 years on the creative side of advertising and the last two building the platform this runs, so read my bias accordingly. Question for others that might be running into route or context failures during mult-step tool calls: what type of solutions have you used to solve larger sets of MCP tools and multi-step execution on your platform?
Original Article

Similar Articles

From Model Scaling to System Scaling: Scaling the Harness in Agentic AI

Hugging Face Daily Papers

This paper argues that advancing agentic AI requires scaling the system architecture around foundation models, focusing on auditable, modular, and verifiable components. The authors introduce CheetahClaws, a reference harness, and outline bottlenecks in context governance, trustworthy memory, and dynamic skill routing.

Engineering Agent Skills at Scale

Reddit r/AI_Agents

The article discusses strategies for engineering AI agent skills at scale, including context minimization, lazy-loading, executable operations, and outcome-based measurement.