Build AI Agent for Company

ok so we spent way too long building an agentic system at work and i wanna share what actually worked, because we tried basically every wrong thing first. the setup sounds simple on paper. user asks the agent something, agent needs to actually go handle the task. easy right. it is not. first thing we tried was one giant system prompt with literally everything the company knows shoved into it. every policy, every product, every edge case. it kinda holds for a week and then it just falls apart. the model gets confused, mixes stuff up, answers a billing question with returns info, and the bigger the prompt gets the dumber the thing feels. plus you cant debug anything cause its all one blob. so we went the other way. one agent but we handed it a huge pile of tools. like 400 of them. turns out the model just cant pick right when it has that many options. it grabs the wrong tool, chains weird stuff together, and every new tool you add makes the picking worse instead of better. zero scaling. next idea was one main agent that spins up a bunch of sub agents on the fly. felt clever. in practice its chaos. no clear ownership, the sub agents step on each other, context leaks all over the place and you have no clue which one actually answered you. what finally clicked was an orchestrator with a fixed set of child agents, kinda like how you structure a project. one router on top whose whole job is figuring out which domain the request belongs to. then each agent under it has its own system prompt and its own small set of tools that it actually specializes in. billing agent only knows billing. orders agent only touches orders. nothing bleeds between them. the routing gets way more accurate this way, because the orchestrator isnt choosing a tool, its just choosing a domain, which is a much easier call to make. and every agent stays sharp because its context is tiny and focused. want a new capability, you just add another child. you dont rewrite anything. anyway thats it. curious if anyone landed on a different structure that actually held up under load.