What an AI agent looks like after months in production on 60k real leads

We've had a text agent and a voice agent working a 60k lead real estate database for a client in Canada for a while now. Their job is figuring out where each old lead is at, and everything lands in a dashboard the realtor works from. What's running today looks a lot different from what we shipped, so here's what changed. The prompt got shorter and the code around it got longer. Version one tried to handle everything in the prompt, like opt-outs and what to do with a wrong number. Those rules kept breaking in weird ways. Now anything with a right answer lives in plain code, and the model only handles the parts that need judgment, which is reading replies and writing the next message. Opt-outs especially, since a "stop" should never depend on a model reading it right. The intent labels collapsed too. We started with way too many categories because people say so many different things, like "after my lease ends" or "talk to my wife." Then we noticed the realtor only ever does four things with a lead. Call now or follow up on a date, keep them warm or drop them. So now there are four labels, one per action, and picking the right one is the model's whole job on intent. If a label doesn't map to something a human does next, it's just trivia. The weird replies are most of the work. Replies show up weeks late, sometimes to a message from two follow-ups ago, so every reply gets read against the whole thread. "Who is this?" comes up constantly lol. And on a list this old, sometimes a family member replies for someone who moved away or passed away. You want to have decided what the agent says to that long before it happens. Voice taught us to shut up. The first version talked in paragraphs and people kept talking over it. Now every turn is a sentence or two and it stops the moment someone starts speaking. A big chunk of calls also hit voicemail, so what it says there ended up mattering more than we expected. Voice is also where the money goes. Text costs almost nothing per conversation and voice minutes add up fast, so text goes first and voice only calls the people who never replied. The dashboard shows volume and how many leads land in each label. It won't tell you the agent has started sounding pushy or keeps misreading one kind of reply. The only thing that catches that is a person reading conversations, so one of us reads a random batch every week. Every conversation that went weird gets saved into a test set. Before we touch a prompt or switch models we replay all of them and read what changed. We pin model versions too, since hearing about a provider update from the client is a bad way to learn about it. That test set is probably the most useful thing we built, and almost all of it came from production. The realtor still closes every lead. The agents figure out who's ready and what they want, and he takes it from there. I don't see that changing and I don't think it should. If I had to sum it up, months in production made the agent smaller and dumber and made everything around it bigger. It works way better for it. TLDR: after months on 60k real leads, the prompt is shorter and the code around it is bigger. Intent labels map to what the realtor actually does next, and someone reads real conversations every week because the dashboard can't tell you when the agent starts going wrong.