We put an AI agent in front of a bank's data warehouse. The part that mattered was not the model.

Disclosure: we run a few AI ventures doing AI engineering for companies. This is client work, names left out. A retail bank in Southeast Asia asked us to let their analysts ask questions in plain English and get answers from the warehouse. Text to SQL, basically. Ten tables, 858 columns, the usual mess of naming conventions. The first version was the obvious one. Schema in the prompt, model writes SQL, run it, show the table. It worked in demos and wobbled in week two. Joins that looked right and were not. Filters on the wrong date column. Confident answers to questions the data could not answer. What got it into production, in order of impact: Three candidate queries per question, generated in parallel, then a separate verifier call that reads the question, the schema and all three candidates and picks one or rejects all. Rejecting is a feature. "I cannot answer this from the available tables" beat a wrong number every time in user testing. A hard gate before any model change. Their governance rule is that nobody swaps the model, not even for a cheaper one, without an eval run ID and a named approver attached. We built a 60 question eval set with the analysts and set the bar at 85 percent correct with a median under 9 seconds. The current model cleared it. Two cheaper ones did not, and we would have shipped one of them on vibes without the gate. Prompt caching on the schema prefix. The static part of the prompt is 9 to 11k tokens. Caching it took per question cost down more than anything we did to the prompts. Now they want the whole thing on a self hosted model inside their own VPC for data residency. We priced it: managed API is about a thousand dollars a month at their volume, the GPU boxes to match come out between 3 and 9k a month plus a couple of engineer weeks for an OpenAI compatible adapter. Still a fair call for a bank. But "self hosting is cheaper" did not survive a spreadsheet. The honest summary: the model was never the hard part. The verifier, the eval set and the rule that says you cannot change anything without proof were. Happy to go deeper on any of it in comments.