The agent failures that get you aren't crashes. They're clean runs that did the wrong thing.

A crash is honestly the good outcome. It's loud, there's a stack trace, and it stops before it can do more damage. What I keep seeing people get burned by is the opposite. Run completes, every tool call returns 200, summary says done, and the thing it did was just wrong. Nobody finds out for a week. Most common version seems to be wrong-target success. Right operation, wrong row or repo or customer or environment. The tool did exactly what it was told to do. Close second is agents reading an empty search result as "this doesn't exist" and moving on confidently. Silence gets treated as data. I don't think most loops even have a separate branch for that case. Then partial completion getting reported as full. 40 of 200 items processed, summary says finished, because from inside the loop it did finish its loop. And self-grading, where the same agent does the work and decides whether the work was good. Everyone knows it's bad. Plenty of prod setups still do it because the alternative costs money. The annoying part is your dashboards catch none of this. Traces fine, latency fine, error rate zero. The run is only wrong relative to intent, and intent isn't in the trace anywhere. Things I've seen suggested, none of which I'm fully sold on: verify with a separate call that has no memory of how the work was done, grade the output not the process have the agent write down its expected outcome before acting, diff it after treat empty/null results as their own branch that escalates, instead of just a value cap irreversible actions per run, after N it has to stop and ask All of them roughly double your cost, which is why they're first to get cut. Two things I'm curious about. What's the silent failure that actually got you? Not an outage, the one that looked fine for a while. And has anyone found detection cheap enough to just leave running in prod? "Run a second model to check the first model" feels like it should have a better answer by now.