If you automated something and stopped checking it, did the errors stop, or did you just stop finding them?

Reddit r/AI_Agents News

Summary

The author reflects on conversations with people running AI automations, noting a pattern where verification is dropped after initial audits, which may hide silent failures. They ask for concrete stories about automations that were wrong without anyone noticing.

I've spent the last few weeks asking people who run AI automations what they won't let an agent do. One answer keeps coming back in a form I can't stop thinking about. Someone running automations for clients described their process like this: start with a manual audit of 100% of what the AI handles. Once you feel confident, drop to a 20% random audit. After a few weeks with no errors, only audit when something breaks. That's a completely reasonable process. It's also the process where, if a quiet failure started on week four, you would probably never know. The thing that struck me across every conversation is that the line people draw isn't risky vs. safe. It's verifiable vs. not. People happily automate high-stakes work when the result is checkable, and refuse low-stakes work when it isn't. One person put it as "anything of importance that cannot be easily verified." And almost nobody trusts the agent's own report of what it did. Everyone had independently built some version of the same workaround: log at the tool layer instead of the agent layer, compare the result against approved source data, keep everything read-only by default, record what was requested separately from what actually executed. So the questions I'm stuck on: If you've scaled back checking on an automation, did you ever go back and verify a sample? What did you find? Has an automation ever reported success while doing the wrong thing, and how long before anyone noticed? What would you need to see to trust a check more than you trust your own spot audit? For context: this started as a university research project and has pushed me toward building something in this area, so I'd rather be upfront about that. No link, nothing to sign up for; I'm trying to find out whether "silently wrong, discovered late" is a real recurring problem or something people have already solved well enough. Concrete stories are far more useful to me than agreement.
Original Article

Similar Articles