how to fix ai agent reliability?

thinking a lot about the gap between an agent that works in a sandbox and one that actually holds up in production. we built a workflow tool, the base model had high sensitivity, which sounds good until you realize it was flagging 4 things per and 3 of them were noise. at that point you don't have a productivity tool, you have something people route around. the fix was adding network that filters alerts before they ever surface to the user. so, what others are doing in those cases - secondary llm evaluators? hard-coded heuristic filters? a cascading architecture? and how much of your dev time ends up on the filtering layer vs. the core task?