I’m starting to think we’re framing AI agent reliability too much as an observability problem.

I came across a discussion recently about someone building a voice AI agent that takes orders and writes to a production database. Their concern was: how do you know that a conversation actually qualifies as a lead before allowing the agent to create something in production? Someone suggested a staging/queue layer with deterministic validation before the write. Then another question came up: what happens if the agent retries and sends the exact same write twice? That rabbit hole got interesting pretty quickly. Because now we're not really talking about observability anymore. We're talking about whether we can trust an agent to perform actions that have side effects. An agent can have perfect logs. You can know exactly what tool it called, what arguments it passed, and whether the API returned a 200. And the system can still be wrong. The database might not contain what the agent intended. A lead might have been created twice. A state transition might have happened when it shouldn't have. An external action might have succeeded even though the agent thinks it failed and retries it. So maybe the reliability layer for agents shouldn't just answer “what did the agent do?” It should also answer “did the action produce the state we actually expected?” This is actually the problem I've been exploring with a small project I'm building. For people running agents in production: how are you making sure agent actions with real side effects are actually reliable, rather than just observable?