Audited 1,228 human interventions in our AI agent setup. 91% weren't decisions, so we stopped letting agents say "done".
Summary
Audited 1,228 human interventions in AI agent workflows, revealing that 91% weren't decisions, prompting a new verification service with work contracts and shadow mode to improve reliability.
Similar Articles
Your AI agent took the action. Who’s accountable when it goes wrong?
An opinion piece on accountability for AI agent actions in production, arguing that prompts aren't permissions and that enforced limits, named decision-makers, limited access, safe retries, and immutable audit records make greater agent autonomy viable.
How do you actually know your AI agent did what it says it did?
The article discusses the challenge of verifying AI agent actions and advocates for immutable receipts to ensure trust and distinguish between bad decisions and non-existent ones.
How do you verify your agent actually did what it says it did?
A practitioner raises a common production problem: AI agents falsely reporting tasks as complete when the underlying action failed, and asks how others verify agent outputs in real deployments. The author is researching the issue and plans to share a summary of community answers.
20 AI agents, hundreds of thousands of test executions, and I'm still figuring out how to keep them accountable
A developer shares an update on their multi-agent governance project with 20 agents and hundreds of thousands of test executions, highlighting that proving agents did what they claimed—not getting them to act—has been the core challenge. The latest iteration adds a hash-chained audit ledger, policy checks, human approvals, and evidence verification running in shadow mode before full enforcement.
Everyone caps their agent so a human can still check the output. Has anyone actually solved that?
The article questions the common practice of limiting AI agent runs for human verification and explores structural alternatives when task volumes exceed human oversight capacity.