An empirical study crashing four production AI agent codebases (Cua, Skyvern, Corsair, Inconvo) mid-action and measuring duplicate side effects on retry, finding a shared root cause: none of them records which actions already completed, so retries re-execute work from an empty history.
I kept hitting the same thing building agents, so I went and measured it against other people's code rather than arguing about it. Four published agent codebases, each pinned at a commit, each killed partway through an action, then retried the way a supervisor or worker pool would. Nothing reimplemented. Where I had to substitute something, I used the project's own documented extension points. Counts come from a ledger the fake service fsyncs before it responds, so a process that dies the instant after acting still counts as having acted. The agent is never asked what it did. What happened: Cua (computer-use): Kill it mid-checkout and the retry charges the card again. Both crash positions, every run. Measured on cua-agent 0.8.4 and again on 0.9.0, identical. Skyvern: Any process death during a step strands the task in running forever. Not failed, not errored, so nothing alerting on failure notices. The only way out is a new run, which orders again. Corsair: Two replicas spend the same rotating OAuth refresh token. A crash leaves the stored credential broken, every run, so the user has to reconnect the integration. Inconvo: An interrupted answer costs 60% more in model calls, because the checkpointer records the graph node but not the fifteen model calls underneath it. Three of those are crash failures. Corsair is the other kind: nothing crashed, two replicas simply both reached for the same token at the same time. Different problem, same missing piece. Four different failures, one shared cause: none of them records which actions already completed. The retry is a fresh process with an empty history, so it does the work again. The most useful thing I read while doing this was arXiv 2609.29095, which graded 25,930 agent episodes against a ledger of committed effects. Agents reported success in 90% of the episodes where they had duplicated an effect. So you can't ask the agent, and anything graded on self-reported traces scores "duplicated and lied about it" as a pass. One thing I won't claim to fix: if a process dies between the action landing and the record of it committing, it still repeats. The effect and the record live in different systems, and nothing closes that. It's also the row any benchmark reporting zero duplicates has quietly excluded by only crashing at already-committed boundaries. The harnesses and raw ledgers are public and need no API key, since every call goes to a local fake. Happy to point anyone at them in the comments. Disclosure: I build a runtime for this problem, so I'm not a neutral party. The measured arm in each case is their code at a pinned commit. Two things I'd genuinely like to know. If you run more than one agent against shared state, what actually stops two of them taking the same action? Every answer I've seen in the wild is a lock, a dedupe table, or nothing. I'd like to know whether there's a fourth. And if you're running agents that touch money or send things, how do you handle the retry today? Keys you pass in yourself, or do you just accept the occasional double?
A discussion query asking developers how they handle recovery when AI agents crash mid-task in production, exploring approaches like restarting, persisting state, using checkpoints, or manual inspection.
The article discusses how AI agents often fail silently by completing tasks incorrectly without crashing, leading to undetected errors. It highlights common failure modes and explores potential detection strategies.
An AI agent (COMMS) repeatedly crashes at the shutdown step, revealing a failure mode specific to on-demand agents where the audit trail fails after work succeeds. The fix involves adjusting spawn timeout at shutdown, highlighting the need for separate lifecycle checkpoints.
AI agents can fail silently without traditional errors, as illustrated by a public postmortem where a pipeline ran into loops and high costs without triggering alarms. The article suggests using tracing and per-agent spend monitoring to detect such issues.
The author shares lessons from putting an AI agent with write access into production, explaining that retry logic inside the agent loop fails when the process dies. They advocate for treating side-effectful tool calls as durable background jobs with idempotency keys.