Five failure cases I’d test before trusting a multi-agent coding workflow

A happy-path demo tells me an agent can finish a task. It tells me less about what happens when the task changes, a worker disappears, or an external action has an uncertain outcome. I’ve been working on a framework for thinking about this while building Backplane with my cofounder. This is a design checklist, not a claim that our product has solved every case. Here’s a hypothetical example: one agent adds CSV export, another reviews it, and a human approves the result. Then someone changes the requirement: the export must exclude personal email addresses. The code hasn’t necessarily changed yet. The tests may still be green. The old review may still say “approved.” But those signals no longer establish that the work meets the current requirement. These are the five cases I’d test. 1. Change the requirement after review Keep the previous review as history, but don’t silently carry its acceptance forward. I’d want the completion record to identify both the code revision and the requirement version it was checked against. A commit alone doesn’t capture a requirement edited outside the repository. Expected result: the workflow identifies which checks or reviews need refreshing. It doesn’t simply keep the task green. 2. Let a worker lose its claim, then return Suppose agent A stops reporting progress. Its claim expires and agent B takes over. A then reconnects and submits its result. Expiry is useful for recovery, but it doesn’t physically stop the old worker. The system still needs a way to reject stale writes where it controls them. Work already sent to an external system needs separate handling. Expected result: A’s output can be inspected, but its old claim cannot silently override B’s current work. 3. Lose the response to an external write Imagine the workflow creates a release, but the response times out. “Failed to receive confirmation” and “failed to create the release” are different states. I’d use an idempotency mechanism where the API supports it, or reconcile against an identifiable operation before deciding what to do next. A single read that finds nothing may not settle it either, depending on the external system. If the outcome remains uncertain, surface that uncertainty instead of letting the model invent a success or blindly repeat the action. Expected result: uncertainty is preserved and routed to a recovery path. 4. Approve one action, then change its scope A human approves a preview deployment. The next step attempts production. An approval in the transcript isn’t enough. I’d want the action boundary to check the actual target and scope against what was approved. Expected result: the preview approval does not authorize production. This only protects actions passing through that boundary, so bypass paths matter too. 5. Restart without the original conversation Stop the run and give the next worker only the saved project record. Can it identify the current requirement, the latest artifact, completed checks, unresolved outcomes, and the next permitted action? Expected result: it resumes from a known checkpoint or asks a specific question. It doesn’t have to reconstruct authority from a long chat. None of these checks proves the implementation is correct. They make it easier to distinguish “a run ended” from “this result was accepted against the current task.” For people running these workflows: which of these failures have you actually had to recover from, and what evidence was missing when it happened?