Retrying a failed agent step is not the same as safely resuming a 14-step run
Summary
The article discusses the difference between retrying a failed agent step and safely resuming a multi-step run, emphasizing that they are not equivalent operations.
Similar Articles
Your agent's retry logic dies when the agent does
Discusses the problem that when an AI agent fails, its retry logic also fails, highlighting a design flaw in agent error handling.
Your agent’s action timed out. Does your code retry it?
Technical article discussing the importance of retry logic when agent actions time out, highlighting a common pitfall in agent-based systems.
Why Retrying Fails: Context Contamination in LLM Agent Pipelines
This paper introduces the Context-Contaminated Restart Model (CCRM) to formally analyze how failed attempts in LLM agent pipelines contaminate context and increase error rates during retries. It provides theoretical proofs and validates the model against SWE-bench data, showing significant discrepancies with standard independent models.
My pipeline ran "successfully" for a week. Turned out my agent had been silently skipping failed API calls the whole time.
A developer recounts how their automated pipeline silently skipped failed API calls due to rate limiting, producing seemingly successful runs with empty data. They discuss the trade-off between retrying and hard-failing, and ask the community for best practices in agent error handling.
Production agent evals should test incident replay not just task success
Discusses that production agent evaluations should include failure replay and resume capabilities, not just happy-path task success, emphasizing the need for observability that enables recovery.