All our scheduled agent jobs died for 19 hours on a usage limit while the plain scripts kept running

Reddit r/AI_Agents News

Summary

The article describes an incident where scheduled AI agent jobs failed for 19 hours due to a usage limit, while plain scripts continued running, and outlines solutions like implementing pre-checks and custom error handling to prevent similar issues.

We run a handful of scheduled agent jobs off one account. In mid-September the account hit its weekly usage limit, and every job that boots a model died on start for about 19 hours. The plain-Python jobs on the same schedule kept running as if nothing had happened, which is what made it easy to miss: the agent that would have reported the failure never got to run. What we changed afterwards: Every scheduled job starts with a plain-code gate that decides whether the model is needed at all. If there's nothing to do, it never boots, so quiet days cost no quota. The gate matches the exact limit message in the run log and labels it as quota, not as a bug. Generic retry logic would have retried straight into the same wall. An expired login is a different failure with a different message and a different fix (a person has to sign in again), so it gets its own check instead of sharing a handler. Routine jobs default to the cheaper model, so the weekly allowance lasts longer. The general rule we took from it: whatever tells you an agent failed can't be the agent. What are you using as the outside-the-agent heartbeat? And has anyone found a clean way to reserve quota for the jobs that matter most?
Original Article

Similar Articles

99.9% uptime is a nightmare when your agent gets stuck in a loop

Reddit r/AI_Agents

A developer recounts an incident where an AI agent got stuck in a runaway loop, burning thousands of dollars on LLM API calls overnight despite high uptime, and asks the community how they build kill switches to stop runaway agent loops.