A month of running agents on a cron in production: four things that broke, none of them the model's fault

Reddit r/AI_Agents News

Summary

The author shares four mechanical failures experienced while running AI agents on a cron job in production for a month, emphasizing that all issues stemmed from setup problems rather than model deficiencies.

I run a small system where agents post on a schedule against shared state, with no human approving individual actions. It has been live for about a month. Every failure that cost me real time turned out to be mechanical, and each one looked like the model being bad at its job. Writing them down because I would have paid for this list a month ago. 1. The context window decided the behaviour, and I blamed the model. Agents were told to reply to a specific opening statement, and the contract said the target id had to come from the feed they were given. Hit rate sat at 33%. The cause: replies are more recent than openings, the feed was a flat count of the twelve newest posts, so by the second phase the openings had been pushed out of it. The instruction pointed at something that was no longer in the window. Hit rate went to 80-100% once openings were injected outside that budget. Someone in this sub named the fix better than I had: a pinned channel sitting next to a recency channel. 2. Silent compliance is worse than refusal. Nothing errored in the case above. The contract said use an id from the feed, the intended target was gone, so the model picked a different valid id and carried on. The output was well-formed and plausible. There is no exception to catch and nothing in the result looks wrong. You only find it by comparing what the agent did against what you meant, or by rendering the exact input it received. 3. Tool descriptions are a contract that gets read on every call. One of my read tools claimed results came oldest first. The API returned newest first. Every test was green, because tests call the endpoint and check the sort, and none of them read the description. No human ever saw that lie. Only agents did, and agents do not file bug reports, they build on it. 4. The expensive failure mode is a well-behaved agent. I braced for runaway loops. What actually threatens the budget is a perfectly obedient agent on a schedule doing work nobody needed. Per-run caps, a kill switch that defaults to on rather than off, and separate daily budgets per capability class did more for me than any loop detection. A round that finds nothing to do now ends at zero cost structurally, rather than because it chose well. The general lesson, if there is one: when an agent behaves badly, check the mechanics before you touch the prompt. Mine were obedient every single time. The prompt was fine and the plumbing was lying to it. Happy to answer anything about the scheduling, the cost caps or the server side. Link in the comments, per rule 3.
Original Article

Similar Articles

I analyzed how 50+ AI teams debug production agent failures and got surprised

Reddit r/AI_Agents

Based on interviews with 50+ AI teams, the author highlights that production agent failures often stem from minor prompt or configuration issues rather than deep model problems. The article advocates for adopting software engineering practices like versioning, A/B testing, and experiment tracking to improve reliability.

The agent failures that cost me the most all reported success

Reddit r/AI_Agents

The author analyzed 155 AI agent jobs and discovered that most failures stemmed from infrastructure issues like timeouts and false success signals, not model errors, leading to practices such as asserting on effects and using multiple verification paths.