Agent failure clusters changed how I think about debugging

I genuinely thought every agent failure was its own isolated thing. Different run, different problem, fix it, move on. That was my mental model for months. Then a coworker pulled up a visualization of failures across like 200+ runs of one of our agents and I just... saw it. I didn't know agent failure clusters were a thing until someone showed me one. now I can't unsee them. The failures weren't scattered randomly across runs. They were grouping. Same point in the workflow. Same type of context conditions. Same category of task going sideways. It was like one of those magic eye pictures where you suddenly see the 3D shape and then you can never not see it again. The thing that got me though, and this is where i think most people stop too early is that seeing the clusters is only half of it. Maybe less than half honestly. My first instinct was "cool, now we tweak the prompt at that step" or "maybe restructure the tool call sequence there." And yeah that helps for that specific failure mode. But the agent has no memory of any of this. Next run it starts completely fresh. It doesn't know it failed 47 times at step 3 when the input had certain characteristics. It doesn't know you already figured out what went wrong. It will cluster again in the exact same place because nothing actually learned from what happened. You learned. The agent didn't. This is the part that's been bugging me. The real unlock isn't pattern detection, it's closing the loop. Taking what you found in those clusters and feeding it back so the agent genuinely improves across runs. Not just "here's your pattern" but "here's your pattern and we did something about it so it doesn't repeat." We've been experimenting with a few approaches: \- manually writing failure-specific instructions and injecting them as context (works but doesn't scale at all) \- logging failure conditions and building lookup tables the agent can reference (janky but kinda works?) \- using a platform called BentoLabs that's specifically built around this closed-loop idea, it traces runs, detects regressions, and actually promotes fixes back into the system as reusable artifacts. but the concept of a "living memory" of failure patterns is exactly what i was trying to duct tape together manually The manual approaches work for like 5-10 failure patterns. After that you're basically maintaining a second codebase of edge case handlers and it gets brittle fast. What I keep coming back to is that we treat agent failures like software bugs, find it, fix it, ship it. But agent failures aren't really bugs in the traditional sense. They're behavioral patterns that emerge from the interaction between the model, the tools, the context, and the input distribution. You can't just patch them the same way. Anyone else working on this problem? How are you handling the "agent has no memory of its own failures" thing? Curious if people have found approaches that actually scale beyond a handful of manually curated fixes.