I built a local control system for agent failures, fixes, evals, and gates to make autoresearch-style self-improvement loops work in real agent codebases

Reddit r/AI_Agents Tools

Summary

A local control system is built to manage agent improvement loops, capturing traces, finding recurring failures, drafting fixes with Codex/Claude Code, and applying changes only after passing checks and evals.

I've been experimenting with autoresearch-style loops for agents: run the agent, inspect the trace, find what went wrong, propose a fix, repeat. The idea is good. Coding agents are already capable of looking at traces, finding failure patterns, and proposing fixes. But the more I tried to use this on real agent codebases, the more challenges I faced. The real difficulty is having a system around the loop: * which failures are actually repeating across runs * what evidence supports each issue * which fixes were proposed * what checks and evals ran * what passed, what failed, and what regressed * what is allowed to change automatically * what needs human review So I built a fully local control system around agent improvement loops. It captures agent runs, finds recurring failures, turns them into evidence-backed issues, lets Codex / Claude Code draft fixes, and applies changes only through a gate after checks and evals. The workflow is: 1. Capture agent runs / traces 2. Find failures that repeat across runs 3. Turn them into reviewable issues with evidence 4. Let Codex, Claude Code draft a fix 5. Rerun the failing trace, run deterministic checks, compare eval results 6. Apply the fix only if it passes the gate, otherwise park it for review Everything is local by default: SQLite database, dashboard, traces, issues, proposals, evals. For the analysis and fix-drafting step, it can use the coding-agent CLI you already have, so there is no separate hosted service or extra model API key. Self-improvement loops are cool in theory but very difficult to transfer to production environments unless you build the infrastructure around them.
Original Article

Similar Articles

Building Self-Repairing Agent Loops (39 minute read)

TLDR AI

This article presents a method for building self-repairing agent loops using OpenAI's Codex, where agents review, repair, and validate outputs iteratively, with a worked example of fixing stale API documentation.