I let an agent pick its own task every morning for 23 days. 41 runs, 19 reached production, 22 died. The 22 are the reason it works.

The "is loop engineering just cron jobs with extra steps" argument comes up here a lot. I've been running one for 23 days and I think the answer is basically yes, and that the cron part is the least interesting piece of it. Quick shape of the thing. A GitHub Actions workflow fires at 6:00 every morning. Before it does anything it reads search data, usage data, a log of every initiative it has attempted before with the outcome attached, what failed and why, what each run cost, and a rules file I maintain by hand. It generates candidate tasks from that, ranks them, and commits to exactly one for the day. Then plan, design, implement, write the copy, test, verify. The last step is where the whole thing actually lives. Nothing reaches production unless it clears 81 automated checks. If it trips one, the run dies. The dead run gets written to the same log that the next morning's ranking reads, so yesterday's failure is an input to today's choice. That path is the only place in the system where anything resembling learning happens. Accounting for Aug 11 to Sep 2: 41 runs, 19 reached production, 22 died before merging. They stalled, tripped a condition, or blew through the cost ceiling I set. So it failed 54% of the time. I'd argue that number is the feature. If I tuned for a high success rate I'd have to loosen the checks, and then I'd be back to reviewing every diff by hand, which is the exact job I was trying to get rid of. The question I care about isn't how often the agent gets it right. It's what happens on the runs where it doesn't, and 22 quiet deaths with a log entry each is a much better answer than 41 merges I have to audit. Of the 19 that shipped, 7 were changes to the loop's own machinery and 12 were content pages. Every word across all 19 was written by the agent. I edited the copy zero times. Two things I'd push back on in the usual autonomy conversation here. One, the autonomy isn't a property of the model. It's how much you'll let it merge without looking, and that's a number you set with checks, not with prompting. I can make this setup meaningfully more or less autonomous without touching the agent at all. Two, and this is the part I have not solved. The ranking step is the only step in the loop with no test that can fail it. Everything downstream of "which task today" gets verified. The choice itself just happens, and a bad choice that clears all 81 checks ships exactly like a good one. I've been using the initiative log as a weak proxy for this, but it's a lagging signal and I know it. So the actual question: if you're running something that picks its own work, how do you evaluate the decision step? Not the execution, the choice. I haven't seen a good answer to this and I'd rather steal one than invent it.