It's impossible to test your own agent. I tried and failed.

After spending almost a year building a personal assistant agent and I still couldn't definitively tell you if it was any good. It runs real work for me every day, and if you'd asked me if this thing's actually any good, I could only have answered you with vague processes descriptions. Asking standard desktop models how they'd rate my agent by uploading all my workspace files didn't really help. Looking for existing benchmarking tools I found nothing. My agent remembered all the wins and quietly forgot the misses, while I just got frustrated. It got worse when the thing seemed to stay the same week to week, even after a shit ton of "improvements". Model swaps and broken promises. Memory failures. Tools that changed without my permission. So even if it was good last month, that told me honestly nothing about today. So I did the obvious thing and started to write our own tests. And it was really good at passing those tests, obviously! What actually helped in the end wasn't a certificate or final score. It was more like a heartbeat. Something recent, fresh, current. Something the agent didn't see coming. That's the underrated part, an agent that knows it's being tested can hold it together for the window. One that doesn't has to actually be good. The day it really landed for me was when my year-old agent got beaten on a whole category by an agent that was a day old. Honestly I was not happy, there was lots of cursing. But that was kind of the whole point (not the cursing). The pointers for improvement were then clear. Anyway, I'm curious how everyone else deals with this. How do you actually know your agent is good and getting better? Do you re-test it with your own or someone else tests? Or just trust it until something fails?