I planted six errors into a month of fake bookkeeping to see if my AI agents would catch them. They caught five. The one they missed is the scary part.
An experiment tested AI agents on fake bookkeeping data with six planted errors; agents caught five, notably refusing to guess when data was insufficient, highlighting the value of uncertainty-awareness over pure accuracy.
Ran an experiment on my own AI agent setup last month that I wasn't planning to post about, but the result stuck with me. I built a fake month of bookkeeping for a business that doesn't exist. Real-looking invoices, vendor payments, a churn curve, the whole thing. Then I quietly planted six specific errors into the data before handing it to the agents: a duplicate invoice, a vendor rate that had drifted without anyone updating it, a threshold that needed sign-off and didn't get one, a customer segment with margin that looked fine on the surface and was actually losing money, a chunk of revenue leakage that only showed up in the churn month, and a vendor payment that was never approved. I wrote an answer key first so I'd know if the agents were actually catching things or just sounding confident. They caught five of six unprompted. The one they missed wasn't a math error, it was a sign-off rule the grading logic itself didn't account for, and two of the agents flagged that gap on their own before I found it. The part that actually built trust wasn't the catches. It was what they refused to do. One agent wouldn't tell me if the business was solvent because it didn't have a number it needed. Another wouldn't estimate customer acquisition cost without real spend data instead of guessing at something reasonable-sounding. A third wouldn't compute a ratio itself when the input was ambiguous, it kicked it back and asked which definition I meant. I also found two actual errors in my own answer key partway through. One was a typo in my math. The other was a rule I'd written that didn't match how the business actually worked. Neither was caught by me, both got flagged because the agents' output didn't match my key and that mismatch forced me to go check who was actually wrong. Whole exercise cost about five and a half dollars and took under half an hour. Everyone talks about agents that get things right. Nobody talks much about agents that know when to say I don't have enough to answer that. That second thing has been more useful to me than the first. Anyone else deliberately breaking their own agent setups to find out where they lie versus where they just say no?
Independent research benchmarked AI agents on fixing 20 real vulnerabilities from Python projects; best solve rate was 50%, expensive models not worth it, and dangerous false positives where agents produced convincing but incomplete fixes.
An engineer recounts discovering that AI agents confidently report completing tasks that never actually occurred, leading to a redesign of verification architecture where the model's claims are treated as hypotheses and external systems provide truth.
The author observes that AI agents exhibit human-like failure patterns, such as overconfidence and skipping steps under context pressure, suggesting that system reliability depends more on robust validation and controlled environments than just model intelligence.
A tweet asks about the most money people have seen an AI agent waste while trying to fix its own mistake, highlighting the financial risks of autonomous agents.
The author describes building a 'gate' that logs tool calls and verifies that claims in an AI agent's answer correspond to actual log entries, forcing the agent to show its work and reducing hallucinated responses.