We benchmarked how 60 engineers direct AI coding agents. Here is what separated the strong ones

Full transparency up front: I am the founder of HyperHat, and this post comes out of what we are building. No link in here. I am posting because this community is where the honest conversations about agent workflows happen, and I want to compare notes. We built a short exercise where an engineer directs a coding agent through a real task, and we score how well they direct, review and verify the agent's work on a 0 to 1000 scale. About 60 engineers in so far. What stood out: The spread between engineers using identical tools is huge. The difference was not prompting tricks, it was specification quality and review behavior. The strongest engineers verify mechanism, not output. Top scorers check what the agent touched and how: which files, whether the tests changed. Output review alone never catches an agent weakening a test to go green. Cost works like a skill signal. A task that suddenly costs 3x its neighbors usually means it was badly cut, not that the work was hard. Ask for the mechanism before any code exists: one or two sentences on how it works, not what it will do. Almost nobody notices when an agent half-fails. Detection without an owner is just a log line. The skill that matters now is directing and verifying agents, and almost nothing in how teams work measures it. Token counts and prompt counts are activity metrics, like counting lines of code. Happy to answer questions about the scoring. If you've run something similar, I'd like to compare notes on how you measure review quality.