ai-metrics

This paper analyzes how different AI performance metrics (bounded vs unbounded) determine whether frontier AI capabilities remain concentrated among wealthy actors or diffuse to smaller models, with implications for regulation.

Alex Atallah highlights that cost per task is more meaningful than price per token, citing Terminal-Bench results where Haiku is 10x the cost of Opus.

The article argues that current metrics for coding agents (e.g., lines of code, speed) miss the more important measure of how much human attention is saved, since constant supervision negates time savings.