@Suhail: Cost per task is going to be a meaningful metric these next 12 months.
Summary
Alex Atallah highlights that cost per task is more meaningful than price per token, citing Terminal-Bench results where Haiku is 10x the cost of Opus.
View Cached Full Text
Cached at: 07/01/26, 10:13 PM
Cost per task is going to be a meaningful metric these next 12 months.
Alex Atallah (@alexatallah): On price/token != cost/task:
To help internalize this fact for yourself, run Terminal-Bench using Haiku and then using Opus.
Here are the results for a 15-task subset. Haiku is 10x the cost!
Similar Articles
Cost illusion in Task vs Token between Opus 4.7 and K2.6 💭
Comparison of cost per token vs cost per task between Kimi K2.6 and Claude Opus 4.7, showing that despite being cheaper per token, Kimi burns more tokens so the savings per task are less significant.
@rohanpaul_ai: Claude Sonnet 5 is more expensive (around +15%) per task than Opus 4.8 and much more expensive (2X) than Sonnet 4.6, ev…
Claude Sonnet 5 costs more per task than previous models due to higher token usage despite lower per-token price, with discounted pricing until August 2026.
under 2% quality gap but 10x cost difference: tested 5 models on identical tool calling tasks[D]
A developer tested five AI models on tool calling tasks and found that cheaper models perform within 2% of expensive models like Opus, with Tencent's Hunyuan under $1.50 vs Opus's $15, leading to a daily cost reduction from $40 to $9 by routing simpler tasks to cheaper models.
Sonnet 5 - its updated tokenizer maps the same text to more tokens (roughly 1.0–1.35× depending on content), so cost per task can be higher.
Anthropic released Claude Sonnet 5 with improved reasoning, tool use, and coding, but its updated tokenizer maps text to more tokens (up to 1.35×), increasing effective cost per task despite the same listed price; introductory pricing applies until August 31, 2026.
@cline: While DeepSeek V4-Flash is significantly cheaper on price per token, this can be misleading if the overall cost per tas…
Discusses DeepSeek V4-Flash's price per token vs. overall cost per task, citing a report that DeepSeek completes benchmark tasks at 105x lower cost than Fable.