I measured what 13 search APIs actually cost to run inside an agent. The pricing page is the smaller half of the bill

Reddit r/AI_Agents News

Summary

A developer benchmarks 13 search API configurations inside an AI agent, revealing that hidden token costs from reading payloads can dominate the total bill and vary by up to 67x across providers.

Disclosure: I build one of the 13 providers below. All payloads, grades and prices are open so you can re-run the whole thing yourself. Every search API prints one price, the one per request. Then it hands your agent a payload and your model bills you again to read it. That second cost never appears on a pricing page and for most providers it is the bigger one. So I ran 13 priced configurations over the same 100 questions. One search call each, same reader model, payloads passed verbatim minus each vendor's own synthesis. Token counts are the real invoice from the answering call, not an estimate. Prices are list pay as you go for everyone, mine included. Provider Tokens/query Total $/1k Correct serpdive-krill 971 $2.91 78% serpdive-mako 1,891 $12.67 88% parallel-turbo 4,155 $13.46 79% tavily-basic 3,166 $17.50 88% parallel-advanced 6,146 $23.44 87% you-web 9,026 $32.08 95% parallel-basic 13,015 $44.05 83% linkup-standard 16,150 $53.45 85% tavily-advanced 12,749 $54.25 91% firecrawl-search 16,458 $55.77 94% serpdive-moby 23,331 $80.49 92% brave-search 47,996 $148.99 93% exa-search 65,543 $196.63 93% Total is the request plus the tokens, read on Sonnet 5 at $3/M. The spread on the token column is 67x between the top and the bottom. Two things worth knowing before you read anything into it. The answer key could not be a fixed file. These are questions whose answers move, and the public dataset I drew from was last republished in April. By August, 13 of the 100 answers had changed under it: every engine returned the current truth and the benchmark scored them as failures. Graded on that stale key, every provider loses 8 to 18 points, and the freshest ones lose most. So the key is rebuilt on the day of the run by the field itself, one vote per provider. And read the last column with error bars: at n=100 it carries about plus or minus 10 points, so the top of it is not separated by this run. The cost column carries no such noise. Repo with every payload, grade and price in the comments.
Original Article

Similar Articles