I benchmarked my browser agent against Browser Use on a live site (150 verified runs, same model). Sending page diffs instead of full re-renders cut token growth by 37%.

Reddit r/AI_Agents Tools

Summary

A developer benchmarks Rote, a memory manager for browser agents that sends page diffs instead of full re-renders, showing a 37% reduction in token growth compared to Browser Use, but with trade-offs on short tasks.

Every major browser agent framework fixed the old problem — they all evict old page observations now (I read the source of Browser Use 0.13.6, Skyvern, Stagehand, and Magnitude to confirm). But every one of them still re-sends a full render of the current page on every step — up to 40K chars in Browser Use — even when the only thing that changed is one checkbox. I’ve been building Rote, a memory manager for browser agents, and its core trick is: after one grounded snapshot of a page, send only the diff, keyed by content-derived element IDs (hash of role + name + ancestry) that survive re-renders and navigations. The benchmark: Browser Use 0.13.6 vs Rote, gpt-4.1-mini on both, live WordPress site, five task lengths (9–25 steps), 15 matched runs per cell. Success judged by independent page-state assertions, not the agent’s self-report. Both harnesses went 75/75. Results: • Input growth: 2,160 tokens per step vs 3,437 — 37.2% slower (95% CI 35.6–38.8) • 849 diffs sent, median 24 characters, vs 9,270-char full snapshots (99.6% median reduction) 2.7x fewer output tokens per run Where it loses, because benchmarks without losses are ads: on short tasks (~9 steps) Browser Use is ~15% cheaper in dollars despite using more tokens — their long immutable prefix is exactly what provider caches reward. Cost crosses in my favor around 13 steps, reaching 16% cheaper at 25. Also, eviction means the agent recalls what it did, not what it saw — tasks needing recall of a left page fail by design until I ship a notes mechanism. And Browser Use ships default-on history compaction, which I don’t have yet. Happy to answer anything about the methodology — and if you think the setup is unfair to Browser Use somewhere, tell me and I’ll run it.
Original Article

Similar Articles

ReactBench v1 (14 minute read)

TLDR AI

ReactBench is a new evaluation benchmark for coding agents on realistic React work, going beyond passing tests to enforce React performance, accessibility, and quality via the open-source React Doctor verifier. Early results show top models solve fewer than half the tasks, with bugs being the most common newly introduced issue.