@JangLawrenceK: We partnered with Browser-Use to release a new live web eval as a followup to the original Odysseys paper Odysseys2 ser…
Summary
Browser-Use partnered with Carnegie Mellon researchers to release Odysseys2, a hard long-horizon live web-agent benchmark built from real user tasks, where the best SOTA agent completes only 47/100 tasks.
View Cached Full Text
Cached at: 10/28/26, 07:39 PM
We partnered with Browser-Use to release a new live web eval as a followup to the original Odysseys paper 😁
Odysseys2 serves as a task set that sources from real user requests to Browser-Use, check it out!
It was great working with @Alezander9 and @mamagnus00 - thank you to Browser-Use for the support and resources!
Browser Use (@browser_use): Introducing: Odysseys2 - hardest long horizon benchmark for web agents based on real user tasks 🏗️
We collaborated with @JangLawrenceK @rsalakhu from Carnegie Mellon University
> SOTA agent fully completes only 47/100 > 5 agents complete none > 8-12 real rubrics per task, 60
Similar Articles
@rsalakhu: Congrats to the @browser_use team for taking the #1 spot on Odysseys, a highly challenging benchmark for long-horizon w…
The browser_use team achieved the #1 spot on the Odysseys benchmark, a challenging evaluation for long-horizon web agents, outperforming models like Opus 4.6 and GPT-5.4.
OSWorld2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
OSWorld 2.0 is a new benchmark for evaluating computer-use agents on 108 long-horizon, real-world workflows. Current agents like Claude Opus 4.8 and GPT-5.5 achieve low completion rates, highlighting significant limitations in handling complex, multi-step tasks.
@browser_use: BrowserCode is incredibly good at long-running tasks It orders pizza for us
BrowserCode achieves #1 spot on Odysseys benchmark for long-horizon web agents, demonstrating strong performance in multi-hour web workflows.
@gregpr07: Browser Use Beta just achieved SOTA on our hardest internal web agent benchmark. Fable is genuinely amazing for optimiz…
Browser Use Beta achieved state-of-the-art results on a difficult internal web agent benchmark, using Fable for optimization and analysis.
I benchmarked my browser agent against Browser Use on a live site (150 verified runs, same model). Sending page diffs instead of full re-renders cut token growth by 37%.
A developer benchmarks Rote, a memory manager for browser agents that sends page diffs instead of full re-renders, showing a 37% reduction in token growth compared to Browser Use, but with trade-offs on short tasks.