EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments
Summary
EdgeBench analyzes 38,000 hours of real-world agent interactions across 134 tasks, revealing log-sigmoid scaling laws for performance and exponential learning speed improvements. The paper introduces a benchmark suite for studying how agents learn from real-world experience.
View Cached Full Text
Cached at: 07/07/26, 06:42 AM
Paper page - EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments
Source: https://huggingface.co/papers/2607.05155 Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Analysis of 38,000 hours of real-world agent interactions reveals log-sigmoid scaling laws for performance and exponential learning speed improvements across 134 diverse tasks.
Pretrainingscaling lawsreveal that model capability improves predictably with data and compute. But learning from real world environments after deployment remains far less understood. Analyzing roughly 38,000 hours ofagent interactionwith the environment across 134real world tasks, we find, to the best of our knowledge, the first evidence that overall performance duringenvironment learningfollows alog-sigmoid scaling lawwith remarkably high precision, reaching R^2 = 0.998. Across model generations, we also find that agent learning speed roughly doubles every three months. This discovery stems fromEdgeBench, a suite of 134real world taskswith ultra-long horizons, spanning scientific discovery, software engineering, combinatorial optimization, professional knowledge work, formal mathematics, and interactive games. Each task sustains at least 12 hours of continuous agent operation under rich,multilevel feedback, and is built through substantial expert effort. We publicly release 51 tasks and our full evaluation framework to accelerate the study of how agents learn from real world experience.
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2607\.05155
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.05155 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.05155 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.05155 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
EdgeBench Reveals the Next Scaling Law: On-the-Fly AI Learning Speed Doubles Every 3 Months
EdgeBench reveals a new scaling law indicating that on-the-fly AI learning speed doubles every three months.
@xiaogaifun: https://x.com/xiaogaifun/status/2073771786202939572
The ByteDance Seed team released the EdgeBench benchmark, which allows AI models to work continuously for 12-72 hours to evaluate their learning ability in long-horizon tasks. They discovered that the relationship between the time spent learning from the environment and performance follows a log-sigmoid curve, revealing a new scaling law.
SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Tasks
SkillLearnBench introduces the first benchmark for evaluating continual skill learning in LLM agents across 20 real-world tasks, revealing that no method dominates and scaling LLMs does not guarantee better skills.
@rohanpaul_ai: ByteDance Seed delivered again. They released EdgeBench, to test whether AI agents can improve through experience, usin…
ByteDance Seed released EdgeBench, a benchmark that tests whether AI agents can improve through experience by performing real-world tasks over 12+ hours, shifting evaluation from static knowledge to dynamic learning.
EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessions
EnterpriseClawBench presents a benchmark for enterprise agents based on real-world workplace sessions, offering 852 reproducible tasks and comprehensive evaluation metrics beyond single performance scores.