Nobody Noticed ByteDance Seed’s Research? It’s Highly Valuable.

Scrolling through X, I saw that the ByteDance Seed team just released a very interesting Benchmark called EdgeBench. I highly recommend everyone check it out.

Currently, the vast majority of Benchmarks give a model a one-time task, the model outputs an answer, and then it’s over. But this approach increasingly fails to reflect the true level and capability of a model.

After all, models today are native Agents. The characteristic of an Agent is its ability to adjust its task execution plan based on feedback in real time.

Just like how we work — if we make a mistake the first time, after seeing the feedback, we can optimize and try again. Traditional Benchmarks simply cannot measure this kind of ability.

I remember a few days ago reading an interview with OpenAI researcher Noam, who said that the common Benchmarks currently prevalent in the industry are basically outdated.

Because models have self-correction capabilities, letting a model work on a task for one minute versus letting it work for one hour or one day yields completely different final results.

Yet, the vast majority of current Benchmarks completely fail to account for the time factor.

EdgeBench incorporates time this time. It allows each model to work continuously for 12 to 72 hours, and finally observes the performance changes over such a long period.

This new Benchmark covers 134 real-world tasks that require long periods of work, including categories like Coding, complex knowledge work, games, mathematical proofs, etc.

For comparison, human experts spend an average of 57 hours on these tasks — not something that can be solved in seconds or minutes.

So traditional Benchmarks cannot accurately measure the true effects of such long-horizon tasks. EdgeBench gives Agents at least 12 hours to work continuously, then observes their performance changes over time.

At the same time, it establishes a feedback mechanism, allowing the Agent to continuously try, continuously submit, and continuously receive new score feedback. This is somewhat similar to what we’ve recently been discussing as Loop Engineering.

They ran tests using five models — Claude Opus 4.8, GPT-5.5, GPT-5.4, GLM-5.1, DeepSeek-V4-Pro — totaling about 38,000 hours of testing, and derived several interesting findings. Let me write down my understanding.

Finding One: A New Scaling Law

Whether for different models or different task types, after averaging the learning curves across the 134 tasks, the performance changes over time almost all follow the same curve, mathematically called a log-sigmoid curve.

Why emphasize “averaging”? Because looking at any single model on any single task, the curve is actually quite messy — there may be no progress for a long time, then a sudden jump, and sometimes even regression.

But after overlaying the results of 134 tasks, the noise of individual tasks is smoothed out, leaving a very regular curve.

This curve, within the observation window of 1 to 12 hours, primarily shows a process of continuous deceleration: the fastest improvement occurs in the first few hours, where the model densely gains experience from environmental feedback, making significant progress.

As time goes on, the easy gains are gradually exhausted, the return on continued time investment becomes smaller and smaller, and the curve gradually flattens.

They mathematically fitted this curve, achieving an average R² of 0.998. The perfect score for R² is 1, so 0.998 indicates that this theoretical curve almost perfectly matches the actual data.

Across a large number of tasks, the relationship between the time a model invests in learning from its environment and the benefits it gains is highly consistent.

We previously knew that pretraining has a Scaling Law — predictable relationships between model size, data volume, compute investment, and performance — which has guided the development path of large models in recent years.

Now the results from EdgeBench show that learning from the environment also has its own Scaling Law.

The relationship between the invested interaction time and the obtained performance gains also follows a highly stable mathematical relationship.

This is the most core finding of this paper.

Finding Two: The Learning Process Can Be Explained Using Graph Exploration Theory

The final score of each task can be broken down into many small score units, distributed on a graph. Each time the model unlocks a unit, it gets the corresponding score.

When the model first enters the task, it has almost no information about this graph, can only explore randomly, and occasionally unlocks a unit.

But once a unit is unlocked, it exerts influence on its surrounding un-unlocked neighbors, making them easier to unlock.

The more units are unlocked, the greater the total influence exerted, and the faster the frontier advances. This is the source of acceleration in the middle phase of learning.

As exploration continues, the number of remaining un-unlocked units on the graph decreases, the advanceable frontier shrinks. Even if influence persists, there are fewer objects to act upon, so the speed naturally drops.

From an overall perspective, the learning process is continuously expanding the unlocked area outward on a task graph. Early on, the main focus is finding the entry point; in the middle, rapid expansion occurs through accumulated influence; later, approaching the boundary, growth slows down.

This structure naturally leads to a learning curve that is slow at first, then fast, then slow again — the log-sigmoid form mentioned in Finding One.

Finding Three: AI’s Speed of Learning from the Environment Is Accelerating

They performed a cross-model comparison: selected several frontier models released between September 2025 and May 2026, placed them in the same long-task environment, controlled for time, uniformly observed only the performance changes in the first two hours, and compared who learned more from the environment within the same time window.

The results show that this learning speed is continuously increasing, roughly doubling every three months. In other words, within the same time window, the efficiency of models extracting useful experience from environmental feedback is improving.

If this trend continues, it means that in the future, Agents will be able to accomplish within a short period the learning process that currently takes several hours to accumulate.

Finding Four: Refactoring Moments in Long-Duration Tasks

The paper selected a gravitational wave signal reconstruction task for detailed tracking.

This task requires the Agent to start from raw LIGO observation data and reproduce the analysis results of a classic gravitational wave paper. It needs to output three types of content: waveforms, spectrograms, and celestial motion curves — a multi-module scientific analysis task.

They let the GPT-5.5 Agent run continuously on this task for 12 hours, recorded a total of 247 scoring attempts, and the final score improved from 42.8 to 67.0.

This process did not advance uniformly. The paper marked 7 key nodes, each representing an obvious leap in the Agent’s understanding of the task.

In the early stage, the Agent’s first action was to make the entire task evaluable.

The original task was open-ended; it first needed to build a complete analysis pipeline that could run, get a feedback score, and then have a foundation for subsequent optimization.

In the middle stage, the Agent hit a bottleneck where the waveform matching effect was not improving.

At this point, it changed its approach, splitting this big problem into several sub-problems: reference signal localization, time-frequency analysis, and detector alignment, tackling them separately. After splitting, the progress speed increased significantly.

Later, itemized feedback showed that the celestial dynamics part was the biggest shortcoming.

The Agent locked onto this direction and concentrated on a breakthrough, raising the score for this sub-item from 64 to 89 within a few hours — the biggest jump during the entire 12-hour period.

In the later stage, the overall plan had stabilized. The Agent kept the core logic unchanged and only fixed the remaining local errors.

The entire process closely resembles how humans approach complex projects: when you reach a certain stage and find that the original understanding is not working, you re-decompose the problem structure, adjust the strategy direction, and then continue pushing forward.

EdgeBench can record the complete trajectory of continuous trial-and-error plus periodic refactoring.

I strongly recommend everyone check out this Benchmark from the ByteDance Seed team and the interview with OpenAI researcher Noam.

I’ve put the links at the end of the article. We truly need a new evaluation method that can measure a model’s learning performance over time in real, feedback-driven task scenarios.

• https://github.com/ByteDance-Seed/EdgeBench

• https://www.youtube.com/watch?v=AZrU6y3pUcU&t=1s