EdgeBench Reveals the Next Scaling Law: On-the-Fly AI Learning Speed Doubles Every 3 Months
Summary
EdgeBench reveals a new scaling law indicating that on-the-fly AI learning speed doubles every three months.
Similar Articles
AI and compute
OpenAI releases an analysis demonstrating that compute used in largest AI training runs has grown exponentially at a 3.4-month doubling time since 2012, representing a 300,000x increase and vastly outpacing Moore's Law. The analysis suggests this trend will likely continue and calls for increased academic AI research funding to address rising computational costs.
AI and efficiency
OpenAI analyzes trends in AI algorithmic efficiency, showing that compute required to reach AlexNet-level performance has halved roughly every 16 months since 2012, outpacing hardware gains. The study draws comparisons across domains like DNA sequencing and transistor density to contextualize AI progress.
@rohanpaul_ai: ByteDance Seed delivered again. They released EdgeBench, to test whether AI agents can improve through experience, usin…
ByteDance Seed released EdgeBench, a benchmark that tests whether AI agents can improve through experience by performing real-world tasks over 12+ hours, shifting evaluation from static knowledge to dynamic learning.
How AI training scales
OpenAI researchers discovered that the gradient noise scale, a simple statistical metric, predicts the parallelizability of neural network training across a wide range of tasks. They found that more complex tasks and more powerful models tolerate larger batch sizes, suggesting future AI systems can scale further through increased parallelization.
@dunik_7: An AI more than doubled its own coding ability while the researchers just watched. 20% -> 50% on SWE-bench. They never …
A paper from Jeff Clune's lab describes an AI that doubled its coding ability on SWE-bench from 20% to 50% by rewriting its own source code without human intervention, using an evolutionary approach.