Harvard & UIUC talent discover a 3rd pretraining axis: 6.2x sample efficiency and 250x faster GenAI generation
Summary
Researchers from Harvard and UIUC discovered a third pretraining axis that improves sample efficiency by 6.2x and speeds up GenAI generation by 250x.
Similar Articles
@songhan_mit: We develop an agent-native approach to accelerate genAI, continuing the success of KDA (Kernel Design Agent) at a highe…
Enze Xie announces Sol Video Inference Engine, an agent-native, training-free full-stack accelerator for video diffusion that auto-tunes cache, sparse attention, token pruning, quantization, and kernel fusion, achieving >2× end-to-end speedup on large models like 64B Cosmos3-Super and 22B LTX-2.3.
EdgeBench Reveals the Next Scaling Law: On-the-Fly AI Learning Speed Doubles Every 3 Months
EdgeBench reveals a new scaling law indicating that on-the-fly AI learning speed doubles every three months.
@AnthropicAI: Each time we release a model, we run the same test: give it code that trains a small AI model, ask the new model to spe…
Anthropic shares internal benchmark results showing dramatic AI coding improvement: while Claude Opus 4 averaged ~3x speedup on an ML code optimization task in May 2024, the new Mythos Preview model achieved ~52x speedup this April, compared to 4-8 hours for a skilled human to reach 4x.
AI and efficiency
OpenAI analyzes trends in AI algorithmic efficiency, showing that compute required to reach AlexNet-level performance has halved roughly every 16 months since 2012, outpacing hardware gains. The study draws comparisons across domains like DNA sequencing and transistor density to contextualize AI progress.
@injaneity: https://x.com/injaneity/status/2075659478096376158
This article explains how batching and parallel operations improve latency and efficiency in AI computer use systems, highlighting open-source implementations like pi-computer-use and cua-driver that achieved significant performance gains before similar features appeared in Codex.