Tag
Luo Fuli's retrospective on MiMo-V2.6 details Xiaomi's innovations in scaling reinforcement learning for agents, focusing on expanded environments, integrated harnesses, and improved grading systems, leading to significant advancements in agent capabilities.
The article discusses the pattern where open-source AI models like Qwen match frontier capabilities about two quarters later, exemplified by GPT-5 and Qwen3.5-27B, and questions if this trend will continue with models like Astra and Fable 5.1.
This paper investigates whether transformers trained on quantum measurement data from Rydberg atom arrays exhibit neural scaling laws similar to large language models, finding that scaling behavior depends on the statistical structure of the data, particularly near critical points.
This study explores scaling laws for sequence weighting in language model training, finding non-monotonic behavior where models transition from learning general patterns to data-specific patterns and back as scale increases.
Nucleus Genomics announces Vitruvian, a set of genetic optimization models trained on over a million people to enhance embryo DNA for increased intelligence, advancing human genetic optimization as proposed by Nick Bostrom.
This paper studies scaling laws for physics-aware surrogate models in AC optimal power flow, demonstrating that physics-informed training objectives improve constraint satisfaction at scale with different scaling rates than standard loss functions.
The paper establishes a theoretical framework showing that data and memory resources in autoregressive prediction are governed by a single predictive-energy spectrum, providing a joint scaling law.
This paper presents theoretical guarantees for one-shot magnitude pruning and compute-adaptive early exit in neural networks, analyzing compute-accuracy tradeoffs and deriving scaling laws through asymptotic and deep network models.
This paper presents a large-scale study comparing distilled byte and token models, finding that byte models achieve higher performance ceilings with more compute and are more data-efficient than token models.
An analysis by Adam Majmudar on the rapid progression of AI capabilities, highlighting how new scaling laws are driving leaps beyond external expectations.
The article explores the difference between internal and external views on AI progress, emphasizing the role of scaling laws and capability jumps in shaping perceptions of future advancements.
This paper quantifies the memorization-to-generalization transition (grokking) in neural networks through scaling laws, revealing that data complexity is the primary driver of transition time compared to model capacity.
This study rigorously tests whether scaling web-video pre-training improves robot task performance, finding that larger models and more compute consistently yield better policies on real industrial tasks, with pre-training quality correlating with performance.
Magic AI Labs announces a new pretraining recipe that matches DeepSeek V4 Pro's performance with 50x less compute, demonstrating significant algorithmic efficiency gains for scaling large AI models.
OpenAI researchers reflect on the rapid scaling of reasoning models and warn that AI systems are likely to achieve recursive self-improvement within years, calling for extreme caution and broader interventions beyond technical solutions.
This paper introduces Coupled Scaling, a task-conditioned framework that explains how neural scaling laws vary based on the relationship between task structure and the geometric representations accessible by architecture-optimization systems.
This paper investigates scaling laws in actuarial ratemaking models using tabular data, finding that TabM exhibits stronger data scaling than standard Transformer and MLP architectures without additional inductive biases.
Open Athena has launched a 535B/A23B MoE model training run that is proceeding as predicted, highlighting their commitment to open development and significant progress in AI scaling.
Isaac 0.5 is a 36-billion-parameter AI model that efficiently adapts to various robotic embodiments by leveraging 1M hours of video and 100K hours of robotic interaction data, demonstrating a scaling law where increased video data significantly reduces teleoperation requirements.
SMELT is a method that loops middle layers in Mixture-of-Experts Transformers to improve training efficiency and downstream performance while matching compute, parameter, and cache budgets, leading to faster loss reduction and practical gains.