scaling

This paper proposes a structured ladder for scaling large reasoning models beyond human supervision, addressing challenges in autonomous rewards and self-generated experience, while identifying risks and evaluation dimensions.

A discussion seeking insights on real-world strategies for tracking and reducing production AI costs, highlighting challenges like cost spikes and trade-offs with quality.

The author critiques the uncritical desire to scale up projects, using J.B.S. Haldane's biological essay to argue that size changes require adaptation and prevent staying the same.

This paper introduces World Model RL to scale automatic research agents by replacing environment execution with a learned world model, thereby accelerating post-training by 3-4x and enabling smaller agents to outperform larger ones on benchmarks.

Clay scaled agent evaluations to over 300 million runs per month, covering their four-quadrant eval framework and the challenges of closing the production-to-eval loop.

The author reflects on the challenges of managing multiple AI agents as they scale, comparing it to infrastructure issues in microservices and mentioning tools like Lyzr for central management.

The paper presents a theoretical framework for steering and scaling large language models via sampling algorithms, such as Sequential Monte Carlo and Replica Exchange, to improve generation quality without external supervision.

Namespace Labs is unboxing MacBooks at scale to use as server racks, suggesting the idea of using MacBooks as a server farm for startups.

This paper scales the Muon optimizer for Diffusion Transformers from 1.3B to 15B parameters, introducing Periodic Row-wise Muon to reduce computational overhead while preserving generative quality improvements over AdamW.

Apodex 1.1 improves sustained, verifiable progress on complex real-world tasks by scaling executable environments and training agents for long-horizon coordination, achieving leading performance with a smaller 35B-parameter model.

Percy Liang shares an overview of the data composition for the Harrier K40 training run, detailing domain clusters and quality buckets as part of the Marin team's joint effort.

Training of the Marin 535B-A23B AI model has begun with an open process, involving pretraining and midtraining on 18.75T tokens using GB200 NVL72 hardware over about 3 months.

The article speculates on the potential impact of concentrating the global compute used for millions of AI prompts onto a single scientific problem, such as curing cancer, and questions whether this could lead to deeper scientific intelligence.

IBM announced a new modular cryogenic architecture to connect quantum processors, enabling scalability and future fault-tolerant quantum systems as part of their quantum roadmap.

The article details the infrastructure and challenges in training the Dyna-2 AI model at a million-hour scale, focusing on data lifecycle and GPU cluster optimizations.

A tutorial on using vLLM to efficiently serve LLM inference for AI agents, covering concepts like continuous batching and PagedAttention to address performance bottlenecks.

This paper reviews eight years of language model evolution from BERT to advanced agentic systems, documenting the collapse of the capability-cost curve and the emergence of task-targeted models through empirical benchmarks and reproducible experiments.

The paper introduces full-bandwidth transformers, which use latent feedback to enhance autoregressive models by allowing non-verbalized computation to re-enter the stack, improving performance with negligible decoding overhead.

The paper 'On the Entropy Calibration of Language Models' interprets rising entropy as increasing diffusion of accessible pathways in autoregressive generation, proposing that scaling has limited benefits due to heavy-tailed data and suggesting a pathway-aware decoding alternative.

This paper identifies a scalability bottleneck in RL-trained automatic research agents—environment execution dominates training cost—and proposes World Model RL (WMRL) with online debiasing and inverse-variance denoising to replace real execution, achieving 3–4x training speedups and better generalization.