@WilliamBarrHeld: To train better open models, we need predictable scaling. Delphi is Marin’s first step: we pretrained many small models…
Summary
Marin AI researchers, led by William Barr Held, introduce Delphi, a methodology that pretrains small models to accurately predict the training outcomes of larger 25B-parameter runs. This research aims to establish predictable scaling for more efficient open-source AI model development.
View Cached Full Text
Cached at: 05/11/26, 08:43 PM
To train better open models, we need predictable scaling.
Delphi is Marin’s first step: we pretrained many small models with one recipe, then extrapolated 300× to predict a 25B-param / 600B-token run with just 0.2% error.
Getting there took some work 🧵 https://t.co/HmlVFl11ag
Similar Articles
@eliebakouch: one of my favorite projects is Marin from the stanford folks, they have a scientific approach to training, are ready to…
Marin is an open-source framework from Stanford for reproducible foundation model research, covering data curation, tokenization, training, and evaluation; it was used to train an 8B parameter model that outperforms Llama 3.1 8B.
@AndrewYNg: In the fight to defend openness in AI, the Marin project is a precious demonstration of openness in model training, wit…
Andrew Ng highlights the Marin project as a valuable example of openness in AI model training, with Percy Liang announcing the start of training for Marin 535B-A23B using open code, data, and processes.
@WilliamBarrHeld: There’s an enormous amount of open training data on Hugging Face. What does it take to make it work together ? For Mari…
The article discusses the process of utilizing open training data from Hugging Face to train Marin’s 535B model, which involved 25T tokens from 152 datasets with permissible licenses.
@yuetai12575: Excited to see the gains further survive as model scaling!
The article reports initial results from OpenRSI's Marin-Scaling-Ladder experiment, where AI agents autonomously propose, implement, and evaluate new optimizers across model scales from 550M to 2.5B, discovering PSPR with Codex (GPT-5.6).
@andykonwinski: i can’t stop checking in on this. the marin team is training the largest fully open model ever. 535B params (23B active…
The Marin team is training the largest fully open model with 535B parameters, providing unprecedented transparency with live tracking and open data logs.