I trained a 1.57B-parameter Dreamer 4 World Model from scratch for under $150
Summary
The author trained a 1.57B-parameter Dreamer 4 world model from scratch for under $150, demonstrating the accessibility of low-cost AI model training and using procedural data for validation.
Similar Articles
A 4b model is now beating 30b ones at web research and the reason is not size
A 4 billion parameter open model from the Apodex family outperforms 30 billion parameter models on web research benchmarks, attributed to careful training data and self-verification techniques rather than raw scale, suggesting a more democratic trajectory for AI capability.
@heyrobinai: THE ENTIRE AI INDUSTRY JUST GOT HUMILIATED a tiny model trained in just a few hours on a single graphics card is planni…
Yann LeCun's team releases LeWorldModel, a tiny 15M-parameter physics model trained on a single GPU in hours that outperforms billion-dollar foundation models in planning speed and physical plausibility, challenging the dominant scaling paradigm.
@vintcessun: Pretraining can be this cost-effective? Train a usable 1B base model from scratch for ~$1000, slashing compute and data by hundreds of times. The key isn't brute-force compute, but hierarchical recursive architecture plus latent space reasoning, combined with PrefixLM packing and FA3 to maximize efficiency. Sounds insane, but the paper and code are open-sourced.
HRM-Text released a 1B-parameter base model, claiming it can be pretrained from scratch for only ~$1000, reducing compute and data volume by hundreds of times. It employs efficient techniques such as hierarchical recursive architecture, latent space reasoning, and PrefixLM packing. The paper and code are open-sourced.
What if i really wanna train an AI from scratch?
A personal reflection on the challenges and allure of training an AI model from scratch, highlighting the difficulties with data, hardware, and scaling, while noting that surprisingly good small models can be trained on modest hardware.
MiniWorld: Democratizing the Training of Video World Models from Scratch
MiniWorld is a reproducible framework for training video world models from scratch using a block-causal Video Diffusion Transformer with Flow Matching, enabling efficient streaming generation and trainable in days on a single 8-GPU server.