HRM Seems To Be Going Off Right Now

TL;DR: Sapient Intelligence released HRM-Text, a 1B-parameter text generation model trained on only 0.04 trillion tokens (cost ~$1,000), which surpasses larger models trained on 100–1000x more data on multiple reasoning benchmarks like Math, DROP, and ARC-Challenge. Its core innovations are "task-completion learning" and "hierarchical recurrence + prefix LM attention," enabling efficient reasoning in both depth and breadth, marking the beginning of a new paradigm for AI training. ## Background: The Scale Hypothesis Is Being Challenged In the past few years, the AI industry has operated on a single hypothesis: progress requires scale—bigger models, more data, more compute. If results aren't good enough, train a bigger model. This hypothesis works, but it's not the only path. It concentrates capabilities in the hands of a few, makes iteration slow and deployment expensive, and makes pretraining nearly impossible for most teams. Sapient Intelligence's co-founders William and Guan explain that they started building HRM (Hierarchical Reasoning Model) because they believed there was a better way—one that doesn't rely on endless scaling. The result is the new generation of HRM proof-of-concept model family, especially HRM-Text: the most streamlined and powerful AI model of its kind, built on a fundamentally different architecture, trained on a fraction of the data, and deployable anywhere. ## HRM-Text Model Overview HRM-Text is a 1-billion-parameter text generation model built on the HRM architecture at the core of SPN. It is trained on approximately 40 billion (0.04 trillion) tokens of structured data. For comparison, models are typically trained on 4 to 36 trillion tokens. HRM-Text is nearly 1000x more data-efficient; the entire model can be pretrained in about a day with a very low budget (~$1,000). After quantization, it takes up only about 0.6 GB and can be deployed on a laptop, mobile device, or edge server. Streamlined training data, streamlined computation, streamlined deployment—but streamlined only matters if the model is equally powerful. ## Benchmark Results (Independent Validation, April 2026) HRM-Text was evaluated on four benchmarks testing different dimensions of intelligence and compared with models including OMO 3 7B, Quan 3.5 2B, Jamma 3 4B, LMA 3.2 3B, and GPT 3.5. ### Math (Mathematical Reasoning) Each problem requires planning a solution path, executing across multiple operations, and tracking intermediate results. HRM-Text scores 56.2, the highest in the table. OMO, with 7x the parameters (trained on 150x more data), scores 40.0—a 16-point gap, yet using only 1/300th of the computation. Llama scores 48.0, Quinn 34.2, GPT 34.1, JMA 24.2. ### DROP (Discrete Reasoning Over Paragraphs) DROP tests a model's discrete and numerical reasoning over a given text. HRM-Text scores 82.2, with the closest open model around 71.5—10 points higher than the 7x larger model. Cohere scores 30.8 (less than 38% of the result), despite training on 900x more data. GPT scores 64.1, JMA 60.1, Llama 45.2. DROP is the benchmark with the largest gap and most directly reflects the design purpose of HRM's recurrent architecture: hold graph-based information and process it precisely. ### ARC-Challenge (Science & Common Sense) A set of science and common sense questions requiring strong understanding of how the physical world works. HRM-Text scores 81.9, again leading the group, with hundreds of times better training efficiency. GPT scores 85.2, but at a massive cost. ### MMLU (Breadth of Knowledge in 57 Domains) From history to medicine, law, physics. HRM-Text scores 60.7, a top-tier performance. Competing models trained on trillions of tokens have an advantage in memorizing facts. HRM-Text's use of only 40 billion tokens is a deliberate choice—a model built for reasoning depth covers fewer random facts compared to models built to absorb the entire internet. ## Efficiency Argument: Training Cost Comparison Two scatter plots show efficiency from different angles: high benchmark average vs. low training cost. HRM's blue star sits alone in the top-left (sweet spot), marked 1x. All other models are farther lower-right. On a log scale, competing models used 100 to about 1000x more tokens, 130 to 600x more FLOPs, and GPT is estimated to have used 44,000x more FLOPs. Even with this incredible efficiency, HRM still delivers better average reasoning performance. ## Core Architectural Innovations How does HRM-Text achieve this? Two key ideas: ### 1. Task-Completion Learning (Making the Model Learn Smarter) Traditional models learn by predicting the next token, giving equal weight to every word (filler words, function words, critical reasoning steps all treated equally). HRM-Text uses structured instruction-response pairs, with loss computed only on the response. The model learns from reasoning steps and solutions, not from surface language. It no longer focuses on "what is the next word" but on "how to complete the task." This allows the model to quickly recognize patterns and derive rules from similar data structures, learning efficiently from fewer samples. ### 2. Deeper and Broader Structure (Making the Model Think Deeper and Broader) Most models run in shallow stacks (~30 layers), reading one word of the problem at a time, only getting the full picture at the last token. HRM-Text takes a different approach in both depth and breadth: - **Depth**: 120 effective layers, more than 4x deeper than standard LMs. - **Breadth**: Can see the complete problem repeatedly in one pass, providing space for true computation over the entire input. How it works: - **Hierarchical Recurrence**: The stack is divided into two modules—a high-level module (updates slowly to set direction) and a low-level module (updates quickly to handle fine-grained steps). This structure keeps training stable. - **Prefix LM Attention**: Enables the model to attend to the full problem in each pass. Deeper and broader, but not larger. ## The Significance of the New Paradigm The significance of HRM-Text goes far beyond a single model release. It marks the beginning of a new paradigm—where AI concepts can be realized and future development can become much more streamlined. Today, you can train a model from scratch in about a day at a cost of roughly $1,000, achieving performance comparable to models that took months to produce. This opens the door to many research possibilities like continual learning, expert models, and more. If building AI requires trillion-token datasets and thousands of GPUs, most people in the world cannot participate. HRM-Text is designed specifically to solve this problem. Efficiency is not an end in itself, but to change who can build, who can deploy, and who can benefit. For the research community, this reopens what scale closed. For years, great architectural ideas went unrealized—not because they were wrong, but because no one could afford pretraining. When pretraining cost drops by nearly a thousand times, the architectural space becomes explorable again. That is the true AGI era. HRM-Text is leading in this area. Task completion, latent space reasoning, hierarchical structure—this is just a foundation, not a ceiling. New scaling laws are now being defined. ## Open Invitation Sapient Intelligence continues to bridge the gap toward general artificial intelligence by decoupling thinking from knowledge, enabling continual learning, and pioneering a new scaling paradigm. They invite everyone to explore HRM-Text and join them in shaping the future of intelligence. Source: https://www.youtube.com/watch?v=jP2HgeLyS30