100 Trillion+ Pretraining data??? This is the largest data I've see a model being trained on.

Reddit r/LocalLLaMA 06/01/26, 04:38 AM Models

Summary

A new AI model is being trained on over 100 trillion tokens, doubling the typical pretraining data size of 27-50 trillion tokens used by other models like Kimi, Mimo, and DeepSeek.

https://preview.redd.it/oss7g2gnll4h1.png?width=894&format=png&auto=webp&s=5d4295707a700ed7541c274b8be8ad75bbd0903d Usually we see 27-50 Trillion tokens in most models, kimi, mimo, deepseek. They seem to have doubled the pretraining data. Minimax-m2.5 was like 27T tokens. If we see mimo, they have done: \- 27T for the Mimo-v2.5-Pro 1 Trillion Parameters \- 48T for the smaller Mimo-v2.5 model which is multimodal. \- 32T for Deepseek V4 Flash and Pro I find it difficult to believe this model will be much bigger than the previous M2 series models. The training data scale is way too big, and will require way more resources for a much bigger model. M3 seems likely to be under 500B params.

Original Article

Similar Articles

Meta’s muse spark 1.3 surpassed fable 5 and GPT 5.6 sol 🫪

Reddit r/singularity

Meta's Muse Spark 1.3 AI model has reportedly surpassed Fable 5 and GPT 5.6 in performance metrics.

Meta slowly catching back up. Muse Spark 1.3 beats Sol on AA

Reddit r/singularity

Meta's Muse Spark 1.3 AI model outperforms Sol on the AA benchmark, indicating progress in Meta's AI development.

Embedded Rust RTOS vs. C RTOS

Hacker News Top

This blog post compares the performance and ease of use of Embassy (async Rust) against FreeRTOS (C) on an STM32F446 microcontroller, focusing on interrupt latency, memory usage, and programming simplicity.

CABiNet (ICRA 2021) vs YOLO26-sem on UAVid: accuracy, compute, and GPU latency [P]

Reddit r/MachineLearning

A benchmark comparison shows CABiNet, a 2021 efficient architecture, achieves better accuracy-to-latency trade-offs than YOLO26-sem on the UAVid dataset for real-time semantic segmentation.

Efficient GPU Retrieval for Semantic Search

arXiv cs.AI

This paper introduces a policy-aligned retrieval framework for semantic search on LinkedIn, leveraging embeddings partitioned into category-supervised segments and a two-stage GPU architecture to improve recall and precision, with significant gains validated in A/B testing.