training

This article provides an update on the Engram model, detailing its 2.6b parameter architecture with a large Engram table and initial training progress at 100m tokens, showing improved completions with context-aware data offloading.

This paper reports on training a language model end-to-end in Rust, detailing failures in Rust ML frameworks like Candle and Burn, and proposing verification methods, with the conclusion that Rust is currently better suited for model serving than training.

This paper introduces methods to bound four key memory peaks in large Mixture-of-Experts training, enabling training at 1M context length with fixed GPU memory and up to 10.4× throughput improvement over baselines.

Kev is a family of small decision models built on Qwen3.5, offering pretrained weights and training code for yes/no, multiple-choice, and rating questions. It includes a web playground and is compatible with TypeSafe's System One API.

DeepSeek is training a 2 trillion-parameter AI model and plans to eventually build an 8 trillion-parameter model.

Kev is a family of small decision models built on Qwen3.5, offering open-source training code and pretrained weights for local deployment with support for various question types.

OpenAI has expanded its Academy with new learning paths for developers, leaders, educators, and college students to build AI skills and apply AI effectively in various roles.

This study explores scaling laws for sequence weighting in language model training, finding non-monotonic behavior where models transition from learning general patterns to data-specific patterns and back as scale increases.

Nathan Lambert predicts that top Chinese AI labs are increasingly using Huawei chips for inference and Nvidia for training, which will accelerate with the rise of agent swarms and scaled post-training.

An announcement for a talk at PyTorch Conference North America covering practical implementations of Muon, Dion, and Dion3 optimizers in training stacks.

Runtime by Modal is an invitation-only AI/ML conference in San Francisco on October 1, 2026, featuring tracks on inference, training, and agents with speakers from leading organizations.

An announcement for the upcoming release of Linux Kernel Exploitation, with discounted reservation spots available.

The author provides an update on building a small 2B parameter AI model with an Engram component, trained on 15m tokens from Wikipedia to achieve surprising coherence, with plans for an Apache 2.0 open-source release.

The article refutes the idea that AI development is slowing down, highlighting that Grok4.8, a 2.5T parameter model, has finished training and that major tech companies are still actively training AI models.

An astronaut recounts the thrill of landing the Space Shuttle Atlantis, detailing the rigorous training and precision required for the dead-stick descent from Mach 1 to touchdown.

ZGCM-1 is a fully open 7B LLM trained from scratch with FP8, MDP mid-training, and AI4AI, achieving performance comparable to Qwen3-235B on math and agentic search tasks.

Elon Musk announces that Grok 4.8, a 2.5 trillion parameter AI model trained with a new C++ software stack, will finish training and start reinforcement learning this week, while @kentcdodds comments on Musk's tendency to discuss future developments early.

This paper decomposes transformer representation updates into parallel and perpendicular components to study evolution geometry, linking it to editing robustness, compression diagnosis, and training improvements.

Tahuna is an open-source AI training infrastructure designed for small teams to train models, run inference, orchestrate GPUs, and experiment with autonomous research, featuring reproducible runs and tools like Hillclimb.

A developer has created a dense 9.4B parameter AI model with technical enhancements like Engram tables and AttnRes modeling, aiming for open-source release and seeking community interest for further training and deployment.