gpu-training

The website documenting Tian Keyu's sabotage has been updated again, with a detailed timeline and evidence covering the 2024 incident in which the former ByteDance intern tampered with T5 weights, modified PyTorch source code, and fabricated research results on a VAR text-to-image generation model.

Maxime Rivest回复网友关于如何入门模型微调的提问，给出了从简单分类任务起步、逐步构建自定义任务、利用coding agent辅助实验的五步学习路径建议。

An update on the LoRA over GGUF series showing that Qwen3.8-Flash-Next (125B-A6B + 51B engram) can now be LoRA-trained in 40 GiB VRAM without CPU offloading, achieving 200 token/s on Strix Halo, with initial GGUF support merged in transformers 5.18 and further kernel optimizations (GGTensile) on the horizon.

The author trained a small language model to replace Gemini Flash for a summarization task, achieving 97% accuracy with 0.06s latency, suitable for deployment in an internal app.

GPT-6 Astra is evolving robotic hand dexterity through simulation training, achieving impressive zero-shot fine manipulation capabilities like twirling a pen and solving a Rubik's Cube, with the goal of transferring these skills to the real world.

A step-by-step repository guiding users to build and train a transformer model from scratch using PyTorch, with comprehensive coverage of data processing, training, and post-training techniques like SFT and RLHF.

Liam Fedus describes the setup of high-throughput materials labs in Menlo Park that create a loop between experiments and AI models, using 1,300 H200 GPUs and experimental data to mid-train models for next-step decisions.

A detailed write-up on training a 210M text-to-image diffusion transformer from scratch on a single GPU, sharing key measurements on attention sinks, loss as a health signal, and timestep shifting benefits.

OpenAI launches GPT-6 Astra, an AI model trained on over 100,000 GPUs at the Texas Stargate site.

A new feature called OpenResearch allows reproducing and experimenting on papers, with a one-click template to train Vector Policy Optimization (VPO) on ToolRL, enabling diverse answer generation and improved test-time search.

Andrej Karpathy open-sourced an autonomous research agent that runs its own ML experiments overnight using a single GPU, automatically iterating on improvements by editing code and keeping changes that lower validation loss.