h100

作者发布了一个免费、代码优先的 GPU 内核工程系列课程（共 44 节）的介绍与路线图视频，从朴素 GEMM 讲到 FlashAttention，覆盖 Hopper 与 Blackwell 架构，并以 roofline 模型和数据移动为核心教学理念。

A hardware-based thought experiment estimating that a hypothetical GPT-6-class model serving 66 tokens/s per user across a 800,000-H100 pool could have an FP8 weight-memory ceiling of about 19.2 trillion parameters, while emphasizing this is a rough upper bound rather than a prediction of GPT-6 Astra's actual size.

A post details the booming underground GPU repair industry in Shenzhen, where technicians service banned NVIDIA H100/A100 cards, alongside reports that China's H20 demand far outstrips NVIDIA's inventory amid ongoing US export restrictions.

The article explains how to determine if a GPU workload is compute-bound or memory-bound by analyzing operations per byte fetched from HBM, using NVIDIA's H100 as an example, and discusses how batching and prompt length affect performance.

An analysis of GPU rental prices on Vast.ai reveals that pricing for multi-GPU configurations deviates significantly from per-GPU-hour headline rates due to supply constraints, with larger clusters often being unavailable or more expensive.

A 117M parameter Silia model was trained on an H100 GPU in 5 hours using 82M tokens. The model is severely under-trained, but comparisons with nanoGPT are provided.

Analysis of spot/interruptible GPU pricing for H100 and A100 across RunPod, Vast.ai, and AWS as of June 2026, noting significant discounts but also variability and availability issues.

Red Hat demonstrates that using speculative decoding can boost LLM inference speed from 145 to 424 tokens per second on the same H100 hardware with no quality loss, highlighting a significant optimization for production serving.

Analyzes the argument that AI compute cannot become a commodity due to non-fungibility, using commodity market analogies to suggest that a standardized reference plus basis trading could enable commoditization.

Internal testing of DiffusionGemma reveals significant performance differences between H100 and A100 GPUs under real-world workloads, with H100s scaling much better under concurrency, and efficiency varying greatly depending on workload type, raising questions about benchmark reliability.

The causal version of the SANA world model has been released, enabling near real-time inference for video generation on a single H100 GPU, with open-source code and a demo.

A user observes that Nvidia H100 GPUs seem to have suddenly disappeared from all cloud platforms, sparking discussion about supply constraints.

This benchmark compares Gemma 4's Multi-Token Prediction (MTP) and z-lab's DFlash speculative decoding methods on a single H100 GPU, showing MTP faster for dense models and DFlash faster for MoE models.

SpaceXAI and Cursor are collaborating to build advanced coding and knowledge-work AI, leveraging Cursor’s developer reach and SpaceX’s massive H100-equivalent Colossus supercomputer.

A quick breakdown of ballpark numbers for a 100k H100 GPU datacenter, covering GPU costs (~$3B), full datacenter build (~$5B), power consumption (~0.2GW), and annual energy costs (~$50M).