@YRSM_Simon: Jensen's precision cuts so sharp it makes your teeth itch. NVIDIA promotes DGX Station: 748GB unified memory. Sounds like it crushes everything—4× RTX PRO 6000's 384GB? Not enough. But look closer—748GB = 252GB HBM3e + 496GB …
Summary
Reveals that the 748GB unified memory advertised for the NVIDIA DGX Station actually only has 252GB of high-speed HBM available. The remaining 496GB of slow LPDDR5X is essentially useless for large model inference, reflecting NVIDIA's precise product differentiation strategy.
View Cached Full Text
Cached at: 07/09/26, 09:34 AM
Jensen’s precision cuts are infuriatingly sharp
NVIDIA promotes the DGX Station: 748GB of unified memory.
It sounds like it crushes everything—4× RTX PRO 6000’s 384GB? Not even close.
But look closer—748GB = 252GB HBM3e + 496GB LPDDR5X.
That 496GB of LPDDR5X has only ~273GB/s bandwidth, 26× slower than HBM’s 7.1TB/s. For large model inference, only the data in HBM is “fast”—the remaining 496GB is basically decoration, fine for storing inactive weights, but when you actually compute, it’s slow motion.
So the real high-speed HBM available: 252GB.
Advertised as 748GB, actual usable high-speed: 252GB. The beauty of this cut:
- The number 748GB just barely beats 4× RTX PRO 6000’s 384GB, making you think you don’t need a multi-card setup.
- But 252GB HBM is exactly 12% less than the full 288GB, making 200B models just a bit tight.
- The 496GB LPDDR5X rounds up the number nicely, but it’s useless for actual inference/training.
Even worse: NVLink disappears from the RTX PRO 6000, so multi-card setups must use PCIe. You think 4× 384GB is the way out? Communication overhead eats into performance. You think the Station’s 748GB is the ultimate answer? Only 252GB can actually run.
The marketing makes it look unbeatable, but in reality it leaves you just hungry enough.
4× RTX PRO 6000: 384GB GDDR7, 7.2TB/s total bandwidth, $60k. True 252GB HBM? Sorry, $85k+ and it’s been gimped.
Jensen doesn’t care which one you buy—he cares that no matter which path you choose, you’re never comfortable.
That’s the knife.
Similar Articles
@AYi_AInotes: Damn, NVIDIA and Jensen Huang really have something huge up their sleeves. It's absolutely insane. Today, the whole internet is sharing this laptop by Jensen Huang that can run 3A games at full frame rate even when unplugged, but most people are missing the point. Gaming is just the sugar coating. The real bomb is the 128GB unified memory, which means that a thin and light laptop on your desk can locally…
The article reviews NVIDIA's new laptop. Its 128GB unified memory enables local execution of a 200B parameter large model, maintains frame rate when unplugged, and targets users needing local AI deployment. It considers this an important step in bringing data center capabilities to portable devices.
@svpino: We are getting 2 new devices from NVIDIA and Microsoft: 1. The DGX Station, with GB300 superchip and up to 748GB of mem…
NVIDIA and Microsoft are releasing two new AI hardware devices: the DGX Station with GB300 superchip and 748GB memory, and the RTX Spark laptop with 1 petaflop AI performance and 128GB unified memory.
@SuJinYan123: NV72: Each tray has 4 GPUs, each tray is equipped with 2 NVIDIA Grace (ARM Neoverse V2, 72 cores, total 144 cores, 3.39 GHz), single CPU ~480 GB LPDDR5X, so 4 GPUs...
The article provides a detailed analysis of the tray design in the NV72 architecture, where every 4 GPUs are paired with 2 Grace CPUs. It discusses optimizing inference performance under coding workloads using parallel strategies such as EP64 and TP8, as well as the complexity of KV cache management and routing algorithms.
@rickawsb: NVIDIA also believes storage is a bigger bottleneck than GPUs — Decoding NVIDIA's latest article. NVIDIA's newly released 'AI Model Co-Design' is a technical article introducing TensorRT-LLM and Blackwell, but also a roadmap for large model design and AI infrastructure in the coming years...
This article provides an in-depth interpretation of NVIDIA's newly released 'AI Model Co-Design' paper, pointing out that in AI inference scenarios, storage (memory bandwidth, weight reading) has replaced GPU compute as the primary bottleneck. It elaborates on the design strategies of TensorRT-LLM and Blackwell architecture around the Roofline model, emphasizing that reducing data movement is more critical than improving compute power.
@FinanceYF5: SK hynix and NVIDIA have signed a multi-year cooperation agreement. Memory chips cannot wait for GPU design to be completed; advanced DRAM requires years of joint design, manufacturing planning, and capital investment. AI supercomputers, personal AI PCs, Jetson robot platforms. Using NVIDIA tools to create a digital twin of the chip factory, first running all tests in the virtual factory before the actual production line...
SK hynix and NVIDIA have signed a multi-year cooperation agreement covering joint design and manufacturing planning of advanced DRAM, as well as AI infrastructure (supercomputers, AI PCs, Jetson) and digital twin factory technology, marking that the AI infrastructure competition has reached the memory level.