hbm

Micron and Samsung executives say the memory shortage, driven by surging AI demand for HBM and server DRAM, will persist through 2028, with production capacity prioritized for AI over consumer devices.

Micron's CEO warned that memory supply will be significantly tighter in 2027 and 2028 than in 2026, signaling continued constraints on DRAM/HBM availability driven by AI demand.

A memory tutorial at hot chips 2026 highlights the growing gap between compute performance and HBM bandwidth, with practical fixes being implemented, such as Samsung's LPDDR5x achieving 3.01x token speed on llama 3.1 8B.

Memory now accounts for 63% of AI accelerator costs, up from 52% in early 2024, shifting industry priorities toward data movement optimization over process shrinks.

The memory industry is experiencing a supercycle due to high AI demand for HBM in Nvidia GPUs, with supply shortages expected to continue and memory manufacturers projected to achieve high profits despite cyclicality.

Apple's 2027 iPhone is expected to use Mobile High Bandwidth Memory (HBM) and Through-Silicon Vias (TSV) technology to enhance AI performance, allowing large language models to run locally with improved efficiency and compact design.

Micron states that High Bandwidth Memory (HBM) requires three times more wafer area compared to DDR5 memory.

SK hynix is building its first high-bandwidth memory (HBM) packaging base in Indiana with a $4 billion investment, backed by US government funding under the CHIPS Act, with operations set to start by 2028-2029.

At Hot Chips 2026, Samsung presented opportunities to utilize unused area on HBM base dies after moving to a logic node, with phases including integrating memory controllers to optimize PHY area and power.

The article analyzes how rising HBM memory costs are affecting NVIDIA's AI server pricing and gross margins, suggesting that customers may bear the increased costs for upcoming systems like Vera Rubin and Grace Blackwell.

A discussion highlights the significant power delivery problem in tHBM technology, requiring routing of thousands of Amps, and critiques Samsung's zHBM for potential melting issues due to stacking memory on hot GPUs, as revealed by HBM creator Kim Jung-ho.

The article explains how to determine if a GPU workload is compute-bound or memory-bound by analyzing operations per byte fetched from HBM, using NVIDIA's H100 as an example, and discusses how batching and prompt length affect performance.

Nvidia is reportedly testing lower-memory variants of its upcoming Rubin Ultra accelerator, including configurations with 192 GB or 256 GB and a switch from HBM4E to HBM4, due to HBM supply constraints.

Report says Samsung, SK Hynix, and Micron have sold out all 2027 DRAM and HBM capacity to AI companies, worsening the RAM shortage and driving up consumer prices for memory, SSDs, and consoles.

SK hynix and SanDisk unveiled the High Bandwidth Flash (HBF) standard to bridge the performance gap between HBM memory and SSDs, targeting up to 3TB/s bandwidth and 512GB capacity to speed AI inference.

Nvidia's DGX GB300 system, featuring 20 terabytes of HBM and 2 miles of copper in one rack, exemplifies peak industrial civilization, turning intelligence into a manufacturing problem.

SK Hynix, a major supplier of memory chips for Nvidia's AI accelerators, had a successful US IPO, opening at $170 per share and reaching a $1 trillion valuation amid surging demand for DRAM and HBM driven by AI data center buildout.

SK Hynix raised $26.5B in the largest foreign IPO in US history, driven by demand for its HBM memory chips used in AI GPUs. The US Commerce Secretary urged the company to build new fabrication plants in the US to reduce reliance on South Korean production.

Reveals that the 748GB unified memory advertised for the NVIDIA DGX Station actually only has 252GB of high-speed HBM available. The remaining 496GB of slow LPDDR5X is essentially useless for large model inference, reflecting NVIDIA's precise product differentiation strategy.

This article outlines key points of the AI supply chain in July earnings reports from multiple tech companies, including trends in memory, equipment, foundry, AI chips, and cloud services, emphasizing TSMC's central role in the AI supply chain.