Stratum: System-Hardware Co-Design with 3D-Stackable DRAM for Efficient Moe
Summary
Introduces Stratum, a system-hardware co-design approach utilizing 3D-stackable DRAM to efficiently accelerate Mixture of Experts (MoE) models.
Similar Articles
Anyone sitting on a lot of slow system memory and a modest GPU.. try Strata + Qwen3.8 Next.
A user reports that Strata with Qwen3-Next 80B IQ3_XXS weights (~80GB) runs at 45-70 tokens/s on a DDR4 + 7900 XTX rig, significantly beating Llama.cpp tuned setups (~22.5 t/s), suggesting Strata is a promising inference tool for large MoE models on modest GPUs and slow system memory.
@MinLiBuilds: I just finished running a round of experiments with Strata. This is truly an awesome inference framework—VRAM is still …
A creator shares benchmark experiments with the Strata inference framework, systematically limiting VRAM budgets for the expert cache on an RTX 4090 to measure how speed degrades across different simulated GPU memory tiers, using Qwen3.8-Flash-Next Q2_0 at 8192 context and temperature 0.
D-Matrix Raptor 3D-DRAM Accelerator for Generative Inference at Hot Chips 2026
d-Matrix presented its Raptor 3D-DRAM accelerator at Hot Chips 2026, which addresses memory capacity and bandwidth challenges in generative AI inference by stacking compute directly on DRAM dies.
@yibie: Announcing my recent project: strata-mlx Strata is fantastic: it can run a 125B parameter Mixture of Experts model Qwen…
yibie announces strata-mlx, an unofficial MLX-based inference engine for Mac that runs the 125B MoE model Qwen3.8-Flash-Next from Strata's GGUF files, achieving 1.7–2.9x faster generation than llama.cpp and enabling lower-RAM Macs to offload experts to SSD.
Samsung Debuts zHBM Prototype, Stacking Memory Directly on AI Accelerators
Samsung unveiled zHBM and zNAND-O memory technologies at the FMS 2026 conference, designed to stack memory directly on AI accelerators for up to 8x higher performance and improved power efficiency in AI systems.