Stratum: System-Hardware Co-Design with 3D-Stackable DRAM for Efficient Moe
Summary
Introduces Stratum, a system-hardware co-design approach utilizing 3D-stackable DRAM to efficiently accelerate Mixture of Experts (MoE) models.
Similar Articles
Are the rich RAM /poor GPU people wrong here?
Discusses the trade-off between dense and Mixture-of-Experts (MoE) models for local AI, noting that high-RAM users have limited MoE options beyond Qwen 3.5 122B, and questioning if large GPU is the only viable path.
MawForge: Memory-Bounded Expert Materialization for Local Mixture-of-Experts Inference
MawForge presents a memory-bounded approach for serving large Mixture-of-Experts (MoE) language models on constrained unified-memory machines by storing the full model on disk and materializing expert tensors into a bounded cache on demand. Experiments on a MacBook Pro M5 Pro demonstrate effective serving of 34GB and 25GB models within a 24GB memory envelope, with analysis of cache size trade-offs and speculative decoding results.
Multi Tier MoE Caching
Discusses multi-tier caching strategies for MoE models to improve inference speed by keeping frequently activated experts on GPU, referencing existing implementations like PowerInfer and llama.cpp branches.
A new way to build chips: Sequentially stacking silicon to extend Moore's Law
Researchers at the University of Illinois have demonstrated a scalable method to sequentially stack high-performance silicon circuits, achieving monolithic 3D integration within strict thermal budgets, which could extend Moore's Law beyond traditional transistor shrinking.
Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation
This paper proposes UniF-MoE, a unified framework for token-adaptive Mixture-of-Experts computation that first shares reusable computation across experts and then routes the remaining residual demand, improving performance while reducing activated computation, latency, and memory on DomainBed and GLUE benchmarks.