I'm eager for a 15x speedup on my strix halo
Summary
Nvidia claims a 15x speedup in text generation using a diffusion model, generating entire blocks at once.
Similar Articles
DiffusionGemma: 4x Faster Text Generation
Google introduces DiffusionGemma, an experimental 26B MoE open model that achieves up to 4x faster text generation on GPUs using text diffusion, targeting speed-critical interactive local workflows.
2x Strix Halo speed-up with an R9700
A user shares how they achieved a 2x performance boost in AI inference by splitting a large MoE model between a Strix Halo APU and an R9700 GPU, detailing configurations and code modifications.
Towards Speed-of-Light Text Generation with Nemotron-Labs Diffusion Language Models
NVIDIA introduces Nemotron-Labs Diffusion, a family of diffusion language models that generate text in parallel and iteratively refine it, offering faster generation and the ability to revise previous tokens.
Block3D: Efficient Text-to-3D Generation via Block-Wise Diffusion
Block3D accelerates text-to-3D generation by using block-wise diffusion with confidence-guided correction to reduce inference time while preserving geometric fidelity, achieving a 5.15x speedup.
@charles_irl: dflash go brr
NVIDIA announces DFlash, an open source block diffusion model for speculative decoding that achieves up to 15x higher inference throughput on Blackwell GPUs while maintaining interactivity.