[OSS] dlmserve - first serving engine for diffusion language models
Summary
dlmserve is the first open-source serving engine for diffusion language models, providing an OpenAI-compatible API, continuous batching, and 2.5x throughput over Hugging Face, all within 12GB VRAM.
Similar Articles
BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
BlockServe introduces block-grained continuous batching to address convergence heterogeneity in diffusion LLMs, enabling 1.9–10.6x throughput improvement over Fast-dLLM while maintaining generation quality.
@vllm_project: Congrats to @GoogleDeepMind on DiffusionGemma A 26B diffusion language model on the Gemma4 backbone, and the first dLLM…
vLLM announces native support for Google DeepMind's DiffusionGemma, a 26B discrete diffusion language model that generates 256-token blocks in parallel, enabling low-latency inference at 1200+ tok/s on a single H200.
@kazukifujii: The UC San Diego Hao AI Lab blog provides a very clear explanation of the usefulness of DistServe's Prefill Decode Disa…
The UC San Diego Hao AI Lab blog provides a clear explanation of DistServe's Prefill Decode Disaggregation, tracing its acceptance from 2024 to 2025 and linking to related technologies like NVIDIA Dynamo, llm-d, Ray Serve LLM, LMCache, and MoonCake, making it a great starting point for learning LLM inference.
Serving Masked Diffusion LLMs: Characterization and Design Principles from Real Hardware
This paper characterizes the serving behavior of masked diffusion language models (dLLMs) using real hardware measurements, identifying key differences from autoregressive models and deriving design principles for efficient inference systems.
@_avichawla: https://x.com/_avichawla/status/2077653695123378321
This article argues that vLLM and similar serving frameworks are inefficient for running multiple small AI models on a single GPU due to design limitations. It introduces the SIE open-source inference engine as a solution for serving many models together to reduce costs.