[OSS] dlmserve - first serving engine for diffusion language models
Summary
dlmserve is the first open-source serving engine for diffusion language models, providing an OpenAI-compatible API, continuous batching, and 2.5x throughput over Hugging Face, all within 12GB VRAM.
Similar Articles
BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
BlockServe introduces block-grained continuous batching to address convergence heterogeneity in diffusion LLMs, enabling 1.9–10.6x throughput improvement over Fast-dLLM while maintaining generation quality.
@vllm_project: Congrats to @GoogleDeepMind on DiffusionGemma A 26B diffusion language model on the Gemma4 backbone, and the first dLLM…
vLLM announces native support for Google DeepMind's DiffusionGemma, a 26B discrete diffusion language model that generates 256-token blocks in parallel, enabling low-latency inference at 1200+ tok/s on a single H200.
@kazukifujii: The UC San Diego Hao AI Lab blog provides a very clear explanation of the usefulness of DistServe's Prefill Decode Disa…
The UC San Diego Hao AI Lab blog provides a clear explanation of DistServe's Prefill Decode Disaggregation, tracing its acceptance from 2024 to 2025 and linking to related technologies like NVIDIA Dynamo, llm-d, Ray Serve LLM, LMCache, and MoonCake, making it a great starting point for learning LLM inference.
@_avichawla: https://x.com/_avichawla/status/2077653695123378321
This article argues that vLLM and similar serving frameworks are inefficient for running multiple small AI models on a single GPU due to design limitations. It introduces the SIE open-source inference engine as a solution for serving many models together to reduce costs.
Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM
This paper proposes Dynamic-dLLM, a training-free framework that accelerates diffusion large language models by dynamically allocating cache-update budgets and calibrating decoding thresholds, achieving over 3x speedup on models like LLaDA and Dream while maintaining performance.