Tag
Modular's kernel team is optimizing serving for MiniMax M3's 1M-token context and native multimodality, with open weights dropping soon for immediate deployment on Modular.
vLLM-Omni v0.22.0 is a major upgrade adding robust support for NVIDIA Cosmos world models, production TTS (Qwen3-TTS, Qwen3-Omni, VoxCPM2), faster diffusion model serving (Wan 2.2, HunyuanVideo 1.5, LTX-2.3), and broader quantization and hardware coverage with 339 commits from 124 contributors.
The author benchmarks serving 1,000 LoRA adapters on one GPU using vLLM, finding that active adapter count and traffic shape are the real bottlenecks, and provides recommendations for tuning max_loras.
MinT is a managed infrastructure system that enables efficient training and serving of millions of LLMs by keeping base models resident and moving lightweight LoRA adapters, scaling across model architectures, storage, and policy management.