audio.cpp release 0.6 introduces new audio models including dots.tts and MiniMax-H3, along with a native WebUI and performance improvements for text-to-audio and music generation.
Hi all :) audio.cpp release 0.6 has been out for a little while, so this is more of an update on what landed and what has been improving around it. 0.6 added 5 new model families: dots.tts, NeuTTS-2e, MuScriptor (Music to MIDI), MiniMax-H3, and SenseVoice-Small, bringing audio.cpp to 49 total model families and 70+ model variants. Since release 0.6, our contributors have added Irodori-TTS v4, IndexTTS 2.5, and ACE-Step 1.5 XL, and MiniMax-Music3. The highlights are: (1) Native WebUI! Many thanks to https://github.com/mirek190 (2) Of course, MiniMax-H3 and MiniMax-Music3. We implemented MiniMax-H3’s text-to-audio pipeline, and one fun use case is TTS/Voice clone/Music gen. It’s more flexible and powerful than dedicated audio models, and the performance is quite decent. Check out the multi-speaker conversation demo in the main post, along with the other demos in the comments. What I’m very excited about with the MiniMax-H3 implementation is that it significantly enriches the framework’s building blocks for DiT models. Now with you don’t need to go through the pain of setting up SageAttention, First Block Cache, or Spectrum manually. Just change a few parameters, and you can experiment with the model. A preliminary inspection of configuration, memory, and performance trade-offs is available in repo's docs/reports/minimax_h3_performance.md Bonus: audio.cpp’s MiniMax-H3 implementation can also produce video frames, because the DiT generates audio and video latents together, and the video VAE path is relatively straightforward to support. For now, the output is saved as RGB frame data plus metadata in JSON, so you need to encode it into a video file yourself. No upscaler or post-processing support. Just for fun. MiniMax-Music3 is currently in preview (preview/minimax-music-3 branch) . CUDA/Vulkan/HIP were tested. Still room for optimization. VRAM usage and RTF depend on audio duration and prompt length.. The demo uses the official demo prompt (4000+ char caption and 1200 char lyrics) and 30 steps plus CFG. Under this setting VRAM is ~11 GB for 30s, 14 GB for 60s, and 17 GB for 180s. It's easy to get faster-than-real-time performance and much lower VRAM usage if you tune the setting. (3) The source code for the IOS-friendly VibeVoice 1.5B is out on the vibevoice-optimizations branch in case you missed it. If you want early access to models in the next release, keep an eye on the dev branch. It will usually be updated with the new release content a few days before the release for testing. Feedback and PRs on the native UI, MiniMax-Music3, and all other aspects of audio.cpp are always welcome!
mlx-audio v0.4.3 releases with 6 new TTS models including Higgs Audio v2 and OmniVoice (646+ languages), plus server improvements like concurrent requests and continuous batching, ~3x faster Voxtral Realtime on 4-bit, and slimmer dependencies for Apple Silicon.
MiniMax releases Music 3, a high-performance music generation model that creates complete songs up to five minutes long using lyrics and detailed descriptions, with an 8B global LLM and 0.6B local LLM for long-range coherence and acoustic detail.
Release 0.4 of audio.cpp adds C++/GGML inference for Higgs Audio v3 TTS 4B (10x real-time) and Fish Audio S2 Pro, with full GGUF loading and Q8 speed/VRAM gains.
audio.cpp releases a major update adding music/SFX generation and source separation with ACE-Step, HeartMuLa, Stable Audio 3, and HTDemucs, achieving up to 10x real-time speed for long music generation in native C++/GGML.
Release of C++/GGML based implementations of Supertonic 3, MOSS-TTS, IndexTTS2, and Irodori-TTS in audio.cpp, capable of generating 10 hours of audio in 3 minutes on an RTX 5090.