Infinite Music Glitch on my Arduino with Magenta Realtime 2

Reddit r/LocalLLaMA Tools

Summary

A developer built a local voice-controlled music system using an ESP32 microcontroller, a MacBook, Magenta Realtime 2 for real-time music generation, MLX Whisper for transcription, and a Qwen model for tool calling, enabling conversational control over music elements like genre and instruments.

I built a local voice AI realtime music setup where my ESP32 microcontroller talks to my MacBook over WebSockets. The microcontroller is just a tiny Arduino-based device with a mic and speaker, and the MacBook M4 Pro runs Magenta Realtime 2 locally and streams the audio back to the device. The fun part is that it’s agentic and conversational. So I can tap the ESP32, speak into it, and it uses MLX Whisper to transcribe what I said. Then after detecting VAD, it sends that to a Qwen model, which decides what tool call to make, like adding drums, making the music Lo-fi, adding Jazz bebop, removing guitar, or changing the instruments in the music. GitHub link: [https://github.com/akdeb/jambox](https://github.com/akdeb/jambox) HF link: [https://huggingface.co/google/magenta-realtime-2](https://huggingface.co/google/magenta-realtime-2)
Original Article

Similar Articles

google/magenta-realtime-2

Hugging Face Models Trending

Google DeepMind released Magenta RealTime 2, an open music generation model for on-device streaming with low-latency control via text, audio examples, and MIDI.

MiniMax Music 3 Infinite Radio

Reddit r/LocalLLaMA

An open-source, self-hosted AI music generation radio tool called TAPEDECK that uses MiniMax Music 3 to create endless, personalized radio stations from a user's music library on a local GPU.

Breaking the music supply constraint

Reddit r/LocalLLaMA

The author describes cancelling music subscriptions and building a self-hosted AI music generation system using DGX Spark, Plex, and Ace-Step 1.5 XL models, creating an infinite personalized music supply.