Voice conversations between Gemma4 12B and E2B on GPU and Jetson Orin

Reddit r/LocalLLaMA News
voice-conversation gpu-inference open-source llm-engine edge-ai performance-benchmark

Summary

The article demonstrates voice conversations between Gemma 4 models on GPU and Jetson Orin hardware, using the open-source Cortexist Little Gemma engine for efficient inference with lip sync and gestures.

Gemma 4 12B runs on an RTX PRO 4500 Blackwell. Gemma 4 E2B run on a Jetson Orin NX 16GB; similar performance is expected on a Jetson Orin Nano Super 8GB. Both systems use a reSpeaker Flex 4-mic array and a 3W speaker. Inference is handled by Cortexist Little Gemma, a small LLM engine written in C for CUDA devices. On Jetson Orin it is faster than llama.cpp, and no degradation after long voice prompt. The pipeline supports lip sync, expressions, and gestures. Everything is open source. They talk to humans too. The engine source code: https://github.com/cortexist/little-gemma
Original Article

Similar Articles

Gemma 4 VLA Demo on Jetson Orin Nano Super

Hugging Face Blog

NVIDIA and Hugging Face publish a hands-on demo showing Gemma 4 running as a vision-language-action model entirely on the Jetson Orin Nano Super, using local STT/TTS and webcam input.

Hugging Face and Cerebras bring Gemma 4 to real-time voice AI

Hugging Face Blog

Hugging Face and Cerebras demonstrate a real-time speech-to-speech pipeline combining open-source models (Nvidia's Parakeet, Gemma 4, Qwen3TTS) with Cerebras' fast inference, enabling natural conversational AI and powering robots like Reachy Mini.