From RTX to Spark: NVIDIA Accelerates Gemma 4 for Local Agentic AI

NVIDIA Blog 04/02/26, 04:15 PM News

nvidia gemma-4 local-ai edge-computing gpu-optimization agentic-ai open-models

Summary

NVIDIA and Google collaborate to optimize Gemma 4 models for local deployment across RTX GPUs, DGX Spark, and Jetson devices, enabling efficient on-device agentic AI with support for reasoning, coding, multimodal capabilities, and 35+ languages.

<div id="bsf_rt_marker"></div>Open models are driving a new wave of on-device AI, extending innovation beyond the cloud to everyday devices. As these models advance, their value increasingly depends on access to local, real-time context that can turn meaningful insights into action. Designed for this shift, Google’s latest additions to the Gemma 4 family introduce a class of small, fast and omni-capable models built for efficient local execution across a wide range of devices. Google and NVIDIA have collaborated to optimize Gemma 4 for NVIDIA GPUs, enabling efficient performance across a range of systems — from data center deployments to NVIDIA RTX-powered PCs and workstations, the <a target="_blank" href="https://www.nvidia.com/en-us/products/workstations/dgx-spark/">NVIDIA DGX Spark</a> personal AI supercomputer and <a target="_blank" href="https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-nano/product-development/">NVIDIA Jetson Orin Nano</a> edge AI modules. <h2>Gemma 4: Compact Models Optimized for NVIDIA GPUs </h2> The latest additions to the Gemma 4 family of open models— spanning E2B, E4B, 26B and 31B variants — are designed for efficient deployment from edge devices to high-performance GPUs. <figure id="attachment_92036" aria-describedby="caption-attachment-92036" style="width: 1149px" class="wp-caption aligncenter"><a href="https://blogs.nvidia.com/wp-content/uploads/2026/04/gemma-4-perf-chart-desktop-light-1.png"><img loading="lazy" decoding="async" class="size-full wp-image-92036" src="https://blogs.nvidia.com/wp-content/uploads/2026/04/gemma-4-perf-chart-desktop-light-1.png" alt="" width="1149" height="489" srcset="https://blogs.nvidia.com/wp-content/uploads/2026/04/gemma-4-perf-chart-desktop-light-1.png 1149w, https://blogs.nvidia.com/wp-content/uploads/2026/04/gemma-4-perf-chart-desktop-light-1-960x409.png 960w, https://blogs.nvidia.com/wp-content/uploads/2026/04/gemma-4-perf-chart-desktop-light-1-630x268.png 630w" sizes="auto, (max-width: 1149px) 100vw, 1149px" /></a><figcaption id="caption-attachment-92036" class="wp-caption-text">All configurations measured using Q4_K_M quantizations BS = 1, ISL = 4096 and OSL = 128 on NVIDIA GeForce RTX 5090 and Mac M3 Ultra desktops. Token generation throughput measured on llama.cpp b7789, using the llama-bench tool.</figcaption></figure> This new generation of compact models supports a range of tasks, including: <ul> <li>Reasoning: Strong performance on complex problem-solving tasks. </li> <li>Coding: Code generation and debugging for developer workflows. </li> <li>Agents: Native support for structured tool use (function calling). </li> <li>Vision, Video and Audio Capabilities: Enables rich multimodal interactions for object recognition, automated speech recognition, and document or video intelligence. </li> <li>Interleaved Multimodal Input: Mix text and images in any order within a single prompt. </li> <li>Multilingual: Out-of-the-box support for 35+ languages, pretrained on 140+ languages. </li> </ul> The E2B and E4B models are built for ultraefficient, low-latency inference at the edge, running completely offline with near-zero latency across many devices including Jetson Nano modules. The 26B and 31B modelsare designed for high-performance reasoning and developer-centric workflows, making them well suited for agentic AI. Optimized to deliver state-of-the-art, accessible reasoning, these models run efficiently on NVIDIA RTX GPUs and DGX Spark — powering development environments, coding assistants and agent-driven workflows. As local agentic AI continues to gain momentum, applications like OpenClaw are enabling always-on AI assistants on RTX PCs, workstations and DGX Spark. The latest Gemma 4 models are compatible with OpenClaw, allowing users to build capable local agents that draw context from personal files, applications and workflows to automate tasks. Learn how to run <a target="_blank" href="https://www.nvidia.com/en-us/geforce/news/open-claw-rtx-gpu-dgx-spark-guide/">OpenClaw for free on RTX GPUs and DGX Spark</a> or using the <a target="_blank" href="https://build.nvidia.com/spark/openclaw">DGX Spark OpenClaw playbook</a>. Check out the <a target="_blank" href="https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/">Google DeepMind announcement blog</a> to learn more about the latest additions to Gemma 4 family. <h2>Getting Started: Gemma 4 on RTX GPUs and DGX Spark </h2> NVIDIA has collaborated with Ollama and llama.cpp to provide the best local deployment experience for each of the Gemma 4 models. To use Gemma 4 locally, users can download Ollama to run Gemma 4 models or install llama.cpp and pair it with the Gemma 4 GGUF Hugging Face checkpoint. Additionally, Unsloth provides day-one support with optimized and quantized models for efficient local fine-tuning and deployment via Unsloth Studio. Start running and fine-tuning Gemma 4 in Unsloth Studio today. Running open models like the Gemma 4 family on NVIDIA GPUs achieves optimal performance because NVIDIA Tensor Cores accelerate AI inference workloads to deliver higher throughput and lower latency for local execution. Plus, the CUDA software stack ensures broad compatibility across leading frameworks and tools, enabling new models to run efficiently from day one. This combination allows open models like Gemma 4 to scale across a wide range of systems — from Jetson Orin Nano at the edge to RTX PCs, workstations and DGX Spark — without requiring extensive optimization. Check out the <a target="_blank" href="https://developer.nvidia.com/blog/bringing-ai-closer-to-the-edge-and-on-device-with-gemma-4/">NVIDIA technical blog</a> for more details on how to get started with Gemma 4 on NVIDIA GPUs and learn more about NVIDIA’s work on <a href="https://blogs.nvidia.com/blog/ai-future-open-and-proprietary/">open models</a>. <h2>#ICYMI: The Latest Updates for RTX AI PCs </h2> <img src="https://s.w.org/images/core/emoji/17.0.2/72x72/2728.png" alt="✨" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Catch up on <a href="https://blogs.nvidia.com/blog/rtx-ai-garage-gtc-2026-nemoclaw">RTX AI Garage</a> blogs for a host of agentic AI announcements from NVIDIA GTC, such as new open models for local agents. These models include NVIDIA Nemotron 3 Nano 4B and Nemotron 3 Super 120B, and optimizations for Qwen 3.5 and Mistral Small 4. NVIDIA recently introduced <a target="_blank" href="https://nvidianews.nvidia.com/news/nvidia-announces-nemoclaw">NVIDIA NemoClaw,</a> an open source stack that optimizes OpenClaw experiences on NVIDIA devices by increasing security and supporting local models. <img src="https://s.w.org/images/core/emoji/17.0.2/72x72/1f680.png" alt="🚀" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <a target="_blank" href="https://accomplish.ai/">Accomplish.ai</a> announced Accomplish FREE, a no-cost version of its open source desktop AI agent with built-in models. It harnesses NVIDIA GPUs to run open weight models locally, while a hybrid router dynamically balances workloads between local RTX hardware and the cloud — enabling fast, private, zero-configuration execution without requiring an application programming interface key. Plug in to NVIDIA AI PC on <a target="_blank" href="https://www.facebook.com/NVIDIA.AI.PC/">Facebook</a>, <a target="_blank" href="https://www.instagram.com/nvidia.ai.pc/">Instagram</a>, <a target="_blank" href="https://www.tiktok.com/@nvidia_ai_pc">TikTok</a> and <a target="_blank" href="https://x.com/NVIDIA_AI_PC">X</a> — and stay informed by subscribing to the <a target="_blank" href="https://www.nvidia.com/en-us/ai-on-rtx/?modal=subscribe-ai">RTX AI PC newsletter</a>. Follow NVIDIA Workstation on <a target="_blank" href="https://www.linkedin.com/showcase/3761136/">LinkedIn</a> and <a target="_blank" href="https://x.com/NVIDIAworkstatn">X</a>.

Original Article

View Cached Full Text

Cached at: 04/21/26, 01:12 AM

# From RTX to Spark: NVIDIA Accelerates Gemma 4 for Local Agentic AI Source: [https://blogs.nvidia.com/blog/rtx-ai-garage-open-models-google-gemma-4/](https://blogs.nvidia.com/blog/rtx-ai-garage-open-models-google-gemma-4/) Open models are driving a new wave of on\-device AI, extending innovation beyond the cloud to everyday devices\. As these models advance, their value increasingly depends on access to local, real\-time context that can turn meaningful insights into action\. Designed for this shift, Google’s latest additions to theGemma 4 familyintroduce a class of small, fast and omni\-capable models built for efficient local execution across a wide range of devices\. Googleand NVIDIA have collaborated to optimize Gemma 4for NVIDIA GPUs, enabling efficient performance across a range of systems — from data center deployments to NVIDIA RTX\-powered PCs and workstations, the[NVIDIA DGX Spark](https://www.nvidia.com/en-us/products/workstations/dgx-spark/)personal AI supercomputer and[NVIDIA Jetson Orin Nano](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-nano/product-development/)edge AI modules\. ## **Gemma 4: Compact Models Optimized for NVIDIA GPUs** The latest additions to theGemma 4 family of open models—spanning E2B, E4B, 26B and 31B variants—are designed for efficient deployment from edge devices to high\-performance GPUs\. [![](https://blogs.nvidia.com/wp-content/uploads/2026/04/gemma-4-perf-chart-desktop-light-1.png)](https://blogs.nvidia.com/wp-content/uploads/2026/04/gemma-4-perf-chart-desktop-light-1.png)All configurations measured using Q4\_K\_M quantizations BS = 1, ISL = 4096 and OSL = 128 on NVIDIA GeForce RTX 5090 and Mac M3 Ultra desktops\. Token generation throughput measured on llama\.cpp b7789, using the llama\-bench tool\.This new generation of compact models supports a range of tasks, including: - **Reasoning:**Strong performance on complex problem\-solving tasks\. - **Coding:**Code generation and debugging for developer workflows\. - **Agents:**Native support for structured tool use \(function calling\)\. - **Vision, Video and Audio Capabilities:**Enables rich multimodal interactions for object recognition, automated speech recognition, and document or video intelligence\. - **Interleaved Multimodal Input:**Mix text and images in any order within a single prompt\. - **Multilingual:**Out\-of\-the\-box support for 35\+ languages, pretrained on 140\+ languages\. TheE2B and E4B modelsare built for ultraefficient, low\-latency inference at the edge, running completely offline with near\-zero latency across many devices including Jetson Nano modules\. The26B and 31B modelsare designed for high\-performance reasoning and developer\-centric workflows, making them well suited for agentic AI\. Optimized to deliver state\-of\-the\-art, accessible reasoning, these models run efficiently on NVIDIA RTX GPUs and DGX Spark — powering development environments, coding assistants and agent\-driven workflows\. As local agentic AI continues to gain momentum, applications likeOpenClaware enabling always\-on AI assistants on RTX PCs, workstations and DGX Spark\. The latest Gemma 4 models are compatible with OpenClaw, allowing users to build capable local agents that draw context from personal files, applications and workflows to automate tasks\. Learn how to run[OpenClaw for free on RTX GPUs and DGX Spark](https://www.nvidia.com/en-us/geforce/news/open-claw-rtx-gpu-dgx-spark-guide/)or using the[DGX Spark OpenClaw playbook](https://build.nvidia.com/spark/openclaw)\. ## **Getting Started: Gemma 4 on RTX GPUs and DGX Spark** NVIDIA has collaborated with Ollama and llama\.cpp to provide the best local deployment experience for each of the Gemma 4 models\. To use Gemma 4 locally, users candownload Ollamato run Gemma 4 modelsorinstallllama\.cppand pair it with the Gemma 4 GGUF Hugging Face checkpoint\.Additionally,Unsloth provides day\-one support with optimized and quantized models for efficient local fine\-tuning and deployment via Unsloth Studio\. Startrunning andfine\-tuningGemma 4 in Unsloth Studio today\. Running open models like the Gemma 4 family on NVIDIA GPUs achieves optimal performance because NVIDIA Tensor Cores accelerate AI inference workloads to deliver higher throughput and lower latency for local execution\. Plus, the CUDA software stack ensures broad compatibility across leading frameworks and tools, enabling new models to run efficiently from day one\. This combination allows open models like Gemma 4 to scale across a wide range of systems — from Jetson Orin Nano at the edge to RTX PCs, workstations and DGX Spark — without requiring extensive optimization\. Check outthe[NVIDIA technical blog](https://developer.nvidia.com/blog/bringing-ai-closer-to-the-edge-and-on-device-with-gemma-4/)for more details on how to get started with Gemma 4 on NVIDIA GPUs and learn more aboutNVIDIA’s work on[open models](https://blogs.nvidia.com/blog/ai-future-open-and-proprietary/)\. ## **\#ICYMI: The Latest Updates for RTX AI PCs** ✨ Catch up on[RTX AI Garage](https://blogs.nvidia.com/blog/rtx-ai-garage-gtc-2026-nemoclaw)blogs for a host of agentic AI announcements from NVIDIA GTC, such as new open models for local agents\. These models include NVIDIA Nemotron 3 Nano 4B and Nemotron 3 Super 120B, and optimizations for Qwen 3\.5 and Mistral Small 4\. NVIDIA recently introduced[NVIDIA NemoClaw,](https://nvidianews.nvidia.com/news/nvidia-announces-nemoclaw)an open source stack that optimizes OpenClaw experiences on NVIDIA devices by increasing security and supporting local models\. **🚀**[Accomplish\.ai](https://accomplish.ai/)announced Accomplish FREE, a no\-cost version of its open source desktop AI agent with built\-in models\. It harnesses NVIDIA GPUs to run open weight models locally, while a hybrid router dynamically balances workloads between local RTX hardware and the cloud — enabling fast, private, zero\-configuration execution without requiring an application programming interface key\. *Plug in to NVIDIA AI PC on*[*Facebook*](https://www.facebook.com/NVIDIA.AI.PC/)*,*[*Instagram*](https://www.instagram.com/nvidia.ai.pc/)*,*[*TikTok*](https://www.tiktok.com/@nvidia_ai_pc)*and*[*X*](https://x.com/NVIDIA_AI_PC)*— and stay informed by subscribing to the*[*RTX AI PC newsletter*](https://www.nvidia.com/en-us/ai-on-rtx/?modal=subscribe-ai)*\.* *Follow NVIDIA Workstation on*[*LinkedIn*](https://www.linkedin.com/showcase/3761136/)*and*[*X*](https://x.com/NVIDIAworkstatn)*\.*

From RTX to Spark: NVIDIA Accelerates Gemma 4 for Local Agentic AI

Similar Articles

NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI

NVIDIA Levels Up Local AI Agents Across RTX PCs and DGX Spark

Bringing Gemma 4 12B to your Laptop: Unlocking Local, Agentic Workflows with Google AI Edge

@JulianGoldieSEO: Google just made local AI 3x faster for FREE. Gemma 4 now runs fast enough on normal laptops that local AI finally feel…

Google announces Gemma 4 optimized for the Pixel 10's TPU (2 minute read)

Submit Feedback

Similar Articles

NVIDIA Accelerates Google DeepMind’s DiffusionGemma for Local AI

NVIDIA Levels Up Local AI Agents Across RTX PCs and DGX Spark

Bringing Gemma 4 12B to your Laptop: Unlocking Local, Agentic Workflows with Google AI Edge

@JulianGoldieSEO: Google just made local AI 3x faster for FREE. Gemma 4 now runs fast enough on normal laptops that local AI finally feel…

Google announces Gemma 4 optimized for the Pixel 10's TPU (2 minute read)