NVIDIA Blackwell Leads on First Agentic AI Infrastructure Benchmark

NVIDIA Blog 06/12/26, 09:00 PM News

nvidia blackwell agentic-ai benchmark agentperf ai-infrastructure

Summary

NVIDIA's Blackwell GB300 NVL72 platform leads the first agentic AI infrastructure benchmark, AgentPerf from Artificial Analysis, delivering up to 20x more agents per megawatt than the previous Hopper generation.

<div id="bsf_rt_marker"></div>AgentPerf from Artificial Analysis, the industry’s first agentic AI benchmark, gives developers, enterprises and infrastructure providers a clear way to compare systems for agentic AI. In the first round of published results, the <a target="_blank" href="https://www.nvidia.com/en-us/data-center/technologies/blackwell-architecture/">NVIDIA Blackwell Ultra NVL72</a> platform delivers leading performance across the agentic AI workloads tested, running 20x more agents per megawatt than NVIDIA Hopper. Agentic AI is a fundamentally different workload than conversational AI. A single chat completion is a sprint: one large language model (LLM) call, one response. An agent functions more like a relay: It breaks a goal into many steps and keeps going until the task is done. <figure id="attachment_94457" aria-describedby="caption-attachment-94457" style="width: 16339px" class="wp-caption alignnone"><img fetchpriority="high" decoding="async" class="wp-image-94457 size-full" src="https://blogs.nvidia.com/wp-content/uploads/2026/06/Agentic-Pipeline_v1-2.png" alt="" width="16339" height="8097" /><figcaption id="caption-attachment-94457" class="wp-caption-text">Agents chain together multiple LLM calls and tool calls to gather context, observe, reason and act.</figcaption></figure> That results in dozens to hundreds of LLM calls chained together, each passing growing context to the next, with tool calls like code compile and execution, database search and web browsing at every handoff. The complexity isn’t additive; it’s multiplicative. The distinction matters enormously for performance measurement. Existing AI inference benchmarks measure one LLM call: how fast an LLM responds to a single request and how many simultaneous requests a system can handle. They weren’t designed for agentic workloads, where chained LLM calls, tool call delays and growing context stress accelerated computing systems in fundamentally different ways than a single LLM call ever could. For companies building and deploying agents at scale, it’s important to understand how responsive agents are, how many can be deployed simultaneously and how much useful work AI infrastructure can deliver for every dollar and watt invested. <h2>NVIDIA GB300 NVL72 Runs 20x More Agents per Megawatt</h2> In this first round, AgentPerf measures agentic performance with <a target="_blank" href="https://artificialanalysis.ai/models/deepseek-v4-pro/providers">DeepSeek V4 Pro</a>, a large mixture-of-experts (MoE) model that represents the class of frontier models powering today’s most capable agents. On this workload, NVIDIA GB300 NVL72 delivers the highest performance in the benchmark, running up to 20x more agents per megawatt than the NVIDIA HGX H200 system. <figure id="attachment_94444" aria-describedby="caption-attachment-94444" style="width: 1996px" class="wp-caption alignnone"><img decoding="async" class="wp-image-94444 size-full" src="https://blogs.nvidia.com/wp-content/uploads/2026/06/agentperf-blackwell-graph-1.jpg" alt="" width="1996" height="1113" srcset="https://blogs.nvidia.com/wp-content/uploads/2026/06/agentperf-blackwell-graph-1.jpg 1996w, https://blogs.nvidia.com/wp-content/uploads/2026/06/agentperf-blackwell-graph-1-960x535.jpg 960w, https://blogs.nvidia.com/wp-content/uploads/2026/06/agentperf-blackwell-graph-1-1680x937.jpg 1680w, https://blogs.nvidia.com/wp-content/uploads/2026/06/agentperf-blackwell-graph-1-1280x714.jpg 1280w, https://blogs.nvidia.com/wp-content/uploads/2026/06/agentperf-blackwell-graph-1-1536x856.jpg 1536w, https://blogs.nvidia.com/wp-content/uploads/2026/06/agentperf-blackwell-graph-1-630x351.jpg 630w" sizes="(max-width: 1996px) 100vw, 1996px" /><figcaption id="caption-attachment-94444" class="wp-caption-text">NVIDIA GB300 NVL72 supports far more concurrent agents per megawatt than NVIDIA H200 at both service-level objectives of 20 and 60 tokens per second per agent.</figcaption></figure> The performance advantage comes from extreme codesign across the full stack. GB300 NVL72 connects 72 GPUs into a single rack-scale system, enabling large MoE models like DeepSeek V4 Pro to distribute model execution efficiently at scale. CUDA kernels accelerate this further by overlapping communication and compute, so the cost of coordinating across experts is absorbed rather than added to latency. NVIDIA TensorRT LLM sustains efficiency as concurrent agent sessions scale. For example, it separates the processing of inputs from the generation of outputs so each can be optimized independently. These results are grounded in a benchmark methodology built from the ground up to reflect how agentic AI actually works in production. <h2>Artificial Analysis AgentPerf: Built on Real-World Agentic Workloads</h2> AgentPerf is built based on real coding agent trajectories: an agent receives a task, reads files, writes and edits code, executes commands and iterates based on the results — all drawn from real public code repositories across 12+ programming languages. The long sequence lengths, tool call patterns and delays are all representative of real-world coding workflows. AgentPerf then measures how many of these agentic tasks a platform can support simultaneously while meeting defined performance thresholds for responsiveness and output token rate. Tool calls are not executed but simulated using representative CPU processing time, so differences in results reflect accelerated computing performance only. The results translate directly into infrastructure decisions: how many concurrent agentic tasks can be run per accelerator and per megawatt of power. For enterprises deploying AI agents at scale, those numbers determine how much productive work a given infrastructure investment can actually deliver. <h2>NVIDIA Ecosystem Partners Harness Blackwell’s Leading Performance</h2> Leading inference providers including Baseten, DeepInfra and Together AI are already serving agentic workloads on frontier models such as <a target="_blank" href="https://artificialanalysis.ai/models/deepseek-v4-pro/providers">DeepSeek V4 Pro</a> on NVIDIA Blackwell and powering production agentic applications today. <a target="_blank" href="https://www.together.ai/blog/learn-how-cursor-partnered-with-together-ai-to-deliver-real-time-low-latency-inference-at-scale">Together AI powers real-time inference for Cursor</a>, an AI-powered agentic coding platform, on NVIDIA Blackwell. Cursor’s agents debug issues, generate features and execute refactors while developers continue working. DeepInfra powers <a target="_blank" href="https://pam.ai">Pam.ai</a>, an AI workforce platform for car dealerships, which deploys agents to book service appointments, handle calls and run outbound sales campaigns, entirely on NVIDIA Blackwell. As NVIDIA and the open source ecosystem continue to optimize inference software, performance and efficiency on agentic workloads will only improve. The NVIDIA Vera Rubin architecture is now in full production, bringing the next generation of infrastructure capacity to meet the growing demands of agentic AI at scale. Dive deeper into AgentPerf’s methodology and NVIDIA’s full-stack optimizations for agentic AI in this <a target="_blank" href="https://developer.nvidia.com/blog/nvidia-achieves-leading-agentic-coding-performance-on-first-agentic-ai-benchmark/">technical blog</a>.

Original Article

View Cached Full Text

Cached at: 06/14/26, 12:17 AM

# NVIDIA Blackwell Leads on First Agentic AI Infrastructure Benchmark Source: [https://blogs.nvidia.com/blog/nvidia-blackwell-agentperf-artificial-analysis/](https://blogs.nvidia.com/blog/nvidia-blackwell-agentperf-artificial-analysis/) AgentPerf from Artificial Analysis, the industry’s first agentic AI benchmark, gives developers, enterprises and infrastructure providers a clear way to compare systems for agentic AI\. In the first round of published results, the[NVIDIA Blackwell Ultra NVL72](https://www.nvidia.com/en-us/data-center/technologies/blackwell-architecture/)platform delivers leading performance across the agentic AI workloads tested, running 20x more agents per megawatt than NVIDIA Hopper\. Agentic AI is a fundamentally different workload than conversational AI\. A single chat completion is a sprint: one large language model \(LLM\) call, one response\. An agent functions more like a relay: It breaks a goal into many steps and keeps going until the task is done\. ![](https://blogs.nvidia.com/wp-content/uploads/2026/06/Agentic-Pipeline_v1-2.png)Agents chain together multiple LLM calls and tool calls to gather context, observe, reason and act\.That results in dozens to hundreds of LLM calls chained together, each passing growing context to the next, with tool calls like code compile and execution, database search and web browsing at every handoff\. The complexity isn’t additive; it’s multiplicative\. The distinction matters enormously for performance measurement\. Existing AI inference benchmarks measure one LLM call: how fast an LLM responds to a single request and how many simultaneous requests a system can handle\. They weren’t designed for agentic workloads, where chained LLM calls, tool call delays and growing context stress accelerated computing systems in fundamentally different ways than a single LLM call ever could\. For companies building and deploying agents at scale, it’s important to understand how responsive agents are, how many can be deployed simultaneously and how much useful work AI infrastructure can deliver for every dollar and watt invested\. ## **NVIDIA GB300 NVL72 Runs 20x More Agents per Megawatt** In this first round, AgentPerf measures agentic performance with[DeepSeek V4 Pro](https://artificialanalysis.ai/models/deepseek-v4-pro/providers), a large mixture\-of\-experts \(MoE\) model that represents the class of frontier models powering today’s most capable agents\. On this workload, NVIDIA GB300 NVL72 delivers the highest performance in the benchmark, running up to 20x more agents per megawatt than the NVIDIA HGX H200 system\. ![](https://blogs.nvidia.com/wp-content/uploads/2026/06/agentperf-blackwell-graph-1.jpg)NVIDIA GB300 NVL72 supports far more concurrent agents per megawatt than NVIDIA H200 at both service\-level objectives of 20 and 60 tokens per second per agent\.The performance advantage comes from extreme codesign across the full stack\. GB300 NVL72 connects 72 GPUs into a single rack\-scale system, enabling large MoE models like DeepSeek V4 Pro to distribute model execution efficiently at scale\. CUDA kernels accelerate this further by overlapping communication and compute, so the cost of coordinating across experts is absorbed rather than added to latency\. NVIDIA TensorRT LLM sustains efficiency as concurrent agent sessions scale\. For example, it separates the processing of inputs from the generation of outputs so each can be optimized independently\. These results are grounded in a benchmark methodology built from the ground up to reflect how agentic AI actually works in production\. ## **Artificial Analysis AgentPerf: Built on Real\-World Agentic Workloads** AgentPerf is built based on real coding agent trajectories: an agent receives a task, reads files, writes and edits code, executes commands and iterates based on the results — all drawn from real public code repositories across 12\+ programming languages\. The long sequence lengths, tool call patterns and delays are all representative of real\-world coding workflows\. AgentPerf then measures how many of these agentic tasks a platform can support simultaneously while meeting defined performance thresholds for responsiveness and output token rate\. Tool calls are not executed but simulated using representative CPU processing time, so differences in results reflect accelerated computing performance only\. The results translate directly into infrastructure decisions: how many concurrent agentic tasks can be run per accelerator and per megawatt of power\. For enterprises deploying AI agents at scale, those numbers determine how much productive work a given infrastructure investment can actually deliver\. ## **NVIDIA Ecosystem Partners Harness Blackwell’s Leading Performance** Leading inference providers including Baseten, DeepInfra and Together AI are already serving agentic workloads on frontier models such as[DeepSeek V4 Pro](https://artificialanalysis.ai/models/deepseek-v4-pro/providers)on NVIDIA Blackwell and powering production agentic applications today\. [Together AI powers real\-time inference for Cursor](https://www.together.ai/blog/learn-how-cursor-partnered-with-together-ai-to-deliver-real-time-low-latency-inference-at-scale), an AI\-powered agentic coding platform, on NVIDIA Blackwell\. Cursor’s agents debug issues, generate features and execute refactors while developers continue working\. DeepInfra powers[Pam\.ai](https://pam.ai/), an AI workforce platform for car dealerships, which deploys agents to book service appointments, handle calls and run outbound sales campaigns, entirely on NVIDIA Blackwell\. As NVIDIA and the open source ecosystem continue to optimize inference software, performance and efficiency on agentic workloads will only improve\. The NVIDIA Vera Rubin architecture is now in full production, bringing the next generation of infrastructure capacity to meet the growing demands of agentic AI at scale\. *Dive deeper into AgentPerf’s methodology and NVIDIA’s full\-stack optimizations for agentic AI in this[technical blog](https://developer.nvidia.com/blog/nvidia-achieves-leading-agentic-coding-performance-on-first-agentic-ai-benchmark/)\.*

NVIDIA Blackwell Leads on First Agentic AI Infrastructure Benchmark

Similar Articles

@rohanpaul_ai: NVIDIA just posted the first agentic AI benchmark results where GB300 NVL72 runs up to 20x more coding agents per megaw…

Fastest, Largest, Strongest: NVIDIA Blackwell Sweeps MLPerf Training 6.0

@CNET: From Nvidia GTC 2026, CEO Jensen Huang talks about investment in AI Natives

HPE AI Factory With NVIDIA Expands for the Era of Agents

@mr_r0b0t: 16 local AI agents streaming at once! MiniMax M2.7 NVFP4 — 2x GB10, no cloud APIs.

Submit Feedback

Similar Articles

@rohanpaul_ai: NVIDIA just posted the first agentic AI benchmark results where GB300 NVL72 runs up to 20x more coding agents per megaw…

Fastest, Largest, Strongest: NVIDIA Blackwell Sweeps MLPerf Training 6.0

@CNET: From Nvidia GTC 2026, CEO Jensen Huang talks about investment in AI Natives

HPE AI Factory With NVIDIA Expands for the Era of Agents

@mr_r0b0t: 16 local AI agents streaming at once! MiniMax M2.7 NVFP4 — 2x GB10, no cloud APIs.
A demonstration shows 16 local AI agents streaming simultaneously using MiniMax M2.7 NVFP4 on two Nvidia GB10 chips, with no cloud APIs required.