@iotcoi: Ran Google’s cookbook with 10 agents on my tiny GB10 GPU. 436 tok/s / 43.6 per agent Qwen3.6-35B + Dflash + DDTree on v…
Summary
A developer ran 10 concurrent agents of the 35B-parameter Qwen3.6 model on a single 74W GB10 GPU at 436 tok/s total using vLLM, demonstrating high-efficiency edge deployment.
View Cached Full Text
Cached at: 04/22/26, 11:28 AM
Ran Google’s cookbook with 10 agents on my tiny GB10 GPU. 436 tok/s / 43.6 per agent Qwen3.6-35B + Dflash + DDTree on vLLM GB10 @ 74W The future isn’t 10,000 GPUs in a nuclear-powered data center. It’s 10 agents on your desk solving your problems while you make your coffee.
Similar Articles
@iotcoi: Qwen3.6-27B-FP8 + Dflash + DDTree, 256k context, 10 agents ~200 tokens/sec max decode 136t/s average on a single tiny G…
Quantized 27B Qwen3.6 model achieves 200 tok/s peak (136 avg) with 256k context and 10 agents on a single 49W GB10 GPU using Dflash+DDTree optimizations.
I measured whether 2 local agents hitting 1 model run in parallel or just take turns. Batching is real, but it is not free using QWEN 3.8 27B 4bit on my MacBook Pro M3Max 128 GB Unified Memory 40 Core GPU
The author experimented with two local agents running in parallel on a MacBook Pro M3Max using the QWEN 3.8 27B 4bit model, finding that batching enables concurrent execution but increases latency, with an optimal agent count around 4.
EXPERIMENT: Qwen3.8-2.4T-A95B running locally on an RTX 5090 + RTX 5060 Ti at ~0.80 tok/s
An experiment running the Qwen3.8-2.4T-A95B MoE model locally on dual consumer GPUs (RTX 5090 + 5060 Ti) with llama.cpp, achieving ~0.8 tok/s with MTP speculative decoding enabled.
If you use Open Code or other agenting programs you are leaving a lot of t/s if you don't actually use agents in parallel. Benchmark : RTX5090, Qwen3.6 35B loaded via LM studio with parallel tasks set to 8
Benchmark shows that running 4-5 parallel agents with LM Studio on RTX 5090 maximizes throughput, while more agents yield diminishing returns due to VRAM and compute splitting.
@mr_r0b0t: 16 local AI agents streaming at once! MiniMax M2.7 NVFP4 — 2x GB10, no cloud APIs.
A demonstration shows 16 local AI agents streaming simultaneously using MiniMax M2.7 NVFP4 on two Nvidia GB10 chips, with no cloud APIs required.