@iotcoi: Ran Google’s cookbook with 10 agents on my tiny GB10 GPU. 436 tok/s / 43.6 per agent Qwen3.6-35B + Dflash + DDTree on v…
Summary
A developer ran 10 concurrent agents of the 35B-parameter Qwen3.6 model on a single 74W GB10 GPU at 436 tok/s total using vLLM, demonstrating high-efficiency edge deployment.
View Cached Full Text
Cached at: 04/22/26, 11:28 AM
Ran Google’s cookbook with 10 agents on my tiny GB10 GPU. 436 tok/s / 43.6 per agent Qwen3.6-35B + Dflash + DDTree on vLLM GB10 @ 74W The future isn’t 10,000 GPUs in a nuclear-powered data center. It’s 10 agents on your desk solving your problems while you make your coffee.
Similar Articles
@iotcoi: Qwen3.6-27B-FP8 + Dflash + DDTree, 256k context, 10 agents ~200 tokens/sec max decode 136t/s average on a single tiny G…
Quantized 27B Qwen3.6 model achieves 200 tok/s peak (136 avg) with 256k context and 10 agents on a single 49W GB10 GPU using Dflash+DDTree optimizations.
How many agents can 2×4090 actually run at once? Three weeks of llama.cpp concurrency data — soft cap 5 @ 64k, hard cap 9, and why.
A benchmarking report on running multiple AI agents concurrently using llama.cpp on 2× RTX 4090 GPUs, revealing performance limits and optimal configurations for Qwen models.
I measured whether 2 local agents hitting 1 model run in parallel or just take turns. Batching is real, but it is not free using QWEN 3.8 27B 4bit on my MacBook Pro M3Max 128 GB Unified Memory 40 Core GPU
The author experimented with two local agents running in parallel on a MacBook Pro M3Max using the QWEN 3.8 27B 4bit model, finding that batching enables concurrent execution but increases latency, with an optimal agent count around 4.
8 agents × 250K context, concurrently, on two 3090s (~600+ tok/s total)
A developer shares open-source vLLM patches and Triton kernels that run 8 coding agents with 250K-token contexts concurrently on two RTX 3090s by keeping the full int4 KV cache in system RAM and fetching only ~1% of tokens per attention step via a rank-64 key sketch, reporting ~545-790 tok/s total and 31/32 recall at 250K.
2×RTX 3090 + EPYC box running qwen3.8-flash-next at ~38 tok/s
The user describes a hardware configuration with dual RTX 3090 GPUs and an AMD EPYC CPU for running the Qwen3-Flash-Next model using llama.cpp, achieving 38 tokens per second in single-stream inference, and seeks advice on whether to add a third GPU or upgrade the CPU to improve performance, especially for running multiple parallel agents.