I measured the 3 claims Users in this Sub all handed me on the last local-agent post. One of you out-predicted my own hypothesis. Learn It All not Know It All rules

Reddit r/AI_Agents News

Summary

The user tested scaling local AI agents with a Qwen 27B model, finding that adding more agents increases throughput only up to a point due to memory bandwidth limits, with long prompts benefiting more from parallelism.

Over the past few days religously (as im suppose to be on holidays) Ive been posting here about whether local agents QWEN 3.8 27B 4 bit in particular. The comments were sharper than my post, so I did not write the follow-up alone. 3 of you here gave me claims, and I turned each into an experiment on my dev rig MacBook Pro M3 Max 128 GB Unified and 40Cores GPU. Same-instant launches so I could not secretly set the queue order. What came out of it: The second agent helps a little, the fourth does not. Aggregate throughput went 16.6 to 20.8 tokens/sec from 1 to 2 agents, then basically flatlined and drifted DOWN at 4 and 8. Meanwhile per-agent decode rate collapsed 17.4 to 12.8 to 6.8 to 3.9, and time-to-first-token climbed from 0.46s to 32s. The total is close to fixed, so every agent you add just cuts everyone's slice thinner. One of you predicted the exact shape. The call was that 1-to-2 agents lands near 1.5x, not 2x, because decode is memory-bandwidth-bound. I measured 1.57x decode-heavy and 1.51x prefill-heavy. Almost dead on. My own hypothesis, that prefill would clearly win, did not show up the way I expected, and I left that miss in the write-up. Longer prompts batch better. Sweeping prompt length from ~170 to ~3,100 tokens, the 1-to-2-agent gain climbed monotonically 1.52x, 1.58x, 1.67x, 1.73x. Prefill is compute-bound and parallelizes; decode does not. So long-context work is the best case for a second agent, short chatty turns the worst. A dense 27B is the hard case. One of my fellow Tech community mates on LinkedIn also pointed out my model re-reads every weight per token, so it is the worst case. An MoE model that activates ~3B of its params per token has more headroom on the same bus. The whole run matrix is on disk and reproducible. Im hoping that this work helps others here either thinking about or doing this and wondering how their results stack up. Paying it forward
Original Article

Similar Articles