Llama-CPP Parallel Agents --> fine for decode, but one agent's prefill will grind all other agents to a halt

Testing with 3-5 agents. Decode performance is superb, however if one performs a web search and needs to process a few thousand tokens, ALL other agents will grind to a halt: I've tried tuning a little bit, but no luck. example command of mine (this server is ONLY used for the sub-agents): ./llama-server \ --model /models/Gemma4-26B/gemma-4-26B-A4B-it-UD-Q5_K_M.gguf \ --model-draft /models/Gemma4-26B/mtp-gemma-4-26B-A4B-it-Q8_0.gguf \ --device Vulkan0 \ --device-draft Vulkan0 \ --split-mode none \ --main-gpu 0 \ --gpu-layers all \ --spec-type draft-mtp \ --spec-draft-n-max 3 \ --ctx-size 240000 \ --parallel 3 \ --batch-size 2048 \ --ubatch-size 512 \ --flash-attn on \ --kv-unified \ --cache-reuse 256 \ --host 0.0.0.0 \ --port 8081 I'm fairly new to parallel agents. Any thoughts/suggestions on what i should be doing differently?