Enabling PCI-E peer-to-peer (P2P) for consumer Nvidia GPUs with patched drivers and vLLM environment variables yields roughly 25% prefill throughput improvement for free, as demonstrated by benchmarks.
Detailed findings on PCIe bifurcation and P2P performance issues with 4x GPU setups, including workarounds and alternatives for tensor and pipeline parallelism.
Testing reveals that Intel consumer platforms like Z890 with Arrow Lake CPUs have hardware/firmware limitations that prevent proper PCIe Peer-to-Peer (P2P) communication between multiple GPUs, making them unsuitable for multi-GPU AI workloads despite adequate lane counts.
A user shares a configuration of 4x RTX 5060 Ti 16GB with P2P to run Qwen3.6-27B-FP8 at 55 tok/s with 262K context, highlighting the low cost of about $1800 for single-user inference.
Llama.cpp now supports Nvidia's Programmatic Dependent Launch (PDL) for Blackwell GPUs, offering a 5-10% performance boost on token generation. The feature is not enabled by default and requires a build flag.
Demonstrates achieving 20GB VRAM and 448GB/s bandwidth for around $100 using two NVIDIA P102-100 cards, running a llama.cpp server with a Qwen model and supporting 3 concurrent users with large context.