Detailed findings on PCIe bifurcation and P2P performance issues with 4x GPU setups, including workarounds and alternatives for tensor and pipeline parallelism.
I dumped the last week deep diving this and I’m I’ve been using Linux for 14 years and am a cloud systems engineer with a focus on supported Linux infrastructure for a private cloud provider. Essentially, if you are using a single 4x4 bifurcation pcie x16 card inserted into your x16 slot on your mobo and you have 4x gpus connected to it. Regardless of pcie generation that card that does the bifurcation is the choke point for p2p communication. It acts as the pcie bridge that connects the gpus and with TP=4 the bandwidth of that fabric that connects the 4 cards on that pci E bridge will become saturated and yield worse performance than with p2p off. The ways to deal with this would be to either: Don’t run p2p. It’s only a 10 to 15% gain and may not justify the cost and effort of having a setup where p2p gets you that 10% performance. Pick up a Chinese slimsas bifurcation bridge. Supposedly you might not encounter it with those. They run between 150 to 250 Buy a 1200 gen 4 pcie bridge from Cpayne. These devices are specifically made for this use case. But 1200 expense for 10% performance gain probably isn’t worth it Don’t use tensor parallelism. Use pipeline parallelism. The downside with this is pipeline parallelism in my benchmarks yielded worse performance at low concurrency than TP=4 + P2P off. PP=4 only yields better performance if you have significant enough concurrency where all the gpus have something they can be working on where none of them are waiting on another GPU to finish their work There are used PLX switches on eBay. But with these you run a risk of them not supporting a multi GPU setup with P2P due to firmware restrictions that limit non storage devices being used with them. Have a motherboard and cpu combo that provides a dedicated x16 lanes to both the primary and secondary x16 slot. You could have both of these with 8i bifurcation with 2 gpus on each. But if that setup requires a retimer to get gen4 or gen 5 then you are talking 130+ for each of these two retimer bifurcation cards. If there is a solution to this that I didn’t list, please let me know and I’ll update this post.
Benchmark results of the unsloth/Qwen3.6-27B-NVFP4 model running on 4x RTX 5060 Ti GPUs with peer-to-peer and pipeline parallelism at various concurrency levels.
A user discovered that a hidden PCIe 2.0 x4 electrical limitation on a Threadripper workstation board was crippling one of four RTX 3090s, causing poor multi-GPU LLM inference performance. Fixing the slot layout and switching to tensor split mode doubled Mistral 128B throughput from ~11 to ~24.7 tok/s.
Testing reveals that Intel consumer platforms like Z890 with Arrow Lake CPUs have hardware/firmware limitations that prevent proper PCIe Peer-to-Peer (P2P) communication between multiple GPUs, making them unsuitable for multi-GPU AI workloads despite adequate lane counts.
Enabling PCI-E peer-to-peer (P2P) for consumer Nvidia GPUs with patched drivers and vLLM environment variables yields roughly 25% prefill throughput improvement for free, as demonstrated by benchmarks.