P40's + MI50's + RPC on 550B Nemotron Ultra Q3_S

Hey guys, I went ahead and installed a 100Gbe NIC card on both my MI50 machine and my P40 machine and loaded Nemotron Ultra IQ3_S across both machines. I was pretty surprised on the throughput for such old hardware. Given the results - I now have my sights on purchasing the Chinese 22GB RTX 2080 Ti's to append more VRAM to the build and continue comparing/contrasting/experimenting. Mi50 Hardware: Asus X99-E-WS (Modded BIOS to support a large number GPU's ) Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz 128GB DDR4 RAM SSD 7x MI50's 112GB VRAM 2x MI50's 64GB VRAM (176 VRAM Total) P40 Hardware: Asus X99-E-WS (Modded BIOS to support a large number GPU's ) Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz 128GB DDR4 RAM (mixed batch of Non-ECC sticks) SSD 5x P40's 120GB VRAM Memory Load: MI50 Box P40 Box Benchmark Results: Context pp512 tg128 pp512+tg128 pp4096+tg128 0 54.42 6.19 21.33 52.28 8,192 53.20 6.08 20.82 50.16 32,768 47.22 5.95 19.86 45.24 65,536 41.50 5.89 18.81 40.04 126,720 34.09 5.59 16.61 33.04 Start up command: HIP_VISIBLE_DEVICES=1,0,2,3,4,5,6,7,8 \ /usr/local/bin/llama-server \ --rpc 10.10.10.2:50052 \ -m "$HOME/.lmstudio/models/unsloth/NVIDIA-Nemotron-3-Ultra-550B-A55B-GGUF/NVIDIA-Nemotron-3-Ultra-550B-A55B-UD-IQ3_S-00001-of-00007.gguf" \ -dev RPC0,RPC1,RPC2,RPC3,RPC4,ROCm0,ROCm1,ROCm2,ROCm3,ROCm4,ROCm5,ROCm6,ROCm7,ROCm8 \ -ts 1,1,1,1,1,1.3,0.65,0.65,1.3,0.65,0.65,0.65,0.65,0.65 \ -ngl 999 \ -fit off \ -sm layer \ -c 131072 \ -b 2048 \ -ub 1024 \ -fa on \ --no-mmap \ --direct-io \ -np 1 \ --host 0.0.0.0