Dual dgx spark (Asus GX10) MiniMax M2.7 results

hi all I have dual 3090 and 8 x mi50 32gb and I was tired of heat and loudness of these machines. So inspired by [this post](https://www.reddit.com/r/LocalLLaMA/comments/1sli7xr/2x_asus_ascent_gx10_minimax_m27_awq_cloud/) and others on nvidia forum I've purchased dual Asus GX10 (dgx spark) and I'm so happy. Each GX10 consumes about 100W during inference. Time to first token is quite high but for me it's a win Without a hassle I can run [https://huggingface.co/cyankiwi/MiniMax-M2.7-AWQ-4bit/](https://huggingface.co/cyankiwi/MiniMax-M2.7-AWQ-4bit/) I've used open code and hermes agent, no errors, just going - I love it! Here are my results using llama benchy --depth 0 4096 8192 16384 32768 --latency-mode generation: | test | t/s | peak t/s | ttfr (ms) | est_ppt (ms) | e2e_ttft (ms) | |----------------:|----------------:|-------------:|------------------:|------------------:|------------------:| | pp2048 | 3452.05 ± 73.32 | | 626.82 ± 19.83 | 511.74 ± 19.83 | 626.84 ± 19.83 | | tg32 | 38.84 ± 0.01 | 40.09 ± 0.01 | | | | | pp2048 @ d4096 | 2848.85 ± 35.82 | | 2022.61 ± 28.98 | 1907.54 ± 28.98 | 2022.65 ± 28.98 | | tg32 @ d4096 | 37.37 ± 0.23 | 38.57 ± 0.24 | | | | | pp2048 @ d8192 | 2579.85 ± 18.26 | | 3523.69 ± 61.33 | 3408.62 ± 61.33 | 3523.73 ± 61.33 | | tg32 @ d8192 | 36.27 ± 0.14 | 37.44 ± 0.15 | | | | | pp2048 @ d16384 | 2411.34 ± 7.68 | | 6791.62 ± 57.14 | 6676.55 ± 57.14 | 6791.66 ± 57.14 | | tg32 @ d16384 | 34.12 ± 0.11 | 35.23 ± 0.12 | | | | | pp2048 @ d32768 | 1988.05 ± 12.95 | | 15512.61 ± 147.98 | 15397.54 ± 147.98 | 15512.65 ± 147.98 | | tg32 @ d32768 | 30.72 ± 0.08 | 31.00 ± 0.00 | | | | | pp2048 @ d102400 | 1167.98 ± 9.19 | | 78208.55 ± 573.73 | 78118.97 ± 573.73 | 78208.59 ± 573.73 | | tg32 @ d102400 | 21.63 ± 0.07 | 23.00 ± 0.00 | | | | I start to consider selling my mi50 ;) Edit: info about llama benchy, added 100k depth