Tag
The article provides a detailed analysis of the tray design in the NV72 architecture, where every 4 GPUs are paired with 2 Grace CPUs. It discusses optimizing inference performance under coding workloads using parallel strategies such as EP64 and TP8, as well as the complexity of KV cache management and routing algorithms.
Fine-tuned LiquidAI's LFM2.5-230M model on Fable-5 coding traces, finding the results better than expected.