Tag
A setup guide for using a custom Docker/Podman toolbox with ROCm/RCCL RDMA support to cluster two AMD Strix Halo nodes, enabling vLLM with tensor parallelism across 256GB unified memory.
Experimental implementation of RDMA over USB4 demonstrated using Thunderbolt and Strix Halo, potentially enabling high-speed data transfer with any USB4 host.
DualPath is a system that breaks the storage bandwidth bottleneck in agentic LLM inference by introducing a dual-path KV-cache loading mechanism, improving throughput by up to 1.87x offline and 1.96x online.
The author shares a technical deep dive into saturating NIC and disk bandwidth using RDMA and io_uring, highlighting how an AI agent achieved the optimization with a flawed explanation, then details the actual debugging process.
A user proposes building a heterogeneous AI cluster using Blackwell GPUs and high-memory servers connected via RDMA, seeking collaboration on Tinygrad driver development.
NVIDIA's Spectrum-X Ethernet fabric with Multipath Reliable Connection (MRC) sets a new standard for gigascale AI networking, improving throughput, load balancing, and resilience for large-scale AI training, as demonstrated by OpenAI, Microsoft, and Oracle.
MLX's RDMA-over-Thunderbolt implementation for macOS is now available as a standalone library, enabling high-speed Mac clusters for local AI workloads.
A demo shows Qwen3.6 35B vision model running across two M5 Max MacBook Pros connected via RDMA over Thunderbolt 5, achieving near-instant responses with prefix caching. The model correctly identifies Apple Park but misidentifies a person in the image.