Tag
A llama.cpp/GGML PR (26291) speeds up RPC-based 300GB model loads by ~300%, cutting load time from ~5 minutes to ~1.5 minutes on consumer hardware with the GGML_RPC_LOAD_THREADS setting.
A comprehensive handbook explaining RPC, Protocol Buffers, and gRPC for building modern distributed systems, including a hands-on implementation with Dart and Flutter.
Describes running the GLM-5.2 model with 4-bit quantization at 12.2 tokens per second on a cluster of 16 AMD MI50 GPUs using llama.cpp's RPC, achieving coherent long-form generation at 10.7k context.
User shares benchmark results running the 550B Nemotron Ultra model across two machines using RPC, achieving impressive throughput on older AMD MI50 and Nvidia P40 GPUs.
RPCForge is a product launch offering a multi-chain Ethereum RPC solution that emphasizes user ownership and control.
gRPC is a modern, open source, high-performance remote procedure call (RPC) framework that enables client and server applications to communicate transparently.