Tag
A llama.cpp/GGML PR (26291) speeds up RPC-based 300GB model loads by ~300%, cutting load time from ~5 minutes to ~1.5 minutes on consumer hardware with the GGML_RPC_LOAD_THREADS setting.
A thread highlighting a technical blog series on how Hugging Face's transformers library loads models efficiently, covering meta device, safetensors, CUDA caching, and more.