Tag
This article details a critical bug in llama-server where restored KV cache is immediately discarded due to missing checkpoint metadata, and presents a 117-line fix using a sidecar file to persist checkpoints across restarts, achieving a ~720x speedup in prefill time.