PSA for DeepSeek-V4-Flash-0731 users — don't blow out your prompt cache with system role messages mid-conversation

Reddit r/LocalLLaMA Tools

Summary

A PSA warning DeepSeek-V4-Flash-0731 users that system role messages are hoisted to the top, ruining prompt caching and proximity, and advising the use of latest_reminder instead.

DSv4F doesn't ship a jinja, but for distributions that do and faithfully reconstruct what DS releases in their chat template python, every system message is hoisted into the system prompt at the top -- the format has no mid-conversation system turn. So, anything you stick at the tail or mid-convo actually fries your prefix (and doesn't have conversational proximity to the injection point). Use latest_reminder, which is the role DS trained for how most templates use system and what most people providing quants are passing through (if they match DS' python template). I use llama.cpp and it happily passes it through no issue; dunno how other engines work with it. Couldn't figure out why my prompt caching was so garbage and there it was, so I'm passing it on to hopefully save others time and frustration (and probably money, if you're using a hosted version).
Original Article

Similar Articles

@YRSM_Simon: Found a bug in vLLM local deployment of DeepSeek V4 Flash. Same session responds instantly, but switching to another session and back triggers a full re-prefill, and the KV cache fails. Stayed up all night, 100% reproducible: A first...

X AI KOLs Following

Discovered a KV cache invalidation bug in DeepSeek V4 Flash with vLLM local deployment: after switching sessions, prefix cache hit rate drops to 0, causing re-prefill, with latency exploding from ~1 second to ~100 seconds. Root cause suspected to be vLLM calling _maybe_evict_cached_block() when reusing free queue blocks, which removes the cached index.