@TheAhmadOsman: To clarify, OMP has 97.5% cache hits Most of my usage is self-hosted models, and not all inference setups report the to…
Summary
A user clarifies that OMP has a 97.5% cache hit rate and discusses the use of self-hosted DeepSeek V4 Flash models, addressing concerns about caching performance in inference setups.
View Cached Full Text
Cached at: 08/23/26, 11:43 PM
To clarify, OMP has 97.5% cache hits
Most of my usage is self-hosted models, and not all inference setups report the token telemetries properly so that’s why it was missing in my original screenshot
These are DeepSeek V4 Flash 0731 running through OMP in the last couple of days https://t.co/BC5EL5Sxet
Theo - t3.gg (@theo): Holy shit is OMP actually that bad at caching???
Similar Articles
I tested DeepSeek Harness with GLM, Kimi, Opus, and GPT to see if prompt caching still works with other models
The article tests whether DeepSeek Harness maintains high prompt caching rates when using alternative AI models, finding that GLM and Kimi achieve 97-99% cache reuse, while Opus shows no cache activity and GPT test failed.
@ciruai: Testing DeepSeek v4 Flash on the AMD Ryzen AI Max+ 395 Strix Halo with 128GB RAM. Getting ~15 TPS over a decently long …
Testing DeepSeek v4 Flash on the AMD Ryzen AI Max+ 395 with 128GB RAM achieves ~15 TPS for a 284B MoE model (13B active) locally, costing $3,000 versus $25,000+ for a datacenter setup, highlighting the feasibility of running large models on consumer hardware.
DeepSeek-V4-Flash 284B on 5.3GB of memory
A developer showcases Mference, a new inference engine that runs MoE models like DeepSeek-V4-Flash on just ~5.3GB of memory by streaming experts from SSD, with a native Mac app and OpenAI-compatible server.
@Michaelzsguo: KV cache is the model’s working memory during generation. As the context window gets longer, the model has to keep more…
DeepSeek's KV cache compression innovations, including MLA and CSA/HCA, reduce KV cache size by 93%, enabling efficient long-context inference and SSD-based caching, as demonstrated by antirez's ds4.c project.
@TheAhmadOsman: OMP with DeepSeek V4 Flash 0731 is SO GOOD
A tweet praising the quality of OMP combined with DeepSeek V4 Flash 0731.