vram-cache

Tag

Cards List
#vram-cache

VRAM disk cache of MoE makes 340 pp/s 9.6 tg/s for Kimi 2.7 on a single dgx spark

Reddit r/LocalLLaMA · 4d ago

A detailed strategy using VRAM as disk cache with unified memory and llama.cpp settings achieves 340 pp/s and 9.6 tg/s for Kimi K2.7 on a single DGX Spark.

0 favorites 0 likes
← Back to home

Submit Feedback