long-context-inference

PuzzleKV is a training-free method for compressing key-value cache in large language models using page-wise low-rank decomposition, achieving over 96% performance with approximately 60% storage.