Tag
CRISP introduces a method for improving sparse prefilling in long-context LLM inference by using structural routing to address noise accumulation, achieving speedups and performance gains on retrieval tasks.
This paper proposes Prefilling-dLLM, a training-free framework that partitions the prefix into chunks and caches KV representations, achieving state-of-the-art quality and up to 28x speedup for long-context inference in diffusion language models.