block-attention

This paper introduces SemanticSeg, a large-scale dataset for semantic segmentation of long texts, and block distillation, a training framework that enables block attention models to approach full-attention performance, improving KV cache reuse in RAG and long-context scenarios.