chunk-wise

Tencent releases HiLS-Attention-7B, a 7B parameter model using a novel chunk-wise sparse attention mechanism that learns chunk selection end-to-end, enabling efficient long-context modeling and strong extrapolation beyond 4x training length.