massive-activations

This paper presents the first systematic study of massive activations in hybrid linear-attention LLMs, uncovering pre-attention spikes and inter-spike plateaus governed by cancellation timing, and showing how their morphology recovers at full-attention limits.