long-context-modeling

SpectralShift introduces a spectral reparameterization approach to effectively extend the context window of Gated DeltaNet models by reshaping the decay spectrum, improving long-context capabilities through continual pretraining.