Tag
SSOG-Attention introduces a novel attention mechanism using Sum of Separable Gaussians to achieve sub-quadratic complexity, outperforming or matching SDPA with faster convergence and improved efficiency.