gated-attention

This paper proposes Hybrid Gated Attention (HyGA), a framework with three gating strategies that improve attention representational capacity and mitigate attention sinks, achieving better performance and efficiency than standard gated attention.