query-key

This paper introduces FourierQK, a method that applies FFT-based frequency-domain preprocessing to learned query and key projections in transformer attention, achieving significant validation loss reductions on character-level language modelling. The approach preserves the full attention score structure and demonstrates reproducible gains over standard dot-product attention.