@che_shr_cat: 1/ We have spent years optimizing KV cache via head-sharing (GQA/MQA), but we ignored a fundamental assumption: why do …
Summary
This thread challenges the fundamental assumption that Transformers require separate Q, K, and V projections, proposing that merging them can yield massive memory savings for KV cache.
View Cached Full Text
Cached at: 06/10/26, 12:25 AM
1/ We have spent years optimizing KV cache via head-sharing (GQA/MQA), but we ignored a fundamental assumption: why do Transformers need three separate Q, K, and V projections in the first place?
Turns out, they don’t. Merging them unlocks massive memory savings.
2/ A new paper, “Do Transformers Need Three Projections?”, systematically dismantles this bottleneck.
Ali Kayyam, Anusha Madan Gopal, and M Anthony Lewis prove that sharing QKV projections is viable and highly effective.
3/ Standard multi-head attention projects Q, K, and V independently.
The authors analyzed trained weights and found high redundancy: Key and Value projection spaces have a 0.73 cosine similarity. Query is distinct (0.42).
This justifies a new variant: Q-K=V.
4/ In the Q-K=V configuration, we project inputs into Q and a unified K space.
During autoregressive decoding, you only store the single unified K tensor in the KV cache. This instantly cuts your KV cache memory footprint by 50% with zero decompression overhead.
5/ This is not a replacement for Grouped Query Attention (GQA) or Multi-Query Attention (MQA). It is orthogonal.
Combining Q-K=V with MQA (Q-MQA) yields an astonishing 96.9% reduction in KV cache memory, creating a massive shift in the serving Pareto frontier.
6/ Does this break the model? Hardly.
At 1.2B scale, the Q-K=V model loses only 0.41% average downstream accuracy (HellaSwag, WinoGrande, etc.) and suffers a negligible 2.4% perplexity hit.
However, symmetric variants like Q=K=V fail catastrophically in causal LM.
7/ Before rewriting your training loops, here are the caveats: • Max scale tested is 1.2B params. • Evaluated up to 2k context length. • To get real-world speedups, we need custom CUDA kernels, as optimized setups (like FlashAttention) expect three distinct QKV tensors.
8/ Fascinating math detail: The authors show that under complete QKV collapse (Q=K=V), linear kernelized attention mathematically reduces to a recurrent State-Space Model (SSM) with adaptive, input-conditioned updates.
An elegant bridge between attention and SSMs.
9/ I think this is a highly underrated architectural shift. As long-context and edge-device serving dominate priorities, we can’t afford redundant weights.
Tying Key and Value projections directly in training is a simple, mathematically sound win.
10/ Read the full paper and code here:
Paper: https://arxiv.org/abs/2606.04032 Code: https://github.com/Brainchip-Inc/Do-Transformers-Need-3-Projections…
Read my complete technical breakdown: https://arxiviq.substack.com/p/do-transformers-need-three-projections…
What are your thoughts on collapsing the QKV space?
11/ I also illustrated this architectural shift as a comic to make the mechanics intuitive. Check it out below!
Similar Articles
Do transformers need three projections? Systematic study of QKV variants
This paper systematically studies variants of QKV projection sharing in transformers, finding that sharing key and value projections (Q-K=V) achieves 50% KV cache reduction with only 3.1% perplexity degradation, and combining with GQA/MQA can reach up to 96.9% cache reduction—enabling practical on-device inference with minimal quality loss.
The KV-cache wall: why fixed-size memory sequence models keep coming back
Explores the growing memory bottleneck of KV-cache in transformer inference, explaining why alternative architectures with fixed-size memory like Mamba and RWKV are gaining renewed attention.
@rohanpaul_ai: Interesting, this paper shows that Transformers may not need separate key and value projections to work well. This pape…
This paper investigates whether Transformers need separate key and value projections, finding that sharing them can reduce KV cache by 50% with only 3.1% higher perplexity, and further cuts when combined with GQA and MQA.
KV Cache Is Becoming the Memory Hierarchy of Inference
The article discusses how the KV cache is evolving into a memory hierarchy for LLM inference, optimizing memory management during decoding.
@songhan_mit: Explore our continued efforts on KV cache compression:
A tweet from Song Han highlights continued work on KV cache compression, featuring a blog by Weian Mao that discusses system-level aspects often overlooked in papers.