depth-sharing

The paper introduces Gated Recurrent Transformer, a model that reuses a shared core across depth with adaptive update gates, achieving comparable or better quality than deeper models with fewer parameters and lower memory.

Introduces FreqDepthKV, a frequency-guided depth sharing method for KV cache compression in long-context LLM inference, which factorizes adjacent-layer KV states into shared low-frequency components and sparse high-frequency residuals, improving memory efficiency and throughput while preserving accuracy on benchmarks.