Tag
This paper identifies flaws in RAG evaluation metrics by measuring context utilization causally, demonstrating that narrow sequential contexts improve recall over wide contexts, and introduces a submodular scheduler for optimized allocation.
This paper introduces Asymmetric Attention Heads (AAH), a framework that assigns different context windows to attention heads in transformers, with experiments showing improved language modeling performance.