Tag
The paper introduces LOGOS, a write-aware detector that identifies attention heads responsible for non-literal retrieval in LLMs by scoring the projection of their OV-circuit output onto the answer-token unembedding direction, outperforming prior attention-based methods across multiple model families.
This paper introduces LOCOS, a method for identifying attention heads responsible for non-literal context synthesis in large language models, outperforming existing techniques on retrieval benchmarks.