long-context-retrieval

This paper investigates the signed nature of FFN residual writes in long-context retrieval, finding that FFN writes act as suppressors or amplifiers depending on layer and task, and proposes a gradient-based diagnostic to distinguish these roles.