Tag
Introduces V-Steer, a training-free inference-time method that edits cached value vectors to restore instruction hierarchy in language models, raising primary constraint accuracy from under 18% to 92% on controlled benchmarks with negligible overhead.
This paper investigates whether deep layer value vectors in transformer attention need context from the residual stream. It proposes Bank of Values (BoV), which uses context-free token-specific value vectors in the last third of layers, improving validation loss and benchmark scores over standard attention.