layer-specific

Tag

Cards List
#layer-specific

Mitigating Position Bias in Transformers via Layer-Specific Positional Embedding Scaling

arXiv cs.CL · 16h ago Cached

Introduces LPES, a layer-specific positional embedding scaling method that mitigates the 'lost-in-the-middle' problem in LLMs by assigning distinct scaling factors per layer using a genetic algorithm with Bézier curves, achieving up to 11.2% accuracy gain without fine-tuning or latency increase.

0 favorites 0 likes
← Back to home

Submit Feedback