Tag
该论文对大语言模型的位置偏差进行了全面研究，覆盖从十亿到万亿参数规模的模型及五种语言。研究发现模型规模增大、思维链提示以及新兴的思考模型能显著缓解甚至几乎消除位置偏差。
PACT introduces a four-term training objective for single-token typed-decision models that exploits contrastive pairs and certificate-verified ablations, matching baseline accuracy while significantly reducing position bias, seed variance, and calibration error.
This paper investigates the position bias phenomenon in online distillation, finding that early tokens provide more useful supervision signals, and proposes the importance-weighted IW-OPD method to improve OPD training.
This paper tests the hypothesis that correcting position bias in attention scores can enable single-pass document sorting for long-context QA. The authors find that while debiasing helps moderately, it does not match the accuracy of iterative sorting, indicating that repeated reordering provides additional benefits beyond bias correction alone.
The paper identifies position bias in on-policy distillation for language models, where later tokens in student-generated answers receive degraded supervision. The proposed Importance-Weighted On-Policy Distillation (IW-OPD) weights corrections based on accumulated drift, improving learning speed and final performance.
This paper investigates the run-to-run reliability of LLM-as-a-Judge evaluations, finding that pairwise preferences flip 13.6% of the time on average, with significant first-position bias in GPT-4o-mini, and recommends multi-trial aggregation and position randomization.
This paper investigates whether positional bias in dense retrievers originates from architecture or training data, finding that training data distribution strongly influences bias and that balanced training can reduce sensitivity by up to 87% while maintaining retrieval performance.
This paper identifies a blind spot in long-context LLM reasoning benchmarks: they fail to control task position within the context, allowing positional failures to go undetected. The authors propose Context Rot Evaluation (CRE) to systematically vary task position, filler content, and context length, revealing severe accuracy drops for some models when reasoning tasks are placed in the middle of long contexts.
This paper reframes pairwise ranking prompting as active learning from noisy comparisons, introducing a noise-robust framework with a randomized-direction oracle to improve ranking quality under call constraints and address position bias.
This research paper investigates position bias in reasoning models, finding that bias scales with the length of the reasoning trajectory rather than being eliminated by 'more thinking.' The study provides causal evidence and a diagnostic toolkit for auditing this length-driven bias in multiple-choice QA evaluations.