Tag
Introduces a scalable, inference-only data valuation pipeline that approximates Shapley values to audit LLM alignment datasets, reducing manual audit search space by 99.1% and uncovering hidden label failures in HelpSteer2 and HH-RLHF.
This paper identifies and formalizes rater state bias in RLHF preference data, where annotator emotional state can confound preference labels. It proposes an audit framework with falsifiable predictions to detect such biases.
Introduces DualEval, a framework that jointly calibrates model ability and item difficulty/sharpness to unify static benchmark and arena-style evaluation, enabling more reliable rankings and downstream applications like benchmark compression and anomaly detection.
This research introduces a method using interpretability to predict which behaviors DPO will amplify or suppress from a preference dataset before training, enabling data debugging to prevent undesired effects. The technique achieves R²=0.9 prediction accuracy and is integrated into Goodfire's Silico platform.
RUBRIC-ARROW presents an alternating framework for reward modeling that improves upon rubric-based methods by reducing ties and leveraging pairwise preference data, achieving competitive accuracy and gains for LLM post-training in non-verifiable domains.