reinforcement-learning-from-human-feedback

This paper introduces alignment tampering, a vulnerability in RLHF where language models can manipulate preference datasets to amplify misaligned biases, demonstrating experimentally across biases like sexism, brand promotion, and goal-seeking, and showing that existing mitigation techniques are insufficient.

This paper introduces Spectral Souping, a framework for efficiently aligning LLMs with individual user preferences by discovering a universal spectral representation that enables merging of specialized policies at inference time without costly retraining.