consistency-training

Self-CTRL is a reinforcement learning method that improves consistency between a language model's self-explanations and its actual behavior. It significantly boosts correlation between self-reported and measured biases and improves refusal prediction accuracy from 36% to 92% while reducing harmful response rates.

This paper introduces a temporal difference (TD) learning objective for diffusion models that enforces cross-time consistency along the denoising trajectory. It reformulates denoising as a reinforcement learning policy evaluation problem, showing significant improvements in sample quality (FID), especially for few-step samplers.

This paper introduces Political Consistency Training (PCT), a reinforcement learning approach to reduce covert political bias in large language models while maintaining helpfulness, and releases metrics for sentiment and helpfulness consistency.