Tag
This paper from OpenAI investigates whether reinforcement learning on beneficial behavior can produce broad and persistent alignment generalization beyond the training distribution. Using a dataset of realistic situations, they show that RL training on beneficial traits improves out-of-distribution alignment and persistence against adversarial attacks.
OpenAI releases research on reinforcement learning for training models to exhibit beneficial traits like honesty and corrigibility, showing that such training generalizes across domains and persists under adversarial pressure.