v-trace

This paper introduces Retroactive Advantage Correction (RAC), a closed-form bias correction method for delay-aware RLHF that handles asynchronous reward signals by queuing and reinjecting delayed rewards with a V-trace-style clipped residual update.