parallel-rl

This paper identifies value mismatch in parallel reinforcement learning where a shared critic degrades performance by mixing environment-specific values, and proposes conditioning the critic on environment indices to improve learning stability and return in benchmarks like Procgen.