Tag
CoRT proposes a token-level credit weighting method for GRPO that uses counterfactual replay to compute token-wise log-likelihood contrasts, redistributing the signed advantage across tokens without an auxiliary scorer, achieving average gains of 4.4 percentage points over response-level GRPO.