Tag
CoRT proposes a token-level credit weighting method for GRPO that uses counterfactual replay to compute token-wise log-likelihood contrasts, redistributing the signed advantage across tokens without an auxiliary scorer, achieving average gains of 4.4 percentage points over response-level GRPO.
FlowTracer is an RL framework that uses attention-induced graphs to trace reasoning flows and assign token-level credit, improving performance on reasoning tasks.