DataFlex-RL: An Evaluation Platform for RLVR Data Policies

Hugging Face Daily Papers Papers
reinforcement-learning rlvr evaluation-platform data-policies grpo benchmarks uniform-sampling

Summary

DataFlex-RL is an evaluation platform for RLVR data policies that finds uniform sampling performs comparably to adaptive methods across mathematics, logic, and science benchmarks.

Data policies for reinforcement learning with verifiable rewards (RLVR) determine which rollouts are used, how strongly they are weighted, and which domains contribute to subsequent training batches. We introduce DataFlex-RL, an evaluation platform for comparing these choices under a common GRPO recipe. Our primary experiment evaluates 13 configurations across 12 matched seeds using Qwen2.5-7B-Base and 12 mathematics, logic, and science benchmarks. Uniform GRPO improves the domain-balanced average accuracy by 7.76 percentage points over the untrained checkpoint. None of the eight rollout-selection or reweighting methods achieves a paired 95% confidence interval that excludes zero relative to uniform sampling, and none of the three adaptive mixtures outperforms a fixed equal mixture at the same level of precision. A corrected 12-seed extension on Llama-3.1-8B-Base places the additional methods on the same score scale as the original controls, but does not reveal a consistent winner in terms of observed mean performance. We also quantify evaluation sensitivity by rescoring nine Qwen2.5-7B-Instruct runs using a math-heavy six-benchmark summary, consisting of five mathematics benchmarks and GPQA-Diamond but no logic benchmark, and comparing it with the domain-balanced 12-benchmark summary. The resulting rankings are negatively correlated, with a correlation coefficient of -0.33, whereas summaries that retain all 12 benchmarks largely agree. Across the controlled settings studied here, changing the data policy measurably changes the training process but does not produce a reproducible improvement over uniform training.
Original Article
View Cached Full Text

Cached at: 09/14/26, 02:35 PM

Paper page - DataFlex-RL: An Evaluation Platform for RLVR Data Policies

Source: https://huggingface.co/papers/2609.06107

Abstract

DataFlex-RL evaluates reinforcement learning data policies and finds that uniform sampling matches or exceeds adaptive rollout selection, reweighting, and domain mixing across math, logic, and science benchmarks.

Data policies for reinforcement learning with verifiable rewards (RLVR) determine which rollouts are used, how strongly they are weighted, and which domains contribute to subsequent training batches. We introduce DataFlex-RL, an evaluation platform for comparing these choices under a commonGRPOrecipe. Our primary experiment evaluates 13 configurations across 12 matched seeds usingQwen2.5-7B-Baseand 12 mathematics, logic, and science benchmarks. UniformGRPOimproves thedomain-balanced average accuracyby 7.76 percentage points over the untrained checkpoint. None of the eightrollout-selectionorreweightingmethods achieves a paired 95% confidence interval that excludes zero relative to uniform sampling, and none of the threeadaptive mixturesoutperforms a fixed equal mixture at the same level of precision. A corrected 12-seed extension onLlama-3.1-8B-Baseplaces the additional methods on the same score scale as the original controls, but does not reveal a consistent winner in terms of observed mean performance. We also quantify evaluation sensitivity by rescoring nine Qwen2.5-7B-Instruct runs using a math-heavy six-benchmark summary, consisting of five mathematics benchmarks andGPQA-Diamondbut no logic benchmark, and comparing it with the domain-balanced 12-benchmark summary. The resulting rankings are negatively correlated, with a correlation coefficient of -0.33, whereas summaries that retain all 12 benchmarks largely agree. Across the controlled settings studied here, changing the data policy measurably changes the training process but does not produce a reproducible improvement over uniform training.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2609\.06107

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.06107 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.06107 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.06107 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

From RLVR to RLSVR (GitHub Repo)

TLDR AI

Introduces RLSVR, a task-transformation paradigm that extends RLVR to open-ended tasks via self-verifiable rewards from self-play games, instantiated in SpyRL and Vision-Zero. It improves LLM performance on summarization, creative writing, and math reasoning.

Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR

Hugging Face Daily Papers

This paper introduces POW3R, a policy-aware rubric reward framework for reinforcement learning with verifiable rewards (RLVR). It shows that static rubric aggregation misallocates learning signal, and POW3R achieves faster convergence and better performance across multiple settings.