WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution

Hugging Face Daily Papers Papers
reinforcement-learning agentic-training long-horizon-agents environment-synthesis credit-assignment cowork-agents

Summary

WorkRover is a framework for training long-horizon cowork agents through recursive environment evolution, introducing Factorial Compatibility Replay and Verifier-to-Turn Credit Assignment to improve training signals. Post-training Qwen3.5-35B-A3B into WorkRover-35B-A3B yields large gains on JobBench, OfficeQAPro, and GDPvalv2 benchmarks.

Scaling interactive environments is important for training long-horizon agents through reinforcement learning. These environments often require further testing and refinement after initial generation, and correcting defects in a task may also require updating its work context and verifier. However, existing construction pipelines offer limited support for such iterative refinement while keeping these components aligned. We introduce WorkRover, a framework for training long-horizon cowork agents through recursive environment evolution. Its harness-driven synthesis enables construction agents to jointly build and validate semantically aligned work contexts, executable tasks, and evidence-grounded verifiers, adapting their actions to workspace state and execution feedback rather than following a fixed construction workflow. To support environment evolution, we introduce Factorial Compatibility Replay (FCR), which uses rollout feedback to evaluate combinations of component revisions through dependency-closed replay and selects the smallest validated repair among tested candidates while preserving task requirements and evaluation constraints. To provide finer-grained training signals, we propose Verifier-to-Turn Credit Assignment (VTCA), which augments trajectory-level GRPO advantages with signals from deterministic workspace-progress checks and assigns a bounded share of positive-progress credit to prerequisite turns using recorded tool-output and artifact dependencies. Using WorkRover, we post-train Qwen3.5-35B-A3B to obtain WorkRover-35B-A3B and evaluate it on several cowork-agent benchmarks. Its reported scores exceed those of Qwen3.5-35B-A3B by 26.27 percentage points on JobBench, 29.32 percentage points on OfficeQA Pro, and 413.25 Elo points on GDPval v2. These results suggest that WorkRover offers a promising foundation for scalable training of long-horizon cowork agents.
Original Article
View Cached Full Text

Cached at: 10/14/26, 06:53 AM

Paper page - WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution

Source: https://huggingface.co/papers/2610.18067 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Scalinginteractiveenvironmentsisimportantfortraininglong-horizonagentsthroughreinforcementlearning.Theseenvironmentsoftenrequirefurthertestingandrefinementafterinitialgeneration,andcorrectingdefectsinataskmayalsorequireupdatingitsworkcontextandverifier.However,existingconstructionpipelinesofferlimitedsupportforsuchiterativerefinementwhilekeepingthesecomponentsaligned.WeintroduceWorkRover,aframeworkfortraininglong-horizoncoworkagentsthroughrecursiveenvironmentevolution.Itsharness-drivensynthesisenablesconstructionagentstojointlybuildandvalidatesemanticallyalignedworkcontexts,executabletasks,andevidence-groundedverifiers,adaptingtheiractionstoworkspacestateandexecutionfeedbackratherthanfollowingafixedconstructionworkflow.Tosupportenvironmentevolution,weintroduceFactorialCompatibilityReplay(FCR),whichusesrolloutfeedbacktoevaluatecombinationsofcomponentrevisionsthroughdependency-closedreplayandselectsthesmallestvalidatedrepairamongtestedcandidateswhilepreservingtaskrequirementsandevaluationconstraints.Toprovidefiner-grainedtrainingsignals,weproposeVerifier-to-TurnCreditAssignment(VTCA),whichaugmentstrajectory-levelGRPOadvantageswithsignalsfromdeterministicworkspace-progresschecksandassignsaboundedshareofpositive-progresscredittoprerequisiteturnsusingrecordedtool-outputandartifactdependencies.UsingWorkRover,wepost-trainQwen3.5-35B-A3BtoobtainWorkRover-35B-A3Bandevaluateitonseveralcowork-agentbenchmarks.ItsreportedscoresexceedthoseofQwen3.5-35B-A3Bby26.27percentagepointsonJobBench,29.32percentagepointsonOfficeQAPro,and413.25ElopointsonGDPvalv2.TheseresultssuggestthatWorkRoveroffersapromisingfoundationforscalabletrainingoflong-horizoncoworkagents.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2610\.18067

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.18067 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.18067 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.18067 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

AutoEnvScaling: Automating the Data Flywheel with Terminal Agents

Hugging Face Daily Papers

AutoEnvScaling treats RL environment design as a terminal task, letting a proposer agent automatically build training environments from solver rollouts to automate the data flywheel. The method raises the valid-environment rate up to 25x at 75% lower cost, improves Qwen terminal-agent scores by up to 12.2 points, and enables a single model to co-evolve as both solver and generator, a step toward recursive self-improvement.

Scaling Verifiable Environments for Long-horizon Work Agents

Hugging Face Daily Papers

论文提出 WorkForge，一个从真实世界资源构建可验证工作智能体训练环境的可扩展合成框架，构建了覆盖 40 个专业领域的 16.7K 个可验证环境，使 Qwen3.5-35B-A3B-Base 在 GDPVal 上从 45.5 提升至 73.6。该论文已被撤稿（withdrawn）。