Paper page - ClawGym II: Exploring Black-Box RL on Agent Harness

Source: https://huggingface.co/papers/2608.16798 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

A unified black-box reinforcement learning framework enables stable, scalable optimization of general agents through complex harnesses via sandbox execution, trajectory reconstruction, and mix-harness training.

Agent harnesses have substantially improved performance on long-horizon tasks by coordinating agent interactions with the environment. However,reinforcement learningthrough complex harnesses remains largely unexplored, as scaling such training to long-horizon agent tasks introduces fundamental challenges. In this work, we present a unifiedblack-box RLframework for stable and scalable optimization of general agents through complex harnesses. Concretely, we first build asandbox-based executioninfrastructure that isolates task environments and harnesses within temporary sandboxes for large-scale concurrent rollouts. We then decouple policy optimization from opaque harness execution and place aserving proxyat the model boundary to capture model calls. To reconstructmulti-turn trajectoriesand improve training efficiency, we organize the captured calls intoprefix treesand further adapt both critic-basedPPOand critic-freeGRPOto optimize over the recovered tree structure. Meanwhile, we maintain training-inference consistency throughout the optimization process. Finally, we introducemix-harness training, allowing a single model to be jointly optimized by heterogeneous harnesses. With Qwen3-30A3B,black-box RLimproves Pass@1 on ClawGym-Bench by 9.98 and 14.81 points through OpenClaw and Claude Code, respectively, while remaining stable over 200-400 optimization steps. Moreover, the framework yields consistent gains on more challenging tasks such as JobBench and OfficeQA. Overall, our framework enables effective, stable, and scalable optimization of general agents through black-box harnesses, supporting unified training across heterogeneous execution systems.

View arXiv pageView PDFProject pageGitHub33Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.16798 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.16798 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.16798 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.