Beyond Next-Token Prediction: An RLVR Proof of Concept for Tool-Use Agents on Atlassian Workflows

# An RLVR Proof of Concept for Tool-Use Agents on Atlassian Workflows
Source: [https://arxiv.org/html/2607.01465](https://arxiv.org/html/2607.01465)
## Beyond Next\-Token Prediction: An RLVR Proof of Concept for Tool\-Use Agents on Atlassian Workflows

\(2026\)

###### Abstract\.

Large language models are trained to predict the next token, not to act inside a specific API\. In niche enterprise SaaS workflows — where success means hitting the right endpoint with the right nested arguments in the right order — this objective mismatch shows up as silent failures: dropped required fields, hallucinated tools, or early stops after a single read\. We ask whether Reinforcement Learning with Verifiable Rewards \(RLVR\), applied directly in the target environment, closes the gap\. As a proof of concept we build a suite of five synthetic environments emulating the Jira REST v3 and Confluence v2 APIs at schema fidelity; rewards are computed entirely from the tool\-call trace, with no live API, no learned judge, and no human label in the loop\. Scoring prompted Qwen3\-1\.7B and Qwen3\.5\-4B on the same checkers that drive GRPO training, we find that on the four scenarios whose rewards are non\-degenerate the RL\-trained policy lifts average reward from a 4B\-baseline range of 0\.35–0\.92 to 0\.95–1\.00, with the largest single gain on Confluence page creation \(0\.35→1\.000\.35\\rightarrow 1\.00\)\. We position this as a preliminary step toward outcome\-optimised small models for niche enterprise APIs, and foreground two limitations a workshop reader should weigh: hand\-crafting verifiable rewards does not scale beyond the handful of endpoints reported here, and one of our five scenarios \(ticket\-transition\) has a saturating reward shape that the prompted 4B already maxes out\.

reinforcement learning, tool use, verifiable rewards, GRPO, synthetic environments, agent evaluation, Jira, Confluence

††copyright:none††conference:KDD 2026 Workshop on AI Agents; August 2026;††journalyear:2026User prompt\(6–12 / scenario\)Qwen3 policy\(1\.7B / 4B, BF16\)Synthetic API\(Jira v3 \+ Conf v2\)Tool\-call trace\(calls \+ args\)Verifiable reward\(R1 args \+ R2 order \+ R3 penalties\)GRPO update\(group\-relative\)tool callresponsescalar rewardparameter updateFigure 1\.The RLVR training loop\. A user prompt enters a Qwen3 policy that emits tool calls into a synthetic, schema\-faithful copy of the Jira REST v3 / Confluence v2 APIs; the resulting tool\-call trace is scored by a hand\-designed verifiable reward \(per\-argument correctness, validate–mutate–verify structural bonuses, hallucination and excess\-call penalties\); GRPO updates the policy on the per\-prompt group of rollouts\. No live API, no learned judge, and no human label are in the loop\.A horizontal pipeline of five boxes: User prompt, Qwen3 policy, Synthetic API, Tool\-call trace, and Verifiable reward\. The reward feeds a scalar into a GRPO update box, which sends a parameter update back to the Qwen3 policy, closing the loop\.## 1\.Introduction

LLM\-driven agents are moving from chat into workflows: opening tickets, transitioning issues, updating wikis\. A growing literature treats tool invocation as a first\-class capability to be elicited via supervised fine\-tuning or RL on real APIs\(Qinet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib9); Schicket al\.,[2023](https://arxiv.org/html/2607.01465#bib.bib11); Wanget al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib15); Yaoet al\.,[2023](https://arxiv.org/html/2607.01465#bib.bib17)\), yet public RL benchmarks for agents target the open web\(Zhouet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib18)\), software engineering\(Jimenezet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib4)\), or broad agentic suites\(Liuet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib6); Trivediet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib13)\)\. Niche enterprise SaaS workflows — narrow, schema\-heavy, dominated by the “check the resource, then mutate it” pattern — get comparatively little attention\.

The deeper issue is an objective mismatch: LLMs are optimised against next\-token likelihood on internet\-scale text, not to act inside a particular API surface\. Asked to create a Jira sub\-task or a Confluence page in our suite, a prompted Qwen3\.5\-4B knows the rough shape of the call but fills slots wrong, omits required fields, or stops after the first read — behaviours that read as fluent under a token\-prediction objective and score poorly under any check that grounds the output back to the environment \(our prompted\-baseline scores in Section[6](https://arxiv.org/html/2607.01465#S6)bear this out\)\.Reinforcement Learning with Verifiable Rewards\(RLVR\)\(Cobbeet al\.,[2021](https://arxiv.org/html/2607.01465#bib.bib1); DeepSeek\-AI,[2025](https://arxiv.org/html/2607.01465#bib.bib3); Lambertet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib5); Shaoet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib12)\)gives up reward modelling in favour of programmatic checkers wherever correctness can be inspected\. Tool\-use is naturally amenable: an agent’s output is a sequence of structured tool calls whose argument values and ordering can be inspected directly\. We exploit that observation to train agents against outcome\-level rewards in synthetic copies of the target environment, with no live API, learned judge, or human label in the loop\.

#### Contribution\.

We present a suite of five synthetic, schema\-faithful Atlassian environments, the verifiable rewards that score them, prompted baselines on the same rewards, and end\-to\-end RL training results\. Concretely:

- •Environments that emulate the Jira REST v3 and Confluence v2 endpoints used in common automation flows \(issue read/create, page read/create, transitions, labels\), with stateful parent–child constraints and per\-batch reset \(Section[3](https://arxiv.org/html/2607.01465#S3)\)\.
- •Verifiable reward functions that decompose into per\-argument correctness, structural bonuses for the validate–mutate–verify pattern, and penalties for hallucinated or duplicate calls — all computable from the tool\-call trace \(Section[4](https://arxiv.org/html/2607.01465#S4)\)\.
- •Prompted baselines for Qwen3\-1\.7B and Qwen3\.5\-4B, run via the HuggingFace Inference Router against the identical reward checkers used at training time \(Section[6](https://arxiv.org/html/2607.01465#S6)\)\.
- •An end\-to\-end RLVR training recipe using GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib12)\)on Qwen3 models\(Yanget al\.,[2025](https://arxiv.org/html/2607.01465#bib.bib16)\)via TRL\(von Werraet al\.,[2020](https://arxiv.org/html/2607.01465#bib.bib14)\), with a convergence\-based early\-stopping callback \(Section[5](https://arxiv.org/html/2607.01465#S5)\), reaching≥0\.95\\geq 0\.95average reward on every non\-degenerate scenario in tens to low hundreds of generation batches and producing the largest absolute lift over prompted 4B baselines on schema\-heavy creation tasks \(e\.g\.\+0\.65\+0\.65on Confluence page creation\)\.

The result is a proof of concept: the recipe works on this cheap substrate, and Section[7](https://arxiv.org/html/2607.01465#S7)foregrounds what it does not yet establish — chiefly that hand\-crafted verifiable rewards do not scale to the full Atlassian public surface, and that one of our five scenarios has a saturating reward shape the prompted 4B already maxes out, which we keep as a transparency control rather than part of the headline claim\.

## 2\.Related Work

#### Tool\-use agents\.

Toolformer\(Schicket al\.,[2023](https://arxiv.org/html/2607.01465#bib.bib11)\)and ToolLLM\(Qinet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib9)\)train models to invoke real APIs; ReAct\(Yaoet al\.,[2023](https://arxiv.org/html/2607.01465#bib.bib17)\)and CodeAct\(Wanget al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib15)\)interleave reasoning with action\. These works prove that LLMs can use tools but largely rely on supervised data; we focus on the RL phase and on training\-time environments that obviate live calls\.

#### RL with verifiable rewards\.

DeepSeek\-R1\(DeepSeek\-AI,[2025](https://arxiv.org/html/2607.01465#bib.bib3)\), DeepSeekMath\(Shaoet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib12)\), and Tülu 3\(Lambertet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib5)\)demonstrate that programmatically checkable rewards drive strong reasoning gains, primarily on math and code\. Concurrent work extends the recipe to tool\-use and agentic settings: ToolRL\(Qianet al\.,[2025](https://arxiv.org/html/2607.01465#bib.bib8)\)argues that reward design is the load\-bearing piece of tool\-use RL, and Agent\-RLVR\(Daet al\.,[2025](https://arxiv.org/html/2607.01465#bib.bib2)\)trains software\-engineering agents with environment rewards and guidance\. Our contribution sits in the same family of methods but is the environment\-and\-reward design for a different surface: narrow enterprise SaaS APIs where the verifier inspects nested argument values and call ordering rather than a final numeric answer, at sub\-second latency and with no live API in the training loop\.

#### Agent benchmarks and synthetic worlds\.

WebArena\(Zhouet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib18)\), AgentBench\(Liuet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib6)\), and SWE\-bench\(Jimenezet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib4)\)evaluate agents on rendered web pages and real GitHub repositories\. AppWorld\(Trivediet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib13)\)and ToolSandbox\(Luet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib7)\)provide stateful synthetic worlds spanning many apps\. Our work is narrower in surface but tailored for training: every reward signal is computed at sub\-second latency from the tool\-call trace, and the environment fits in a few hundred lines of Python, making it suitable for the inner loop of RL\.

## 3\.Environment Design

### 3\.1\.Design principles

The environments target four properties:

#### \(P1\) Schema fidelity\.

Tool signatures and response payloads mirror the public Jira REST v3 and Confluence v2 contracts \(e\.g\.POST/rest/api/3/issueexpectsfields\.project\.key,fields\.parent\.key, andfields\.issuetype\.id="10003"for a sub\-task\), so policies trained here speak the same wire format as their live counterparts\.

#### \(P2\) Stateful but resettable\.

Each environment exposes a mutable resource pool \(issues, pages\) so thatcreate\_\*calls have side effects observable by subsequentget\_\*calls; areset\_synthetic\_data\(\)hook is invoked before every reward computation to restore the pristine state, preventing state\-pollution across rollouts\.

#### \(P3\) Determinism\.

No randomness in the environment: identical tool\-call sequences yield identical responses\. This makes reward attribution exact and the substrate usable as a regression fixture\.

#### \(P4\) Verifiability\.

The ground\-truth solution for each prompt is a small dictionary of expected argument values, so the reward function never needs to invoke an oracle LLM or hit a live service\.

### 3\.2\.Scenarios

We instantiate five scenarios spanning Jira, Confluence, and the cross\-product \(Table[1](https://arxiv.org/html/2607.01465#S3.T1)\)\. All scenarios share a common interaction pattern: the user prompt names a resource by key, the agent is expected to validate that the resource exists, mutate state with a write call, and optionally verify the result with a final read\.

Table 1\.Scenarios in the synthetic Atlassian suite\. Tools are surfaced to the model via the OpenAI\-compatible function\-calling interface; the synthetic data layer holds 6–12 prompts per scenario with hand\-written ground\-truth arguments\.
### 3\.3\.Example tasks

One user prompt per scenario, drawn verbatim from the training set:

- •Ticket transition\.“Please resolve ticket ISK1\.”
- •Sub\-task creation\.“Create a sub\-task ‘Write QA test cases’ under ABC\-123 and assign to admin123\.”
- •Page creation\.“Create a page ‘Oncall Runbook \- Payments’ under parent page 789 in space PAY\.”
- •Page labeling\.“Can you correctly label page 10001?”
- •Cross\-product\.“Create a sub\-task ‘Write Release Notes’ under ABC\-123 and assign to admin123\. Also create a Confluence page ‘Release Notes 1\.0’ under parent 789 in space PAY\.”

At convergence on the cross\-product scenario the agent reads the named parent issue and the named parent page, then issues both create calls with correct nested arguments; the reward function \(Section[4](https://arxiv.org/html/2607.01465#S4)\) attributes credit for each correct argument and for the read\-before\-write ordering\. The converged tool\-call sequence is reproduced in Appendix[A](https://arxiv.org/html/2607.01465#A1)\(Listing[3](https://arxiv.org/html/2607.01465#LST3)\)\.

## 4\.Verifiable Reward Functions

For each scenario we hand\-design a dense reward function that maps the list of tool calls in a rollout to a scalar in\[0,1\]\[0,1\]\. The reward decomposes into three additive groups:

#### \(R1\) Per\-argument correctness\.

For each expected argument in the gold dictionary, we award a small constant \(0\.10 in the cross\-product reward; 0\.10–0\.15 per slot in the single\-product creation rewards\) if the agent’s emitted argument matches\. This is the dominant signal: e\.g\., in the sub\-task scenario, five fields \(summary,parent\.key,project\.key,assignee,issuetype\.id\) each contribute 0\.10, for a correctness ceiling of 0\.50\. For Confluence page labeling, R1 is instead a per\-label recall term: 0\.10 for every gold label the agent eventually adds \(Table[1](https://arxiv.org/html/2607.01465#S3.T1)\)\.

#### \(R2\) Structural bonuses\.

We award the validate–mutate–verify pattern: aget\_\*call before anycreate\_\*call, the correct write next, and an optionalget\_\*after the create to confirm\. The cross\-product reward awards 0\.05 per step; the single\-product creation rewards tune these higher \(typically 0\.05–0\.20 per step\) so that an isolated structural signal still drives learning when the per\-argument ceiling is smaller\. Bonuses are gated by per\-platform “valid state” predicates so that they cannot be earned without producing a usable mutation\.

#### \(R3\) Penalties\.

We subtract for behaviour that would be costly or destructive against a real API: missing requiredcreate\_\*calls, invalid payload shape, duplicate creates, hallucinated tool names, and excessive call counts beyond a per\-scenario budget\. Penalty magnitudes are scenario\-tuned \(e\.g\.−0\.25\-0\.25for a missing create in the cross\-product reward,−0\.30\-0\.30in the sub\-task reward\); the cross\-product values are listed in Table[2](https://arxiv.org/html/2607.01465#S4.T2)\. The final reward is clamped to\[0,1\]\[0,1\]\.

Table[2](https://arxiv.org/html/2607.01465#S4.T2)decomposes the cross\-product reward, the most complex; the single\-product rewards follow the same R1/R2/R3 template with per\-scenario magnitudes, and ticket\-transition is the known outlier \(Section[7](https://arxiv.org/html/2607.01465#S7)\)\.

Table 2\.Cross\-product reward decomposition\. Maximum positive budget is 1\.35 before clamping; penalties are subtracted before clamping to\[0,1\]\[0,1\]\.ComponentValueJira per\-arg correctness \(5×0\.105\\times 0\.10\)0\.50Confluence per\-arg correctness \(4×0\.104\\times 0\.10\)0\.40Jira structure: get\-before\-create, verify\-after0\.15Confluence structure: get\-before\-create, verify\-after0\.15Cross\-platform completion bonus \(gated on both\)0\.15Missingcreate\_issue/create\_page−0\.25\-0\.25eachInvalid Jira / Confluence payload−0\.25\-0\.25eachDuplicatecreate\_issue/create\_page−0\.15\-0\.15eachHallucinated tool name−0\.15\-0\.15eachExcess calls beyond budget \(\>6\>6\)−0\.05\-0\.05each

## 5\.Training Setup

We train Qwen3\-1\.7B\(Yanget al\.,[2025](https://arxiv.org/html/2607.01465#bib.bib16)\)\(ticket transition\) and Qwen3\.5\-4B\(Qwen Team,[2026](https://arxiv.org/html/2607.01465#bib.bib10)\)\(all other scenarios\) with GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2607.01465#bib.bib12)\)as implemented in TRL\(von Werraet al\.,[2020](https://arxiv.org/html/2607.01465#bib.bib14)\)\. We use 4–16 generations per prompt \(4 for the larger\-prompt\-budget scenarios; 16 for ticket\-transition and labeling, which expose fewer distinct prompts per batch\), two optimisation iterations per batch, BF16 weights, gradient accumulation of 4, and a maximum completion length of 2048–4096 tokens depending on scenario\. A reward\-convergence callback stops training once the epoch\-average reward delta falls below 0\.01 for a per\-scenario patience window \(5–10 epochs\), rotating checkpoints to retain the most recent two on disk\.

#### Hardware\.

The 1\.7B runs fit on a single Nvidia RTX PRO 6000 Blackwell; the 4B runs use two — intentionally sized for a small on\-prem or single\-node cloud budget\.

## 6\.Results

![Refer to caption](https://arxiv.org/html/2607.01465v1/x1.png)Figure 2\.Prompted\-baseline vs\. RLVR\-trained average reward per scenario, scored by the identical reward\. Baselines \(Qwen3\-1\.7B, Qwen3\.5\-4B\) run via the HuggingFace Inference Router; “RLVR\-trained” is the final\-epoch GRPO average\. Ticket\-transition \(asterisked\) saturates trivially and is not informative on its own \(Section[7](https://arxiv.org/html/2607.01465#S7)\)\.Grouped bar chart of average reward for five scenarios, with three bars per scenario: prompted Qwen3\-1\.7B, prompted Qwen3\.5\-4B, and the RLVR\-trained policy\. The RLVR\-trained bars reach 0\.95–1\.00 across scenarios, above both baselines except on ticket transition where the 4B baseline already saturates\.
![Refer to caption](https://arxiv.org/html/2607.01465v1/x2.png)Figure 3\.Average group reward vs\. generation\-batch index, four scenarios trained with GRPO on Qwen3\.5\-4B\. Solid lines: 5\-batch moving average; shaded bands: per\-batch min/max over the four\-generation group\. All four cross 0\.95 within≤100\\leq 100batches\.Line plot of average group reward versus training step for four scenarios\. Cross\-product and sub\-task creation rise fastest, page labeling rises more gradually to about 0\.95, and page creation stays flat near 1\.0; shaded min/max bands narrow as training proceeds\.

### 6\.1\.Prompted baselines vs\. RLVR\-trained

We score prompted Qwen3\-1\.7B and Qwen3\.5\-4B against the same reward functions used during training, run via the HuggingFace Inference Router \(temperature 0\.7, 25\-turn cap\), with synthetic state reset between every task and tool\-call arguments normalised to dicts so baseline scores are directly comparable\. Figure[3](https://arxiv.org/html/2607.01465#S6.F3)reports the average reward per scenario\. We keep ticket\-transition only as a transparency control — its reward saturates \(Section[7](https://arxiv.org/html/2607.01465#S7)\), so the prompted 4B already scores 1\.00; the four non\-degenerate scenarios carry every claim below\.

On the four scenarios whose rewards are non\-degenerate, the RL\-trained policy beats prompted Qwen3\.5\-4B by a meaningful margin\. The largest absolute lifts come on the most schema\-heavy creation tasks: Confluence page creation jumps from 0\.35 to 1\.00 \(\+0\.65\+0\.65\), Confluence labeling from 0\.52 to 0\.95 \(\+0\.43\+0\.43\), and Jira sub\-task creation from 0\.68 to 1\.00 \(\+0\.32\+0\.32\)\. On the cross\-product task, where the 4B baseline is already strong at 0\.92, RL closes the residual tail to 1\.00 — every one of the 12 evaluation prompts becomes a perfect rollout rather than 8 of 12\. Failure modes in the prompted baselines mirror the objective mismatch flagged in Section[1](https://arxiv.org/html/2607.01465#S1): plausible\-looking calls with the wrong nested key, dropped required fields, and early\-stopping after the read\.

### 6\.2\.Training dynamics

Figure[3](https://arxiv.org/html/2607.01465#S6.F3)plots the GRPO reward trajectories \(ticket\-transition is omitted; its logging schema was coarser\)\. The three from\-scratch scenarios reach≥0\.95\\geq 0\.95in under 100 generation batches; the Confluence page\-creation curve is a warm\-started stability trace \(flat near 1\.0 across 16 batches\)\. The shaded bands report per\-batch min/max across the four\-generation group, and contract over training as the policy stops emitting low\-reward rollouts\.

Appendix[A](https://arxiv.org/html/2607.01465#A1)contrasts the two dominant pre\-training failure modes \(early stop after the read calls; mis\-shaped nested parent arg followed by a chain of duplicatecreate\_\*retries\) against the converged trace\. These traces are human\-inspectable evidence that the gain is behavioural, not reward\-hacking: the converged policy reads before it writes, shapes nested arguments correctly, and stops without hallucinated or duplicate calls — visible in the call sequence, independent of the scalar\.

## 7\.Limitations and Future Work

#### Narrow prompt coverage; no held\-out evaluation\.

Each scenario ships with 6–12 training prompts\. Convergence to≥0\.95\\geq 0\.95average reward on those prompts shows that GRPO with the proposed verifiable reward can fit the training distribution; it does not measure generalisation to unseen prompts\. A held\-out set — same schema, new entities, summaries, and parent–child combinations — is the immediate next step\.

#### Reward\-shape artifacts\.

The ticket\-transition reward awards\+0\.25\+0\.25per tool call, clamped at 1\.0, with no penalty for excess or hallucinated calls, so the prompted 4B already saturates at 1\.00 \(one rollout invokedtransition\_issuenine times and still maxed out\)\. A separate type bug compares the gold transition id as an integer against a string\-typed schema, denying the schema\-compliant 1\.7B a bonus the 4B collects\. We keep these shapes verbatim — they are the rewards under which the rollouts were generated — but flag per\-scenario reward\-hacking audits as unfinished work\.

#### Reward engineering does not scale by hand\.

Each reward was hand\-tuned through pilot GRPO loops that exposed one of three failure modes: no convergence \(signal too weak\), reward hacking, or premature plateau \(group\-relative advantages collapse on a partial solution\)\. Scaling this to hundreds of Atlassian endpoints is the load\-bearing scalability question\. Much of each reward is mechanically derivable, though — R1 per\-argument terms from a schema’srequiredfields, R3 type penalties from itstype/enumconstraints, leaving only R2 structural bonuses to hand\-tune — so OpenAPI\-driven reward synthesis, checked by pilot\-run signal or an LLM\-as\-reward\-designer, is the most important follow\-up\.

#### Live\-API transfer\.

Schema fidelity at the wire format does not guarantee robustness to real\-world failure modes \(rate limits, eventual consistency, permission errors, partial responses\); the synthetic environments are intended as a cheap inner loop paired with a smaller live\-tenant validation loop: record–replay of captured responses, fault injection \(429/403, stale eventual\-consistency reads\), and a low\-volume sandbox\-tenant canary\.

#### Stronger baselines and open questions\.

Our prompted baselines hold the base model fixed, isolating the post\-training objective; they are a controlled test, not a claim that RL is the only route\. SFT on successful traces, ReAct\- or plan\-execute\-style scaffolding, and a frontier model with a Python REPL over these environments are the next rungs — separating whether RL is necessary from merely sufficient — alongside an order\-of\-magnitude more prompts, larger base models, and multi\-turn user simulators\.

#### Reproducibility\.

Rewards are deterministic functions of the tool\-call list, so any rollout re\-scores post\-hoc against the same checker; Tables[1](https://arxiv.org/html/2607.01465#S3.T1)–[2](https://arxiv.org/html/2607.01465#S4.T2), Sections[3\.3](https://arxiv.org/html/2607.01465#S3.SS3)and[5](https://arxiv.org/html/2607.01465#S5), and the Appendix[A](https://arxiv.org/html/2607.01465#A1)rollouts specify the substrate at the level needed to reimplement it\.

## References

- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman \(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§1](https://arxiv.org/html/2607.01465#S1.p2.1)\.
- J\. Da, C\. Wang, X\. Deng, Y\. Ma, N\. Barhate, and S\. Hendryx \(2025\)Agent\-rlvr: training software engineering agents via guidance and environment rewards\.arXiv preprint arXiv:2506\.11425\.Cited by:[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px2.p1.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§1](https://arxiv.org/html/2607.01465#S1.p2.1),[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px2.p1.1)\.
- C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. Narasimhan \(2024\)SWE\-bench: can language models resolve real\-world github issues?\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2607.01465#S1.p1.1),[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px3.p1.1)\.
- N\. Lambert, J\. Morrison, V\. Pyatkin, S\. Huang, H\. Ivison, F\. Brahman, L\. J\. V\. Miranda, A\. Liu, N\. Dziri, S\. Lyu,et al\.\(2024\)Tülu 3: pushing frontiers in open language model post\-training\.arXiv preprint arXiv:2411\.15124\.Cited by:[§1](https://arxiv.org/html/2607.01465#S1.p2.1),[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px2.p1.1)\.
- X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang, S\. Zhang, X\. Deng, A\. Zeng, Z\. Du, C\. Zhang, S\. Shen, T\. Zhang, Y\. Su, H\. Sun, M\. Huang, Y\. Dong, and J\. Tang \(2024\)AgentBench: evaluating llms as agents\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2607.01465#S1.p1.1),[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px3.p1.1)\.
- J\. Lu, T\. Holleis, Y\. Zhang, B\. Aumayer, F\. Nan, F\. Bai, S\. Ma, S\. Ma, M\. Li, G\. Yin, Z\. Wang, and R\. Pang \(2024\)ToolSandbox: a stateful, conversational, interactive evaluation benchmark for llm tool use capabilities\.arXiv preprint arXiv:2408\.04682\.Cited by:[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px3.p1.1)\.
- C\. Qian, E\. C\. Acikgoz, Q\. He, H\. Wang, X\. Chen, D\. Hakkani\-Tür, G\. Tur, and H\. Ji \(2025\)ToolRL: reward is all tool learning needs\.arXiv preprint arXiv:2504\.13958\.Cited by:[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Qin, S\. Liang, Y\. Ye, K\. Zhu, L\. Yan, Y\. Lu, Y\. Lin, X\. Cong, X\. Tang, B\. Qian,et al\.\(2024\)ToolLLM: facilitating large language models to master 16000\+ real\-world apis\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2607.01465#S1.p1.1),[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px1.p1.1)\.
- Qwen Team \(2026\)Qwen3\.5: hybrid mixture\-of\-experts and linear\-attention foundation models\.Note:[https://huggingface\.co/collections/Qwen/qwen35](https://huggingface.co/collections/Qwen/qwen35)Cited by:[§5](https://arxiv.org/html/2607.01465#S5.p1.1)\.
- T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. Scialom \(2023\)Toolformer: language models can teach themselves to use tools\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2607.01465#S1.p1.1),[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[4th item](https://arxiv.org/html/2607.01465#S1.I1.i4.p1.2),[§1](https://arxiv.org/html/2607.01465#S1.p2.1),[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.01465#S5.p1.1)\.
- H\. Trivedi, T\. Khot, M\. Hartmann, R\. Manku, V\. Dong, E\. Li, S\. Gupta, A\. Sabharwal, and N\. Balasubramanian \(2024\)AppWorld: a controllable world of apps and people for benchmarking interactive coding agents\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§1](https://arxiv.org/html/2607.01465#S1.p1.1),[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px3.p1.1)\.
- L\. von Werra, Y\. Belkada, L\. Tunstall, E\. Beeching, T\. Thrush, N\. Lambert, S\. Huang, K\. Rasul, and Q\. Gallouédec \(2020\)TRL: transformer reinforcement learning\.Note:[https://github\.com/huggingface/trl](https://github.com/huggingface/trl)Cited by:[4th item](https://arxiv.org/html/2607.01465#S1.I1.i4.p1.2),[§5](https://arxiv.org/html/2607.01465#S5.p1.1)\.
- X\. Wang, Y\. Chen, L\. Yuan, Y\. Zhang, Y\. Li, H\. Peng, and H\. Ji \(2024\)Executable code actions elicit better llm agents\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2607.01465#S1.p1.1),[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Yang, B\. Yang,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[4th item](https://arxiv.org/html/2607.01465#S1.I1.i4.p1.2),[§5](https://arxiv.org/html/2607.01465#S5.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2607.01465#S1.p1.1),[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. Neubig \(2024\)WebArena: a realistic web environment for building autonomous agents\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2607.01465#S1.p1.1),[§2](https://arxiv.org/html/2607.01465#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix APre\- and Post\-Training Tool\-Call Sequences

The listings below are the ordered tool\-call sequences \(tool name and argument dictionary\) emitted by the policy during a single cross\-product rollout, lifted fromatlassian\_rollouts\.jsonwith tool responses and intermediate reasoning omitted\. Listings[1](https://arxiv.org/html/2607.01465#LST1)and[2](https://arxiv.org/html/2607.01465#LST2)are representative pre\-convergence failure modes; Listing[3](https://arxiv.org/html/2607.01465#LST3)is a converged success\.

#### Failure mode A: early stop\.

Listing[1](https://arxiv.org/html/2607.01465#LST1)is a generation\-batch 0 rollout \(reward=0\.00=0\.00\)\. The policy correctly reads both parent resources, then halts before issuing the two required write calls\. This is the dominant pre\-training failure mode in the cross\-product scenario and the one captured by the qualitative claim in Section[6](https://arxiv.org/html/2607.01465#S6)\.

Listing 1:Pre\-training rollout, step 0, reward = 0\.00\. The policy stops after the read calls and never emits acreate\_\*\.get\_issue\(issueIdOrKey="XYZ\-456"\)

get\_page\(page\_id="100"\)

#### Failure mode B: structural OK, nested args wrong\.

Listing[2](https://arxiv.org/html/2607.01465#LST2)is a generation\-batch 2 rollout \(reward=0\.52=0\.52\)\. The policy emits the full read\-then\-write sequence, but the firstcreate\_issuepassesparentas a bare string"ABC\-123"instead of\{"key": "ABC\-123"\}, the Jira\-v3 wire format\. The policy then issues five morecreate\_issuecalls trying alternate shapes \(string parent, dropped assignee,projectKeyinstead ofproject\.key\); each collects the duplicate\-create penalty\.

Listing 2:Pre\-training rollout, step 2, reward = 0\.52\. Five duplicatecreate\_issuecalls probe alternate argument shapes after the first fails the payload\-shape check\.get\_issue\(issueIdOrKey="ABC\-123"\)

get\_page\(page\_id="789"\)

create\_issue\(fields=\{"project":\{"key":"ABC"\},

"parent":"ABC\-123",

"summary":"WriteReleaseNotes",

"issuetype":\{"id":"10003"\},

"assignee":\{"accountId":"admin123"\}\}\)

create\_page\(spaceId="PAY",title="ReleaseNotes1\.0",

parentId="789",

body=\{"representation":"storage",

"value":"<h2\>ReleaseNotes1\.0</h2\>\.\.\."\}\)

create\_issue\(fields=\{\.\.\.,"parent":"ABC\-123",

"assignee":"admin123"\}\)

create\_issue\(fields=\{\.\.\.,"parent":"ABC\-123"\}\)

create\_issue\(fields=\{"projectKey":"ABC",

\.\.\.,"parent":"ABC\-123"\}\)

create\_issue\(fields=\{\.\.\.,"parent":"ABC\-123"\}\)

#### Success: converged\.

Listing[3](https://arxiv.org/html/2607.01465#LST3)is a generation\-batch 53 rollout \(reward=1\.00=1\.00\)\. The policy reads both parents, then issues exactly onecreate\_issueand onecreate\_pagewith correct nested shape — including theissuetype\.id="10003"subtask discriminator that the system prompt names but does not demonstrate\.

Listing 3:Post\-training rollout, step 53, reward = 1\.00\. Read both parents, then one correctly shaped write per platform\.get\_issue\(issueIdOrKey="ABC\-123"\)

get\_page\(page\_id="789"\)

create\_issue\(fields=\{

"project":\{"key":"ABC"\},

"parent":\{"key":"ABC\-123"\},

"summary":"WriteReleaseNotes",

"issuetype":\{"id":"10003"\},

"assignee":\{"accountId":"admin123"\}\}\)

create\_page\(spaceId="PAY",title="ReleaseNotes1\.0",

parentId="789",

body=\{"representation":"storage",

"value":"<p\>\.\.\.</p\>"\}\)