Speculative Macro Commit for Faster Tool-Using Agents

# Speculative Macro Commit for Faster Tool-Using Agents
Source: [https://arxiv.org/html/2609.03236](https://arxiv.org/html/2609.03236)
###### Abstract

Tool\-using LLM agents spend wall\-clock time not only on model inference but also in serial action–observation turns, where each tool call, environment transition, and observation can delay subsequent decisions\. We introduceSpeculative Macro Commit\(SMC\), a runtime mechanism for a two\-tier agent system: a large authoritative actor model produces the official trajectory, while a faster speculative drafter model continuously predicts and executes future action chains on an isolated environment snapshot\. SMC mines recurring multi\-action skeletons from training traces and stores them in a macro library used to match against action chains predicted by the drafter at runtime\. When the actor’s next tool call matches the first drafted action, SMC commits the remaining pre\-executed draft steps, together with their observations, to the official trajectory\. Using Qwen3\.5\-27B INT4 as the authoritative actor model and Qwen3\.5\-4B as the speculative drafter model, SMC matches the sequential agent’s overall accuracy while reducing latency by 10\.23% over the Speculative Actions \(SA\) baseline and 18\.59% over sequential execution on theτ2\\tau^\{2\}\-Bench Telecom subset\. On AppWorld, SMC reduces wall time by 7\.7% over SA baseline and 44\.9% over sequential execution, with a small reduction in task completion\. Overall, SMC provides a practical way to reuse multi\-step speculative execution and reduce agent latency beyond single\-step speculative actions\. Our code is publicly available[here](https://github.com/zeyuliu1037/speculative-macro-commit)\.

Zeyu Liu⋆Souvik Kundu†Peter A\. Beerel⋆⋆University of Southern California, Los Angeles, USA†Intel Labs, San Diego, USA
Index Terms—LLM Agents, speculative execution, inference latency

## 1Introduction

Language agents are becoming a practical interface to software, data, web, and customer\-service workflows\[[11](https://arxiv.org/html/2609.03236#bib.bib1),[5](https://arxiv.org/html/2609.03236#bib.bib2),[10](https://arxiv.org/html/2609.03236#bib.bib3)\]\. Unlike a model that makes a single prediction, a tool agent runs a loop: it calls the model to choose an action, executes that action in an environment, observes the result, and conditions its next decision on what it observed\[[12](https://arxiv.org/html/2609.03236#bib.bib9)\]\. This iterative interaction gives agents much of their flexibility, but it also creates a wall\-clock bottleneck that model throughput alone does not capture: a single task can wait through dozens of sequential turns even when the model server is well batched\.

This latency has several sources\. Repeated large\-model calls reprocess long contexts and stress cache management\[[6](https://arxiv.org/html/2609.03236#bib.bib8)\]\. Tool orchestration adds scheduling, state, and environment\-lifecycle overhead\[[3](https://arxiv.org/html/2609.03236#bib.bib7)\]\. The tools and environments can themselves dominate a request; PASTE reports tool execution at 35% to 61% of total latency in representative agents\[[8](https://arxiv.org/html/2609.03236#bib.bib6)\]\. Across these sources, the main constraint is sequential dependence: the next prompt usually cannot be built until the previous action returns its observation\. Token\-level speculative decoding\[[4](https://arxiv.org/html/2609.03236#bib.bib10)\]drafts several tokens with a cheap model and verifies them with the expensive one, but it operates strictly inside one model call and cannot remove this cross\-step dependency\.

Speculative Actions \(SA\) lifts the draft\-then\-verify idea from tokens to actions\[[13](https://arxiv.org/html/2609.03236#bib.bib4)\]\. A fast speculative drafter predicts likely future actions while a slower authoritative actor computes the next verified action; when the actor later agrees, an action that was already launched can be committed\. However, the core SA mechanism only accepts progress at the granularity of adjacent action steps: a correct guess lets the executor reuse work for the next step, while naively longer lookahead has limited benefits\. Therefore, how to effectively commit multi\-step actions still remains underexplored\. A complementary line of work observes that tool\-agent traces contain repeated sequences of tool calls\. AWO\[[1](https://arxiv.org/html/2609.03236#bib.bib5)\], for example, mines recurring tool\-call sequences and turns them into deterministic composite meta\-tools that bundle multiple actions into one invocation\. This can remove several reasoning steps when the model chooses the right meta\-tool\. In our experiments, however, simply exposing mined action sequences as additional callable tools is unreliable: open models like Qwen3\.5\-27B\[[7](https://arxiv.org/html/2609.03236#bib.bib13)\]rarely select these mined macros\. This raises the central question of the paper:

> *Can recurring action patterns reduce agent latency without requiring the model to choose new meta tools?*

Our answer isSpeculative Macro Commit\(SMC\), where a*macro*is a recurring multi\-action pattern mined from execution traces\. A*commit*means accepting already executed draft tool calls and their returned results into the agent’s real execution history, so that the large model does not need to choose and execute those steps again\. When one macro from the mined macro library is identified in this long action chain, and the first action of the macro matches the authoritative action, SMC attempts a*macro commit*: it commits the remaining matched draft steps and their observations into the official trajectory, skipping the corresponding large\-model calls as well as environment delays\. As a macro commit does not ask the actor to regenerate and verify every skipped step, SMC is approximate rather than strictly lossless but is empirically shown to preserve task quality while reducing latency\.

1. 1\.An end\-to\-end runtime for action\-level speculation\.We build a two\-tier runtime in which an authoritative actor model and a speculative drafter model run concurrently throughout complete long\-horizon tool\-agent tasks\. The executor measures actual wall\-clock latency, lets the drafter run ahead on draft action–observation chains, and falls back to ordinary execution when speculation cannot be used\.
2. 2\.Speculative Macro Commit\.We introduce SMC, which shifts mined recurring action patterns from model\-side macro selection to executor\-side validation\. The executor first identifies macro matches in the drafter’s pre\-executed action chain; after the authoritative actor agrees with the first matched draft action, SMC can commit the remaining matched draft steps and their observations into the official trajectory\. This skips the corresponding large\-model calls and avoids waiting again for already returned tool results, making SMC an approximate but guarded latency optimization\.
3. 3\.Evaluation on full tool\-agent benchmarks\.With Qwen3\.5\-27B INT4 as the authoritative actor and Qwen3\.5\-4B as the speculative drafter on theτ2\\tau^\{2\}\-bench Telecom subset, the SMC run matches the single\-model baseline’s reward while cutting latency by 18\.59%; and reduces latency by 10\.23% over SA\. On AppWorld benchmark, SA reduces the latency by 40\.37% compared with the baseline, and SMC further reduces wall time by 7\.64% with a small accuracy tradeoff\.

## 2Related Work

Speculative decoding and speculative agent execution\.Speculative decoding accelerates a single model call by letting a cheap drafter model propose tokens that a target model verifies in batches\[[4](https://arxiv.org/html/2609.03236#bib.bib10)\]\. This line of work reduces decoding latency inside one model invocation, but a ReAct\-style tool agent, which interleaves model reasoning with tool calls over many steps\[[12](https://arxiv.org/html/2609.03236#bib.bib9)\], also pays latency across invocations, since each tool result must return before the next prompt can be formed\. Speculative Actions extends the draft\-and\-verify idea from tokens to agent actions, launching predicted future API calls in parallel with an authoritative actor and committing them only when predictions match\[[13](https://arxiv.org/html/2609.03236#bib.bib4)\]\. Our SMC shares this action\-level view but differs in what a commit covers: rather than verifying one action at a time, it commits several already executed draft steps, trading the lossless guarantee of one\-step verification for a guarded multi\-step approximation\.

SMC builds on the same actor–speculator setting but uses a different commit rule\. After the actor verifies the first drafted action, SMC may commit a matched suffix of already executed action–observation pairs without verifying each skipped action with the actor\. This enables multi\-step runtime skipping, at the cost of making SMC approximate rather than strictly lossless\.

Speculative tool execution systems\.Recent LLM\-serving systems attack the same serial LLM–tool loop from complementary angles\. PASTE speculatively executes likely tool calls while the model is still computing, exploiting stable application\-level control flow and data dependencies\[[8](https://arxiv.org/html/2609.03236#bib.bib6)\], while ThunderAgent\[[3](https://arxiv.org/html/2609.03236#bib.bib7)\]and KVFlow\[[6](https://arxiv.org/html/2609.03236#bib.bib8)\]reduce latency through program\-aware serving and prefix/cache management without changing which actions enter the trajectory\. These systems motivate the runtime setting of our work: latency depends on whether useful tool work can move off the serial critical path, not only on model calls\.

Workflow compression and meta\-tools\.Agent Workflow Optimization \(AWO\) mines recurrent tool\-call sequences from traces and registers the resulting composites as meta\-tools available to the model\[[1](https://arxiv.org/html/2609.03236#bib.bib5)\]\. The key difference is the interface: AWO changes the model\-visible toolset and lets the model choose when to call a composite, whereas our SMC keeps the mined steps as hidden runtime state and commits them only after the anchor call is verified\.

## 3Speculative Macro Commit

### 3\.1Executor Roles and State

SMC is implemented in the*agent executor*, the program that runs the tool\-agent loop\. Given the current task history, the executor prompts the model for the next tool call, executes that call, records the returned result, and repeats this process until the task finishes or the step budget is exhausted\. The executor is therefore distinct from both the language model and the tool environment: it alone decides which action–result pairs become part of the task history seen by future model calls\.

SMC augments this executor with two models that run concurrently\. Theauthoritative actoris the large model whose tool calls define the real execution of the agent\. Thespeculative drafteris a smaller and faster model that runs ahead of the actor, trying to speculate short future tool\-call sequences from the committed historyHH\. The executor maintainsHH, consisting of the tool calls and returned results that future actor prompts will condition on, and a live task stateEE, which is the state reached by executing the calls inHH\. The drafter’s calls are executed in an isolated draft stateEiE^\{i\}, so they produce returned results without immediately changing the live task state\. We write the drafter’s current output as a draft chainQ=\(q1,…,qD\)Q=\(q\_\{1\},\\ldots,q\_\{D\}\), where eachqiq\_\{i\}contains a drafted tool call and the result of executing it in the draft state\. A*macro*is a recurring multi\-action pattern mined from past executions\. A*commit*is the executor operation that appends executed tool calls and results pairs toHHand advancesEEconsistently with them\. When no macro commit is performed,HHgrows in the usual one\-step way: the executor appends the authoritative actor’s tool call and its returned result\. SMC changes this loop through a single addition, the*macro commit rule*of Section[3\.3](https://arxiv.org/html/2609.03236#S3.SS3), which under specific conditions lets one actor step be followed by several already executed draft steps\.

When the macro commit rule is disabled, the same executor still runs the actor and drafter concurrently and still reuses single\-step speculative work: whenever the actor proposes the same call as the first drafted step, the drafter’s already executed call and result are committed directly\. We call this configuration the*SA\-only baseline*and use it as the matched\-runtime comparison in Section[4](https://arxiv.org/html/2609.03236#S4)\.

### 3\.2Macro Mining

SMC first extracts candidate multi\-action patterns from successful training traces\. It then evaluates these candidates using drafter predictions from sampled training states and keeps only those that satisfy the selection criteria below\. The selected candidates form the runtime*macro library*ℳ\\mathcal\{M\}\. Each macro records an ordered tool\-call sequence together with the argument structure needed to match it, replacing task\-specific values such as user IDs, document IDs, or timestamps with slots\. This normalization lets one macro cover repeated behavior across tasks without requiring identical concrete arguments\.

Because a macro is useful only if the drafter can reproduce it at deployment, we label candidates under the same actor–drafter setting used at runtime\. For each sampled training state, the drafter proposes a short future action chain, which we compare against a reference future trajectory from that state\. A proposal is labeled correct when its tool\-call sequence matches the reference future trajectory after task\-specific argument values are normalized as described above\. Each candidate macro thus carries two kinds of evidence: how often the pattern occurs, and how reliably the drafter reproduces it when the relevant state is encountered\.

For a candidate macromm, letnmn\_\{m\}be the number of labeled opportunities andkmk\_\{m\}be the number of correct drafter proposals\. We compute a conservative reliability estimate using the lowerδ\\delta\-quantile of a Beta posterior,

p¯m=FBeta⁡\(km\+1,nm−km\+1\)−1​\(δ\)\\underline\{p\}\_\{m\}=F^\{\-1\}\_\{\\mathrm\{Beta\}\(k\_\{m\}\+1,\\;n\_\{m\}\-k\_\{m\}\+1\)\}\(\\delta\)\(1\)and retainmmonly whennm≥nminn\_\{m\}\\geq n\_\{\\min\}andp¯m≥τ\\underline\{p\}\_\{m\}\\geq\\tau\. The thresholdsnminn\_\{\\min\}andτ\\tauare selected on held\-out tasks using the same executor configuration as the final evaluation\.

These criteria are used only to filter the candidate macros\. Whether a retained macro is committed is determined separately at runtime by the macro commit rule\.

### 3\.3Macro Commit Rule

At runtime, the drafter continuously extends the draft chainQQfrom the currentHH\. Consider a retained macrom=\(u1,…,uK\)m=\(u\_\{1\},\\ldots,u\_\{K\}\)\. A runtime alignment matchesu1,…,uju\_\{1\},\\ldots,u\_\{j\}to a suffix ofHHand the remainingℓ=K−j\\ell=K\-jactions to the draft prefixq1,…,qℓq\_\{1\},\\ldots,q\_\{\\ell\}\. The library storesmmonce;jjandℓ\\ellare determined by the state reached at runtime\. An alignment is only a commit candidate; it does not changeHHby itself\. The executor waits for the actor’s next tool call\. If that call matchesq1q\_\{1\}, we callq1q\_\{1\}the*anchor call*\. The drafter has already executedq1q\_\{1\}, so the executor commits it together with the drafter’s returned result; onlyq2,…,qℓq\_\{2\},\\ldots,q\_\{\\ell\}are committed without individual actor confirmation\. This skipsℓ−1\\ell\-1actor decisions, so candidate selection requiresℓ−1≥Lmin\\ell\-1\\geq L\_\{\\min\}\. The floor cannot be enforced fromKKalone: for the same four\-action macro, aligning one versus two actions withHHleaves three versus two draft actions and therefore skips two versus one actor decisions\.

Online checks then discard stale draft work and apply benchmark\-specific action safety\. In AppWorld, manually specified API\-name rules reject irreversible or unknown calls, while forkable mutations require a matching live\-state replay\. If all checks pass, the executor appendsq2,…,qℓq\_\{2\},\\ldots,q\_\{\\ell\}and their returned results toHHand advancesEEaccordingly\. The drafter is restarted only when the committed history actually diverges from the draft chain, that is, when the actor’s call does not matchq1q\_\{1\}\. In that case the executor executes the actor’s call in the live state, waits for its result, commits this step, discards the draft work, and restarts the drafter from the updated history\. Whenever the actor confirmsq1q\_\{1\}, the executor instead reuses the drafter’s execution: it removes the committed steps fromQQ\(q1q\_\{1\}alone if no macro commit,q1,…,qrq\_\{1\},\\ldots,q\_\{r\}if one does\), and the drafter keeps extending the remainder of the draft chain, which the executor continues to search for macro matches in later iterations\. If candidate selection finds no eligible alignment \(including the depth floor\), or if an online check rejects the selected actions, only the anchor step is committed and the executor continues as the SA\-only baseline\. Algorithm[1](https://arxiv.org/html/2609.03236#alg1)summarizes this rule\.

Algorithm 1Speculative Macro CommitExecutor1:macro library

ℳ\\mathcal\{M\}, live task state

EE, task

xx
2:authoritative actor

AactorA\_\{\\mathrm\{actor\}\}, speculative drafter

AdraftA\_\{\\mathrm\{draft\}\}
3:draft depth

DD, minimum skipped length

LminL\_\{\\min\}
4:

H←\[\]H\\leftarrow\[\\,\]⊳\\trianglerightcommitted task history

5:launch

AdraftA\_\{\\mathrm\{draft\}\}to maintain a draft chain

Q=\(q1,…,qD\)Q=\(q\_\{1\},\\ldots,q\_\{D\}\)from

HH
6:

Factor←SubmitActor​\(Aactor,H,E\)F\_\{\\mathrm\{actor\}\}\\leftarrow\\textsc\{SubmitActor\}\(A\_\{\\mathrm\{actor\}\},H,E\)
7:whiletask not finished and budget remainsdo

8:

aactor←Wait​\(Factor\)a\_\{\\mathrm\{actor\}\}\\leftarrow\\textsc\{Wait\}\(F\_\{\\mathrm\{actor\}\}\)⊳\\trianglerightactor’s proposed tool call \(not yet executed\)

9:

Q←LatestDraftChainQ\\leftarrow\\textsc\{LatestDraftChain\}
10:ifFirstActionMatch\(

aactor,q1a\_\{\\mathrm\{actor\}\},q\_\{1\}\)then

11:Commit\(

H,E,q1H,E,q\_\{1\}\)⊳\\trianglerightreuse the drafter’s executed call and result

12:

r←1r\\leftarrow 1
13:

c←SelectMacro​\(ℳ,H,Q,Lmin\)c\\leftarrow\\textsc\{SelectMacro\}\(\\mathcal\{M\},H,Q,L\_\{\\min\}\)
14:if

c≠⊥c\\neq\\botthen

15:

\(m,r′\)←c\(m,r^\{\\prime\}\)\\leftarrow c⊳\\trianglerightmmmatchesq1,…,qr′q\_\{1\},\\ldots,q\_\{r^\{\\prime\}\}andr′−1≥Lminr^\{\\prime\}\-1\\geq L\_\{\\min\}

16:ifOnlineChecks\(

m,H,\(q2,…,qr′\),x,Em,H,\(q\_\{2\},\\ldots,q\_\{r^\{\\prime\}\}\),x,E\)then

17:for

q∈\(q2,…,qr′\)q\\in\(q\_\{2\},\\ldots,q\_\{r^\{\\prime\}\}\)do

18:Commit\(

H,E,qH,E,q\)

19:endfor

20:

r←r′r\\leftarrow r^\{\\prime\}
21:endif

22:endif

23:PopDraftPrefix\(

Q,rQ,r\)⊳\\trianglerightdrafter keeps extending the remainder ofQQ

24:else

25:

o←Execute​\(aactor,E\)o\\leftarrow\\textsc\{Execute\}\(a\_\{\\mathrm\{actor\}\},E\)⊳\\trianglerightrun the actor’s call in the live state

26:Commit\(

H,E,\(aactor,o\)H,E,\(a\_\{\\mathrm\{actor\}\},o\)\)

27:discard draft work; restart

AdraftA\_\{\\mathrm\{draft\}\}from

HH
28:endif

29:

Factor←SubmitActor​\(Aactor,H,E\)F\_\{\\mathrm\{actor\}\}\\leftarrow\\textsc\{SubmitActor\}\(A\_\{\\mathrm\{actor\}\},H,E\)
30:endwhile

Overall, unlike registered meta\-tools, the actor model never emits a macro call: the actor sees the same tool schema as the baseline, and a macro step is a runtime decision over already executed drafter work, not a new action the model must learn\.

## 4EXPERIMENTAL Results

### 4\.1Setup

Models and serving\.The authoritative actor is a quantized Qwen3\.5\-27B model, sized to fit on a single GPU; the speculative drafter is Qwen3\.5\-4B\[[7](https://arxiv.org/html/2609.03236#bib.bib13)\], and both use greedy decoding\. The sequential baseline runs only the actor on a single GPU\. The two speculative runtimes use three GPUs: the actor, an actor\-class replica that serves speculative requests so they do not queue behind the authoritative request stream, and the drafter\. The SA\-only runtime \(hereafter SA\) and SMC share this serving configuration and differ only in whether the macro commit rule of Section[3\.3](https://arxiv.org/html/2609.03236#S3.SS3)is enabled\.

Benchmarks and metrics\.We evaluate on two full tool\-agent benchmarks,τ2\\tau^\{2\}Telecom\[[2](https://arxiv.org/html/2609.03236#bib.bib12)\]and AppWorld\[[9](https://arxiv.org/html/2609.03236#bib.bib11)\]\. We use binary task accuracy forτ2\\tau^\{2\}Telecom and task\-goal completion \(TGC\) for AppWorld as performance metrics, and average per\-task wall time as the latency metric\.

### 4\.2Main Results

As shown in Table[1](https://arxiv.org/html/2609.03236#S4.T1), onτ2\\tau^\{2\}Telecom, SA reduces average latency from 27\.60s to 25\.03s per task, 9\.31% below the sequential baseline, and SMC further reduces it to 22\.47s, 18\.59% below the baseline and 10\.23% below SA\. The paired outcomes are unchanged: SMC agrees with the sequential baseline on every task \(2,274 correct and 11 incorrect under both runs\), and relative to SA it fixes one task and regresses none\. On this benchmark, macro commit converts committed draft steps into latency savings without changing any task outcome\.

AppWorld shows the same direction in a harder regime\. SA preserves TGC while reducing latency by 40\.37% relative to the sequential baseline, and SMC reaches to 195\.9s per task, 44\.93% below the baseline and 7\.64% below SA\. Unlikeτ2\\tau^\{2\}\-Bench, this additional speedup comes with a small TGC drop, from 70/168 to 68/168 tasks\.

Table 1:Full benchmark results\. Latency is average seconds per task;Δ\\Deltais relative to the sequential baseline within each benchmark\.τ2\\tau^\{2\}\-Bench accuracy is binary task accuracy; AppWorld accuracy is TGC\.BenchmarkRunAcc\.Lat\.Δ\\Delta\(%\)τ2\\tau^\{2\}TelecomBaseline99\.5227\.60–SA99\.4725\.03\-9\.31SMC99\.5222\.47\-18\.59AppWorldBaseline41\.67355\.7–SA41\.67212\.1\-40\.37SMC40\.48195\.9\-44\.93
### 4\.3Analysis of the AppWorld Gain

The smaller AppWorld gain over SA is not explained by a lack of macro opportunities\. Table[2](https://arxiv.org/html/2609.03236#S4.T2)reports skip density, defined as the number of skipped steps divided by total number of steps in the benchmark\. On AppWorld, SMC commits at least one macro on 62\.0% of tasks and successfully skips 512 actor steps, yielding a skip density of 3\.81%, close to the 3\.91% observed onτ2\\tau^\{2\}Telecom\. This similarity indicates that, after normalizing for differences in dataset size and step budget, SMC commits a comparable amount of skipped work on both benchmarks\. The two benchmarks therefore differ not in whether SMC finds reusable patterns, but in how reliably the saved work survives into end\-to\-end wall time\.

Table 2:Macro\-step coverage in the main benchmark runs\. Skip density divides skipped steps bytasks×\\timesmax\_steps\.BenchmarkCommit rateHitsSkippedSkip densityAppWorld62\.0%2195123\.81%τ2\\tau^\{2\}Telecom86\.2%3,3527,1543\.91%AppWorld is the harder case because task completion, retries, and the per\-task step cap \(Section[4\.1](https://arxiv.org/html/2609.03236#S4.SS1)\) all interact with latency\. An all\-task average therefore mixes three effects: skipped draft steps on trajectories that proceed as before, trajectories whose outcome changes, and steps that consume budget without issuing a tool call\. Table[3](https://arxiv.org/html/2609.03236#S4.T3)controls for the latter two\. On the 143 tasks where SA and SMC reach the same correctness outcome, SMC reduces total wall time by 13\.5%, substantially more than the 7\.64% all\-task gain\. On the 85 tasks where neither run emits a no\-tool\-call \(NTC\) step, the gain is 10\.7%\. This slice removes idle, recovery\-heavy trajectories whose wasted steps inflate wall time and dilute the macro saving It does not hide a side effect of SMC, as SMC produces fewer such steps than SA \(68 NTC steps over 31 tasks, versus 105 over 78 for SA\), so macro commit reduces rather than induces idle recovery\. On comparable trajectories, macro commit thus converts skipped draft steps into latency reduction more effectively than the aggregate number suggests; the all\-task figure is diluted by outcome shifts and by the idle trajectories isolated above\.

Table 3:AppWorld controlled wall\-time slices\.SubsetSASMCΔ\\Delta\(%\)Same\-acc\.28,920s25,014s\-13\.5NTC\-free16,375s14,625s\-10\.7

## 5Mechanism Ablations

### 5\.1Ablations

The main results suggest that a macro commit helps only when it satisfies the three conditions built into the commit rule of Section[3\.3](https://arxiv.org/html/2609.03236#S3.SS3)\. The reused steps must stay hidden from the model, since asking the model to select mined patterns adds a decision problem rather than skipping any actor call; the runtime must filter mined candidates aggressively, since a library match alone weakly predicts that the trajectory will follow the mined pattern; and a commit must skip enough actor calls on the critical path to overcome speculation, verification, and synchronization overhead\. The three ablations below test these conditions in turn\.

### 5\.2Interface: hidden runtime state

Table[4](https://arxiv.org/html/2609.03236#S5.T4)evaluates two simpler ways of using mined routines on a single GPU\. The first exposes mined patterns as registered meta\-tools\[[1](https://arxiv.org/html/2609.03236#bib.bib5)\], making reuse a model\-visible action\. The second leaves the model interface unchanged but commits matched patterns passively, without the online checks of the full commit rule\.

Neither alternative has desirable results\. Registered meta\-tools slightly increase latency and lower accuracy, because the actor almost never selects them: exposing mined routines as ordinary tools does not reliably skip actor calls, it poses an extra tool\-selection problem\. Passive committing has the opposite failure mode\. It cuts latency by 11\.34%, confirming that hidden runtime reuse can skip actor calls, but accuracy falls from 99\.52% to 96\.48%\. Hidden reuse alone is therefore insufficient: the runtime must also verify that the committed steps are anchored to the authoritative trajectory\. SMC keeps the latency benefit of hidden reuse while adding the online commit conditions\.

Table 4:Results onτ2\\tau^\{2\}\-Bench with single GPU setting\.RunAcc\.Lat\.Δ\\Delta\(%\)Baseline99\.5227\.60–Baseline \+ AWO\-like99\.3427\.89\+1\.05Baseline \+ passive96\.4824\.47\-11\.34
### 5\.3Commit Precision

The passive\-committing result raises the central safety question: how does SMC avoid committing incorrect mined patterns? Table[5](https://arxiv.org/html/2609.03236#S5.T5)answers with a staged audit on 100 held\-outτ2\\tau^\{2\}Telecom tasks\. For the first four rows, SMC suppresses the commit and lets the actor continue, so the realized trajectory provides an event\-level counterfactual for whether the candidate steps were exact; the final row reports task\-outcome preservation for the steps SMC actually committed\.

The mined library by itself is far too noisy to act as a commit rule: a library match reproduces the exact future steps in only 34\.6% of events\. Requiring the drafter to have already executed the steps raises precision to 70\.6%, since SMC no longer commits patterns that exist only as offline matches\. Verifying the anchor call raises it to 87\.9% by tying the candidate to the authoritative trajectory, and the depth guard \(Lmin=1L\_\{\\min\}=1, at least one committed step after the anchor\) raises it to 90\.4% while removing shallow commits unlikely to repay synchronization overhead\. Under full rule, all committed events preserve the outcome in this audit\. As noted in Section[3\.3](https://arxiv.org/html/2609.03236#S3.SS3), this is not a proof that every committed pattern is uniquely correct; it shows that the online checks turn a low\-precision library into a high\-precision set of commits on this held\-out audit\.

Table 5:Conditional precision of the commit stages on 100 held\-outτ2\\tau^\{2\}Telecom tasks\. Rows above the rule are event\-level exact\-match checks; the committed row is a task\-outcome check\.Filter stageEventsCorrect /outcome\-preservingLibrary match only1,968681/1,968 \(34\.6%\)\+ drafter\-executed885625/885 \(70\.6%\)\+ verified anchor call711625/711 \(87\.9%\)\+ depth guard \(Lmin=1L\_\{\\min\}=1\)343310/343 \(90\.4%\)Actually fired158158/158 \(100\.0%\)
### 5\.4Critical\-Path Depth

High precision alone does not guarantee speedup: a correct macro is unhelpful if it skips only one or two actor calls or skips calls that are off the critical path\. Table[6](https://arxiv.org/html/2609.03236#S5.T6)compares the final SMC runtime with an earlier runtime that committed more aggressively\. The legacy runtime commits almost twice as often as the final runtime and skips more total steps, yet it is 1\.64% slower than SA\. Raw macro count is thus the wrong objective: many additional legacy commits are too shallow or poorly aligned with already executed draft work, adding commit overhead without reducing end\-to\-end latency\. The final runtime suppresses 6,281 depth\-one opportunities, enforces theLminL\_\{\\min\}depth floor, and commits only steps already executed on the draft branch\. While it commits fewer macros and skips fewer total steps, it is 10\.23% faster than SA\. The useful unit is therefore not a macro hit but a guarded, sufficiently deep commit that skips actor calls on the critical path\.

Table 6:τ2\\tau^\{2\}Telecom full\-run comparison: raw macro count versus critical\-path commits\. Hits is the number of macro commits; Skip is the total committed draft steps after anchors\.RunAcc\.Lat\.HitsSkipΔ\\Delta\(%\)SA99\.4725\.0300–Legacy macro99\.5625\.446,41010,528\+1\.64Final SMC99\.5222\.473,3527,154\-10\.23

## 6Conclusion

We presented Speculative Macro Commit \(SMC\), a runtime mechanism that extends speculative action execution from single\-step reuse to committing several already executed draft steps after a verified anchor call\. Because a macro commit is approximate rather than losslessly verified, SMC guards it with conservative offline mining, a verified anchor call, a minimum skipped depth, and online state and argument checks\. Onτ2\\tau^\{2\}Telecom, SMC gives a quality\-preserving latency gain over the full dataset, beyond both the sequential baseline and an equal\-hardware speculation\-only baseline \(SA\); on AppWorld it gives a larger wall\-time gain with a small completion tradeoff, and larger gains on same\-accuracy slices\. More broadly, workflow macros can live as hidden runtime state rather than model\-visible tools, but they help only when the pattern is predictable enough to mine, the drafter has already executed it in the current run, and the commit survives the online checks\.

## References

- \[1\]\(2026\)Optimizing agentic workflows using meta\-tools\.External Links:2601\.22037,[Link](https://arxiv.org/abs/2601.22037)Cited by:[§1](https://arxiv.org/html/2609.03236#S1.p3.1),[§2](https://arxiv.org/html/2609.03236#S2.p4.1),[§5\.2](https://arxiv.org/html/2609.03236#S5.SS2.p1.1)\.
- \[2\]V\. Barres, H\. Dong, S\. Ray, X\. Si, and K\. Narasimhan\(2025\)τ2\\tau^\{2\}\-Bench: evaluating conversational agents in a dual\-control environment\.External Links:2506\.07982,[Link](https://arxiv.org/abs/2506.07982)Cited by:[§4\.1](https://arxiv.org/html/2609.03236#S4.SS1.p2.1)\.
- \[3\]H\. Kang, Z\. Li, X\. Yang, W\. Xu, Y\. Chen, J\. Wang, B\. Chen, T\. Krishna, C\. Xu, and S\. Arora\(2026\)ThunderAgent: a simple, fast and program\-aware agentic inference system\.External Links:2602\.13692,[Link](https://arxiv.org/abs/2602.13692)Cited by:[§1](https://arxiv.org/html/2609.03236#S1.p2.1),[§2](https://arxiv.org/html/2609.03236#S2.p3.1)\.
- \[4\]Y\. Leviathan, M\. Kalman, and Y\. Matias\(2023\)Fast inference from transformers via speculative decoding\.InInternational Conference on Machine Learning,pp\. 19274–19286\.Cited by:[§1](https://arxiv.org/html/2609.03236#S1.p2.1),[§2](https://arxiv.org/html/2609.03236#S2.p1.1)\.
- \[5\]G\. Ling, S\. Zhong, and R\. Huang\(2026\)Agent skills: a data\-driven analysis of claude skills for extending large language model functionality\.External Links:2602\.08004,[Link](https://arxiv.org/abs/2602.08004)Cited by:[§1](https://arxiv.org/html/2609.03236#S1.p1.1)\.
- \[6\]Z\. Pan, A\. PATEL, Y\. Shen, Z\. Hu, Y\. Guan, W\. Li, L\. Qin, Y\. Wang, and Y\. Ding\(2026\)KVFlow: efficient prefix caching for accelerating LLM\-based multi\-agent workflows\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=5Iw1nDtYmT)Cited by:[§1](https://arxiv.org/html/2609.03236#S1.p2.1),[§2](https://arxiv.org/html/2609.03236#S2.p3.1)\.
- \[7\]Qwen Team\(2026\)Qwen3\.5: towards native multimodal agents\.External Links:[Link](https://qwen.ai/blog?id=qwen3.5)Cited by:[§1](https://arxiv.org/html/2609.03236#S1.p3.1),[§4\.1](https://arxiv.org/html/2609.03236#S4.SS1.p1.1)\.
- \[8\]Y\. Sui, H\. Zhao, R\. Ma, Z\. He, H\. Wang, J\. Li, and Y\. Yang\(2026\)Act while thinking: accelerating llm agents via pattern\-aware speculative tool execution\.External Links:2603\.18897,[Link](https://arxiv.org/abs/2603.18897)Cited by:[§1](https://arxiv.org/html/2609.03236#S1.p2.1),[§2](https://arxiv.org/html/2609.03236#S2.p3.1)\.
- \[9\]H\. Trivedi, T\. Khot, M\. Hartmann, R\. Manku, V\. Dong, E\. Li, S\. Gupta, A\. Sabharwal, and N\. Balasubramanian\(2024\)AppWorld: a controllable world of apps and people for benchmarking interactive coding agents\.InACL,Cited by:[§4\.1](https://arxiv.org/html/2609.03236#S4.SS1.p2.1)\.
- \[10\]T\. Wei, T\. Li, Z\. Liu, X\. Ning, Z\. Yang, J\. Zou, Z\. Zeng, R\. Qiu, X\. Lin, D\. Fu, Z\. Li, M\. Ai, D\. Zhou, W\. Bao, Y\. Li, G\. Li, C\. Qian, Y\. Wang, X\. Tang, Y\. Xiao, L\. Fang, H\. Liu, X\. Tang, Y\. Zhang, C\. Wang, J\. You, H\. Ji, H\. Tong, and J\. He\(2026\)Agentic reasoning for large language models\.External Links:2601\.12538,[Link](https://arxiv.org/abs/2601.12538)Cited by:[§1](https://arxiv.org/html/2609.03236#S1.p1.1)\.
- \[11\]H\. Xu, C\. Li, X\. Ma, X\. Ou, Z\. Zhang, T\. He, X\. Liu, Z\. Wang, J\. Liang, Z\. Chu, R\. Liu, R\. Mu, D\. Tu, M\. Liu, and B\. Qin\(2026\)The evolution of tool use in llm agents: from single\-tool call to multi\-tool orchestration\.External Links:2603\.22862,[Link](https://arxiv.org/abs/2603.22862)Cited by:[§1](https://arxiv.org/html/2609.03236#S1.p1.1)\.
- \[12\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2609.03236#S1.p1.1),[§2](https://arxiv.org/html/2609.03236#S2.p1.1)\.
- \[13\]N\. Ye, A\. Ahuja, G\. Liargkovas, Y\. Lu, K\. Kaffes, and T\. Peng\(2026\)Speculative actions: a lossless framework for faster AI agents\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=P0GOk5wslg)Cited by:[§1](https://arxiv.org/html/2609.03236#S1.p3.1),[§2](https://arxiv.org/html/2609.03236#S2.p1.1)\.