JevSpawn：通过组合式动作空间实现自适应智能体推理

# JevSpawn: Adaptive Agentic Inference through Compositional Action Spaces
Source: [https://arxiv.org/html/2610.00437](https://arxiv.org/html/2610.00437)
###### Abstract

LLM agents generate intermediate reasoning and actions token by token, making extended interactions slow and computationally expensive\. Jev\-style models offer fast probabilistic predictions over finite fields, but require those fields to be specified in advance\. This requirement limits autonomous task solving, where the available actions must be derived from natural language instructions and adapted through interaction\. We introduce JevSpawn, a compositional policy that connects natural language task specifications to finite probabilistic exploration\. Parallel action spawning is coupled with feedback driven branch selection, representation revision, and recovery from retained alternatives\. Shared action structure and model prefixes reduce repeated generation and context computation without additional training\. Evaluations on eight benchmark tasks against seven agent baselines and a TypeSafe Jev variant establish JevSpawn as a promising approach to structured agentic inference, with improved task performance and faster navigation\. Code is available at[https://github\.com/Hoyant\-Su/JevSpawn](https://github.com/Hoyant-Su/JevSpawn)\.

Fudan UniversityShanghai Jiao Tong UniversityShanghai Innovation InstituteShanghai Innovation Institute
\(a\)Quality and latency\(b\)Task scores over timeFigure 1:Quality and latency across eight benchmark tasks for JevSpawn and seven agent baselines\. \(a\) Average quality and latency with 95% paired bootstrap intervals\. \(b\) Scores accumulated as tasks finish\.
## 1Introduction

LLM agents solve tasks through tool use and repeated interaction with an environment\([Yao et al\., 2023b](https://arxiv.org/html/2610.00437#bib.bib11);[Shinn et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib27);[Guo et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib18)\)\. Reasoning and actions are generated token by token, while observations enlarge the context for later steps\. Exploring alternative actions adds further cost, limiting both the breadth and duration of interaction under a fixed inference budget\.

Existing methods use this budget more effectively by organizing model calls and reducing intermediate information\([Yang et al\., 2026](https://arxiv.org/html/2610.00437#bib.bib19);[Yan et al\., 2025](https://arxiv.org/html/2610.00437#bib.bib20)\)\. Search favors promising trajectories, while workflow optimization removes redundant calls and parallelizes independent tools\([Zhou et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib14);[Kim et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib3);[Zhang et al\., 2025b](https://arxiv.org/html/2610.00437#bib.bib25);[Hu et al\., 2025b](https://arxiv.org/html/2610.00437#bib.bib26);[Wang et al\., 2025a](https://arxiv.org/html/2610.00437#bib.bib10)\)\. Communication pruning, memory management, and latent exchange reduce the information passed between steps\([Zhang et al\., 2025a](https://arxiv.org/html/2610.00437#bib.bib13);[Wang et al\., 2025b](https://arxiv.org/html/2610.00437#bib.bib21);[Hu et al\., 2025a](https://arxiv.org/html/2610.00437#bib.bib2);[Sun et al\., 2026](https://arxiv.org/html/2610.00437#bib.bib8);[Zou et al\., 2026](https://arxiv.org/html/2610.00437#bib.bib15)\)\. Further savings are possible within the actions themselves, since alternatives often share a form and differ in only a few arguments\.

To exploit this shared structure, repeated action descriptions can be replaced by predictions over finite fields\. Constrained decoding enforces output rules and can skip deterministic continuations\([Beurer\-Kellner et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib28);[Geng et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib22);[Dong et al\., 2025](https://arxiv.org/html/2610.00437#bib.bib16);[Zheng et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib1)\)\. TypeSafe Jev returns typed values and probabilities for supplied questions without generating response strings\([Almeida, 2026](https://arxiv.org/html/2610.00437#bib.bib24)\)\. The application must, however, define the fields in advance\. For autonomous agents, suitable action fields must be inferred from instructions and revised as new observations arrive\. The central challenge is to retain the speed of finite prediction while allowing the action space to change during interaction\.

JevSpawn addresses this challenge by making the action fields part of the policy\. Field values are combined into executable actions, and joint probabilities guide parallel spawning\. Execution feedback guides branch selection and field revision, while retained branches support recovery from unsuccessful choices\. Shared action structure and model context reduce repeated generation and computation\. The resulting policy extends Jev\-style prediction to sustained task solving without additional training\.

Our contributions are threefold\. First, a compositional action representation connects natural language goals and interaction rules to executable finite fields, extending Jev\-style prediction to tasks including spatial navigation\. Second, a unified Jev\-style architecture couples parallel spawning with feedback driven state transitions, representation revision, and recovery through retained branches\. Third, parallel causal evaluation and shared prefix reuse provide a common computational basis for the finite policy, reducing repeated action generation and context processing\.

## 2Related Work

### 2\.1Efficient Agentic Inference

Efficient agentic inference depends on both the computation requested by an agent and the cost of executing those requests\([Guo et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib18);[Yang et al\., 2026](https://arxiv.org/html/2610.00437#bib.bib19);[Yan et al\., 2025](https://arxiv.org/html/2610.00437#bib.bib20)\)\. At the agent level, existing methods organize exploration or reduce intermediate information\. Search and workflow optimization determine where model calls are spent\([Yao et al\., 2023a](https://arxiv.org/html/2610.00437#bib.bib12);[Zhang et al\., 2025b](https://arxiv.org/html/2610.00437#bib.bib25);[Hu et al\., 2025b](https://arxiv.org/html/2610.00437#bib.bib26)\)\. LATS uses environment feedback to guide search\([Zhou et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib14)\), LLMCompiler parallelizes tool calls\([Kim et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib3)\), and DyFlow adapts workflows during execution\([Wang et al\., 2025a](https://arxiv.org/html/2610.00437#bib.bib10)\)\. Fewer or better scheduled calls reduce wasted computation, but each interaction still carries messages and history needed for later actions\.

Reducing this information burden is the focus of communication and memory management\([Wu et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib30);[Gao et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib31);[Park et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib33);[Packer et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib32);[Shinn et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib27);[Wang et al\., 2024b](https://arxiv.org/html/2610.00437#bib.bib29)\)\. AgentPrune removes communication edges\([Zhang et al\., 2025a](https://arxiv.org/html/2610.00437#bib.bib13)\), while AgentDropout removes unnecessary agents\([Wang et al\., 2025b](https://arxiv.org/html/2610.00437#bib.bib21)\)\. HiAgent organizes subgoal memory\([Hu et al\., 2025a](https://arxiv.org/html/2610.00437#bib.bib2)\), and Context Folding summarizes history\([Sun et al\., 2026](https://arxiv.org/html/2610.00437#bib.bib8)\)\. LatentMAS further reduces reliance on text by exchanging latent representations\([Zou et al\., 2026](https://arxiv.org/html/2610.00437#bib.bib15)\)\. Together, call scheduling and information reduction determine how much work reaches the model\. The cost of that remaining work depends on how context is processed and outputs are generated\.

At this computational level, inference methods accelerate individual model requests\. Transformer inference\([Vaswani et al\., 2017](https://arxiv.org/html/2610.00437#bib.bib34)\)benefits from efficient attention\([Dao et al\., 2022](https://arxiv.org/html/2610.00437#bib.bib43);[Dao, 2023](https://arxiv.org/html/2610.00437#bib.bib44);[Shah et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib45);[Ye et al\., 2025](https://arxiv.org/html/2610.00437#bib.bib53)\), compact key and value representations\([Shazeer, 2019](https://arxiv.org/html/2610.00437#bib.bib46);[Ainslie et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib47)\), and improved scheduling and memory management\([Zhong et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib48);[Patel et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib49);[Agrawal et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib50);[Agrawal et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib51);[Prabhu et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib52)\)\. Partitioning, offloading, and activation sparsity address additional resource constraints\([Aminabadi et al\., 2022](https://arxiv.org/html/2610.00437#bib.bib54);[Sheng et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib55);[Song et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib56)\)\. For repeated agent interactions, these per\-request improvements can be extended by reusing the context shared across successive calls\.

Prefix and text caching provide such reuse\([Yao et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib67);[Gim et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib68);[Juravsky et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib69);[Ye et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib70)\), with prefix aware scheduling extending reuse across workers\([Srivatsa et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib57)\)\. Cache compression and selective retention or retrieval reduce the cost of accumulated history\([Xiao et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib58);[Zhang et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib59);[Liu et al\., 2023c](https://arxiv.org/html/2610.00437#bib.bib60);[Li et al\., 2024a](https://arxiv.org/html/2610.00437#bib.bib61);[Cai et al\., 2024b](https://arxiv.org/html/2610.00437#bib.bib62);[Tang et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib63);[Liu et al\., 2024b](https://arxiv.org/html/2610.00437#bib.bib64);[Liu et al\., 2024a](https://arxiv.org/html/2610.00437#bib.bib65);[Liu et al\., 2023b](https://arxiv.org/html/2610.00437#bib.bib66);[Wang et al\., 2024a](https://arxiv.org/html/2610.00437#bib.bib71);[Ge et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib72);[Ribar et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib73)\), while alternative context models use recurrence, compression, sparse attention, or linear attention\([Dai et al\., 2019](https://arxiv.org/html/2610.00437#bib.bib35);[Rae et al\., 2019](https://arxiv.org/html/2610.00437#bib.bib36);[Beltagy et al\., 2020](https://arxiv.org/html/2610.00437#bib.bib37);[Zaheer et al\., 2020](https://arxiv.org/html/2610.00437#bib.bib38);[Kitaev et al\., 2020](https://arxiv.org/html/2610.00437#bib.bib39);[Choromanski et al\., 2020](https://arxiv.org/html/2610.00437#bib.bib40);[Wang et al\., 2020](https://arxiv.org/html/2610.00437#bib.bib41);[Katharopoulos et al\., 2020](https://arxiv.org/html/2610.00437#bib.bib42)\)\. These methods reduce the cost of shared inputs, but exploring alternative actions can still require repeated output generation\. JevSpawn targets this remaining cost by reusing compositional action fields across branches and selecting field values through finite probabilities\.

### 2\.2Constrained Decoding

Replacing repeated action generation requires a representation that preserves executable outputs\. Constrained decoding provides a basis for such representations through fixed or input dependent grammars\([Beurer\-Kellner et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib28);[Geng et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib22)\)\. Syntactic and semantic constraints improve program validity\([Scholak et al\., 2021](https://arxiv.org/html/2610.00437#bib.bib96);[Poesia et al\., 2022](https://arxiv.org/html/2610.00437#bib.bib97)\), although valid structure alone does not ensure accurate predictions\([Geng et al\., 2025](https://arxiv.org/html/2610.00437#bib.bib98);[Tam et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib100)\)\. Formatting, option order, and context position can still affect the selected output\([Sclar et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib99);[Liu et al\., 2023a](https://arxiv.org/html/2610.00437#bib.bib101);[Zhao et al\., 2021](https://arxiv.org/html/2610.00437#bib.bib102);[Lu et al\., 2021](https://arxiv.org/html/2610.00437#bib.bib103);[Zheng et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib104)\)\. A useful structured policy must therefore preserve both output validity and meaningful choices within the permitted space\.

Meeting these requirements during decoding raises two further issues, the cost of enforcing constraints and the effect of constraints on output probabilities\. XGrammar largely hides this overhead through cached token checks and overlap with GPU execution\([Dong et al\., 2025](https://arxiv.org/html/2610.00437#bib.bib16)\), while SGLang skips deterministic grammar paths and reuses context\([Zheng et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib1)\)\. Constraint overhead is reduced, while distributional fidelity requires separate treatment\. DOMINO preserves valid continuations across tokenization boundaries\([Beurer\-Kellner et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib23)\), and grammar aligned decoding corrects distributional distortion from local token masking\([Park et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib17)\)\. Even with efficient enforcement and improved fidelity, choices not fixed by the grammar are still generated sequentially\.

This remaining sequential cost motivates methods that produce or verify several tokens at once\. Speculative decoding accelerates generation through parallel verification\([Leviathan et al\., 2022](https://arxiv.org/html/2610.00437#bib.bib74);[Chen et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib75);[Li et al\., 2025](https://arxiv.org/html/2610.00437#bib.bib79);[Fu et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib80);[Miao et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib81)\), using drafts from auxiliary models, prediction heads, retrieval, or earlier layers\([Cai et al\., 2024a](https://arxiv.org/html/2610.00437#bib.bib76);[Li et al\., 2024c](https://arxiv.org/html/2610.00437#bib.bib77);[Li et al\., 2024b](https://arxiv.org/html/2610.00437#bib.bib78);[He et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib82);[Zhang et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib83);[Elhoushi et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib84);[Bhendawade et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib85);[Cheng et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib86)\)\. Nonautoregressive and insertion based models change the output factorization\([Gu et al\., 2017](https://arxiv.org/html/2610.00437#bib.bib87);[Ghazvininejad et al\., 2019](https://arxiv.org/html/2610.00437#bib.bib88);[Gu et al\., 2019](https://arxiv.org/html/2610.00437#bib.bib89);[Stern et al\., 2019](https://arxiv.org/html/2610.00437#bib.bib90);[Wang et al\., 2018](https://arxiv.org/html/2610.00437#bib.bib91)\), while diffusion models refine sequences iteratively\([Li et al\., 2022](https://arxiv.org/html/2610.00437#bib.bib92);[Lou et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib93);[Sahoo et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib94);[Nie et al\., 2025](https://arxiv.org/html/2610.00437#bib.bib95)\)\. These methods accelerate the production of output sequences\. When the possible values are already known, the task can instead be expressed as finite prediction over those alternatives\.

TypeSafe Jev provides this finite interface by evaluating supplied questions in parallel and returning typed values with probabilities\([Almeida, 2026](https://arxiv.org/html/2610.00437#bib.bib24)\)\. Response strings need not be generated, but the application must define the fields being predicted\. JevSpawn addresses this dependence on predefined fields by making the action space part of the policy\. Compositional fields are inferred from task context and revised through feedback, coupling finite prediction with parallel spawning and recovery across interaction rounds\.

## 3Method

JevSpawn extends Jev\-style prediction to agentic tasks by allowing the model to define and revise finite action fields\. Spawning explores combinations of field values in parallel, with actions selected by joint probability\. Execution feedback guides branch selection and field revision\. Figure[2](https://arxiv.org/html/2610.00437#S3.F2)presents the method\.

![Refer to caption](https://arxiv.org/html/2610.00437v1/jevspawn.png)Figure 2:JevSpawn architecture\. \(a\) Compositional actions derived from natural language\. \(b\) Probabilistic spawning, feedback\-driven transitions, and recovery\. \(c\) Finite action probabilities\. \(d\) Shared computation in the interaction loop\.### 3\.1Compositional Jev Policies

The action representation is inferred as part of the policy\. Letvvindex a branch,xxcontain the task context and interaction rules,HvH\_\{v\}denote the action, observation, and declaration feedback history ofvv, andMMcollect observations shared across branches\. The model receiveshv=prompt⁡\(x,M,Hv\)h\_\{v\}=\\operatorname\{prompt\}\(x,M,H\_\{v\}\)together with the current declaration and finite alternatives\. A pretrained model with fixed parametersθ\\thetaconstructs

Σv=\(\{\(rj,Dj​\(⋅\)\)\}j=1mv,gv\),\\Sigma\_\{v\}=\\bigl\(\\\{\(r\_\{j\},D\_\{j\}\(\\cdot\)\)\\\}\_\{j=1\}^\{m\_\{v\}\},g\_\{v\}\\bigr\),\(1\)HereΣv\\Sigma\_\{v\}is the action declaration for branchvv,mvm\_\{v\}is the number of fields, andjjindexes a field\. Eachrjr\_\{j\}specifies a field name and meaning, whileDj​\(⋅\)D\_\{j\}\(\\cdot\)gives the finite values permitted under preceding assignments\. For a complete assignmentz=\(z1,…,zmv\)z=\(z\_\{1\},\\ldots,z\_\{m\_\{v\}\}\),gv​\(z\)g\_\{v\}\(z\)renders the field values as an executable action\. Shared syntax follows the task rules, while missing fields are generated from context\. Appendix[A\.1](https://arxiv.org/html/2610.00437#A1.SS1)gives a worked construction\.

Fields are evaluated in conditional blocksb1,…,bLb\_\{1\},\\ldots,b\_\{L\}, wherebℓb\_\{\\ell\}indexes the fields evaluated jointly at composition stepℓ\\ellandLLis the maximum composition depth\. The tuplezbℓz\_\{b\_\{\\ell\}\}contains the values assigned to that block, andz<bℓz\_\{<b\_\{\\ell\}\}contains all preceding assignments\. Completed assignments receive only an empty extension of probability one\. The compositional policy assigns probabilityπθ​\(z∣hv,Σv\)\\pi\_\{\\theta\}\(z\\mid h\_\{v\},\\Sigma\_\{v\}\)to a complete assignment,

πθ​\(z∣hv,Σv\)=∏ℓ=1LQ¯θ​\(zbℓ∣hvz<bℓ,𝒟bℓ​\(z<bℓ\)\),\\pi\_\{\\theta\}\(z\\mid h\_\{v\},\\Sigma\_\{v\}\)=\\prod\_\{\\ell=1\}^\{L\}\\bar\{Q\}\_\{\\theta\}\\bigl\(z\_\{b\_\{\\ell\}\}\\mid h\_\{v\}^\{z\_\{<b\_\{\\ell\}\}\},\\mathcal\{D\}\_\{b\_\{\\ell\}\}\(z\_\{<b\_\{\\ell\}\}\)\\bigr\),\(2\)The contexthvz<bℓh\_\{v\}^\{z\_\{<b\_\{\\ell\}\}\}augmentshvh\_\{v\}with the preceding assignments\. The joint domain𝒟bℓ​\(z<bℓ\)\\mathcal\{D\}\_\{b\_\{\\ell\}\}\(z\_\{<b\_\{\\ell\}\}\)contains the block assignments allowed by the field domainsDjD\_\{j\}\. The finite distributionQ¯θ\\bar\{Q\}\_\{\\theta\}gives the probability of a block assignment within that domain and is normalized over a value tree, with equal probability shares for duplicate serialized values\. Equation[2](https://arxiv.org/html/2610.00437#S3.E2)defines a policy over the declared action space without prescribing a trajectory\. The distribution and absorbing empty extension are derived in Appendix[A\.2](https://arxiv.org/html/2610.00437#A1.SS2)\.

### 3\.2Adaptive Branch Exploration

Spawning expands a state into executable alternatives under the compositional policy\. A beam of widthKKretains up toKKassignments\. Each retained assignmentzwz\_\{w\}defines a child branchwwand an actionaw=gv​\(zw\)a\_\{w\}=g\_\{v\}\(z\_\{w\}\)\. Letsvs\_\{v\}be the parent environment state\. The interaction operatorℰ\\mathcal\{E\}executesawa\_\{w\}in an independent copy ofsvs\_\{v\}, producing the child statesws\_\{w\}and observationowo\_\{w\},

\(sw,ow\)=ℰ⁡\(sv,aw\),Hw=Hv∥\(aw,ow\)\.\(s\_\{w\},o\_\{w\}\)=\\mathcal\{E\}\(s\_\{v\},a\_\{w\}\),\\qquad H\_\{w\}=H\_\{v\}\\mathbin\{\\\|\}\(a\_\{w\},o\_\{w\}\)\.\(3\)Here∥\\\|appends the action–observation pair to the parent history, giving the child historyHwH\_\{w\}\. The continuation policy scores retained branches using returned observations\. Action preferences can therefore be revised after execution, and earlier branches can be resumed without regenerating the preceding trajectory\.

The policy can also reviseΣv\\Sigma\_\{v\}from accumulated feedback\. Recovery explores retained states, while revision changes the available action space\. Answer submission or budget exhaustion terminates interaction\. Algorithm[1](https://arxiv.org/html/2610.00437#alg1)specifies branch retention, operation selection, and expansion\.

### 3\.3Parallel Policy Evaluation

Declared values are known token sequences, whose causal hidden states can be evaluated in parallel\. JevSpawn projects onto vocabulary entries that distinguish continuations and combines the resulting probabilities intoQ¯θ\\bar\{Q\}\_\{\\theta\}\. The same finite evaluation supports action proposals, branch selection, and operation choice\. Appendix[A](https://arxiv.org/html/2610.00437#A1)derives the distribution and computation\.

Shared action structure avoids regenerating complete descriptions, while prefix reuse avoids recomputing common context\. ForBBevaluations with prefix lengthnpn\_\{p\}and suffix lengthsn1,…,nBn\_\{1\},\\ldots,n\_\{B\}, processed input tokens decrease fromB​np\+∑b=1BnbBn\_\{p\}\+\\sum\_\{b=1\}^\{B\}n\_\{b\}tonp\+∑b=1Bnbn\_\{p\}\+\\sum\_\{b=1\}^\{B\}n\_\{b\}, before padding and value prefix evaluation\. Suffix attention to the shared prefix is retained\. Appendix[A\.5](https://arxiv.org/html/2610.00437#A1.SS5)establishes the conditions for equivalent attention and recurrent computation\.

Text generation supplies missing declarations, revisions, and final answers when required\. Repeated exploration uses the finite policy, with the pretrained parameters unchanged\.

## 4Experiments

Table 1:Task performance \(↑\\uparrow\) across eight benchmark tasks\. PPNL, Maze, and Grid use success rates on\[0,1\]\[0,1\], and the remaining tasks use benchmark rewards or scores\. Bold and underlined values indicate the best and second best results, including ties\. The TypeSafe Jev variant replaces finite scoring within JevSpawn\.Table 2:E2E latency \(s,↓\\downarrow\), including failed tasks and timeouts\. Bold and underlined values indicate the lowest and second lowest latencies, including ties\. The TypeSafe Jev variant replaces finite scoring within JevSpawn\.\(a\)PPNL\(b\)Maze\(c\)Grid\(d\)LightsOut\(e\)RushHour\(f\)Sokoban\(g\)2048\(h\)Nullify
Figure 3:Stability of the quality–latency tradeoff under paired bootstrap resampling\. Symbols mark paired bootstrap medians, and error bars show 95% marginal intervals from 2,000 resamples\. Horizontal bars show Pareto frontier frequency under resampling of paired instances\.Table 3:Text decoding throughput \(tok/s,↑\\uparrow\), excluding prefill\. TypeSafe Jev uses Qwen3\.8\-27B for text generation\. / denotes not applicable\. Bold and underlined values indicate the best and second best results, including ties\.\(a\)PPNL\(b\)Maze\(c\)Grid\(d\)LightsOut\(e\)RushHour\(f\)Sokoban\(g\)2048\(h\)Nullify
Figure 4:Instance\-level effects of component ablations\. Stacked bars show fractions with lower, equal, and higher scores than complete JevSpawn\. Boxes summarize paired latency ratios with medians, interquartile ranges, and fifth to ninety\-fifth percentile whiskers\. Ratios below one indicate shorter execution\.### 4\.1Experimental Setup

JevSpawn and the seven agent baselines use Qwen3\.8\-27B as the pretrained policy and run on four H100 GPUs with tensor parallelism and bfloat16 precision\. The batch size is 8, the context limit is 16,384 tokens, and each generation is limited to 2,048 tokens\. Tasks receive up to 36 exploration rounds and a final submission step, within a 300 second limit\. Temperature is zero, thinking mode is disabled, and the seed is 42\. JevSpawn selects one parent and spawns up to four distinct actions per expansion\. A further variant replaces finite scoring with TypeSafe Jev and retains Qwen for declarations and text generation\. End\-to\-end \(E2E\) latency measures elapsed time from task submission to termination\. Appendix[B](https://arxiv.org/html/2610.00437#A2)provides timing definitions and execution details\.

Eight tasks from five benchmarks are used to compare JevSpawn with LATS\([Zhou et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib14)\), LLMCompiler\([Kim et al\., 2024](https://arxiv.org/html/2610.00437#bib.bib3)\), AgentPrune\([Zhang et al\., 2025a](https://arxiv.org/html/2610.00437#bib.bib13)\), HiAgent\([Hu et al\., 2025a](https://arxiv.org/html/2610.00437#bib.bib2)\), FoldAgent\([Sun et al\., 2026](https://arxiv.org/html/2610.00437#bib.bib8)\), DyFlow\([Wang et al\., 2025a](https://arxiv.org/html/2610.00437#bib.bib10)\), and LatentMAS\([Zou et al\., 2026](https://arxiv.org/html/2610.00437#bib.bib15)\)\. PPNL evaluates path planning from natural language specifications\([Aghzal et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib7)\)\. Maze from LMRL Gym and Grid from LLF Bench evaluate navigation with action feedback\([Abdulhai et al\., 2025](https://arxiv.org/html/2610.00437#bib.bib6);[Cheng et al\., 2023](https://arxiv.org/html/2610.00437#bib.bib5)\)\. LightsOut, RushHour, and Sokoban from TextArena assess sequential puzzle solving\([Guertler et al\., 2025](https://arxiv.org/html/2610.00437#bib.bib9)\), while 2048 and Nullify from KORGym assess numerical game play\([Shi et al\., 2025](https://arxiv.org/html/2610.00437#bib.bib4)\)\. Evaluation uses the public PPNL test collections, fixed Maze initial states, and environment seeds 42 through 141 for the remaining tasks\.

#### Ablation settings\.

Component ablations vary expansion width over\{1,2,4,8,10\}\\\{1,2,4,8,10\\\}, distribute four actions across parent–child allocations of1×41\\times 4,2×22\\times 2, or4×14\\times 1, retain the first valid declaration, or restrict observations to the current branch\. Each variant is compared with a separate run of the full method on the same 1,761 instances\.

Long horizon experiments form a separate series with a 131,072 token context and round limits of\{36,54,72,108\}\\\{36,54,72,108\\\}, each evaluated with and without the 300 second deadline\. Batch scaling retains the 16,384 token context and varies physical batch size over\{8,16,32,64,128\}\\\{8,16,32,64,128\\\}until memory is exhausted\. Other settings are fixed within each comparison\. Appendix[C](https://arxiv.org/html/2610.00437#A3)reports the complete results\.

### 4\.2Main Results

#### Task performance\.

Against the seven agent baselines, JevSpawn achieves the highest scores on five of eight tasks in Table[1](https://arxiv.org/html/2610.00437#S4.T1)\. Success rates of 0\.96 and 0\.95 are achieved on Maze and Grid, compared with 0\.52 and 0\.73 for FoldAgent\. LightsOut reward reaches 0\.61 against AgentPrune’s 0\.34\. Nullify reward reaches 0\.21 against 0\.09 for HiAgent and FoldAgent, and the 2048 score reaches 305\.12 against LLMCompiler’s 5\.00\. AgentPrune remains strongest on PPNL, RushHour, and Sokoban\.

Within the same JevSpawn architecture, TypeSafe Jev outperforms Qwen scoring on PPNL, LightsOut \(0\.66 versus 0\.61\), and Sokoban, matches Maze, and scores lower on Grid, RushHour, 2048, and Nullify\.

#### Execution efficiency\.

JevSpawn achieves joint quality and latency gains on Maze and Grid\. Alongside the success rate improvements, E2E latency is reduced from the fastest baseline values of 47\.91 and 61\.44 seconds to 40\.91 and 40\.58 seconds\. Figure[3](https://arxiv.org/html/2610.00437#S4.F3)places JevSpawn on the Pareto frontier in nearly all bootstrap resamples for seven tasks\. Sokoban frontier membership is less stable, reflecting a 1\.50 second latency margin over LLMCompiler and a lower score\.

On LightsOut and 2048, higher scores accompany longer execution\. JevSpawn takes 111\.05 seconds on LightsOut against 84\.17 for FoldAgent, and 169\.94 seconds on 2048 against 33\.51 for LLMCompiler\. The TypeSafe Jev variant takes 1\.4 to 2\.1 times as long as Qwen scoring across all eight tasks, including API communication\. Appendix[B\.5](https://arxiv.org/html/2610.00437#A2.SS5)compares per\-instance scores and latencies with AgentPrune\.

Text decoding throughput reaches 544 and 580 tokens per second on PPNL and LightsOut, respectively, the highest values in Table[3](https://arxiv.org/html/2610.00437#S4.T3)\. Maze and Grid use finite evaluation throughout\. On Nullify, text generation and finite evaluation overlap in every decoding interval\. Separate finite evaluation measurements are given in Appendix[B](https://arxiv.org/html/2610.00437#A2)\.

### 4\.3Ablation Studies

Figure[4](https://arxiv.org/html/2610.00437#S4.F4)compares expansion width, parent allocation, declaration revision, and branch information sharing against the full JevSpawn method\. Appendix[C](https://arxiv.org/html/2610.00437#A3)reports scores and E2E latency for every condition\.

Parallel spawning contributes strongly to navigation and LightsOut performance\. Reducing width from four to one lowers Maze success from 0\.88 to 0\.16, Grid success from 0\.94 to 0\.58, and LightsOut reward from 0\.61 to 0\.06\. Increasing width to ten solves every Maze instance, but halves Nullify reward and approximately doubles RushHour latency\.

Parent allocation matters at a fixed budget of four actions\. Success rates of 1\.00 are achieved on both Maze and Grid with two parents and two actions each, compared with 0\.88 and 0\.94 for one parent\. Four parents with one action each reduce these rates to 0\.12 and 0\.75, while increasing the 2048 score\.

Retaining the first valid declaration improves scores on Maze, LightsOut, Sokoban, and 2048 and reduces latency on six tasks\. Adaptive declarations score higher on the other four tasks\. Restricting observations to the current branch improves Maze, Grid, LightsOut, and RushHour scores, whereas shared observations yield higher scores on the remaining tasks and lower latency on all eight\.

Without a deadline, increasing the round limit from 36 to 108 raises the 2048 score from 315\.16 to 1102\.24 and LightsOut reward from 0\.61 to 0\.876\. Under the 300 second deadline, 85 percent of 2048 tasks time out at 108 rounds\. Figure[10](https://arxiv.org/html/2610.00437#A3.F10)and Table[7](https://arxiv.org/html/2610.00437#A3.T7)compare both time settings\.

## 5Conclusion

JevSpawn extends Jev\-style prediction to agentic inference by making the finite action space adaptive\. A compositional policy couples parallel spawning with feedback driven exploration, representation revision, and branch recovery\. Shared structure and prefix computation reduce the cost of evaluating alternatives without additional training\. Experiments demonstrate higher scores across several tasks and reduced latency in navigation\.

## References

- Abdulhaiet al\.\(2025\)M\. Abdulhai, I\. White, C\. V\. Snell, C\. Sun, J\. Hong, Y\. Zhai, K\. Xu, and S\. LevineLMRL gym: benchmarks for multi\-turn reinforcement learning with language models\.InProceedings of the 42nd International Conference on Machine Learning,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267,pp\. 126–153\.External Links:[Link](https://proceedings.mlr.press/v267/abdulhai25a.html)Cited by:[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.
- Aghzalet al\.\(2023\)M\. Aghzal, E\. Plaku, and Z\. YaoCan large language models be good path planners? a benchmark and investigation on spatial\-temporal reasoning\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2310.03249),[Link](https://arxiv.org/abs/2310.03249)Cited by:[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.
- Agrawalet al\.\(2024\)A\. Agrawal, N\. Kedia, A\. Panwar, J\. Mohan, N\. Kwatra, B\. S\. Gulavani, A\. Tumanov, and R\. RamjeeTaming throughput\-latency tradeoff in llm inference with sarathi\-serve\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2403.02310),[Link](https://arxiv.org/abs/2403.02310)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Agrawalet al\.\(2023\)A\. Agrawal, A\. Panwar, J\. Mohan, N\. Kwatra, B\. S\. Gulavani, and R\. RamjeeSARATHI: efficient llm inference by piggybacking decodes with chunked prefills\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2308.16369),[Link](https://arxiv.org/abs/2308.16369)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Ainslieet al\.\(2023\)J\. Ainslie, J\. Lee\-Thorp, M\. de Jong, Y\. Zemlyanskiy, F\. Lebrón, and S\. SanghaiGQA: training generalized multi\-query transformer models from multi\-head checkpoints\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2305.13245),[Link](https://arxiv.org/abs/2305.13245)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Almeida \(2026\)D\. AlmeidaIntroducing system one models & Jev\.Note:TypeSafe AI\.[https://typesafe\.ai/blog/introducing\-system\-one\-models\-and\-jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p3.1),[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p4.1)\.
- Aminabadiet al\.\(2022\)R\. Y\. Aminabadi, S\. Rajbhandari, M\. Zhang, A\. A\. Awan, C\. Li, D\. Li, E\. Zheng, J\. Rasley, S\. Smith, O\. Ruwase, and Y\. HeDeepSpeed inference: enabling efficient inference of transformer models at unprecedented scale\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2207.00032),[Link](https://arxiv.org/abs/2207.00032)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Beltagyet al\.\(2020\)I\. Beltagy, M\. E\. Peters, and A\. CohanLongformer: the long\-document transformer\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2004.05150),[Link](https://arxiv.org/abs/2004.05150)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Beurer\-Kellneret al\.\(2023\)L\. Beurer\-Kellner, M\. Fischer, and M\. VechevPrompting is programming: a query language for large language models\.Proceedings of the ACM on Programming Languages7\(PLDI\),pp\. 1946–1969\.External Links:ISSN 2475\-1421,[Link](http://dx.doi.org/10.1145/3591300),[Document](https://dx.doi.org/10.1145/3591300)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p3.1),[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p1.1)\.
- Beurer\-Kellneret al\.\(2024\)L\. Beurer\-Kellner, M\. Fischer, and M\. VechevGuiding LLMs the right way: fast, non\-invasive constrained generation\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 3658–3673\.External Links:[Link](https://proceedings.mlr.press/v235/beurer-kellner24a.html)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p2.1)\.
- Bhendawadeet al\.\(2024\)N\. Bhendawade, I\. Belousova, Q\. Fu, H\. Mason, M\. Rastegari, and M\. NajibiSpeculative streaming: fast llm inference without auxiliary models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2402.11131),[Link](https://arxiv.org/abs/2402.11131)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Caiet al\.\(2024a\)T\. Cai, Y\. Li, Z\. Geng, H\. Peng, J\. D\. Lee, D\. Chen, and T\. DaoMedusa: simple llm inference acceleration framework with multiple decoding heads\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2401.10774),[Link](https://arxiv.org/abs/2401.10774)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Caiet al\.\(2024b\)Z\. Cai, Y\. Zhang, B\. Gao, Y\. Liu, Y\. Li, T\. Liu, K\. Lu, W\. Xiong, Y\. Dong, J\. Hu, and W\. XiaoPyramidKV: dynamic kv cache compression based on pyramidal information funneling\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2406.02069),[Link](https://arxiv.org/abs/2406.02069)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Chenet al\.\(2023\)C\. Chen, S\. Borgeaud, G\. Irving, J\. Lespiau, L\. Sifre, and J\. JumperAccelerating large language model decoding with speculative sampling\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2302.01318),[Link](https://arxiv.org/abs/2302.01318)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Chenget al\.\(2023\)C\. Cheng, A\. Kolobov, D\. Misra, A\. Nie, and A\. SwaminathanLLF\-bench: benchmark for interactive learning from language feedback\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2312.06853),[Link](https://arxiv.org/abs/2312.06853)Cited by:[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.
- Chenget al\.\(2024\)Y\. Cheng, A\. Zhang, X\. Zhang, C\. Wang, and Y\. WangRecurrent drafter for fast speculative decoding in large language models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2403.09919),[Link](https://arxiv.org/abs/2403.09919)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Choromanskiet al\.\(2020\)K\. Choromanski, V\. Likhosherstov, D\. Dohan, X\. Song, A\. Gane, T\. Sarlos, P\. Hawkins, J\. Davis, A\. Mohiuddin, L\. Kaiser, D\. Belanger, L\. Colwell, and A\. WellerRethinking attention with performers\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2009.14794),[Link](https://arxiv.org/abs/2009.14794)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Daiet al\.\(2019\)Z\. Dai, Z\. Yang, Y\. Yang, J\. Carbonell, Q\. V\. Le, and R\. SalakhutdinovTransformer\-xl: attentive language models beyond a fixed\-length context\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.1901.02860),[Link](https://arxiv.org/abs/1901.02860)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Daoet al\.\(2022\)T\. Dao, D\. Y\. Fu, S\. Ermon, A\. Rudra, and C\. RéFlashAttention: fast and memory\-efficient exact attention with io\-awareness\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2205.14135),[Link](https://arxiv.org/abs/2205.14135)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Dao \(2023\)T\. DaoFlashAttention\-2: faster attention with better parallelism and work partitioning\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2307.08691),[Link](https://arxiv.org/abs/2307.08691)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Donget al\.\(2025\)Y\. Dong, C\. F\. Ruan, Y\. Cai, Z\. Xu, Y\. Zhao, R\. Lai, and T\. ChenXGrammar: flexible and efficient structured generation engine for large language models\.InProceedings of Machine Learning and Systems,M\. Zaharia, G\. Joshi, and Y\. Lin \(Eds\.\),Vol\.7,pp\.\.External Links:[Link](https://proceedings.mlsys.org/paper_files/paper/2025/file/5c20ca4b0b20b0bd2f1d839dc605e70f-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p3.1),[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p2.1)\.
- Elhoushiet al\.\(2024\)M\. Elhoushi, A\. Shrivastava, D\. Liskovich, B\. Hosmer, B\. Wasti, L\. Lai, A\. Mahmoud, B\. Acun, S\. Agarwal, A\. Roman, A\. Aly, B\. Chen, and C\. WuLayerSkip: enabling early exit inference and self\-speculative decoding\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 12622–12642\.External Links:[Link](http://dx.doi.org/10.18653/v1/2024.acl-long.681),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.681)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Fuet al\.\(2024\)Y\. Fu, P\. Bailis, I\. Stoica, and H\. ZhangBreak the sequential dependency of llm inference using lookahead decoding\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2402.02057),[Link](https://arxiv.org/abs/2402.02057)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Gaoet al\.\(2024\)D\. Gao, Z\. Li, X\. Pan, W\. Kuang, Z\. Ma, B\. Qian, F\. Wei, W\. Zhang, Y\. Xie, D\. Chen, L\. Yao, H\. Peng, Z\. Zhang, L\. Zhu, C\. Cheng, H\. Shi, Y\. Li, B\. Ding, and J\. ZhouAgentScope: a flexible yet robust multi\-agent platform\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2402.14034),[Link](https://arxiv.org/abs/2402.14034)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p2.1)\.
- Geet al\.\(2023\)S\. Ge, Y\. Zhang, L\. Liu, M\. Zhang, J\. Han, and J\. GaoModel tells you what to discard: adaptive kv cache compression for llms\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2310.01801),[Link](https://arxiv.org/abs/2310.01801)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Genget al\.\(2025\)S\. Geng, H\. Cooper, M\. Moskal, S\. Jenkins, J\. Berman, N\. Ranchin, R\. West, E\. Horvitz, and H\. NoriJSONSchemaBench: a rigorous benchmark of structured outputs for language models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2501.10868),[Link](https://arxiv.org/abs/2501.10868)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p1.1)\.
- Genget al\.\(2023\)S\. Geng, M\. Josifoski, M\. Peyrard, and R\. WestGrammar\-constrained decoding for structured NLP tasks without finetuning\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 10932–10952\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.674/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.674)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p3.1),[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p1.1)\.
- Ghazvininejadet al\.\(2019\)M\. Ghazvininejad, O\. Levy, Y\. Liu, and L\. ZettlemoyerMask\-predict: parallel decoding of conditional masked language models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.1904.09324),[Link](https://arxiv.org/abs/1904.09324)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Gimet al\.\(2023\)I\. Gim, G\. Chen, S\. Lee, N\. Sarda, A\. Khandelwal, and L\. ZhongPrompt cache: modular attention reuse for low\-latency inference\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2311.04934),[Link](https://arxiv.org/abs/2311.04934)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Guet al\.\(2017\)J\. Gu, J\. Bradbury, C\. Xiong, V\. O\. K\. Li, and R\. SocherNon\-autoregressive neural machine translation\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.1711.02281),[Link](https://arxiv.org/abs/1711.02281)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Guet al\.\(2019\)J\. Gu, C\. Wang, and J\. ZhaoLevenshtein transformer\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.1905.11006),[Link](https://arxiv.org/abs/1905.11006)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Guertleret al\.\(2025\)L\. Guertler, B\. Cheng, S\. Yu, B\. Liu, L\. Choshen, and C\. TanTextArena\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2504.11442),[Link](https://arxiv.org/abs/2504.11442)Cited by:[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.
- Guoet al\.\(2024\)T\. Guo, X\. Chen, Y\. Wang, R\. Chang, S\. Pei, N\. V\. Chawla, O\. Wiest, and X\. ZhangLarge language model based multi\-agents: a survey of progress and challenges\.InProceedings of the Thirty\-Third International Joint Conference on Artificial Intelligence, IJCAI\-24,K\. Larson \(Ed\.\),pp\. 8048–8057\.Note:Survey TrackExternal Links:[Document](https://dx.doi.org/10.24963/ijcai.2024/890),[Link](https://doi.org/10.24963/ijcai.2024/890)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p1.1)\.
- Heet al\.\(2023\)Z\. He, Z\. Zhong, T\. Cai, J\. D\. Lee, and D\. HeREST: retrieval\-based speculative decoding\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2311.08252),[Link](https://arxiv.org/abs/2311.08252)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Huet al\.\(2025a\)M\. Hu, T\. Chen, Q\. Chen, Y\. Mu, W\. Shao, and P\. LuoHiAgent: hierarchical working memory management for solving long\-horizon agent tasks with large language model\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 32779–32798\.External Links:[Link](https://aclanthology.org/2025.acl-long.1575/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1575),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.
- Huet al\.\(2025b\)S\. Hu, C\. Lu, and J\. CluneAutomated design of agentic systems\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 21344–21377\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/36b7acf6f6010652b3f2a433774a66fe-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p1.1)\.
- Juravskyet al\.\(2024\)J\. Juravsky, B\. Brown, R\. Ehrlich, D\. Y\. Fu, C\. Ré, and A\. MirhoseiniHydragen: high\-throughput llm inference with shared prefixes\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2402.05099),[Link](https://arxiv.org/abs/2402.05099)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Katharopouloset al\.\(2020\)A\. Katharopoulos, A\. Vyas, N\. Pappas, and F\. FleuretTransformers are rnns: fast autoregressive transformers with linear attention\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2006.16236),[Link](https://arxiv.org/abs/2006.16236)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Kimet al\.\(2024\)S\. Kim, S\. Moon, R\. Tabrizi, N\. Lee, M\. W\. Mahoney, K\. Keutzer, and A\. GholamiAn LLM compiler for parallel function calling\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 24370–24391\.External Links:[Link](https://proceedings.mlr.press/v235/kim24y.html)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.
- Kitaevet al\.\(2020\)N\. Kitaev, Ł\. Kaiser, and A\. LevskayaReformer: the efficient transformer\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2001.04451),[Link](https://arxiv.org/abs/2001.04451)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Leviathanet al\.\(2022\)Y\. Leviathan, M\. Kalman, and Y\. MatiasFast inference from transformers via speculative decoding\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2211.17192),[Link](https://arxiv.org/abs/2211.17192)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Liet al\.\(2022\)X\. L\. Li, J\. Thickstun, I\. Gulrajani, P\. Liang, and T\. B\. HashimotoDiffusion\-lm improves controllable text generation\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2205.14217),[Link](https://arxiv.org/abs/2205.14217)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Liet al\.\(2024a\)Y\. Li, Y\. Huang, B\. Yang, B\. Venkitesh, A\. Locatelli, H\. Ye, T\. Cai, P\. Lewis, and D\. ChenSnapKV: llm knows what you are looking for before generation\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2404.14469),[Link](https://arxiv.org/abs/2404.14469)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Liet al\.\(2024b\)Y\. Li, F\. Wei, C\. Zhang, and H\. ZhangEAGLE\-2: faster inference of language models with dynamic draft trees\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2406.16858),[Link](https://arxiv.org/abs/2406.16858)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Liet al\.\(2024c\)Y\. Li, F\. Wei, C\. Zhang, and H\. ZhangEAGLE: speculative sampling requires rethinking feature uncertainty\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2401.15077),[Link](https://arxiv.org/abs/2401.15077)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Liet al\.\(2025\)Y\. Li, F\. Wei, C\. Zhang, and H\. ZhangEAGLE\-3: scaling up inference acceleration of large language models via training\-time test\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2503.01840),[Link](https://arxiv.org/abs/2503.01840)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Liuet al\.\(2024a\)A\. Liu, J\. Liu, Z\. Pan, Y\. He, G\. Haffari, and B\. ZhuangMiniCache: kv cache compression in depth dimension for large language models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2405.14366),[Link](https://arxiv.org/abs/2405.14366)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Liuet al\.\(2023a\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2307.03172),[Link](https://arxiv.org/abs/2307.03172)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p1.1)\.
- Liuet al\.\(2023b\)Y\. Liu, H\. Li, Y\. Cheng, S\. Ray, Y\. Huang, Q\. Zhang, K\. Du, J\. Yao, S\. Lu, G\. Ananthanarayanan, M\. Maire, H\. Hoffmann, A\. Holtzman, and J\. JiangCacheGen: kv cache compression and streaming for fast large language model serving\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2310.07240),[Link](https://arxiv.org/abs/2310.07240)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Liuet al\.\(2023c\)Z\. Liu, A\. Desai, F\. Liao, W\. Wang, V\. Xie, Z\. Xu, A\. Kyrillidis, and A\. ShrivastavaScissorhands: exploiting the persistence of importance hypothesis for llm kv cache compression at test time\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2305.17118),[Link](https://arxiv.org/abs/2305.17118)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Liuet al\.\(2024b\)Z\. Liu, J\. Yuan, H\. Jin, S\. Zhong, Z\. Xu, V\. Braverman, B\. Chen, and X\. HuKIVI: a tuning\-free asymmetric 2bit quantization for kv cache\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2402.02750),[Link](https://arxiv.org/abs/2402.02750)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Louet al\.\(2023\)A\. Lou, C\. Meng, and S\. ErmonDiscrete diffusion modeling by estimating the ratios of the data distribution\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2310.16834),[Link](https://arxiv.org/abs/2310.16834)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Luet al\.\(2021\)Y\. Lu, M\. Bartolo, A\. Moore, S\. Riedel, and P\. StenetorpFantastically ordered prompts and where to find them: overcoming few\-shot prompt order sensitivity\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2104.08786),[Link](https://arxiv.org/abs/2104.08786)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p1.1)\.
- Miaoet al\.\(2024\)X\. Miao, G\. Oliaro, Z\. Zhang, X\. Cheng, Z\. Wang, Z\. Zhang, R\. Y\. Y\. Wong, A\. Zhu, L\. Yang, X\. Shi, C\. Shi, Z\. Chen, D\. Arfeen, R\. Abhyankar, and Z\. JiaSpecInfer: accelerating large language model serving with tree\-based speculative inference and verification\.InProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3,ASPLOS ’24,pp\. 932–949\.External Links:[Link](http://dx.doi.org/10.1145/3620666.3651335),[Document](https://dx.doi.org/10.1145/3620666.3651335)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Nieet al\.\(2025\)S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. LiLarge language diffusion models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2502.09992),[Link](https://arxiv.org/abs/2502.09992)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Packeret al\.\(2023\)C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. GonzalezMemGPT: towards llms as operating systems\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2310.08560),[Link](https://arxiv.org/abs/2310.08560)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p2.1)\.
- Parket al\.\(2023\)J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. BernsteinGenerative agents: interactive simulacra of human behavior\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2304.03442),[Link](https://arxiv.org/abs/2304.03442)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p2.1)\.
- Parket al\.\(2024\)K\. Park, J\. Wang, T\. Berg\-Kirkpatrick, N\. Polikarpova, and L\. D'AntoniGrammar\-aligned decoding\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 24547–24568\.External Links:[Document](https://dx.doi.org/10.52202/079017-0774),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/2bdc2267c3d7d01523e2e17ac0a754f3-Paper-Conference.pdf)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p2.1)\.
- Patelet al\.\(2023\)P\. Patel, E\. Choukse, C\. Zhang, A\. Shah, Í\. Goiri, S\. Maleki, and R\. BianchiniSplitwise: efficient generative llm inference using phase splitting\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2311.18677),[Link](https://arxiv.org/abs/2311.18677)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Poesiaet al\.\(2022\)G\. Poesia, O\. Polozov, V\. Le, A\. Tiwari, G\. Soares, C\. Meek, and S\. GulwaniSynchromesh: reliable code generation from pre\-trained language models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2201.11227),[Link](https://arxiv.org/abs/2201.11227)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p1.1)\.
- Prabhuet al\.\(2024\)R\. Prabhu, A\. Nayak, J\. Mohan, R\. Ramjee, and A\. PanwarVAttention: dynamic memory management for serving llms without pagedattention\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2405.04437),[Link](https://arxiv.org/abs/2405.04437)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Raeet al\.\(2019\)J\. W\. Rae, A\. Potapenko, S\. M\. Jayakumar, and T\. P\. LillicrapCompressive transformers for long\-range sequence modelling\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.1911.05507),[Link](https://arxiv.org/abs/1911.05507)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Ribaret al\.\(2023\)L\. Ribar, I\. Chelombiev, L\. Hudlass\-Galley, C\. Blake, C\. Luschi, and D\. OrrSparQ attention: bandwidth\-efficient llm inference\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2312.04985),[Link](https://arxiv.org/abs/2312.04985)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Sahooet al\.\(2024\)S\. S\. Sahoo, M\. Arriola, Y\. Schiff, A\. Gokaslan, E\. Marroquin, J\. T\. Chiu, A\. Rush, and V\. KuleshovSimple and effective masked diffusion language models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2406.07524),[Link](https://arxiv.org/abs/2406.07524)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Scholaket al\.\(2021\)T\. Scholak, N\. Schucher, and D\. BahdanauPICARD: parsing incrementally for constrained auto\-regressive decoding from language models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2109.05093),[Link](https://arxiv.org/abs/2109.05093)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p1.1)\.
- Sclaret al\.\(2023\)M\. Sclar, Y\. Choi, Y\. Tsvetkov, and A\. SuhrQuantifying language models’ sensitivity to spurious features in prompt design or: how i learned to start worrying about prompt formatting\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2310.11324),[Link](https://arxiv.org/abs/2310.11324)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p1.1)\.
- Shahet al\.\(2024\)J\. Shah, G\. Bikshandi, Y\. Zhang, V\. Thakkar, P\. Ramani, and T\. DaoFlashAttention\-3: fast and accurate attention with asynchrony and low\-precision\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2407.08608),[Link](https://arxiv.org/abs/2407.08608)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Shazeer \(2019\)N\. ShazeerFast transformer decoding: one write\-head is all you need\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.1911.02150),[Link](https://arxiv.org/abs/1911.02150)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Shenget al\.\(2023\)Y\. Sheng, L\. Zheng, B\. Yuan, Z\. Li, M\. Ryabinin, D\. Y\. Fu, Z\. Xie, B\. Chen, C\. Barrett, J\. E\. Gonzalez, P\. Liang, C\. Ré, I\. Stoica, and C\. ZhangFlexGen: high\-throughput generative inference of large language models with a single gpu\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2303.06865),[Link](https://arxiv.org/abs/2303.06865)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Shiet al\.\(2025\)J\. Shi, J\. Yang, J\. Liu, X\. Bu, J\. Chen, J\. Zhou, K\. Ma, Z\. Wen, B\. Wang, Y\. He, L\. Song, H\. Zhu, S\. Li, X\. Wang, W\. Zhang, R\. Yuan, Y\. Yao, W\. Yang, Y\. Wang, S\. Fang, S\. Yuan, Q\. He, R\. Tang, Y\. Tan, W\. Zhou, Z\. ZHANG, Z\. Li, W\. Huang, and G\. ZhangKORGym: a dynamic game platform for llm reasoning evaluation\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38, Main Conference,pp\. 161286–161314\.External Links:[Document](https://dx.doi.org/10.52202/085713-5384),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/ebfa4297cd6419f64efe86f657ba49d0-Paper-Conference.pdf)Cited by:[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 8634–8652\.External Links:[Document](https://dx.doi.org/10.52202/075280-0377),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/1b44b878bb782e6954cd888628510e90-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p2.1)\.
- Songet al\.\(2023\)Y\. Song, Z\. Mi, H\. Xie, and H\. ChenPowerInfer: fast large language model serving with a consumer\-grade gpu\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2312.12456),[Link](https://arxiv.org/abs/2312.12456)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Srivatsaet al\.\(2024\)V\. Srivatsa, Z\. He, R\. Abhyankar, D\. Li, and Y\. ZhangPreble: efficient distributed prompt scheduling for llm serving\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2407.00023),[Link](https://arxiv.org/abs/2407.00023)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Sternet al\.\(2019\)M\. Stern, W\. Chan, J\. Kiros, and J\. UszkoreitInsertion transformer: flexible sequence generation via insertion operations\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.1902.03249),[Link](https://arxiv.org/abs/1902.03249)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Sunet al\.\(2026\)W\. Sun, M\. Lu, Z\. Ling, K\. Liu, X\. Yao, Y\. Yang, and J\. ChenScaling long\-horizon agent via context folding\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=lNRgWoGfYg)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.
- Tamet al\.\(2024\)Z\. R\. Tam, C\. Wu, Y\. Tsai, C\. Lin, H\. Lee, and Y\. ChenLet me speak freely? a study on the impact of format restrictions on performance of large language models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2408.02442),[Link](https://arxiv.org/abs/2408.02442)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p1.1)\.
- Tanget al\.\(2024\)J\. Tang, Y\. Zhao, K\. Zhu, G\. Xiao, B\. Kasikci, and S\. HanQuest: query\-aware sparsity for efficient long\-context llm inference\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2406.10774),[Link](https://arxiv.org/abs/2406.10774)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Vaswaniet al\.\(2017\)A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. PolosukhinAttention is all you need\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.1706.03762),[Link](https://arxiv.org/abs/1706.03762)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Wanget al\.\(2018\)C\. Wang, J\. Zhang, and H\. ChenSemi\-autoregressive neural machine translation\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.1808.08583),[Link](https://arxiv.org/abs/1808.08583)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Wanget al\.\(2020\)S\. Wang, B\. Z\. Li, M\. Khabsa, H\. Fang, and H\. MaLinformer: self\-attention with linear complexity\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2006.04768),[Link](https://arxiv.org/abs/2006.04768)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Wanget al\.\(2025a\)Y\. Wang, Z\. Xu, Y\. Huang, X\. Wang, Z\. Song, L\. Gao, C\. Wang, R\. Tang, Y\. Zhao, A\. Cohan, X\. Zhang, and X\. ChenDyFlow: dynamic workflow framework for agentic reasoning\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38, Main Conference,pp\. 174148–174181\.External Links:[Document](https://dx.doi.org/10.52202/085713-5793),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/fe9910d2b03324faeb5371a9658277bb-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.
- Wanget al\.\(2025b\)Z\. Wang, Y\. Wang, X\. Liu, L\. Ding, M\. Zhang, J\. Liu, and M\. ZhangAgentDropout: dynamic agent elimination for token\-efficient and high\-performance LLM\-based multi\-agent collaboration\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 24013–24035\.External Links:[Link](https://aclanthology.org/2025.acl-long.1170/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1170),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p2.1)\.
- Wanget al\.\(2024a\)Z\. Wang, B\. Cui, and S\. GanSqueezeAttention: 2d management of kv\-cache in llm inference via layer\-wise optimal budget\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2404.04793),[Link](https://arxiv.org/abs/2404.04793)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Wanget al\.\(2024b\)Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. NeubigAgent workflow memory\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2409.07429),[Link](https://arxiv.org/abs/2409.07429)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p2.1)\.
- Wuet al\.\(2023\)Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu, A\. H\. Awadallah, R\. W\. White, D\. Burger, and C\. WangAutoGen: enabling next\-gen llm applications via multi\-agent conversation\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2308.08155),[Link](https://arxiv.org/abs/2308.08155)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p2.1)\.
- Xiaoet al\.\(2023\)G\. Xiao, Y\. Tian, B\. Chen, S\. Han, and M\. LewisEfficient streaming language models with attention sinks\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2309.17453),[Link](https://arxiv.org/abs/2309.17453)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Yanet al\.\(2025\)B\. Yan, Z\. Zhou, L\. Zhang, L\. Zhang, Z\. Zhou, D\. Miao, Z\. Li, C\. Li, and X\. ZhangBeyond self\-talk: a communication\-centric survey of llm\-based multi\-agent systems\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2502.14321),[Link](https://arxiv.org/abs/2502.14321)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p1.1)\.
- Yanget al\.\(2026\)X\. Yang, L\. Li, H\. Zhou, T\. Zhu, X\. Qu, Y\. Fan, Q\. Wei, R\. Ye, L\. Kang, Y\. Qin, D\. Liu, Q\. Li, N\. Ding, S\. Chen, and J\. ShaoToward efficient agents: memory, tool learning, and planning\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2601.14192),[Link](https://arxiv.org/abs/2601.14192)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p1.1)\.
- Yaoet al\.\(2024\)J\. Yao, H\. Li, Y\. Liu, S\. Ray, Y\. Cheng, Q\. Zhang, K\. Du, S\. Lu, and J\. JiangCacheBlend: fast large language model serving for rag with cached knowledge fusion\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2405.16444),[Link](https://arxiv.org/abs/2405.16444)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Yaoet al\.\(2023a\)S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. Griffiths, Y\. Cao, and K\. NarasimhanTree of thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 11809–11822\.External Links:[Document](https://dx.doi.org/10.52202/075280-0517),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/271db9922b8d1f4dd7aaef84ed5ac703-Paper-Conference.pdf)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p1.1)\.
- Yaoet al\.\(2023b\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p1.1)\.
- Yeet al\.\(2024\)L\. Ye, Z\. Tao, Y\. Huang, and Y\. LiChunkAttention: efficient self\-attention with prefix\-aware kv cache and two\-phase partition\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2402.15220),[Link](https://arxiv.org/abs/2402.15220)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Yeet al\.\(2025\)Z\. Ye, L\. Chen, R\. Lai, W\. Lin, Y\. Zhang, S\. Wang, T\. Chen, B\. Kasikci, V\. Grover, A\. Krishnamurthy, and L\. CezeFlashInfer: efficient and customizable attention engine for llm inference serving\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2501.01005),[Link](https://arxiv.org/abs/2501.01005)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Zaheeret al\.\(2020\)M\. Zaheer, G\. Guruganesh, A\. Dubey, J\. Ainslie, C\. Alberti, S\. Ontanon, P\. Pham, A\. Ravula, Q\. Wang, L\. Yang, and A\. AhmedBig bird: transformers for longer sequences\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2007.14062),[Link](https://arxiv.org/abs/2007.14062)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Zhanget al\.\(2025a\)G\. Zhang, Y\. Yue, Z\. Li, S\. Yun, G\. Wan, K\. Wang, D\. Cheng, J\. Yu, and T\. ChenCut the crap: an economical communication pipeline for llm\-based multi\-agent systems\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 75389–75428\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/bbc461518c59a2a8d64e70e2c38c4a0e-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.
- Zhanget al\.\(2025b\)J\. Zhang, J\. Xiang, Z\. Yu, F\. Teng, X\. Chen, J\. Chen, M\. Zhuge, X\. Cheng, S\. Hong, J\. Wang, B\. Zheng, B\. Liu, Y\. Luo, and C\. WuAFlow: automating agentic workflow generation\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 34040–34077\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/5492ecbce4439401798dcd2c90be94cd-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p1.1)\.
- Zhanget al\.\(2024\)J\. Zhang, J\. Wang, H\. Li, L\. Shou, K\. Chen, G\. Chen, and S\. MehrotraDraft& verify: lossless large language model acceleration via self\-speculative decoding\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 11263–11282\.External Links:[Link](http://dx.doi.org/10.18653/v1/2024.acl-long.607),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.607)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p3.1)\.
- Zhanget al\.\(2023\)Z\. Zhang, Y\. Sheng, T\. Zhou, T\. Chen, L\. Zheng, R\. Cai, Z\. Song, Y\. Tian, C\. Ré, C\. Barrett, Z\. Wang, and B\. ChenH2\{\}\_\{2\}o: heavy\-hitter oracle for efficient generative inference of large language models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2306.14048),[Link](https://arxiv.org/abs/2306.14048)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p4.1)\.
- Zhaoet al\.\(2021\)T\. Z\. Zhao, E\. Wallace, S\. Feng, D\. Klein, and S\. SinghCalibrate before use: improving few\-shot performance of language models\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2102.09690),[Link](https://arxiv.org/abs/2102.09690)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p1.1)\.
- Zhenget al\.\(2023\)C\. Zheng, H\. Zhou, F\. Meng, J\. Zhou, and M\. HuangLarge language models are not robust multiple choice selectors\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2309.03882),[Link](https://arxiv.org/abs/2309.03882)Cited by:[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p1.1)\.
- Zhenget al\.\(2024\)L\. Zheng, L\. Yin, Z\. Xie, C\. Sun, J\. Huang, C\. H\. Yu, S\. Cao, C\. Kozyrakis, I\. Stoica, J\. E\. Gonzalez, C\. Barrett, and Y\. ShengSGLang: efficient execution of structured language model programs\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 62557–62583\.External Links:[Document](https://dx.doi.org/10.52202/079017-2000),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/724be4472168f31ba1c9ac630f15dec8-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p3.1),[§2\.2](https://arxiv.org/html/2610.00437#S2.SS2.p2.1)\.
- Zhonget al\.\(2024\)Y\. Zhong, S\. Liu, J\. Chen, J\. Hu, Y\. Zhu, X\. Liu, X\. Jin, and H\. ZhangDistServe: disaggregating prefill and decoding for goodput\-optimized large language model serving\.arXiv\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2401.09670),[Link](https://arxiv.org/abs/2401.09670)Cited by:[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p3.1)\.
- Zhouet al\.\(2024\)A\. Zhou, K\. Yan, M\. Shlapentokh\-Rothman, H\. Wang, and Y\. WangLanguage agent tree search unifies reasoning, acting, and planning in language models\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 62138–62160\.External Links:[Link](https://proceedings.mlr.press/v235/zhou24r.html)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.
- Zouet al\.\(2026\)J\. Zou, R\. Qiu, G\. Li, X\. Yang, K\. Tieu, P\. Lu, K\. Shen, H\. Tong, Y\. Choi, J\. He, J\. Zou, M\. Wang, and L\. YangLatent collaboration in multi\-agent systems\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=syG9I9ofd8)Cited by:[§1](https://arxiv.org/html/2610.00437#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00437#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2610.00437#S4.SS1.p2.1)\.

## Appendix AAnalysis of Structured Execution

The following analysis derives the finite action distribution, bounded exploration procedure, and conditions for equivalent shared computation\.

Algorithm 1JevSpawn0:Task

xx, model

θ\\theta, environment

ℰ\\mathcal\{E\}, budgets

T,R,P,KT,R,P,K
1:Initialize

\(ℱ,M\)←\(\{v0\},∅\)\(\\mathcal\{F\},M\)\\leftarrow\(\\\{v\_\{0\}\\\},\\varnothing\)
2:

\(sv0,Hv0,Σv0\)←\(s0,∅,∅\)\(s\_\{v\_\{0\}\},H\_\{v\_\{0\}\},\\Sigma\_\{v\_\{0\}\}\)\\leftarrow\(s\_\{0\},\\varnothing,\\varnothing\)
3:Set

𝒰T​\(v\)=\{submit\}\\mathcal\{U\}\_\{T\}\(v\)=\\\{\\mathrm\{submit\}\\\}for every branch

vv
4:for

t=0,…,Tt=0,\\ldots,Tdo

5:Render

hℱ←prompt⁡\(x,M,\{Hb\}b∈ℱ\)h\_\{\\mathcal\{F\}\}\\leftarrow\\operatorname\{prompt\}\(x,M,\\\{H\_\{b\}\\\}\_\{b\\in\\mathcal\{F\}\}\)
6:Score

ρt←Q¯θ\(⋅∣hℱ,ℱ\)\\rho\_\{t\}\\leftarrow\\bar\{Q\}\_\{\\theta\}\(\\cdot\\mid h\_\{\\mathcal\{F\}\},\\mathcal\{F\}\)
7:Retain

ℱ←TopR⁡\(ℱ,ρt\)\\mathcal\{F\}\\leftarrow\\operatorname\{Top\}\_\{R\}\(\\mathcal\{F\},\\rho\_\{t\}\)and select

v←arg⁡maxb∈ℱ​ρt​\(b\)v\\leftarrow\\arg\\max\_\{b\\in\\mathcal\{F\}\}\\rho\_\{t\}\(b\)
8:Choose

u←arg⁡maxu′∈𝒰t​\(v\)​Q¯θ​\(u′∣hv,𝒰t​\(v\)\)u\\leftarrow\\arg\\max\_\{u^\{\\prime\}\\in\\mathcal\{U\}\_\{t\}\(v\)\}\\bar\{Q\}\_\{\\theta\}\(u^\{\\prime\}\\mid h\_\{v\},\\mathcal\{U\}\_\{t\}\(v\)\)
9:if

u=submitu=\\mathrm\{submit\}then

10:return

Submitθ⁡\(hv,sv\)\\operatorname\{Submit\}\_\{\\theta\}\(h\_\{v\},s\_\{v\}\)
11:elseif

u=reviseu=\\mathrm\{revise\}then

12:

\(Σv,ev\)←Reviseθ⁡\(hv,Σv\)\(\\Sigma\_\{v\},e\_\{v\}\)\\leftarrow\\operatorname\{Revise\}\_\{\\theta\}\(h\_\{v\},\\Sigma\_\{v\}\)
13:

Hv←Hv∥evH\_\{v\}\\leftarrow H\_\{v\}\\mathbin\{\\\|\}e\_\{v\}
14:endif

15:Remove

ℱ←ℱ∖\{v∣u=discard\}\\mathcal\{F\}\\leftarrow\\mathcal\{F\}\\setminus\\\{v\\mid u=\\mathrm\{discard\}\\\}
16:if

ℱ=∅\\mathcal\{F\}=\\varnothingthen

17:returnFailure

18:endif

19:Select parents

ℬ←TopP\(\{b∈ℱ∣χ\(b\)χ\(v\)=1,u∈\{expand,revise\}\},ρt\)\\mathcal\{B\}\\leftarrow\\operatorname\{Top\}\_\{P\}\(\\\{b\\in\\mathcal\{F\}\\mid\\chi\(b\)\\chi\(v\)=1,\\ u\\in\\\{\\mathrm\{expand\},\\mathrm\{revise\}\\\}\\\},\\rho\_\{t\}\)
20:Compose

𝒜b←\{gb\(z\)∣z∈BeamK\(πθ\(⋅∣hb,Σb\)\)\},b∈ℬ\\mathcal\{A\}\_\{b\}\\leftarrow\\\{g\_\{b\}\(z\)\\mid z\\in\\operatorname\{Beam\}\_\{K\}\(\\pi\_\{\\theta\}\(\\cdot\\mid h\_\{b\},\\Sigma\_\{b\}\)\)\\\},\\ b\\in\\mathcal\{B\}
21:Spawn

𝒞←\{w=\(b,a\)∣b∈ℬ,a∈𝒜b\}\\mathcal\{C\}\\leftarrow\\\{w=\(b,a\)\\mid b\\in\\mathcal\{B\},\\ a\\in\\mathcal\{A\}\_\{b\}\\\}
22:for all

w=\(b,a\)∈𝒞w=\(b,a\)\\in\\mathcal\{C\}in paralleldo

23:Execute

\(sw,ow\)←ℰ⁡\(sb,a\)\(s\_\{w\},o\_\{w\}\)\\leftarrow\\mathcal\{E\}\(s\_\{b\},a\)
24:

Hw←Hb∥\(a,ow\),Σw←ΣbH\_\{w\}\\leftarrow H\_\{b\}\\mathbin\{\\\|\}\(a,o\_\{w\}\),\\quad\\Sigma\_\{w\}\\leftarrow\\Sigma\_\{b\}
25:endfor

26:Share

M←M∥\{\(w,a,ow\)\}w=\(b,a\)∈𝒞M\\leftarrow M\\mathbin\{\\\|\}\\\{\(w,a,o\_\{w\}\)\\\}\_\{w=\(b,a\)\\in\\mathcal\{C\}\}
27:Update

ℱ←\(ℱ∖ℬ\)∪𝒞\\mathcal\{F\}\\leftarrow\(\\mathcal\{F\}\\setminus\\mathcal\{B\}\)\\cup\\mathcal\{C\}
28:endfor

In Algorithm[1](https://arxiv.org/html/2610.00437#alg1),s0s\_\{0\}is the initial environment state andv0v\_\{0\}is the root branch\. The budgetsTT,RR,PP, andKKbound interaction rounds, retained branches, expanded parents, and actions per parent, respectively\. The contexthv=prompt⁡\(x,M,Hv\)h\_\{v\}=\\operatorname\{prompt\}\(x,M,H\_\{v\}\)combines task context, branch history, and shared observations\. The operation set𝒰t​\(v\)\\mathcal\{U\}\_\{t\}\(v\)requires revision when a declaration is missing or rejected, permits submission or removal at a terminal branch, and otherwise permits expansion, revision, or submission\. RoundTTpermits submission alone\. The indicatorχ⁡\(v\)\\chi\(v\)identifies nonterminal branches with executable declarations\. Rejected revisions preserve any previous executable declaration and return feedback\. Rejected submission, an empty frontier, or resource exhaustion yields failure\.

Each round selects one operation for the highest ranked branch\. Revision updates that branch alone\. Expansion uses up toPPranked executable parents when the selected branch remains executable, with existing declarations retained for additional parents\. Removal produces no expansion\. Unselected branches remain available until pruning\.

### A\.1Constructing a compositional action space

A declaration defines finite choices and a shared action renderergvg\_\{v\}\. With conditional field domains, the assignment and action spaces are

𝒵⁡\(Σv\)=\{z∈∏j=1mv𝒳j\|zj∈Dj​\(z<j\)\},𝒜⁡\(Σv\)=\{gv​\(z\)∣z∈𝒵⁡\(Σv\)\},\\mathcal\{Z\}\(\\Sigma\_\{v\}\)=\\left\\\{z\\in\\prod\_\{j=1\}^\{m\_\{v\}\}\\mathcal\{X\}\_\{j\}\\,\\middle\|\\,z\_\{j\}\\in D\_\{j\}\(z\_\{<j\}\)\\right\\\},\\qquad\\mathcal\{A\}\(\\Sigma\_\{v\}\)=\\\{g\_\{v\}\(z\)\\mid z\\in\\mathcal\{Z\}\(\\Sigma\_\{v\}\)\\\},\(4\)where𝒳j\\mathcal\{X\}\_\{j\}is the value universe of fieldjj\. Independent domains yield a Cartesian product\. Conditional domains exclude assignments that violate declared dependencies\.

The rules permit one step north or up to two steps east\. Actions usedirectionandstepsfields,r1\\displaystyle r\_\{1\}=movement direction,\\displaystyle=\\text\{movement direction\},D1\\displaystyle D\_\{1\}=\{north,east\},\\displaystyle=\\\{\\mathrm\{north\},\\mathrm\{east\}\\\},\(5\)r2\\displaystyle r\_\{2\}=number of steps,\\displaystyle=\\text\{number of steps\},D2​\(z1\)\\displaystyle D\_\{2\}\(z\_\{1\}\)=\{1,…,1\+𝕀\[z1=east\]\},\\displaystyle=\\\{1,\\ldots,1\+\\mathbb\{I\}\[z\_\{1\}=\\mathrm\{east\}\]\\\},\(6\)gv​\(z1,z2\)\\displaystyle g\_\{v\}\(z\_\{1\},z\_\{2\}\)=\{"direction":z1, "steps":z2\}\.\\displaystyle=\\texttt\{\\lx@text@lbrace\\char 34\\relax direction\\char 34\\relax: \}z\_\{1\}\\texttt\{, \\char 34\\relax steps\\char 34\\relax: \}z\_\{2\}\\texttt\{\\lx@text@rbrace\}\.\(7\)The renderer inserts quoted directions and integer step counts into shared syntax\. Joint evaluation uses the block𝒟b1​\(∅\)=\{\(north,1\),\(east,1\),\(east,2\)\}\.\\mathcal\{D\}\_\{b\_\{1\}\}\(\\varnothing\)=\\\{\(\\mathrm\{north\},1\),\(\\mathrm\{east\},1\),\(\\mathrm\{east\},2\)\\\}\.\(8\)The induced policy satisfiesπθ​\(z∣hv,Σv\)=Q¯θ​\(z∣hv,𝒟b1​\(∅\)\),∑z∈𝒟b1​\(∅\)πθ​\(z∣hv,Σv\)=1\.\\pi\_\{\\theta\}\(z\\mid h\_\{v\},\\Sigma\_\{v\}\)=\\bar\{Q\}\_\{\\theta\}\(z\\mid h\_\{v\},\\mathcal\{D\}\_\{b\_\{1\}\}\(\\varnothing\)\),\\qquad\\sum\_\{z\\in\\mathcal\{D\}\_\{b\_\{1\}\}\(\\varnothing\)\}\\pi\_\{\\theta\}\(z\\mid h\_\{v\},\\Sigma\_\{v\}\)=1\.\(9\)Direction and step count are scored jointly\. The two highest ranked assignments can be executed in separate environment copies, with returned observations determining the next branch\.

Optional arguments and variable length objects introduce presence or length fields before dependent values\. Open strings require model generated structure when no finite domain is supplied\. Syntax and type checks return declaration feedback for revision\. Serialization fixes the output form, while the domains remain model predictions\.

### A\.2Probability on a finite value tree

Let𝒪\\mathcal\{O\}be a nonempty finite option list\. Entryiiis serialized asτ⁡\(i\)\\tau\(i\)with a reserved termination token, yielding the prefix free set𝒯=\{τ⁡\(i\)∣i∈𝒪\}\\mathcal\{T\}=\\\{\\tau\(i\)\\mid i\\in\\mathcal\{O\}\\\}of distinct sequences\. The contexthhcontains task evidence, scoring instructions, and option descriptions\. Action options specify complete arguments, while integer branch identifiers refer to states and observations\.

Let𝒩\\mathcal\{N\}contain every proper prefix of a sequence in𝒯\\mathcal\{T\}, includingϵ\\epsilon, and letC⁡\(u\)C\(u\)be the outgoing tokens atu∈𝒩u\\in\\mathcal\{N\}\. The pretrained hidden statefθ​\(h,u\)f\_\{\\theta\}\(h,u\)and output rowwyw\_\{y\}define

qθ​\(y∣h,u,C⁡\(u\)\)=exp⁡\(wy⊤​fθ​\(h,u\)\)∑y′∈C⁡\(u\)exp⁡\(wy′⊤​fθ​\(h,u\)\),y∈C⁡\(u\)\.q\_\{\\theta\}\(y\\mid h,u,C\(u\)\)=\\frac\{\\exp\\bigl\(w\_\{y\}^\{\\top\}f\_\{\\theta\}\(h,u\)\\bigr\)\}\{\\sum\_\{y^\{\\prime\}\\in C\(u\)\}\\exp\\bigl\(w\_\{y^\{\\prime\}\}^\{\\top\}f\_\{\\theta\}\(h,u\)\\bigr\)\},\\qquad y\\in C\(u\)\.\(10\)A singleton continuation has probability one\. For a leafτ∈𝒯\\tau\\in\\mathcal\{T\},

Qθ​\(τ∣h,𝒯\)=∏\(u,y\)∈path⁡\(τ\)\|\|C⁡\(u\)\|\>1qθ​\(y∣h,u,C⁡\(u\)\)\.Q\_\{\\theta\}\(\\tau\\mid h,\\mathcal\{T\}\)=\\prod\_\{\(u,y\)\\in\\operatorname\{path\}\(\\tau\)\\,\\mid\\,\|C\(u\)\|\>1\}q\_\{\\theta\}\(y\\mid h,u,C\(u\)\)\.\(11\)
###### Proposition A\.1\.

For finite, nonempty, prefix free𝒯\\mathcal\{T\},

Qθ​\(τ∣h,𝒯\)≥0,∑τ∈𝒯Qθ​\(τ∣h,𝒯\)=1\.Q\_\{\\theta\}\(\\tau\\mid h,\\mathcal\{T\}\)\\geq 0,\\qquad\\sum\_\{\\tau\\in\\mathcal\{T\}\}Q\_\{\\theta\}\(\\tau\\mid h,\\mathcal\{T\}\)=1\.\(12\)

###### Proof\.

μ⁡\(ϵ\)\\displaystyle\\mu\(\\epsilon\)=1,μ⁡\(u​y\)=μ⁡\(u\)​qθ​\(y∣h,u,C⁡\(u\)\)≥0,\\displaystyle=1,\\qquad\\mu\(uy\)=\\mu\(u\)q\_\{\\theta\}\(y\\mid h,u,C\(u\)\)\\geq 0,∑y∈C⁡\(u\)μ⁡\(u​y\)\\displaystyle\\sum\_\{y\\in C\(u\)\}\\mu\(uy\)=μ⁡\(u\),u∈𝒩,\\displaystyle=\\mu\(u\),\\qquad u\\in\\mathcal\{N\},∑τ∈𝒯Qθ​\(τ∣h,𝒯\)\\displaystyle\\sum\_\{\\tau\\in\\mathcal\{T\}\}Q\_\{\\theta\}\(\\tau\\mid h,\\mathcal\{T\}\)=∑τ∈𝒯μ⁡\(τ\)=μ⁡\(ϵ\)=1\.∎\\displaystyle=\\sum\_\{\\tau\\in\\mathcal\{T\}\}\\mu\(\\tau\)=\\mu\(\\epsilon\)=1\.\\qed

Definec⁡\(τ\)=\|\{i∈𝒪∣τ⁡\(i\)=τ\}\|c\(\\tau\)=\|\\\{i\\in\\mathcal\{O\}\\mid\\tau\(i\)=\\tau\\\}\|\. The option distribution follows from Proposition[A\.1](https://arxiv.org/html/2610.00437#A1.Thmproposition1),

Q¯θ​\(i∣h,𝒪\)=Qθ​\(τ​\(i\)∣h,𝒯\)c⁡\(τ⁡\(i\)\),∑i∈𝒪Q¯θ​\(i∣h,𝒪\)=∑τ∈𝒯c⁡\(τ\)​Qθ​\(τ∣h,𝒯\)c⁡\(τ\)=1\.\\bar\{Q\}\_\{\\theta\}\(i\\mid h,\\mathcal\{O\}\)=\\frac\{Q\_\{\\theta\}\(\\tau\(i\)\\mid h,\\mathcal\{T\}\)\}\{c\(\\tau\(i\)\)\},\\qquad\\sum\_\{i\\in\\mathcal\{O\}\}\\bar\{Q\}\_\{\\theta\}\(i\\mid h,\\mathcal\{O\}\)=\\sum\_\{\\tau\\in\\mathcal\{T\}\}c\(\\tau\)\\frac\{Q\_\{\\theta\}\(\\tau\\mid h,\\mathcal\{T\}\)\}\{c\(\\tau\)\}=1\.\(13\)The same distribution scores block assignments, branches, and operations\. Figure[2](https://arxiv.org/html/2610.00437#S3.F2)abbreviates the conditional distribution asQ¯θ​\(i\)\\bar\{Q\}\_\{\\theta\}\(i\)\. Duplicate serialized values share mass equally, and duplicate executable actions are collapsed within each parent state\.

The locally restricted distribution differs in general from conditioning the unrestricted model on𝒯\\mathcal\{T\},

pθ​\(τ∣h,τ∈𝒯\)=pθ​\(τ∣h\)∑τ′∈𝒯pθ​\(τ′∣h\)\.p\_\{\\theta\}\(\\tau\\mid h,\\tau\\in\\mathcal\{T\}\)=\\frac\{p\_\{\\theta\}\(\\tau\\mid h\)\}\{\\sum\_\{\\tau^\{\\prime\}\\in\\mathcal\{T\}\}p\_\{\\theta\}\(\\tau^\{\\prime\}\\mid h\)\}\.\(14\)Equation[14](https://arxiv.org/html/2610.00437#A1.E14)includes forced token likelihoods and the probability of remaining in the tree\. Local normalization omits these factors and defines a relative action preference, not a probability of execution success\.

### A\.3Parallel evaluation and value ordering

Let𝒩b=\{u∈𝒩∣\|C⁡\(u\)\|\>1\}\\mathcal\{N\}\_\{b\}=\\\{u\\in\\mathcal\{N\}\\mid\|C\(u\)\|\>1\\\}\. Scoring requires hidden states only at these branching prefixes\. Known prefix tokens permit parallel evaluation with the same ancestor visibility, positions, and recurrent initial states as independent evaluation\.

Forced tokens affect subsequent hidden states but require no output score\. At prefixuu, projection usesdmodel​\|C⁡\(u\)\|d\_\{\\mathrm\{model\}\}\|C\(u\)\|multiplications instead ofdmodel​\|𝒱\|d\_\{\\mathrm\{model\}\}\|\\mathcal\{V\}\|, wheredmodeld\_\{\\mathrm\{model\}\}is model width and𝒱\\mathcal\{V\}is the vocabulary\. Prefix computation is retained\.

Action options are sorted by serialized field assignments\. Field grouping, branch indexing, and tie resolution remain representation dependent\.

### A\.4Conditional composition and bounded exploration

Each unfinished block has a nonempty conditional domain𝒟bℓ​\(z<bℓ\)\\mathcal\{D\}\_\{b\_\{\\ell\}\}\(z\_\{<b\_\{\\ell\}\}\)\. Completed assignments have the absorbing domain\{ϵ\}\\\{\\epsilon\\\}, withz∥ϵ=zz\\mathbin\{\\\|\}\\epsilon=zandQ¯θ​\(ϵ∣hvz,\{ϵ\}\)=1\\bar\{Q\}\_\{\\theta\}\(\\epsilon\\mid h\_\{v\}^\{z\},\\\{\\epsilon\\\}\)=1\. Bound assignments enterhvz<bℓh\_\{v\}^\{z\_\{<b\_\{\\ell\}\}\}\. The complete log probability is

Sθ​\(z∣hv,Σv\)=log⁡πθ​\(z∣hv,Σv\)=∑ℓ=1Llog⁡Q¯θ​\(zbℓ∣hvz<bℓ,𝒟bℓ​\(z<bℓ\)\)\.S\_\{\\theta\}\(z\\mid h\_\{v\},\\Sigma\_\{v\}\)=\\log\\pi\_\{\\theta\}\(z\\mid h\_\{v\},\\Sigma\_\{v\}\)=\\sum\_\{\\ell=1\}^\{L\}\\log\\bar\{Q\}\_\{\\theta\}\\bigl\(z\_\{b\_\{\\ell\}\}\\mid h\_\{v\}^\{z\_\{<b\_\{\\ell\}\}\},\\mathcal\{D\}\_\{b\_\{\\ell\}\}\(z\_\{<b\_\{\\ell\}\}\)\\bigr\)\.\(15\)The partial score is defined recursively,

S0​\(∅\)\\displaystyle S\_\{0\}\(\\varnothing\)=0,\\displaystyle=0,\(16\)Sℓ​\(z≤bℓ\)\\displaystyle S\_\{\\ell\}\(z\_\{\\leq b\_\{\\ell\}\}\)=Sℓ−1​\(z<bℓ\)\+log⁡Q¯θ​\(zbℓ∣hvz<bℓ,𝒟bℓ​\(z<bℓ\)\)\.\\displaystyle=S\_\{\\ell\-1\}\(z\_\{<b\_\{\\ell\}\}\)\+\\log\\bar\{Q\}\_\{\\theta\}\\bigl\(z\_\{b\_\{\\ell\}\}\\mid h\_\{v\}^\{z\_\{<b\_\{\\ell\}\}\},\\mathcal\{D\}\_\{b\_\{\\ell\}\}\(z\_\{<b\_\{\\ell\}\}\)\\bigr\)\.\(17\)Let𝒥ℓ\\mathcal\{J\}\_\{\\ell\}denote the beam after blockℓ\\ell\. With𝒥0=\{∅\}\\mathcal\{J\}\_\{0\}=\\\{\\varnothing\\\},

Extend⁡\(𝒥,b\)\\displaystyle\\operatorname\{Extend\}\(\\mathcal\{J\},b\)=\{z∥zb∣z∈𝒥,zb∈𝒟b\(z\)\},\\displaystyle=\\\{z\\mathbin\{\\\|\}z\_\{b\}\\mid z\\in\\mathcal\{J\},\\ z\_\{b\}\\in\\mathcal\{D\}\_\{b\}\(z\)\\\},\(18\)𝒥ℓ\\displaystyle\\mathcal\{J\}\_\{\\ell\}=TopK⁡\(Extend⁡\(𝒥ℓ−1,bℓ\),Sℓ\)\.\\displaystyle=\\operatorname\{Top\}\_\{K\}\\bigl\(\\operatorname\{Extend\}\(\\mathcal\{J\}\_\{\\ell\-1\},b\_\{\\ell\}\),S\_\{\\ell\}\\bigr\)\.\(19\)Completed assignments retain the accumulated scores without another model evaluation\. The final beam definesBeamK\\operatorname\{Beam\}\_\{K\}in Algorithm[1](https://arxiv.org/html/2610.00437#alg1)\.

Before pruning, normalization follows by summing over the conditional domains from the final block backward,

∑z∈𝒵⁡\(Σv\)πθ​\(z∣hv,Σv\)\\displaystyle\\sum\_\{z\\in\\mathcal\{Z\}\(\\Sigma\_\{v\}\)\}\\pi\_\{\\theta\}\(z\\mid h\_\{v\},\\Sigma\_\{v\}\)=∑zb1∈𝒟b1​\(∅\)⋯∑zbL∈𝒟bL​\(z<bL\)∏ℓ=1LQ¯θ\(zbℓ∣hvz<bℓ,𝒟bℓ\(z<bℓ\)\)\\displaystyle=\\sum\_\{z\_\{b\_\{1\}\}\\in\\mathcal\{D\}\_\{b\_\{1\}\}\(\\varnothing\)\}\\cdots\\sum\_\{z\_\{b\_\{L\}\}\\in\\mathcal\{D\}\_\{b\_\{L\}\}\(z\_\{<b\_\{L\}\}\)\}\\prod\_\{\\ell=1\}^\{L\}\\bar\{Q\}\_\{\\theta\}\\bigl\(z\_\{b\_\{\\ell\}\}\\mid h\_\{v\}^\{z\_\{<b\_\{\\ell\}\}\},\\mathcal\{D\}\_\{b\_\{\\ell\}\}\(z\_\{<b\_\{\\ell\}\}\)\\bigr\)=1\.\\displaystyle=1\.\(20\)Pruning removes probability mass and may discard a partial assignment with a higher scoring completion\. Beam selection therefore approximates exhaustive ranking\.

IfPt≤PP\_\{t\}\\leq Pparents are expanded in roundt<Tt<Tandδt\\delta\_\{t\}branches are discarded after retention, then

\|𝒞t\|≤Pt​K,\|ℱt\+1\|≤R−δt−Pt\+Pt​K\.\|\\mathcal\{C\}\_\{t\}\|\\leq P\_\{t\}K,\\qquad\|\\mathcal\{F\}\_\{t\+1\}\|\\leq R\-\\delta\_\{t\}\-P\_\{t\}\+P\_\{t\}K\.\(21\)Discarded branches and expanded parents are disjoint\. Further pruning or deduplication can only reduce this upper bound\. Since roundTTpermits submission alone,

NT=∑t=0T−1\|𝒞t\|≤∑t=0T−1Pt​K≤T​P​K\.N\_\{T\}=\\sum\_\{t=0\}^\{T\-1\}\|\\mathcal\{C\}\_\{t\}\|\\leq\\sum\_\{t=0\}^\{T\-1\}P\_\{t\}K\\leq TPK\.\(22\)The bound counts executed child actions\. Declaration, branch selection, and submission are excluded\. Domain saturation, deduplication, and early termination reduce the realized count\.

### A\.5Equivalence of shared attention

For each query row𝐪\\mathbf\{q\}, partition the attended tokens into prefix indicesIpI\_\{p\}and the visible suffix indicesIsI\_\{s\}\. Letdheadd\_\{\\mathrm\{head\}\}be the attention head dimension\. Forα∈\{p,s\}\\alpha\\in\\\{p,s\\\},

Zα=∑k∈Iαexp⁡\(𝐪⊤​𝐤kdhead\),λα=log⁡Zα,𝐀α=Zα−1​∑k∈Iαexp⁡\(𝐪⊤​𝐤kdhead\)​𝐯k\.Z\_\{\\alpha\}=\\sum\_\{k\\in I\_\{\\alpha\}\}\\exp\\\!\\left\(\\frac\{\\mathbf\{q\}^\{\\top\}\\mathbf\{k\}\_\{k\}\}\{\\sqrt\{d\_\{\\mathrm\{head\}\}\}\}\\right\),\\quad\\lambda\_\{\\alpha\}=\\log Z\_\{\\alpha\},\\quad\\mathbf\{A\}\_\{\\alpha\}=Z\_\{\\alpha\}^\{\-1\}\\sum\_\{k\\in I\_\{\\alpha\}\}\\exp\\\!\\left\(\\frac\{\\mathbf\{q\}^\{\\top\}\\mathbf\{k\}\_\{k\}\}\{\\sqrt\{d\_\{\\mathrm\{head\}\}\}\}\\right\)\\mathbf\{v\}\_\{k\}\.\(23\)For nonempty partitions, shared attention is recovered by

𝐀=eλp​𝐀p\+eλs​𝐀seλp\+eλs\.\\mathbf\{A\}=\\frac\{e^\{\\lambda\_\{p\}\}\\mathbf\{A\}\_\{p\}\+e^\{\\lambda\_\{s\}\}\\mathbf\{A\}\_\{s\}\}\{e^\{\\lambda\_\{p\}\}\+e^\{\\lambda\_\{s\}\}\}\.\(24\)Indeed, for each query row,

Zp​𝐀p\+Zs​𝐀sZp\+Zs=∑k∈Ip∪Isexp⁡\(𝐪⊤​𝐤k/dhead\)​𝐯k∑k∈Ip∪Isexp⁡\(𝐪⊤​𝐤k/dhead\)\.\\frac\{Z\_\{p\}\\mathbf\{A\}\_\{p\}\+Z\_\{s\}\\mathbf\{A\}\_\{s\}\}\{Z\_\{p\}\+Z\_\{s\}\}=\\frac\{\\sum\_\{k\\in I\_\{p\}\\cup I\_\{s\}\}\\exp\\\!\\left\(\\mathbf\{q\}^\{\\top\}\\mathbf\{k\}\_\{k\}/\\sqrt\{d\_\{\\mathrm\{head\}\}\}\\right\)\\mathbf\{v\}\_\{k\}\}\{\\sum\_\{k\\in I\_\{p\}\\cup I\_\{s\}\}\\exp\\\!\\left\(\\mathbf\{q\}^\{\\top\}\\mathbf\{k\}\_\{k\}/\\sqrt\{d\_\{\\mathrm\{head\}\}\}\\right\)\}\.\(25\)An empty partition contributes zero mass, leaving attention over the nonempty partition unchanged\.

A recurrent layer initialized from the shared prefix state follows the recurrence of independent sequence evaluation\. Induction over layers establishes equivalence for matching positions, causal visibility, parameters, and initial states in exact arithmetic\.

### A\.6Sequential latency and shared work

For a completed trajectory, letΔdecl\\Delta\_\{\\mathrm\{decl\}\}cover declaration and revision,Δsubmit\\Delta\_\{\\mathrm\{submit\}\}submission, andΔother\\Delta\_\{\\mathrm\{other\}\}scheduling\. Per round,Δrank,t\\Delta\_\{\\mathrm\{rank\},t\},Δcontrol,t\\Delta\_\{\\mathrm\{control\},t\},Δspawn,t\\Delta\_\{\\mathrm\{spawn\},t\}, andΔenv,t\\Delta\_\{\\mathrm\{env\},t\}denote branch selection, operation choice, action scoring, and environment execution\. Partitioning elapsed time gives

Δtask\\displaystyle\\Delta\_\{\\mathrm\{task\}\}=Δdecl\+Δsubmit\+Δother\\displaystyle=\\Delta\_\{\\mathrm\{decl\}\}\+\\Delta\_\{\\mathrm\{submit\}\}\+\\Delta\_\{\\mathrm\{other\}\}\+∑t\(Δrank,t\+Δcontrol,t\+Δspawn,t\+Δenv,t\)\.\\displaystyle\\quad\+\\sum\_\{t\}\\bigl\(\\Delta\_\{\\mathrm\{rank\},t\}\+\\Delta\_\{\\mathrm\{control\},t\}\+\\Delta\_\{\\mathrm\{spawn\},t\}\+\\Delta\_\{\\mathrm\{env\},t\}\\bigr\)\.\(26\)Concurrent trajectories may share GPU computation\.

A common prefix ofnpn\_\{p\}tokens saves\(B−1\)​np\(B\-1\)n\_\{p\}input tokens acrossBBrequests\. Reusing an unchanged history ofnhist,tn\_\{\\mathrm\{hist\},t\}tokens saves the corresponding prefill at the next compatible request\. Suffix attention, projection, padding, and scheduling costs remain\.

## Appendix BExperimental Details

### B\.1Tasks and Environment Interfaces

Task context contains the goal, initial observation, interaction rules, and answer format, shared across methods\. Method prompts are supplied separately, with core JevSpawn excerpts in Appendix[B\.6](https://arxiv.org/html/2610.00437#A2.SS6)\. Benchmark interfaces execute and score actions, and copied environment states support branch exploration\. JevSpawn infers missing fields and reuses finite domains provided by task rules\. Maze and Grid specify complete domains, so execution requires no text generation\.

PPNL contains 1,136 instances, with 250 each from the 5 by 5 and 7 by 7 test collections, 336 from the standard collection, and 300 from the additional obstacle collection\. Maze contains 25 initial states\. Each remaining task contains 100 instances generated from seeds 42 through 141\. Invalid submissions and timeouts count as failures\. TextArena rewards and KORGym scores retain the benchmark scales\.

### B\.2Execution and Timing

Up to eight ready requests are processed together, with occupancy determined by method dependencies\. E2E latency includes action construction, queueing, model computation, environment interaction, and submission\. Model loading is excluded\. Observed elapsed times are retained for timed out tasks\.

Time to first token \(TTFT\) spans batch preparation and prefill through the first generated token\. Inter\-token latency \(ITL\) measures successive token intervals within a sequence, including interleaved finite evaluations\. Finite evaluation latency covers preparation, context processing, and value scoring for a complete batch\. Queueing and environment execution are timed separately\.

Text decoding throughput divides tokens emitted after the first token by the corresponding batch decoding duration measured with CUDA events\. Intervals with interleaved finite evaluation are excluded\. Isolated decoding uses the same token and time definitions on replayed requests\. Text generation throughput in the batch scaling experiment includes prefill and decoding\. Finite throughput counts requests per second, with one request evaluating an option set containing multiple assignments\.

\(a\)Asynchronous execution\(b\)Prefix reuse
Figure 5:Request scheduling and prefix reuse during JevSpawn execution on PPNL\. \(a\) Request durations and returned observations during asynchronous execution\. Teal spans indicate spawn scoring, with the mean batch latency annotated\. Text generation can span intervening finite evaluations\. \(b\) Computed and reused input tokens for successive finite evaluation batches\.In Figure[5](https://arxiv.org/html/2610.00437#A2.F5), batches containing spawn scoring or state selection average 0\.53 seconds, and batches containing operation selection average 0\.47 seconds\. Different scoring operations can share a batch\. During mixed text and finite evaluation, text generation averages 0\.52 seconds TTFT and 137 milliseconds between emitted tokens\. The isolated decoding experiment below measures token intervals without intervening finite requests\.

### B\.3Action Probability Evaluation

Profiling replays fixed requests in batches of eight\. Per task, sixteen text requests and sixteen finite requests from each of the short and long history groups are used\. Each batch has one warmup and three measured repetitions\.

Isolated text decoding reaches 630 to 660 tokens per second with a mean ITL of 12 milliseconds\. Prefill processes 29,000 to 33,000 uncached tokens per second\. Finite evaluation takes 0\.26 to 0\.49 seconds per batch with short histories and 0\.26 to 2\.56 seconds with long histories, including context extension and probability computation\.

Figure[6](https://arxiv.org/html/2610.00437#A2.F6)shows four GPU traces at a fixed number of alternatives\. Longer uncached histories increase matrix computation and data movement\. Collective communication includes device synchronization\.

\(a\)Short history, 5,290 computed tokens\(b\)Long history, 21,450 computed tokens
Figure 6:GPU activity during finite action evaluation of eight requests and 77 assignments\. Teal indicates model computation, purple indicates collective communication, and light gray indicates gaps between kernels\. Each timeline begins at the first kernel on the corresponding GPU\.
### B\.4TypeSafe Jev Comparison

The TypeSafe Jev variant uses API modeljev\-1\.13\.0for finite scoring and Qwen for declarations and text generation\. Both scorers receive the same task context and history under the common input limit\. Branch exploration, action execution, and task evaluation are unchanged\. E2E latency includes network and API service time\.

### B\.5Further Performance Analysis

Among the seven agent baselines, AgentPrune achieves the highest score on four tasks\. Figure[7](https://arxiv.org/html/2610.00437#A2.F7)compares paired instance latencies and scores with JevSpawn\. Timeout pairs are marked separately and excluded from the median latency curve\.

\(a\)PPNL\(b\)Maze\(c\)Grid\(d\)LightsOut
Figure 7:Paired task times for JevSpawn and AgentPrune\. Each point represents the same instance under both methods\. Points below the diagonal indicate shorter JevSpawn execution\. Filled teal, gray, and open navy marks denote higher, equal, and lower JevSpawn scores\. Crosses identify pairs containing a timeout\. Marginal histograms show elapsed time distributions\. The dark curve and band summarize median and interquartile JevSpawn times within equal\-count groups of AgentPrune times, excluding timeout pairs\.\(a\)RushHour\(b\)Sokoban\(c\)2048\(d\)Nullify
Figure 8:Continued\. \(e\) RushHour, \(f\) Sokoban, \(g\) 2048, and \(h\) Nullify\.
### B\.6Core Prompts

The following excerpts specify action construction, joint scoring, and feedback guided continuation\. The task context and observed history accompany these instructions\.

Declare the reusable commands for one environment transition\. Use the smallest executable transaction accepted by the public interface\. When a command permits a sequence of moves, declare one move and let subsequent turns execute subsequent moves after observing their results\. When the interface requires a joint tuple in one call, declare that complete tuple\. Output one signature per primitive command form\. The same signature is reused at every applicable state\. Fixed native prefixes, separators, spacing and punctuation remain literally in the signature\. Replace each variable component with a named slot whose complete finite domain is declared inline as \{name:enum\("value\_a", "value\_b"\)\} or \{name:range\(start, stop\)\}\. Here range uses integers with an inclusive start and exclusive stop\.

Select the joint field assignment whose resulting action makes the best progress toward the task’s stated goal from the current state\. Each option binds the pending fields together\. Evaluate their compatibility, the action’s applicability, and the latest feedback as one decision\. Previously bound fields remain fixed\.

Use the task rules, executed actions and returned observations\. A stopped environment may represent failure\. Determine the outcome from its observation\. The following operation can continue execution, revise the action declaration, submit, or discard a stopped branch\.Choose the next operation for the selected branch using its active declaration and the latest actual action\-observation pairs below\. A declaration with a reported native syntax or argument\-format error needs revise so that subsequent values produce executable commands\. A well\-formed command rejected by the current environment state needs a different value combination through expand, unless the required command or value is absent from the declaration\. A completed task supported by actual observations needs submit\.

## Appendix CAblation Studies

### C\.1Expansion Width

Expansion width controls the maximum number of child actions from one parent\. Table[4](https://arxiv.org/html/2610.00437#A3.T4)compares widths of one, two, four, eight, and ten against the default width of four\. Width one reduces latency but substantially lowers scores on Maze, Grid, and LightsOut\. Wider expansion yields task dependent gains\.

Table 4:Effect of exploration width on score \(↑\\uparrow\) and E2E latency \(s,↓\\downarrow\)\. Bold values mark the best result for each metric and task, including ties\.\(a\)PPNL\(b\)Maze\(c\)Grid\(d\)LightsOut\(e\)RushHour\(f\)Sokoban\(g\)2048\(h\)Nullify
Figure 9:Task performance and E2E latency across expansion widths\. Circled labels indicate the maximum number of spawned actions,KK\. Dotted lines mark the reference setting,K=4K=4\.
### C\.2Expansion Topology

Table[5](https://arxiv.org/html/2610.00437#A3.T5)allocates four actions as1×41\\times 4,2×22\\times 2, or4×14\\times 1parent–child configurations\. Two parents achieve the highest Maze and Grid success, while four parents achieve the highest 2048 score\. The action budget is fixed across configurations\.

Table 5:Effect of exploration topology under a fixed expansion budget\. Score \(↑\\uparrow\) and E2E latency \(s,↓\\downarrow\) are reported\. Bold values mark the best result for each metric and task, including ties\.
### C\.3Declaration Revision

Declaration revision is disabled after the first executable declaration in Table[6](https://arxiv.org/html/2610.00437#A3.T6)\. Feedback and branch selection remain active\. Fixed declarations improve scores on four tasks and reduce latency on six, while adaptive declarations score higher on the other four tasks\.

Table 6:Ablations of action declaration revision and cross\-branch observation sharing\. Score \(↑\\uparrow\) and E2E latency in seconds \(↓\\downarrow\) are reported for each setting\. Bold values mark the best result for each metric and task, including ties\.
### C\.4Branch Information Sharing

Table[6](https://arxiv.org/html/2610.00437#A3.T6)restricts observations to the current branch history\. Shared observations reduce latency on all eight tasks and improve scores on PPNL, Sokoban, 2048, and Nullify\. Restricting observations improves scores on Maze, Grid, LightsOut, and RushHour\.

### C\.5Interaction Length and Time Budget

Round limits of 36, 54, 72, and 108 are evaluated at a context limit of 131,072 tokens, with all other settings fixed\. Each condition is run with a 300 second deadline and without a deadline\. Table[7](https://arxiv.org/html/2610.00437#A3.T7)reports scores, E2E latency, and timeout rates for the timed series\. Figure[10](https://arxiv.org/html/2610.00437#A3.F10)shows the score trends\.

Table 7:Interaction length with and without a 300 s time budget\. Score \(↑\\uparrow\), E2E latency in seconds \(↓\\downarrow\), and timeout rate TO in percent \(↓\\downarrow\) are reported\. Bold values mark the best result within each budget condition for each metric and task, including ties\.\(a\)PPNL\(b\)Maze\(c\)Grid\(d\)LightsOut\(e\)RushHour\(f\)Sokoban\(g\)2048\(h\)Nullify
Figure 10:Task performance and timeout incidence across interaction horizons\. Shading separates scores with a fixed deadline and unrestricted time\. Brackets mark the gap at 108 rounds, with timeout rates below\.Without a deadline, the 2048 score rises from 315\.16 at 36 rounds to 603\.72, 912\.52, and 1102\.24 at 54, 72, and 108 rounds\. LightsOut reward rises from 0\.61 to 0\.876\. Navigation success changes little, and RushHour reward peaks at 0\.47 at 72 rounds before declining to 0\.44\.

With the deadline enforced, 82 and 85 percent of 2048 tasks time out at 72 and 108 rounds, reducing scores to 96\.60 and 124\.84\. Sokoban reaches a timeout rate of 77 percent at 108 rounds\. Timed out tasks receive zero reward\.

At 108 rounds without a deadline, mean E2E latency reaches 464\.67 seconds on 2048 and 415\.77 seconds on Sokoban, with 99th percentiles of 612 and 559 seconds\. Sokoban reward increases from 0\.12 at 36 rounds to 0\.18\. In 2048, 92 percent of tasks exceed 300 seconds and nine percent exceed 600 seconds\. The longest observed run takes 623 seconds\.

Removing only the deadline at the main context limit of 16,384 tokens and 36 rounds yields success rates of 0\.951, 0\.96, and 0\.94 on PPNL, Maze, and Grid\.

### C\.6Batch Scaling

Batch scaling replays 128 fixed requests for each of text generation and finite evaluation, with one warmup and three measured repetitions per batch\. Throughput includes input processing\. Finite evaluation is separated by history length and measured in requests per second, with each request scoring one option set\.

\(a\)Throughput scaling\(b\)Latency scaling\(c\)GPU memory\(d\)Effect of history length
Figure 11:Throughput, latency, and memory under increasing physical batch size\. \(a\) The throughput curve highlights the best measured batch and the gain over batch 8\. \(b\) ITL and TTFT are normalized to batch 8, with absolute values annotated\. \(c\) Memory intervals connect allocated and reserved maxima across four GPUs\. \(d\) Paired finite throughput measurements contrast short and long histories, with ratios and batch latencies shown\. Finite throughput includes context processing\. The next tested text and finite batches, 128 and 32, exceed memory\.Text generation throughput, including prefill, peaks at batch size 16, increasing from 138\.2 to 164\.2 tokens per second relative to batch size 8\. Mean ITL rises from 12\.28 to 17\.54 milliseconds between batch sizes 8 and 64\. Finite throughput remains near 28 requests per second for short histories and six for long histories at both batch sizes 8 and 16\.

The largest completed batch sizes are 64 for text generation and 16 for finite evaluation\. The next tested sizes, 128 and 32, exceed GPU memory\.

## Appendix DCase Studies

One instance from each benchmark task illustrates how finite action fields support exploration\. JevSpawn attains the maximum reward on seven selected instances, whereas most baselines do not complete the corresponding task\. The 2048 example is compared by accumulated score\. Consecutive rounds are shown in alternating colors, with observations condensed to the returned state changes and terminal feedback\. Branch identifiers link states across rounds, and bold identifiers mark the submitted path\. Recovery in a round title denotes a return to an earlier branch\. Field identifiers label the slots in each action template\.

### D\.1PPNL

The goal lies beyond two obstacles in the top row\. The selected path descends to the second row, crosses below both obstacles, and returns to the goal in seven moves\.

Additional obstacle test, instance 164\. JevSpawn score 1\.LATSLLMCompilerAgentPruneHiAgentFoldAgentDyFlowLatentMAS0010010

Figure 12:PPNL path around the upper row obstacles\. The circle and star mark the start and goal\. Arrows show the submitted moves\.
Initial state\.Fieldsf0\(movement direction\)∈\\in\{up,down,left,right\}\.Action templateexecute\(actions = $\{f0\}\)\.Selected branchn2, selection probability 0\.996\.Selected branchn4, selection probability 0\.970\.Selected branchn10, selection probability 0\.510\.Selected branchn14, selection probability 0\.562\.Selected branchn18, selection probability 0\.824\.Selected branchn23, selection probability 0\.992\.Selected branchn26, selection probability 0\.994\.Submitted actionsright,down,right,right,right,up,right\.

### D\.2Maze

Only local wall observations are available\. An initial right move is blocked\. Subsequent selections follow an observed corridor to the goal without requiring a complete map\.

Initial state 10\. JevSpawn score 1\.LATSLLMCompilerAgentPruneHiAgentFoldAgentDyFlowLatentMAS0000000

Figure 13:Maze navigation from local observations\. The route contains one blocked right move followed by nine moves to the goal\.
Initial state\.Fieldsf0\(movement command\)∈\\in\{move left,move right,move up,move down\}\.Action templatemove\(action = $\{f0\}\)\.Selected branchn2, selection probability 0\.349\.Selected branchn4, selection probability 0\.411\.Selected branchn8, selection probability 0\.486\.Selected branchn12, selection probability 0\.310\.Selected branchn18, selection probability 0\.422\.Selected branchn22, selection probability 0\.570\.Selected branchn26, selection probability 0\.767\.Selected branchn28, selection probability 0\.618\.Selected branchn32, selection probability 0\.405\.Selected branchn36, selection probability 0\.983\.Submitted actionsmove right,move down,move down,move down,move right,move right,move right,move down,move down,move down\.

### D\.3Grid

Room descriptions reveal adjacent doors and visible objects\. The six submitted actions include one blocked move before the treasure is reached\. The action declaration remains unchanged throughout exploration\. Direction indices 0, 1, 2, and 3 denote north, east, west, and south, respectively\.

Environment seed 104\. JevSpawn score 1\.LATSLLMCompilerAgentPruneHiAgentFoldAgentDyFlowLatentMAS0000000

Figure 14:Grid navigation through observed rooms\. Direction labels follow the selected actions, with the treasure found in the final room\.
Initial state\.Fieldsf0\(direction index\)∈\\in\{0,1,2,3\}\.Action templateexecute\(action = $\{f0\}\)\.Selected branchn3, selection probability 0\.500\.Selected branchn7, selection probability 0\.587\.Selected branchn11, selection probability 0\.529\.Selected branchn13, selection probability 0\.324\.Selected branchn19, selection probability 0\.914\.Selected branchn21, selection probability 0\.854\.Submitted actions3,3,3,1,3,1\.

### D\.4LightsOut

Row and column fields define sixteen possible button presses\. Three presses extinguish the board\. Four actions are explored at each expansion, and subsequent states are selected from the returned boards\.

Environment seed 124\. JevSpawn score 1\.LATSLLMCompilerAgentPruneHiAgentFoldAgentDyFlowLatentMAS0000000

Figure 15:LightsOut solution in three presses\. Filled circles denote lights remaining on the observed boards\. The final panel shows the environment’s success message\.
Initial state\.Fieldsf0\(row\)∈\\in\{0,1,2,3\},f1\(column\)∈\\in\{0,1,2,3\}\.Action templateexecute\(action = $\{f0\} $\{f1\}\)\.Selected branchn0, selection probability 0\.803\.Selected branchn6, selection probability 0\.891\.Selected branchn11, selection probability 0\.995\.Submitted actions0 0,3 0,3 3\.

### D\.5RushHour

Vehicle and direction fields express the available sliding actions\. Exploration returns to retained alternatives before selecting a path of nine actions that releases the red car\.

Environment seed 138\. JevSpawn score 1\.

Figure 16:RushHour states along the submitted branch\. The middle panel follows J\+, B\+, C\-, B\+\. The car marked X exits after the final forward move\.
Initial state\.Fieldsf0\(vehicle\)∈\\in\{A,B,C,D,F,H,J,X\},f1\(direction\)∈\\in\{\+,\-\}\.Action templateexecute\(action = $\{f0\}$\{f1\}\)\.Selected branchn3, selection probability 0\.666\.Selected branchn4, selection probability 0\.488\.Selected branchn1, selection probability 0\.693\.Selected branchn5, selection probability 0\.435\.Selected branchn9, selection probability 0\.395\.Selected branchn18, selection probability 0\.403\.Selected branchn20, selection probability 0\.445\.Selected branchn24, selection probability 0\.550\.Selected branchn28, selection probability 0\.770\.Selected branchn39, selection probability 0\.902\.Selected branchn40, selection probability 0\.144\.Selected branchn44, selection probability 0\.211\.Selected branchn51, selection probability 0\.990\.Submitted actionsJ\+,B\+,C\-,B\+,B\+,X\+,B\-,C\+,X\+\.

### D\.6Sokoban

A declaration over four movement commands supports both navigation and box pushing\. Exploration revisits earlier branches before selecting a solution with four moves that places both boxes on their goals\.

Environment seed 88\. JevSpawn score 1\.LATSLLMCompilerAgentPruneHiAgentFoldAgentDyFlowLatentMAS0010000

Figure 17:Sokoban solution in four moves\. Squares denote boxes, rings denote goals, and P marks the player\.
Initial state\.Fieldsf0\(movement command\)∈\\in\{\[up\],\[down\],\[left\],\[right\]\}\.Action templateexecute\(action = $\{f0\}\)\.Selected branchn1, selection probability 0\.921\.Selected branchn6, selection probability 0\.830\.Selected branchn11, selection probability 0\.460\.Selected branchn0, selection probability 0\.337\.Selected branchn8, selection probability 0\.458\.Selected branchn21, selection probability 0\.991\.Submitted actions\[left\],\[right\],\[down\],\[left\]\.

### D\.72048

The selected trajectory contains 31 moves and is submitted at the round limit with a merge score of 1076\. KORGym draws new tiles from powers of two up to half the largest tile, or inserts a two when the largest tile is below four\. Newly inserted tiles do not contribute to the merge score\.

Environment seed 141\. JevSpawn score 1076\.LATSLLMCompilerAgentPruneHiAgentFoldAgentDyFlowLatentMAS0000000

Figure 18:Selected 2048 states along the submitted trajectory\. The final score is 1076 and the largest tile is 256\.
Initial state\.Fieldsf0\(movement direction\)∈\\in\{LEFT,RIGHT,UP,DOWN\}\.Action templateexecute\(action = $\{f0\}\)\.Selected branchn3, selection probability 0\.770\.Selected branchn6, selection probability 0\.573\.Selected branchn0, selection probability 0\.513\.Selected branchn9, selection probability 0\.221\.Rounds 6 through 33 are condensed\. The action declaration is retained and 112 actions are executed across these rounds\. Several earlier branches are revisited before the final high scoring path is submitted\.Selected branchn129, selection probability 0\.364\.Selected branchn135, selection probability 0\.923\.Selected branchn139, selection probability 0\.909\.Selected branchn141, selection probability 0\.436\.Submitted actionsUP,RIGHT,LEFT,UP,UP,UP,UP,LEFT,DOWN,UP,LEFT,LEFT,LEFT,UP,UP,UP,LEFT,LEFT,UP,UP,LEFT,UP,LEFT,UP,LEFT,UP,UP,LEFT,UP,UP,LEFT\.

### D\.8Nullify

Two index fields select units for arithmetic operations, with the goal of eliminating all units\. A solution with seven actions is found after revisiting retained branches\. The initial domain omits unit 8, valued at \-7\. Reindexing after the first action brings that unit into the declared range, and the third action eliminates the unit\.

Environment seed 129\. JevSpawn score 1\.LATSLLMCompilerAgentPruneHiAgentFoldAgentDyFlowLatentMAS0000000

Figure 19:Nullify units during the selected trajectory\. The final action rounds the remaining negative value up to zero\. The terminal observation echoes the pre\-action units and reports reward one\.
Initial state\.Fieldsf0\(first unit index\)∈\\in\{0,1,2,3,4,5,6,7\},f1\(second unit index\)∈\\in\{0,1,2,3,4,5,6,7\}\.Action templateexecute\(action = $\{f0\} $\{f1\}\)\.Selected branchn3, selection probability 0\.779\.Selected branchn7, selection probability 0\.254\.Selected branchn6, selection probability 0\.208\.Selected branchn15, selection probability 0\.307\.Selected branchn8, selection probability 0\.328\.Selected branchn19, selection probability 0\.249\.Selected branchn24, selection probability 0\.360\.Selected branchn27, selection probability 0\.273\.Selected branchn31, selection probability 0\.427\.Selected branchn35, selection probability 0\.594\.Selected branchn30, selection probability 0\.426\.Selected branchn40, selection probability 0\.468\.Submitted actions6 0,5 7,5 6,4 1,3 0,1 2,0 1\.