上帝不掷骰子：在智能体内核生成中强制实现批次不变性

# God Does Not Play Dice: Enforcing Batch Invariance in Agentic Kernel Generation
Source: [https://arxiv.org/html/2610.36623](https://arxiv.org/html/2610.36623)
\\staraffiliation

1STAR Lab, George Mason University2National University of Singapore\\staremails\\starshorttitleBatch Invariance in Agentic Kernel Generation\\startrackARTIFICIAL INTELLIGENCE / MACHINE LEARNING\\starstatusPREPRINT\\starversionv1\\starkeywordsLLM Agents, GPU Kernel Optimization, Code Generation, Batch Invariance, Reproducibility\\stargmulogoassets/gmu\-logo\.png\\starpartnerNational University of Singaporeassets/nus\-logo\-color\.jpg

###### Abstract

Large language model \(LLM\) serving systems dynamically batch user requests for GPU execution, so the same request may be processed alongside different requests across runs\. If the GPU kernels lack batch invariance \(BI\), this changing batch context can alter the numerical output for an unchanged input, undermining reproducibility\. Existing kernel generation agents focus on correctness and efficiency but*overlook BI*\. We find 19 confirmed BI violations among 83 batch\-applicable released artifacts\. To address this limitation, we proposeBIEvolve, which combines contract\-guided static dependency analysis with dynamic validation of bitwise consistency across batch contexts\.BIEvolveis designed as a plug\-in component for general kernel agents, allowing their original optimization capabilities to be reused while adding BI enforcement\. We integrate it as successive acceptance gates in the optimization loop, rejecting structurally risky or dynamically non\-invariant candidates before performance benchmarking and returning feedback to the agent\. Experiments across three agent frameworks under baseline and BI\-aware generation on real world LLM serving tasks show thatBIEvolveimproves the BI pass rate from 67\.6% to 91\.9% while retaining 91\.8% of the baseline geometric mean speedup\. These results demonstrate thatBIEvolvecan be incorporated into kernel optimization by agents as an additional reliability objective alongside correctness and speed\.

11footnotetext:This work is done during the internship in[STAR Lab](https://star-computing.github.io/), George Mason University\.\*Corresponding author\.## 1Introduction

Large language model \(LLM\) agents are increasingly used to generate and optimize high performance GPU kernels\[[8](https://arxiv.org/html/2610.36623#bib.bib5);[5](https://arxiv.org/html/2610.36623#bib.bib4);[45](https://arxiv.org/html/2610.36623#bib.bib26);[4](https://arxiv.org/html/2610.36623#bib.bib27)\]\. Existing kernel agents are typically evaluated along two axes: functional correctness and execution efficiency\. A generated kernel is considered successful if it matches a reference implementation on test inputs while achieving lower latency, as exemplified by KernelBench\[[25](https://arxiv.org/html/2610.36623#bib.bib12)\]\. However, this evaluation paradigm overlooks a critical reliability requirement:whether an unchanged input produces consistent numerical results as its execution context changes\.

We focus on such overlooked reliability property,*batch invariance*\(BI\)\. For workloads in which samples are semantically independent, BI requires that an unchanged sample produce exactly the same output regardless of batch size or batch composition\[[16](https://arxiv.org/html/2610.36623#bib.bib9);[37](https://arxiv.org/html/2610.36623#bib.bib14)\]\. A BI violation occurs when changing the batch context changes that sample’s output, as illustrated in Figure[1](https://arxiv.org/html/2610.36623#S1.F1)\(a\)\. Existing generated kernels are not necessarily BI because changing the batch context can alter low\-level execution decisions, such as kernel tiling, parallel decomposition, and reduction order\. Since floating point arithmetic is non\-associative, these implementation changes can produce different results even with identical model weights and deterministic sampling settings\[[16](https://arxiv.org/html/2610.36623#bib.bib9);[13](https://arxiv.org/html/2610.36623#bib.bib8);[6](https://arxiv.org/html/2610.36623#bib.bib23)\]\. Thus, a kernel may pass conventional correctness tests while still exhibiting batch\-dependent behavior, undermining reproducibility, reliability and security\[[26](https://arxiv.org/html/2610.36623#bib.bib25);[6](https://arxiv.org/html/2610.36623#bib.bib23)\]\.

Evidence from released agent\-generated kernels confirms that the problem occurs in practice \(Table[1](https://arxiv.org/html/2610.36623#S1.T1)\)\. Among 131 released artifacts, 83 have a meaningful semantic batch dimension, and 19 of these 83 \(22\.9%\) contain confirmed BI violations\.

Table 1:BI evidence from released kernel artifacts\.BI is particularly important in modern LLM systems because batch contexts change continuously in practice\. Dynamic and continuous batching repeatedly regroup user prompt request, so the same request may be executed under different batch configurations across runs\[[19](https://arxiv.org/html/2610.36623#bib.bib10)\]\. Even small numerical deviations can propagate through autoregressive decoding and alter the final outputs\. Enforcing BI therefore improves reproducibility and predictable serving, facilitates debugging by making divergent operations easier to localize\[[51](https://arxiv.org/html/2610.36623#bib.bib21)\], and can reduce batch\-dependent rollout, training mismatch in LLM reinforcement learning\[[43](https://arxiv.org/html/2610.36623#bib.bib16);[48](https://arxiv.org/html/2610.36623#bib.bib18);[50](https://arxiv.org/html/2610.36623#bib.bib20)\]\.

Existing work has introduced BI kernels and deterministic execution modes for specific inference systems\[[16](https://arxiv.org/html/2610.36623#bib.bib9);[37](https://arxiv.org/html/2610.36623#bib.bib14)\]\. However, these methods rely on specialized, system\-specific changes and cover only a limited set of kernels, making them hard to scale to agent\-generated kernels\. For example,[Thinking Machines Lab \[35\]](https://arxiv.org/html/2610.36623#bib.bib13)reports replacing onlyfourPyTorch operators, providing limited coverage of the operator space\. Moreover, recovering performance after enforcing BI execution still requires substantial kernel engineering, motivating an optimization agent that can search for efficient invariant implementations across operators and hardware\. At the same time, replacing existing kernel agents would lose the strong optimization capabilities they already provide\. This motivates a more general approach:

Adding batch invariance as a lightweight plug\-in to existing kernel generation agents, while reusing their original optimization pipelines\.

Figure[1](https://arxiv.org/html/2610.36623#S1.F1)shows the working mechnisms ofBIEvolve, which combines a domain specific language \(DSL\)\-guided static analysis module with dynamic validation module\. The DSL specifies BI\-relevant task semantics, and the static analysis module consumes this specification to detect forbidden dependencies from declared batch symbols to protected parameters\. The dynamic validation module compares an unchanged target sample across batch contexts\. Candidates pass through DSL\-guided static analysis, reference correctness testing, and dynamic validation before performance benchmarking\. Rejected candidates receive actionable diagnostics that guide subsequent optimization attempts\.

![Refer to caption](https://arxiv.org/html/2610.36623v1/fig1.png)Figure 1:Batch dependence and theBIEvolveworkflow\. \(a\) An illustrative request can receive different outputs when its batch context changes\. \(b\) A DSL\-guided static analysis module rejects forbidden dependencies specified by contracts, while dynamic validation checks realized behavior across batch contexts\.We plugBIEvolveinto three representative LLM\-based kernel optimization agents and evaluate it, evolving existing kernel agents to generate BI kernels\. Across controlled agent reruns,BIEvolveimproves the BI pass rate from 67\.6% to 91\.9% while retaining 91\.8% of the baseline geometric mean speedup on tasks covering diverse kernel families and subtypes\. We report tolerance\-based reference correctness separately from exact BI and analyze the complementary violations detected by the static analysis module and dynamic validation module\.

Our contributions are:

- •Problem Novelty\.We identify BI as an overlooked reliability requirement in LLM\-driven GPU kernel generation and optimization\.
- •Technique Novelty\.We introduceBIEvolve, a plug\-in for general kernel generation and optimization agents, combining a DSL\-guided static analysis module with dynamic validation across semantically matched batch contexts\.
- •Empirical Evaluation\.We integrateBIEvolveinto three agent pipelines and evaluateBIEvolveon a task suite drawn from KernelBench\.com and FlashInfer Bench workloads\.

To our knowledge, this is the first work to make BI an acceptance criterion during LLM\-driven GPU kernel optimization\.

## 2Background and Problem Statement

### 2\.1GPU Kernel Execution and Nondeterminism

A GPU executes a kernel by launching a grid of thread blocks, each containing multiple threads\. To improve hardware utilization, GPU operator implementations often specialize execution to the input shape through library dispatch or shape\-dependent kernel logic\. Numerical outputs can therefore vary across executions for several distinct reasons, depending on what changes between them\. Run\-to\-run nondeterminism can arise even for the same input and execution context, originating from unordered floating point atomics, although such atomics are uncommon in modern LLM forward inference\. Numerical differences can also arise when the hardware or software environment changes\[[7](https://arxiv.org/html/2610.36623#bib.bib24);[6](https://arxiv.org/html/2610.36623#bib.bib23);[38](https://arxiv.org/html/2610.36623#bib.bib35);[23](https://arxiv.org/html/2610.36623#bib.bib36);[30](https://arxiv.org/html/2610.36623#bib.bib37);[21](https://arxiv.org/html/2610.36623#bib.bib38)\]\. These sources are distinct from batch\-dependent variation, where the target input and execution environment remain unchanged but its surrounding batch context changes\.

We focus on batch\-dependent variation because batch contexts naturally change in modern LLM serving\[[18](https://arxiv.org/html/2610.36623#bib.bib50);[28](https://arxiv.org/html/2610.36623#bib.bib49);[27](https://arxiv.org/html/2610.36623#bib.bib48)\]\. Dynamic batching and scheduling can alter the batch size, composition, and associated workload shapes seen by a kernel, even when the target request itself is unchanged\. Consequently, changing the batch context can select a different kernel variant, alter tile dimensions, or change how work is partitioned across thread blocks, including the use of split reductions\.

For reductions such as summation, matrix multiplication, and normalization, these threads compute partial results in parallel and subsequently combine them\. Since floating\-point addition is not associative, rounding can cause\(a\+b\)\+c\(a\+b\)\+cto differ froma\+\(b\+c\)a\+\(b\+c\)\. The numerical result can therefore depend on how a reduction is partitioned and in which order its partial results are accumulated\[[32](https://arxiv.org/html/2610.36623#bib.bib22);[6](https://arxiv.org/html/2610.36623#bib.bib23)\]\. Changing the batch context can consequently change the reduction order, and thus the output bits for a target sample, even though the sample and its mathematical computation remain unchanged\[[24](https://arxiv.org/html/2610.36623#bib.bib11);[16](https://arxiv.org/html/2610.36623#bib.bib9);[6](https://arxiv.org/html/2610.36623#bib.bib23)\]\.

In attention, for example, changes in workload shape can alter tiling or Split\-KV strategies and thereby change the floating point reduction order\. Prior studies therefore identify batch\-dependent changes in floating point reduction and decomposition as a central source of batch variance in LLM kernels\[[16](https://arxiv.org/html/2610.36623#bib.bib9);[3](https://arxiv.org/html/2610.36623#bib.bib3)\]\. In Figure[2](https://arxiv.org/html/2610.36623#S2.F2), we provide a concrete code example identified in our study of a batch dependent split/reduction:batch\_sizechangesbase\_blocksandavg\_pages, which determinenum\_splitsand thus the page ranges reduced by each split\.

![Refer to caption](https://arxiv.org/html/2610.36623v1/fig2.png)Figure 2:Representative kernel code patterns that introduce batch variance\.
### 2\.2Batch Invariance

Unless otherwise stated, all executions considered below use the same hardware and software enviroment\. LetK⁡\(B\)K\(B\)denote the output of a candidate kernelKKon a batchBB, and let𝒳\\mathcal\{X\}denote the input domain\. For each target samplex∈𝒳x\\in\\mathcal\{X\}, let𝒞⁡\(x\)\\mathcal\{C\}\(x\)be the set of admissible batch contexts that preservexxand its reference semantics while allowing the batch size, composition, and position ofxxto vary\. A contextc∈𝒞⁡\(x\)c\\in\\mathcal\{C\}\(x\)produces a batchBcB\_\{c\}in whichxxappears at positionici\_\{c\}\. We say thatKKis batch invariant over𝒳\\mathcal\{X\}if∀x∈𝒳\\forall x\\in\\mathcal\{X\}and∀c,c′∈𝒞⁡\(x\)\\forall c,c^\{\\prime\}\\in\\mathcal\{C\}\(x\),K\(Bc\)ic≡bitK\(Bc′\)ic′K\(B\_\{c\}\)\_\{i\_\{c\}\}\\equiv\_\{\\mathrm\{bit\}\}K\(B\_\{c^\{\\prime\}\}\)\_\{i\_\{c^\{\\prime\}\}\}\.

Thus, the output associated with a preserved target sample must be bitwise identical across all admissible batch contexts\. This definition treats batch invariance as an umbrella property encompassing invariance to batch size, batch composition, and batch position; restricting𝒞⁡\(x\)\\mathcal\{C\}\(x\)to changes along one dimension yields the corresponding narrower property\.

#### Problem definition\.

Given a kernel task, its reference implementation, and a set of admissible batch contexts, our goal is to generate an optimized kernel that \(i\) satisfies the task’s numerical correctness criterion, \(ii\) is BI across admissible contexts under a fixed hardware and software environment, and \(iii\) achieves high kernel execution performance\.

## 3Methodology

BIEvolveis designed as a plug\-in workflow for existing kernel agents\. As illustrated in Figure[1](https://arxiv.org/html/2610.36623#S1.F1), it combines a DSL\-guided static analysis module with dynamic validation module, while retaining each framework’s native correctness test and performance benchmark\. Figure[3](https://arxiv.org/html/2610.36623#S3.F3)provides a detailed view ofBIEvolve\. Both modules are driven by domain knowledge, the design rules could also be automatically extracted by an agent from representative examples\. The static module uses contracts that specify forbidden batch dependencies into protected parameters; the dynamic module tests whether a preserved sample produces bitwise identical outputs across valid batch contexts\. When a candidate fails either module,BIEvolvereturns actionable diagnostics through the agent’s existing feedback channel to guide the next optimization round\.

![Refer to caption](https://arxiv.org/html/2610.36623v1/Fig3.png)Figure 3:BIEvolvecandidate evaluation workflow\. BI failures produce actionable feedback, while accepted candidates proceed to speedup measurement\.### 3\.1DSL\-Guided Static Analysis Module

#### BI contract DSL\.

The static analysis module detects source level dependencies that are known to threaten BI before executing a generated kernel\. It takes two inputs: task metadata describing the task semantics and the generated source bundle\. The metadata is used to select the applicable BI contract, while the source code is parsed into a representation suitable for dependency analysis\. The analyzer then checks whether any protected kernel parameter depends on the batch context\. The module outputs either a pass decision or an actionable diagnosis identifying the violating dependency\.

The contract DSL specifies two elements: batch context variables, including batch size and batch\-derived workload quantities, and protected parameters, whose values must remain independent of the batch context variables\. Contracts are organized hierarchically as global, kernel family, and subtype rules; task metadata resolves the applicable contracts, allowing a new task to inherit protections from its semantic family\. Aliases map implementation specific variable names to canonical protected parameters\. The simplified DSL grammar is shown below in Figure[4](https://arxiv.org/html/2610.36623#S3.F4.fig1)\.

⟨contract⟩\\langle\\textit\{contract\}\\rangle:⁣:=::="kernel"⟨id⟩\\langle\\textit\{id\}\\rangle⟨inherit⟩\\langle\\textit\{inherit\}\\rangle"\{"⟨axis⟩\\langle\\textit\{axis\}\\rangle⟨params⟩\\langle\\textit\{params\}\\rangle"\}"⟨inherit⟩\\langle\\textit\{inherit\}\\rangle:⁣:=::=ε\\varepsilon∣\\mid"extends"⟨names⟩\\langle\\textit\{names\}\\rangle⟨axis⟩\\langle\\textit\{axis\}\\rangle:⁣:=::="invariant\_axis""batch" "symbols""\["⟨names⟩\\langle\\textit\{names\}\\rangle"\]" ";"⟨params⟩\\langle\\textit\{params\}\\rangle:⁣:=::=ε\\varepsilon∣\\mid⟨parameter⟩\\langle\\textit\{parameter\}\\rangle⟨params⟩\\langle\\textit\{params\}\\rangle⟨parameter⟩\\langle\\textit\{parameter\}\\rangle:⁣:=::="parameter"⟨id⟩\\langle\\textit\{id\}\\rangle"\{"⟨fields⟩\\langle\\textit\{fields\}\\rangle"\}"⟨fields⟩\\langle\\textit\{fields\}\\rangle:⁣:=::=⟨field⟩\\langle\\textit\{field\}\\rangle∣\\mid⟨field⟩\\langle\\textit\{field\}\\rangle⟨fields⟩\\langle\\textit\{fields\}\\rangle⟨field⟩\\langle\\textit\{field\}\\rangle:⁣:=::=⟨semantic⟩\\langle\\textit\{semantic\}\\rangle∣\\mid⟨policy⟩\\langle\\textit\{policy\}\\rangle∣\\mid⟨aliases⟩\\langle\\textit\{aliases\}\\rangle⟨semantic⟩\\langle\\textit\{semantic\}\\rangle:⁣:=::="semantic"⟨sem⟩\\langle\\textit\{sem\}\\rangle";"⟨sem⟩\\langle\\textit\{sem\}\\rangle:⁣:=::="split\_structure"∣\\mid"recurrence\_structure"∣\\mid"rng\_stream"⟨policy⟩\\langle\\textit\{policy\}\\rangle:⁣:=::="policy"⟨policy\_kind⟩\\langle\\textit\{policy\\\_kind\}\\rangle";"⟨policy\_kind⟩\\langle\\textit\{policy\\\_kind\}\\rangle:⁣:=::="fixed\_across\_batch"∣\\mid"may\_depend\_on" "batch"∣\\mid"free"⟨aliases⟩\\langle\\textit\{aliases\}\\rangle:⁣:=::="aliases" "\["⟨names⟩\\langle\\textit\{names\}\\rangle"\]" ";"⟨names⟩\\langle\\textit\{names\}\\rangle:⁣:=::=⟨id⟩\\langle\\textit\{id\}\\rangle∣\\mid⟨id⟩\\langle\\textit\{id\}\\rangle","⟨names⟩\\langle\\textit\{names\}\\rangle⟨id⟩\\langle\\textit\{id\}\\rangle:⁣:=::=\[A\-Za\-z\_\]\[A\-Za\-z0\-9\_\]\*
*Semantic constraints: inheritance is resolved before analysis; aliases denote the same parameter; policies respectively forbid batch dependence, permit dependence on batch, or leave dependence unrestricted\.*

Figure 4:Simplified Backus\-Naur Form \(BNF\) grammar of the BI contract DSL\.The current DSL protects key mechanisms inclduing: split/reduction structure \(NUM\_SPLITS\), recurrence decomposition \(RECURRENCE\_SEGMENTS\), and random stream state \(RNG\_SEED\)\.

#### Source parsing\.

Given task metadata, the contract parser resolves the applicable global, family, and subtype contracts and exposes their declared batch sources and protected parameters to the analyzer\. In parallel, the source parser processes the complete generated source bundle\. Python and Triton code are parsed using abstract syntax tree \(AST\)\-based analysis that tracks dependencies through argument passing, container accesses, control flow, and a restricted set of same file helper functions\. C/C\+\+/CUDA code is handled with targeted dependency patterns, and embedded generated native code is included in the same analysis\. The resulting representation provides the dependency information required by the subsequent analyzer\.

#### Contract\-guided dependency analysis\.

The analyzer treats contract\-declared batch symbols and their derived global quantities as dependency sources, and protected parameters as sinks\. It propagates both data and control dependence through the parsed program\. A candidate is rejected if a protected parameter is assigned directly or transitively from batch context, or if its value is selected through a batch dependent conditional or loop\. Thus, the contract determines what must remain batch independent, while the analyzer determines whether the generated implementation satisfies that requirement\. Dependencies that cannot be reliably resolved statically, such as opaque runtime dispatch or autotuning behavior, are left to the dynamic validation module\.

#### Diagnostic feedback\.

For a rejected candidate, the analyzer converts the detected dependency into compact feedback for the kernel agent\. It reports the protected mechanism, source code location, and dependency path from the batch source to the protected parameter, together with mechanism specific repair guidance\.

### 3\.2Dynamic Validation Module

#### Test policy selection\.

The dynamic validation module tests whether an executable candidate produces the same output for an unchanged sample under different batch contexts\. Its input is a reference\-correct candidate together with task metadata, and its output is either a pass decision or a diagnostic describing the first detected cross batch mismatch\. Task metadata selects a predefined batch test policy according to the kernel family or subtype\. General kernels use less batch contexts, while higher risk families such as attention use additional contexts to exercise more execution boundaries\.

#### Cross batch comparison\.

For each selected batch context, the test harness constructs a valid workload around the same target sample\. The target sample is first executed on the baseline batch context, and the target output is recorded\. It is then executed under each larger batch context, after which the corresponding target output is extracted and compared bitwise with the baseline\. BI requires identical structure, shape, dtype, and numerical values\. Testing stops at the first mismatch\.

#### Feedback\.

If all tested contexts produce identical target outputs, the candidate passes dynamic validation\. Otherwise, dynamic module reports the failing batch context through the agent’s existing feedback path\.

### 3\.3Plug into the Agent Loop

#### Candidate evaluation pipeline\.

BIEvolveis integrated into the existing candidate evaluation pipeline rather than modifying the agent’s generation or search algorithm\. A lightweight framework adapter packages the information required by the static and dynamic modules, inserts both modules into the candidate evaluation pipeline, and converts BI failures into the framework’s original feedback format\. At each optimization round, a generated candidate passes through four sequential stages: \(1\) static BI analysis, \(2\) reference correctness test, \(3\) dynamic BI validation, and \(4\) performance benchmarking\. The static analysis first removes candidates with recognized BI violation patterns\. The framework’s native correctness test then filters incorrect implementations\. Dynamic validation checks the remaining candidates across batch contexts, and performance is measured only after both correctness andBIEvolveare passed\. This ordering prevents unnecessary execution of statically invalid candidates and avoids interpreting correctness failures as BI violations\.

#### Framework adapter\.

For each agent framework, the adapter supplies the information needed for the BI modules: the generated source and task metadata for static analysis, the test harness and task metadata for dynamic validation\. The framework’s original correctness criterion, benchmark procedure, prompt, candidate selection policy, and search state remain unchanged\. Thus, theBIEvolvecan be attached to agents with different optimization strategies\.

In addition, the adapter routes the outputs of the BI modules\. If a candidate passes, the evaluation pipeline proceeds\. If either BI module rejects the candidate, subsequent evaluation stages are skipped, and the diagnostic is recorded through the framework’s existing evaluation history or log and inserted into its feedback path\. This allows subsequent optimization rounds to condition on previous BI failures and revise the implementation accordingly\.

## 4Experiments

In this section, we evaluate whetherBIEvolvereduces BI violations in agent\-generated kernels while preserving ordinary correctness and useful performance\. The evaluation combines controlled kernel generation with three agent frameworks, subgroup performance analysis, and an ablation of theBIEvolvecomponents\.

### 4\.1Experimental Setup

#### Datasets\.

Our controlled kernel generation experiment uses a compact, risk\-focused suite drawn from the hard tasks on KernelBench\.com[Arledge \[1\]](https://arxiv.org/html/2610.36623#bib.bib1)and the FlashInfer trace and contest workloads described by[Xing et al\. \[42\]](https://arxiv.org/html/2610.36623#bib.bib15)and[FlashInfer Team \[11\]](https://arxiv.org/html/2610.36623#bib.bib7)\. We select batch applicable kernels spanning attention, linear attention, mixture\-of\-experts \(MoE\), normalization, and sampling\. Attention covers GQA, MLA, and DSA across representative prefill and decode settings, while linear attention includes KDA and Gated DeltaNet variants\.

We emphasize attention and linear attention because their execution commonly involves batch dependent partitioning, algorithm selection, and reductions, while the remaining families broaden coverage beyond these higher risk regimes\.

#### Agent frameworks\.

We evaluate K\-Search from[Cao et al\. \[5\]](https://arxiv.org/html/2610.36623#bib.bib4)and the Iterative Agent and Evolve Agent from[FlashInfer Team \[12\]](https://arxiv.org/html/2610.36623#bib.bib6)to cover three representative search paradigms in feedback\-driven kernel generation\. The Iterative Agent refines a current implementation along a single trajectory; the Evolve Agent explores multiple implementations through recent and elite candidate pools; and K\-Search guides implementation generation using a co\-evolving world model that separates planning from code instantiation\.

These frameworks differ in how they retain search history, select candidates for improvement, and explore alternative designs\. Together, they provide broad architectural coverage for evaluating whetherBIEvolvegeneralizes beyond a particular refinement policy or search state representation\.

For each framework, we compare vanilla generation with a rerun usingBIEvolveas described in Section[3](https://arxiv.org/html/2610.36623#S3)\. We use 20 optimization turns for K\-Search and 25 turns for the Iterative Agent and Evolve Agent, keeping the turn budget unchanged between vanilla and BI\-aware generation within each framework\. Controlled generation uses Gemini 3\.7 Flash for all three frameworks on an NVIDIA H100 NVL GPU\.

#### Evaluation\.

We evaluate the selected kernels using the same protocol across generation configurations\. Ordinary correctness compares each kernel with its task reference using the prescribed numerical tolerance\. We report these outcomes separately, since tolerance\-based agreement with a reference does not imply BI\.

Performance is measured as speedup relative to the corresponding task reference and summarized using geometric means\. Performance retention is the ratio of the BI\-aware and vanilla geometric mean speedups over the same tasks\.

### 4\.2Main Results

#### Controlled kernel generation\.

We report the final\-solution BI pass rate, defined as the fraction of correctness\-passing kernels with no confirmed BI violation, and the BI intervention rate, defined as the fraction of tasks for whichBIEvolverejects at least one generated candidate during optimization\. Correctness is evaluated using the task\-specific reference tolerance\. Table[2](https://arxiv.org/html/2610.36623#S4.T2)compares baseline and BI\-aware generation under these metrics\. GM denotes geometric mean\. Speedup retained is reported once per framework for the BI\-aware/baseline comparison\. Retained speedup is100×GMBI​\-​aware/GMbaseline100\\times\\mathrm\{GM\}\_\{\\mathrm\{BI\\text\{\-\}aware\}\}/\\mathrm\{GM\}\_\{\\mathrm\{baseline\}\}, using the same tasks and reference implementations\.

In the K\-Search study,BIEvolveincreases the BI pass rate from 75\.7% to 97\.4% under the final solution evaluation, while ordinary correctness changes from 94\.9% to 97\.4%\. For the Iterative Agent and Evolve Agent, BI pass rates change from 48\.6% to 89\.2% and from 78\.4% to 89\.2%, respectively, while ordinary correctness remains 94\.9% in both conditions for both frameworks\.

The BI intervention rate is highest for the Iterative Agent at 43\.6%, compared with 23\.1% for K\-Search and 25\.6% for Evolve Agent, indicating that the framework with the weakest baseline BI pass rate required intervention on substantially more tasks and also exhibited the largest final BI pass rate improvement\. Notably, intervention rates may exceed the corresponding final BI pass gains, indicating thatBIEvolvealso filters BI violating intermediate candidates on tasks whose baseline final solution is BI\.

Table 2:Controlled kernel generation results across agent frameworks\.
#### Performance retention\.

For K\-Search, the BI\-aware rerun achieves a geometric mean speedup of 35\.39×\\times, compared with 40\.94×\\timesfor vanilla generation, retaining 86\.4% of the vanilla speedup\. The corresponding retention for the Iterative Agent and Evolve Agent is 87\.7% and 102\.0%\.

Figure[5](https://arxiv.org/html/2610.36623#S4.F5)compares performance retention across baseline BI outcomes\. We partition tasks according to whether their vanilla kernels have a confirmed BI violation\.

Figure 5:Speedup retained by baseline BI outcome across agent frameworks\. Subgroups are grouped by the baseline final kernel’s BI outcome\.Aggregate retention is lower for tasks with confirmed violations than for tasks without violations \(86\.0% versus 95\.7%\), consistent with BI gating excluding fast but batch\-dependent optimization choices\[[16](https://arxiv.org/html/2610.36623#bib.bib9)\]\. The Evolve Agent retains nearly all performance in both subgroups, suggesting that broader search can recover invariant alternatives and that the BI cost depends partly on search strategy\.

### 4\.3BIEvolveComponent Ablation

To isolate the contribution of eachBIEvolvecomponent from changes in the agent search trajectory, Table[3](https://arxiv.org/html/2610.36623#S4.T3)compares static\-analysis\-only, dynamic\-validation\-only, and fullBIEvolveconfigurations offline on the same balanced, independently labeled candidate pool\. The fullBIEvolvemeasurement preserves sequential early exit\.

Table 3:BIEvolvecomponent ablation on a shared candidate pool \(N=200N=200\)\.ComponentBI violationdetection \(%\)False rejection\(%\)Median evaluationtime \(ms/candidate\)OverallUniqueOverlapStatic module83374627\.69Dynamic module5594602\.24BIEvolve92––210\.21BIEvolvedetects 92% of confirmed violations, compared with 83% for the static analysis module and 55% for the dynamic validation module\. It rejects 2% of candidates with no confirmed BI violation\. Median evaluation time is 10\.21 ms per candidate for fullBIEvolve, versus 7\.69 ms and 2\.24 ms for the static analysis module and dynamic validation module, respectively\. The higherBIEvolvedetection rate supports the intended complementarity: static analysis covers modeled dependency patterns, while dynamic validation can expose realized behavior outside those rules\.

## 5Related Work

In this section, we review LLM\-based GPU kernel generation and complementary approaches to deterministic inference\.

#### Kernel generation and optimization\.

Existing work\[[9](https://arxiv.org/html/2610.36623#bib.bib28);[17](https://arxiv.org/html/2610.36623#bib.bib29);[10](https://arxiv.org/html/2610.36623#bib.bib30);[22](https://arxiv.org/html/2610.36623#bib.bib31);[44](https://arxiv.org/html/2610.36623#bib.bib32);[15](https://arxiv.org/html/2610.36623#bib.bib33);[14](https://arxiv.org/html/2610.36623#bib.bib34);[41](https://arxiv.org/html/2610.36623#bib.bib39);[33](https://arxiv.org/html/2610.36623#bib.bib40);[31](https://arxiv.org/html/2610.36623#bib.bib41);[40](https://arxiv.org/html/2610.36623#bib.bib42);[29](https://arxiv.org/html/2610.36623#bib.bib43);[47](https://arxiv.org/html/2610.36623#bib.bib44);[39](https://arxiv.org/html/2610.36623#bib.bib45)\]combines kernel evaluation with feedback\-driven generation, learned refinement, or structured search\. KernelBench, introduced by[Ouyang et al\. \[25\]](https://arxiv.org/html/2610.36623#bib.bib12), formulates kernel generation as replacing a PyTorch computation with a functionally equivalent, faster implementation, and studies repeated sampling and refinement using execution and profiling feedback\. FlashInfer\-Bench, proposed by[Xing et al\. \[42\]](https://arxiv.org/html/2610.36623#bib.bib15), extends evaluation to production LLM inference workloads through a unified representation of kernel tasks, implementations, and measurements\. Its framework integrates correctness validation, performance benchmarking, and deployment\. These are evaluation frameworks rather than individual agent architectures\. Representative agent designs include Kevin by[Baronio et al\. \[2\]](https://arxiv.org/html/2610.36623#bib.bib2), which trains multi\-turn refinement through reinforcement learning with correctness and speedup rewards, and K\-Search by[Cao et al\. \[5\]](https://arxiv.org/html/2610.36623#bib.bib4), which separates high\-level optimization planning from code implementation\. K\-Search updates its search tree and optimization priorities through a co\-evolving intrinsic world model informed by execution feedback\. Despite their different designs, these approaches primarily emphasize correctness, performance, or broader evaluation robustness\. Their validation criteria do not explicitly protect BI during optimization: numerical agreement with a reference does not establish bitwise consistency across batch configurations\[[34](https://arxiv.org/html/2610.36623#bib.bib51);[52](https://arxiv.org/html/2610.36623#bib.bib46);[20](https://arxiv.org/html/2610.36623#bib.bib52);[36](https://arxiv.org/html/2610.36623#bib.bib47)\]\. In this work, we focus on explicit BI protection inside the kernel optimization loop\.

#### BI kernels and deterministic inference\.

[He and Thinking Machines Lab \[16\]](https://arxiv.org/html/2610.36623#bib.bib9)and[vLLM Project \[37\]](https://arxiv.org/html/2610.36623#bib.bib14)provide specialized BI kernels or serving modes for supported implementations\. Their designs control batch\-dependent tiling and reduction schedules, requiring careful engineering to maintain performance tuning\. In addition, they do not provide BI validation for arbitrary agent\-generated candidates\.[Yuan et al\. \[46\]](https://arxiv.org/html/2610.36623#bib.bib17)systematically links changes in evaluation batch size and hardware configuration to numerical differences and divergent generated responses\.

There are alternative serving designs reduce reliance on BI kernels\. LLM\-42, introduced by[Gond et al\. \[13\]](https://arxiv.org/html/2610.36623#bib.bib8), generates candidate tokens through ordinary decoding, then replays them under a fixed\-shape verification schedule and rolls back mismatches\. CoRun, introduced by[Zhao et al\. \[49\]](https://arxiv.org/html/2610.36623#bib.bib19), instead controls execution shapes through scheduling and padding: it processes each prompt separately and pads decoding batches to a fixed size, while restricting attention reductions that would otherwise vary with batching\. These methods illustrate complementary ways to obtain deterministic inference, whereas our contribution concerns protecting BI as agents optimize kernel implementations\.

## 6Conclusion and Future Work

BI should be an explicit evaluation dimension for generated GPU kernels when task semantics treat samples independently\. Our analysis of released artifacts shows that conventional correctness and speed evaluation can miss batch\-dependent outputs\. We introduceBIEvolve, which plugs into kernel generation frameworks, making BI actionable within existing generation and optimization workflows\. On the configured test domain,BIEvolveremoves most observed violations while preserving conventional correctness and speedup\. Together, these findings support evaluating BI alongside correctness and performance\.

Several limitations motivate future work\. The static analysis module currently provides limited cross\-file analysis\. Extending this analysis would improve coverage of dependencies spanning implementation boundaries\. Furthermore, agent\-based extraction and learning from representative examples provide paths for automatic design rule synthesis and refinement\. Finally, our experiments measure batch dependence at the kernel output level, future work should assess its end\-to\-end effects on token generation, evaluation, serving, and reinforcement learning\.

## References

- \[1\]E\. Arledge\(2026\)KernelBench\.com: agentic GPU kernel benchmark results\.Note:Benchmark websiteAccessed 2026\-09\-24External Links:[Link](https://kernelbench.com/)Cited by:[§4\.1](https://arxiv.org/html/2610.36623#S4.SS1.SSS0.Px1.p1.1)\.
- \[2\]C\. Baronio, P\. Marsella, B\. Pan, S\. Guo, and S\. Alberti\(2026\)Kevin: multi\-turn RL for generating CUDA kernels\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2026/hash/86ba836d4c5dd859d795a172911745e2-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[3\]N\. Cankaya\(2026\)Bit\-exact AI inference verification without performance tradeoffs\.InICML 2026 Workshop on Technical AI Governance Research,External Links:[Link](https://openreview.net/forum?id=ECU7tSBQY3)Cited by:[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p4.1)\.
- \[4\]S\. Cao, Z\. Mao, J\. E\. Gonzalez, and I\. Stoica\(2026\)K\-search: llm kernel generation via co\-evolving intrinsic world model\.arXiv preprint arXiv:2602\.19128\.Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p1.1)\.
- \[5\]S\. Cao, Z\. Mao, J\. E\. Gonzalez, and I\. Stoica\(2026\)K\-Search: LLM kernel generation via co\-evolving intrinsic world model\.arXiv preprint arXiv:2602\.19128\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2602.19128),[Link](https://arxiv.org/abs/2602.19128)Cited by:[§C\.2](https://arxiv.org/html/2610.36623#A3.SS2.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.36623#S1.p1.1),[§4\.1](https://arxiv.org/html/2610.36623#S4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[6\]S\. Chen, J\. Peng, Y\. He, J\. Yang, and B\. Ray\(2026\)Your compiler is backdooring your model: understanding and exploiting compilation inconsistency vulnerabilities in deep learning compilers\.In2026 IEEE Symposium on Security and Privacy \(SP\),pp\. 233–251\.External Links:[Document](https://dx.doi.org/10.1109/SP63933.2026.00097),[Link](https://doi.org/10.1109/SP63933.2026.00097)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p3.1)\.
- \[7\]S\. Chen, S\. Wei, C\. Liu, and W\. Yang\(2023\)DyCL: dynamic neural network compilation via program rewriting and graph optimization\.InProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis,pp\. 614–626\.External Links:[Document](https://dx.doi.org/10.1145/3597926.3598082),[Link](https://doi.org/10.1145/3597926.3598082)Cited by:[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p1.1)\.
- \[8\]J\. Dong, Y\. Yang, T\. Liu, Y\. Wang, F\. Qi, V\. Tarokh, K\. Rangadurai, and S\. Yang\(2026\)STARK: strategic team of agents for refining kernels\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2026/hash/229608737599da1f292d4cf5a18c99f0-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p1.1)\.
- \[9\]H\. Du, Q\. Ge, J\. Hu, A\. Yang, Z\. Cai, Z\. Huang, S\. Yuan, Q\. Cheng, X\. Xie, Y\. Chen,et al\.\(2026\)Kernel\-smith: a unified recipe for evolutionary kernel optimization\.arXiv preprint arXiv:2603\.28342\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[10\]Q\. Fan, A\. Zou, and Y\. Ma\(2026\)CUDA\-harness: harnessing agentic cuda kernel generation and optimization from natural language\.arXiv preprint arXiv:2609\.00058\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[11\]FlashInfer Team\(2026\)MLSys 2026 FlashInfer AI Kernel Generation Contest\.Note:Contest websiteAccessed 2026\-09\-24External Links:[Link](https://mlsys26.flashinfer.ai/)Cited by:[§4\.1](https://arxiv.org/html/2610.36623#S4.SS1.SSS0.Px1.p1.1)\.
- \[12\]FlashInfer Team\(2026\)MLSys 2026 FlashInfer AI Kernel Generation Contest: agent baseline\.Note:GitHub repositoryAccessed 2026\-09\-24External Links:[Link](https://github.com/flashinfer-ai/mlsys26-agent-baseline)Cited by:[§C\.2](https://arxiv.org/html/2610.36623#A3.SS2.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.36623#S4.SS1.SSS0.Px2.p1.1)\.
- \[13\]R\. Gond, A\. K\. Kamath, R\. Ramjee, and A\. Panwar\(2026\)LLM\-42: enabling determinism in LLM inference with verified speculation\.InProceedings of the ACM SIGOPS 32nd Symposium on Operating Systems Principles,pp\. 916–931\.External Links:[Document](https://dx.doi.org/10.1145/3830418.3843901),[Link](https://doi.org/10.1145/3830418.3843901)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p2.1),[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px2.p2.1)\.
- \[14\]J\. Guo, S\. Lu, T\. Ma, W\. Gao, Z\. Ye, X\. Zhang, and D\. Lin\(2026\)SpecGen: accelerating agentic kernel optimization with speculative generation\.arXiv preprint arXiv:2606\.17518\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[15\]S\. K\. S\. Hari, V\. Balaji, S\. Damani, Q\. Huang, and C\. Kozyrakis\(2026\)Improving efficiency of gpu kernel optimization agents using a domain\-specific language and speed\-of\-light guidance\.arXiv preprint arXiv:2603\.29010\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[16\]H\. He and Thinking Machines Lab\(2025\)Defeating nondeterminism in LLM inference\.Note:Thinking Machines Lab, ConnectionismExternal Links:[Document](https://dx.doi.org/10.64434/tml.20250910),[Link](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/)Cited by:[Appendix A](https://arxiv.org/html/2610.36623#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.36623#S1.p2.1),[§1](https://arxiv.org/html/2610.36623#S1.p5.1),[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p3.1),[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p4.1),[§4\.2](https://arxiv.org/html/2610.36623#S4.SS2.SSS0.Px2.p3.1),[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px2.p1.1)\.
- \[17\]C\. Ke, R\. Zhang, Z\. Fang, Z\. Li, Y\. Wen, J\. Shen, S\. Wang, J\. Guo, L\. Li, Q\. Guo,et al\.\(2026\)KernelZero: co\-evolving proposer and coder for continuously improved gpu kernel generation\.arXiv preprint arXiv:2609\.33074\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[18\]W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica\(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles,Cited by:[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p2.1)\.
- \[19\]W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica\(2023\)Efficient memory management for large language model serving with PagedAttention\.InProceedings of the 29th Symposium on Operating Systems Principles,pp\. 611–626\.External Links:[Document](https://dx.doi.org/10.1145/3600006.3613165),[Link](https://doi.org/10.1145/3600006.3613165)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p4.1)\.
- \[20\]T\. Li, Z\. Chen, and Y\. Tang\(2026\)LEAP: lean environment\-feedback via adaptive pruning for code rl in gpu kernel generation\.arXiv preprint arXiv:2608\.01804\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[21\]N\. Loose, J\. Sander, F\. Mächtle, and T\. Eisenbarth\(2026\)FloatDoor: platform\-triggered backdoors in llms\.arXiv preprint arXiv:2606\.19535\.Cited by:[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p1.1)\.
- \[22\]H\. Mai, X\. Guo, X\. Ding, D\. Li, Q\. Yu, C\. Guo, C\. Wang, J\. Zhao, C\. Kozyrakis, and B\. Yuan\(2026\)Ave: guiding agentic gpu optimization using data\-flow invariants\.InProceedings of the ACM SIGOPS 32nd Symposium on Operating Systems Principles,pp\. 1055–1069\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[23\]J\. Möller, E\. Imgrund, T\. Eisenhofer, and K\. Rieck\(2026\)Hardware\-triggered backdoors\.arXiv preprint arXiv:2601\.21902\.Cited by:[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p1.1)\.
- \[24\]NVIDIA\(2025\)Floating point and IEEE 754\.Note:CUDA Toolkit documentation, version 13\.0\.3Accessed 2026\-09\-24External Links:[Link](https://docs.nvidia.com/cuda/archive/13.0.3/floating-point/index.html)Cited by:[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p3.1)\.
- \[25\]A\. Ouyang, S\. Guo, S\. Arora, A\. L\. Zhang, W\. Hu, C\. Ré, and A\. Mirhoseini\(2025\)KernelBench: can LLMs write efficient GPU kernels?\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 47356–47415\.External Links:[Link](https://proceedings.mlr.press/v267/ouyang25a.html)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p1.1),[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[26\]S\. Ouyang, J\. M\. Zhang, M\. Harman, and M\. Wang\(2025\)An empirical study of the non\-determinism of ChatGPT in code generation\.ACM Transactions on Software Engineering and Methodology34\(2\),pp\. 42:1–42:28\.External Links:[Document](https://dx.doi.org/10.1145/3697010),[Link](https://doi.org/10.1145/3697010)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p2.1)\.
- \[27\]W\. Pang, X\. Luo, K\. Chen, D\. Ji, L\. Qiao, and W\. Yi\(2023\)Efficient cuda stream management for multi\-dnn real\-time inference on embedded gpus\.Journal of Systems Architecture139,pp\. 102888\.Cited by:[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p2.1)\.
- \[28\]P\. Pochelu, S\. G\. Petiton, and B\. Conche\(2021\)An efficient and flexible inference system for serving heterogeneous ensembles of deep neural networks\.In2021 IEEE International Conference on Big Data \(Big Data\),pp\. 5225–5232\.Cited by:[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p2.1)\.
- \[29\]Q\. Qu, Y\. Sui, Y\. Sun, R\. Chen, X\. Zhang, Y\. Zhang, H\. Wang, and G\. Lan\(2026\)A two\-stage gpu kernel tuner combining semantic refactoring and search\-based optimization\.arXiv preprint arXiv:2601\.12698\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[30\]H\. Rahaman, Q\. Xu, Z\. Haider, P\. Chakraborty, S\. Bhunia, and F\. Suya\(2026\)\(A\) ispy: parasitic trojans for machine learning infrastructure\.arXiv preprint arXiv:2607\.17550\.Cited by:[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p1.1)\.
- \[31\]D\. Ran, S\. Xie, M\. Ji, Z\. Hua, M\. Wu, Y\. Cao, Y\. Guo, Y\. Hao, L\. Li, Y\. Hu,et al\.\(2025\)Kernelband: boosting llm\-based kernel optimization with a hierarchical and hardware\-aware multi\-armed bandit\.arXiv e\-prints,pp\. arXiv–2511\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[32\]R\. Rathnasuriya, N\. Majoju, Z\. Song, and W\. Yang\(2025\)An investigation on numerical bugs in GPU programs towards automated bug detection\.Proceedings of the ACM on Software Engineering2\(ISSTA\),pp\. 1654–1677\.External Links:[Document](https://dx.doi.org/10.1145/3728950),[Link](https://doi.org/10.1145/3728950)Cited by:[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p3.1)\.
- \[33\]T\. Sereda, B\. Bartan, A\. Nayak, T\. S\. John, N\. Serrino, and Z\. Asgar\(2026\)KForge: llm\-driven cross\-platform kernel generation for ai accelerators\.arXiv preprint arXiv:2606\.02963\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[34\]Q\. Sun, J\. Han, T\. Li, Z\. Tang, S\. Chen, F\. Yang, A\. Liu, X\. Liu, and Y\. Liu\(2026\)Kernelskill: a multi\-agent framework for gpu kernel optimization\.arXiv preprint arXiv:2603\.10085\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[35\]Thinking Machines Lab\(2025\)Batch invariant ops\.Note:GitHub repositoryAccessed 2026\-09\-24External Links:[Link](https://github.com/thinking-machines-lab/batch_invariant_ops)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p5.1)\.
- \[36\]A\. Tschand, M\. Awad, R\. Swann, K\. Ramakrishnan, J\. Ma, K\. Lowery, G\. Dasika, and V\. J\. Reddi\(2025\)Swizzleperf: hardware\-aware llms for gpu kernel performance optimization\.arXiv preprint arXiv:2508\.20258\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[37\]vLLM Project\(2026\)Batch invariance\.Note:vLLM documentationAccessed 2026\-09\-24External Links:[Link](https://docs.vllm.ai/en/latest/features/batch_invariance/)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p2.1),[§1](https://arxiv.org/html/2610.36623#S1.p5.1),[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px2.p1.1)\.
- \[38\]Y\. Wang, T\. Li, X\. Zhang, X\. Zhang, W\. Ma, M\. Cheng, and L\. Pan\(2026\)Hidden reliability risks in large language models: systematic identification of precision\-induced output disagreements\.arXiv preprint arXiv:2604\.19790\.Cited by:[§2\.1](https://arxiv.org/html/2610.36623#S2.SS1.p1.1)\.
- \[39\]A\. Wei, T\. Sun, Y\. Seenichamy, H\. Song, A\. Ouyang, A\. Mirhoseini, K\. Wang, and A\. Aiken\(2025\)Astra: a multi\-agent system for gpu kernel performance optimization\.arXiv preprint arXiv:2509\.07506\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[40\]N\. Wiedemann, Q\. Leboutet, M\. Paulitsch, D\. Wofk, and B\. Ummenhofer\(2026\)KernelFoundry: hardware\-aware evolutionary gpu kernel optimization\.arXiv preprint arXiv:2603\.12440\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[41\]X\. Xiao, Z\. Song, D\. Ran, W\. Yang, and T\. Xie\(2026\)TileCas: cascading kernel development across programming abstractions\.arXiv preprint arXiv:2610\.14740\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[42\]S\. Xing, Y\. Zhai, A\. Jiang, Y\. Dong, Y\. Wu, Z\. Ye, C\. F\. Ruan, Y\. Huang, Y\. Zhang, L\. Yin, A\. Bayyapu, L\. Ceze, and T\. Chen\(2026\)FlashInfer\-Bench: building the virtuous cycle for AI\-driven LLM systems\.InProceedings of Machine Learning and Systems,Vol\.8\.External Links:[Link](https://proceedings.mlsys.org/paper_files/paper/2026/hash/37e44c4b5321605735be9761f9b758fc-Abstract-Conference.html)Cited by:[§4\.1](https://arxiv.org/html/2610.36623#S4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[43\]F\. Yao, L\. Liu, D\. Zhang, C\. Dong, J\. Shang, and J\. Gao\(2025\)On the rollout\-training mismatch in modern RL systems\.InNeurIPS Workshop on Efficient Reasoning,External Links:[Link](https://openreview.net/forum?id=8MHqvb4lK9)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p4.1)\.
- \[44\]Z\. Ye, Y\. Huang, H\. Jin, B\. Hou, J\. Shao, Z\. Yu, J\. Chen, M\. Cowan, S\. Cao, S\. Xing,et al\.\(2026\)CAKE: compiler\-agent co\-design for frontier kernel evolution\.arXiv preprint arXiv:2608\.12629\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[45\]Y\. Yu, P\. Zang, C\. H\. Tsai, H\. Wu, Y\. Shen, J\. Zhang, H\. Wang, Z\. Xiao, J\. Shi, Y\. Luo,et al\.\(2026\)Towards automated kernel generation in the era of llms\.arXiv preprint arXiv:2601\.15727\.Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p1.1)\.
- \[46\]J\. Yuan, H\. Li, X\. Ding, W\. Xie, Y\. Li, W\. Zhao, K\. Wan, J\. Shi, X\. Hu, and Z\. Liu\(2025\)Understanding and mitigating numerical sources of nondeterminism in LLM inference\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 188045–188077\.External Links:[Document](https://dx.doi.org/10.52202/085713-5653),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/hash/f80094a824ba5912d4a2de169c404a40-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px2.p1.1)\.
- \[47\]S\. Zhang, Q\. Yu, Y\. Zhang, R\. Xu, X\. Shi, G\. Li, X\. Feng, H\. Cui, and J\. Zhao\(2026\)Learning when to optimize: verified optimization skills from expert gpu\-kernel lineages\.arXiv preprint arXiv:2605\.28213\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.
- \[48\]Z\. Zhang, X\. Ding, J\. Yuan, R\. Liu, H\. Mao, J\. Xing, and Z\. Liu\(2026\)Deterministic inference across tensor parallel sizes that eliminates training\-inference mismatch\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.306,pp\. 155026–155042\.External Links:[Link](https://proceedings.mlr.press/v306/zhang26ag.html)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p4.1)\.
- \[49\]S\. Zhao, J\. Yang, Q\. Chen, J\. Hu, J\. Zheng, G\. Chen, and X\. Chen\(2026\)CoRun: padding is simple and efficient for deterministic LLM inference\.arXiv preprint arXiv:2608\.14376\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2608.14376),[Link](https://arxiv.org/abs/2608.14376)Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px2.p2.1)\.
- \[50\]T\. Zhong, N\. Ling, Y\. Pi, Z\. Wei, T\. Yu, G\. Fox, P\. Wu, and X\. Yu\(2026\)Diagnosing training inference mismatch in LLM reinforcement learning via a zero\-mismatch reference\.arXiv preprint arXiv:2605\.14220\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2605.14220),[Link](https://arxiv.org/abs/2605.14220)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p4.1)\.
- \[51\]Z\. Zhou, Y\. Zhao, H\. Zhu, W\. Wang, Z\. Bai, Y\. Zhang, S\. Wang, H\. Lin, and P\. Huang\(2026\)OpGuard: bitwise alignment for precise and general debugging of production LLM training\.In20th USENIX Symposium on Operating Systems Design and Implementation \(OSDI 26\),pp\. 1385–1406\.External Links:[Link](https://www.usenix.org/conference/osdi26/presentation/zhou-ziming)Cited by:[§1](https://arxiv.org/html/2610.36623#S1.p4.1)\.
- \[52\]X\. Zhu, S\. Peng, J\. Guo, Y\. Chen, Q\. Guo, Y\. Wen, H\. Qin, R\. Chen, Q\. Zhou, K\. Gao,et al\.\(2026\)Qimeng\-kernel: macro\-thinking micro\-coding paradigm for llm\-based high\-performance gpu kernel generation\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 29168–29176\.Cited by:[§5](https://arxiv.org/html/2610.36623#S5.SS0.SSS0.Px1.p1.1)\.

## Appendix Table of Contents

- •
- •
- •
- •

## Appendix AProblem evidence by kernel family and mechanism

Table[A1](https://arxiv.org/html/2610.36623#A1.T1)complements the aggregate results in Table[1](https://arxiv.org/html/2610.36623#S1.T1)by summarizing where confirmed BI violations and structural risks appear across kernel families\. Together, the two tables provide evidence that batch dependence occurs in released kernels and spans multiple implementation mechanisms\.

Table A1:Batch invariance evidence by kernel family and mechanism\. Confirmed: test confirmed BI violation with the associated mechanism identified\. Risk: potential risk identified by static inspection, with no test confirmed violation\. None: no evidence reported for that mechanism, not proof of absence\. N/A: no semantic batch dimension in the analyzed artifacts\. Mechanisms are not mutually exclusive\. GEMM = general matrix–matrix multiplication\.#### Evidence categories\.

We summarize the evidence as follows: High denotes a confirmed violation rate of at least 50% among batch\-applicable artifacts \(batch\-applicable denotes kernels whose per\-sample outputs are expected to be independent of the surrounding batch context\); Medium denotes at least one confirmed violation with a rate below 50%; Low denotes no confirmed violation but at least one identified structural risk; N/A denotes no true semantic batch dimension in the analyzed artifacts\. Low is an evidence category, not a measured low violation probability\.

#### Mechanism definitions\.

*Work partitioning*includes split\-K/split\-KV, sequence chunking, split–merge, recurrence segmentation, and blocks per row\.*Reduction order*includes partial\-sum grouping, reduction\-tree structure,BLOCK\_Kor tiling changes that alter accumulation order, and partial\-result merge order\.*Algorithm selection*includes autotuning, shape\- or batch\-dependent dispatch, backend kernel selection, and alternate implementation paths\.*Expert routing*includes expert–token grouping, variable expert sizes, and grouped\-GEMM scheduling\. These patterns threaten BI when a permitted change in batch context alters the computation for a preserved target sample; a pattern alone is not evidence of a violation\. Here, K is the matrix reduction dimension, KV denotes keys and values, andBLOCK\_Kis the reduction tile size\. Related BI inference analyses discuss the numerical consequences of changing reduction and partition choices\[[16](https://arxiv.org/html/2610.36623#bib.bib9)\]\.

## Appendix BExample DSL Contract

The following listing reproduces the implementedbatch\_identity\.bidcontract\. The registry loads this file directly throughparse\_contract\_file\(\); therefore, the listing uses executable contract syntax rather than pseudocode\.

kernel BatchIdentity \{ invariant\_axis batch symbols \[B, batch, batch\_size, bsz, num\_seqs, len\_indptr, num\_requests, num\_tokens, total\_q, total\_kv, num\_kv\_indices, total\_pages\]; parameter NUM\_SPLITS \{ semantic split\_structure; policy fixed\_across\_batch; aliases \[num\_splits, max\_splits, num\_partitions, split\_factor, split\_k, splits, ns, k\_split, S, \_S, SPLIT, SPLITS, K\_SPLIT, SPLIT\_K, NUM\_PARTITIONS\]; \} parameter RNG\_SEED \{ semantic rng\_stream; policy fixed\_across\_batch; aliases \[seed, rng\_seed, seed\_counter, rng\_counter\]; \} \}

Theinvariant\_axis batch symbolsdeclaration defines the batch\-source vocabulary\. ForNUM\_SPLITS, thefixed\_across\_batchpolicy marks the canonical parameter and every listed alias as protected sinks\. The analyzer consequently rejects direct or transitive data dependence, as well as batch\-derived control dependence through conditionals or loops, from any declared batch symbol toNUM\_SPLITSor an alias\. Dependencies that cannot be traced to a batch symbol declared in the contract are outside the current static\-analysis scope\.

Here, the three protected parameters are semantic abstractions rather than exhaustive code patterns:NUM\_SPLITScovers batch\-dependent partitioning and reduction structure across aliases and kernel families, while opaque algorithm selection, expert\-routing behavior, and other undeclared runtime effects are covered by the complementary dynamic validator\.

## Appendix CTask selection and evaluation details

### C\.1Task coverage

We construct the suite to provide compact coverage of BI applicable kernel semantics rather than exhaustively enumerate FlashInfer configurations\. Tasks are required to have a meaningful semantic batch dimension and are selected to cover distinct operator families and execution regimes\. We prioritize higher risk attention and linear\-attention kernels while retaining MoE, normalization, and sampling tasks to broaden operator coverage\. Within attention, the suite spans GQA, MLA, and DSA as well as representative paged/ragged and prefill/decode regimes; linear attention covers KDA and Gated DeltaNet\. We prioritize these semantic and operator axes over additional configurations that would largely duplicate the same coverage\. Overall, we evaluate diverse kernel definitions covering 1,180 benchmark workloads/shapes\.

### C\.2Protocol details

#### Evaluation protocol\.

We use each framework’s original correctness, performance benchmarking, and failure\-handling procedures\[[5](https://arxiv.org/html/2610.36623#bib.bib4);[12](https://arxiv.org/html/2610.36623#bib.bib6)\]\. Controlled generation uses Gemini 3\.7 Flash for all three frameworks\. For dynamic validation, we construct sample 0 at batch size one and embed the same input unchanged as sample 0 in each larger batch, comparing only its output bitwise\. Each context uses one preserved target sample; testing stops at the first mismatch\. TheBIEvolveworkflow and acceptance gates were designed independently of evaluation data, which are used only for measurement\. Dynamic validation intentionally samples a lightweight finite subset of the formal BI context space; static analysis complements it by tracing all declared batch\-context symbols, not only batch size, to protected sinks\. Thus, the combined modules are intended to enforce BI without exhaustive context enumeration\.

#### Shared candidate pool\.

We perform anBIEvolvecomponent ablation on one fixed pool of 200 generated candidates: 100 independently confirmed BI violations and 100 candidates with no confirmed BI violation\. Candidates are drawn across the three agent frameworks, vanilla and BI\-aware generation, and the major task families\. Selection is independent of the observed decisions under the static\-analysis\-only, dynamic\-validation\-only, and fullBIEvolveconfigurations\. We prioritize distinct violation mechanisms and limit near\-duplicate candidates from the same task and run\.

#### Independent BI labeling\.

Labels are assigned independently ofBIEvolveoutput\. Static inspection identifies candidate mechanisms and relevant batch boundaries; an independent local test then crosses the boundary and compares the preserved output bitwise\. A candidate is confirmed only when this test observes an actual bitwise output difference for the unchanged target sample\. Candidates are labeled as confirmed BI violation, or no confirmed BI violation; all unclassifiable candidates are excluded from the ablation pool\.

#### BIEvolvecomponent evaluation and timing\.

Static\-analysis\-only, dynamic\-validation\-only, and fullBIEvolveconfigurations are applied to the shared candidate pool\. Violation detection is the fraction of confirmed violations rejected by the evaluated configuration\. False rejection is the fraction of candidates without confirmed violations rejected by the evaluated configuration\. Evaluation time excludes LLM and application programming interface \(API\) calls, candidate compilation and setup, reference correctness testing, and autotuning or performance benchmarking\. Dynamic validation time includes all GPU kernel launches issued by the dynamic validation module and device synchronization\. Each configured batch context is warmed up 3 times and then measured over 10 synchronized repetitions\. We report the median across candidates; the fullBIEvolvemeasurement preserves its sequential execution and early exit behavior\.

## Appendix DLate\-stage search gains

Table[A2](https://arxiv.org/html/2610.36623#A4.T2)reports the change in best\-so\-far speedup over the final five optimization turns, grouping tasks by the BI outcome of the baseline final kernel\. For each subgroup and condition, gain is the relative change in geometric mean best\-so\-far speedup over the final five optimization turns:100​\(Send/Sstart−1\)100\(S\_\{\\mathrm\{end\}\}/S\_\{\\mathrm\{start\}\}\-1\), whereSSis the geometric mean best\-so\-far speedup across tasks\. Two tasks whose final kernels failed correctness are excluded, leaving 37 tasks\. Thus, Table[A2](https://arxiv.org/html/2610.36623#A4.T2)characterizes late\-stage search progress, whereas main Figure[5](https://arxiv.org/html/2610.36623#S4.F5)compares final speedups\.

Table A2:Late\-stage search gains by baseline BI outcome\.For tasks whose baseline final kernels have a confirmed BI violation, BI\-aware generation achieves a larger late\-stage gain \(26\.30% versus 6\.11%\)\. This pattern indicates that useful optimization continues later under BI constraints for this subgroup, but it does not establish performance beyond the fixed search budget\.