Boltzmann MapReduce: A Partition-Function Reduce for Forkable Sandboxes

# Boltzmann MapReduce: A Partition-Function Reduce for Forkable Sandboxes
Source: [https://arxiv.org/html/2607.09689](https://arxiv.org/html/2607.09689)
Yossi Eliaz Incredibuild \| islo\.dev \| HIT CS Department, Israel yossi\.eliaz@incredibuild\.comyossi@islo\.develiazy@hit\.ac\.il

###### Abstract

To leading order under local asymptotic normality \(LAN\), the confidence density a worker emits over a chunk of sizennis a Gibbs–Boltzmann measureexp⁡\{−β​E​\(θ\)\}\\exp\\\{\-\\beta E\(\\theta\)\\\}whose inverse temperature is the sample size,β=n\\beta=n\. Three consequences are exact in the Gaussian/linear case and first\-order otherwise: disjoint chunks carry independent Boltzmann factors, so the MapReduce*reduce*, read literally, is a partition functionZ=∫∏khk​d​θZ=\\int\\prod\_\{k\}h\_\{k\}\\,d\\thetawhose mode is precision\-weighted \(inverse\-variance\) pooling; frequentist consistency is the zero\-temperature limitT=1/n→0T=1/n\\to 0; and the same precision weighting gives pooling an unbounded influence function, so a single “cold” liar reporting false precision hijacks the consensus unless an explicit clip is added\. This matters now because the commodity machine of the AI era is the forkable copy\-on\-write microVM sandbox\. When the unit of execution is a forked ensemble rather than a deterministic path, the reduce should equilibrate that ensemble, not average it\. We*measure*the statistical core on an open, deterministic \(seed0\) reference implementation: the partition\-function reduce equals closed\-form inverse\-variance pooling to machine precision \(an algebraic check\); the pooled estimate tracks the full\-data oracle to a sampling gap of4\.3×10−44\.3\\times 10^\{\-4\}on a1212\-shard mean; the temperature cools as1/N1/\\sqrt\{N\}; a confident liar pulls the naive pool to17\.017\.0while the clip recovers4\.954\.95and flags it; and on a non\-linear, heterogeneous logistic estimator the precision\-weighted reduce recovers the full\-data MLE while naive equal\-weight averaging is≈24×\\approx\\\!24\\timesworse \(mean over88seeds\)\. A seven\-casepytestsuite spanning all experiments passes\. We additionally*run*the pipeline end\-to\-end on the islo forkable\-sandbox cloud \(one trial\): a distinct four\-shard configuration forked from a141141MB OCI snapshot, each shard computing its density on a separate microVM, reduces to4\.9424\.942versus an oracle4\.9454\.945—a substrate existence proof \(for a fixed shard set and seed the deterministic workers make islo and local outputs identical\), and by batching under each account’s concurrency cap we fork to256256–10241024total sandboxes at100%100\\%success across*three*real forkable\-sandbox clouds—islo, Daytona, and Tensorlake \(createp50p\_\{50\}spanning0\.200\.20–6\.96\.9s\)\. Per\-restore microVM latency and large\-scale Byzantine accuracy remain*projections*from published figures, labeled throughout\. Artifact:[https://github\.com/zozo123/boltzmann\-mapreduce](https://github.com/zozo123/boltzmann-mapreduce)\.

## 1 Introduction

MapReduce did not win on cleverness; it won on fit\. Dean and Ghemawat matched a two\-primitive model to the commodity machine of their era—the cheap, individually unreliable PC, racked by the thousand—and let the runtime absorb partitioning, scheduling, failure, and communication\[[5](https://arxiv.org/html/2607.09689#bib.bib1)\]\. The abstraction was canonical because the substrate was: when the unit of hardware is a slow\-booting, shared\-nothing box, you*reduce*by summing, counting, and concatenating across them\. The arithmetic reduce was a faithful image of the machine beneath it\.

The commodity machine of the AI era is a different object\. It is the forkable microVM sandbox: a parent boots once, warms its runtime, pauses, and is serialized to a snapshot—an OCI image of content\-addressed, deduplicated layers\[[16](https://arxiv.org/html/2607.09689#bib.bib27)\]—after which children are cloned not by rebooting but by copy\-on\-write \(CoW\) demand paging, each childmmaping the shared backing image and faulting in only the pages it touches\[[1](https://arxiv.org/html/2607.09689#bib.bib12),[12](https://arxiv.org/html/2607.09689#bib.bib15),[6](https://arxiv.org/html/2607.09689#bib.bib13)\]\. Forking a sandbox is now cheap enough to be a default for real work: running a test suite under many seeds, evaluating a coding agent across parallel rollouts, sweeping hyperparameters, benchmarking a model on sampled shards, or fanning an AI harness over branches of a decision tree to discard divergent ones\[[14](https://arxiv.org/html/2607.09689#bib.bib26)\]\. The map step—fork, run, isolate—is commoditized\. The reduce step is not\.

The trouble is that what these forks emit is noisy and uncertain: a pass rate over flaky tests, an eval score on a sampled suite, a coefficient with a standard error, an agent’s success fraction over stochastic rollouts—each from a different amount of data, each deserving a different amount of trust\. The inherited reduce averages and concatenates, discarding exactly the information the ensemble was created to expose: how much data each fork saw, how concentrated its estimate is, and whether one fork’s confident\-looking number should move the consensus at all\. We argue the reduce over a forked ensemble should be statistical, and that the right statistics fall out of a single observation about what a fork emits\.

#### The durable kernel\.

Under LAN, the confidence density a worker emits over a chunk of sizennis, to leading order, a Gibbs–Boltzmann measureh​\(θ\)∝exp⁡\{−β​E​\(θ\)\}h\(\\theta\)\\propto\\exp\\\{\-\\beta E\(\\theta\)\\\}in which the energy is the local quadratic loss and the inverse temperature is the sample size,β=n\\beta=n\(Eq\. \([2](https://arxiv.org/html/2607.09689#S3.E2)\), Eq\. \([6](https://arxiv.org/html/2607.09689#S3.E6)\)\)\. A data\-poor chunk is hot and diffuse; a data\-rich chunk is cold and concentrated\. What is empirically falsifiable here is the ratio structure \(relative weights tracknkn\_\{k\}\) and the cooling rate \(width∼1/N\\sim 1/\\sqrt\{N\}\); takingβ=n\\beta=nper observation is the natural convention that makes reduce a partition function\. Because disjoint chunks are independent, their factors multiply, so reduce computes a partition function whose mode is precision\-weighted pooling; consistency is the zero\-temperature limit; and the precision weighting that makes pooling efficient also gives it an unbounded influence function, so a confident liar mandates an explicit clip\.

#### What is new\.

The divide\-and\-conquer statistics already exist: parameter servers\[[11](https://arxiv.org/html/2607.09689#bib.bib21)\], one\-shot\[[23](https://arxiv.org/html/2607.09689#bib.bib22)\]and surrogate\-likelihood estimation\[[9](https://arxiv.org/html/2607.09689#bib.bib23)\], the bag\-of\-little\-bootstraps\[[10](https://arxiv.org/html/2607.09689#bib.bib24)\], and confidence\-distribution meta\-analysis\[[22](https://arxiv.org/html/2607.09689#bib.bib2),[18](https://arxiv.org/html/2607.09689#bib.bib3)\]\. Our reduce*is*the Rao\-type confidence\-distribution \(Rao\-CD\) pooling of Tang and Song\[[19](https://arxiv.org/html/2607.09689#bib.bib4)\]; the Gaussian confidence density is textbook LAN\. We do not claim these\. Our contribution is \(i\) naming the reduce as a partition function withβ=n\\beta=n, which makes precision weighting, consistency, and outlier behavior a single thermodynamic statement rather than a family of weighting recipes; and, operationally, \(ii\) the observation that a cheap forking substrate makes the unit of work a per\-shard*ensemble*of forked replicas rather than one local estimate, and that an arithmetic reduce is no longer the faithful image of that machine\.

Contributions\.

- •Identity \(proven\)\.The asymptotic confidence density is a Gibbs measure withβ=n\\beta=n\(Eq\. \([2](https://arxiv.org/html/2607.09689#S3.E2)\), Eq\. \([6](https://arxiv.org/html/2607.09689#S3.E6)\)\), recasting reduce as a partition\-function product \(Eq\. \([7](https://arxiv.org/html/2607.09689#S3.E7)\)\) and consistency as the zero\-temperature limit, exact under Gaussian/linear energy and first\-order otherwise\.
- •Reduce \(measured\)\.An open reference implementation: pooling equals closed\-form inverse\-variance pooling to machine precision; the pooled estimate recovers the full\-data oracle; the temperature cools as1/N1/\\sqrt\{N\}; pooling is Byzantine\-fragile but clip\-repaired; and on a non\-linear, heterogeneous logistic estimator pooling recovers the oracle MLE while naive averaging is markedly worse\. One end\-to\-end run on a real forkable\-sandbox cloud \(islo\)\.
- •Substrate \(enabling design\)\.Forkable CoW microVM sandboxes as the commodity machine that makes the forked ensemble the unit of execution, with a compact agentic control plane and a measured clip as its one validated robustness piece; per\-restore latency and large\-scale robustness are stated as labeled projections \(Tables[2](https://arxiv.org/html/2607.09689#S5.T2),[3](https://arxiv.org/html/2607.09689#S5.T3)\)\.

## 2 Programming Model and Forkable\-Sandbox Substrate

The model keepsmap/reduce\[[5](https://arxiv.org/html/2607.09689#bib.bib1)\]but makes estimates and their variances first\-class\. The user supplies two functions; forking, snapshotting, and metadata tracking are abstracted beneath them\.mapreturns, per key, astatistical\_result: a point estimate \(mean, regression coefficients, empirical CDF\), an uncertainty measure \(confidence interval, standard error, sandwich covariance\[[21](https://arxiv.org/html/2607.09689#bib.bib20)\], or full confidence density\), the local sample sizenn, and execution metadata\. The implemented contract is two calls: a worker emits a confidence\-density JSON\(θ^k,Jnk,nk\)\(\\hat\{\\theta\}\_\{k\},J\_\{n\_\{k\}\},n\_\{k\}\), andreduce\_partitionmultiplies the Gibbs factors of Eq\. \([7](https://arxiv.org/html/2607.09689#S3.E7)\)—precision\-weighted pooling—merging heterogeneous estimates without centralizing raw individual\-participant data\[[18](https://arxiv.org/html/2607.09689#bib.bib3),[19](https://arxiv.org/html/2607.09689#bib.bib4)\]\.

#### Worker substrate\.

Three properties govern parallelism: instantiation speed, per\-worker overhead, and isolation\. Containers cold\-start in0\.40\.4–22s with heavy memory overhead; V8 isolates boot sub\-millisecond but cannot run the scientific Python stack\. We use lightweight micro\-virtualization with memory snapshotting on a minimal VMM such as Firecracker\[[1](https://arxiv.org/html/2607.09689#bib.bib12)\], KVM\-isolated behind a restricted device model\. A parent microVM boots once, warms its runtime, pauses, and is serialized to a snapshot whose information layer is an*OCI image*: content\-addressed base\-OS, runtime, and user\-code layers\[[16](https://arxiv.org/html/2607.09689#bib.bib27),[6](https://arxiv.org/html/2607.09689#bib.bib13)\]deduplicated and shared read\-only across every fork, so a child is restored rather than rebuilt and only its writes diverge\. Each childmmaps its guest memoryMAP\_PRIVATEand is demand\-paged on first touch viauserfaultfd\[[12](https://arxiv.org/html/2607.09689#bib.bib15)\]with working\-set prefetch\[[20](https://arxiv.org/html/2607.09689#bib.bib14)\]; a write traps a fault and copies only that page into a per\-child overlay, so shared base layers are never duplicated and disk growth is bounded as fanout rises—a CoW property our local backend exercises directly and the islo run of Sec\.[5](https://arxiv.org/html/2607.09689#S5)exercises on a real cloud\.*Live branching*extends this to running workers: an active worker snapshots in\-flight memory and forks a clone with a source\-pause window on the order of tens of milliseconds \(a reported∼\\sim56 ms on a1\.51\.5GiB source\[[14](https://arxiv.org/html/2607.09689#bib.bib26)\]\), the copy completing asynchronously—so an AI harness can fork at a decision node, test competing branches in parallel, and discard divergent ones\. Commercial forkable\-sandbox clouds \(islo, Daytona\[[4](https://arxiv.org/html/2607.09689#bib.bib28)\], Morph\[[14](https://arxiv.org/html/2607.09689#bib.bib26)\], Modal\) expose this snapshot\-and\-fork primitive directly\.

## 3 The Boltzmann Reduce

When the data\-generating density cannot be fully specified, inference uses*estimating functions*built from partial aspects of the mechanism, such as moment conditions\[[7](https://arxiv.org/html/2607.09689#bib.bib5)\]\. A*confidence distribution*is a frequentist, sample\-dependent distribution encoding confidence intervals of all levels\[[22](https://arxiv.org/html/2607.09689#bib.bib2),[18](https://arxiv.org/html/2607.09689#bib.bib3)\]\. Processing chunkW\(k\)W^\{\(k\)\}, a worker yields a confidence densityhk​\(θ\)h\_\{k\}\(\\theta\)\. Under LAN,

nk1/2​\(θ^k−θk,0\)​∼asy\.​N​\(0,jk−1​\(θk,0\)\),n\_\{k\}^\{1/2\}\(\\hat\{\\theta\}\_\{k\}\-\\theta\_\{k,0\}\)\\overset\{\\text\{asy\.\}\}\{\\sim\}N\\\!\\left\(0,\\,j\_\{k\}^\{\-1\}\(\\theta\_\{k,0\}\)\\right\),\(1\)withjkj\_\{k\}the per\-observation sandwich \(inverse Godambe\) information\[[7](https://arxiv.org/html/2607.09689#bib.bib5),[21](https://arxiv.org/html/2607.09689#bib.bib20)\]\. The confidence density is, to leading order,

hk​\(θ\)∝exp⁡\{−nk2​\(θ^k−θ\)T​Jnk​\(θ^k\)​\(θ^k−θ\)\},h\_\{k\}\(\\theta\)\\propto\\exp\\\!\\left\\\{\-\\tfrac\{n\_\{k\}\}\{2\}\(\\hat\{\\theta\}\_\{k\}\-\\theta\)^\{T\}J\_\{n\_\{k\}\}\(\\hat\{\\theta\}\_\{k\}\)\(\\hat\{\\theta\}\_\{k\}\-\\theta\)\\right\\\},\(2\)whereJnkJ\_\{n\_\{k\}\}estimates the per\-observation informationjkj\_\{k\}, so chunkkkcarries total precisionnk​Jnkn\_\{k\}J\_\{n\_\{k\}\}\.Regime\.Under correct specification,hk=exp⁡\{−nk​Ek\}​\(1\+O​\(nk−1/2\)\)h\_\{k\}=\\exp\\\{\-n\_\{k\}E\_\{k\}\\\}\\,\(1\+O\(n\_\{k\}^\{\-1/2\}\)\); the Gibbs form is*exact*iff the estimating function is linear inθ\\theta\(Gaussian location, linear regression\) and otherwise holds toO​\(nk−1/2\)O\(n\_\{k\}^\{\-1/2\}\)by Laplace approximation\. Higher\-order corrections \(skewness, curvature\) are exactly what a CD carries beyond first order and are dropped here\.

The reduce merges the independent densities via Rao\-CD\[[19](https://arxiv.org/html/2607.09689#bib.bib4)\], which generalizes MLE to robust estimating equations\. With full data one would solve

ψfull​\(W;θ\)=n−1​∑i=1nψ​\(Wi;θ\)=0\.\\psi\_\{\\text\{full\}\}\(W;\\theta\)=n^\{\-1\}\\sum\_\{i=1\}^\{n\}\\psi\(W\_\{i\};\\theta\)=0\.\(3\)Since data cannot be centralized, reduce solves the combined equation forθ^rcd\\hat\{\\theta\}\_\{\\text\{rcd\}\},

ΨR​\(θ\)=n−1/2​∑k=1Knk​SnkT​\(θ\)​V^nk−1​ψsub\(k\)​\(W\(k\);θ\)=0,\\Psi\_\{R\}\(\\theta\)=n^\{\-1/2\}\\sum\_\{k=1\}^\{K\}n\_\{k\}\\,S\_\{n\_\{k\}\}^\{T\}\(\\theta\)\\,\\hat\{V\}\_\{n\_\{k\}\}^\{\-1\}\\,\\psi\_\{\\text\{sub\}\}^\{\(k\)\}\(W^\{\(k\)\};\\theta\)=0,\(4\)withSnk​\(θ\)=−ψ˙sub\(k\)​\(θ\)S\_\{n\_\{k\}\}\(\\theta\)=\-\\dot\{\\psi\}\_\{\\text\{sub\}\}^\{\(k\)\}\(\\theta\)andV^nk\\hat\{V\}\_\{n\_\{k\}\}estimating the sub\-estimating\-function variance atθ^k\\hat\{\\theta\}\_\{k\}\. SolvingΨR=0\\Psi\_\{R\}=0combines local*scores*, whereas the mode of∏khk\\prod\_\{k\}h\_\{k\}averages local*point estimates*; the two coincide exactly only under quadratic \(Gaussian/linear\) energy, where the score is linear inθ\\theta, and otherwise to first order\. Under correct specification and a common truthθk,0=θ0\\theta\_\{k,0\}=\\theta\_\{0\}, Rao\-CD is asymptotically first\-order efficient,*recovering*—never exceeding—the infeasible full\-data estimator, and dominating equal\-weight pooling under heterogeneous information \(strictly so when precisions differ; the two agree under equalnkn\_\{k\}\), connecting to GMM and quadratic estimating\-function optimality\[[8](https://arxiv.org/html/2607.09689#bib.bib6),[3](https://arxiv.org/html/2607.09689#bib.bib7)\]\.

#### The Gibbs view\.

Writing the local quadratic loss as an energy,

Ek​\(θ\)=12​\(θ^k−θ\)T​Jnk​\(θ^k\)​\(θ^k−θ\),E\_\{k\}\(\\theta\)=\\tfrac\{1\}\{2\}\(\\hat\{\\theta\}\_\{k\}\-\\theta\)^\{T\}J\_\{n\_\{k\}\}\(\\hat\{\\theta\}\_\{k\}\)\(\\hat\{\\theta\}\_\{k\}\-\\theta\),\(5\)Eq\. \([2](https://arxiv.org/html/2607.09689#S3.E2)\) readshk​\(θ\)∝exp⁡\{−βk​Ek​\(θ\)\}h\_\{k\}\(\\theta\)\\propto\\exp\\\{\-\\beta\_\{k\}E\_\{k\}\(\\theta\)\\\}with

βk=nk\.\\beta\_\{k\}=n\_\{k\}\.\(6\)This is a normalization convention—energy defined*per observation*—not a discovery: one could absorbnkn\_\{k\}intoJJand setβ=1\\beta=1\. What is empirically falsifiable is the ratio structure \(relative weights tracknkn\_\{k\}\) and the cooling rate \(width∼1/N\\sim 1/\\sqrt\{N\}\), not an absolute temperature scale\. Under the per\-observation convention, a data\-poor chunk is hot and diffuse, a data\-rich chunk cold and concentrated\. For disjoint chunks the factors multiply and energies and inverse temperatures add:

h​\(θ\)∝∏k=1Khk​\(θ\)∝exp⁡\{−∑k=1Kβk​Ek​\(θ\)\},h\(\\theta\)\\propto\\prod\_\{k=1\}^\{K\}h\_\{k\}\(\\theta\)\\propto\\exp\\\!\\Big\\\{\-\\sum\_\{k=1\}^\{K\}\\beta\_\{k\}E\_\{k\}\(\\theta\)\\Big\\\},\(7\)withZ=∫∏khk​\(θ\)​d​θZ=\\int\\prod\_\{k\}h\_\{k\}\(\\theta\)\\,d\\theta\. \(Additivity needs disjoint partitions; within\-shard bootstrap replicas are not independentβ=nk\\beta=n\_\{k\}replicas and are not multiplied\.\) The mode minimizes∑kβk​Ek\\sum\_\{k\}\\beta\_\{k\}E\_\{k\}, i\.e\. precision\-weighted pooling, since chunkkkenters with weightβk​Jnk\\beta\_\{k\}J\_\{n\_\{k\}\}\.*Statistical optimality*is additivity of precisions under the product measure\.*Robustness to honest heterogeneity*is Boltzmann suppression: an honest outlier with modest reported precision is a high\-energy, exponentially damped configuration\. This is*not*a Byzantine defense: an adversary reporting false high precision \(falsely “cold”\) receives large weight and captures the consensus—precision\-weighted pooling has an unbounded influence function—which is why the clip of Sec\.[4](https://arxiv.org/html/2607.09689#S4)is mandatory\. Separately, fixKKand let everynk→∞n\_\{k\}\\to\\infty\(soN=∑knk→∞N=\\sum\_\{k\}n\_\{k\}\\to\\infty\); the per\-observation temperature1/nk→01/n\_\{k\}\\to 0and the measure concentrates by Laplace’s method on the minimizer of the averaged energy, which is consistent forθ0\\theta\_\{0\}when the estimating function is unbiased with the usual regularity\.*Frequentist consistency is the zero\-temperature limit\.*

## 4 Orchestration and Security \(Design Sketch\)

The master is replaced by a decentralized, durable agentic control plane \(envisioned, not yet measured\)\. The one piece validated below is its core mechanism: a median\-absolute\-deviation precision clip that caps a confident liar’s reported precision and down\-weights location outliers\. The remaining design elements are unmeasured\. A Command Center receives a high\-level JobSpec and realizes it across an agent mesh of planning, execution, and scheduling agents that place work and bridge edge and serverless backends\[[13](https://arxiv.org/html/2607.09689#bib.bib11)\]\. Agentic workflows break synchronous scheduling, so the control plane runs on durable execution: every hop is logged, an OOM kill or timeout triggers replay rather than a crash, and a failed map task is re\-forked from the same base template; the durable log, sharded and replicated, is the component to scale at high fanout\. Where a map and its reduce are co\-located in a single\-address\-space tier, software\-enforced isolation enables a zero\-copy hand\-off of the confidence density \(reported single\-system gains of over2×2\\times\[[17](https://arxiv.org/html/2607.09689#bib.bib25)\]are a design target we do not reproduce\)\. Because nodes span hybrid clouds, vTPM boot\-integrity attestation anchors trust before a sandbox joins the mesh \(boot integrity only, not statistical behavior\), and dynamic information\-flow control mitigates co\-location side channels\. For poisoning, where IID\-based defenses such as Krum\[[2](https://arxiv.org/html/2607.09689#bib.bib16)\]degrade under non\-IID inputs, reduce agents may discard or clip implausibly precise updates; large\-scale robustness is left to future evaluation\.

## 5 Evaluation

We separate what we*measured*from what we*project*\. The measurements \(Sec\.[5](https://arxiv.org/html/2607.09689#S5)A\) are deterministic \(seed0\), Python with onlynumpy/matplotlib, and reproduce bit\-for\-bit from the artifact\. The projections \(Sec\.[5](https://arxiv.org/html/2607.09689#S5)B\) are drawn from figures published for the cited systems and from stated design targets; no head\-to\-head experiment against the named baselines was run\.

### A\. Measured: the Boltzmann reduce, on real runs

The implementation realizes Sec\.[2](https://arxiv.org/html/2607.09689#S2)directly:mapemits\(θ^k,Jnk,nk\)\(\\hat\{\\theta\}\_\{k\},J\_\{n\_\{k\}\},n\_\{k\}\)andreduce\_partitionmultiplies the Gibbs factors of Eq\. \([7](https://arxiv.org/html/2607.09689#S3.E7)\)\. Two backends sit behind one interface: alocalbackend that forks OS processes, and anislobackend that snapshots a parent microVM once \(its environment auv\-locked Python\) and forks one sandbox per shard via theisloCLI \(islo snapshot save bmr\-base;islo use bmr\-shard\-kk\-\-snapshot bmr\-base \.\.\.\)\. We verified that islo launches a Claude Code harness inside the fork; the deterministic numbers below come from a fixed worker invocation \(\-\- uv run python \-m bmr\.worker\) so the reduce is reproducible\.

#### Pooling is the partition function \(algebraic\); efficiency is recovered \(statistical\)\.

On1212homoscedastic shards \(sd=1\.5\\mathrm\{sd\}=1\.5\) with heterogeneous sizesnk∈\[107,355\]n\_\{k\}\\in\[107,355\]aroundθ=5\.0\\theta=5\.0, the partition\-function reduce yieldsθ^pool=4\.9542\\hat\{\\theta\}\_\{\\text\{pool\}\}=4\.9542, CI\[4\.8980,5\.0104\]\[4\.8980,5\.0104\], against a full\-data oracle4\.95464\.9546\. Two distinct facts: reduce reproduces the closed\-form inverse\-variance pooled estimator to*machine precision*—an algebraic check, since multiplying Gaussian factors*is*that estimator—while the pooled estimator tracks the oracle to a*sampling*gap\|θ^pool−θ^oracle\|=4\.29×10−4\|\\hat\{\\theta\}\_\{\\text\{pool\}\}\-\\hat\{\\theta\}\_\{\\text\{oracle\}\}\|=4\.29\\times 10^\{\-4\}, a finite\-sample efficiency check, attaining the oracle ceiling without exceeding it\. The per\-shard weights tracknkn\_\{k\}exactly asβk=nk\\beta\_\{k\}=n\_\{k\}predicts: the largest shard \(n=355n=355\) carries12\.6%12\.6\\%of the weight, the smallest \(n=107n=107\) carries3\.6%3\.6\\%\. The same holds for a1212\-shard linear regression \(trueβ=\[1\.5,−2\.0,0\.75\]\\beta=\[1\.5,\-2\.0,0\.75\]\): pooled\[1\.494,−1\.983,0\.748\]\[1\.494,\-1\.983,0\.748\]versus oracle\[1\.516,−1\.966,0\.730\]\[1\.516,\-1\.966,0\.730\],L2L\_\{2\}gap3\.3×10−23\.3\\times 10^\{\-2\}\. The larger regression gap is finite\-sample variation in the design\-matrix information across shards, not a failure of the identity; both cases are exactly the linear regime where Eq\. \([2](https://arxiv.org/html/2607.09689#S3.E2)\) is Gibbs\-exact, so machine\-precision agreement is expected, not surprising\.

#### The informative regime: non\-linear and heterogeneous\.

To test where the identity is only first\-order, we run logistic regression—a non\-linear estimating function—across five strongly heterogeneous shards \(nk∈\{60,120,400,2000,5000\}n\_\{k\}\\in\\\{60,120,400,2000,5000\\\}, trueβ=\[0\.5,−1\.2,2\.0\]\\beta=\[0\.5,\-1\.2,2\.0\]\)\. Here the Gibbs energy is the leading\-order LAN quadratic, not exact\. Across88seeds the precision\-weighted reduce recovers the full\-data maximum\-likelihood estimator \(pooled gap mean8\.3×10−38\.3\\times 10^\{\-3\}, sd4\.2×10−34\.2\\times 10^\{\-3\}\), whereas*naive equal\-weight*averaging of the per\-shard coefficients is mean0\.180\.18away—24×24\\timesworse on average \(range66–46×46\\timesover seeds, scaling with the spread of per\-shard information\)\. This is the non\-confirmatory result: when shards disagree in size and information, the thermodynamic weightβk=nk\\beta\_\{k\}=n\_\{k\}is what tracks the oracle, and the arithmetic average—MapReduce’s inherited reduce—visibly fails\.

![Refer to caption](https://arxiv.org/html/2607.09689v1/fig_gibbs.png)Figure 1:Per\-shard Gibbs confidence densities \(hot, data\-poor replicas\) collapse into the sharp pooled density obtained by the partition\-function reduce of Eq\. \([7](https://arxiv.org/html/2607.09689#S3.E7)\); withβk=nk\\beta\_\{k\}=n\_\{k\}, multiplying Boltzmann factors recovers the full\-data oracle, which overlaps the pool\. \(Measured, seed0\.\)
#### Cooling\.

As shards accrue, the pooled95%95\\%CI half\-width shrinks while tracking the1/N1/\\sqrt\{N\}reference in cumulative sample sizeNN—the zero\-temperature limit made concrete:T→0T\\to 0as data accrue\.

#### Byzantine fragility, and the clip\.

Adding to the1212honest shards a single confident liar \(θ=17\.0\\theta=17\.0, falsely tiny variance,50×50\\timesinflated precision, fabricatedn=2000n=2000\), naive pooling is hijacked to17\.000417\.0004—the unbounded influence function in action\. The detection layer \(byzantine\_clip,κ=3\\kappa=3\) caps the liar’s reported precision atmedian\+κ⋅MAD\\mathrm\{median\}\+\\kappa\\cdot\\mathrm\{MAD\}; with its fabricated precision capped to the honest scale, its location no longer dominates and the reduce returns to4\.95424\.9542, matching the clean1212\-shard pool, and the liar is flagged\. Both halves hold on real data: pooling alone is not Byzantine\-robust, and the clip recovers the truth\. A seven\-casepytestsuite spanning all experiments locks these numbers: the algebraic identity, oracle tracking for mean and precision, regression recovery, clip\-versus\-liar, JSON round\-trip of the Gibbs identityneg\_log\_density​\(θ\)=β⋅energy​\(θ\)\\texttt\{neg\\\_log\\\_density\}\(\\theta\)=\\beta\\cdot\\texttt\{energy\}\(\\theta\), and the heterogeneous logistic recovery\.

#### End\-to\-end on a real forkable\-sandbox cloud\.

We ran the full pipeline on theislobackend in a four\-shard configuration: from a141141MB uv\-locked OCI parent snapshot we forked one sandbox per shard, each computing its confidence density on a separate microVM, reducing to a pooled mean of4\.9424\.942against a full\-data oracle of4\.9454\.945\(gap2\.9×10−32\.9\\times 10^\{\-3\}; committed run log\)\. Because the workers are deterministic and seed\-pinned, for a*fixed shard set and seed*theisloandlocalbackends produce identical statistical output by construction \(this scoping is why the four\-shard islo pair differs from the1212\-shard headline run above\); the islo trial therefore evidences that the snapshot\-once/fork\-per\-shard*substrate executes end\-to\-end*, not that islo yields a different number\. Instrumenting the forks \(perf\_counter, committed run log\), each dispatched in≈6\.7\\approx\\\!6\.7s \(min6\.676\.67, max6\.706\.70\), with the four concurrent forks completing in6\.706\.70s total\. These figures are the restore\-plus\-run\-plus\-capture round\-trip dominated by client–cloud latency,*not*the in\-host restore \(Sec\.[5](https://arxiv.org/html/2607.09689#S5)B; see “What we did not do”\)\.

Table 1:Batched fanout from one OCI snapshot on three real forkable\-sandbox clouds, concurrency held under each account’s quota\. All reach their total at100%100\\%success\. \(†\\daggerislo is round\-trip\-inclusive dispatch, not in\-host create\.\)
#### Fanout, measured on three clouds\.

Holding concurrency*under*each account’s cap and batching, we climb the total fanout from one OCI snapshot on three real forkable\-sandbox clouds—islo,Daytona, andTensorlake\(Table[1](https://arxiv.org/html/2607.09689#S5.T1)\)—each reaching256256–10241024total forks at100%100\\%success\. Two things stand out\. First, per\-fork*create*latency spans nearly two orders of magnitude across platforms exposing the*same*snapshot\-and\-fork abstraction \(0\.200\.20s on Daytona to≈6\.9\{\\approx\}6\.9s round\-trip on islo\): “forkable sandbox” is one interface but many performance classes\. Second, on Daytona create latency is*flat*from6464to10241024forks—per\-fork cost stays bounded as fanout rises\. Concurrency itself is account\-quota\-capped \(∼16\{\\sim\}16–3232islo,1010Daytona,88Tensorlake here\), so high total fanout comes from batching under the cap, not raw concurrency—a tier limit, not an architectural one\. The*statistical*ensemble scales identically in the abstract: local process\-fork fanout toK=1024K=1024shrinks the pooled95%95\\%CI half\-width from0\.0790\.079\(K=16K\{=\}16\) to0\.01020\.0102\(K=1024K\{=\}1024\)—the1/K1/\\sqrt\{K\}cooling ofβ=n\\beta=n—with the pooled mean within2\.1×10−32\.1\\times 10^\{\-3\}of the truth \(committed run logs\)\.

### B\. Projected: latency and large\-scale robustness

#### Forking latency\.

Our CoW microVM substrate recovers polyglot, KVM\-enforced isolation within single\-digit milliseconds of isolate\-class restore\. With snapshot layering and UFFD prefetch\[[1](https://arxiv.org/html/2607.09689#bib.bib12),[6](https://arxiv.org/html/2607.09689#bib.bib13),[20](https://arxiv.org/html/2607.09689#bib.bib14)\], published figures suggest low\-millisecond restores for a small \(∼\\sim64 MB\) working set; a warmed NumPy/PyTorch runtime faults in far more and will exceed this range\. Sub\-2020ms p99 dispatch assumes a warm\-pool hit; a cold miss degrades to the100100–150150ms cold\-start row\. These caveats hold for the whole row; we do not re\-litigate them below\.

Table 2:*Projected*latency/isolation/density across serverless environments \(figures from published systems; the microVM restore column assumes a small∼\\sim64 MB working set under UFFD prefetch, not full\-image fault\-in, and the p99 assumes a warm\-pool hit\)\.
#### Large\-scale Byzantine robustness\.

Under a threat model of up to a30%30\\%fraction of non\-colluding Byzantine workers, our design target is mid\-90% accuracy with roughly one\-third lower communication than IID\-robust baselines, stemming from transmitting condensed confidence densities rather than raw data\. This is a*design target, not a measurement*: we present published Krum/FLAME numbers \(Table[3](https://arxiv.org/html/2607.09689#S5.T3)\) only as context, on intrusion\-detection benchmarks \(Edge\-IIoTset, CIC\-IDS2017, UNSW\-NB15\) whose protocol and threat model differ from CD aggregation, and we deliberately omit a projected row from the table to avoid implying a head\-to\-head win we did not run\.

Table 3:Published Byzantine\-robustness figures for Krum/FLAME on intrusion\-detection benchmarks, shown*as context only*\. Our mid\-90% accuracy at a30%30\\%Byzantine fraction with∼\\sim1/3 lower communication is a stated*design target*, not a measurement, and is intentionally not tabulated alongside these rows\.
#### What we did not do\.

We have not run a head\-to\-head against Krum/FLAME, have not isolated the in\-host microVM restore latency \(our islo figures are round\-trip\-inclusive, so the33–88ms restore of Table[2](https://arxiv.org/html/2607.09689#S5.T2)remains projected and is not contradicted by them\), reached256256–10241024total forks across islo, Daytona, and Tensorlake by batching under the per\-account concurrency caps \(tier limits, not architectural\) but did not exceed those concurrent caps, and have not validated the LM\-agent control plane of Sec\.[4](https://arxiv.org/html/2607.09689#S4)\. The measured claims are confined to Sec\.[5](https://arxiv.org/html/2607.09689#S5)A\.

## 6 Conclusion

The commodity machine of the AI era is the forkable microVM sandbox, and when the unit of execution is a forked ensemble, the reduce should equilibrate it rather than average it\. The kernel is one identity: to leading order under LAN, the confidence density a worker emits is a Gibbs measure withβ=n\\beta=n, so reduce is a partition function, consistency is the zero\-temperature limit, and a confident liar—a falsely cold report—is the failure mode that mandates a clip\. We measured this core on an open artifact \(Sec\.[5](https://arxiv.org/html/2607.09689#S5)A\): pooling equals the partition function and recovers the full\-data oracle \(gap4\.3×10−44\.3\\times 10^\{\-4\}\), the temperature cools as1/N1/\\sqrt\{N\}, the clip repairs the Byzantine fragility, and the pipeline runs end\-to\-end on a real forkable\-sandbox cloud\. The substrate’s per\-restore latency and large\-scale robustness remain labeled projections, to be established by systems experiments at scale\.

#### Outlook: distributed software engineering for AI coding agents\.

We close by naming the larger program this paper opens, as an agenda rather than a measured result\. MapReduce gave distributed*data*a faithful primitive; what is still missing is the analogue for distributed*code*—a theory and paradigm for code evolution via snapshots, now that the commodity machine is a forkable sandbox rather than a racked PC\. The picture is an army of coding agents that fork warm OCI snapshots per task, branch, or hypothesis, evolve code fire\-and\-forget, and are reaped; the reduce that turns the noisy outcomes these forks emit \(test pass\-rates, eval scores, build and patch successes, benchmark deltas\) into a trustworthy decision has no principled foundation yet\. We do not benchmark that program here and claim no software\-engineering results for it; our measurements concern only the statistical reduce\. What this paper contributes is a first principled primitive for that reduce: each agent’s outcome carries an implicit confidence—how much it ran, how concentrated, how trustworthy—so the right combination is the partition\-function, precision\-weighted reduce with a Byzantine clip, not an arithmetic average\.

## References

- \[1\]\(2020\)Firecracker: lightweight virtualization for serverless applications\.InProceedings of the 17th USENIX Symposium on Networked Systems Design and Implementation \(NSDI\),pp\. 419–434\.Cited by:[§1](https://arxiv.org/html/2607.09689#S1.p2.1),[§2](https://arxiv.org/html/2607.09689#S2.SS0.SSS0.Px1.p1.4),[§5](https://arxiv.org/html/2607.09689#S5.SSx2.SSS0.Px1.p1.4)\.
- \[2\]P\. Blanchard, E\. M\. El Mhamdi, R\. Guerraoui, and J\. Stainer\(2017\)Machine learning with adversaries: byzantine tolerant gradient descent\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.30,pp\. 119–129\.Cited by:[§4](https://arxiv.org/html/2607.09689#S4.p1.1),[Table 3](https://arxiv.org/html/2607.09689#S5.T3.9.2.1.1)\.
- \[3\]M\. Crowder\(1987\)On linear and quadratic estimating functions\.Biometrika74\(3\),pp\. 591–597\.Cited by:[§3](https://arxiv.org/html/2607.09689#S3.p2.9)\.
- \[4\]Daytona\(2024\)Daytona: secure and elastic infrastructure for running ai\-generated code\.Note:[https://www\.daytona\.io](https://www.daytona.io/)Forkable AI sandbox platform; accessed 2026Cited by:[§2](https://arxiv.org/html/2607.09689#S2.SS0.SSS0.Px1.p1.4),[Table 1](https://arxiv.org/html/2607.09689#S5.T1.8.4.4)\.
- \[5\]J\. Dean and S\. Ghemawat\(2004\)MapReduce: simplified data processing on large clusters\.InProceedings of the 6th USENIX Symposium on Operating Systems Design and Implementation \(OSDI\),pp\. 137–150\.Cited by:[§1](https://arxiv.org/html/2607.09689#S1.p1.1),[§2](https://arxiv.org/html/2607.09689#S2.p1.2)\.
- \[6\]D\. Du, T\. Yu, Y\. Xia, B\. Zang, G\. Yan, C\. Qin, Q\. Wu, and H\. Chen\(2020\)Catalyzer: sub\-millisecond startup for serverless computing with initialization\-less booting\.InProceedings of the 25th International Conference on Architectural Support for Programming Languages and Operating Systems \(ASPLOS\),pp\. 467–481\.Cited by:[§1](https://arxiv.org/html/2607.09689#S1.p2.1),[§2](https://arxiv.org/html/2607.09689#S2.SS0.SSS0.Px1.p1.4),[§5](https://arxiv.org/html/2607.09689#S5.SSx2.SSS0.Px1.p1.4)\.
- \[7\]V\. P\. Godambe\(1960\)An optimum property of regular maximum likelihood estimation\.The Annals of Mathematical Statistics31\(4\),pp\. 1208–1211\.Cited by:[§3](https://arxiv.org/html/2607.09689#S3.p1.2),[§3](https://arxiv.org/html/2607.09689#S3.p1.3)\.
- \[8\]L\. P\. Hansen\(1982\)Large sample properties of generalized method of moments estimators\.Econometrica50\(4\),pp\. 1029–1054\.Cited by:[§3](https://arxiv.org/html/2607.09689#S3.p2.9)\.
- \[9\]M\. I\. Jordan, J\. D\. Lee, and Y\. Yang\(2019\)Communication\-efficient distributed statistical inference\.Journal of the American Statistical Association114\(526\),pp\. 668–681\.Cited by:[§1](https://arxiv.org/html/2607.09689#S1.SS0.SSS0.Px2.p1.1)\.
- \[10\]A\. Kleiner, A\. Talwalkar, P\. Sarkar, and M\. I\. Jordan\(2014\)A scalable bootstrap for massive data\.Journal of the Royal Statistical Society: Series B76\(4\),pp\. 795–816\.Cited by:[§1](https://arxiv.org/html/2607.09689#S1.SS0.SSS0.Px2.p1.1)\.
- \[11\]M\. Li, D\. G\. Andersen, J\. W\. Park, A\. J\. Smola, A\. Ahmed, V\. Josifovski, J\. Long, E\. J\. Shekita, and B\. Su\(2014\)Scaling distributed machine learning with the parameter server\.In11th USENIX Symposium on Operating Systems Design and Implementation \(OSDI\),pp\. 583–598\.Cited by:[§1](https://arxiv.org/html/2607.09689#S1.SS0.SSS0.Px2.p1.1)\.
- \[12\]Linux man\-pages projectUserfaultfd\(2\) — linux programmer’s manual\.Note:[https://man7\.org/linux/man\-pages/man2/userfaultfd\.2\.html](https://man7.org/linux/man-pages/man2/userfaultfd.2.html)Accessed: 2026\-06\-17Cited by:[§1](https://arxiv.org/html/2607.09689#S1.p2.1),[§2](https://arxiv.org/html/2607.09689#S2.SS0.SSS0.Px1.p1.4)\.
- \[13\]B\. McMahan, E\. Moore, D\. Ramage, S\. Hampson, and B\. A\. y\. Arcas\(2017\)Communication\-efficient learning of deep networks from decentralized data\.InProceedings of the 20th International Conference on Artificial Intelligence and Statistics \(AISTATS\),pp\. 1273–1282\.Cited by:[§4](https://arxiv.org/html/2607.09689#S4.p1.1)\.
- \[14\]Morph Labs\(2024\)Infinibranch: instant snapshotting and branching of running sandboxes\.Note:[https://cloud\.morph\.so](https://cloud.morph.so/)Accessed 2026Cited by:[§1](https://arxiv.org/html/2607.09689#S1.p2.1),[§2](https://arxiv.org/html/2607.09689#S2.SS0.SSS0.Px1.p1.4)\.
- \[15\]T\. D\. Nguyen, P\. Rieger, R\. De Viti, H\. Chen, B\. B\. Brandenburg, H\. Yalame, H\. Möllering, H\. Fereidooni, S\. Marchal, M\. Miettinen, A\. Mirhoseini, S\. Zeitouni, F\. Koushanfar, A\. Sadeghi, and T\. Schneider\(2022\)FLAME: taming backdoors in federated learning\.InProceedings of the 31st USENIX Security Symposium \(USENIX Security\),pp\. 1415–1432\.Cited by:[Table 3](https://arxiv.org/html/2607.09689#S5.T3.9.3.2.1)\.
- \[16\]Open Container Initiative\(2017\)OCI image format specification\.Note:[https://github\.com/opencontainers/image\-spec](https://github.com/opencontainers/image-spec)Content\-addressed, layered container image format; accessed 2026Cited by:[§1](https://arxiv.org/html/2607.09689#S1.p2.1),[§2](https://arxiv.org/html/2607.09689#S2.SS0.SSS0.Px1.p1.4)\.
- \[17\]S\. Shillaker and P\. Pietzuch\(2020\)Faasm: lightweight isolation for efficient stateful serverless computing\.In2020 USENIX Annual Technical Conference \(ATC\),pp\. 419–433\.Cited by:[§4](https://arxiv.org/html/2607.09689#S4.p1.1)\.
- \[18\]K\. Singh, M\. Xie, and W\. E\. Strawderman\(2005\)Combining information from independent sources through confidence distributions\.The Annals of Statistics33\(1\),pp\. 159–183\.Cited by:[§1](https://arxiv.org/html/2607.09689#S1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.09689#S2.p1.2),[§3](https://arxiv.org/html/2607.09689#S3.p1.2)\.
- \[19\]L\. Tang, L\. Zhou, and P\. X\.\-K\. Song\(2020\)Distributed simultaneous inference in generalized linear models via confidence distribution\.Journal of the American Statistical Association115\(531\),pp\. 1357–1370\.Cited by:[§1](https://arxiv.org/html/2607.09689#S1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2607.09689#S2.p1.2),[§3](https://arxiv.org/html/2607.09689#S3.p2.10)\.
- \[20\]D\. Ustiugov, P\. Petrov, M\. Kogias, E\. Bugnion, and B\. Grot\(2021\)Benchmarking, analysis, and optimization of serverless function snapshots\.InProceedings of the 26th International Conference on Architectural Support for Programming Languages and Operating Systems \(ASPLOS\),pp\. 559–572\.Cited by:[§2](https://arxiv.org/html/2607.09689#S2.SS0.SSS0.Px1.p1.4),[§5](https://arxiv.org/html/2607.09689#S5.SSx2.SSS0.Px1.p1.4)\.
- \[21\]H\. White\(1982\)Maximum likelihood estimation of misspecified models\.Econometrica50\(1\),pp\. 1–25\.Cited by:[§2](https://arxiv.org/html/2607.09689#S2.p1.2),[§3](https://arxiv.org/html/2607.09689#S3.p1.3)\.
- \[22\]M\. Xie and K\. Singh\(2013\)Confidence distribution, the frequentist distribution estimator of a parameter: a review\.International Statistical Review81\(1\),pp\. 3–39\.Cited by:[§1](https://arxiv.org/html/2607.09689#S1.SS0.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2607.09689#S3.p1.2)\.
- \[23\]Y\. Zhang, J\. C\. Duchi, and M\. J\. Wainwright\(2013\)Communication\-efficient algorithms for statistical optimization\.Journal of Machine Learning Research14,pp\. 3321–3363\.Cited by:[§1](https://arxiv.org/html/2607.09689#S1.SS0.SSS0.Px2.p1.1)\.