If a process’s fork() is the operating system’s cloning spell, then forkd brings this magic into AI agents’ micro virtual machines. Built on Firecracker and KVM, it enables a pre-warmed parent VM to fork 100 child VMs in about 100ms, with BRANCH operations on live sandboxes taking as little as 56ms. The core mechanism is copy-on-write snapshots: child VMs share the parent VM’s memory image, inheriting loaded models and caches while maintaining KVM-level hardware isolation. Version 0.5 adds differential snapshot chaining, allowing pip-installed layers to be stacked for storage reuse.

Targeted primarily at agent parallel exploration scenarios—for example, a code interpreter forking a sandbox for each conversation turn—it remains in alpha, with production-grade features not yet complete. Review the documentation before diving in.

:https://github.com/deeplethe/forkd

deeplethe/forkd

Source: https://github.com/deeplethe/forkd

Fork 100 microVMs in 101 ms. BRANCH a live VM in 56 ms (v0.4 live mode).

Unedited: forkd pull a portable snapshot, then fork 100 microVMs — 100/100 alive, ~200 ms wall-clock. asciicast. (First pull also fetches the rootfs sidecar once; cached here.)

A microVM sandbox runtime for AI agent fan-out. Children fork from a warmed parent snapshot, inheriting its address space copy-on-write instead of cold-booting their own kernel.

forkd is built on Firecracker. The parent VM boots once, imports your runtime (Python + your dependencies, a JIT-warmed JVM, an already-loaded ML model) and is paused to disk. Each child is a separate Firecracker process that mmaps the parent’s memory image with MAP_PRIVATE; the kernel implements copy-on-write at the page level, so children share the parent’s resident memory until they diverge. The result is two properties at once: per-child KVM isolation, and a spawn cost that’s closer to fork(2) than to a cold-boot VM.

forkd also supports BRANCH: pause a running sandbox, snapshot its in-flight state, and resume — all in ~150 ms — so an agent can fork mid-thought, not only at warm-up.

v0.3.4 fixed a slow-path regression where repeated BRANCHes on the same parent ballooned from 150 ms to 2.7 s (#146); the chain now stays flat (17.6× faster on the 6th consecutive BRANCH).

v0.4 live BRANCH collapses the source-pause window from ~200 ms (Diff) to 56 ms p50 / 64 ms p90 on a 1.5 GiB source — measured on a real BRANCH workload, bench/live-fork-pause-window/RESULTS-v0.4.md. 3.6× faster pause vs v0.3 Diff at p50, and the gap widens on slower storage because Live’s pause is disk-independent (memory copy runs after resume, not during). With wait: false the caller returns in ~70 ms while the background copy completes asynchronously — a 200× RT improvement for fire-and-forget BRANCH from agent code.

Pass --live / --no-wait on the CLI, mode: "live" / wait: false on REST, or the same on the Python / TypeScript / MCP SDKs.

from forkd import Controller

c = Controller()
# Source must boot with live_fork=True (memfd-backed RAM, the prereq
# for UFFD_WP to see writes from the running parent).
parent = c.spawn_sandboxes("pyagent", n=1, live_fork=True)[0]
# ... drive parent ...

# then BRANCH live + fire-and-forget:
branch = c.branch_sandbox(parent["id"], mode="live", wait=False)
# Returns after ~10 ms with status="writing"; poll list_snapshots
# until status="ready" for the background copy to finish.

# CLI: spawn live-fork-capable children locally, then live-BRANCH the
# daemon-tracked one. The two paths don't compose yet — daemon-side
# spawn from the CLI is the next gap (see issue #209 for status).
sudo -E forkd fork --tag pyagent -n 1 --per-child-netns --live-fork
sudo -E forkd snapshot --from-sandbox --live --no-wait

Requires Linux ≥ 5.7, vm.unprivileged_userfaultfd=1 (or CAP_SYS_PTRACE), and the vendored Firecracker fork from deeplethe/firecracker:forkd-v0.4-mem-backend-shared-v1.12 (https://github.com/deeplethe/firecracker/tree/forkd-v0.4-mem-backend-shared-v1.12) — forkd doctor probes both.

Full design: DESIGN-v0.4.md. Empirical PoC data: experiments/v0.4-*-poc/. Tracking issue #101.

v0.5: stacking diff snapshots into a chain

Once an agent starts caching pip install numpy, pip install pandas, pip install scikit-learn as separate snapshots, you want them stacked — not three copies of the same 1.5 GiB base.

v0.5 ships diff-snapshot chains: each layer records a parent_tag + content-hash edge to the layer below; the daemon walks the chain at spawn time and assembles the memory image in one pass.

# Build a 3-layer chain off a python:3.12-slim base
forkd snapshot-diff --from py-base --tag py-numpy --exec "pip install numpy==2.0.2"
forkd snapshot-diff --from py-numpy --tag py-pandas --exec "pip install pandas==2.2.3"

# Spawn from the chain head — daemon walks edges, verifies parent
# content hashes, assembles memory transparently. Caller sees one
# POST /v1/sandboxes round-trip.
forkd fork --tag py-pandas -n 1

# Inspect a chain before deciding to delete or compact
forkd snapshot-info py-numpy
# chain depth: 1
# parent_tag: py-base
# ancestors: py-base
# dependents: py-pandas (would be orphaned by `rmi`)

# rmi refuses to orphan a chain parent — explicit cascade or force
forkd rmi py-numpy              # HTTP 409, names the dependent
forkd rmi py-numpy --cascade    # delete subtree
forkd rmi py-numpy --force      # orphan children

# Flatten a deep chain when the per-link tax bites
forkd snapshot-compact --from py-pandas --to py-pandas-flat

# Ship the whole chain in one tarball (bundles every ancestor)
forkd pack --tag py-pandas --out py-pandas-chain.tar.zst
forkd unpack py-pandas-chain.tar.zst  # restores all 3 link dirs

Phase 5 bench numbers (bench/chain-spawn/RESULTS-v0.5.md) on a 512 MiB base, ext4, i7-12700:

Per-link tax tracks SHA-256 of the base (~460 ms for 512 MiB at 1.1 GiB/s on this CPU) — the mmap-once-then-incremental verify optimization is queued as v0.6.

Correctness: 90/90 (100%) probe passes across L1/L2/L3 plus the flat-equivalent — the design’s vmstate-drift question is empirically closed.

Tracking issue #216 lists every phase + PR. Design doc: DESIGN-v0.5-diff-snapshot-chains.md.

Demo: branch a thinking agent

A 24-second walkthrough of the LangGraph branch-and-fan-out demo — source agent runs a ReAct loop, gets BRANCHed mid-thought, three grandchildren each receive a different steering hint, all three produce divergent itineraries while inheriting the same prior reasoning state.

Headline divergence: the source (no hint) picks Nishiki Market for Day 1; all three hinted children independently substitute Arashiyama Bamboo Grove. The cost-focused child also adds “may be pricey” annotations the others don’t.

The model wasn’t told to swap places — each hint perturbed the next LLM call, the rest of the prior reasoning came along unchanged.

Full mechanism + numbers + raw transcripts in recipes/langgraph-react/ and recipes/langgraph-react/DEMO.md.

And: filesystem state, not just reasoning

For the “but couldn’t you just call the LLM 3 times in parallel?” objection, see recipes/coding-agent-fork/ — a 50 MiB binary blob travels byte-identically across all 4 sandboxes through a single BRANCH. Three grandchildren each apply a different fix to a buggy Python package; their __pycache__/ and edits stay isolated, but the 50 MiB inheritance is shared. Bytes can’t fit in a prompt.

3.3 s pause for the BRANCH operation.

Properties

• Hardware isolation. Each child is its own Firecracker microVM backed by KVM. Escape requires a hypervisor or kernel vulnerability, not a runc regression.
• Warmed runtimes inherit for free. Imports, JIT compilation, model weights, prefetched caches — anything the parent did is already resident in the child.
• Real Linux per child. Multi-vCPU, full TCP networking, apt install, outbound HTTPS. Unlike function-level snapshot runtimes that trade single-vCPU + serial-I/O for raw spawn speed, forkd children can run real Python servers, model inference, or any workload that needs a full kernel.
• Multi-tenant by construction. Per-child network namespace, per-child cgroup v2 memory limit, independent /dev/urandom re-seeded by vmgenid (Linux 5.20+).
• Built for agent fan-out. AI agent workloads that fan out into many short-lived sandboxes — code-interpreter, tool-use, evaluation rollouts — are the design point. The warmed parent collapses the per-request import numpy / import torch cost across the entire cohort.
• Operable. Daemon process owning state, REST API on Unix or TCP, Prometheus /metrics, append-only JSON audit log, systemd unit.
• Open source. Apache 2.0, no vendor SDK.

Benchmarks

Same Linux host (Ubuntu 24.04, Linux 6.14, 20 vCPU, 30 GiB, KVM).
Workload: spawn 100 sandboxes that each run import numpy; numpy.zeros(5).tolist().

Spawn time at N=100
Host memory per sandbox

¹ CubeSandbox: 1.06 s wall-clock is the fast-path N=100 figure on this host (1056 ± 14 ms over five runs, 100 % success every run), measured with a bench script that pre-warms Python’s ThreadPoolExecutor to keep client-side lazy-init out of the timing. An earlier slow-path measurement on the same host returned 20.3 s with 77/100 success — that template had a 2 GiB writable-layer size that didn’t match the default 1 GiB pool, so every sandbox went through a live mkfs.ext4 + reflink-copy; after the upstream maintainer at #235 (https://github.com/TencentCloud/CubeSandbox/issues/235) clarified the distinction, we added 2Gi to pool_default_format_size_list and re-ran. The host runs cube v0.2.0, which carries a ~50 ms latency regression that PR #234 (https://github.com/TencentCloud/CubeSandbox/pull/234) fixes in v0.2.1; the value above is the v0.2.0 baseline. Cube advertises <60 ms single-instance cold-start on a 96 vCPU host; we did not retest that shape. See bench/CUBESANDBOX.md for the full methodology, both rows, and the cmdTimeout race we filed two PRs upstream against (#236 (https://github.com/TencentCloud/CubeSandbox/pull/236) / #237 (https://github.com/TencentCloud/CubeSandbox/pull/237)).

² BoxLite is optimised for one long-lived stateful Box per workload, not 100 concurrent fresh microVMs. The cold fan-out is included for direct comparability. See bench/BOXLITE.md.

³ OpenSandbox is an abstraction layer over Docker / K8s / gVisor / Kata / Firecracker; the number is for its default Docker runtime. See bench/OPENSANDBOX.md.

Reproduce: bench/bench-spawn-100.sh then bench/generate_charts.py.

For one sandbox doing the same numpy expression two ways:

How it works

flowchart TB
    %% ─── parent ───────────────────────────────────────────────
    subgraph PARENT["Parent VM (booted once, warmed)"]
        direction TB
        runtime["PID 1\nPython + numpy + your deps\nimported into RAM"]
    end
    PARENT -- "pause + snapshot" --> SNAP["Snapshot on disk\nmemory.bin (CoW source)\nvmstate (vCPU + devices)"]

    %% ─── controller ───────────────────────────────────────────
    CLIENT["Client (CLI / Python SDK)"] -- "POST /v1/sandboxes\nn=100" --> CTL["forkd-controller\nREST · auth · audit · /metrics"]
    CTL -- "restore_many_with(...)" --> SNAP

    %% ─── children ─────────────────────────────────────────────
    subgraph CHILDREN["100 Child Firecracker processes (kernel CoW per page)"]
        direction LR
        subgraph NS1["netns forkd-child-1"]
            C1["Child 1\nmmap MAP_PRIVATE\ncgroup memory.max"]
        end
        subgraph NS2["netns forkd-child-2"]
            C2["Child 2\nmmap MAP_PRIVATE\ncgroup memory.max"]
        end
        subgraph NSN["netns forkd-child-100"]
            CN["Child 100\nmmap MAP_PRIVATE\ncgroup memory.max"]
        end
    end
    SNAP -. "shared file\n(read-mostly)" .-> C1
    SNAP -. "shared file" .-> C2
    SNAP -. "shared file" .-> CN

    %% ─── network ──────────────────────────────────────────────
    C1 -- "veth" --> BR["host bridge\nforkd-br0\nMASQUERADE"]
    C2 -- "veth" --> BR
    CN -- "veth" --> BR
    BR --> UPLINK(("uplink → internet"))

    %% styling
    classDef parent fill:#e8f3ec,stroke:#4c956c,color:#1f2933;
    classDef snap fill:#fff3df,stroke:#d4a259,color:#1f2933;
    classDef ctl fill:#e6efff,stroke:#5b7dba,color:#1f2933;
    classDef child fill:#ffffff,stroke:#52606d,color:#1f2933;
    classDef net fill:#f1f3f5,stroke:#8d99ae,color:#1f2933;

    class PARENT,runtime parent;
    class SNAP snap;
    class CTL,CLIENT ctl;
    class NS1,NS2,NSN,C1,C2,CN child;
    class BR,UPLINK net;

See DESIGN.md for the full design and the open problems the architecture leaves on the table.

How forkd compares

The sandbox-runtime space has a wide spread of designs. The table below summarises positioning of forkd against the most-cited open-source projects. Numbers in quotes are as advertised by the upstream project unless they match a row in our benchmark chart above. forkd does not measure other projects on workloads they were not designed for.

¹ Wall-clock at N=100 concurrent on this bare-metal host (systemd-detect-virt: none, i7-12700, 20 vCPU, no nested virt). This is the fast-path number — pool_default_format_size_list was extended to include the template’s writable-layer size, so each sandbox reuses a pre-formatted pool entry rather than going through a live mkfs.ext4 + reflink-copy. 1056 ± 14 ms over five runs, 100 % success every run, measured with a bench script that pre-warms Python’s ThreadPoolExecutor to keep client-side lazy-init out of the timing. Host runs cube v0.2.0, which carries a ~50 ms latency regression that PR #234 (https://github.com/TencentCloud/CubeSandbox/pull/234) fixes in v0.2.1 — the figure above is the v0.2.0 baseline. An earlier slow-path measurement on the same host (writable-layer size that didn’t match the default pool) returned 20.3 s with 77/100 success — that mismatch was on our side and the maintainer corrected it at #235 (https://github.com/TencentCloud/CubeSandbox/issues/235). Cube advertises <60 ms single-instance cold-start on a 96 vCPU host; we did not retest that shape. See bench/CUBESANDBOX.md for the full methodology, both rows, and the cmdTimeout race we filed two PRs upstream against (#236 (https://github.com/TencentCloud/CubeSandbox/pull/236) / #237 (https://github.com/TencentCloud/CubeSandbox/pull/237)).

² BoxLite is optimised for one long-lived stateful Box per workload, not 100 concurrent fresh microVMs. The cold fan-out is included for direct comparability. See bench/BOXLITE.md.

³ OpenSandbox is an abstraction layer over Docker / K8s / gVisor / Kata / Firecracker; the number is for its default Docker runtime. See bench/OPENSANDBOX.md.