@yifanxu_ephai: 这是我正在做的 DSH-LayerFS。 https://github.com/Ephemeral-AI-Lab/layerfs/tree/main/dsh… 基于 LayerFS 的存算隔离架构和存储压缩能力,做了一个极简的 multi…

X AI KOLs Timeline 工具

摘要

DSH-LayerFS 是基于 LayerFS 存算隔离与存储压缩能力构建的极简 multi-agent harness,支持执行记录可追溯、独立 Workspace 运行 Bash 命令、冲突检测与递归嵌套 Fork。

这是我正在做的 DSH-LayerFS。 https://github.com/Ephemeral-AI-Lab/layerfs/tree/main/dsh… 基于 LayerFS 的存算隔离架构和存储压缩能力,做了一个极简的 multi-agent harness: - Agent 的每一步执行都有记录可追溯 - 每条 Bash 命令都在独立的 Workspace 中运行 - 内置冲突检测机制 - 支持递归嵌套 Fork(Recursive Nested Forking) 再配上这么可爱的鲨鱼娘视图—— 大家愿意再给 DSH 一次机会吗?🦈 @tianyi
查看原文
查看缓存全文

缓存时间: 2026/10/17 14:59

这是我正在做的 DSH-LayerFS。

https://github.com/Ephemeral-AI-Lab/layerfs/tree/main/dsh…

基于 LayerFS 的存算隔离架构和存储压缩能力,做了一个极简的 multi-agent harness:

  • Agent 的每一步执行都有记录可追溯
  • 每条 Bash 命令都在独立的 Workspace 中运行
  • 内置冲突检测机制
  • 支持递归嵌套 Fork(Recursive Nested Forking)

再配上这么可爱的鲨鱼娘视图——

大家愿意再给 DSH 一次机会吗?🦈

@tianyi


Ephemeral-AI-Lab/layerfs

Source: https://github.com/Ephemeral-AI-Lab/layerfs

LayerFS icon

LayerFS

Ephemeral Workspaces. Durable Shared History.

MIT license Developer preview WeChat QR invite Join Discord Follow @yifanxu_ephai on X

v0.1.6 Developer Preview: release and downloads, every measured selection, and compatibility boundary. Live Workspace state now lives in the sandbox and the Store format is unchanged; one construction worker, a bounded sandbox spool and every measured regression, waiver and declared allowance stay explicit.

Source notice (2026-10-10): the v0.1.6 implementation described below is not on main any more. Its source under root crates/ was retired once the replacement in core/ reached its closing examination; the release tag v0.1.6 still carries it, and the sections below describe that release. core/ is under development and is not a release.

🚀 What is LayerFS?

Ephemeral Workspaces. Durable Shared History.

LayerFS is a SQLite-backed, content-addressed time machine for agent Workspaces. Give every filesystem-affecting tool call its own ephemeral Workspace; when retained, that call becomes one immutable Commit—the base unit of the filesystem timeline. CAS, CDC, and COW store one shared base plus unique deltas instead of cloning full environments. From any Layer or eligible Commit, agents can fork zero-copy Branches, run parallel rollouts, discard failures, roll back by forking an earlier state, and promote a winning Branch with Add. The filesystem remains load-bearing for recursive multi-agent exploration without multiplying storage.

LayerFS 0.1.6 is a Developer Preview. It is intended for local evaluation, agent-runtime integration, and performance research—not production storage. It does not provide crash- or power-loss-durability guarantees. Keep an independent copy of important data.

🧱 LayerStack storage model

⚙️ Core storage mechanisms

CAS, CDC, and COW make LayerStack history storage-efficient by reusing unchanged objects, file regions, and filesystem structure.

The governing invariant is simple: every filesystem state is complete logically, but incremental physically. A new state should cost what changed, not the size of the Workspace it exposes. Read the CAS + CDC + COW foundations for the design rationale and step-by-step storage model.

🔐 01 · Identity📍 02 · Byte locality🌳 03 · Structural locality
Content-addressed storage — Names immutable objects from their canonical bytes, verifies reads, and reuses exact duplicates across files, LayerStacks, and agents.Content-defined chunking — Keeps chunk boundaries stable around localized edits, so changing a small region does not require storing an entirely new large file.Copy-on-write — Rebuilds only the changed file and directory path when publishing a new Commit, preserving unchanged subtrees; a Branch head can then be added as the next Layer.

🗃️ Check out an ephemeral filesystem from any layer

A LayerStack records complete filesystem checkpoints. Agent A can start from L1 while Agent B independently starts from L3. Each gets a private place to work while the selected history remains shared.

This is the conceptual checkout view; the public lifecycle is Layer or Commit → Branch → Workspace.

Top-down LayerStack with Branches, Commits, and parallel agent Workspaces

One LayerStack, many zero-copy Branches, and ephemeral COW Workspaces sharing immutable history.

📦 Measured deduplication

The primary storage signal is semantic content growth: the canonical bytes needed by a new state. SQLite page allocation is reported separately because existing database pages can absorb new objects without changing the logical content represented by the Store.

WorkloadLayerFS semantic growth
Sixteen deterministic edits0.2250 MiB
Prepend 10 bytes to 32 MiB0.0256 MiB

These measurements come from the final public-SDK, real-FUSE campaign. See the full benchmark report for physical allocation, equations, source identity, and raw evidence.


🧭 System boundaries

🧩 LayerFS components

The storage engine, SDK, CLI, and filesystem projection are implemented in the 0.1.6 Developer Preview. They remain separate public boundaries so callers do not depend on private CAS handles or storage formats.

StatusComponentRole
Implemented coreStorageOwns identities, canonical objects, CDC, file manifests, structural COW, immutable CAS admission, lifecycle coordination, and verified reads.
Implemented previewSDK and CLIExpose filesystem, Workspace, LayerStack, Branch, Commit, publication, query, monitoring, and container-lifecycle operations through the public surfaces.
Implemented previewFilesystem projectionExposes a Workspace through host materialization or real container FUSE and captures bounded filesystem effects while delegating identity, CDC, COW, and admission to storage.
Implemented previewContainer runtimeCreates and controls resource-bounded Linux FUSE containers for Workspaces without placing the durable Store inside the container.

The one-Store boundary is intentional: a Client binds one SQLite Store, one Monitor, and one Workspace manager. A second Store uses a separate Client.

Each workspace exec starts a fresh process. commit publishes the Workspace state to its Branch. end removes the ephemeral projection and never commits implicitly.

🛠️ Quickstart

Releases are built from source. You need macOS or Linux and Rust 1.85 or newer. Docker, /dev/fuse, and CAP_SYS_ADMIN are needed only for managed container-FUSE workspaces. No packages are published to crates.io for this release.

From the repository root, at the v0.1.6 release tag:

git clone https://github.com/Ephemeral-AI-Lab/layerfs.git
cd layerfs
git checkout v0.1.6

cargo build --release -p layerfs-cli
export LAYERFS_BIN="$PWD/target/release/layerfs"
export LAYERFS_CONTEXT="$PWD/.layerfs/context"
mkdir -p "$PWD/.layerfs"

"$LAYERFS_BIN" db create "$PWD/.layerfs/store.sqlite"
"$LAYERFS_BIN" context use --store "$PWD/.layerfs/store.sqlite"
"$LAYERFS_BIN" layerstack init --name demo --empty
"$LAYERFS_BIN" query layerstacks

This creates a SQLite Store and an empty LayerStack with a genesis Layer. Continue with the complete quickstart for Branch creation, Workspace execution, commits, cleanup, directory imports, managed containers, and real FUSE.

When importing an existing directory, keep the Store file outside the directory being imported or projected:

mkdir -p "$PWD/import-root"
printf 'hello\n' > "$PWD/import-root/hello.txt"
"$LAYERFS_BIN" layerstack init --name imported "$PWD/import-root"

🗂️ Repository layout

core/                          replacement implementation (in development)
├── crates/                    content, storage, history, persistence, project,
│                              telemetry, overlay, workspace, fuse, daemon,
│                              bridge, sandbox and the SDK
├── benchmark/                 core harnesses, registrations and proof tools
└── docs/                      architecture, design contracts and run records
benchmark/                     v0.1.6 harness, historical; not runnable on main
docs/versioned/0.1.6           v0.1.6 manual
docs/versioned/0.1.2           previous versioned product manual
release-notes/                 release records, evidence and limitations
tools/production_loc.py        pinned production source-line counter

The v0.1.6 crates, tools/layerfs-eval and containers/layerfs-fuse are in the v0.1.6 release tag.

⚠️ Current limitations

LayerFS is suitable for evaluation and integration work, but the preview boundary matters:

  • 0.1.6 operates against one Store per Client; there is no cross-host synchronization;
  • live-process transaction visibility does not imply crash or power-loss durability, and Workspace backing is not fsynced;
  • a v0.1.6 sandbox owner and a v0.1.5 host do not share a live Workspace; match SDK, CLI, daemon and runtime components across a session;
  • the sandbox owns live mutable state and a private payload backing for its mount; the host learns it only when a Commit transfers it, and that transfer pays a storage read (bounded sandbox spool residency ≤ 2.6 MiB) instead of page-cache service. Kernel-dirty shared mmap is still not captured by a Commit;
  • canonical construction runs one worker (namespace init is the only exception), so six dedup/CDC construction cells are 1.50–1.65× slower than v0.1.5 and are owner-accepted, and the cold namespace-100000 Init target is owner-waived (4.986 s against 2.7 s). Read the release record, acceptance and waivers;
  • explicit compaction stays removed; compacted Stores from earlier builds remain readable through the retained authenticated read path, and ordinary storage is the only supported route;
  • the measured v0.1.6 set is 36 selections: 28 performance receipts and 36 independent verifications, three performance rows above the 15 s family target under declared allowances, one verification row under a declared 30 s ceiling and two verify-only rows under frozen 120 s / 300 s watchdogs. Endurance is not qualified;
  • the retained SQLite Store still misses the 600 MB primary-control goal; storage-efficiency tradeoffs are accepted for v0.1.4 in #18, with optimization design still to be discussed;
  • the detached CLI context owner does not forward an interactive PTY; and
  • CLI JSON output is a preview text envelope, not a stable machine API.

Read the full limitations before using LayerFS with important data.

📚 Documentation

Start with the documentation index, or jump directly to a focused guide:

GoalGuide
Learn the conceptsCore concepts
Run the CLI and SDKQuickstart
Find a CLI commandCLI reference
Integrate with RustRust SDK reference
Configure container FUSEContainer runtime
Understand storageStorage format
Review v0.1.6 changesNumbered changelog
Review released evidence0.1.6 release record
Review completed 0.1.2 work0.1.2 checklist
Contribute changes0.1.x development guide

The first-principles learning site is educational material and may describe future work. The versioned repository manual defines the current product contract.

🗺️ Roadmap

StageFocusStatus
0.1.0 Developer PreviewOne SQLite Store, immutable LayerStack history, Branches, Workspaces, public SDK/CLI, host materialization, container FUSE, monitoring, and benchmark evidence.Released as source under v0.1.0.
0.1.1Measure and harden existing-directory initialization through localized Commit, with focused FUSE and Docker proof.Released as source under v0.1.1; see the release record.
0.1.2Three SDK-only 1/10/100/500 MiB edit families, namespace and Store refresh, and universal regular-file editing.Released as source under v0.1.2; see the release record.
0.1.3Shared live Workspaces, optimized Commit and reads, and the 198-case/226-proof checkpoint.Released as source under v0.1.3; see the release record.
0.1.4Packed storage and bounded Init/Commit improvements; measured tradeoffs accepted.Closed; release qualified; release record and limitations.
0.1.5Ordinary schema-10 storage, bounded pending edits, coalesced pack rows and the removal of explicit compaction; full campaign qualification with published regressions.Released as source under v0.1.5; see the release record, acceptance and limitations.
0.1.6Sandbox-local Workspace state: the sandbox owner holds the live namespace and file data, the host keeps canonical construction and publication, and Commit transfers mutable state with no pause fence and one construction worker.Released as source under v0.1.6; see the release record, acceptance and limitations.
0.2.0Establish a portable projection foundation, including capability-detected reflink/clonefile paths and a future OverlayFS projection.Planned; requires a new compatibility contract.
LaterAdd platform/runtime expansion and verified Store export, import, and synchronization.Research; no cross-host synchronization is part of 0.1.0.

See the roadmap checklist and roadmap architecture notes for acceptance gates, ownership boundaries, and sequencing. Use the Rust SDK reference to integrate the current public SDK.

🤝 Contributing

Bug reports, reproducible performance evidence, documentation corrections, and focused pull requests are welcome. Before opening a change, review the 0.1.x development guide and run the repository’s relevant verification commands.

💬 Community

WeChat QR code for the Ephemeral AI Lab group

📄 License

LayerFS is licensed under the MIT License.

相似文章

LayerFS:面向 AI 编程智能体的文件系统

Hacker News Top

LayerFS 为 AI 编码代理提供专用文件系统:每个代理挂载一个临时可写工作区并提交到共享的持久历史,实现大规模并行开发、零成本分叉、去重压缩存储以及即时依赖挂载。

@yifanxu_ephai: 在LayerFS跑一个RUST编写的DB Engine任务: 将 2.50 TB 的快照数据、30 个分支,压缩到仅 4.66 GB, 光是 main 分支就有 8.23 GB。 先吹个牛逼,我怕明天发布翻车,就吹不了了。 半年前我和VC…

X AI KOLs Timeline

Ephemeral AI Lab 宣布 LayerFS 1.0.0 正式发布,这是一款为 AI 智能体设计的文件系统基座,其 Rust 编写的 DB Engine 将 2.50 TB 快照数据和 30 个分支压缩到仅 4.66 GB,主打短暂工作区与共享持久历史。

@d0r1anX:我正在开发 BoxLite:为 AI 代理打造的计算机。在隔离的 microVM 中运行代码,在会话之间保留文件,并将你的代理所构建的内容部署上线……

X AI KOLs Timeline

BoxLite 是一个面向 AI 编码代理的开源运行时与托管云平台,可在隔离的 microVM 中运行代码、持久化文件,并为部署的应用提供公开 URL。它提供 Python SDK 与 104 个 REST 端点,支持 Claude Code 等代理在安全沙箱中执行任务。