PS: if you like the visuals on this post, I created a data catalog viewer for EgoSuite-Open100k built on @rerundotio. You can search, sort, and filter episodes. Download individual episodes and inspect the data.

Code: https://t.co/QhkgM0qJVD

jonstephens85/egosuite-explorer

Source: https://github.com/jonstephens85/egosuite-explorer

EgoSuite Interactive Explorer

A local, manifest-first browser and interactive Rerun viewer for the gated LightwheelAI/EgoDemo dataset. Browse episode metadata without downloading the dataset, fetch exact MCAP episodes on demand, and automatically convert Lightwheel’s custom hand, body, and semantic protobuf streams into native Rerun visualizations.

The application code is open source. EgoDemo data is not bundled or redistributed; every user must accept Lightwheel’s dataset terms and authenticate with their own Hugging Face account.

Current status

This repository contains an MVP supporting the four annotated EgoDemo variants:

• EgoStand
• EgoStand-body
• EgoProStandard
• EgoProStandard-body

Setup from source

Requirements:

• Python 3.11+
• Node.js 20+
• uv
• A recent Chromium or Firefox browser with WebAssembly support

Rerun does not need to be installed separately. uv sync installs the pinned Python rerun-sdk used by the MCAP converter, and npm ci installs the matching @rerun-io/web-viewer package used by the embedded browser viewer. Both are currently pinned to Rerun 0.36.3 to keep generated RRD files and the web viewer compatible. The standalone Rerun desktop application is optional.

Linux and macOS

uv sync
uv run hf auth login
cd frontend
npm ci
npm run build
cd ..
./launch_demo.sh

Windows

Use 64-bit Windows 10 or 11 and PowerShell. Clone the repository into a short path such as C:\src\egosuite-explorer to leave room for long episode names. From the repository directory, run:

powershell -ExecutionPolicy Bypass -File .\setup_windows.ps1
uv run hf auth login
.\launch_demo.cmd

The setup script creates a Python 3.12 environment, installs the pinned Rerun SDK, installs the frontend packages, and builds the embedded viewer. The execution-policy override applies only to that setup command; it does not change the machine-wide PowerShell policy. The .cmd launcher works from both PowerShell and Command Prompt without changing that policy.

Command Prompt users can launch the prepared application with:

launch_demo.cmd

To confirm the Python Rerun installation:

uv run python -c "import rerun; print(rerun.__version__)"

If you also want the optional native desktop viewer, the rerun-sdk dependency provides the rerun command inside the project environment:

uv run rerun --version

The server listens only on 127.0.0.1:8765 by default and opens the local app in your browser. Use ./launch_demo.sh --no-browser on Linux/macOS or .\launch_demo.cmd --no-browser on Windows to suppress that behavior.

If ./EgoDemo already exists, it is reused. Otherwise data is stored under ~/.cache/egosuite-explorer/EgoDemo. Override this location with:

EGOSUITE_EXPLORER_DATA=/path/to/EgoDemo ./launch_demo.sh

On Windows PowerShell:

$env:EGOSUITE_EXPLORER_DATA = "D:\EgoDemo"
.\launch_demo.cmd

Without an override, Windows stores downloaded data under %USERPROFILE%\.cache\egosuite-explorer\EgoDemo. MCAP and RRD data remain excluded from Git on every platform.

Workflow

1. Click Sync manifests. Only the four manifests/** trees are requested.
2. Search tasks, scenes, environments, and modality combinations.
3. Choose Download & prepare for an episode.
4. The exact manifest-referenced MCAP is downloaded and converted locally.
5. Choose Open interactive viewer to inspect synchronized video, native 3D hand/body pose, and semantic annotations.

The generated Rerun recording includes Lightwheel-inspired pose styling, pixel-space hand skeleton overlays on both head-camera streams, calibrated head-camera frustums, rolling head and foot motion trails, and action labels on the video. Wrist video remains synchronized but does not receive a projected overlay because the EgoDemo MCAPs do not provide wrist-camera extrinsics.

The Four-Variant Quick Start chooses one interaction-rich, moderate-duration episode per locally available manifest.

Development

Run the API and Vite development server separately:

.venv/bin/python -m egosuite_explorer.server --no-browser
cd frontend && npm run dev

The frontend proxies /api to http://127.0.0.1:8765.

On Windows, use .venv\Scripts\python.exe in place of .venv/bin/python, or use the platform-neutral command uv run egosuite-explorer --no-browser. GitHub Actions runs the Python tests and production frontend build on both Windows and Linux for every push and pull request.

Data and licensing

• Application source: Apache-2.0.
• Lightwheel joint ordering/topology is attributed in egodemo_mcap_to_rerun.py to the Apache-2.0 LightwheelAI/LW-Egosuite-DevKit.
• EgoDemo recordings remain governed by the dataset’s own access terms and license. Do not commit MCAP, RRD, extracted video, or Hugging Face tokens.