General-Instinct/InstinctFlash

Source: https://github.com/General-Instinct/InstinctFlash

A high-performance serving framework for robotics models.

What’s new 🔥

• [2026/09/17] RTX 5090 support. Deploy on your workstation with the same Runtime API used on Jetson Thor. Setup · Reproduce.
• [2026/09/16] RTX 4090 support. Desktop inference and WebSocket serving with dedicated installation profiles. Setup · Reproduce.
• [2026/09/15] Full-source release. Eight robotics model families, acceleration kernels, and Python / WebSocket serving through one Runtime. Get started.
• [2026/09/15] Jetson Thor benchmarks. Up to 33.78× speedup with LingBot-VA @2V/4A, using FP8 and fewer sampling steps. Results · Reproduce.

Results

Prediction p50 on Jetson Thor (ms), measured September 15, 2026.

We’ve seen up to 33.78× speedup with no observed loss in task performance in our real-robot tests.

VA measures early continuations; each row compares the same schedule. The 33.78× headline includes 25V/50A → 2V/4A. FP8 and sampling changes are optional.

Protocol and raw results · Native VA 2V/4A · Reproduction commands

Install

git clone https://github.com/General-Instinct/InstinctFlash && cd InstinctFlash
python3 -m venv .venv-core
source .venv-core/bin/activate
python -m pip install . uv==0.12.5

The Python 3.10+ core inspects checkpoints and plans without PyTorch or a GPU. Inference uses a separate, pinned environment for each model family. For RTX 4090:

python3 scripts/bootstrap_vendor.py install pi05 --target rtx4090 \
  --python python3.12 --root ~/ifl-pi05-4090 --ptxas /usr/local/cuda/bin/ptxas
source ~/ifl-pi05-4090/activate.sh

Use va, vla4, vla2, pi05, groot, edge, nano or dreamzero. Edge and Nano use Python 3.13; the other families use Python 3.12. The bootstrap installs the upstream source, compatibility patches, core and adapter. Model weights are downloaded separately. See RTX 5090 setup, RTX 4090 setup or Jetson Thor setup, which selects --target jetson_thor and uses the Thor CUDA backend build.

Load a model

Your fine-tuned checkpoint — the expected case. Point serve at the training output; it detects the family, writes the small instinctflash.json declaration from what the checkpoint itself proves, and starts serving. One command:

instinctflash serve /path/to/your/checkpoint

Anything the checkpoint cannot prove is asked for explicitly, never guessed. Once the declaration exists (serve writes it on first run), the same directory also loads in Python:

from instinctflash import Runtime

runtime = Runtime.from_pretrained("/path/to/your/checkpoint")

A stock release — use its Hub id after installing the family’s environment:

runtime = Runtime.from_pretrained("robbyant/lingbot-va-posttrain-robotwin")

Fine-tunes reuse their family’s adapter; quality is evaluated per checkpoint.

The same Runtime defaults to precision="native" with a BITEXACT transformation ceiling. Use tier_ceiling="numeric" to allow numerical changes, or precision="fp8" (CLI: --fp8) to explicitly enable FP8. Step schedules are selected separately. See precision policy and FP8 support and validation.

DreamZero’s opt-in dynamic step cache requires tier_ceiling="behavioral" with either precision. See the Thor measurements.

Get actions

In process — this is the whole Python API:

with runtime.episode(prompt="put the bottle in the dustbin") as episode:
    while not done:
        result = episode.predict(observation)
        action = result["action"]

observation is a dict in the model’s own format; result["action"] contains its action array. For LingBot-VA, pass executed_action=... when the controller changes a predicted action chunk, so the next prediction uses the actions actually executed.

Over the network — the serve command above hosts the same runtime behind the msgpack-over-websocket wire protocol the pi0/openpi ecosystem already speaks, so existing robot-side clients connect unchanged (pip install openpi-client):

from openpi_client.websocket_client_policy import WebsocketClientPolicy

client = WebsocketClientPolicy("my-server", 8000)
result = client.infer(observation)
action = result["action"]

The prompt rides in the observation; a changed prompt starts a new episode, and a client can say it explicitly with {"reset": True, ...}. Four flags cover the rest:

• --serve.dry_run — preflight only: device, declaration, plan. No weights, no GPU.
• --serve.smoke — load, produce one action, exit.
• --serve.seed — seed native execution for paired comparisons; FP8 serving rejects this option.
• --serve.viz — stream observations, actions and latency to a Rerun viewer.

The second verb, instinctflash validate <dir>, checks a checkpoint is publishable; given --validate.teacher_outcomes/.student_outcomes/.margin it also certifies non-inferiority and stamps the certificate into the package.

Benchmark acceleration and quantization

After the vendor and auxiliary-asset preparation, reproduce paired eager/default/selected Runtime measurements with the included inputs and fixed checkpoint revision. Thor also requires its native backend. Keep the model and asset environments activated. For RTX 4090:

python -I -m benchmarks.regression.reproduce prepare --target rtx4090 \
  --model pi05 --mode fp8 --output pi05-inputs
python -I -m benchmarks.regression.reproduce run --prepared pi05-inputs --output pi05-results
python -I -m benchmarks.regression.serve_smoke --prepared pi05-inputs --output pi05-serving

run writes checked JSON/CSV reports and full action arrays. serve_smoke tests the actual CLI and WebSocket pipeline across two episodes. Use --mode native for default precision; FP8, numerical compilation and changed schedules are explicit selections. Reproduction guide. For additional framework comparisons, use the pinned comparison recipes.

Compare original and optimized models with instinctflash eval. Reports separate latency, action agreement and simulator task success.

instinctflash eval adapters
instinctflash eval coverage --run /path/to/run
instinctflash eval --registry plan.registry.json report --run /path/to/run

See the evaluation guide to create and run paired LIBERO / RoboTwin experiments, or benchmark details for acceleration and quantization protocols. Results: simulator screening and repeatability, checkpoints and edge latency. The expanded V2 evaluation binds latency and quality evidence to execution profiles and checks explicit control budgets. The native qualification workflow adds fresh-start admission, retained failures and checkpoint-specific evidence for each device. LingBot-VA Hub IDs retain native step counts; 2V/4A requires an explicit nfe selection. The September 9 Thor comparison separates native acceleration, FP8 Runtime gains and paired task outcomes; historical engine controls isolate additional implementation effects.

Shared BF16 fusion provides an opt-in NUMERIC path, with per-model compatibility and paired Thor regression results. Shared tensor caching and prefill separation extend native Cosmos optimization to Edge and Nano; exact caching and NUMERIC compilation remain separate options.

Framework overview

InstinctFlash keeps model declarations, optimization planning, runtime execution, and evidence in one inspectable path, whether it is called from Python or the command line.

Architecture

A checkpoint carries a short declaration of what it is. The runtime reads the declaration, decides which optimizations are provably valid for those weights, applies them, and shows its work:

checkpoint ─▶ adapter          ─▶ planner            ─▶ engine passes        ─▶ actions
              declares what        decides what          apply and measure
              the model is         is valid (no GPU,     each optimization
                                   no weights needed)

Optimization is organized in six layers, by what each one changes:

Layer 1 changes the weights and produces a checkpoint; it lives in the companion repos. Layers 2–6 change how the weights execute and produce a plan; they are the runtime in this repo. The layers are not a priority order — the runtime measures where the time actually goes and starts there.

Add a model

To add your own model family, declare an instinctflash.adapters entry point and pip install your package — see examples/external_plugin/.

Roadmap

• Few-step distillation, when needed — only after native optimizations miss a declared edge control budget; compare each student with its teacher and the matched untrained schedule using paired closed-loop evaluation.
• LingBot-VA on the edge engine — native and FP8 serving on Jetson Thor, with paired inference and WebSocket checks for full and 2V/4A schedules.
• Attention upgrades — a faster NUMERIC-tier attention arm beside the BITEXACT default for pi05-class models; hybrid and linear attention for long-context world models.
• Cosmos3 and DreamZero on Thor — pinned installation, paired inference and installed CLI/WebSocket checks; task quality remains a separate evaluation.
• Device-specific serving defaults — measure each family and operating point; select a verified path within the caller’s precision constraints. LingBot-VLA-V2 native Thor capture and LingBot-VA saturation profiling are complete; selective VA action capture showed no speedup and stays experimental. Results and evidence. Execution-bound budget selection is available; the expanded V2 H100 evaluation remains SCREEN.