ZipDepth [ECCV 2026] Official implementation of “ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device”. A compact 6.1M-parameter network for zero-shot monocular depth estimation, running in real time from server GPUs to mobile phones via knowledge distillation from foundation models. https://github.com/fabiotosi92/ZipDepth… ZipDepth is a lightweight zero-shot monocular depth estimation model that achieves the best accuracy–efficiency trade-off among lightweight methods, approaching transformer-based foundation models at a fraction of their cost. It combines reparameterizable convolutions (RepVGG), efficient channel and spatial attention (strip pooling and global context blocks), and a compact FPN decoder — all designed to run fast on any device, from edge hardware to server GPUs.

fabiotosi92/ZipDepth

Source: https://github.com/fabiotosi92/ZipDepth

📢 News

• [Jul 2026] — 🚀 Code and pretrained model released.
• [Jun 2026] — 🎉 ZipDepth accepted at ECCV 2026.

📱 See ZipDepth running live on an iPhone on the project page.

ZipDepth is a lightweight zero-shot monocular depth estimation model that achieves the best accuracy–efficiency trade-off among lightweight methods, approaching transformer-based foundation models at a fraction of their cost. It combines reparameterizable convolutions (RepVGG), efficient channel and spatial attention (strip pooling and global context blocks), and a compact FPN decoder — all designed to run fast on any device, from edge hardware to server GPUs.

🖼️ Qualitative Results

ZipDepth generalizes across diverse domains without any fine-tuning — nighttime driving, outdoor objects, close-up textures, and synthetic content.

RGB	Depth	RGB	Depth

📊 Quantitative Results

ZipDepth achieves state-of-the-art accuracy among lightweight embedded models on NYUv2, KITTI, ETH3D, ScanNet, and DIODE, while being significantly more efficient than large pretrained models.

🏗️ Architecture

The encoder is organized in four hierarchical stages. Stages 1–2 use RepVGG reparameterizable blocks (3×3 + 1×1 + identity branches fused into a single 3×3 at inference) augmented with Strip Pooling Attention for horizontal/vertical context. Stage 3 adds Squeeze-and-Excitation channel attention and a Global Context Block. Stage 4 deepens the representation with additional RepVGG blocks.

The neck combines SPPF multi-scale pooling with a Cross-Scale Fusion module. The decoder is a lightweight FPN with a Convex Upsampling head for sub-pixel-accurate depth maps.

🔧 Installation

Requirements: Python ≥ 3.9, PyTorch ≥ 2.4, CUDA (recommended)

git clone https://github.com/fabiotosi92/ZipDepth
cd ZipDepth

# Create and activate a virtual environment (recommended)
python -m venv venv
source venv/bin/activate

pip install -r requirements.txt
pip install -e .

PyTorch / CUDA: requirements.txt installs the default PyTorch wheel, which on Linux ships with a bundled CUDA runtime. If you need a specific CUDA version (or a CPU-only build), install PyTorch first from the official selector, then run pip install -r requirements.txt.

# Faster JPEG decoding (recommended)
pip install PyTurboJPEG

# FLOPs measurement in benchmark
pip install fvcore thop

# ONNX graph simplification during export
pip install onnx onnxsim

🗂️ Checkpoints

Pretrained checkpoints are included in the repository under checkpoints/:

Both variants share identical encoder and decoder weights. The only difference is the final upsampling step:

• zipdepth_base.pth — uses torch.nn.Unfold for sub-pixel convex upsampling. Best accuracy on GPU.
• zipdepth_base_npu.pth — replaces the unfold operation with an NPU-friendly equivalent. Use this for ONNX export targeting mobile or edge devices.

🚀 Quick Start

Single image

python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/im0.jpg \
  --input-size 384

Folder of images

python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/imgs/ \
  --output output/depth/

Video

python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/clip.mp4

⚡ Inference

python scripts/infer.py --checkpoint <ckpt> --input <path> [options]

Batch and video inference use an asynchronous pipeline: GPU inference and CPU colormap/write are overlapped, keeping the GPU continuously busy. Timing is measured with CUDA events.

More examples

# Raw depth maps only — maximum speed, no visualization
python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/imgs/ \
  --no-colormap --save-raw

# FP16 + torch.compile — highest throughput
python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/imgs/ \
  --fp16 --compile

# 4K video → 1080p output
python scripts/infer.py \
  --checkpoint checkpoints/zipdepth_base.pth \
  --input assets/examples/clip.mp4 \
  --output-size 1080

📦 Export

ONNX

# GPU checkpoint (default)
python scripts/export.py \
  --ckpt checkpoints/zipdepth_base.pth \
  --format onnx \
  --height 384 --width 384

# NPU / mobile checkpoint — add --npu
python scripts/export.py \
  --ckpt checkpoints/zipdepth_base_npu.pth \
  --format onnx \
  --height 384 --width 384 --npu

Install onnxsim for automatic graph simplification (applied transparently if available).

TorchScript

# Traced
python scripts/export.py \
  --ckpt checkpoints/zipdepth_base.pth \
  --format torchscript

# Frozen — smaller and faster on CPU
python scripts/export.py \
  --ckpt checkpoints/zipdepth_base.pth \
  --format torchscript-frozen

The output is saved next to the checkpoint by default. Pass --output to override the path.

📱 Mobile / Edge Deployment

Both checkpoints can be exported and run on-device — you’re free to try either, but the two upsampling variants are not equally portable:

• zipdepth_base_npu.pth — recommended for mobile / edge / NPU. Its unfold-free convex upsampling is built from operators that convert cleanly across mobile runtimes. Export with --npu.
• zipdepth_base.pth relies on torch.nn.Unfold + pixel_shuffle, which several mobile runtimes lower poorly (or not at all). Great on GPU/server, less reliable on-device.

Typical path: export to ONNX (see above), then convert to your target runtime — ONNX Runtime Mobile, CoreML (iOS), TFLite (Android), or NCNN. Starting from the NPU checkpoint maximizes the chance of a clean, fully-supported conversion.

On-device latency and operator-level profiling across hardware are reported in the Deployment Profiling section of the supplementary material.

📈 Benchmark

Measures parameters, GFLOPs, and latency across backends with IQR-filtered statistics.

python scripts/benchmark.py --height 384 --width 384
python scripts/benchmark.py --height 384 --width 384 --fp16

Representative latency on an RTX 3090 (ZipDepth-base, 384×384, PyTorch 2.4.1+cu121, CUDA 12.1), from the paper’s deployment profiling — median over 200 forward passes (20 warm-up):

  Backend                       Precision   Latency     FPS   Speedup
  ─────────────────────────────────────────────────────────────────────
  PyTorch Eager                 FP32         3.9 ms     255    1.0×
  PyTorch Fused                 FP32         2.5 ms     389    1.5×
  PyTorch Fused                 FP16         3.2 ms     307    1.2×
  torch.compile (max-autotune)  FP16         0.9 ms    1117    4.4×
  TensorRT (static)             FP16         0.8 ms    1317    4.9×

Fused = Conv-BN + reparameterizable branch fusion via fuse_for_inference(). torch.compile uses mode="max-autotune"; TensorRT is a static-shape FP16 engine (measured separately from benchmark.py).

🧪 Evaluation

We follow the zero-shot protocol of Marigold: predictions are aligned to the ground truth with a least-squares scale and shift, then standard depth metrics are computed. Please refer to Marigold for downloading and preparing the benchmark datasets.

Evaluate a checkpoint on any supported benchmark with a single command:

python scripts/eval.py \
  --dataset nyuv2 \
  --data_dir /path/to/NYUv2/test \
  --checkpoint checkpoints/zipdepth_base.pth

The per-dataset depth range, KITTI crop, and Eigen mask are applied automatically. Metrics are printed and saved to eval_results/<dataset>/accuracy_<dataset>.json, with per-sample values in per_sample.csv. Add --fp16 for half-precision inference, or --npu when evaluating the zipdepth_base_npu.pth checkpoint.

🎓 Training

Training Data

ZipDepth was trained via knowledge distillation using pseudo depth maps generated by Depth Anything V2 Large. The training set spans 17 domains and contains approximately 14.1 million RGB–depth pairs:

ACDC · ADE20K · bdd100k · Cityscapes · COCO · DrivingStereo · Flickr1024 · Gated2Depth · GoogleLandmarks · HRWSI · HoloPix50K · Mapillary · MegaDepth · Object365 · OpenImagesv7 · SA-1B · Trans10K

The exact file list used for training — 14,069,951 images across these 17 domains — is released as a downloadable asset:

📄 training_files.txt.gz (71 MB compressed)

Each line is domain<TAB>relative_path. This is the file list only — the RGB images must be obtained from the original public datasets listed above, under their respective licenses.

The proxy depth labels are likewise not distributed: they can be regenerated by running the official Depth Anything V2 Large inference code on the RGB images, exactly as done in the paper.

Dataset Index

The dataloader expects a JSON index of RGB–depth pairs. To build one from your own data, organize images into parallel RGB and depth directories with matching relative paths, then run:

# Single domain
python scripts/prepare_index.py build \
    --domains MyDataset /path/to/rgb /path/to/depth \
    --output dataset_index.json

# Multiple domains via a YAML config
python scripts/prepare_index.py build \
    --config domains.yaml --output dataset_index.json

YAML config format:

MyDataset:
    rgb:   /path/to/rgb
    depth: /path/to/depth
AnotherSet:
    rgb:   /path/to/rgb2
    depth: /path/to/depth2

The depth maps can be PNG (uint16) or .npy/.npz files. Before training, convert the JSON index to numpy memmap format (much faster I/O at millions of samples):

python scripts/prepare_index.py convert --input dataset_index.json

Or do both in one shot with --convert:

python scripts/prepare_index.py build \
    --domains MyDataset /path/to/rgb /path/to/depth \
    --output dataset_index.json --convert

Set index_file in configs/default.json to the JSON path — the dataloader auto-detects the converted .npy files.

Launch

Edit configs/default.json to set your dataset path and hyperparameters, then launch:

# Single GPU
python scripts/train.py --config configs/default.json

# Multi-GPU with DDP (e.g. 2 GPUs)
torchrun --nproc_per_node=2 scripts/train.py --config configs/default.json

# Resume from a checkpoint
python scripts/train.py \
  --config configs/default.json \
  --resume checkpoints/base_384x384/epoch_3.pth

Key configuration fields:

{
  "model":    { "variant": "base" },
  "data":     { "index_file": "/path/to/dataset_index.json", "height": 384, "width": 384 },
  "training": { "epochs": 5, "batch_size": 96 },
  "optimizer":{ "lr": 1e-3, "weight_decay": 0.05 },
  "amp":      { "enabled": true, "dtype": "bfloat16" }
}

Training uses a scale-and-shift invariant loss with gradient regularization, AdamW with OneCycleLR scheduling, and bfloat16 mixed precision.

🙏 Acknowledgements

We thank the authors of Marigold and Depth Anything V2 for their excellent work and for releasing code and evaluation protocols that made this research possible.

📝 Citation

@inproceedings{tosi2026zipdepth,
  title     = {ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device},
  author    = {Tosi, Fabio and Bartolomei, Luca and Poggi, Matteo and Mattoccia, Stefano},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}

📧 Contact

For questions about the paper or the code, feel free to reach out:

• Fabio Tosi — [email protected]
• Luca Bartolomei — [email protected]
• Matteo Poggi — [email protected]
• Stefano Mattoccia — [email protected]