Looking to synchronize multiple Insta360 or action cameras for multi-view data capture without hardware trigger lines? I’ve created an open-source tool that uses a sequence of dynamic QR codes to offline-align videos from various cameras onto a single UTC millisecond timeline, achieving multi-camera synchronization for embodied intelligence data collection. Python 3.11 + uv, ready to use.
🔗 https://t.co/KNAcBbH6Go

XiaomingX/multicam-timestamp-align

Source: https://github.com/XiaomingX/multicam-timestamp-align

Multi-Camera Timestamp Alignment (multicam-align)

Uses a sequence of dynamically played QR codes to offline-align videos from multiple cameras (Insta360, action cameras, etc.) to a single UTC millisecond timeline, achieving synchronized multi-view capture. Designed for embodied intelligence multi-perspective data collection.

Principle

Relying on “each camera records separately, then aligning afterward” cannot achieve millisecond-level precision. This tool works by:

1. Playing a sequence of QR codes on a screen at a fixed frequency freq (e.g., 10 Hz).

2. Each QR code encodes:

   NTP1|||  
   

   • T0_ms: UTC millisecond timestamp corresponding to the 0th frame of the sequence (same for the entire group).
   • frame_index: Incrementing counter (0, 1, 2…) with fixed width (4 digits).

3. All cameras start recording, capturing the screen with the QR codes.

4. After recording, perform offline processing: Recognize frame_index frame-by-frame using pyzbar, mapping each frame to an absolute time:

   t_abs_ms = T0_ms + frame_index / freq * 1000  
   

   Because frame_index increments between screen refreshes, any frame captured by a camera can be interpolated to a sub-frame level absolute timing, compensating for screen refresh delays and camera shutter delays.

5. Perform linear regression t_abs = a·vf + b on each video (vf = video frame time in seconds). The slope accounts for the actual frame rate (including non-integer rates like 29.97), and the intercept gives the absolute time of frame 0 for that camera’s video.

6. Using the camera that appears earliest in the sequence as the reference, calculate the offset for each camera, resulting in a unified timeline.

Error Sources and Mitigations

Key constraint: The sequence playback frequency must be a divisor of the camera frame rate. For example, with cameras at 30 fps, use a sequence at 10 Hz (one QR code every 3 video frames), ensuring no video frame captures two adjacent QR codes, which would cause ambiguity.

Installation

Requires Python 3.11+ and uv. On macOS, also install system libraries zbar and ffmpeg:

# macOS
brew install zbar ffmpeg

# Or install ffmpeg from source/package manager (Linux: apt install zbar ffmpeg)

# Install Python dependencies
uv sync

If import pyzbar raises a Symbol not found error, the system lacks the zbar library. First run brew install zbar (macOS) or apt install libzbar0 (Linux).

Usage Workflow

1. Generate QR Code Sequence + Playback Page

uv run multicam-align gen --freq 10 --seq-len 20 --out qr_seq/

• Generates qr_seq/qr_0000.png … qr_seq/qr_0019.png and a full-screen playback page qr_seq/play.html.
• T0 defaults to “current time + 3 seconds”, giving you enough time to open the playback page and start all cameras. You can also specify it explicitly with --t0 <timestamp_ms>.

2. Recording

1. Open play.html in full-screen mode (disable system sleep/screensaver, keep the browser tab in the foreground).
2. Start recording on all cameras, ensuring the QR code sequence on the screen is clearly captured (the sequence appears for a few seconds in each loop; you don’t need to capture the entire playback).
3. Begin your data collection actions.
4. Stop recording and copy the video files to your local machine.

3. Alignment

uv run multicam-align align \
  --videos cam1.mp4 cam2.mp4 cam3.mp4 \
  --names cam1 cam2 cam3 \
  --out aligned/

• Recognizes QR codes frame-by-frame, fits the timeline, and outputs aligned/result.json along with each camera’s offset, coverage, and RMS error.
• Optional: --reference <camera_name> specifies the reference camera (default: the one appearing earliest in the sequence).
• --step N decodes every Nth frame to speed up processing (coverage statistics become coarser).

4. Remux Videos (Aligned Start Point)

uv run multicam-align remux --result aligned/result.json --out synced/

Uses each camera’s offset with ffmpeg -ss to trim, ensuring all output videos start at the same UTC moment (synced/_aligned.mp4).

5. Generate Alignment Report

uv run multicam-align report --result aligned/result.json --out report.html

The HTML report includes: each camera’s offset, coverage, RMS, long-missed detection alerts, as well as an “absolute timeline fit plot” and “residual scatter plot” for assessing alignment quality.

6. Extract Synchronized Frame at a Specific Time

uv run multicam-align extract --result aligned/result.json --at 1703123457000 --out frames/

Extracts the frame closest to the target UTC millisecond from each camera (frames/_.png), useful for stitching synchronized multi-view snapshots.

Module Structure

src/multicam_align/
├── cli.py          # Unified CLI (gen/align/remux/report/extract)
├── qr_generator.py # Generate dynamic QR sequence + play.html + payload parsing
├── decoder.py      # Frame-by-frame frame_index recognition + linear timeline fitting
├── align.py        # Multi-camera offset + global timeline + result.json
├── remux.py        # ffmpeg remuxing based on offset
├── report.py       # matplotlib + HTML report generation
└── extract.py      # Extract nearest frame given UTC time

Testing

uv run pytest

tests/test_roundtrip.py verifies that each generated QR code can be recognized and restored to (t0, frame_index, seq_len), which is the core invariant of the alignment scheme.

Notes / Troubleshooting

• Frequency divisor constraint: The sequence frequency must be a divisor of the camera frame rate; otherwise, ambiguity arises from a frame spanning two QR codes.
• Missed detection alerts: If a camera’s coverage is too low or there are long stretches of missed detections, the report will mark ⚠. Check if the QR codes in that camera’s footage are clear and free from motion blur.
• Long recording sessions: Camera clock drift accumulates over time. For long captures, consider filming the QR sequence again midway for segment re-anchoring (subsequent versions may support multiple anchors).
• ffmpeg precision: -ss input-level seeking is generally sufficient. For frame-accurate remuxing, modify remux.py to re-encode (slower).