Centralized fusion in large-scale surveillance: when you have dozens or hundreds of cameras, the computational bottleneck directly stalls, making scaling impossible, and a single central station burns through most of the budget. That’s why distributed multi-view tracking cannot truly be deployed without a distributed approach — the expansion cost of centralized solutions skyrockets exponentially with the number of nodes, while engineering demands a one-time large-scale, low-cost deployment. MV3DT provides a practical solution: each camera runs monocular 3D perception + local visual geometric association, nodes only exchange small amounts of state and confidence, running at 30 FPS in real-time with communication overhead compressed to 2.2%, and zero-shot calibration ready to use.

Fully Distributed Multi-View 3D Tracking in Real-Time

Source: https://arxiv.org/html/2606.13127 11institutetext:University of Florida, Gainesville, FL, USA 11email:{bhernandezosorio,hmedeiros}@ufl.edu22institutetext:NVIDIA Corporation, Santa Clara, CA, USA 22email:{bhernandez,fangyul,aotianw,pshin,kpurandare}@nvidia.comFangyu LiAotian WuPaul J. ShinKaustubh PurandareHenry Medeiros

Abstract

Multi-camera tracking with overlapping fields of view typically relies on centralized fusion, which creates computational bottlenecks that prevent deployment at scale. We presentMV3DT, a fully distributed framework for real-time multi-view 3D tracking that achieves accurate identity propagation and occlusion recovery through peer-to-peer coordination, eliminating the need for central aggregation. Each camera node executes a lightweight modular pipeline comprising monocular 3D perception, distributed multi-view association, and collaborative fusion via lightweight messaging. MV3DT achieves 96.5% IDF1, 93.1% MOTA, and 94.6% MOTP on WILDTRACK, competitive with state-of-the-art centralized methods, and unprecedented 41.7% IDF1 and 50.9% MOTA on SCOUT while demonstrating superior scalability: sustaining 30 FPS on 100 cameras with<<10ms inter-camera latency and only 2.2% communication overhead. MV3DT operates in a zero-shot regime given camera calibrations, requiring no scene-specific learning and making it directly deployable in new environments. These results establish MV3DT as a practical solution for real-time multi-view tracking in large-scale overlapping camera networks.

1Introduction

Refer to captionFigure 1:MV3DT Overview.MV3DT deploys a modular pipeline on each camera node without requiring a central server.Monocular Detectionextracts 2D bounding boxes. Then, 3D foot location estimates and full-body bounding boxes are computed forData Association, where detection-to-targets matches, both intra-view and multi-view, are found using several similarity measures.Target Managementmaintains target state and ID consistency across overlapping cameras through distributed ID propagation, and integrates Kalman filtering for multi-view measurement fusion.Distributed Communicationuses MQTT publish/subscribe messaging for peer-to-peer coordination. Each camera maintains a local database of shared target states, enabling coordinated tracking without centralized aggregation. MV3DT achieves highly effective ID propagation and multi-view integration throughfully-distributed interactions, allowing for online and real-time deployment of large camera networks.Multi-camera multi-target tracking (MCMT) is a prevalent problem in computer vision. Large-scale applications such as warehouse monitoring and intelligent cities require tens to thousands of cameras to effectively cover the region of interest[24 (https://arxiv.org/html/2606.13127#bib.bib24)]. The increasing participation in the AI City Challenge reflects the growing importance of large-scale MCMT[25 (https://arxiv.org/html/2606.13127#bib.bib25),45 (https://arxiv.org/html/2606.13127#bib.bib45),41 (https://arxiv.org/html/2606.13127#bib.bib41)]. MCMT techniques can be classified as centralized, decentralized, or distributed, depending on how they execute processes and aggregate data[14 (https://arxiv.org/html/2606.13127#bib.bib14)]. They may also focus on camera topologies with overlapping or non-overlapping fields of view (FOV). The amount of FOV overlap determines the availability of multi-view geometric cues to improve tracking accuracy. On the other hand, non-overlapping camera systems typically use appearance representations and trajectory prediction based on camera linking models to achieve effective tracking[31 (https://arxiv.org/html/2606.13127#bib.bib31),1 (https://arxiv.org/html/2606.13127#bib.bib1)].

Centralized approaches leverage overlapping camera setups to exploit global information from all cameras on asingle fusion stage, which can improve accuracy but concentrates all computation and communication in one logical node[14 (https://arxiv.org/html/2606.13127#bib.bib14)]. In large deployments, such centralized fusion often becomes impractical due to bandwidth, latency, and robustness constraints. Many distributed MCMT methods target non-overlapping or sparsely overlapping camera topologies, where cross-camera association relies primarily on appearance and temporal constraints. In these settings, cameras often operate without explicit 3D calibration and maintain consistency mainly through label and appearance exchange. The current paradigm of distributed techniques for overlapping camera setups relies on parallel single-camera tracking (SCT) processes followed by a centralized multi-camera aggregation stage. Thistwo-stagedependency on a central entity hinders real-time deployment and limits scalability.

Overlapping FOVs provide complementary 3D geometric information that reduces the impact of occlusions, which is one of the most challenging issues in multi-object tracking[4 (https://arxiv.org/html/2606.13127#bib.bib4),18 (https://arxiv.org/html/2606.13127#bib.bib18)]. Although centralized methods have long benefited from overlapping FOVs, fully distributed MCMT systems that operate directly in the 3D ground plane and are demonstrated at large scale on fixed, calibrated overlapping camera networks remain scarce[1 (https://arxiv.org/html/2606.13127#bib.bib1),51 (https://arxiv.org/html/2606.13127#bib.bib51),33 (https://arxiv.org/html/2606.13127#bib.bib33)]. The fundamental difficulty is a self-conflicting requirement: the multi-view geometric cues improve tracking accuracy, yet exploiting them in a distributed manner is challenging and has limited the scalability of accurate multi-view tracking.

One of the main challenges for scalable deployment is the availability of computing resources. Advances in computer and communication technology have enabled larger interconnected camera networks for real-time supervision. Simultaneously, models and algorithm complexity have grown proportionally. Therefore, accurately tracking multiple objects in real-time across large camera networks while leveraging overlapping views remains an open challenge. Recent fully distributed approaches already demonstrate that peer-to-peer coordination can sustain online multi-camera tracking[51 (https://arxiv.org/html/2606.13127#bib.bib51),33 (https://arxiv.org/html/2606.13127#bib.bib33)], but they primarily operate in 2D image space and do not exploit calibrated 3D ground-plane geometry in large-scale overlapping camera networks.

As illustrated inFig. ̃1 (https://arxiv.org/html/2606.13127#S1.F1), we propose a fully distributed MCMT framework in which a modular pipeline processes each video feed in parallel without a central aggregator. Each camera node executes a pipeline comprising modules for data association, target management, motion estimation, monocular 3D perception, inter-camera communication, and distributed multi-view fusion. The framework enables multi-view identity propagation and 3D fusion through lightweight inter-camera communication, allowing each node to reason locally while achieving globally consistent associations across overlapping views.

Our main contributions are highlighted below:

• •A fully distributed multi-view 3D tracking framework for calibrated overlapping cameras, in which each node performs local 3D tracking, peer‐to‐peer ID propagation, and multi-view fusion without a central aggregation server.
• •An occlusion-aware monocular 3D detector with mechanisms that turn noisy single‐view detections into reliable multi‐view measurements.
• •A three‐stage distributed ID propagation protocol that ensures global ID convergence without a central node.
• •Large‐scale evaluation with state-of-the-art accuracy on standard benchmarks, measured communication overhead, and synchronized deployment.

2Related Work

Early object tracking systems based on multiple cameras[39 (https://arxiv.org/html/2606.13127#bib.bib39),19 (https://arxiv.org/html/2606.13127#bib.bib19)]used triangulation techniques or exploited the geometry of the scene[22 (https://arxiv.org/html/2606.13127#bib.bib22)]to combine information from different perspectives. One of the main motivations for the development of multi-camera tracking techniques was the resolution of target occlusions in single-view systems[4 (https://arxiv.org/html/2606.13127#bib.bib4)]. These approaches were designed to operate on a single computer or on multiple computers orchestrated by a leader. As the scale of multi-camera systems increased, it became clear that these earlycentralizedapproaches were limited to small areas covered by few cameras[40 (https://arxiv.org/html/2606.13127#bib.bib40)].

Decentralizedapproaches group cameras into clusters and designate lead nodes or cluster heads to aggregate information from neighboring cameras, reducing communication overhead[21 (https://arxiv.org/html/2606.13127#bib.bib21),53 (https://arxiv.org/html/2606.13127#bib.bib53)]. Cluster heads coordinate within their groups and communicate summaries across cluster boundaries. Fullydistributedapproaches rely on peer-to-peer strategies in which all camera nodes operate as equal participants without hierarchical coordination entities[7 (https://arxiv.org/html/2606.13127#bib.bib7)]. In such systems, cameras exchange information and reach consensus through decentralized algorithms, eliminating any dependency on leader nodes or coordinators. Distributed methods have mainly focused on camera networks with disjoint FOVs so far, where appearance-based object re-identification (reID) and inter-camera linking suffice for cross-camera association. Recent fully distributed MCMT systems further demonstrate that peer-to-peer coordination can sustain online multi-camera tracking. Some approaches tackle the association problem by sharing ID labels and appearance features across cameras and maintaining a distributed label–appearance table to reach ID consensus[51 (https://arxiv.org/html/2606.13127#bib.bib51)]; others share full tracklets across cameras and fuse these hypotheses into consistent multi-camera trajectories[33 (https://arxiv.org/html/2606.13127#bib.bib33)]. However, these methods operate primarily in 2D image space and cannot perform 3D ground-plane tracking in large-scale overlapping camera networks.

Camera topology shapes algorithm design[31 (https://arxiv.org/html/2606.13127#bib.bib31),1 (https://arxiv.org/html/2606.13127#bib.bib1)]. For non-overlapping or sparsely overlapping networks, linking models discover spatial or topological connectivity between camera views, learning which cameras observe adjacent or connected regions to establish inter-camera associations[15 (https://arxiv.org/html/2606.13127#bib.bib15),34 (https://arxiv.org/html/2606.13127#bib.bib34)]. Recent approaches present reID strategies when targets are not visible for prolonged periods[37 (https://arxiv.org/html/2606.13127#bib.bib37),26 (https://arxiv.org/html/2606.13127#bib.bib26)]. These methods rely on discriminative appearance features, which are also used for multi-camera associations.

For overlapping FOVs, geometric approaches are commonly used for association. Examples include the projection of image coordinates onto a global coordinate system[19 (https://arxiv.org/html/2606.13127#bib.bib19)]and homography-based matching[38 (https://arxiv.org/html/2606.13127#bib.bib38)]. Recent methods leverage transformer-based architectures and bird’s-eye view (BEV) representations to aggregate multi-view information early in the pipeline[42 (https://arxiv.org/html/2606.13127#bib.bib42),46 (https://arxiv.org/html/2606.13127#bib.bib46)]. BEVFormer[20 (https://arxiv.org/html/2606.13127#bib.bib20)]introduces spatiotemporal transformers that project multi-camera features onto a unified BEV space, enabling robust 3D object detection and tracking. Building on this, TrackTacular[42 (https://arxiv.org/html/2606.13127#bib.bib42)]combines temporal feature aggregation with appearance and motion cues for multi-view pedestrian and vehicle tracking. BEV-SUSHI[46 (https://arxiv.org/html/2606.13127#bib.bib46)]extends the BEV paradigm with hierarchical graph neural networks for long-term identity association. MVTrajecter[50 (https://arxiv.org/html/2606.13127#bib.bib50)]incorporates BEV motion and appearance costs, achieving state-of-the-art performance on pedestrian benchmarks. Other recent methods explore end-to-end temporal aggregation[52 (https://arxiv.org/html/2606.13127#bib.bib52)]and unified graph-based frameworks, such as the Unified Message Passing Network (UMPN)[12 (https://arxiv.org/html/2606.13127#bib.bib12)].

Tracking accuracy depends on detection quality[3 (https://arxiv.org/html/2606.13127#bib.bib3)], and multi-view detection fusion methods follow the same taxonomy: in non-overlapping networks, each camera runs a 2D detector and reID or linking models handle cross-camera association; in overlapping scenarios, centralized methods may perform fusion at detection time with multi-view or BEV detectors, while distributed systems keep detection local (single-view 2D per node) and perform 3D reasoning at association. Widely used single-view 2D detectors include YOLO[35 (https://arxiv.org/html/2606.13127#bib.bib35),36 (https://arxiv.org/html/2606.13127#bib.bib36),16 (https://arxiv.org/html/2606.13127#bib.bib16)], DETR and its variants[5 (https://arxiv.org/html/2606.13127#bib.bib5),57 (https://arxiv.org/html/2606.13127#bib.bib57),56 (https://arxiv.org/html/2606.13127#bib.bib56)], and recent YOLO iterations[44 (https://arxiv.org/html/2606.13127#bib.bib44),43 (https://arxiv.org/html/2606.13127#bib.bib43)]. Multi-view detectors[9 (https://arxiv.org/html/2606.13127#bib.bib9),8 (https://arxiv.org/html/2606.13127#bib.bib8),11 (https://arxiv.org/html/2606.13127#bib.bib11),2 (https://arxiv.org/html/2606.13127#bib.bib2)]improve accuracy in overlapping FOVs but require centralized aggregation and do not scale to distributed deployments.

3Multi-Object Tracking Framework

MV3DT introduces a novel fully distributed and modular MCMT paradigm. Our framework aims for a real-time, online, and accurate pipeline for multi-object tracking on multiple cameras; occlusion handling and scalability are key objectives. Similar in philosophy to[21 (https://arxiv.org/html/2606.13127#bib.bib21),47 (https://arxiv.org/html/2606.13127#bib.bib47)], we exploit simultaneous information from multiple views in a fully 3D setting. We use a lightweight peer-to-peer communication strategy to share multi-view information and resolve tracking ambiguities in real-time. Rather than resorting to a centralized multi-view tracking mechanism or aggregating the results of multiple single-view trackers, our method treats each camera as an independent agent. Hence, it can be instantiated as a single process that communicates with other cameras within a communication network. This section describes the core components: detection, data association, target management, multi-view fusion, and communications.

3.1Object Detection Module

This module produces a set of object bounding boxes{bd}\{\mathbf{b}_{d}\}for each input frame, wherebd=[u,v,w,h]\mathbf{b}_{d}=[u,v,w,h], and(u,v)(u,v)are the pixel coordinates of its top-left corner and(w,h)(w,h)are its width and height.

3.1.1Monocular Foot Localization with Occlusion Handling.

To enable 3D geometric reasoning, we model target objects as cylindersC