SAM-MT: Real-Time Interactive Multi-Target Video Segmentation

Hugging Face Daily Papers Papers

Summary

This paper introduces SAM-MT, an extension of SAM2 for real-time interactive multi-target video segmentation, achieving high FPS independent of target count.

Modern Video Object Segmentation (VOS) involves tracking and segmenting user-specified targets. While recent approaches have achieved remarkable performance in single-target scenarios, extending them to multi-target settings typically involves replicating the single-target processing for each individual object, resulting in reduced frame rates (FPS) with unbounded latency as target count increases. Built upon Segment Anything 2 (SAM2), we propose SAM-MT, which addresses this by transforming the model into an interactive framework for real-time Multi-Target video segmentation. SAM-MT uses explicit queries to represent different individual targets, in parallel with a shared representation for global context. It employs decoupled masked attention to keep individual identities distinct from cross-target interference, and sparse memory for stable temporal evolution, along with specialized strategies for occlusion handling and overlap prevention. SAM-MT successfully decouples latency from the number of targets, achieving real-time speed on par with single-target baselines (>36 FPS for 10 targets) while maintaining SAM2's robust video segmentation performance.
Original Article
View Cached Full Text

Cached at: 07/10/26, 06:11 PM

Paper page - SAM-MT: Real-Time Interactive Multi-Target Video Segmentation

Source: https://huggingface.co/papers/2607.08688

Abstract

ModernVideoObjectSegmentation(VOS)involvestrackingandsegmentinguser-specifiedtargets.Whilerecentapproacheshaveachievedremarkableperformanceinsingle-targetscenarios,extendingthemtomulti-targetsettingstypicallyinvolvesreplicatingthesingle-targetprocessingforeachindividualobject,resultinginreducedframerates(FPS)withunboundedlatencyastargetcountincreases.BuiltuponSegmentAnything2(SAM2),weproposeSAM-MT,whichaddressesthisbytransformingthemodelintoaninteractiveframeworkforreal-timeMulti-Targetvideosegmentation.SAM-MTusesexplicitqueriestorepresentdifferentindividualtargets,inparallelwithasharedrepresentationforglobalcontext.Itemploysdecoupledmaskedattentiontokeepindividualidentitiesdistinctfromcross-targetinterference,andsparsememoryforstabletemporalevolution,alongwithspecializedstrategiesforocclusionhandlingandoverlapprevention.SAM-MTsuccessfullydecoupleslatencyfromthenumberoftargets,achievingreal-timespeedonparwithsingle-targetbaselines(>36FPSfor10targets)whilemaintainingSAM2’srobustvideosegmentationperformance.

View arXiv pageView PDFProject pageGitHub3Add to collection

Get this paper in your agent:

hf papers read 2607\.08688

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.08688 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.08688 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.08688 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

SAM2Matting: Generalized Image and Video Matting

Hugging Face Daily Papers

SAM2Matting introduces a tracker-to-matting framework that enhances foundational trackers like SAM2 with region-proposal bridges and matting heads, achieving state-of-the-art video matting performance even when trained only on images, with strong generalization across diverse scenarios.

SAM 3: Segment Anything with Concepts

Papers with Code Trending

SAM 3 introduces a unified model for promptable concept segmentation and tracking, achieving state-of-the-art performance with a decoupled recognition and localization architecture and a scalable data engine.