MatAnyone-E: Sparse Memory Propagation for Efficient High-Resolution Video Matting

Hugging Face Daily Papers Papers
video-matting efficiency sparse-memory high-resolution computer-vision inference-speedup

Summary

MatAnyone-E is an efficient extension of MatAnyone 2 for high-resolution video matting that applies high-resolution memory matching only to uncertain regions and uses a lightweight decoder, achieving up to 7.4x speedup and reducing peak GPU memory to 7.6% of the original at 4K while preserving matting quality.

High-resolution video matting requires preserving fine details, such as hair strands and soft transparency, while maintaining consistency across frames. Memory-based methods achieve strong matting quality by matching current-frame features with information stored from earlier frames, but this process becomes increasingly costly as resolution grows. We present MatAnyone-E, an efficient extension of MatAnyone 2 that focuses computation where it is most needed. Our approach first performs low-resolution propagation to identify uncertain regions that require closer inspection and the memory candidates most relevant to them. It then performs high-resolution matching only for these regions using the selected memory, while reusing low-resolution matching results elsewhere. This strategy reduces both the number of locations requiring detailed matching and the amount of memory searched, while preserving the information needed to produce a complete matte. We further reduce the cost of generating high-resolution mattes with a lightweight decoder designed around measured runtime bottlenecks and trained to reproduce the original decoder's representations. Experiments show that MatAnyone-E maintains matting quality comparable to MatAnyone 2 while achieving an estimated 7.4x inference speedup and reducing peak GPU memory usage to 7.6% of the usage of MatAnyone 2 at 4K. With compilation-enabled inference on a single NVIDIA H200 GPU, it reaches 73.07 FPS at 1080p and 28.62 FPS at 4K.
Original Article
View Cached Full Text

Cached at: 10/28/26, 05:27 AM

Paper page - MatAnyone-E: Sparse Memory Propagation for Efficient High-Resolution Video Matting

Source: https://huggingface.co/papers/2610.35724

Abstract

MatAnyone-E accelerates high-resolution video matting by selectively applying high-resolution matching to uncertain regions and using a lightweight decoder, achieving large speedups with preserved quality.

High-resolutionvideo mattingrequires preserving fine details, such as hair strands and soft transparency, while maintaining consistency across frames.Memory-based methodsachieve strong matting quality by matching current-frame features with information stored from earlier frames, but this process becomes increasingly costly as resolution grows. We present MatAnyone-E, an efficient extension of MatAnyone 2 that focuses computation where it is most needed. Our approach first performslow-resolution propagationto identify uncertain regions that require closer inspection and thememory candidatesmost relevant to them. It then performshigh-resolution matchingonly for these regions using the selected memory, while reusing low-resolution matching results elsewhere. This strategy reduces both the number of locations requiring detailed matching and the amount of memory searched, while preserving the information needed to produce a complete matte. We further reduce the cost of generating high-resolution mattes with alightweight decoderdesigned around measured runtime bottlenecks and trained to reproduce the original decoder’s representations. Experiments show that MatAnyone-E maintains matting quality comparable to MatAnyone 2 while achieving an estimated 7.4xinference speedupand reducing peakGPU memoryusage to 7.6% of the usage of MatAnyone 2 at 4K. With compilation-enabled inference on a single NVIDIA H200 GPU, it reaches 73.07 FPS at 1080p and 28.62 FPS at 4K.

View arXiv pageView PDFProject pageGitHub54Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.35724 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.35724 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.35724 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

SAM2Matting: Generalized Image and Video Matting

Hugging Face Daily Papers

SAM2Matting introduces a tracker-to-matting framework that enhances foundational trackers like SAM2 with region-proposal bridges and matting heads, achieving state-of-the-art video matting performance even when trained only on images, with strong generalization across diverse scenarios.