Show HN: MultiMatte, a Promptable Image Background Removal Model

# MultiMatte: Keep What You Want, Cut the Rest — Feyn Source: [https://usefeyn.com/blog/multimatte/](https://usefeyn.com/blog/multimatte/) We’re introducing MultiMatte, a background removal model you can aim with words\.**MultiMatte keeps the object you name and removes everything else\.** Try MultiMatte on your own images at[usefeyn\.com/multimatte](https://usefeyn.com/multimatte)\. MultiMatte is built on[SAM 3](https://huggingface.co/facebook/sam3)\([Meta, 2025](https://usefeyn.com/blog/multimatte/#ref-1)\)\. We used low\-rank fine\-tuning to modify 19\.49M of its 860M parameters\. That update touches only 2\.27% of the model weights, yet MultiMatte improves substantially on image segmentation\. On the DIS\-VD benchmark, it scores a 0\.901 S\-measure against SAM 3’s 0\.667\. Prompt steeringdrag to compare inputPrompt: the dog ## From Segmentation to Matting SAM 3 is a concept\-promptable detector\. Given a phrase, the model returns binary masks for every matching object\. These masks classify each pixel as either inside or outside the object\. This binary approach fails on fine or translucent elements with fuzzy boundaries, like hair or a blurred screen\. MultiMatte addresses this gap with alpha mattes\. Instead of assigning a boolean to each pixel, an alpha matte assigns a continuous opacity value to every pixel\. Fuzzy boundaries can then be described as they actually appear\. Alpha mattes earn their keep on benchmarks\. Across five high\-resolution DIS splits, SAM 3 scores between 0\.649 and 0\.703\. MultiMatte scores between 0\.893 and 0\.923\. ## Training MultiMatte SAM 3 is not one network but several aligned parts: a vision tower, a CLIP text tower, a geometry encoder, a detection encoder and decoder, and a mask decoder\. The ability to aim the model with a phrase lives in the alignment between those parts\. Our objective was to carry this ability over to matting\. We trained the model with PEFT, using LoRA \([Hu et al\., 2021](https://usefeyn.com/blog/multimatte/#ref-2)\)\. Every targeted linear layer kept its pretrained weight frozen and learned two small matrices that added a low\-rank update alongside it\. Training this way let us retain SAM 3’s text alignment and original vocabulary while improving its segmentation ability\. The rank\-16 adapter followed the configuration principles in Thinking Machines’ LoRA analysis \([2025](https://usefeyn.com/blog/multimatte/#ref-3)\)\. It targeted the attention and MLP projections in every tower, including the CLIP text tower\. We merged the final adapter into the released weights, so inference doesn’t require a separate adapter library\. The training run used 19,953 images spanning salient objects, camouflage, high\-resolution subjects, hair, and marine scenes\. We trained for 14,000 steps with the semantic segmentation objective SAM 3 already uses: focal loss \([Lin et al\., 2017](https://usefeyn.com/blog/multimatte/#ref-4)\) plus Dice loss\. For prompt supervision, we used 4,949 images with human\-written labels, so 24\.8% of the training set named the specific object in each image\. This was enough to teach the new matte head to use the text alignment SAM 3 already has\. ## Results We report S\-measure, which rewards complete subjects and faithful shapes on a scale from 0 to 1, and mean absolute error over opacity values, where lower is better\. Prompt\-free S\-measure · SAM 3 → MultiMatte Prompt\-free S\-measure for SAM 3 and MultiMatte on twelve benchmark splits\.SplitSAM 3MultiMatteChangeDIS\-VD0\.6670\.901\+0\.233DIS\-TE10\.6670\.901\+0\.234DIS\-TE20\.7030\.923\+0\.220DIS\-TE30\.6850\.921\+0\.235DIS\-TE40\.6490\.893\+0\.244DAVIS\-S \(no sibling in training mix\)0\.9130\.979\+0\.066HRSOD\-TE0\.9300\.969\+0\.039UHRSD\-TE0\.8770\.961\+0\.084DUTS\-TE0\.8920\.954\+0\.062DUT\-OMRON \(no sibling in training mix\)0\.7920\.901\+0\.109COD10K\-TE0\.7870\.934\+0\.148CAMO\-TE0\.8270\.914\+0\.086Higher is better\.MultiMatte improves on SAM 3 in every split\. We treat changes below 0\.002 S\-measure or 0\.0005 MAE as measurement noise\. DAVIS\-S and DUT\-OMRON are the strongest generalization checks because neither has a sibling in the training mix\. MultiMatte reaches its highest absolute score on DAVIS\-S at 0\.979 and lifts DUT\-OMRON from 0\.792 to 0\.901\. Concept names help even without fine\-tuning\. On DIS\-VD, a real concept name adds 0\.150 S\-measure to SAM 3 with no gradient steps and still adds 0\.036 to MultiMatte\. The prompt pathway survived the retrain and contributes on its own\. CAMO’s named scores use the same annotator’s phrasing style seen during training, so that result measures in\-family prompt generalization rather than cross\-domain prompting\. ## Run MultiMatte MultiMatte is built with[NoBg](https://github.com/feyninc/nobg), the same library we used to train[FeyNoBg](https://usefeyn.com/blog/feynobg)\. Install it with`pip install nobg`: ``` from nobg import AutoModel, AutoProcessor model = AutoModel.from_pretrained("feyninc/multimatte") processor = AutoProcessor.from_pretrained("feyninc/multimatte") # Uses the model's generic default prompt. cutout = model.predict(processor, "photo.jpg") cutout.save("output.png") # Names the concept to keep. cutout = model.predict(processor, "photo.jpg", "the dog") ``` `predict`returns an RGBA cutout\. Pass`return\_type="tensor"`for the raw matte\. The adapter is already merged into the released weights for ease of use\. Try MultiMatte on your own images at[usefeyn\.com/multimatte](https://usefeyn.com/multimatte)\. MultiMatte and NoBg are built by Feyn\. Find us on[X](https://x.com/feynai),[GitHub](https://github.com/feyninc), or[LinkedIn](https://www.linkedin.com/company/107081181/)\. ## Acknowledgements MultiMatte is built on Meta’s SAM 3\. FlowDIS \([Sargsyan and Navasardyan, 2026](https://usefeyn.com/blog/multimatte/#ref-5)\) supplied the human\-written DIS5K phrases used for training and evaluation\. Thinking Machines’ LoRA analysis informed the adapter configuration\. We are grateful to the dataset authors whose released work made the training mix and evaluation possible\. References \[1\]Meta\. "SAM 3: Segment Anything with Concepts\."*arXiv:2511\.16719*\(2025\)\. \[2\]Hu et al\. "LoRA: Low\-Rank Adaptation of Large Language Models\."*arXiv:2106\.09685*\(2021\)\. \[3\]Thinking Machines Lab\. "LoRA Without Regret" \(2025\)\. https://thinkingmachines\.ai/blog/lora/ \[4\]Lin et al\. "Focal Loss for Dense Object Detection\."*arXiv:1708\.02002*\(2017\)\. \[5\]Sargsyan and Navasardyan\. "FlowDIS: Language\-Guided Dichotomous Image Segmentation with Flow Matching\."*CVPR*2026\.*arXiv:2605\.05077*\. Cite this note ``` @note{multimatte2026, title = {MultiMatte: Keep What You Want, Cut the Rest}, author = {Hichri, Hafedh and Nigam, Shreyash and Feyn Research}, year = {2026}, venue = {Feyn Field Notes} } ```