VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation

Hugging Face Daily Papers Papers
multimodal distillation visual-attribution counterfactual on-policy

Summary

Introduces Visual Attribution Distillation (VAD), a counterfactual target-reconstruction method for multimodal on-policy distillation that estimates the visually-attributable part of teacher corrections. Outperforms existing approaches across fine-grained visual benchmarks at 4B and 9B scales.

Multimodal on-policy distillation (OPD) transfers fine-grained visual knowledge by supervising student-generated trajectories with a privileged-view teacher. Yet its next-token corrections are source-mixed, combining visual signals with linguistic priors and teacher-specific effects. The key challenge is to estimate which corrections are supported by visual evidence, not merely where or how strongly to distill. We introduce Visual Attribution Distillation (VAD), a counterfactual target-reconstruction algorithm that estimates the visually attributable part of a teacher correction. At each student-generated prefix, VAD evaluates the same fixed teacher with the relevant evidence present and removed. The corresponding change in centered log-probabilities defines ut, a signed proxy for the visual evidence direction that estimates how revealing the evidence supports or refutes candidate tokens. VAD projects the original correction onto this proxy to obtain an intervention-aligned component and a proxy-unexplained residual, then reconstructs a student-anchored target from the former. During training, this reconstructed target supplies the primary supervision signal, while the privileged teacher contributes a weak regularizer. Across six fine-grained visual benchmarks at 4B and 9B scales, VAD outperforms direct privileged-view distillation and visual-advantage weighting. Token- level and controlled-target analyses show that the proxy-aligned component is enriched in task-relevant visual corrections and yields stronger target shifts, especially when evidence refutes a mistaken answer. These results support counterfactual target reconstruction as an effective alternative to source-mixed supervision.
Original Article
View Cached Full Text

Cached at: 08/04/26, 05:37 AM

Paper page - VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation

Source: https://huggingface.co/papers/2607.28590 Published on Jul 30

·

Submitted byhttps://huggingface.co/zhangkangning

张康宁on Aug 4

#2 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

Multimodalon-policydistillation(OPD)transfersfine-grainedvisualknowledgebysupervisingstudent-generatedtrajectorieswithaprivileged-viewteacher.Yetitsnext-tokencorrectionsaresource-mixed,combiningvisualsignalswithlinguisticpriorsandteacher-specificeffects.Thekeychallengeistoestimatewhichcorrectionsaresupportedbyvisualevidence,notmerelywhereorhowstronglytodistill.WeintroduceVisualAttributionDistillation(VAD),acounterfactualtarget-reconstructionalgorithmthatestimatesthevisuallyattributablepartofateachercorrection.Ateachstudent-generatedprefix,VADevaluatesthesamefixedteacherwiththerelevantevidencepresentandremoved.Thecorrespondingchangeincenteredlog-probabilitiesdefinesut,asignedproxyforthevisualevidencedirectionthatestimateshowrevealingtheevidencesupportsorrefutescandidatetokens.VADprojectstheoriginalcorrectionontothisproxytoobtainanintervention-alignedcomponentandaproxy-unexplainedresidual,thenreconstructsastudent-anchoredtargetfromtheformer.Duringtraining,thisreconstructedtargetsuppliestheprimarysupervisionsignal,whiletheprivilegedteachercontributesaweakregularizer.Acrosssixfine-grainedvisualbenchmarksat4Band9Bscales,VADoutperformsdirectprivileged-viewdistillationandvisual-advantageweighting.Token-levelandcontrolled-targetanalysesshowthattheproxy-alignedcomponentisenrichedintask-relevantvisualcorrectionsandyieldsstrongertargetshifts,especiallywhenevidencerefutesamistakenanswer.Theseresultssupportcounterfactualtargetreconstructionasaneffectivealternativetosource-mixedsupervision.

View arXiv pageView PDFGitHub16Add to collection

Get this paper in your agent:

hf papers read 2607\.28590

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.28590 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.28590 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.28590 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

OPD-V: Visual On-Policy Self-Distillation with Modality Balance

Hugging Face Daily Papers

Introduces OPD-V, a visual on-policy self-distillation paradigm for multimodal large language models that leverages positive and negative teachers to exploit modality balance as privileged information, improving reasoning performance across benchmarks while reducing training cost.

Visual Contrastive Self-Distillation

Hugging Face Daily Papers

VCSD removes the need for external teachers, privileged answers, or visual evidence in on-policy self-distillation by using content-erased control images to produce contrastive signals, consistently outperforming existing methods on vision-language benchmarks.