PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents

Hugging Face Daily Papers Papers
diffusion-models image-generation video-generation test-time-search multimodal-critique prompt-adherence latent-search

Summary

PreviewDiff introduces a training-free test-time search method that turns diffusion sampling into a multimodal critic-guided search over intermediate latents, improving compositional prompt adherence in image and video generation beyond budget-matched Best-of-N baselines.

Diffusion models can produce striking images and videos, but they still struggle with the compositional details that make a generation faithful to a prompt, such as object counts, attribute binding, spatial relations, and temporally grounded actions. A common way to improve prompt satisfaction is to spend more compute at test time through Best-of-N sampling, but final-sample selection is fixed. Best-of-N can only choose among completed outputs and cannot repair a promising trajectory before it fails. We introduce PreviewDiff, a training-free test-time search method that turns diffusion sampling from scalar search into a multimodal critic-guided search over intermediate latents. At selected denoising checkpoints, PreviewDiff decodes a partial preview, asks a multimodal judge to score and critique it, and uses the resulting natural-language feedback to branch over semantic prompt edits and locally re-noised latent continuations. These branches are then scored and selectively rolled forward, allowing verifier compute to guide generation while the sample is still editable. Across image and video generation benchmarks, PreviewDiff consistently improves over budget-matched Best-of-N selection and strong scalar-search baselines. Ablations show that earlier interventions and increased search width provide the largest gains, while deeper search and additional semantic variants offer complementary improvements. PreviewDiff demonstrates that multimodal feedback is most useful not only as a final verifier, but as an active controller inside the denoising process.
Original Article
View Cached Full Text

Cached at: 10/01/26, 12:19 AM

Paper page - PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents

Source: https://huggingface.co/papers/2609.36199

Abstract

Diffusionmodelscanproducestrikingimagesandvideos,buttheystillstrugglewiththecompositionaldetailsthatmakeagenerationfaithfultoaprompt,suchasobjectcounts,attributebinding,spatialrelations,andtemporallygroundedactions.AcommonwaytoimprovepromptsatisfactionistospendmorecomputeattesttimethroughBest-of-Nsampling,butfinal-sampleselectionisfixed.Best-of-Ncanonlychooseamongcompletedoutputsandcannotrepairapromisingtrajectorybeforeitfails.WeintroducePreviewDiff,atraining-freetest-timesearchmethodthatturnsdiffusionsamplingfromscalarsearchintoamultimodalcritic-guidedsearchoverintermediatelatents.Atselecteddenoisingcheckpoints,PreviewDiffdecodesapartialpreview,asksamultimodaljudgetoscoreandcritiqueit,andusestheresultingnatural-languagefeedbacktobranchoversemanticprompteditsandlocallyre-noisedlatentcontinuations.Thesebranchesarethenscoredandselectivelyrolledforward,allowingverifiercomputetoguidegenerationwhilethesampleisstilleditable.Acrossimageandvideogenerationbenchmarks,PreviewDiffconsistentlyimprovesoverbudget-matchedBest-of-Nselectionandstrongscalar-searchbaselines.Ablationsshowthatearlierinterventionsandincreasedsearchwidthprovidethelargestgains,whiledeepersearchandadditionalsemanticvariantsoffercomplementaryimprovements.PreviewDiffdemonstratesthatmultimodalfeedbackismostusefulnotonlyasafinalverifier,butasanactivecontrollerinsidethedenoisingprocess.

View arXiv pageView PDFProject pageAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.36199 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.36199 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.36199 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention

arXiv cs.AI

PreDiff-LM proposes a hybrid attention mechanism that preserves causal attention for prompt tokens and bidirectional attention for masked target tokens, enabling adaptation of pretrained autoregressive models for discrete masked diffusion language modeling, achieving improvements in perplexity and downstream tasks over prior diffusion baselines.

PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models

Hugging Face Daily Papers

PerceptionDLM introduces a multimodal diffusion language model that enables parallel region perception via structured attention masking and efficient prompting, achieving faster inference without sacrificing caption quality. Experiments show competitive performance with substantial speed improvements for multi-region perception tasks.

Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models

arXiv cs.CL

Proposes AdaLook, an adaptive multi-step lookahead decoding framework for masked diffusion language models that dynamically determines rollout depth and branch expansion based on candidate-score variance, achieving better accuracy-decoding steps trade-off compared to existing one-step lookahead decoding methods.