Think Before You Score: Thinking Reward Model for Visual Generation

Hugging Face Daily Papers Papers

Summary

This paper introduces the Thinking Reward Model (TRM) that explicitly formulates case-adaptive rubrics for evaluating visual generation, achieving state-of-the-art performance among open-source reward models.

Visual reward models are essential for evaluating and improving visual generation models, yet existing approaches typically map task conditions and candidate outputs directly to scalar rewards, leaving implicit what should be evaluated for each individual case. We introduce Think Before You Score, a paradigm that explicitly determines what matters for each case before judging how well the candidate performs. Following this principle, we propose the Thinking Reward Model (TRM), which formulates case-adaptive rubrics, performs rubric-guided assessment, and produces fine-grained pointwise rewards. We further observe that conventional pairwise preference optimization can induce score polarization, and introduce Pairwise Dual-Group Relative Policy Optimization (PD-GRPO), which leverages pairwise supervision to improve reward discrimination while preserving fine-grained pointwise scoring. Extensive experiments on image generation and editing reward-modeling benchmarks demonstrate that TRM achieves state-of-the-art performance among open-source reward models while remaining highly competitive with proprietary alternatives. Moreover, using TRM as a reward for reinforcement learning consistently improves diverse visual generation models, demonstrating that its fine-grained, case-adaptive rewards translate into effective optimization signals for visual generation.
Original Article
View Cached Full Text

Cached at: 09/30/26, 04:22 AM

Paper page - Think Before You Score: Thinking Reward Model for Visual Generation

Source: https://huggingface.co/papers/2609.37372 Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

Visualrewardmodelsareessentialforevaluatingandimprovingvisualgenerationmodels,yetexistingapproachestypicallymaptaskconditionsandcandidateoutputsdirectlytoscalarrewards,leavingimplicitwhatshouldbeevaluatedforeachindividualcase.WeintroduceThinkBeforeYouScore,aparadigmthatexplicitlydetermineswhatmattersforeachcasebeforejudginghowwellthecandidateperforms.Followingthisprinciple,weproposetheThinkingRewardModel(TRM),whichformulatescase-adaptiverubrics,performsrubric-guidedassessment,andproducesfine-grainedpointwiserewards.Wefurtherobservethatconventionalpairwisepreferenceoptimizationcaninducescorepolarization,andintroducePairwiseDual-GroupRelativePolicyOptimization(PD-GRPO),whichleveragespairwisesupervisiontoimproverewarddiscriminationwhilepreservingfine-grainedpointwisescoring.Extensiveexperimentsonimagegenerationandeditingreward-modelingbenchmarksdemonstratethatTRMachievesstate-of-the-artperformanceamongopen-sourcerewardmodelswhileremaininghighlycompetitivewithproprietaryalternatives.Moreover,usingTRMasarewardforreinforcementlearningconsistentlyimprovesdiversevisualgenerationmodels,demonstratingthatitsfine-grained,case-adaptiverewardstranslateintoeffectiveoptimizationsignalsforvisualgeneration.

View arXiv pageView PDFProject pageGitHub16Add to collection

Get this paper in your agent:

hf papers read 2609\.37372

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.37372 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.37372 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.37372 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

MatrixReward: Reward from Rubric Matrix for Open-Ended Generation

arXiv cs.LG

MatrixReward proposes constructing rewards for open-ended generation from a rollout-by-rubric win-rate matrix, using column spread and correlation statistics to derive data-dependent rubric weights and ideal profiles. Evaluated with Qwen3-8B on four benchmarks, it achieves 63.02 average score, outperforming the strongest baseline by ~2%.

Think, then Score: Decoupled Reasoning and Scoring for Video Reward Modeling

Hugging Face Daily Papers

This paper introduces DeScore, a video reward model that decouples reasoning and scoring processes to improve training efficiency and generalization. It addresses the limitations of existing discriminative and generative reward models by using a 'think-then-score' paradigm with multimodal large language models.