Think Before You Score: Thinking Reward Model for Visual Generation
Summary
This paper introduces the Thinking Reward Model (TRM) that explicitly formulates case-adaptive rubrics for evaluating visual generation, achieving state-of-the-art performance among open-source reward models.
View Cached Full Text
Cached at: 09/30/26, 04:22 AM
Paper page - Think Before You Score: Thinking Reward Model for Visual Generation
Source: https://huggingface.co/papers/2609.37372 Authors:
,
,
,
,
,
,
,
,
,
,
Abstract
Visualrewardmodelsareessentialforevaluatingandimprovingvisualgenerationmodels,yetexistingapproachestypicallymaptaskconditionsandcandidateoutputsdirectlytoscalarrewards,leavingimplicitwhatshouldbeevaluatedforeachindividualcase.WeintroduceThinkBeforeYouScore,aparadigmthatexplicitlydetermineswhatmattersforeachcasebeforejudginghowwellthecandidateperforms.Followingthisprinciple,weproposetheThinkingRewardModel(TRM),whichformulatescase-adaptiverubrics,performsrubric-guidedassessment,andproducesfine-grainedpointwiserewards.Wefurtherobservethatconventionalpairwisepreferenceoptimizationcaninducescorepolarization,andintroducePairwiseDual-GroupRelativePolicyOptimization(PD-GRPO),whichleveragespairwisesupervisiontoimproverewarddiscriminationwhilepreservingfine-grainedpointwisescoring.Extensiveexperimentsonimagegenerationandeditingreward-modelingbenchmarksdemonstratethatTRMachievesstate-of-the-artperformanceamongopen-sourcerewardmodelswhileremaininghighlycompetitivewithproprietaryalternatives.Moreover,usingTRMasarewardforreinforcementlearningconsistentlyimprovesdiversevisualgenerationmodels,demonstratingthatitsfine-grained,case-adaptiverewardstranslateintoeffectiveoptimizationsignalsforvisualgeneration.
View arXiv pageView PDFProject pageGitHub16Add to collection
Get this paper in your agent:
hf papers read 2609\.37372
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.37372 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.37372 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.37372 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
MatrixReward: Reward from Rubric Matrix for Open-Ended Generation
MatrixReward proposes constructing rewards for open-ended generation from a rollout-by-rubric win-rate matrix, using column spread and correlation statistics to derive data-dependent rubric weights and ideal profiles. Evaluated with Qwen3-8B on four benchmarks, it achieves 63.02 average score, outperforming the strongest baseline by ~2%.
AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment
AutoRubric-T2I automatically generates and selects explicit rubrics to guide Vision-Language Model judges for text-to-image generation, achieving high-quality reward signals with minimal human annotation and improving generation quality in downstream tasks.
Think, then Score: Decoupled Reasoning and Scoring for Video Reward Modeling
This paper introduces DeScore, a video reward model that decouples reasoning and scoring processes to improve training efficiency and generalization. It addresses the limitations of existing discriminative and generative reward models by using a 'think-then-score' paradigm with multimodal large language models.
Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteria
This paper introduces the Auto-Rubric as Reward (ARR) framework, which externalizes implicit preference knowledge into explicit rubrics for multimodal alignment. It proposes Rubric Policy Optimization (RPO) to stabilize policy gradients, achieving better performance in text-to-image and image editing tasks.
RewardVerse: Rubric-Guided Policy Optimization for Video Reward Modeling
RewardVerse presents a rubric-based framework for video reward modeling to mitigate scalar drift and provide stable evaluation criteria, enhancing reinforcement learning in video generation.