Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression

Hugging Face Daily Papers Papers
reward-modeling latent-reasoning efficient-inference compression rlhf evaluation

Summary

LatentGRM 提出了一种基于语义分块与压缩重建的潜在评估框架，通过在连续隐空间中完成 rubric 引导的成对偏好判断，将评估轨迹压缩 8.9–9.2 倍并把推理时间减少 6.1–7.0 倍，同时保持与显式 SFT 评委相当的偏好准确率。

Reward modeling often requires jointly representing and reasoning over multiple evaluation criteria, yet verbalizing this process token by token can incur substantial inference cost. Recent work on latent reasoning suggests that continuous states may support this computation more compactly. We introduce LatentGRM, a latent evaluation framework built on semantic chunking, compression, and reconstruction. By using the structure of rubric-guided evaluations to guide compression, LatentGRM learns compact continuous trajectories that support autonomous pairwise judgments without generating textual assessments. A separate interpreter reconstructs evaluation text from these trajectories, providing an offline view of the information retained under compression. Under matched training data and backbones, LatentGRM achieves competitive aggregate preference accuracy relative to explicit Supervised Fine-Tuning (SFT) judges at both 4B and 8B scales. Across four benchmark domains, LatentGRM-8B compresses evaluation trajectories by 8.9--9.2x and reduces total judge inference time by 6.1--7.0x at vote@5. Controlled rubric interventions show that criterion-dependent preference information is carried through the latent sequence. Together, these results demonstrate that continuous latent evaluation can substantially reduce inference cost while preserving competitive judgment quality.
Original Article
View Cached Full Text

Cached at: 10/12/26, 02:37 AM

Paper page - Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression

Source: https://huggingface.co/papers/2610.09788

Abstract

Rewardmodelingoftenrequiresjointlyrepresentingandreasoningovermultipleevaluationcriteria,yetverbalizingthisprocesstokenbytokencanincursubstantialinferencecost.Recentworkonlatentreasoningsuggeststhatcontinuousstatesmaysupportthiscomputationmorecompactly.WeintroduceLatentGRM,alatentevaluationframeworkbuiltonsemanticchunking,compression,andreconstruction.Byusingthestructureofrubric-guidedevaluationstoguidecompression,LatentGRMlearnscompactcontinuoustrajectoriesthatsupportautonomouspairwisejudgmentswithoutgeneratingtextualassessments.Aseparateinterpreterreconstructsevaluationtextfromthesetrajectories,providinganofflineviewoftheinformationretainedundercompression.Undermatchedtrainingdataandbackbones,LatentGRMachievescompetitiveaggregatepreferenceaccuracyrelativetoexplicitSupervisedFine-Tuning(SFT)judgesatboth4Band8Bscales.Acrossfourbenchmarkdomains,LatentGRM-8Bcompressesevaluationtrajectoriesby8.9--9.2xandreducestotaljudgeinferencetimeby6.1--7.0xatvote@5.Controlledrubricinterventionsshowthatcriterion-dependentpreferenceinformationiscarriedthroughthelatentsequence.Together,theseresultsdemonstratethatcontinuouslatentevaluationcansubstantiallyreduceinferencecostwhilepreservingcompetitivejudgmentquality.

View arXiv pageView PDFGitHub1Add to collection

Get this paper in your agent:

hf papers read 2610\.09788

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.09788 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.09788 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.09788 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

MetaRubric: Learning to Reward for Rubric-Based Reinforcement Learning

Hugging Face Daily Papers

MetaRubric 通过交替进行证据感知的策略优化与响应引导的评分标准适配，解决基于评分标准的强化学习中「空洞奖励（Vacuous Credit）」问题，在 PubMedQA 上比静态评委 GRPO 提升 6.00–20.40 个百分点。

Rubrics from the Task, Verdicts from Evidence: Pointwise Rubric Rewards with Tool-Augmented Judges

Hugging Face Daily Papers

本文提出 ReVeR，一种点式（pointwise）rubric 奖励模型，仅通过分析指令本身生成硬性规则与分级原则，并借助工具增强的独立评判来评估响应，避免了从候选响应中推导标准所引入的偏置。实验表明 ReVeR 在偏好准确率、测试时扩展及下游策略优化方面优于成对 rubric 奖励模型。