Pushing Biomolecular Utility-Diversity Frontiers with Supergroup Relative Policy Optimization
Summary
This paper introduces SGRPO, a policy optimization framework that improves biomolecular generation by incorporating set-level diversity rewards alongside utility. It demonstrates improved utility-diversity trade-offs in tasks such as small-molecule and protein design.
View Cached Full Text
Cached at: 05/12/26, 07:33 AM
Paper page - Pushing Biomolecular Utility-Diversity Frontiers with Supergroup Relative Policy Optimization
Source: https://huggingface.co/papers/2605.08659
Abstract
SGRPO is a policy optimization framework that enhances biomolecular generation by directly incorporating set-level diversity rewards, improving both utility and diversity across multiple design tasks.
Biomolecular generatorsare often adapted withreward feedbackto improve task-specific utility, but pushing utility alone can concentrate generation on a narrow family of candidates. Maintaining diversity is difficult because sample diversity is a set-level property. We introduceSupergroup Relative Policy Optimization(SGRPO), a flexibleGRPO-style framework that directly constructs rewards fromset-level diversity. For each condition, SGRPOsamples a supergroup of candidate sets, compares their diversity under the same condition, and redistributes the group diversity reward to individual rollouts through leave-one-out diversity contributions before combining it with rollout-level utility. This design decouples SGRPOfrom a particular generator, utility reward, or diversity metric, and allows instantiation with differentGRPO-style approaches. We evaluate SGRPOonde novo small-molecule design,pocket-based small-molecule design, andde novo protein design, instantiating it with bothGRPOand Coupled-GRPOacross autoregressive anddiscrete diffusion generators. Across decoding sweeps, SGRPOexpands the utility-diversity Pareto frontier and achieves the best frontier-level metrics relative to pretrained generators,GRPO, and memory-assistedGRPOwhen applicable. Our analyses further show that directset-level diversityrewards remain effective with small groups and help preserve broader generation-distribution coverage during post-training. The code is available at https://github.com/IDEA-XL/SGRPO.
View arXiv pageView PDFGitHub1Add to collection
Get this paper in your agent:
hf papers read 2605\.08659
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.08659 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.08659 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.08659 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Multi-module GRPO: Composing Policy Gradients and Prompt Optimization for Language Model Programs
The paper introduces mmGRPO, a multi-module extension of Group Relative Policy Optimization (GRPO) that improves accuracy in modular AI systems by optimizing language model calls and prompts. It reports an average 11% accuracy improvement across various tasks and provides an open-source implementation in DSPy.
BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization
BiasGRPO proposes a framework using Group Relative Policy Optimization (GRPO) to stabilize social bias mitigation in LLMs by normalizing rewards across sampled completions, outperforming DPO and PPO on multiple benchmarks. The authors also release a compute-efficient bias reward model designed for integration into multi-objective RLHF pipelines.
Group Entropy-Controlled Policy Optimization
This paper proposes Group Entropy-Controlled Policy Optimization (GEPO), a lightweight extension to GRPO that uses group entropy to perform entropy-conditioned asymmetric advantage shaping, addressing heterogeneous entropy regimes across tasks during RL-based alignment of LLMs. Experiments show consistent improvements over GRPO and recent entropy-controlled methods across multiple benchmarks.
F-GRPO: Factorized Group-Relative Policy Optimization for Unified Candidate Generation and Ranking
F-GRPO proposes a factorized group-relative policy optimization framework that unifies candidate generation and ranking in a single autoregressive LLM, addressing credit assignment issues and improving top-ranked performance across sequential recommendation and multi-hop QA benchmarks.
Active-GRPO: Adaptive Imitation and Self-Improving Reasoning for Molecular Optimization
Active-GRPO introduces an adaptive imitation and self-improving reasoning framework that dynamically decides when to imitate references and when to reinforce the model's own discoveries for molecular optimization, achieving statistically significant improvements over previous methods on the TOMG-Bench-MolOpt benchmark.