Training Reasoning Models to Generate Diverse Scientific Ideas
Summary
Researchers propose a set-level reinforcement learning objective that trains language models to generate diverse, high-quality, document-grounded scientific ideas, with an 8B model outperforming Claude Opus 4.8 and GPT-6 Astra on creative utility.
View Cached Full Text
Cached at: 10/21/26, 07:06 AM
Paper page - Training Reasoning Models to Generate Diverse Scientific Ideas
Source: https://huggingface.co/papers/2610.22324
Abstract
Researchers propose a set-level reinforcement learning objective to train language models for diverse, high-quality, document-grounded idea generation, achieving stronger creative utility than larger proprietary models.
One of the first steps of a research pipeline is generating ideas. Large Language Models (LLMs) are increasingly used to automate this step, but most of this work optimizes the quality of individual ideas, while overlooking diversity. In our work, we studydocument-grounded ideation, focusing on increasing the diversity of LLM generated ideas and ensuring they are high quality. Given prior papers, a model must propose a set of ideas that are each high-quality and clearly distinct from one another, while also being grounded in the input documents. To optimize for this, we propose training with aset-level reinforcement learningobjective. In our setup, in a single generation, the model writes K structuredidea cardsgrounded in the source papers, and the set is scored jointly, where each idea’s quality is weighted by its distance from the ideas written before it. This teaches the model to steer each new idea away from its predecessors. We show that our trained 8B model achieves highercreative utility(a measure combining quality and diversity) than Claude Opus 4.8 and GPT-6 Astra on ourcontamination-free benchmark. In a pilot human study with four experts, our sets are judged as promising as Claude Opus 4.8’s and contain marginally more unique ideas. When a coding agent runs proof-of-concept experiments on the ideas, our model’s success@k is comparable to Claude Opus 4.8’s, so we see that the added diversity does not cost any executability.
View arXiv pageView PDFProject pageGitHub2Add to collection
Get this paper in your agent:
hf papers read 2610\.22324
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.22324 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.22324 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.22324 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
@omarsar0: Great application of small language models from Salesforce and colleagues. AI-scientist pipelines are well known to str…
Salesforce and Stony Brook researchers train an 8B reasoning model with a set-level reinforcement learning objective to generate diverse, high-quality scientific ideas, achieving higher creative utility than Claude Opus 4.8 and GPT-6 Astra on a contamination-free benchmark.
Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation
This paper explores teaching language models to forecast the empirical success of research ideas by comparing pairs of ideas. Using a dataset of 11,488 idea pairs from PapersWithCode, the authors show that fine-tuning (SFT) boosts accuracy to 77.1%, outperforming GPT-5, and reinforcement learning with verifiable rewards achieves 71.35% with interpretable reasoning.
Assessing the Creativity of Large Language Models: Testing, Limits, and New Frontiers
This paper systematically evaluates human creativity tests for LLMs and finds they fail to predict scientific ideation. It introduces the DRAT, a new test that combines convergent and divergent thinking to reliably predict scientific ideation ability in language models.
Mid-Training with Self-Generated Data Improves Reinforcement Learning in Language Models
This paper investigates how using diverse self-generated data during mid-training improves the effectiveness of Reinforcement Learning in Large Language Models, particularly for reasoning tasks.
Reinforcing Agentic Creativity in Scientific Ideation with Night Science
This paper introduces AI Night-Scientist, a framework using reinforcement learning to enhance creative reasoning in large language models for scientific ideation, significantly improving idea diversity and impact.