Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation
Summary
Introduces IG-Bench, a benchmark for scientific lineage reasoning and idea generation, representing scientific works as Idea Genome objects. Evaluates 14 LLM-based scientists, finding a compositional bottleneck with the strongest system achieving only 27.3% exact accuracy on lineage reasoning.
View Cached Full Text
Cached at: 07/10/26, 06:17 AM
Paper page - Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation
Source: https://huggingface.co/papers/2607.08758 Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
A benchmark for scientific lineage reasoning and idea generation is introduced, organizing scientific works as genetic-like Idea Genome objects and evaluating both reasoning and generation capabilities.
Scientific ideas rarely start from a blank page. They inherit mechanisms, repair known limitations, and recombine pieces of earlier work, much like biological genomes. Current benchmarks still say little about whether AI systems can follow this inheritance structure. We present IdeaGene-Bench (IG-Bench), a benchmark for scientificlineage reasoningand lineage-groundedidea generation.IG-Benchis organized around theIdeaGene framework: each paper or proposal is represented as a set of minimal, typed, evidence-groundedIdea Genome objects, and aGenomeDiffaligns these objects to record inheritance, mutation, loss, external import, and novel insertion under six operationalevolutionary dynamics. The benchmark contains 1,961 golden lineage traces, 1,085 curatedIdea Genome objects, and 920 pairwiseGenomeDiffrecords across 10 scientific domains. It supports two evaluations.IG-Exam(42 task types, 1,029 instances) tests closed-formlineage reasoningacross Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification.IG-Arenaevaluates generation with a lineage-conditionedPopulation-Evolution Score(PES), asking whether a proposal can be inserted as a coherent descendant of a given lineage population: it should inherit the rightIdea Genome objects, vary meaningfully from nearby work, and offer selection value for future research. Experiments on 14LLM-based scientistsexpose a compositional bottleneck. The strongest system reaches only 27.3% exact accuracy onlineage reasoning, and structured lineage context reshuffles system rankings rather than helping every participant uniformly.
View arXiv pageView PDFProject pageGitHub2Add to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.08758 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.08758 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.08758 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
GENEB: Why Genomic Models Are Hard to Compare
GENEB is a large-scale diagnostic benchmark that evaluates 40 genomic foundation models across 100 tasks in 13 functional categories under a unified probing protocol, exposing that aggregate leaderboards are unstable and that architectural alignment often outweighs model scale. The work addresses the fragmented evaluation landscape in genomic machine learning, analogous to what MTEB did for NLP.
Agentic-Ideation: Sample Efficient Agentic Trajectories Synthesis for Scientific Ideation Agents
Proposes Agentic-Ideation, a framework for efficient synthesis of agentic trajectories to train LLMs for scientific ideation, achieving over 10x improvement in sample efficiency and outperforming existing workflow-based baselines.
IsoSci: A Benchmark of Isomorphic Cross-Domain Science Problems for Evaluating Reasoning versus Knowledge Retrieval in LLMs
Introduces IsoSci, a benchmark of isomorphic cross-domain science problem pairs that separates reasoning ability from domain knowledge retrieval in LLM evaluation. The study finds that 91.3% of reasoning-mode gains are knowledge-dependent, challenging common assumptions about chain-of-thought reasoning.
Measuring the Gap Between Human and LLM Research Ideas
This research paper introduces a framework to measure the distributional gap between human-generated and LLM-generated research ideas, finding that LLM ideas are concentrated around specific opportunity patterns and synthesis methods, while human ideas are more diverse.
IdeaTrail: Full-Process Agent Trajectories for Scientific Ideation
IdeaTrail is a dataset of multi-turn process trajectories for scientific ideation, synthesizing research processes from evidence gathering to proposal construction using a Generator–Advisor loop to ensure grounding.