建模真实的研究轨迹：研究构想生成的图谱视角

# Modeling Authentic Research Trajectories: A Graph View of Research Ideation
Source: [https://arxiv.org/html/2610.27337](https://arxiv.org/html/2610.27337)
Ruochen Liruochen\.li@utdallas\.eduAffiliation:Zhiyu Chenzhiyu\.chen2@utdallas\.eduAffiliation:Department of Computer Science, University of Texas at Dallas

###### Abstract

The research idea presented in a published paper is rarely the initial idea of the project\. Between the two lies an iterative, closed\-loop cognitive process, in which researchers conduct experiments, derive insights from the results, identify new challenges, explore the literature, and revise the idea, with each transition motivated by evidence and reasoning that they can articulate\. We refer to the path that one idea takes through this process, with the evidence and reasoning behind each step, as its authentic research trajectory\. Although it underlies every scientific contribution, it is rarely recorded\. Existing LLM\-based research agents generate an idea once, refine it against reviews, retrieved literature, or the results of their own experiments, or search over candidate ideas and experiments\. In all of them the object that evolves is the text of the idea, yet the evidence and reasoning that connected one version to the next are discarded\. In this paper, we propose the Research Progress Graph \(RPG\), a new representation of authentic and complete research trajectories\. An RPG records the evolution of an idea as typed research states connected by cognitive transitions, each annotated with a temporal index and the reasoning process behind it\. The representation thus makes the evolution of the idea an explicit object of study, preserving the intermediate states, their temporal order, and the reasoning that connects each piece of evidence to the resulting idea update\. We ground the representation in authentic research trajectories whose steps their authors justify in their annotations and papers\. We compare the representations of thirty\-five existing research agents with the RPG and find that each is an incomplete RPG; none maintains the state that changed, the evidence that changed it, and the reasoning that connects the two as a single evolving record\. Modeling research ideation with the RPG changes how research agents should be designed, evaluated, and trained\. Agents predict the next research state instead of a complete idea, ideas are evaluated as trajectories instead of final texts, and authentic trajectories serve as training data\. Together, these directions shift research agents from generating static research ideas to explicitly modeling the evolution of research ideas\.

## 1Introduction

Research Progress GraphRQICERINCLSLLINCINICER11234567891010111213evidence from the worldevidence from the literaturea state is read again later \(10\)numbers: temporal indexreasoning process on transition 11: why this evidence changes this part of the ideacognition \(RQ, I, IN, C\)literature \(LS, L\)world \(CE, R\)untyped text or scoreExisting representationsprompt, papersI\(1\) one\-shot generationIreviewregenerate\(2\) revision against self\-critiqueIfixedCECERR\(3\) fixed idea, tree of experimentsIRIscore\(4\) results to a new idea through a scoreRQICI\(5\) typed idea graph, no evidence entersFigure 1:The Research Progress Graph \(top\) and the structures by which existing research agents represent ideation \(bottom\)\. An RPG records typed states, the temporal order of the transitions between them, and the reasoning process behind each transition, with results and literature entering as states that change the idea\. Each existing structure keeps a subset of these elements and can be viewed as an incomplete RPG\.Scientific ideation is an iterative, closed\-loop cognitive process, and the research idea presented in a published paper is rarely the initial idea of the project\. Instead, the idea evolves through a continuous cycle of literature exploration, hypothesis formulation, implementation, experimentation, insight generation, challenge identification, and idea revision, in which researchers recognize patterns in heterogeneous evidence and gradually transform an incomplete idea into a mature scientific contribution\. Each step of this cycle is driven by evidence, whether an experimental result or a paper, and by a reasoning process that the researcher can articulate\. We refer to the complete sequence of research states and transitions, together with the evidence and reasoning behind each transition, as the*authentic research trajectory*of an idea\. Figure[1](https://arxiv.org/html/2610.27337#S1.F1)\(top\) shows one such trajectory\. This trajectory is only one example\. Trajectories are free\-form, in that the states an idea passes through, their order, and the number of rounds all vary from one idea to the next \(§[4](https://arxiv.org/html/2610.27337#S4)\)\.

Existing LLM\-based research agents represent this process only partially, and in an artificial form\. Early work formulated ideation as one\-shot generation, in which an LLM produces a research idea from a topic or a set of papers\([Si et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib18)\), so this formulation represents only the final idea\. Other agents refine the idea iteratively against reviewer feedback\([Baek et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib23)\), retrieved literature\([Wang et al\., 2024b](https://arxiv.org/html/2610.27337#bib.bib27);[Hu et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib29)\), or the results of experiments the agent runs itself\([Yuan et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib42);[Weng et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib43)\), and the most recent agents organize the refinement as a search over candidate ideas or experiments\([Yamada et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib31);[Yang et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib51)\)\. These agents differ in what they feed back, but they all represent the same object, the current text of the idea, and each round turns it into the next idea \(Figure[1](https://arxiv.org/html/2610.27337#S1.F1), bottom\)\. In most of them, a round takes the whole idea as input and returns a new idea, while the evidence that caused the change and the reasoning that connected the two versions stay in a prompt that is discarded afterward\. Insights, challenges, and the literature searched for in response are rarely kept as objects of their own\. Even when an agent records a result or a reason, it stores only a score or a summary\. Each representation therefore keeps one part of the trajectory, whether the versions of the idea, the evidence, or the reasoning, and none records a change to the idea together with the evidence and reasoning behind it\.

In this paper, we propose the Research Progress Graph \(RPG\), a new representation of authentic and complete research trajectories\. An RPG records the evolution of an idea as a graph of typed research states connected by cognitive transitions, each annotated with a temporal index and the reasoning process behind it\. The states are of eight types: research question, idea update, insight, challenge, literature search, literature, code and experiment plan, and experimental result\. The representation makes the evolution of an idea an explicit object of study\. It preserves the intermediate states that a paper omits and an agent discards, their chronological order, and the reasoning that connects each piece of evidence to the resulting idea update, so that a trajectory can be recorded as it happens, read by the next step, learned from, and evaluated\. The RPG is free\-form, in that it prescribes no order of states and does not require a round to visit every type, and minimal, in that none of the eight state types can be removed without losing a stage of the ideation cycle, so that an RPG can be annotated by hand and generated by a model\.

We ground the representation in authentic research trajectories from machine learning and natural language processing and use it to examine existing research agents and the design of future ones\. §[2](https://arxiv.org/html/2610.27337#S2)describes how research ideas evolve and what existing research agents model of the process, and §[3](https://arxiv.org/html/2610.27337#S3)defines the RPG\. We then use it in three ways\. First, we represent as RPGs eight authentic trajectories whose steps their authors justify in annotations or in their papers \(§[4](https://arxiv.org/html/2610.27337#S4)\)\. In each of them, the decisive change is confined to one state, caused by a specific result or paper, and explained by the reasoning of the authors, which the RPG records on the transition\. The eight state types cover every state of each trajectory\. Second, we compare the representations of thirty\-five existing research agents with the RPG \(§[5](https://arxiv.org/html/2610.27337#S5)\)\. Each is an incomplete RPG, and the comparison locates for each agent precisely which states and transitions it lacks\. Third, we show that modeling research ideation with the RPG changes how research agents should be designed, evaluated, and trained \(§[6](https://arxiv.org/html/2610.27337#S6), §[7](https://arxiv.org/html/2610.27337#S7)\)\. Instead of a complete idea, an agent predicts the next research state from the trajectory so far\. An idea is evaluated as a trajectory instead of a final text, and a blind study of 26 idea pairs shows that a judge reading only the final text rates an untested rewrite above the version its authors carried out in ten of the pairs, and that giving the judge retrieval does not prevent this\. Authentic trajectories, recorded while a project runs, serve as training data\. §[8](https://arxiv.org/html/2610.27337#S8)discusses open questions and limitations\.

## 2Background: Research Idea Evolution

### 2\.1Scientific discovery as idea evolution

The view that research ideas are developed rather than generated has a long history in cognitive science and the history of science\.[Boden \(2004\)](https://arxiv.org/html/2610.27337#bib.bib17)distinguishes three kinds of creativity, combinational, exploratory, and transformational, and in each of them a new idea is built from what already exists, by combining familiar ideas, exploring an existing conceptual space, or transforming that space\. Computational models describe discovery as a step\-by\-step heuristic search through a space of possible hypotheses\([Langley et al\., 1987](https://arxiv.org/html/2610.27337#bib.bib58)\)\. Laboratory studies of scientific reasoning\([Klahr and Dunbar, 1988](https://arxiv.org/html/2610.27337#bib.bib59)\), observations of real research laboratories\([Dunbar, 1997](https://arxiv.org/html/2610.27337#bib.bib60)\), and the history of a single mathematical idea\([Lakatos, 1976](https://arxiv.org/html/2610.27337#bib.bib61)\)describe a common cycle, in which a researcher proposes a conjecture, tests it, and revises it according to what the test shows\. The trajectories in §[4](https://arxiv.org/html/2610.27337#S4)show this cycle in more detail\. A researcher reads the literature, proposes a hypothesis, implements and tests it, draws insights from the results, identifies the challenges that these insights reveal, and revises the idea, often after a new literature search aimed at the challenge\. Figure[1](https://arxiv.org/html/2610.27337#S1.F1)\(top\) shows one round of this cycle\. Two observations follow from it\. First, the evidence that changes an idea arrives at specific moments, as an experimental result or a paper\. Second, each revision is local and has a reason\. It changes one part of the idea, and the researcher can explain how the evidence led to that change\. A representation of how an idea evolves must therefore keep the states the idea passes through, their order in time, and the reasoning that links each piece of evidence to the change it caused\.

This process is largely missing from the published record\. A paper is written to present a result, not the path that led to it\([Medawar, 1963](https://arxiv.org/html/2610.27337#bib.bib62)\)\. It reports the final idea and the experiments that support it, and it leaves out the intermediate reasoning, the implementation decisions, and the attempts that failed\. The trajectory of an idea therefore has to be reconstructed, and reconstruction faces three problems\. First, papers rarely say in what order the intermediate states occurred\. Second, only the researchers know the reasons for each step\. Third, any reconstruction happens after the fact, and people have limited access to their own past reasoning\([Nisbett and Wilson, 1977](https://arxiv.org/html/2610.27337#bib.bib16)\)\. The order of the states and the evidence can be checked against code versions, experiment logs, and publication dates, but the stated reasons cannot\. For this reason, each trajectory in this paper is annotated by its authors, traced and then confirmed by its authors, or traced from what its authors wrote in papers, reports, and blog posts\. Existing datasets for hypothesis discovery reconstruct only the first step of a trajectory\. They record, for each paper, the research background, the inspirations, and the hypothesis that combines them, annotated by PhD students for a few dozen papers\([Yang et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib2);[Yang et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib3)\)or extracted by an LLM from over 100,000 papers\([Yang and Bing, 2026](https://arxiv.org/html/2610.27337#bib.bib4)\), but not the experiments, insights, and revisions that come after the hypothesis\. Recording an RPG while the research is in progress avoids all three problems of reconstruction\.

### 2\.2LLM\-based research agents

LLMs are now used to propose research ideas and to automate parts of the research process\. The simplest approach generates an idea in a single pass, by prompting an LLM with a problem description, a topic, or a set of papers\([Si et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib18)\)\. Other agents revise the idea over several rounds, and they have developed in three directions\. The first two change what drives the revision, feedback or literature, and the third changes how many candidates an agent keeps\.

The first direction revises the idea in response to feedback, first critiques of the text and later the results of experiments\. Outside ideation, Self\-Refine and Reflexion showed that a model can improve its output by iterating on its own critique\([Madaan et al\., 2023](https://arxiv.org/html/2610.27337#bib.bib64);[Shinn et al\., 2023](https://arxiv.org/html/2610.27337#bib.bib65)\)\. ResearchAgent applies this to ideas and revises them in response to the reviews and scores of reviewing agents\([Baek et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib23)\)\. The AI Scientist revises its idea through self\-reflection and then revises its experiments according to their results\([Lu et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib37)\)\. By late 2025, Dolphin, DeepScientist, ASI\-Arch, AlphaResearch, Kosmos, and freephdlabor use the results of experiments to produce the next round of ideas\([Yuan et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib42);[Weng et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib43);[Liu et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib45);[Yu et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib46);[Mitchener et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib44);[Li et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib47)\)\. The second direction gives the agent literature, first as retrieved papers and later as structured collections selected for the current idea\.[Swanson \(1986\)](https://arxiv.org/html/2610.27337#bib.bib63)proposed, long before LLMs, that new hypotheses can come from connecting literatures that do not cite each other\. MOOSE\-Chem, which draws this parallel, pairs a research background with inspiration papers not previously known to be related to it\([Yang et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib3)\)\. SciMON retrieves related work along several routes\([Wang et al\., 2024b](https://arxiv.org/html/2610.27337#bib.bib27)\), ResearchAgent builds a store of entities from the literature, and Chain of Ideas arranges papers into chains in the order in which the field developed\([Li et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib28)\)\. In these agents, the literature is a structured object rather than text added to a prompt\. The third direction also changes how the candidates are related\. Early agents keep one candidate and replace it in each round\. Tree search keeps several branches and expands the most promising one, and graph search also allows branches to merge\. Tree and graph search were first applied to reasoning steps\([Yao et al\., 2023](https://arxiv.org/html/2610.27337#bib.bib66);[Besta et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib67)\)and later to ideas and experiments\. Nova and IRIS build trees of ideas\([Hu et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib29);[Garikaparthi et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib34)\)\. AIDE and AI Scientist v2 build trees of code and experiments while the task or idea stays fixed\([Jiang et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib50);[Yamada et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib31)\)\. R&D\-Agent merges parallel traces and, in its second version, chooses parent nodes in a directed acyclic graph of solutions\([Yang et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib51)\)\.

Beyond ideation, end\-to\-end research agents combine LLMs with tools for literature search, coding, and the analysis of results, and coding agents increasingly carry out the implementation\([Wang et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib9)\)\. Across the three directions, agents have gained richer inputs and more candidates to compare, but they judge each revision by its outcome\. The insights, challenges, and reasoning through which a researcher moves from one version of an idea to the next are represented only in part\. §[5](https://arxiv.org/html/2610.27337#S5)compares how each agent represents ideation\.

## 3The Research Progress Graph

The Research Progress Graph \(RPG\) represents the authentic research trajectory of an idea as a directed graph\. Its nodes are research states of eight types, namely the research question, idea updates, insights, challenges, literature searches, literature, code and experiment plans, and experimental results\. Its edges are the cognitive transitions by which the researcher moves from one or more states to the next, such as analyzing an experimental result, identifying a challenge, searching the literature, or integrating these signals into an idea update\. Each transition carries two annotations\. A*temporal index*gives the global chronological order of the transition, and each state inherits the index of the transition that produced it, so that the indices define the temporal progression of the project and the graph can be replayed as a timeline\. A*reasoning process*describes the cognitive step by which the researcher moved from the source states to the target state\. At the start of a project, the researcher provides a research question, optionally with a set of relevant papers, an initial idea, and an initial code and experiment plan with its first results, and these form the*initial RPG*\.

Figure[2](https://arxiv.org/html/2610.27337#S4.F2)shows the RPG of Case 1\. Its initial RPG holds the question, the pointer\-generator paper, the initial idea, its implementation, and the first result\. Transitions 4 to 11 take the project from that result to the final idea, and transitions 12 and 13 implement and run it\.

The representation is not tied to a field\. Its states and transitions are those of empirical research in general, and the trajectories and agents in this paper come from machine learning and natural language processing, the fields in which we can read the papers and code closely and have the authors confirm a trajectory\. Nor is the representation limited to ideation\. The code and experiment plan state and the experimental result state hold the implementation and the experiments of a project, so the same graph can represent the research process as a whole\. This paper centers on ideation, the part of the process that existing agents represent least \(§[5](https://arxiv.org/html/2610.27337#S5)\)\.

### 3\.1Research states

Table[1](https://arxiv.org/html/2610.27337#S3.T1)defines the eight state types with the running example of Case 1\. The types fall into three groups by where a state comes from\. The research question, idea updates, insights, and challenges are produced by the researcher’s own cognition\. Literature searches and literature are how the written record, and existing resources such as model and dataset repositories, enter the graph\. Code and experiment plans and experimental results are how the world, or a derivation that can be checked, enters it\. The eight types are minimal, in that none of them can be removed without losing a stage of the ideation cycle, and they have been sufficient for every trajectory we have examined\. They are not a closed ontology\. Finer distinctions, such as an assumption, a mechanism, or a decision between alternatives, can be introduced as subtypes without changing the graph\. A state records what was known or produced at a point in the project, not how it was produced\. The representation does not specify which model generated a state or whether a person or an agent produced it\.

Table 1:State types of an RPG, with the running example of Case 1 \(few\-shot table\-to\-text generation\)\.
### 3\.2Cognitive transitions

A transition connects one or more source states to one target state\. Its source states are the evidence for the target\. An insight drawn from a result has that result as a source, and an idea update justified by an insight and a paper has both as sources, so the evidence behind a step is read off the graph rather than stated separately\. Many transitions in the case trajectories have more than one source, and the note under each case figure names them\. The temporal index orders transitions, not states, which matters because a state can be the source of transitions long after it was produced\. In Figure[2](https://arxiv.org/html/2610.27337#S4.F2), Literature \#0 enters at the start of the project and is read again at transition 10\. The reasoning process annotated on a transition describes the cognitive step as the researcher took it, and it is specific to that step\. For the transition that produced Idea Update \#1 in Case 1, the reasoning records not that a pointer\-generator network and a pre\-trained language model were combined but why they were compatible\. The pointer\-generator is itself a switch between a language model and a copy distribution, so its language\-model half could be replaced while the switch and the copy half were kept\.

Grouped by the type of the target state, transitions are a small set of operations\.*Deriving an insight*reads one or more results, papers, or earlier states into an insight\.*Identifying a challenge*turns an insight, a result, or the research question into a challenge about one part of the current idea\.*Searching the literature*turns a challenge into a literature search whose query is recorded, and the search returns literature\.*Updating the idea*produces a new idea update from the previous one and from the insights and literature that justify the change; the update is localized, in that the new version differs from the previous one in named components and keeps the rest\.*Implementing and experimenting*produce a code and experiment plan and an experimental result from the current idea\. A*re\-reading*takes a state that is already in the graph, usually a paper, as the source of a new transition under a new challenge\. Re\-readings are common in the recorded trajectories and are how a graph reuses what it already holds\.

### 3\.3Idea evolution on the graph

Idea evolution is a path on the graph that returns to an idea update\. The initial RPG holds what the researcher starts with\. From there the project adds states and transitions until a new idea update is reached, and a mature idea is the result of one or more such rounds\. The RPG is*free\-form*\. It prescribes no order of state types, any type may follow any other, several sources may feed one target, and a round need not visit every type\. What the recorded trajectories most often show is the shape of Figure[1](https://arxiv.org/html/2610.27337#S1.F1)\(top\)\. An experimental result or a paper enters the graph, an insight is derived from it, the insight raises a challenge about one component of the idea, the challenge directs a literature search, the literature yields a further insight, and the insights justify a localized change to the idea\. The two states that enter from outside the researcher’s cognition, the result and the literature, are where the evidence arrives, and every change to the idea in the recorded trajectories traces back, usually through insights and challenges, to one of them\.

The trajectories depart from this shape in several ways, and the representation accommodates each without a new construct\. A round can re\-read a state already in the graph instead of searching, as transition 10 of Case 1 does\. A challenge can be answered by a derivation rather than by a paper or an experiment, as when the closed\-form optimum of the KL\-constrained objective let direct preference optimization train the policy on preferences without a separate reward model\([Rafailov et al\., 2023](https://arxiv.org/html/2610.27337#bib.bib15)\)\. A derivation that can be checked is recorded as a result state, since, like an experiment, it returns an outcome that can contradict what the researcher expected\. A result can come from people rather than from an experiment, as in Case 2, and an external resource rather than a paper can enter as a literature state, as the Hugging Face Hub does in Case 4\. And a trajectory can span several papers, with each earlier paper recorded as a literature state that the next paper reads, as Case 5 records the first paper of its group as a literature state of the second; the same holds when the papers come from different groups\. What the rounds share is that the new idea is a change to the previous one, that the change is caused by evidence that entered the graph, and that the reasoning behind it is recorded on the transition\.

### 3\.4Properties of the representation

An authentic research trajectory consists of the states an idea passes through, their order, and the evidence and reasoning behind each transition\. A representation of such a trajectory must preserve each of these, and the five properties below state what this requires\. The first three keep the parts of a trajectory that a paper omits, the intermediate states, their order, and the reasoning between them\. The fourth makes idea evolution a closed loop driven by evidence rather than a text revised against itself\. The fifth concerns who writes the graph, a researcher or an agent\. Each property is stated with what is lost without it\.

1. 1\.*Typed states\.*The trajectory passes through states of different kinds, and a question, an insight, a challenge, a paper, a plan, and a result play different roles in it\. Keeping them as distinct objects is what lets a later step act on one of them in particular\. Without types, a challenge is a sentence in a paragraph and cannot direct a search, and an insight cannot be checked against the result it was drawn from\.
2. 2\.*A temporal order that later steps read\.*The order of the states is part of the trajectory, since what the researcher knew at each step is what had entered the graph by then\. States and transitions are therefore kept in chronological order, and the next step takes the earlier ones as input\. A stored history that nothing reads does not have this property, because the order then plays no part in what happens next\.
3. 3\.*Reasoning bound to the change it explains\.*The reasoning behind a step is the part of the trajectory that only its author can supply, and it is specific to one change\. It is therefore annotated on the transition that produced the change, not kept as a review detached from any particular edit or in a chain of thought that is discarded at the end of the round\. Without this property, the record shows that the idea changed but not why, which is exactly what a published paper already shows\.
4. 4\.*Evidence enters the graph and changes the idea\.*In an authentic trajectory, every change to the idea traces back to a result or a literature state\. After an initial idea exists, literature retrieved for it or results of running it lead, through insights and challenges, to a new idea update\. Self\-critique is not evidence, and a result that changes the code but not the idea does not count, since in neither case has the idea evolved in response to something that could contradict it\.
5. 5\.*Humans can intervene at any stage\.*The trajectories in this paper were produced by researchers, and in a project that runs an agent the researchers still edit, veto, or redirect the idea between steps\. A representation that held only an agent’s steps could record neither the researchers’ trajectories nor their interventions\. The RPG therefore does not distinguish who produced a state\. A coding agent can produce code and experiment plans and experimental results, a retriever can produce literature, an ideation agent can produce insights, challenges, literature searches, and idea updates, and a person can produce any of them\. The graph is the interface between them, so whether a person intervenes is a question of which transitions the person takes, not of whether a person is present\. The same holds across research groups\. A line of work handed from one group to another is one graph in which the hand\-overs are literature states, and updating another group’s idea is a transition that every follow\-up paper takes\.

These are properties of the representation rather than of the evidence we offer for it\. The trajectories of §[4](https://arxiv.org/html/2610.27337#S4)are taken from what their authors annotated or wrote, because only the authors can state the reasoning behind a transition\.

#### Why the structure is stored explicitly\.

A model with a long context window, or an agent with a memory store, could be given the whole history of an idea as text and left to find the structure itself\. Such a transcript lacks three things that the properties above require\. It does not say which sentence is an assumption, which is a claim already tested, and which the last result contradicted, so a challenge cannot be addressed to one component and an insight cannot be checked against its source \(property 1\)\. It records order but not attribution, since it contains the result and the next version but not the edge saying that one caused the other \(properties 3 and 4\)\. Finally, it cannot be edited locally, so the operation the cases show to be decisive, replacing one component for a stated reason while keeping the rest, has no counterpart in regenerating a text from a longer prompt\. The RPG is compatible with long contexts, which are the natural way to present a linearized graph to a model\. What it requires is only that the structure be explicit in what is stored and read\.

## 4Case Studies: Authentic Research Trajectories

In this section we represent the trajectories of real ideas as RPGs\. The resulting RPGs show that real trajectories fit the representation without a new construct, which is the evidence for the claim that the representation is authentic and complete\. For each idea we traced the path from the initial idea to the published idea, typed every state, and drew the result as an RPG\. Table[2](https://arxiv.org/html/2610.27337#S4.T2)summarizes eight cases\. Five are presented in this section\. Case 1 is given in full as the worked example, and Cases 2 to 5 are each given as one paragraph and one figure\. Cases 6 to 8 are given in Appendix[A](https://arxiv.org/html/2610.27337#A1), which also contains the state and transition listings from which every figure is drawn\. For each case we ask whether the idea evolved through the cycle of §[2\.1](https://arxiv.org/html/2610.27337#S2.SS1), where evidence entered, and which parts of the trajectory an agent that regenerates the whole idea from a score or a summary could not have recorded\.

#### Scope of the evidence\.

Every case is an authentic trajectory, that is, a trajectory produced by the idea’s own authors, with the reason for each step taken from what they annotated or wrote\. This is required because the claim under test is that the reasoning behind an idea update is bound to the evidence that caused it, and only the authors can state what the reasoning was\. Cases 1 to 4 are annotated, or traced and confirmed, by their authors, Case 2 spans two papers by one group, and Case 4 spans two versions of one paper\. Cases 5 to 8 are traced from what their authors published\. Case 5 spans two papers by one group, linked by their shared title, authors, and citations, and of the cases in the appendix, Case 6 is built from its authors’ published narrative, Case 7 spans two papers by the same three authors, and Case 8’s report records two abandoned attempts with their reasons\.

Table 2:Eight real ideas as RPGs\. Cases 1 to 5 are presented in this section and Cases 6 to 8 in Appendix[A](https://arxiv.org/html/2610.27337#A1)\. Rounds counts the passes through the cycle of §[2\.1](https://arxiv.org/html/2610.27337#S2.SS1); Evidence lists the experimental result \(R\) and literature \(L\) states through which evidence entered\.CaseIdea typeInitial ideaPublished ideaRoundsEvidence1MethodPointer\-generator network for few\-shot table\-to\-textPre\-trained LM with a copy switch\([Chen et al\., 2020](https://arxiv.org/html/2610.27337#bib.bib1)\)1L \(pointer\-generator, re\-read\), R \(own pilot\), L \(GPT\-2\), R \(final\)2ApplicationPrompt GPT\-4 to role\-play a patientPATIENT\-Ψ\\Psi, GPT\-4 conditioned on clinician\-written cognitive models and evaluated by mental\-health experts and trainees\([Wang et al\., 2024c](https://arxiv.org/html/2610.27337#bib.bib8)\); then CBT\-Bench on the therapist’s side\([Zhang et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib55)\)4R \(expert interviews\), L \(CBT\), L \(transcripts\), R \(user study\), L \(prior CBT tasks\), R \(benchmark\)3Method \(ideation agent\)Fine\-tune an LLM on paper\-to\-idea pairsLDC: reward models per dimension, dimensional controllers, sentence\-level dynamic control\([Li et al\., 2024a](https://arxiv.org/html/2610.27337#bib.bib20)\)3L \(prompting\-based ideation\), L \(ResearchAgent’s iterative feedback\), L \(trade\-off in prior work\), R×\\times2, R with human judges, R \(later sweep\)4System \(research agent\)A prompted idea agent and an experiment agent that runs the source paper’s codeA ReAct experiment agent with retrieval and human feedback; the idea agent trained with fine\-tuning and reward\-model reinforcement learning\([Li et al\., 2024b](https://arxiv.org/html/2610.27337#bib.bib41)\)3L \(prior systems\), L \(MLAgentBench\), L \(ResearchAgent\), R \(agent drift\), L \(Hugging Face Hub\), R \(five tasks\), R \(single\-prompt baseline\), L \(the group’s idea\-generation method\), R \(retrained idea agent\)5Empirical insightApply the one\-shot RLVR lens to on\-policy distillation“Data\-overfed but algorithm\-starved”; inputs act as state generators\([Fu et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib6)\)3L \(one\-shot RLVR, Part I\), R×\\times36MethodFine\-tune the student on teacher samplesOn\-policy distillation with per\-token reverse KL, later used for recovery after mid\-training \(blog post,[Lu and Thinking Machines Lab, 2025](https://arxiv.org/html/2610.27337#bib.bib5)\)3\.5R \(SFT plateau\), L \(Qwen3 report\), L \(DAgger, GKD\), R×\\times67Evaluation of ideationA large blind human study of LLM and expert ideas\([Si et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib18)\)Execute a subset of the same ideas; LLM ideas lose more on all four metrics rated at both stages\([Si et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib19)\)1, a second left openL, R \(expert study\), R \(self\-analysis\), R \(execution study\), R \(attribution\)8MethodGRPO after SFT on mathematics\([Shao et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib53)\)DeepSeek\-R1: pure\-RL R1\-Zero, then cold start and multi\-stage RL; distillation for small models; two branches abandoned\([DeepSeek\-AI, 2025](https://arxiv.org/html/2610.27337#bib.bib54)\)3, two abandoned branches, one forkL \(own prior work\), R×\\times4
### 4\.1Case 1, worked example: few\-shot table\-to\-text generation

Research Question: few\-shot table\-to\-text generationLiterature \#0: pointer\-generator paperIdea Update \#0: pointer\-generator networkCode & Exp\. Plan \#0: train on 50–500 pairsExp\. Results \#0: copies values; connective text ungrammaticalInsight \#0: copying is learnable from few examples, language modeling is notChallenge \#1: how to obtain the language\-modeling skillLiterature Search \#1: few\-shot language modelingLiterature \#1: GPT\-2 and other pre\-trained LMsInsight \#1: the LM skill can come from pre\-trainingChallenge \#2: how to combine a pre\-trained LM with copyingInsight \#2: the pointer\-generator is a switch between an LM and a copy distributionIdea Update \#1: pre\-trained LM in the LM role; switch and copy keptCode & Exp\. Plan \#1: same training sizesExp\. Results \#1: 36\.1 BLEU at 200 examples on Humans11234567891010, re\-read111213Reasoning on transition 11: the pointer\-generator already contains a language model and a copy distribution joined by a learned switch\. Only the language\-modeling half failed with few examples, so replace that half with the pre\-trained model and keep the switch and the copy supervision\.Every transition carries a temporal index \(the numbers\) and the reasoning process behind it; one is shown\. Transition 1 has two sources and transition 10 re\-reads a state already in the graph\. The RPG prescribes no order of states; this is one recorded trajectory\.Figure 2:Case 1, few\-shot table\-to\-text generation, as an RPG\. One round, two experimental and two literature states of evidence\. Transitions are numbered by temporal index; transition 1 has two sources, transition 10 re\-reads a state already in the graph, and the reasoning process on transition 11 is shown\. Listing in Appendix[A\.1](https://arxiv.org/html/2610.27337#A1.SS1)\.Case 1 is a method idea\([Chen et al\., 2020](https://arxiv.org/html/2610.27337#bib.bib1)\), annotated, or traced and confirmed, by its authors\. The project started from the question of how to generate text from a table with only a few hundred training pairs, at the moment GPT\-2 had just appeared\. The initial idea was the natural one in 2019, a pointer\-generator network\([See et al\., 2017](https://arxiv.org/html/2610.27337#bib.bib57)\), since the output mostly copies values from the table\. The initial RPG therefore held a research question, a small literature set around the pointer\-generator, an idea, an implementation, and a first result\.

The first result was where outside information entered\. With 50 to 500 examples the pointer\-generator copied values reasonably but produced ungrammatical connective text\. Analyzing that result gave the insight that the task needs two separable skills, copying and language modeling, and that copying is the easy one to learn from little data\. That insight defined the challenge of learning the language\-modeling skill from few examples\. The challenge triggered a literature search, which returned the second piece of outside information, the pre\-trained language models that had just been released\. Reading them gave the insight that the language\-modeling skill need not be learned from the task data at all, which narrowed the challenge to how a pre\-trained language model can be combined with copying\.

Resolving that challenge needed no new information\. It needed a state already in the graph to be read again\. Revisiting the pointer\-generator paper with the new challenge in mind gave the decisive insight that the pointer\-generator is structurally a switch between a language model and a copy distribution, so its language\-model half can be replaced by the pre\-trained model while the switch and the copy half are kept\. The final idea is that hybrid, with the copy switch supervised explicitly, and the second implementation and its result closed the project \(Figure[2](https://arxiv.org/html/2610.27337#S4.F2)\)\.

The trajectory is one round with two experimental and two literature states of evidence\. The decisive step, transition 11, combines components, but it is not “pointer\-generator plus GPT\-2”\. It maps the new component onto one slot of the functional structure that transition 10 abstracted from the pointer\-generator\.

*What a whole\-idea regeneration would have lost\.*A loop that receives the first result as a score \(“BLEU 21 at 200 examples”\) and regenerates the idea might well arrive at “use GPT\-2”\. It cannot represent the actual edit, in which the language\-model half of the switch was replaced and the switch and the copy half were kept, nor the reason on transition 11\. That reason is why the final model keeps explicit copy supervision instead of fine\-tuning GPT\-2 alone, and it exists only as the annotation on the edge from the insight drawn from the re\-read paper to the new idea\. In a regeneration loop the diagnosis that copying was learned and language modeling was not would sit in a discarded prompt, so the next version could not be checked against it\.

### 4\.2Case 2: from a prompted patient to PATIENT\-Ψ\\Psiand CBT\-Bench

Research Question: how trainees can practice formulating a cognitive modelIdea Update \#0: prompt GPT\-4 to role\-play a patientExp\. Results \#0: 12 experts, trainees: role\-play unrealistic; styles differInsight \#0: realism = behavior generated from a structured cognitionChallenge \#1: how to give an LLM a structured cognitionLiterature Search \#1: clinical psychology; CBT’s patient modelLiterature \#1: Beck’s cognitive model; the CCD \(8 components\)Insight \#1: an LLM given its parameters can simulate a cognition; CCD as inputIdea Update \#1: GPT\-4 conditioned on a CCD \+ one of six conversational stylesChallenge \#2: where do CCDs come from? No dataset existsLiterature \#2: Alexander Street CBT transcripts \(data\)Code & Exp\. Plan \#1: GPT\-4 Turbo summaries; two psychologists write 106 CCDsChallenge \#3: how to evaluate a simulated patient with no ground truthIdea Update \#2: expert fidelity, expert\-rated CCD accuracy, perceived effectivenessCode & Exp\. Plan \#2: user study: 20 experts, 13 trainees, randomized orderExp\. Results \#1: fidelity, effectiveness above baseline; CCDs rated 4\.1–4\.3/5Insight \#2: LLM judges rank the baseline above PATIENT\-Ψ\\Psi, unlike expertsInsight \#3: the cognitive model is the hinge on both sides; can a model analyze it?Challenge \#4 \(open\): objective skill gains need longitudinal RCTsChallenge \#5: can LLMs do the therapist\-side cognitive\-model work?Literature \#3: prior work on isolated CBT tasks; PATIENT\-Ψ\\Psias premiseIdea Update \#3: CBT\-Bench: QA \(220\); CD, PC, FC \(146, 184, 112\); DP \(156\)Code & Exp\. Plan \#3: open and closed LLMs; human references for DPExp\. Results \#2: 405B: QA 95\.0%; F1 43\.3 CD, 58\.2 FC; behind humans on DPInsight \#4: knowing CBT and analyzing a cognition are different capabilitiesChallenge \#6 \(open\): how to get cognitive\-structure analysis, not recitationInitial RPG123455678991011Reasoning on transition 8: LLM judges gave the GPT\-4 baseline higher fidelity scores than PATIENT\-Ψ\\Psi, the opposite of the experts, although the baseline talks like a therapist\. A judge that rewards the failure the experts named cannot score this application, so the paper rests its claims on expert ratings, a conclusion about how the idea is judged, not about the idea\.Transitions 5 and 9 have two sources\. Exp\. Results \#0 and \#1 come from people \(formative interviews with experts and trainees, a user study\), not from runs\. One reasoning record is shown\.Figure 3:Case 2, PATIENT\-Ψ\\Psiand CBT\-Bench, as an RPG\. Four rounds, six states of evidence, two of them from people\. Full account and listing in Appendix[A\.2](https://arxiv.org/html/2610.27337#A1.SS2)\.Case 2 is an application idea from one group\([Wang et al\., 2024c](https://arxiv.org/html/2610.27337#bib.bib8);[Zhang et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib55)\), annotated, or traced and confirmed, by its authors\. Trainees in cognitive behavioral therapy have few realistic ways to practice formulating a patient’s cognitive model before seeing real patients, and the initial idea was to prompt GPT\-4 to role\-play a patient\. The first evidence was a formative interview study with twelve mental\-health experts and trainees, who reported that peer role\-play is unrealistic and that real patients differ in conversational style\. The insight was that realism requires a structured cognition from which behavior is generated, and the challenge of giving an LLM one led to a literature search in clinical psychology, which returned Beck’s cognitive conceptualization diagram as a parameterization an LLM can follow\. Later rounds built the missing data, 106 diagrams written by two clinical psychologists from transcript summaries, and evaluated the simulated patient without ground truth\. That evaluation found that LLM judges give the GPT\-4 baseline higher fidelity scores than PATIENT\-Ψ\\Psi, the opposite of the experts, and the paper concludes that automatic evaluation with LLMs cannot assess the fidelity of a simulated patient, so its claims rest on the expert ratings\. In the fourth round the group moved the object of the idea from the patient to the therapist\. CBT\-Bench tests whether a model can analyze a patient’s cognitive structure, and its result separates knowing CBT from analyzing a patient’s cognition\. Llama\-3\.1\-405B reaches 95\.0% on its CBT question set but an F1 of only 43\.3 on cognitive\-distortion classification and 58\.2 on fine\-grained core\-belief classification \(Figure[3](https://arxiv.org/html/2610.27337#S4.F3)\)\. Each change is confined to one state and tied to a specific result or paper\. The conclusion about the judge, caused by a result about the judge, is the one a whole\-idea regeneration would overwrite silently, and a judge failure of the same kind recurs in our pilot \(§[6\.4](https://arxiv.org/html/2610.27337#S6.SS4)\)\.

### 4\.3Case 3: LDC, from fine\-tuning an idea generator to controlling it

Research Question: ideas that meet expert standardsLiterature \#0: prompting\-based ideationInsight \#0: prompting is misaligned with experts; learn the taskIdea Update \#0: SFT on 1,000 \(paper→\\toidea\) pairsInsight \#1: SFT gives the format; the quality dimensions are left to RLChallenge \#1: optimize each dimension and balance themIdea Update \#1: RL with three reward models, one per dimensionExp\. Results \#0: RL 5\.7 \(LLaMA\-3\-8B\) vs ResearchAgent 5\.3Literature \#1: innovation\-feasibility trade\-off in prior workLiterature \#2: ResearchAgent’s iterative feedbackInsight \#2: the dimensions are interdependent and trade offChallenge \#2: control each dimension separately at inferenceIdea Update \#2: per\-dimension controllers with steering valuesε\\varepsilonExp\. Results \#1: novelty controller alone 5\.8 vs 5\.7; all, static, 6\.1Insight \#3: emphasis depends on the part of the ideaIdea Update \#3: sentence\-level decoder setsε\\varepsilonper sentenceExp\. Results \#2: dynamic 6\.3; human 5\.8, Pearson 0\.871Exp\. Results \#3 \(later\): novelty weight 1→\\to4; novelty 6\.4→\\to7\.3, feasibility 6\.1→\\to4\.9Challenge \#3: interpretability, label bias, citation judge \(open\)Initial RPG123456798Reasoning on transition 4, as the paper states it: prior work reports that highly novel ideas often lack feasibility, and the dimensions “often exhibit interdependency and trade\-offs”\. The paper therefore adds one controller per dimension to “adjust the generation to prioritize specific dimensions when necessary”, with steering values that can be set at inference time\.Transitions 1, 3, and 6 have two sources each\. Exp\. Results \#0 feeds no later transition\. The trade\-off in prior work leads to both the controllers \(transition 4\) and the decoder \(transition 6\), and the sweep \(Exp\. Results \#3\) confirms it later\. One reasoning record is shown\.Figure 4:Case 3, LDC, as an RPG\. Three rounds, seven states of evidence, three of them from the literature\. The trade\-off enters from prior work at transition 4 and leads to both the controllers and the decoder \(transition 6\); the sweep that confirms it \(Exp\. Results \#3\) is a later analysis\. Full account and listing in Appendix[A\.3](https://arxiv.org/html/2610.27337#A1.SS3)\.Case 3 is a method for research\-idea generation, LDC\([Li et al\., 2024a](https://arxiv.org/html/2610.27337#bib.bib20)\), based on version 3 of its paper and annotated, or traced and confirmed, by its authors\. Prompting\-based ideation agents use pre\-trained models without task\-specific learning, and the paper describes their ideas as “misaligned with expert standards”\. The initial idea is therefore to fine\-tune a model on pairs of a paper and the idea extracted from it\. Fine\-tuning gives the task format, and the paper leaves the quality dimensions to a later stage, so the first update adds reinforcement learning with three reward models, for novelty, feasibility, and effectiveness, a feedback design taken from the iterative feedback of ResearchAgent\([Baek et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib23)\)\. The evidence for the next step comes from the literature, not from the paper’s own results\. The paper takes from prior work, including[Si et al\. \(2025b\)](https://arxiv.org/html/2610.27337#bib.bib18), an “inevitable innovation\-feasibility trade\-off”, in which highly novel ideas often lack feasibility, and it reads the dimensions as interdependent\. It therefore adds, on top of the per\-dimension rewards, one controller per dimension whose weight can be set at inference time\. The same trade\-off motivated the last update\. A fixed setting of the controllers raises novelty at the cost of feasibility across the whole idea, so a decoder sets them sentence by sentence, with novelty emphasized in the method and feasibility in the experiment plan \(Figure[4](https://arxiv.org/html/2610.27337#S4.F4)\)\. A later sweep on the finished system confirms the trade\-off\. Raising the weight of the novelty controller from 1\.0 to 4\.0 lifts novelty from 6\.4 to 7\.3 and drops feasibility from 6\.1 to 4\.9\. An agent that receives only the overall score after the first update, 5\.7 for reinforcement learning on Llama\-3\-8B against 5\.3 for ResearchAgent, would see a method that already wins, since the score hides the trade\-off between novelty and feasibility\. The two updates that mattered came from reading prior work as the statement that raising one dimension lowers another\. That reasoning sits on the transitions from the literature state to the controllers and to the decoder, and a representation that keeps only the current idea has no place for it or for the paper it came from\.

### 4\.4Case 4: MLR\-Copilot, from a prompted idea agent to a trained one

Research Question: from a paper to run experimentsLiterature \#0: ideas not executed; small edits onlyChallenge \#1: connect ideas to code that runsIdea Update \#0: prompted idea agent; agent runs the paper’s codeIdea Update \#1: ReAct experiment agent with iterative feedbackLiterature \#1: MLAgentBench, a ReAct agentLiterature \#2: ResearchAgent, iterative feedbackExp\. Results \#0: the agent drifts further from the goalInsight \#1: human correction largely fixes the directionLiterature \#3: Hugging Face Hub, models and datasetsIdea Update \#2: \+ Hugging Face retrieval, \+ human feedbackExp\. Results \#1: innovativeness 3\.9 vs 3\.1; \+39\.7%Insight \#2: execution succeeds through its feedback loopExp\. Results \#2: single prompt fails on all tasksIdea Update \#3: trained idea agent; experiment stages keptLiterature \#4: the group’s SFT \+ RL method \(Case 3\)Exp\. Results \#3: 3\.9 vs 3\.8 vs 3\.1; \+44\.2%Challenge \#2: go back from results to the idea \(open\)Initial RPG12345678910Reasoning on transition 3, as the authors annotated or confirmed it: the experiment agent adopts the ReAct framework of the agent in MLAgentBench, in which each action follows a stated thought and is followed by an observation, and its iterative feedback follows the design of ResearchAgent\.Transitions 1, 3, 5, 7, and 8 have several sources\. Literature \#3 is an external resource, and Insight \#2 feeds no later transition\. Exp\. Results \#2 precedes Idea Update \#3\. One reasoning record is shown\.Figure 5:Case 4, MLR\-Copilot, as an RPG\. Three rounds and an open challenge, nine states of evidence, five of them literature states\. Literature \#3 is an external resource, and Literature \#4 is a later paper by the same group \(Case 3\)\. Full account and listing in Appendix[A\.4](https://arxiv.org/html/2610.27337#A1.SS4)\.Case 4 is a system for automated machine learning research, MLR\-Copilot\([Li et al\., 2024b](https://arxiv.org/html/2610.27337#bib.bib41)\), based on the first and third arXiv versions of its paper \(2024 and 2025\) and annotated, or traced and confirmed, by its authors\. It starts from a gap between two kinds of prior system\. Ideation systems produce ideas that are never executed, and systems that automate experiments make small edits such as tuning hyperparameters, without robust feedback\. The initial idea connects the two\. A prompted idea agent reads a paper and writes a hypothesis and an experiment plan, and an experiment agent adapts the paper’s prototype code to the plan and runs it, with no retrieval of models or datasets and no channel for human feedback\. The first idea update takes two components from two papers\. The experiment agent is rebuilt on the ReAct framework used by the agent of MLAgentBench\([Huang et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib71)\), and its iterative feedback follows the design of ResearchAgent\([Baek et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib23)\)\. The second update has two sources of different kinds\. The Hugging Face Hub enters as an external resource, through one researcher’s idea that retrieving candidate models and datasets from it would match models to the task better, with no result behind it\. A tool through which a person can be asked for help was added after the agent was seen to drift further from the goal over its iterations, a drift that human correction largely fixed\. This gives the system of the first version\. On five tasks its ideas score higher than those of a prompt\-only baseline \(innovativeness 3\.9 against 3\.1\) and its experiments improve on the prototype by 39\.7% on average, while a single\-prompt baseline without execution feedback fails on every task\. The third version keeps the experiment stages and replaces the idea agent with a trained model, fine\-tuned on extracted ideas and refined by reinforcement learning with reward models for novelty, feasibility, and effectiveness\. This is the method of a later paper by the same group \(Case 3\), used without its inference\-time control, and the authors give the reason for the change as the availability of that better ideation method\. The paper ends with an open challenge, that its stages run largely in sequence and need transitions back from implementation to ideation, which is the loop that property 4 requires \(Figure[5](https://arxiv.org/html/2610.27337#S4.F5)\)\. Every idea update draws on a literature state, two papers for the first, the Hugging Face Hub for the second, and the group’s own later paper for the third, and only the second also rests on a result of the system’s own runs\. A representation that keeps only the current system description records the final system\. It does not record which component came from which paper, nor why each was added\.

### 4\.5Case 5: on\-policy distillation from one training example

Research Question: how much data does OPD need, and whyLiterature \#0: one\-shot RLVR as a controlled experiment; Part I of OPDIdea Update \#0: bring the one\-shot lens to OPD: train on one queryCode & Exp\. Plan \#0: one query vs 17k; 4 domains×\\times3 familiesExp\. Results \#0: 68\.5 vs 69\.8 at step 300; 87% of full\-data gainInsight \#0: rollouts of one query reach 71\.5% of state clustersChallenge \#1: query content, or only the states its rollouts generate?Code & Exp\. Plan \#1: content\-free prompts; 1–64 rollouts; 1–16 queriesExp\. Results \#1: templates nearly match; 16 queries reach 98\.9% coverageInsight \#1: inputs are state generators, not task instructionsChallenge \#2: why does absorption still take hundreds of steps?Code & Exp\. Plan \#2: absorption rate on 1 to 17k queries; frozen states \(64 trajectories\)Exp\. Results \#2: absorption rate declines the same way; frozen run gains for 200 stepsInsight \#2: the bottleneck is absorption rate, not supervision quantityIdea Update \#1: “data\-overfed, algorithm\-starved”; select queries by the states they induceInitial RPG12345678Reasoning on transition 5: Insight \#0 attributes the gain to the states the rollouts reach, not to the query\. If so, the query’s content should be replaceable: content\-free templates and off\-domain prompts should drive OPD nearly as well, and more rollouts from one query should help\. The mechanism, not the observation, predicts the ablation\.Transition 8 has three sources\. The dashed box is the initial RPG; Part I is recorded as a literature state of Part II\. One reasoning record is shown\.Figure 6:Case 5, on\-policy distillation from one training example, as an RPG\. Three rounds, four states of evidence\. Full account and listing in Appendix[A\.5](https://arxiv.org/html/2610.27337#A1.SS5)\.Case 5 is a finding carried across two papers by one group\([Li et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib7);[Fu et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib6)\), which share eight authors and a title\. The second paper names one\-shot RLVR\([Wang et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib56)\), not the first paper, as the source of its idea, and the first paper enters the graph of the second as a literature state\. The initial idea transferred an experimental lens rather than a technique, training on\-policy distillation \(OPD\) on a single query to isolate the role of data, and as an idea it had no mechanism, no prediction, and no boundary\. The first result, that one\-shot OPD keeps improving for hundreds of steps and recovers 69% of the teacher–student gap, led to the insight that one query’s rollouts already reach most of the states that full\-data training reaches, 65\.9% of the state clusters by step 100\. That insight raised a challenge that a text idea could not have raised, whether the query’s content matters or only the states its rollouts generate, and an ablation answered it, with content\-free templates and off\-domain prompts nearly matching the full training set\. A third round found that the share of the teacher–student distance closed by each update declines the same way at every data size\. The final idea is a reframing, that OPD is “data\-overfed but algorithm\-starved” \(Figure[6](https://arxiv.org/html/2610.27337#S4.F6)\)\. Its final idea is a re\-scoped claim rather than a new idea, each challenge asks about one state, and the ablation exists because the insight predicted it\. An agent that receives “68\.5 against 69\.8”, one\-shot against full\-data accuracy at step 300, as a score would either stop or start an unrelated experiment\.

### 4\.6What the cases establish

The cases establish five things, which Appendix[A\.9](https://arxiv.org/html/2610.27337#A1.SS9)tabulates case by case\. First, the cycle is not specific to method ideas\. The finding \(Case 5\) passes through it three times with almost only experimental evidence, the application \(Case 2\) with mostly human and literature evidence, the system \(Case 4\) with each update confined to one of its stages, and the evaluation idea in the appendix \(Case 7\) with a change to nothing but the judge\. Second, the most valuable transitions are the ones where a*challenge*redirects the search, such as “how to obtain the language\-modeling skill” in Case 1, “how to give an LLM a structured cognition” in Case 2, and whether a small amount of cold\-start data can improve reasoning and give a readable model in Case 8\. This is why challenges are states of their own and not a sentence in a review\. Third, idea updates are often not additions or regenerations\. Examples are one slot of a structure substituted \(Case 1\), a claim re\-scoped by two ablations \(Case 5\), the object of the idea moved from patient to therapist \(Case 2\), the emphasis set sentence by sentence rather than for the whole idea \(Case 3\), one stage of a system replaced by a method from another paper \(Case 4\), and the judge replaced \(Case 7\)\. An agent that can only regenerate a fuller idea cannot express any of these\. Fourth, forks and abandoned attempts \(Case 8, in the appendix\) are sibling transitions out of one challenge with different reasons, which a representation holding only the current idea cannot hold at all\. Fifth, in every case the decisive change is confined to one state and caused by a specific result or paper\. Its reasoning is read from the authors’ annotation, their narrative, or their paper, and outside the RPG it is kept, if at all, in prose apart from the state it explains\. Agents that pass back a score, a review, or a summary and regenerate the whole idea cannot keep the state, the evidence, or the reasoning; typed idea graphs that no evidence enters have the states and not the evidence\. Idea evolution needs both at once, which is what the comparison of §[5](https://arxiv.org/html/2610.27337#S5)finds missing in every existing representation\.

## 5Existing Representations Compared with the RPG

Every LLM\-based research agent represents research ideation with some data structure, whether a proposal schema that is revised in place, a tree of experiments under a fixed hypothesis, or a graph of typed claims\. This section groups these representations and compares each group with the RPG, using the eight state types of §[3\.1](https://arxiv.org/html/2610.27337#S3.SS1)and the five properties of §[3\.4](https://arxiv.org/html/2610.27337#S3.SS4)as the dimensions of comparison\. The result is the same for every group\. Each representation keeps some of the states and transitions of an RPG and collapses the rest into a text, a score, or a prompt that is discarded, so each can be viewed as an*incomplete RPG*, and none of them represents the authentic research trajectory of an idea\. The missing parts are not left blank\. Each representation fills them with a device of the agent, a review text, a score, or a regeneration of the whole idea, that has no counterpart in a human trajectory, which is why these representations are artificial as well as incomplete\. The comparison therefore has two aims, to locate, for each representation, which parts of the trajectory it lacks, and to show in what sense the RPG is complete and authentic where they are not\. The comparison covers thirty\-five agents, thirty\-two in the five group tables and three coding agents \(AIDE, R&D\-Agent, and Gome\) that appear in Table[3](https://arxiv.org/html/2610.27337#S5.T3)but in no group table, and rests on their papers and on public code as of October 2026\.

### 5\.1How the representations are compared

Six questions are asked of every representation, and the answers come from the agent’s paper and, where available, its code\.

1. 1\.Which of the eight state types exist as distinct objects in its data structures, and which are folded into the idea text or into a prompt?
2. 2\.What evidence enters after the initial idea exists, whether nothing, a score, a review text, retrieved passages, a result label or summary, or typed states?
3. 3\.What does an update modify, the whole idea or named components of it?
4. 4\.Are earlier states and the evidence that produced them inputs to the next step, or are they only stored?
5. 5\.Where is the reasoning behind a change kept, bound to the change, detached from it as a review, or only in a chain of thought that is discarded?
6. 6\.Where can a person enter?

The answers score the five properties\. Question 1 scores typed states \(P1\), which are partial when some kinds of state, such as fields of the idea, retrieved papers, or results, are kept as separate records and absent when the idea and its evidence form one text; question 4 scores the temporal order that later steps read \(P2\), question 5 reasoning bound to the change \(P3\), questions 2 and 3 together evidence that enters and changes the idea \(P4\), and question 6 human intervention \(P5\)\. The answers to questions 2 and 3, what enters and what it revises, also place the representation in one of the five groups that follow, and the table of each group records them in its loop column\.

Table[3](https://arxiv.org/html/2610.27337#S5.T3)shows the answers state by state for twenty of the agents\. Each column is one element of the RPG, the eight state types and the two annotations on a transition, and each row is an agent in order of first arXiv version\. A cell is full when the element is a distinct object that a later step reads, partial when the information exists but is folded into a text, a prompt, or a score, or is stored and not read, and absent otherwise\. The ideation agents type the idea, usually as a proposal schema\([Si et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib18);[Baek et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib23)\), and the literature, where a retriever returns records\([Wang et al\., 2024b](https://arxiv.org/html/2610.27337#bib.bib27);[Hu et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib29)\)\. The coding and end\-to\-end research agents type the code and the results as nodes of an experiment tree or graph\([Yamada et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib31);[Yang et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib51)\)and read their history\. Insights and challenges are distinct states in almost none of them\. ASI\-Arch’s per\-node shortcoming analysis\([Liu et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib45)\)is the only full insight state, and DeepScientist’s findings memory\([Weng et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib43)\)is the nearest after it\. Evolving Idea Graphs’ evidence\-need nodes\([Dong et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib49)\)and the per\-hypothesis challenges of R&D\-Agent and Gome\([Yang et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib51);[Zhang et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib52)\)are the only full challenge states\. The research question is usually a fixed prompt rather than a state\. The temporal index has analogs in ResearchTown’s timesteps\([Yu et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib24)\)and in the parent pointers of the tree\-search agents, which the next step reads in the coding agents but rarely in the ideation agents\. The reasoning behind a change is bound to the change in two rows only, in R&D\-Agent, where it concerns an experiment on a fixed task, and in Evolving Idea Graphs, which no evidence enters\. No agent has all ten elements in full\. Each kind of agent has built one part of the trajectory around a different object, the text for the ideation agents, the code for the coding agents, and a closed graph of claims for the typed\-idea agents\. Ten further agents are not scored because we have not examined their representations in detail\. They are Deep Ideation\([Zhao et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib72)\), FlowPIE\([Wang et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib73)\), Idea2Story\([Xu et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib74)\), DeepInnovator\([Fan et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib81)\),[Ueda et al\. \(2025\)](https://arxiv.org/html/2610.27337#bib.bib75), Denario\([Villaescusa\-Navarro et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib76)\), SciAgents\([Ghafarollahi and Buehler, 2025a](https://arxiv.org/html/2610.27337#bib.bib77)\), Sparks of Science\([O’Neill et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib78)\), GraphEval\([Feng et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib79)\), and Sparks\([Ghafarollahi and Buehler, 2025b](https://arxiv.org/html/2610.27337#bib.bib80)\)\.

Table 3:What twenty agents keep of an RPG, by element\.*v1*is the month of the first arXiv version, and the other columns are the eight state types of §[3\.1](https://arxiv.org/html/2610.27337#S3.SS1)and the two edge annotations of §[3\.2](https://arxiv.org/html/2610.27337#S3.SS2):*Index*, the temporal index, read by the next step;*Reasoning*, the reasoning process bound to the change it explains\.✓\\checkmarka distinct object that a later step reads;⊙\\odotthe information exists but is folded into a text, a prompt or a score, or is stored and not read;×\\timesabsent; ? not disclosed \(Kosmos is closed source\)\. Entries condense the group tables of §[5\.2](https://arxiv.org/html/2610.27337#S5.SS2)to §[5\.6](https://arxiv.org/html/2610.27337#S5.SS6); AIDE, R&D\-Agent, and Gome appear in no group table, and their entries are read from their papers and code\.aA reason and a targeted problem on most hypotheses \(the class also defines observation and justification fields\), but about a change to the code of an experiment on a fixed task, not to an idea state\.

### 5\.2A single idea text revised against itself

The first group \(Table[4](https://arxiv.org/html/2610.27337#S5.T4)\) represents the idea as a single text that is generated once or revised against the model’s own critique\.[Si et al\. \(2025b\)](https://arxiv.org/html/2610.27337#bib.bib18)overgenerate thousands of seed ideas, deduplicate them, expand the remaining seeds into full proposals, and rank the proposals; the novelty filter retrieves papers but only discards ideas\. LDC\([Li et al\., 2024a](https://arxiv.org/html/2610.27337#bib.bib20)\)generates a proposal from a target paper’s context with controllable decoding; the Ramon Llull’s Thinking Machine\([Zhao et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib21)\)recombines theme, domain, and method elements; ResearchTown\([Yu et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib24)\)simulates a community of proposers and reviewers; CycleResearcher\([Weng et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib22)\)trains a generator against a learned reviewer\. ResearchAgent\([Baek et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib23)\)is the clearest review loop\. Reviewing agents score the problem, the method, and the experiment design, each on five criteria, and each part is refined iteratively, with the scores saturating after three rounds\. Compared with the RPG, this representation holds a research question, a set of literature states read once, and a single idea update\. The review loop adds a transition from one idea update to the next whose intermediate state, the review text, is untyped and is attached to a whole section of the idea, the problem, the method, or the experiment design, rather than to the part of it that should change, and no result or literature enters after the initial idea exists\. What this representation records is a sequence of texts, and the steps between them are set by the prompt\. This is the representation of Figure[1](https://arxiv.org/html/2610.27337#S1.F1), structures \(1\) and \(2\)\.

Table 4:Group A: a single idea text, generated once or revised against the model’s own critique\. Scores:✓\\checkmarkfull,⊙\\odotpartial,×\\timesabsent; P1 to P5 are the properties of §[3\.4](https://arxiv.org/html/2610.27337#S3.SS4); in column P4, lit, exp, and both name the evidence that enters after the initial idea and changes it, and⊙\\odotmarks evidence that changes only the code, the plan, or a hypothesis inside a fixed question; ? marks a score that cannot be given because the representation is not disclosed\.AgentIdea representationLoop: feedback→\\rightarrowwhat is revisedP1P2P3P4P5Si et al\.\([Si et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib18);[Si et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib19)\)Proposal JSON: title, problem, motivation, method, experiment plan, test cases, fallbackNone; novelty filter only discards; closed\-loop feedback named as future work⊙\\odot×\\times×\\times×\\times×\\timesLDC\([Li et al\., 2024a](https://arxiv.org/html/2610.27337#bib.bib20)\)Text: problem, method, experiment plan, related work; input = a supporting paperNone; literature enters once before generation; novelty reward uses retrieval on the evaluation side only⊙\\odot×\\times×\\times×\\times×\\timesRamon Llull’s Thinking Machine\([Zhao et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib21)\)Theme×\\timesdomain×\\timesmethod combination expanded to a title and abstractNone⊙\\odot×\\times×\\times×\\times⊙\\odotCycleResearcher\([Weng et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib22)\)Full paper textTraining\-time only: review scores train the next generator×\\times×\\times×\\times×\\times×\\timesResearchAgent\([Baek et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib23)\)Problem, method, experiment, each with rationaleYes: 15 reviewing agents, 5 criteria→\\rightarrowwhole\-section regeneration, 3 rounds to saturation; no evidence enters the loop⊙\\odot⊙\\odot⊙\\odot×\\times×\\timesResearchTown\([Yu et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib24)\)Typed objects: insight, idea, proposal, review, and meta\-review, with keys and timesteps; a rebuttal type is defined but its generation is disabled in the codeNone at run time; the review→\\rightarrowproposal transition is declared and never called⊙\\odot⊙\\odot⊙\\odot×\\times×\\timesGoAI\([Gao et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib25)\)Literature graph with citation\-semantic edges; idea = motivation, novelty, method, with reviewer scoresReview score→\\rightarrowregeneration \(mechanism unclear\)⊙\\odot×\\times×\\times×\\times×\\times
### 5\.3An idea text with literature retrieved for it

The second group \(Table[5](https://arxiv.org/html/2610.27337#S5.T5)\) retrieves literature for the current idea inside the loop\. SciMON\([Wang et al\., 2024b](https://arxiv.org/html/2610.27337#bib.bib27)\)retrieves the closest prior work to a generated idea and asks for a “significantly different” one; Nova\([Hu et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib29)\)plans a search around the current idea and generates child ideas from the results; MOOSE\-Star\([Yang and Bing, 2026](https://arxiv.org/html/2610.27337#bib.bib4)\)plans a motivation from the research background, searches a corpus hierarchically for the next inspiration paper, and adds one hypothesis increment per inspiration, with trained models for inspiration retrieval and hypothesis composition; Chain of Ideas\([Li et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib28)\)orders retrieved papers into a chain before generating; InternAgent \(NovelSeek\)\([InternAgent Team et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib32)\)adds literature from a survey agent while it evolves ideas, and VirSci\([Su et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib30)\)retrieves papers during multi\-agent discussion; AI Scientist v2\([Yamada et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib31)\)retrieves literature that bears on the idea only during ideation, and its later searches only gather citations for the paper\. Literature now enters after the initial idea, so the fourth property holds for literature\. But no challenge directs the search, the query is derived from the whole idea or from the research goal rather than from an identified gap in the idea, and the update regenerates the idea rather than editing the component that the literature bears on\. The insight that the literature yields, and the reasoning from that insight to the change, exist only in the prompt\. Nova writes its full idea tree to disk with parent pointers and reads only the last step; the history exists and is not an input\.

Table 5:Group B: an idea text with literature retrieved for it\. Results, where experiments are run, do not revise the idea\.AgentIdea representationLoopP1P2P3P4P5SciMON\([Wang et al\., 2024b](https://arxiv.org/html/2610.27337#bib.bib27)\)One sentence with a record of input, prediction, nearest prior work, regenerationLiterature similarity→\\rightarrowregeneration until no retrieved work is too close⊙\\odot⊙\\odot⊙\\odotlit×\\timesMOOSE\-Star\([Yang and Bing, 2026](https://arxiv.org/html/2610.27337#bib.bib4)\)Hypothesis built in increments from a research background, one increment \(motivation, mechanism, methodology\) per inspiration paperMotivation→\\rightarrowhierarchical search for the next inspiration→\\rightarrowincrement composed on the previous hypothesis, one step per inspiration; models trained on 108,717 papers decomposed by an LLM; no feedback loop⊙\\odot⊙\\odot⊙\\odotlit×\\timesChain of Ideas\([Li et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib28)\)Chain of papers in development order; idea = motivation, novelty, methodNovelty check→\\rightarrowrefine from chain; review \+ retrieval→\\rightarrowexperiment plan, 1 round⊙\\odot⊙\\odot⊙\\odotlit×\\timesNova\([Hu et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib29)\)Idea JSON with parent pointerSearch plan on current idea→\\rightarrowchild ideas, 3 rounds; tree stored, not read back⊙\\odot⊙\\odot⊙\\odotlit×\\timesVirSci\([Su et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib30)\)Idea JSON with title, experiment, and self\-ratings→\\rightarrowabstractTeam discussion→\\rightarrowidea; similarity→\\rightarrowabstract⊙\\odot⊙\\odot⊙\\odotlit×\\timesAI Scientist v2\([Yamada et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib31)\)Idea JSON frozen to a file; experiment tree nodes typedLiterature→\\rightarrowproposal in ideation only; results→\\rightarrowexperiment tree, never the idea⊙\\odot⊙\\odot⊙\\odotlit⊙\\odotInternAgent \(NovelSeek\)\([InternAgent Team et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib32)\)Text idea; evolution treeSelf\-review \+ literature \+ human→\\rightarrowidea tree; results→\\rightarrowcode⊙\\odot⊙\\odot⊙\\odotlit✓\\checkmarkAI co\-scientist\([Gottweis et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib33)\)Text hypothesis, rationale, protocol, reviews, EloReview, debate, meta\-review, web search→\\rightarrownew hypotheses in a tournament; no lineage described \(closed source\); wet\-lab outside the system⊙\\odot⊙\\odot⊙\\odotlit✓\\checkmarkIRIS\([Garikaparthi et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib34)\)MCTS state: brief, reward, review, retrieved itemsReview, literature, user→\\rightarrowchild briefs⊙\\odot✓\\checkmark⊙\\odotlit✓\\checkmarkIdeaSynth\([Pu et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib35)\)Canvas of facet nodes \(problem, solution, evaluation, contribution\)Literature feedback \+ user→\\rightarrowchild facets⊙\\odot⊙\\odot⊙\\odotlit✓\\checkmarkScideator\([Radensky et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib36)\)Purpose, mechanism, evaluation facetsNovelty check→\\rightarrowsuggested facet swap→\\rightarrowuser regenerates⊙\\odot×\\times⊙\\odotlit✓\\checkmark
### 5\.4A fixed idea with a tree of experiments

The third group \(Table[6](https://arxiv.org/html/2610.27337#S5.T6)\) runs experiments and feeds the results back, but into the implementation or the plan rather than into the idea\. The AI Scientist\([Lu et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib37)\)re\-plans experiments conditional on results up to five times while the idea is fixed in a file; AI Scientist v2 keeps one experiment tree per stage and starts each new stage from a copy of the previous stage’s best node whose parent is set to null, so the lineage is cut at every stage transition, with the hypothesis frozen before the first tree starts; AIDE\([Jiang et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib50)\), Curie\([Kon et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib40)\), Jr\. AI Scientist\([Miyai et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib26)\), MLR\-Copilot\([Li et al\., 2024b](https://arxiv.org/html/2610.27337#bib.bib41)\), and Agent Laboratory\([Schmidgall et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib39)\)revise code, plans, or a hypothesis inside a fixed experimental question from execution feedback\. The representation in this group has typed code, experiment plans, and experimental results, and some of these agents read their history\. This is the part of the trajectory that these agents represent best\. But the cycle between plan and result runs many times under one idea update that never changes\. The fourth property holds only in part, because the evidence changes the code, the plan, or a hypothesis inside a fixed question and not the idea, and the reasoning that is recorded, where it is recorded, explains why a piece of code was changed\.

Table 6:Group C: a fixed idea with a tree of experiments; results change the code, the plan, or a hypothesis inside a fixed question, not the idea\.AgentIdea representationLoopP1P2P3P4P5AI Scientist v1\([Lu et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib37)\)Idea JSONUp to two self\-reflection rounds per idea in the released pipeline; results→\\rightarrownext experiment code and paper; score feedback to ideas described but not wired in the released code⊙\\odot⊙\\odot×\\times⊙\\odot×\\timesAI\-Researcher\([Tang et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib38)\)Sectioned proposalAdvisor and judge→\\rightarrowcode⊙\\odot⊙\\odot⊙\\odot⊙\\odot×\\timesAgent Laboratory\([Schmidgall et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib39)\)Text topic and planLLM review \+ last results→\\rightarrownew plan; human checkpoints⊙\\odot⊙\\odot⊙\\odot⊙\\odot✓\\checkmarkCurie\([Kon et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib40)\)Experiment knowledge module with a DAGResults→\\rightarrowhypothesis within a fixed experimental question⊙\\odot✓\\checkmark⊙\\odot⊙\\odot×\\timesJr\. AI Scientist\([Miyai et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib26)\)Idea JSON as AI Scientist v2Results→\\rightarrowcode; hypothesis fixed⊙\\odot⊙\\odot×\\times⊙\\odot×\\timesMLR\-Copilot\([Li et al\., 2024b](https://arxiv.org/html/2610.27337#bib.bib41)\)Task, related work, hypothesis, planResults→\\rightarrowresearcher→\\rightarrowhypothesis⊙\\odot⊙\\odot×\\times⊙\\odot✓\\checkmark
### 5\.5Ideas revised from experimental results

The fourth group \(Table[7](https://arxiv.org/html/2610.27337#S5.T7)\) closes the loop through experiments and into a new idea\. Dolphin\([Yuan et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib42)\)labels each idea’s result as improvement, maintenance, or decline and lists the ideas that improved performance as effective in the next round’s prompt; DeepScientist\([Weng et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib43)\)keeps a findings memory that the next hypothesis step retrieves from; ASI\-Arch\([Liu et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib45)\)stores a shortcoming analysis on every architecture node and retrieves cognitions to address it; AlphaResearch\([Yu et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib46)\), Kosmos\([Mitchener et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib44)\), freephdlabor\([Li et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib47)\), and Zochi\([Intology, 2025](https://arxiv.org/html/2610.27337#bib.bib48)\)pass results or summaries back to a new idea\. R&D\-Agent\([Yang et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib51)\), a coding agent scored only in Table[3](https://arxiv.org/html/2610.27337#S5.T3), runs the same kind of loop over experiments on one fixed task, with several traces in parallel that are fused at the end\. Results now enter and produce new idea updates, so the fourth property holds in full\. What is missing is the middle of the trajectory\. The transition from a result to a new idea carries a label, a score, or a summary rather than an insight and a challenge about a named component, and the new idea is regenerated in full, except in ASI\-Arch, which edits the code of a parent architecture and writes a new motivation for the child\. The reasoning behind the change, where it is recorded, is therefore not bound to the component that changed\. ASI\-Arch’s shortcoming analysis and R&D\-Agent’s hypothesis fields come closest to a reasoning process annotated on a transition\.

Table 7:Group D: ideas revised from experimental results\.AgentIdea representationLoopP1P2P3P4P5Dolphin\([Yuan et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib42)\)Idea JSONResult label \(improvement, maintenance, decline\)→\\rightarrowimproved ideas marked effective in the next prompt, others used as a similarity filter; new ideas, not revisions; numbers and analysis do not reach the prompt⊙\\odot⊙\\odot×\\timesexp×\\timesDeepScientist\([Weng et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib43)\)Findings memory typed by lifecycle stageResults \+ analysis records retrieved by the next hypothesis step⊙\\odot✓\\checkmark⊙\\odotboth⊙\\odotKosmos\([Mitchener et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib44)\)Structured world model, format not publicLiterature \+ data analysis summaries→\\rightarrowworld model→\\rightarrownext tasks?⊙\\odot?both×\\timesASI\-Arch\([Liu et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib45)\)Node: motivation, code, results, analysis, retrieved cognition; parent pointerResults \+ shortcoming analysis \+ retrieved cognition→\\rightarrowchild architecture⊙\\odot✓\\checkmark⊙\\odotboth×\\timesAlphaResearch\([Yu et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib46)\)Ordered trajectory of idea, program, and result entriesResults→\\rightarrownext idea; a simulated review score rejects ideas before they run⊙\\odot⊙\\odot×\\timesexp×\\timesfreephdlabor\([Li et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib47)\)Working idea JSONFree\-text results→\\rightarrowmanager→\\rightarrowre\-ideation; human can interject⊙\\odot⊙\\odot×\\timesexp✓\\checkmarkZochi\([Intology, 2025](https://arxiv.org/html/2610.27337#bib.bib48)\)Not publicClaimed only???exp \(claimed\)✓\\checkmark
### 5\.6Typed idea graphs that no evidence enters

The fifth group \(Table[8](https://arxiv.org/html/2610.27337#S5.T8)\) types the internal structure of the idea\. Evolving Idea Graphs\([Dong et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib49)\)has nine node types, among them problem, hypothesis, method, assumption, risk, and evidence need, records a rationale and a timestamp for every edit, although no later step reads this log at run time; ResearchTown \(Table[4](https://arxiv.org/html/2610.27337#S5.T4)\) keys every insight, idea, proposal, and review with a timestep, and Scideator and IdeaSynth \(Table[5](https://arxiv.org/html/2610.27337#S5.T5)\) decompose an idea into facets that a person recombines; these three type parts of the idea in the same way but are grouped by their loops\. Evolving Idea Graphs satisfies the first and third properties and the second in part, and these typed representations are the only ones in the comparison that can say which part of an idea changed\. But no result or literature enters after the initial idea, since the literature is given in advance and no experiment is run\. The graph is edited from the inside, and so the changes it records are not driven by evidence\.

Table 8:Group E: typed idea graphs that no evidence enters\.AgentIdea representationLoopP1P2P3P4P5Evolving Idea Graphs\([Dong et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib49)\)Typed nodes \(problem, hypothesis, method, assumption, risk, evaluation plan, novelty claim, evidence need, repair\) and typed edges; every operation logged with rationale and timestampController edit and commit steps inside the graph; literature given in advance; no retrieval, no experiments✓\\checkmark⊙\\odot✓\\checkmark×\\times×\\times
### 5\.7Summary of the comparison

Across the five groups, what flows around the existing loops is a score, a label, a review text, or a summary, and, except in ASI\-Arch, the loop uses it to regenerate the whole idea\. ResearchAgent’s loop carries the reviewers’ text and regenerates the reviewed part of the idea in full; Dolphin’s carries a label of improvement, maintenance, or decline and an embedding filter that discards new ideas similar to earlier ones, and the numerical results never reach the prompt; AlphaResearch’s carries the previous idea, program, and execution result, with a review score used only to reject ideas before they run; Kosmos’s carries free\-text summaries and freephdlabor’s carries file references with short summaries\. None of these representations can therefore answer, for a given change to an idea, which part of the idea changed, because of which piece of evidence, into what, and why\. A recurring pattern in the code is that the trajectory is stored and not read\. Nova writes its idea tree to disk and reads only the last step; AI Scientist v2 cuts the lineage of its experiment tree at every stage boundary and never revises the hypothesis; ResearchTown declares a transition from a rejection back to a new proposal that is never called; ResearchAgent keeps every round’s versions and reviews in a history list that its code reads only to pick the best version and its paper does not mention\. These agents already store the trajectory, but they do not give it to the next step to reason over\.

None of the thirty\-five has all five properties, and the two nearest fall short in different ways\. R&D\-Agent is the nearest on reasoning\. In its code a trace is a list of nodes of type \(experiment, feedback\), its second version organizes these nodes as a directed acyclic graph and selects a parent for each new node, and the default hypothesis object carries a hypothesis, the pipeline component it targets, the problem it addresses, and a reason, so a reason is recorded on most steps\. But the node is an experiment on a fixed data\-science task, the hypothesis is a free\-text statement tagged with a pipeline component rather than a typed idea state, and the reason explains why a code or feature change was tried, so the graph is over candidate solutions to one task rather than over versions of an idea\. Evolving Idea Graphs is the nearest on typing\. It has the states and the rationales, but no evidence arrives after the initial idea\. Behind these two are DeepScientist and ASI\-Arch \(real loops and a kept history, but lifecycle or engineering types and reasons linked by retrieval rather than annotated on transitions\), IRIS \(a version tree, reviews stored per node, and a person, but literature only\), and the AI co\-scientist \(the strongest literature loop and human intervention, but free\-text hypotheses with no lineage described\)\. Many representations with human intervention have no experiment loop, and most with an experiment loop are fully automatic\. Agent Laboratory, MLR\-Copilot, InternAgent, freephdlabor, and Zochi have both, but they keep the idea as text, or do not disclose it, and record human intervention and experimental evidence separately rather than on one graph\.

The RPG is complete in the sense that the parts each group has built, the idea text of the first two, the plans and results of the third, the result\-to\-idea transition of the fourth, and the typed internal structure of the fifth, sit in one graph together with the two states that almost none of them has, the insight and the challenge, and with the temporal index and the reasoning process on every transition between them\. Nothing in an RPG is new to the agents taken together; what is new is that the pieces form a single evolving record, so that the question above has an answer for every change\. The RPG is authentic in the sense that the trajectories of §[4](https://arxiv.org/html/2610.27337#S4), recorded from their authors, fit the representation without a new construct, including their re\-readings, derivations, and hand\-overs, whereas each representation in this section would keep only a fragment of them\.

## 6Evaluating Improvement

This section argues that if an idea is a trajectory, an improvement to it must be evaluated on the trajectory and not only on the final text\.

### 6\.1From absolute quality to improvability

None of the signals used to judge research measures the quality of an idea at the time the idea has to be judged\. Awards, citations, and venue acceptance score a paper, that is, the idea together with its execution, writing, timing, and authors, and they arrive months or years later\. Experts can judge an idea directly, but in the blind study of[Si et al\. \(2025b\)](https://arxiv.org/html/2610.27337#bib.bib18)they agree on only 56\.1% of rankings, against 66\.0% and 71\.9% that the same authors report for reviewers of complete NeurIPS 2021 and ICLR 2024 submissions, and their judgment is expensive and concerns a description of the idea rather than what becomes of it\. A reward model trained on any of these signals inherits its bias, and it becomes circular when the same group trains both the judge and the system being judged\. We therefore do not know how to define an absolute score for the text of an idea, and the framework does not require one\.

The framework replaces the question of how good an idea is with the question of how far the idea can still be improved from its current state, and at what cost\. This depends on where the idea stands in its graph, namely which of its assumptions have been tested, which challenges remain open, what evidence it rests on, and what the next round of the cycle would cost and could change\. Two ideas with the same text, one with a result and the insight that explains it and one without, therefore stand in different places, and a judge who reads only the text cannot tell them apart\. Improvability is not a score assigned to a text either\. What can be measured is which of two processes moved an idea further from the same starting point\. Whether improvability can be measured without running the loop, and whether it predicts later outcomes, are open questions\.

### 6\.2Relative evaluation at three levels

Improvement is measured by comparison\. Two processes start from the same seed with the same model, budget, and tools, and a blind pairwise judge, who cannot see how either result was produced, decides which result is better\. On a trajectory this comparison can be made at three levels, none of which assigns a score to a single text\. For a single transition, the question is whether the new state is supported by the graph so far, for example whether an insight follows from the result it cites or whether a challenge concerns a component of the current idea\. A person can check this in minutes\. For a sequence of transitions, the question is whether the sequence reaches the idea update that the authors’ own trajectory reached, in no more steps, which requires recorded reference trajectories\. For a whole trajectory, the question is whether it shows the cognitive patterns of human trajectories, such as substituting one slot of a structure \(Case 1\) or sharpening a claim and then ablating what its mechanism predicts \(Case 5\), which requires a taxonomy of such patterns that does not yet exist\. The comparison proposed next works on final versions and, through its hidden target, at the second level\.

### 6\.3A proposed comparison and its pilot

The comparison that would test the framework asks whether the RPG loop improves an idea more than whole\-idea rewriting does under equal resources\. This paper specifies the comparison and does not run it\. Five conditions share one generator and one output\-token budget, and those that retrieve share one retrieval tool\. The conditions are a one\-shot rewrite, a review loop, a literature loop, the RPG loop, and the RPG loop without outside information\. In the RPG loop each step is an insight, a challenge, a search, or an idea update, and an idea update edits named fields of the idea and must cite a node created since the previous update\. Each seed is an early version of an idea from a paper posted after the generator’s training cutoff, and the authors’ own next version is kept as a hidden target\. A blind pairwise judge compares the final ideas on the five criteria of[Si et al\. \(2025b\)](https://arxiv.org/html/2610.27337#bib.bib18), and a second measure records how close each condition came to the step the authors actually took\. Appendix[B](https://arxiv.org/html/2610.27337#A2)gives the full protocol\.

A pilot on three seeds changed the protocol in two ways and is reported for that reason only \(Table[12](https://arxiv.org/html/2610.27337#A2.T12)in Appendix[B](https://arxiv.org/html/2610.27337#A2)\)\. On overall preference the RPG loop lost to the review loop and to the literature loop, three pairs to zero each, and beat its own ablation three to zero and the one\-shot rewrite two to one\. The logs point to two causes, the budget and the judge \(§[6\.4](https://arxiv.org/html/2610.27337#S6.SS4)\)\. At an equal number of calls the review and literature loops generated about 13,000 to 14,000 output tokens per run and the RPG loop about 5,000, because four of its six steps produce no idea text, so the protocol now fixes generated tokens rather than calls\. Three seeds are too few to say anything about the framework, and we draw no conclusion about it from the pilot\.

### 6\.4The judge problem

A judge that reads only the final text cannot tell a claim that was checked against the literature from one that was not, so it can reward an idea for hiding its prior work and penalize one for citing it\. The known biases of language\-model judges, toward position, length, and their own generations\([Zheng et al\., 2023](https://arxiv.org/html/2610.27337#bib.bib68);[Wang et al\., 2024a](https://arxiv.org/html/2610.27337#bib.bib69);[Panickssery et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib70)\), are of the same kind, since each rewards a property of the text rather than of the idea\. Three instances show the problem, one published and two from our own runs\.

In the first instance, from Case 2, LLM\-based evaluators gave the unconstrained GPT\-4 patient higher fidelity scores than PATIENT\-Ψ\\Psi, the reverse of the experts’ ratings, and the authors conclude that automatic evaluation with LLMs cannot assess the fidelity of a simulated patient\([Wang et al\., 2024c](https://arxiv.org/html/2610.27337#bib.bib8)\)\.

In the second instance, from our pilot, the judge read a citation as a lack of originality\. Its rationales repeatedly hold the RPG version’s own citations against it, for example “B’s main hypothesis, the passive\-adaptive gap, is already reported by The Distillation Game \(2026\), which B cites itself”, and it awarded novelty to the review loop, whose claims were bolder and, since the loop has no tools, never checked\. The RPG loop cites its literature because each of its updates must name the node that motivated it\.

The third instance, a blind study on the 26 seeds of Appendix[B](https://arxiv.org/html/2610.27337#A2)run after the pilot, shows that a text\-only judge often cannot tell an executed idea from an untested one\. For each seed, the authors’ own next version, written into the fixed schema by the generator from the abstract of the real paper that followed, with its results removed, was paired with a one\-shot rewrite of the seed by the same generator, written without tools and never executed\. Every target postdates the generator’s training cutoff\. A text\-only judge, a newer model than the generator, compared each pair on the five criteria in both orders, and a preference counts only when the two orders agree \(Table[9](https://arxiv.org/html/2610.27337#S6.T9)\)\. The judge preferred the authors’ version on novelty \(19 pairs to 3\) and on excitement \(21 to 4\)\. On feasibility and effectiveness it preferred the rewrite, 16 to 8 and 13 to 10, although neither preference is significant with 26 pairs, and on overall quality it was close to even, 13 to 10 for the authors’ version\. In ten of the 26 pairs, therefore, it rated an untested rewrite above the version the authors carried out\. The rewrites were also longer, 641 words against 504 on average\. Because the judge is newer than the generator, it may have seen some target papers in training, but a judge that recognized a target would favor the authors’ version, so this possible bias works against the finding\.

Giving the judge retrieval did not remove the problem\. A verified\-novelty judge extracts each version’s novelty claims, retrieves literature posted before the target for each claim, and decides with that evidence visible\. It preferred the authors’ version in 13 pairs and the rewrite in 9 \(last row of Table[9](https://arxiv.org/html/2610.27337#S6.T9)\), and of the pairs that both judges decided, the four on which they disagree all moved toward the rewrite, although with four such pairs the asymmetry is not significant on its own \(exact McNemarp=0\.125p=0\.125\)\. The rationales explain the shift\. The authors’ claims sit in a crowded 2025 to 2026 literature in which retrieval finds concurrent work, while several of the rewrite’s claims are too general for retrieval to match and pass unchallenged\. A verified judge must therefore record a claim that retrieval cannot match as unsupported rather than as unrefuted\. The study measures the judge and says nothing about the RPG\.

Table 9:Blind judges on 26 pairs, each pairing the authors’ own next version with a tool\-free one\-shot rewrite of the same seed\. The first five rows are the text\-only judge; the last row \(“Novelty, verified”\) is the verified\-novelty judge, whose retrieval is restricted to work posted before the target\. A preference counts only when both presentation orders agree; the remainder are inconsistent\.pp\(rewrite\) is the share of decisive pairs in which the judge preferred the rewrite, with a 95% Wilson interval\.The three instances set three requirements for the judge of the proposed comparison, all fixed before any run\. The judge has the same retrieval access as the systems it judges, it records a claim that retrieval cannot match as unsupported, and a subset judged by people calibrates it\. The text\-only judgment is reported beside the verified one\. The first two requirements give the judge what the trajectory records and the final text omits, namely whether a claim was checked and against what evidence, and the third measures how far the model judge can be trusted\.

## 7Actionable Insights

This section turns the findings of the cases, the comparison of agents, and the evaluation into five recommendations, on what a research agent should store, how its output should be judged, what it should learn, what data it should learn from, and how agents and people should share one record\. Each recommendation is presented in the same order\. A first paragraph states what to do and which finding of this paper motivates it, a second states what a system or the field gains by following it, and a list gives the concrete steps\. Table[10](https://arxiv.org/html/2610.27337#S7.T10)summarizes the five, which build on one another\. Evaluation and learning at the level of transitions both need the trajectory to be stored, learning also needs annotated trajectories, and the system design combines the first four\.

Table 10:The five recommendations of this section, the problem each addresses, where this paper shows the problem, and what is gained by following it\.#### Store the trajectory, not the current text\.

A system that improves ideas should store each idea as a graph of typed states connected by time\-indexed edges that carry their reasons, and should give the next step this graph rather than the latest text\. The comparison of existing agents found that much of this information is already produced and then discarded\. Nova writes its idea tree to disk with parent pointers and reads only the last step, AI Scientist v2 cuts the lineage of its experiment tree at every stage boundary, and ResearchTown declares a transition from a rejection back to a new proposal that is never called\. In the loops that do run, what passes from one round to the next is a score, a label, a review text, or a summary, and the next round regenerates the whole idea from it\. Case 1 shows what this loses\. An agent given the first result as a BLEU score could arrive at “use GPT\-2”, but not at the decision to keep the copy switch, which came from re\-reading the pointer\-generator after the insight that copying and language modeling are separate skills\.

Storing the trajectory gives three things that a text cannot\. First, each change to an idea can be traced to the component it changed, the evidence that caused it, and the stated reason, so a person reviewing an agent’s output checks one reason against one piece of evidence instead of comparing two full versions\. Second, an idea that failed is kept together with the reason it failed, so a later step can avoid proposing it again\. Dolphin comes closest with an embedding filter of earlier ideas, which prevents an idea from being proposed again but does not keep the reason it failed\. Third, the stored trajectory is the input that the other four recommendations require\.

- •*Make idea updates localized edits\.*An update names the components it changes, cites the insight or literature node that justifies each change, and keeps the other components verbatim\. This can be added to any structured proposal schema, and the resulting edit log records which component changed because of which node\. Evolving Idea Graphs already logs a rationale for every edit, but no evidence enters its graph after the initial idea, so none of its edits can cite a result or a paper\.
- •*Type what drives the loop\.*A review, a result, or a retrieved paper becomes a node rather than prompt text, so that a later step can refer to it and check an insight against it\. R&D\-Agent already records a reason with most hypotheses, which shows that a working agent can record reasons as it runs\. What is missing is a reason about a change to the idea rather than about a change to the code\.
- •*Read the history\.*The next step’s input includes the earlier versions and the evidence that produced them, which is property 2\. Several systems already store this information, so what has to change is what the next step reads\.

#### Evaluate relatively, with a grounded judge\.

An improvement process should be evaluated by comparing results produced from the same seed under the same budget, blind and pairwise, at one of the three levels of §[6\.2](https://arxiv.org/html/2610.27337#S6.SS2), and never by giving a single idea a number\. The reason is that the judges in common use can be wrong in a direction that works against grounded ideas\. In the pilot the text\-only judge preferred the unchecked claims of a tool\-free loop to the cited claims of the RPG loop, in the blind study it rated an untested rewrite above the authors’ executed version in ten of 26 pairs, and in Case 2 LLM evaluators reversed the experts’ ranking of two simulated patients\. A field that optimizes against such a judge selects for ideas that sound bold and cannot be checked, and penalizes the grounding that the RPG records\.

A relative protocol with a grounded judge removes this incentive, so that an agent is no longer penalized for citing the literature behind its claims\. Equal budgets measured in generated tokens separate the effect of the representation from the effect of compute\. Reporting how close a process came to the authors’ own next step, beside the blind verdict, adds a signal that does not depend on any judge’s preferences\.

- •*Give the judge retrieval\.*The judge has the same retrieval access as the systems it judges, restricted to work that predates the version under judgment\. It checks the novelty claims of both sides before comparing them and records a claim that retrieval cannot match as unsupported, since the blind study showed that such claims otherwise pass\. The text\-only judgment is reported beside it\.
- •*Equalize generated tokens, not calls\.*A loop whose steps are insights, challenges, and searches produces little idea text per call, so a budget counted in calls penalizes it for the work the framework asks for\. In the pilot the RPG loop generated about 5,000 tokens per run against about 13,000 to 14,000 for the review and literature loops at the same number of calls\.
- •*Calibrate against people\.*A subset of pairs is also judged by people\. Each of the three examples above is a disagreement between a model judge and people or evidence, and only human judgments show how large that disagreement is under a given protocol\.
- •*Use the authors’ own next step as a reference where it exists\.*A process is credited for reaching the update the authors reached, or a later one, and for changing the idea along the dimension the authors changed\. This is reported beside the blind judgment rather than combined with it, since an idea better than the reference can score low against it, and a disagreement between the two signals is itself informative\.

#### Train on next\-state prediction\.

An ideation agent should be trained to predict the next state of a trajectory\. Given a partial RPG, it predicts the next transition and the state that transition produces, whether an insight, a challenge, a search, or an idea update, together with the reasoning on the transition\. Training an agent to produce a complete idea gives one training signal per idea, and the only signal available for a complete idea is a judge or a reward model, whose problems §[6](https://arxiv.org/html/2610.27337#S6)describes\. The cases show that the information lies in individual steps, since in every case the decisive change is confined to one state and caused by a specific result or paper\. MOOSE\-Star\([Yang and Bing, 2026](https://arxiv.org/html/2610.27337#bib.bib4)\)already trains this way for two transitions, retrieving the next inspiration and composing the next hypothesis increment, and finds that the decomposition makes training tractable where generating the whole hypothesis at once is not\. DeepInnovator\([Fan et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib81)\)calls its objective “next idea prediction”, but what it predicts at each step is a complete improved idea, scored by a reward model against the idea of the target paper\.

Training per step has four benefits\. A trajectory with one wrong turn still supervises its correct steps, so the same data yields more supervision\. A person can check each target, a single transition, in minutes, so the training data can be audited\. No reward model over whole ideas is needed, which avoids the circularity that arises when one group trains both the judge and the system being judged\. Finally, an agent trained on trajectories that include abandoned branches, such as the process reward model and the Monte Carlo tree search that DeepSeek\-R1 dropped for stated reasons \(Case 8\), can learn not to propose a step that the graph already records as failed, because the reason for the failure is recorded with it\.

- •*Limit the agent to the cognitive transitions\.*A retriever produces the literature state from the search state, and a coding agent or a person produces the plan and result states from the idea state\. The ideation agent therefore owns the insight, the challenge, the search, and the idea update, which are exactly the transitions that existing systems compress into one prompt\. It can be trained and judged on each of them separately\.
- •*Compare continuations, not final texts\.*The corresponding reinforcement learning formulation is preference learning over two continuations of the same partial graph\. It keeps the same locality, since a person or a grounded judge compares two next steps rather than two complete ideas\.

#### Annotate update paths\.

We propose that the lead authors of published papers annotate the update paths of their own ideas\. Each author types the states the project passed through, orders them, and writes the reason on each edge, starting from the initial idea rather than from the paper’s narrative, which is the protocol used for Case 1\. Such data is needed because next\-state prediction and the second level of evaluation both require trajectories, while the corpora that train ideation systems hold only final ideas, in abstracts, proposals, and papers\. The nearest exception is MOOSE\-Star’s corpus\([Yang and Bing, 2026](https://arxiv.org/html/2610.27337#bib.bib4)\), in which an LLM decomposes 108,717 published papers into a background, inspiration papers, and hypothesis increments\. It holds the steps from the literature to the hypothesis but not the results, insights, and challenges that revised it\. The reasons can come only from the authors\. Of our eight cases four are annotated, or traced and confirmed, by their authors\. In the other four the reasons are those the authors chose to publish\.

An annotated corpus would provide both the training data for next\-state prediction and the reference trajectories for evaluation at the second level\. Checked against each project’s code and logs, it would also measure how often authors misremember the order of their own steps and the evidence behind them\.

- •*Validate against artifacts\.*Earlier code, logs, drafts, and submission histories exist for most projects, and the annotated order and evidence are checked against them, with every disagreement recorded\.
- •*Annotate close to the work\.*Recall of the reasoning behind a step fades with time\. The best record is kept while the project proceeds, which makes the annotation an RPG recorded during the work rather than reconstructed after it\.
- •*Keep the schema minimal and shared\.*Annotators use the eight state types and add finer types only when they need them, so that corpora from different groups can be combined and compared\.
- •*Add cheaper sources\.*Three sources need less of an author’s time\. Trajectories can be traced from papers whose evolution is transparent and then confirmed by their authors, or recovered by retrospection from the final idea as in the seeds of §[6\.3](https://arxiv.org/html/2610.27337#S6.SS3)\. Result and analysis pairs extracted from papers train the transition from a result to an insight, which, to our knowledge, no ideation system currently learns\. Community\-level trajectories, in which each paper’s related work is its incoming literature and its limitations are its outgoing challenges, need no annotator\. Each source has a different bias, and the practical question is which mixture trains a next\-state predictor that transfers to held\-out recorded trajectories\.

#### Record agents and people on one graph\.

A research system should be built from specialists, agents and people, that read and write one RPG, so that the question of whether a human is in the loop becomes the question of which transitions the human takes\. Agent Laboratory, MLR\-Copilot, InternAgent, freephdlabor, and Zochi have both human intervention and an experiment loop, but each keeps the idea as text, or does not disclose it, and records the two kinds of input separately, so a later reader cannot tell whether a change came from a person’s comment or from a result\. Because the RPG does not distinguish who produced a state \(property 5 of §[3\.4](https://arxiv.org/html/2610.27337#S3.SS4)\), one RPG can hold both\.

One graph gives the project a single record, in which every change has an author, a time, and a reason, whoever made it\. The role of the person is set per transition, so a group can give the agent the searches and keep the idea updates, or the reverse, without changing the representation\. A project that reaches a paper has also produced an author\-annotated trajectory for the data agenda above, at no extra cost\. The same graph covers the research process beyond ideation, since the ideation agent writes the cognitive transitions, a retriever and an experiment runner write the literature, plan, and result states, and the graph records all of them\. When the six questions of §[5\.1](https://arxiv.org/html/2610.27337#S5.SS1)are asked of the system under design rather than of an existing one, their answers fix its states, its transitions, and the place of the person\.

- •*Fix the states and who writes each\.*The idea follows the group’s proposal template \(problem, key insight, method, experiment plan, expected results, assumptions and risks\)\. Members write the research question and the initial idea, the agent writes insights, challenges, searches, and idea updates, a retriever writes literature, and the group’s experiment runner writes plans and results from the current idea\. Every node carries its author and time index\.
- •*Define one agent transition\.*Given the whole graph, including pruned and abandoned branches, the agent emits one transition with its reasoning\. It is an insight citing the result or literature nodes it reads, a challenge naming the component of the idea it concerns, a search whose query is recorded, or an idea update that lists the changed components with the nodes justifying each and keeps the rest verbatim\. A round that brings in no new outside information cannot produce an update, which implements property 4\.
- •*Place the person\.*Members can veto or edit any transition before the next one, redirect the work by adding a challenge, and add a result or a paper as a node\. Because their operations are the same as the agent’s, the graph remains the single record\.
- •*Keep alternatives as sibling updates\.*If the group wants alternatives, the agent proposes several updates from the same insight, each with its reason, and the choice among them is itself a transition with a reason\. A pruned branch therefore keeps the reason it was pruned, and a later turn does not regenerate it\.
- •*Evaluate transitions as they are made\.*Members rate single transitions as they review them\. When the project reaches a paper, its recorded graph becomes an author\-annotated trajectory, and the agent’s predicted updates can be scored against it at the second level\.

A system built in this way can answer, for every change to the idea, which component changed, which node caused the change, what the component became, and why, which is the question that none of the existing representations can answer\.

## 8Limitations

The claims of this paper are limited by the scope of its evidence, by how that evidence was obtained, and by questions the framework leaves open\. We state each, and close with the results that would count against the framework\.

#### Scope\.

The cases and the systems are drawn from machine learning and natural language processing, where results are cheap to obtain and a round of the loop takes days\. In the experimental sciences, results are expensive and infrequent, so a trajectory may be dominated by literature and insight states and by challenges that stay open for long periods\. Even within machine learning the evidence can be a derivation rather than a run, as in the step from reward modeling to direct preference optimization\([Rafailov et al\., 2023](https://arxiv.org/html/2610.27337#bib.bib15)\), which the RPG records as a result state, and in theoretical fields this is the usual case\. The state types do not depend on the field, but the shape of the loop and the relative weight of its entry points do, and only recorded trajectories from other fields can test this\. The cases are also restricted to trajectories within one group of authors wherever the reason on an edge is to be checked, because only the authors can confirm why they made a change\. Trajectories across groups, such as the line of on\-policy distillation papers in which Cases 5 and 6 sit, can be checked only for what each paper took from the previous one\. Finally, since a published paper presents the result rather than the path \(§[2\.1](https://arxiv.org/html/2610.27337#S2.SS1)\), the orderings in our cases may be tidier than the projects were\.

#### Evidence\.

The ideal trajectory is annotated by its authors while the research proceeds, with each state and each reason recorded as it happens, and none of our eight cases was collected in this way\. Cases 1 to 4 were annotated by their authors after the work was done, or traced from papers that state their steps clearly and then confirmed by their authors\. Cases 5 to 8 were traced from what their authors published in papers, a technical report, and a blog post, so the reasons on their edges are the ones the authors chose to state\. Where these sources leave the order of states or the reason for a step unstated, we inferred it from the surrounding text, and such inferred steps are not marked in the figures or listings\. Tracing a trajectory from papers that describe their evolution clearly and having their authors confirm it can enlarge the set of trajectories at little cost, but it remains retrospective, and annotation during the research remains the goal \(§[7](https://arxiv.org/html/2610.27337#S7.SS0.SSS0.Px4)\)\. The comparison of systems rests on papers and public code as of October 2026\. Closed systems are scored on what their papers state, and the paper on Kosmos does not disclose its representation\. The controlled comparison of §[6\.3](https://arxiv.org/html/2610.27337#S6.SS3)is specified and piloted but not run, so this paper offers no evidence that a process built on the RPG improves ideas more than whole\-idea rewriting\. The evidence of §[4](https://arxiv.org/html/2610.27337#S4)supports the representation, in that real trajectories fit it, and not any process built on it\. The pilot is evidence about judges and budget units only, and the protocol uses literature as the only outside information, so the result loop that several recent systems implement is not compared\. The input of the RPG loop is the linearized graph, which costs more input tokens than any baseline, and we report this cost without correcting for it\. The blind study of §[6\.4](https://arxiv.org/html/2610.27337#S6.SS4)uses one judge model and 26 pairs, its preferences on feasibility and effectiveness are not significant, and the starting points of its 21 retro seeds are reconstructed by a model\.

#### Open questions\.

The eight state types are minimal in the sense of §[3\.1](https://arxiv.org/html/2610.27337#S3.SS1), and the cases already suggest where more may be needed\. Application ideas have people as sources of evidence \(Case 2\)\. Many idea updates are substitutions, re\-scopings, or abandoned branches \(Cases 1, 5, and 8\), and the RPG records the difference in the reasoning on the transition rather than in the type of the state\. Assumptions, mechanisms, and the boundary of a claim are inside the idea text and cannot yet be the target of a challenge\. Conclusions about which judge can score an idea are recorded as insights and a change of judge as an idea update, although both alter the idea’s criterion of success and may need a state of their own\. The reasoning on transitions, which §[6\.2](https://arxiv.org/html/2610.27337#S6.SS2)refers to only informally as cognitive patterns, needs a taxonomy grounded in recorded trajectories\. Two patterns can already be named, abstracting the structure of a method and mapping a new component onto one slot of it \(Case 1\), and transferring a clinical instrument as the parameterization a model follows \(Case 2\)\. Three further questions are open\. Whether improvability \(§[6\.1](https://arxiv.org/html/2610.27337#S6.SS1)\) can be measured without running the loop, whether it predicts the signals that arrive later, such as expert preference, execution outcomes, and citations, and whether searching over typed idea states improves on the search over idea texts that Nova and IRIS already show to help\([Hu et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib29);[Garikaparthi et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib34)\)are all questions that the framework poses and does not answer\.

#### What would count against the framework\.

The claims of this paper can be refuted in three places\. The first is the claim of §[3](https://arxiv.org/html/2610.27337#S3)that the decisive change to an idea is localized to one state and that its reason is bound to the evidence that caused it \(properties 3 and 4\)\. If trajectories recorded as the work proceeds showed decisive changes spread across states, or reasons that cannot be bound to evidence, this claim would fail and the state types with it\. The second is the claim of §[6\.4](https://arxiv.org/html/2610.27337#S6.SS4)that a judge reading only the final text cannot tell grounded versions from ungrounded ones, and that retrieval alone does not fix this\. If a text\-only judge, or one given retrieval, were shown to agree with experts on such pairs, the requirements we set for the judge would be unnecessary\. The third is the design claim of §[7](https://arxiv.org/html/2610.27337#S7)\. If, under the protocol of Appendix[B](https://arxiv.org/html/2610.27337#A2), a loop built on the RPG at an equal output budget were preferred no more often than whole\-idea rewriting by a judge with retrieval, and came no closer to the authors’ real next step, the design claim would fail, although the descriptive claims of §[4](https://arxiv.org/html/2610.27337#S4)and §[5](https://arxiv.org/html/2610.27337#S5)would stand\. We state these so that the proposed protocol can serve as a test of the framework\.

## 9Conclusion

Existing research agents represent a research idea as text and its improvement as rewriting, and each keeps only a fragment of the trajectory through which ideas actually evolve\. The authentic trajectories of eight real ideas show what this trajectory contains\. Evidence enters as a result or a paper, an insight and a challenge are drawn from it, and the idea changes locally for reasons that only its authors can state\. The Research Progress Graph is a minimal representation that keeps these states, their temporal order, and the reasoning on every transition\. Compared with it, each of thirty\-five existing representations is an incomplete RPG, and the two nearest fall short in different ways\. We have argued that the graph, rather than the text, should be the unit of research ideation, that is, what an agent predicts, what it is trained on, and what it is judged by\. The judge problem, documented in a published case and in our blind study, shows what is lost when the unit is the text\. The same graph, with its literature, plan, and result states written by retrievers, experiment runners, and people, records the research process as a whole; ideation is the part of it this paper has modeled\. The most immediate consequence is a data agenda\. Authentic trajectories, recorded by their own authors as the work proceeds, are the corpus that research agents lack\.

## References

- R\. Agarwal, N\. Vieillard, Y\. Zhou, P\. Stanczyk, S\. Ramos, M\. Geist, and O\. BachemOn\-policy distillation of language models: learning from self\-generated mistakes\.InProceedings of ICLR,Cited by:[§A\.6](https://arxiv.org/html/2610.27337#A1.SS6.p1.1.1)\.
- Baeket al\.\(2025\)J\. Baek, S\. K\. Jauhar, S\. Cucerzan, and S\. J\. HwangResearchAgent: iterative research idea generation over scientific literature with large language models\.InProceedings of NAACL,Note:arXiv:2404\.07738Cited by:[§A\.3](https://arxiv.org/html/2610.27337#A1.SS3.SSS0.Px1.p2.1),[§A\.4](https://arxiv.org/html/2610.27337#A1.SS4.SSS0.Px1.p2.1),[§1](https://arxiv.org/html/2610.27337#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§4\.3](https://arxiv.org/html/2610.27337#S4.SS3.p1.1),[§4\.4](https://arxiv.org/html/2610.27337#S4.SS4.p1.1),[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1),[§5\.2](https://arxiv.org/html/2610.27337#S5.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.27337#S5.T4.9.6.1.1.1)\.
- Bestaet al\.\(2024\)M\. Besta, N\. Blach, A\. Kubicek, R\. Gerstenberger, M\. Podstawski, L\. Gianinazzi, J\. Gajda, T\. Lehmann, H\. Niewiadomski, P\. Nyczyk, and T\. HoeflerGraph of Thoughts: solving elaborate problems with large language models\.InProceedings of AAAI,Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1)\.
- Boden \(2004\)M\. A\. BodenThe creative mind: myths and mechanisms\.2nd edition,Routledge\.Cited by:[§2\.1](https://arxiv.org/html/2610.27337#S2.SS1.p1.1)\.
- Chenet al\.\(2020\)Z\. Chen, H\. Eavani, W\. Chen, Y\. Liu, and W\. Y\. WangFew\-shot NLG with pre\-trained language model\.InProceedings of ACL,Note:arXiv:1904\.09521Cited by:[§A\.1](https://arxiv.org/html/2610.27337#A1.SS1.p1.1.1),[§4\.1](https://arxiv.org/html/2610.27337#S4.SS1.p1.1),[Table 2](https://arxiv.org/html/2610.27337#S4.T2.6.2.4.1.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-AIDeepSeek\-R1: incentivizing reasoning capability in LLMs via reinforcement learning\.arXiv preprint arXiv:2501\.12948v1\.Note:Version 1, January 2025Cited by:[§A\.8](https://arxiv.org/html/2610.27337#A1.SS8.p1.1.1),[Table 2](https://arxiv.org/html/2610.27337#S4.T2.6.9.4.1.1)\.
- Donget al\.\(2026\)J\. Dong, B\. Li, and W\. LinEvolving idea graphs with learnable edits\-and\-commits for multi\-agent scientific ideation\.arXiv preprint arXiv:2605\.04922\.Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1),[§5\.6](https://arxiv.org/html/2610.27337#S5.SS6.p1.1),[Table 8](https://arxiv.org/html/2610.27337#S5.T8.4.2.1.1.1)\.
- Dunbar \(1997\)K\. DunbarHow scientists think: on\-line creativity and conceptual change in science\.InCreative Thought: An Investigation of Conceptual Structures and Processes,T\. B\. Ward, S\. M\. Smith, and J\. Vaid \(Eds\.\),pp\. 461–493\.Cited by:[§2\.1](https://arxiv.org/html/2610.27337#S2.SS1.p1.1)\.
- Fanet al\.\(2026\)T\. Fan, F\. Zhang, Y\. Zheng, B\. Chen, X\. Niu, C\. Huang, J\. Lin, and C\. HuangDeepInnovator: triggering the innovative capabilities of LLMs\.arXiv preprint arXiv:2602\.18920\.Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1),[§7](https://arxiv.org/html/2610.27337#S7.SS0.SSS0.Px3.p1.1)\.
- Fenget al\.\(2025\)T\. Feng, Y\. Sun, and J\. YouGraphEval: a lightweight graph\-based LLM framework for idea evaluation\.InProceedings of ICLR,Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1)\.
- Fuet al\.\(2026\)Z\. Fu, B\. He, Y\. Zuo, H\. Huang, J\. Zhang,et al\.Rethinking on\-policy distillation of large language models II: one training example\.arXiv preprint arXiv:2609\.04172\.Cited by:[§4\.5](https://arxiv.org/html/2610.27337#S4.SS5.p1.1),[Table 2](https://arxiv.org/html/2610.27337#S4.T2.6.6.4.1.1)\.
- Gaoet al\.\(2025\)X\. Gao, Z\. Zhang, T\. Liu, and Y\. FuGoAI: enhancing AI students’ learning paths and idea generation via graph of AI ideas\.arXiv preprint arXiv:2503\.08549\.Cited by:[Table 4](https://arxiv.org/html/2610.27337#S5.T4.9.8.1.1.1)\.
- Garikaparthiet al\.\(2025\)A\. Garikaparthi, M\. Patwardhan, L\. Vig, and A\. CohanIRIS: interactive research ideation system for accelerating scientific discovery\.InProceedings of ACL \(System Demonstrations\),Note:arXiv:2504\.16728Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[Table 5](https://arxiv.org/html/2610.27337#S5.T5.4.10.1.1.1),[§8](https://arxiv.org/html/2610.27337#S8.SS0.SSS0.Px3.p1.1)\.
- Ghafarollahi and Buehler \(2025a\)A\. Ghafarollahi and M\. J\. BuehlerSciAgents: automating scientific discovery through bioinspired multi\-agent intelligent graph reasoning\.Advanced Materials37\(22\),pp\. 2413523\.Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1)\.
- Ghafarollahi and Buehler \(2025b\)A\. Ghafarollahi and M\. J\. BuehlerSparks: multi\-agent artificial intelligence model discovers protein design principles\.arXiv preprint arXiv:2504\.19017\.Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1)\.
- Gottweiset al\.\(2025\)J\. Gottweis, W\. Weng, A\. Daryin, T\. Tu, A\. Palepu, P\. Sirkovic, A\. Myaskovsky, F\. Weissenberger, K\. Rong, R\. Tanno,et al\.Towards an AI co\-scientist\.arXiv preprint arXiv:2502\.18864\.Note:Later version titled “Accelerating Scientific Discovery with Co\-Scientist”Cited by:[Table 5](https://arxiv.org/html/2610.27337#S5.T5.4.9.1.1.1)\.
- Guet al\.\(2024\)Y\. Gu, L\. Dong, F\. Wei, and M\. HuangMiniLLM: knowledge distillation of large language models\.InProceedings of ICLR,Cited by:[§A\.6](https://arxiv.org/html/2610.27337#A1.SS6.p1.1.1)\.
- Huet al\.\(2024\)X\. Hu, H\. Fu, J\. Wang, Y\. Wang, Z\. Li, R\. Xu, Y\. Lu, Y\. Jin, L\. Pan, and Z\. LanNova: an iterative planning and search approach to enhance novelty and diversity of LLM generated ideas\.arXiv preprint arXiv:2410\.14255\.Cited by:[§1](https://arxiv.org/html/2610.27337#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1),[§5\.3](https://arxiv.org/html/2610.27337#S5.SS3.p1.1),[Table 5](https://arxiv.org/html/2610.27337#S5.T5.4.5.1.1.1),[§8](https://arxiv.org/html/2610.27337#S8.SS0.SSS0.Px3.p1.1)\.
- Huanget al\.\(2024\)Q\. Huang, J\. Vora, P\. Liang, and J\. LeskovecMLAgentBench: evaluating language agents on machine learning experimentation\.InProceedings of ICML,Note:arXiv:2310\.03302Cited by:[§A\.4](https://arxiv.org/html/2610.27337#A1.SS4.SSS0.Px1.p2.1),[§4\.4](https://arxiv.org/html/2610.27337#S4.SS4.p1.1)\.
- InternAgent Teamet al\.\(2025\)InternAgent Team, B\. Zhang, S\. Feng, X\. Yan, J\. Yuan, others, and L\. BaiInternAgent \(NovelSeek\): when agent becomes the scientist – building closed\-loop system from hypothesis to verification\.arXiv preprint arXiv:2505\.16938\.Cited by:[§5\.3](https://arxiv.org/html/2610.27337#S5.SS3.p1.1),[Table 5](https://arxiv.org/html/2610.27337#S5.T5.4.8.1.1.1)\.
- Intology \(2025\)IntologyZochi technical report\.Note:[https://www\.intology\.ai/blog/zochi\-tech\-report](https://www.intology.ai/blog/zochi-tech-report)March 17, 2025Cited by:[§5\.5](https://arxiv.org/html/2610.27337#S5.SS5.p1.1),[Table 7](https://arxiv.org/html/2610.27337#S5.T7.4.8.1.1.1)\.
- Jianget al\.\(2025\)Z\. Jiang, D\. Schmidt, D\. Srikanth, D\. Xu, I\. Kaplan, D\. Jacenko, and Y\. WuAIDE: AI\-driven exploration in the space of code\.arXiv preprint arXiv:2502\.13138\.Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.4](https://arxiv.org/html/2610.27337#S5.SS4.p1.1)\.
- Klahr and Dunbar \(1988\)D\. Klahr and K\. DunbarDual space search during scientific reasoning\.Cognitive Science12\(1\),pp\. 1–48\.Cited by:[§2\.1](https://arxiv.org/html/2610.27337#S2.SS1.p1.1)\.
- Konet al\.\(2025\)P\. T\. J\. Kon, J\. Liu, Q\. Ding, Y\. Qiu, Z\. Yang, Y\. Huang, J\. Srinivasa, M\. Lee, M\. Chowdhury, and A\. ChenCurie: toward rigorous and automated scientific experimentation with AI agents\.arXiv preprint arXiv:2502\.16069\.Cited by:[§5\.4](https://arxiv.org/html/2610.27337#S5.SS4.p1.1),[Table 6](https://arxiv.org/html/2610.27337#S5.T6.4.5.1.1.1)\.
- Lakatos \(1976\)I\. LakatosProofs and refutations: the logic of mathematical discovery\.Cambridge University Press\.Cited by:[§2\.1](https://arxiv.org/html/2610.27337#S2.SS1.p1.1)\.
- Langleyet al\.\(1987\)P\. Langley, H\. A\. Simon, G\. L\. Bradshaw, and J\. M\. ZytkowScientific discovery: computational explorations of the creative processes\.MIT Press\.Cited by:[§2\.1](https://arxiv.org/html/2610.27337#S2.SS1.p1.1)\.
- Liet al\.\(2025a\)E\. Li, J\. Ren, X\. Pan, C\. Yan, C\. Li, D\. Bergemann, and Z\. YangBuild your personalized research group: a multiagent framework for continual and interactive science automation\.arXiv preprint arXiv:2510\.15624\.Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.5](https://arxiv.org/html/2610.27337#S5.SS5.p1.1),[Table 7](https://arxiv.org/html/2610.27337#S5.T7.4.7.1.1.1)\.
- Liet al\.\(2025b\)L\. Li, W\. Xu, J\. Guo, R\. Zhao, X\. Li, Y\. Yuan, B\. Zhang, Y\. Jiang, Y\. Xin, R\. Dang,et al\.Chain of Ideas: revolutionizing research via novel idea development with LLM agents\.InFindings of EMNLP,Note:arXiv:2410\.13185Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.3](https://arxiv.org/html/2610.27337#S5.SS3.p1.1),[Table 5](https://arxiv.org/html/2610.27337#S5.T5.4.4.1.1.1)\.
- Liet al\.\(2024a\)R\. Li, L\. Jing, C\. Han, J\. Zhou, and X\. DuLDC: learning to generate research idea with dynamic control\.arXiv preprint arXiv:2412\.14626v3\.Note:Version 3, September 2026Cited by:[§A\.3](https://arxiv.org/html/2610.27337#A1.SS3.p1.1.1),[§4\.3](https://arxiv.org/html/2610.27337#S4.SS3.p1.1),[Table 2](https://arxiv.org/html/2610.27337#S4.T2.6.4.4.1.1),[§5\.2](https://arxiv.org/html/2610.27337#S5.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.27337#S5.T4.9.3.1.1.1)\.
- Liet al\.\(2024b\)R\. Li, T\. Patel, Q\. Wang, and X\. DuMLR\-Copilot: autonomous machine learning research based on large language models agents\.arXiv preprint arXiv:2408\.14033\.Cited by:[§A\.4](https://arxiv.org/html/2610.27337#A1.SS4.p1.1.1),[§4\.4](https://arxiv.org/html/2610.27337#S4.SS4.p1.1),[Table 2](https://arxiv.org/html/2610.27337#S4.T2.6.5.4.1.1),[§5\.4](https://arxiv.org/html/2610.27337#S5.SS4.p1.1),[Table 6](https://arxiv.org/html/2610.27337#S5.T6.4.7.1.1.1)\.
- Liet al\.\(2026\)Y\. Li, Y\. Zuo, B\. He, J\. Zhang, C\. Xiao,et al\.Rethinking on\-policy distillation of large language models: phenomenology, mechanism, and recipe\.arXiv preprint arXiv:2604\.13016\.Cited by:[Table 11](https://arxiv.org/html/2610.27337#A1.T11h.2.3.3.1.1),[§4\.5](https://arxiv.org/html/2610.27337#S4.SS5.p1.1)\.
- Lightmanet al\.\(2023\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.arXiv preprint arXiv:2305\.20050\.Cited by:[§A\.6](https://arxiv.org/html/2610.27337#A1.SS6.p1.1.1)\.
- Liuet al\.\(2025\)Y\. Liu, Y\. Nan, W\. Xu, X\. Hu, L\. Ye, Z\. Qin, and P\. LiuAlphaGo moment for model architecture discovery\.arXiv preprint arXiv:2507\.18074\.Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1),[§5\.5](https://arxiv.org/html/2610.27337#S5.SS5.p1.1),[Table 7](https://arxiv.org/html/2610.27337#S5.T7.4.5.1.1.1)\.
- Luet al\.\(2024\)C\. Lu, C\. Lu, R\. T\. Lange, J\. Foerster, J\. Clune, and D\. HaThe AI Scientist: towards fully automated open\-ended scientific discovery\.arXiv preprint arXiv:2408\.06292\.Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.4](https://arxiv.org/html/2610.27337#S5.SS4.p1.1),[Table 6](https://arxiv.org/html/2610.27337#S5.T6.4.2.1.1.1)\.
- Lu and Thinking Machines Lab \(2025\)K\. Lu and Thinking Machines LabOn\-policy distillation\.Note:Blog post,[https://thinkingmachines\.ai/blog/on\-policy\-distillation/](https://thinkingmachines.ai/blog/on-policy-distillation/)Cited by:[Table 2](https://arxiv.org/html/2610.27337#S4.T2.6.7.4.1.1)\.
- Madaanet al\.\(2023\)A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. ClarkSelf\-Refine: iterative refinement with self\-feedback\.InAdvances in Neural Information Processing Systems,Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1)\.
- Medawar \(1963\)P\. B\. MedawarIs the scientific paper a fraud?\.The Listener70,pp\. 377–378\.Cited by:[§2\.1](https://arxiv.org/html/2610.27337#S2.SS1.p2.1)\.
- Mitcheneret al\.\(2025\)L\. Mitchener, A\. Yiu, B\. Chang, M\. Bourdenx, T\. Nadolski,et al\.Kosmos: an AI scientist for autonomous discovery\.arXiv preprint arXiv:2511\.02824\.Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.5](https://arxiv.org/html/2610.27337#S5.SS5.p1.1),[Table 7](https://arxiv.org/html/2610.27337#S5.T7.4.4.1.1.1)\.
- Miyaiet al\.\(2025\)A\. Miyai, M\. Toyooka, T\. Otonari, Z\. Zhao, and K\. AizawaJr\. AI Scientist and its risk report: autonomous scientific exploration from a baseline paper\.arXiv preprint arXiv:2511\.04583\.Cited by:[§5\.4](https://arxiv.org/html/2610.27337#S5.SS4.p1.1),[Table 6](https://arxiv.org/html/2610.27337#S5.T6.4.6.1.1.1)\.
- Nisbett and Wilson \(1977\)R\. E\. Nisbett and T\. D\. WilsonTelling more than we can know: verbal reports on mental processes\.Psychological Review84\(3\),pp\. 231–259\.Cited by:[§A\.6](https://arxiv.org/html/2610.27337#A1.SS6.SSS0.Px1.p4.1),[§2\.1](https://arxiv.org/html/2610.27337#S2.SS1.p2.1)\.
- O’Neillet al\.\(2025\)C\. O’Neill, T\. Ghosal, R\. Răileanu, M\. Walmsley, T\. Bui, K\. Schawinski, and I\. CiucăSparks of Science: hypothesis generation using structured paper data\.arXiv preprint arXiv:2504\.12976\.Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1)\.
- Panicksseryet al\.\(2024\)A\. Panickssery, S\. R\. Bowman, and S\. FengLLM evaluators recognize and favor their own generations\.InAdvances in Neural Information Processing Systems,Cited by:[§6\.4](https://arxiv.org/html/2610.27337#S6.SS4.p1.1)\.
- Puet al\.\(2025\)K\. Pu, K\. J\. K\. Feng, T\. Grossman, T\. Hope, B\. D\. Mishra, M\. Latzke, J\. Bragg, J\. C\. Chang, and P\. SiangliulueIdeaSynth: iterative research idea development through evolving and composing idea facets with literature\-grounded feedback\.InProceedings of CHI,Note:arXiv:2410\.04025Cited by:[Table 5](https://arxiv.org/html/2610.27337#S5.T5.4.11.1.1.1)\.
- Qwen Team \(2025\)Qwen TeamQwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§A\.6](https://arxiv.org/html/2610.27337#A1.SS6.p1.1.1)\.
- Radenskyet al\.\(2024\)M\. Radensky, S\. Shahid, R\. Fok, P\. Siangliulue, T\. Hope, and D\. S\. WeldHuman\-LLM compound system for scientific ideation through facet recombination and novelty evaluation\.arXiv preprint arXiv:2409\.14634\.Cited by:[Table 5](https://arxiv.org/html/2610.27337#S5.T5.4.12.1.1.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems,Note:arXiv:2305\.18290Cited by:[§3\.3](https://arxiv.org/html/2610.27337#S3.SS3.p2.1),[§8](https://arxiv.org/html/2610.27337#S8.SS0.SSS0.Px1.p1.1)\.
- Rosset al\.\(2011\)S\. Ross, G\. Gordon, and D\. BagnellA reduction of imitation learning and structured prediction to no\-regret online learning\.InProceedings of AISTATS,Cited by:[§A\.6](https://arxiv.org/html/2610.27337#A1.SS6.p1.1.1)\.
- Schmidgallet al\.\(2025\)S\. Schmidgall, Y\. Su, Z\. Wang, X\. Sun, J\. Wu, X\. Yu, J\. Liu, M\. Moor, Z\. Liu, and E\. BarsoumAgent Laboratory: using LLM agents as research assistants\.arXiv preprint arXiv:2501\.04227\.Cited by:[§5\.4](https://arxiv.org/html/2610.27337#S5.SS4.p1.1),[Table 6](https://arxiv.org/html/2610.27337#S5.T6.4.4.1.1.1)\.
- Seeet al\.\(2017\)A\. See, P\. J\. Liu, and C\. D\. ManningGet to the point: summarization with pointer\-generator networks\.InProceedings of ACL,Cited by:[§4\.1](https://arxiv.org/html/2610.27337#S4.SS1.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§A\.8](https://arxiv.org/html/2610.27337#A1.SS8.p1.1.1),[Table 2](https://arxiv.org/html/2610.27337#S4.T2.6.9.3.1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems,Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1)\.
- Siet al\.\(2025a\)C\. Si, T\. Hashimoto, and D\. YangThe ideation\-execution gap: execution outcomes of LLM\-generated versus human research ideas\.arXiv preprint arXiv:2506\.20803\.Cited by:[§A\.7](https://arxiv.org/html/2610.27337#A1.SS7.p1.1.1),[Table 2](https://arxiv.org/html/2610.27337#S4.T2.6.8.4.1.1),[Table 4](https://arxiv.org/html/2610.27337#S5.T4.9.2.1.1.1)\.
- Siet al\.\(2025b\)C\. Si, D\. Yang, and T\. HashimotoCan LLMs generate novel research ideas? A large\-scale human study with 100\+ NLP researchers\.InProceedings of ICLR,Note:arXiv:2409\.04109Cited by:[§A\.3](https://arxiv.org/html/2610.27337#A1.SS3.SSS0.Px1.p2.1),[§A\.7](https://arxiv.org/html/2610.27337#A1.SS7.p1.1.1),[Appendix B](https://arxiv.org/html/2610.27337#A2.p8.1),[§1](https://arxiv.org/html/2610.27337#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p1.1),[§4\.3](https://arxiv.org/html/2610.27337#S4.SS3.p1.1),[Table 2](https://arxiv.org/html/2610.27337#S4.T2.6.8.3.1.1),[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1),[§5\.2](https://arxiv.org/html/2610.27337#S5.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.27337#S5.T4.9.2.1.1.1),[§6\.1](https://arxiv.org/html/2610.27337#S6.SS1.p1.1),[§6\.3](https://arxiv.org/html/2610.27337#S6.SS3.p1.1)\.
- Suet al\.\(2025\)H\. Su, R\. Chen, S\. Tang, Z\. Yin, X\. Zheng, J\. Li, B\. Qi, Q\. Wu, H\. Li, W\. Ouyang, P\. Torr, B\. Zhou, and N\. DongMany heads are better than one: improved scientific idea generation by a LLM\-based multi\-agent system\.InProceedings of ACL,Cited by:[§5\.3](https://arxiv.org/html/2610.27337#S5.SS3.p1.1),[Table 5](https://arxiv.org/html/2610.27337#S5.T5.4.6.1.1.1)\.
- Swanson \(1986\)D\. R\. SwansonFish oil, Raynaud’s syndrome, and undiscovered public knowledge\.Perspectives in Biology and Medicine30\(1\),pp\. 7–18\.Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1)\.
- Tanget al\.\(2025\)J\. Tang, L\. Xia, Z\. Li, and C\. HuangAI\-Researcher: autonomous scientific innovation\.arXiv preprint arXiv:2505\.18705\.Cited by:[Table 6](https://arxiv.org/html/2610.27337#S5.T6.4.3.1.1.1)\.
- Uedaet al\.\(2025\)K\. Ueda, W\. Hirota, T\. Asakura, T\. Omi, K\. Takahashi, K\. Arima, and T\. IshigakiExploring the design of multi\-agent LLM dialogues for research ideation\.InProceedings of SIGDIAL,Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1)\.
- Villaescusa\-Navarroet al\.\(2025\)F\. Villaescusa\-Navarro, B\. Bolliet, P\. Villanueva\-Domingo,et al\.The Denario project: deep knowledge AI agents for scientific discovery\.arXiv preprint arXiv:2510\.26887\.Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1)\.
- Wanget al\.\(2024a\)P\. Wang, L\. Li, L\. Chen, Z\. Cai, D\. Zhu, B\. Lin, Y\. Cao, L\. Kong, Q\. Liu, T\. Liu, and Z\. SuiLarge language models are not fair evaluators\.InProceedings of ACL,Cited by:[§6\.4](https://arxiv.org/html/2610.27337#S6.SS4.p1.1)\.
- Wanget al\.\(2024b\)Q\. Wang, D\. Downey, H\. Ji, and T\. HopeSciMON: scientific inspiration machines optimized for novelty\.InProceedings of ACL,Note:arXiv:2305\.14259Cited by:[§1](https://arxiv.org/html/2610.27337#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1),[§5\.3](https://arxiv.org/html/2610.27337#S5.SS3.p1.1),[Table 5](https://arxiv.org/html/2610.27337#S5.T5.4.2.1.1.1)\.
- Wanget al\.\(2026\)Q\. Wang, H\. Wang, L\. Chen, Z\. Yang, G\. Chen, H\. Alinejad\-Rokny, H\. Li, Y\. Lin, and M\. YangFlowPIE: test\-time scientific idea evolution with flow\-guided literature exploration\.arXiv preprint arXiv:2603\.29557\.Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1)\.
- Wanget al\.\(2024c\)R\. Wang, S\. Milani, J\. C\. Chiu, J\. Zhi, S\. M\. Eack, T\. Labrum, S\. M\. Murphy, N\. Jones, K\. Hardy, H\. Shen, F\. Fang, and Z\. Z\. ChenPATIENT\-Ψ\\Psi: using large language models to simulate patients for training mental health professionals\.InProceedings of EMNLP,Note:arXiv:2405\.19660Cited by:[§A\.2](https://arxiv.org/html/2610.27337#A1.SS2.p1.1.1),[§4\.2](https://arxiv.org/html/2610.27337#S4.SS2.p1.1),[Table 2](https://arxiv.org/html/2610.27337#S4.T2.6.3.4.1.1),[§6\.4](https://arxiv.org/html/2610.27337#S6.SS4.p2.1)\.
- Wanget al\.\(2025a\)X\. Wang, B\. Li, Y\. Song, F\. F\. Xu, X\. Tang, M\. Zhuge, J\. Pan, Y\. Song, B\. Li, J\. Singh,et al\.OpenHands: an open platform for AI software developers as generalist agents\.InProceedings of ICLR,Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p3.1)\.
- Wanget al\.\(2025b\)Y\. Wang, Q\. Yang, Z\. Zeng, L\. Ren, L\. Liu, B\. Peng, H\. Cheng, X\. He, K\. Wang, J\. Gao, W\. Chen, S\. Wang, S\. S\. Du, and Y\. ShenReinforcement learning for reasoning in large language models with one training example\.InAdvances in Neural Information Processing Systems,Note:arXiv:2504\.20571Cited by:[§A\.5](https://arxiv.org/html/2610.27337#A1.SS5.SSS0.Px1.p1.1),[Table 11](https://arxiv.org/html/2610.27337#A1.T11h.2.3.3.1.1),[§4\.5](https://arxiv.org/html/2610.27337#S4.SS5.p1.1)\.
- Wenget al\.\(2025a\)Y\. Weng, M\. Zhu, G\. Bao, H\. Zhang, J\. Wang, Y\. Zhang, and L\. YangCycleResearcher: improving automated research via automated review\.InProceedings of ICLR,Note:arXiv:2411\.00816Cited by:[§5\.2](https://arxiv.org/html/2610.27337#S5.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.27337#S5.T4.9.5.1.1.1)\.
- Wenget al\.\(2025b\)Y\. Weng, M\. Zhu, Q\. Xie, Q\. Sun, Z\. Lin, S\. Liu, and Y\. ZhangDeepScientist: advancing frontier\-pushing scientific findings progressively\.arXiv preprint arXiv:2509\.26603\.Cited by:[§1](https://arxiv.org/html/2610.27337#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1),[§5\.5](https://arxiv.org/html/2610.27337#S5.SS5.p1.1),[Table 7](https://arxiv.org/html/2610.27337#S5.T7.4.3.1.1.1)\.
- Xuet al\.\(2026\)T\. Xu, Z\. Qian, G\. Liu,et al\.Idea2Story: an automated pipeline for transforming research concepts into complete scientific narratives\.arXiv preprint arXiv:2601\.20833\.Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1)\.
- Yamadaet al\.\(2025\)Y\. Yamada, R\. T\. Lange, C\. Lu, S\. Hu, C\. Lu, J\. Foerster, J\. Clune, and D\. HaThe AI Scientist\-v2: workshop\-level automated scientific discovery via agentic tree search\.arXiv preprint arXiv:2504\.08066\.Cited by:[§1](https://arxiv.org/html/2610.27337#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1),[§5\.3](https://arxiv.org/html/2610.27337#S5.SS3.p1.1),[Table 5](https://arxiv.org/html/2610.27337#S5.T5.4.7.1.1.1)\.
- Yanget al\.\(2025a\)X\. Yang, X\. Yang, S\. Fang, Y\. Zhang, J\. Wang, B\. Xian,et al\.R&D\-Agent: an LLM\-agent framework towards autonomous data science\.arXiv preprint arXiv:2505\.14738\.Cited by:[§1](https://arxiv.org/html/2610.27337#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1),[§5\.5](https://arxiv.org/html/2610.27337#S5.SS5.p1.1)\.
- Yang and Bing \(2026\)Z\. Yang and L\. BingMOOSE\-Star: unlocking tractable training for scientific discovery by breaking the complexity barrier\.InProceedings of ICML,Cited by:[§2\.1](https://arxiv.org/html/2610.27337#S2.SS1.p2.1),[§5\.3](https://arxiv.org/html/2610.27337#S5.SS3.p1.1),[Table 5](https://arxiv.org/html/2610.27337#S5.T5.4.3.1.1.1),[§7](https://arxiv.org/html/2610.27337#S7.SS0.SSS0.Px3.p1.1),[§7](https://arxiv.org/html/2610.27337#S7.SS0.SSS0.Px4.p1.1)\.
- Yanget al\.\(2024\)Z\. Yang, X\. Du, J\. Li, J\. Zheng, S\. Poria, and E\. CambriaLarge language models for automated open\-domain scientific hypotheses discovery\.InFindings of ACL,Cited by:[§2\.1](https://arxiv.org/html/2610.27337#S2.SS1.p2.1)\.
- Yanget al\.\(2025b\)Z\. Yang, W\. Liu, B\. Gao, T\. Xie, Y\. Li, W\. Ouyang, S\. Poria, E\. Cambria, and D\. ZhouMOOSE\-Chem: large language models for rediscovering unseen chemistry scientific hypotheses\.InProceedings of ICLR,Cited by:[§2\.1](https://arxiv.org/html/2610.27337#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1)\.
- Yaoet al\.\(2023\)S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. NarasimhanTree of Thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems,Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1)\.
- Yuet al\.\(2025a\)H\. Yu, Z\. Hong, Z\. Cheng, K\. Zhu, K\. Xuan, J\. Yao, T\. Feng, and J\. YouResearchTown: simulator of human research community\.InProceedings of ICML,Note:arXiv:2412\.17767Cited by:[Appendix B](https://arxiv.org/html/2610.27337#A2.p7.1),[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1),[§5\.2](https://arxiv.org/html/2610.27337#S5.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.27337#S5.T4.9.7.1.1.1)\.
- Yuet al\.\(2025b\)Z\. Yu, K\. Feng, Y\. Zhao, S\. He, X\. Zhang, and A\. CohanAlphaResearch: accelerating new algorithm discovery with language models\.arXiv preprint arXiv:2511\.08522\.Cited by:[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.5](https://arxiv.org/html/2610.27337#S5.SS5.p1.1),[Table 7](https://arxiv.org/html/2610.27337#S5.T7.4.6.1.1.1)\.
- Yuanet al\.\(2025\)J\. Yuan, X\. Yan, S\. Feng, B\. Zhang, T\. Chen, B\. Shi, W\. Ouyang, Y\. Qiao, L\. Bai, and B\. ZhouDolphin: moving towards closed\-loop auto\-research through thinking, practice, and feedback\.arXiv preprint arXiv:2501\.03916\.Cited by:[§1](https://arxiv.org/html/2610.27337#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.27337#S2.SS2.p2.1),[§5\.5](https://arxiv.org/html/2610.27337#S5.SS5.p1.1),[Table 7](https://arxiv.org/html/2610.27337#S5.T7.4.2.1.1.1)\.
- Zhanget al\.\(2025\)M\. Zhang, X\. Yang, X\. Zhang, T\. Labrum, J\. C\. Chiu, S\. M\. Eack, F\. Fang, W\. Y\. Wang, and Z\. Z\. ChenCBT\-Bench: evaluating large language models on assisting cognitive behavior therapy\.InProceedings of NAACL,Note:arXiv:2410\.13218Cited by:[§A\.2](https://arxiv.org/html/2610.27337#A1.SS2.SSS0.Px1.p4.1),[§A\.2](https://arxiv.org/html/2610.27337#A1.SS2.p1.1.1),[§4\.2](https://arxiv.org/html/2610.27337#S4.SS2.p1.1),[Table 2](https://arxiv.org/html/2610.27337#S4.T2.6.3.4.1.1)\.
- Zhanget al\.\(2026\)Y\. Zhang, X\. Yang, X\. Yang, B\. Xian, Q\. Li, S\. Fang,et al\.Reasoning as gradient: scaling MLE agents beyond tree search\.arXiv preprint arXiv:2603\.01692\.Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1)\.
- Zhaoet al\.\(2025a\)K\. Zhao, W\. Lin, Q\. Zheng, F\. Xu, and Y\. LiDeep Ideation: designing LLM agents to generate novel research ideas on scientific concept network\.arXiv preprint arXiv:2511\.02238\.Cited by:[§5\.1](https://arxiv.org/html/2610.27337#S5.SS1.p2.1)\.
- Zhaoet al\.\(2025b\)X\. Zhao, B\. Zheng, C\. Si, H\. Yu, K\. Liu,et al\.The Ramon Llull’s Thinking Machine for automated ideation\.arXiv preprint arXiv:2508\.19200\.Cited by:[Appendix B](https://arxiv.org/html/2610.27337#A2.p7.1),[§5\.2](https://arxiv.org/html/2610.27337#S5.SS2.p1.1),[Table 4](https://arxiv.org/html/2610.27337#S5.T4.9.4.1.1.1)\.
- Zhenget al\.\(2023\)L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. StoicaJudging LLM\-as\-a\-judge with MT\-Bench and Chatbot Arena\.InAdvances in Neural Information Processing Systems, Datasets and Benchmarks Track,Cited by:[§6\.4](https://arxiv.org/html/2610.27337#S6.SS4.p1.1)\.

## Appendix AThe Cases in Full

This appendix gives each case of §[4](https://arxiv.org/html/2610.27337#S4)in full\. For every case except Case 1, whose description is in the body \(§[4\.1](https://arxiv.org/html/2610.27337#S4.SS1)\), the full account comes first\. It is followed by the listings from which the figure is drawn, one table of typed states and one table of transitions, with the reasoning annotation of each transition abbreviated to a phrase\. Node identifiers \(RQ, L, LS, I, IN, C, CE, R with an index\) follow Table[1](https://arxiv.org/html/2610.27337#S3.T1), and a letter suffix \(I3a, I3b\) marks the branches of a fork\. An abandoned idea update is an I node marked “abandoned” and numbered X1, X2, joined by a dashed edge \(⇢\\dashrightarrow\) that carries the reason for abandoning it\. Within a case, section, table and figure numbers in parentheses refer to the cited source, not to this paper\.

### A\.1Case 1: few\-shot table\-to\-text generation

*Source: annotated, or traced and confirmed, by its authors;[Chen et al\. \(2020\)](https://arxiv.org/html/2610.27337#bib.bib1)\.*

The description is in §[4\.1](https://arxiv.org/html/2610.27337#S4.SS1)\.

#### Listings\.

idtypecontent \(short\)RQ0RQHow to do table\-to\-text NLG with only a few hundred examplesL0LPointer\-generator network \(See et al\. 2017\), built on pointer networks; template NLG baselinesI0IPointer\-generator network trained on the few\-shot setCE0CEImplement pointer\-generator; train at 50/100/200/500 examplesR0RCopies table values well; connective language is poor at all sizesIN0INTwo skills are needed: copy \(easy from few shots\) and language modeling \(hard from few shots\)C1CHow to acquire language modeling with little in\-domain dataLS1LSSearch: few\-shot / pre\-trained language modelingL1LGPT\-2 and contemporaneous pre\-trained LMsIN1INThe LM skill can come from pre\-training instead of task dataC2CHow to combine a pre\-trained LM with copyingIN2IN\(from re\-reading L0\) The pointer\-generator is a switch between an LM and a copy distribution; the LM slot can be replacedI1IPre\-trained LM as decoder \+ copy mechanism \+ learned switch; explicit copy supervision on matched positionsCE1CEImplement I1; same training sizesR1RLarge gains, e\.g\., 36\.1 BLEU at 200 examples on Humans against 21\.3 for the pointer\-generator variant \(Base \+ switch\) and 4\.9 for the original baseline \(Table 1\); project concluded\#edgetransition1RQ0, L0→\\rightarrowI0Choose the standard architecture for copy\-heavy generation2I0→\\rightarrowCE0Implement the pointer\-generator3CE0→\\rightarrowR0Train and evaluate at 50 to 500 examples4R0→\\rightarrowIN0Analyze errors; copying succeeded, fluency failed5IN0→\\rightarrowC1Isolate the hard skill as the challenge6C1→\\rightarrowLS1Search for ways to get language modeling cheaply7LS1→\\rightarrowL1Find GPT\-2 and contemporaneous pre\-trained LMs8L1→\\rightarrowIN1Pre\-training supplies the LM skill9IN1→\\rightarrowC2The challenge narrows to combination10C2, L0→\\rightarrowIN2Re\-read L0; abstract the pointer\-generator to “switch between LM and copy”11IN2→\\rightarrowI1Substitute the LM slot, keep the switch and the copy supervision12I1→\\rightarrowCE1Implement I113CE1→\\rightarrowR1Run at the same training sizes; stop

### A\.2Case 2: from a prompted patient to PATIENT\-Ψ\\Psiand CBT\-Bench

*Sources:[Wang et al\. \(2024c\)](https://arxiv.org/html/2610.27337#bib.bib8)for the first three rounds, with the order of the intermediate states annotated, or traced and confirmed, by its authors;[Zhang et al\. \(2025\)](https://arxiv.org/html/2610.27337#bib.bib55)for the fourth\.*

#### Full account\.

The research question came from a gap reported by practitioners\. Their training did not prepare them for real patients, and trainees have few realistic ways to practice formulating a patient’s cognitive model, the core skill of cognitive behavioral therapy, before seeing real patients\. The initial idea is the simplest one, to prompt GPT\-4 to role\-play a patient with a given diagnosis\. The first outside information was not an experiment but a formative interview study with twelve mental\-health experts and trainees, and it played the role R plays in the other cases\. The interviewees reported that peer role\-play is unrealistic because it does not reflect actual therapy sessions, and that real patients differ in conversational style\. The paper reports the therapist\-like behavior of the GPT\-4 baseline only later, from its user study \(§4\.1\), where experts described it as “too forthcoming with responses” and one called it “almost like doing therapy with a therapist”\. The insight is that realism is a property of the patient’s*cognition being structured*, not of the surface text; a trainee learns to infer beliefs from behavior, so the simulated behavior must be generated from beliefs\. The paper states that integrating a patient’s cognitive model with an LLM can achieve high fidelity \(§1\), and the step from the interviews to this insight is annotated, or traced and confirmed, by the authors\. The challenge is how to give an LLM a structured cognition\.

That challenge sends the search into clinical psychology rather than NLP, and the literature returned is Beck’s cognitive model and its clinical instrument, the cognitive conceptualization diagram \(relevant history, core beliefs, intermediate beliefs, coping strategies, situations, automatic thoughts, emotions, behaviors\)\. The insight is a transfer of a general kind\. An LLM can simulate a cognitive process when given its parameters, so a CCD can be the conditioning input\. The first idea update is GPT\-4 conditioned on a CCD and instructed to let the diagram inform its answers without stating it\. The same formative interviews contributed a second component, six conversational styles \(plain, upset, verbose, reserved, tangent, pleasing\), because the experts emphasized that real patients show different conversational styles in therapy\.

The second round is about where CCDs come from, which is a challenge with no analog in the method cases, since there is no dataset of cognitive models\. The outside information is clinical transcripts \(Alexander Street, CBT\-filtered\); the implementation is GPT\-4 Turbo summaries of transcripts, then two clinical psychologists writing 106 CCDs from those summaries with their own expertise\. The third round is evaluation without ground truth\. The group evaluated three dimensions, fidelity to real patients, accuracy to the cognitive model as rated by experts, and perceived training effectiveness, with 20 experts and 13 trainees against a GPT\-4 baseline\. The results are the fourth injection, and one of them is an insight the paper reports explicitly\. LLM\-based evaluators give the GPT\-4 baseline higher fidelity scores than PATIENT\-Ψ\\Psi, the opposite of human experts, although the baseline conveys CBT knowledge “in a manner resembling a therapist speaking directly and explicitly, rather than a real patient” \(§5\)\. From that finding the paper concludes that automatic evaluation with LLMs cannot assess the fidelity of a simulated patient, so its claims rest on the expert ratings\. The paper leaves objective skill gains for future work, which it notes could be measured in longitudinal randomized controlled trials, since all its measures of training effectiveness are perceived improvements\.

The fourth round is the group’s next paper, CBT\-Bench\([Zhang et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib55)\), which shares five authors with PATIENT\-Ψ\\Psi, including its senior author\. The reason for the step, as read from the two papers, has been annotated, or traced and confirmed, by the authors\. Two results of PATIENT\-Ψ\\Psipoint the same way\. The expert ratings measured how accurately each simulated patient followed its cognitive model, so the cognitive model is central to the whole evaluation; and the finding that LLM judges give the unstructured baseline higher fidelity scores than PATIENT\-Ψ\\Psi, although GPT\-4 rates accuracy to the cognitive model much as the experts do \(§5\), showed that the models judge the realism of a structured patient poorly\. The insight is that the same cognitive model is needed on both sides of therapy, to simulate the patient and to assist the therapist\. PATIENT\-Ψ\\Psiused it to make a model behave like a patient, but whether a model can*analyze*a patient’s cognition, which is what assisting the therapist would require, had not been tested\. CBT\-Bench states the gap it fills as “work on professional assistance in real therapy settings remain limited”, with prior work addressing isolated tasks such as cognitive\-distortion classification while “many other critical stages in CBT … could potentially be enhanced” \(§1\), and it cites PATIENT\-Ψ\\Psifor the premise that “with clear modeling of the cognitive model of patients, LLMs could act more like real patients” \(§2\)\. The idea update is a three\-level benchmark on the therapist side, built with the same clinical collaborators, which tests CBT knowledge \(CBT\-QA, 220 questions\), cognitive\-model understanding \(cognitive\-distortion, primary core\-belief and fine\-grained core\-belief classification; 146, 184 and 112 examples\) and therapeutic response generation \(CBT\-DP, 156 deliberate\-practice exercises\)\. The result is the sixth injection\. Llama\-3\.1\-405B reaches 95\.0% on CBT\-QA against 75\.9% for Mistral\-7B, but current models “struggle with detecting fine\-grained cognitive disorders”, with Llama\-3\.1\-405B at an F1 of 43\.3 on cognitive\-distortion classification and 58\.2 on fine\-grained core\-belief classification \(Table 5\), and, by the authors’ account, lag behind human reference responses on deliberate practice \(§5\.3; Table 6\)\. The resulting insight is that knowing CBT and analyzing a patient’s cognitive structure are different capabilities, and only the first is present\. The challenge left open is how to get the second\.

The trajectory has four rounds and six states of evidence\. Two of the six are studies with people \(the formative interviews and the user study\) rather than papers or runs, which is what application ideas look like in an RPG, and the fourth round shows the group’s idea moving from the patient to the therapist because of its own evaluation results\.

*What a whole\-idea regeneration loop would have lost\.*A review loop would receive “not realistic” from the interviewees and regenerate a better patient prompt\. The actual changes were three localized steps, each attributed to one piece of evidence\. The conditioning input of I0 was replaced by a CCD \(transitions 4 and 5\) because R0 found practice through role\-play unrealistic and L1 supplied a parameterization an LLM can follow; a conversational\-style component was added \(transition 5\) because the same experts named the styles real patients show; and the paper concluded that its claims must rest on expert ratings \(transition 8\) because R1 showed LLM judges give the baseline higher fidelity scores than PATIENT\-Ψ\\Psi\. The third step is the one a regeneration loop would have overwritten silently\. It is a conclusion about how the idea can be judged, caused by a result about the judge, and it is only recordable as a node \(IN2\) with a transition back to R1\. The fourth round adds a fourth kind of edit\. The idea’s*object*changed, from the patient to the therapist, and the reason is a pair of results about the evaluation \(expert\-rated accuracy to the cognitive model; LLM judges that misjudge the realism of a structured patient\) that a regeneration loop would have consumed as a score and forgotten\.

#### Listings\.

idtypecontent \(short\)sourceRQ0RQHow can CBT trainees practice formulating a patient’s cognitive model before seeing real patientsWang et al\. 2024cI0IPrompt GPT\-4 to role\-play a patient with a diagnosisWang et al\. 2024cR0R \(formative interviews, 12 experts and trainees\)Training did not prepare them for real practice; peer role\-play is unrealistic; real patients show different conversational stylesWang et al\. 2024c §1, App\. BIN0INRealism = behavior generated from a structured cognition; trainees must infer beliefs from behaviorWang et al\. 2024cC1CHow to give an LLM a structured cognitionWang et al\. 2024cLS1LSSearch in clinical psychology: how CBT models a patientWang et al\. 2024cL1LBeck’s cognitive model; cognitive conceptualization diagram \(8 components\)Wang et al\. 2024cIN1INAn LLM can simulate a cognitive process when given its parameters; a CCD can be the conditioning inputWang et al\. 2024cI1IGPT\-4 conditioned on a CCD \+ one of six conversational styles; answers informed by, but not stating, the diagramWang et al\. 2024cC2CWhere do CCDs come from? No dataset existsWang et al\. 2024cL2L \(data\)Alexander Street CBT therapy transcriptsWang et al\. 2024cCE1CEGPT\-4 Turbo summaries of transcripts→\\rightarrowtwo clinical psychologists write 106 CCDs \(PATIENT\-Ψ\\Psi\-CM\)Wang et al\. 2024cC3CHow to evaluate a simulated patient with no ground\-truth conversationWang et al\. 2024cI2IEvaluation on three dimensions: fidelity rated by experts \+ accuracy to the CCD rated by experts \+ perceived effectiveness rated by experts and trainees, against the GPT\-4 baseline; PATIENT\-Ψ\\Psi\-TRAINER interfaceWang et al\. 2024c §3CE2CEUser study: 20 experts, 13 trainees, randomized orderWang et al\. 2024cR1RFidelity and effectiveness significantly above baseline and above textbooks/role\-play; CCD components rated 4\.1 to 4\.3 / 5Wang et al\. 2024cIN2INLLM judges rank the baseline above PATIENT\-Ψ\\Psion fidelity, opposite to experts: automatic evaluation is unreliable for this applicationWang et al\. 2024cC4C \(open\)Objective skill gains need longitudinal RCTsWang et al\. 2024cIN3INThe cognitive model is central on both sides: PATIENT\-Ψ\\Psiused it to make a model behave like a patient; whether a model can analyze a patient’s cognition \(the therapist’s task\) is untested; LLM judges misjudged the realism of a structured patient \(reason annotated, or traced and confirmed, by the authors\)Wang et al\. 2024c §4\.2, §5; Zhang et al\. 2025 §1C5CCan LLMs do the therapist\-side cognitive\-model work, beyond isolated distortion classification?Zhang et al\. 2025 §1 \(quoted\)L3LPrior work: isolated CBT tasks \(distortion classification\); “work on professional assistance in real therapy settings remain limited”; PATIENT\-Ψ\\Psias premiseZhang et al\. 2025 §1, §2I3ICBT\-Bench: three levels: CBT\-QA \(220\); cognitive\-model understanding: CBT\-CD, CBT\-PC, CBT\-FC \(146, 184, 112\); CBT\-DP deliberate practice \(156\); built with the clinical collaboratorsZhang et al\. 2025 §3CE3CEEvaluate open and closed LLMs on all levels; human references for CBT\-DPZhang et al\. 2025 §4, §5R2RCBT\-QA 95\.0% \(Llama\-3\.1\-405B\) vs 75\.9% \(Mistral\-7B\); Llama\-3\.1\-405B F1 43\.3 on CBT\-CD and 58\.2 on CBT\-FC \(75\.3 on CBT\-PC\); models lag behind human references on deliberate practice by the authors’ accountZhang et al\. 2025 Tables 5, 6, §5\.3IN4INReciting CBT knowledge and analyzing a patient’s cognitive structure are different capabilities; only the first is presentZhang et al\. 2025 abstract, §5C6C \(open\)How to get cognitive\-structure analysis rather than recitationZhang et al\. 2025 §6\#edgetransition1RQ0→\\rightarrowI0Default choice2I0→\\rightarrowR0Formative interviews with experts and trainees instead of a run3R0→\\rightarrowIN0→\\rightarrowC1Diagnose lack of realism as lack of structured cognition \(reason annotated, or traced and confirmed, by the authors; the paper states that a cognitive model gives high fidelity, §1\)4C1→\\rightarrowLS1→\\rightarrowL1→\\rightarrowIN1Search a different field; identify the CCD as the parameterization an LLM can simulate5IN1, R0→\\rightarrowI1Combine CCD conditioning with the styles experts named6I1→\\rightarrowC2→\\rightarrowL2→\\rightarrowCE1Build the missing data with clinicians7I1→\\rightarrowC3→\\rightarrowI2→\\rightarrowCE2→\\rightarrowR1Design and run evaluation without ground truth8R1→\\rightarrowIN2→\\rightarrowC4Learn that LLM evaluation fails; leave objective outcomes open9R1, IN2→\\rightarrowIN3→\\rightarrowC5Read the evaluation results as a statement about the cognitive model on both sides; move the question to the therapist’s side \(reason annotated, or traced and confirmed, by the authors\)10C5→\\rightarrowL3→\\rightarrowI3→\\rightarrowCE3→\\rightarrowR2Survey what exists; build the therapist\-side benchmark with the same clinicians; evaluate11R2→\\rightarrowIN4→\\rightarrowC6Separate knowledge from analysis; leave analysis open

### A\.3Case 3: LDC, from fine\-tuning an idea generator to controlling it dimension by dimension

*Source:[Li et al\. \(2024a\)](https://arxiv.org/html/2610.27337#bib.bib20), in version 3 of the arXiv paper \(2026\); quotations, section numbers, and table numbers follow that version\. The states are traced from the paper, and the states and reasons are annotated, or traced and confirmed, by its authors\.*

#### Full account\.

The states and reasons below are taken from the published paper and from its authors, and the order of the intermediate states follows the paper’s own presentation, which, like every published paper, may be tidier than the project was, as §[2\.1](https://arxiv.org/html/2610.27337#S2.SS1)discusses\. The research question is how to make a language model produce research ideas that meet the standards experts actually apply\. The literature the idea starts from is the line of ideation systems built on pre\-trained models, such as ResearchAgent \(§1, §2\)\. The paper’s reading of it is the first insight\. These systems “primarily rely on pre\-trained models without task\-specific learning” \(§1\), so their ideas are “misaligned with expert standards” \(abstract\)\. The initial idea is therefore to*learn*the task by fine\-tuning an LLM on pairs of a paper and the research idea extracted from it, with 1,000 ICLR papers in the fine\-tuning split \(§4\.1\) and the most supporting prior work chosen by prompting LLaMA with the paper and the citation counts of the candidates \(§3\.2\)\. The paper states what this stage is for\. Supervised fine\-tuning is meant to “introduce the model with the general task format” and to “improve model training stability in RL”, and its training data “does not need to achieve high scores in terms of the metrics”, which are left to reinforcement learning \(§3\.2\)\. The challenge, stated in the introduction, is that “optimizing idea generation towards each of the key dimensions while achieving a balanced trade\-off remains a critical yet unresolved question” \(§1\)\.

The first idea update answers the first half of that challenge with reinforcement learning and three reward models, for novelty, feasibility, and effectiveness, whose labels come from review scores or from prompting an LLM on the paper and its reviews \(§3\.3\)\. The authors confirm that this feedback design follows ResearchAgent\([Baek et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib23)\), which refines ideas iteratively with feedback from reviewing agents on several criteria\. In LDC the feedback on each dimension is given by a reward model during training\. With the Llama\-3\-8B backbone used for every later number in this case, the result is 5\.7 overall against 5\.3 for ResearchAgent \(Table 1\)\. The second update is motivated by the literature rather than by this result\. The introduction takes from prior work,[Si et al\. \(2025b\)](https://arxiv.org/html/2610.27337#bib.bib18)and one further study, the “inevitable innovation\-feasibility trade\-off”, in which highly novel ideas often lack feasibility \(§1\), and the method section gives the same reason for the controllers, since the dimensions “often exhibit interdependency and trade\-offs” \(§3\.4\)\. The insight is that the dimensions are interdependent, so the generator needs a separate handle for each\. The second idea update adds dimensional controllers on top of the per\-dimension rewards, one per dimension, implemented as layer\-wise multiplicative perturbations of the hidden states with a steering value each\. The steering values are set to 1 during training and can be set per dimension at inference time \(§3\.4\)\. With the same backbone, the novelty controller alone scores 5\.8 against 5\.7 for reinforcement learning without controllers, the feasibility and effectiveness controllers alone score 5\.3 and 5\.7, and all three together reach 6\.1 \(Table 1\)\.

The third round comes from the same trade\-off\. As the authors confirm, a fixed steering value raises one dimension at the cost of another across the whole idea, so the emphasis has to change within it\. The paper states where it should change\. Emphasis should be context\-aware, “such as prioritizing novelty in the method part and feasibility in the experiment planning” \(§1\), and its analysis of how the dimensions vary with sentence position supports this \(§5\.2, Figure 4\)\. The idea update is a sentence\-level decoder, a small recurrent network that predicts the three steering values for each sentence from the text generated so far, trained against per\-sentence targets \(§3\.5\)\. Dynamic control reaches 6\.3 overall, significant against all baselines \(Table 1\), and human evaluation agrees with the automatic scores \(5\.8 overall, Table 2; Pearson 0\.871 on the overall score, Table 3\)\. The experts’ comment names the trade\-off again, since novelty steering produced “more creative, though sometimes less practical, ideas” \(§4\.4\)\. A later analysis of the finished system confirms the trade\-off directly\. Raising the weight of the novelty controller from 1\.0 to 4\.0 lifts novelty from 6\.4 to 7\.3 and drops feasibility from 6\.1 to 4\.9 \(§5\.1, Table 6\)\. The challenges the paper leaves open are the interpretability of the learned controllers, the bias that prompted reward labels may inherit, and whether citation prediction could serve as a judge \(Limitations\)\.

The trajectory has three rounds and seven states of evidence, three from the literature and four experimental\. Every idea update is a change to one component of the generator, and each is justified by a reading of prior work, a reading of the task, or, for the decoder, the same trade\-off together with an analysis of sentence positions\. LDC optimizes a one\-shot generator against three rewards, and its trade\-off finding leaves open whether the unit of improvement should be the generated text or the path by which it is updated, which is the question this paper takes up\.

*What a whole\-idea regeneration loop would have lost\.*A loop that receives “5\.7 vs 5\.3” regenerates a better reward or a bigger model\. The update that mattered came from a trade\-off reported in prior work \(L1\), read as a property of the task \(IN2\)\. Raising one dimension lowers another, so the fix is not a better reward but a separate handle per dimension\. The paper’s own sweep \(R3, Table 6\) confirms the trade\-off only later, on the finished controllers\. The same trade\-off then led the next update to localize the handles to parts of the idea\. Both changes are to the*architecture of the generator*, and the reason for the first sits on the edge from the literature to the controllers\. In the paper that reason is a sentence in the introduction and one in the method section, separate from the sweep that later confirms it\.

#### Listings\.

idtypecontent \(short\)sourceRQ0RQHow to generate research ideas that meet expert standards of novelty, feasibility and effectivenessLi et al\. §1L0LIdeation systems built on pre\-trained models without task\-specific learning \(e\.g\., ResearchAgent\)Li et al\. §1, §2L2LResearchAgent: ideas refined iteratively with feedback from reviewing agents on several criteriaannotated, or traced and confirmed, by the authorsIN0INPrompting alone leaves outputs “misaligned with expert standards”; the task must be learnedLi et al\. abstract \(quoted\), §1I0ISupervised fine\-tuning on 1,000 \(paper→\\rightarrowextracted idea\) pairs; supporting work chosen by prompting with citation countsLi et al\. §3\.2, §4\.1IN1INSFT gives the task format and RL stability; the quality dimensions are left to RLLi et al\. §1, §3\.2 \(stated purpose of the SFT stage\)C1CHow to optimize toward each dimension and balance them \(“critical yet unresolved”\)Li et al\. §1 \(quoted\)I1IRL with three reward models \(novelty, feasibility, effectiveness\); labels from review scores or prompted from paper and review contentLi et al\. §3\.3R0RSFT \+ RL 5\.7 overall \(LLaMA\-3\-8B\) vs ResearchAgent 5\.3Li et al\. Table 1L1LPrior work: the “inevitable innovation\-feasibility trade\-off”; highly novel ideas often lack feasibilityLi et al\. §1 \(quoted\), citing Si et al\. 2024 and Yang et al\. 2024IN2INThe dimensions “often exhibit interdependency and trade\-offs”; each needs its own handleLi et al\. §1, §3\.4 \(quoted\)C2CHow to control each dimension separately at inference timeLi et al\. §3\.4I2IDimensional controllers on top of the per\-dimension rewards: per\-dimension layer\-wise multiplicative perturbations with steering valuesε\\varepsilonLi et al\. §3\.4R1RNovelty controller alone 5\.8 \(feasibility 5\.3, effectiveness 5\.7\) vs 5\.7 without controllers; all controllers, static, 6\.1 \(LLaMA\-3\-8B\)Li et al\. Table 1IN3INEmphasis should depend on the part of the idea: novelty in the method, feasibility in the experiment planLi et al\. §1 \(quoted\), §5\.2I3ISentence\-level decoder: an RNN predicts \(εn\\varepsilon\_\{n\},εf\\varepsilon\_\{f\},εe\\varepsilon\_\{e\}\) per sentence from the generated contextLi et al\. §3\.5R2RDynamic 6\.3 overall \(significant against all baselines\); human eval 5\.8 overall; Pearson 0\.871 with automatic on the overall score; experts: more creative, “sometimes less practical”Li et al\. Tables 1–3, §4\.4R3R \(later analysis\)Novelty controller weight 1\.0→\\rightarrow4\.0: novelty 6\.4→\\rightarrow7\.3, feasibility 6\.1→\\rightarrow4\.9; confirms the trade\-off of L1Li et al\. §5\.1, Table 6C3C \(open\)Controller interpretability; bias in prompted reward labels; citation prediction as a judgeLi et al\. Limitations\#edgetransition1RQ0, L0→\\rightarrowIN0→\\rightarrowI0Read prompting\-based ideation as unlearned; fine\-tune2I0→\\rightarrowIN1→\\rightarrowC1State what SFT does and does not optimize; pose the multi\-dimension challenge3C1, L2→\\rightarrowI1→\\rightarrowR0Add per\-dimension reward models, with the feedback design taken from ResearchAgent \(annotated, or traced and confirmed, by the authors\); measure4L1→\\rightarrowIN2→\\rightarrowC2Read the trade\-off reported in prior work as a property of the task; ask for separate control \(reason stated in §1 and §3\.4\)5C2→\\rightarrowI2→\\rightarrowR1Add dimensional controllers; measure6IN2, R1→\\rightarrowIN3→\\rightarrowI3A fixed setting trades one dimension against another across the whole idea, so localize emphasis to parts of the idea; add the sentence\-level decoder \(trade\-off annotated, or traced and confirmed, by the authors; parts stated in §1 and §5\.2\)7I3→\\rightarrowR2Measure with automatic and human judges8I2→\\rightarrowR3Later analysis: sweep the novelty controller weight; the trade\-off of L1 reappears9R2→\\rightarrowC3Record open questions

### A\.4Case 4: MLR\-Copilot, from a prompted idea agent to a trained one

*Source:[Li et al\. \(2024b\)](https://arxiv.org/html/2610.27337#bib.bib41), versions 1 \(August 2024\) and 3 \(November 2025\) of the arXiv paper; section and table numbers follow the version named\. The states are traced from the two versions, and the states and reasons are annotated, or traced and confirmed, by the authors\.*

#### Full account\.

The research question is how to carry machine learning research from an existing paper to executed experiments automatically\. The first version places the system between two kinds of prior work\. Ideation systems produce ideas that are never executed, and systems that automate experiments make small edits such as tuning hyperparameters and lack robust feedback\. The initial idea has two agents\. An idea agent, a prompted LLM, reads a paper and recent related work and writes first a hypothesis and then an experiment plan, and an experiment agent adapts the paper’s prototype code to the plan and runs it\. This initial system retrieved no models or datasets and had no channel for human feedback\.

The first idea update takes one component from each of two papers\. The experiment agent is rebuilt on the ReAct framework, in which each action follows a stated thought and is followed by an observation, as in the research agent of MLAgentBench\([Huang et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib71)\), and the iterative feedback through which execution results refine the work follows the design of ResearchAgent\([Baek et al\., 2025](https://arxiv.org/html/2610.27337#bib.bib23)\)\. The second update has two sources of different kinds\. The Hugging Face Hub, a repository of models and datasets, enters as an external resource, through one researcher’s idea that retrieving candidates from it would match models to the task better, and no result preceded it\. The tool through which a person can be asked for help was added after the agent was seen to drift further from the goal over its iterations, and human correction largely fixed the direction\. The result is the system of the first version\. On five tasks, with eight trials each, its ideas score higher than those of a baseline prompted with the core paper alone, for example 3\.9 against 3\.1 on innovativeness as rated by three experts, and its experiments improve on the prototype by 39\.7% on average with GPT\-4, with 40% of runs improving by at least 10%\. A single\-prompt baseline, reported in the third version but run before the idea agent changed, fails on every task because it cannot detect and correct execution errors\.

The third version keeps the experiment stages and replaces the idea agent with a trained Llama model\. It is first fine\-tuned on ideas and plans extracted from 1,000 papers and then refined by reinforcement learning with reward models for novelty, feasibility, and effectiveness, trained on OpenReview feedback for 4,271 ICLR and NeurIPS papers from 2023 and 2024\. The paper attributes the method to a later paper by the same group \(Case 3\) and uses it “without steering at inference time”\. The reason for the change, as the authors give it, is that this better ideation method had become available\. The limitations section states the open challenge that the stages run largely in sequence and need transitions back from implementation to ideation\.

*What a whole\-idea regeneration loop would have lost\.*A loop that receives the third version’s scores would record a better system\. It would not record that two of the three idea updates took components from other papers \(a benchmark’s agent framework, another system’s feedback design, and the group’s own idea generator\), that one component came from an external resource on a researcher’s idea and another from a correction for an observed drift, or that the paper ends by asking for the loop from results back to the idea that the RPG’s fourth property describes\.

#### Listings\.

idtypecontent \(short\)sourceRQ0RQCarry ML research from an existing paper to executed experiments automaticallyv1 §1L0LIdeation systems whose ideas are not executed; experiment automation limited to small edits, without robust feedbackv1 §1; v3 §1C1CConnect ideas to code that runsv1 §1, §2I0IPrompted idea agent \(paper \+ related work→\\rightarrowhypothesis→\\rightarrowplan\); experiment agent adapts and runs the paper’s prototype code; no model or data retrieval, no human feedbackannotated, or traced and confirmed, by the authorsL1LMLAgentBench: a research agent built on the ReAct framework \(thought, action, observation\)annotated, or traced and confirmed, by the authorsL2LResearchAgent: iterative feedback that refines the workannotated, or traced and confirmed, by the authorsI1IExperiment agent rebuilt on ReAct, with iterative feedback from executionv1 §2\.2–2\.3, §4; sources annotated, or traced and confirmed, by the authorsR0RThe agent drifts further from the goal over its iterationsannotated, or traced and confirmed, by the authorsIN1INHuman correction largely fixes the directionannotated, or traced and confirmed, by the authorsL3LHugging Face Hub: a repository of models and datasets \(external resource\)v1 §2\.2; annotated, or traced and confirmed, by the authorsI2IAdd retrieval of candidate models and datasets from Hugging Face and a tool for asking a person for help \(the system of v1\)v1 §2\.2–2\.3, App\. A–BR1RFive tasks \(SemRel, ELLIPSE, IMDB, Spaceship\-Titanic, Identify\-Contrails\), eight trials\. Ideas: innovativeness 3\.1→\\rightarrow3\.9, similarity 0\.32→\\rightarrow0\.16; execution: \+39\.7% over prototype \(GPT\-4\), 40% of runs≥\\geq10%v1 §3, Tables 1–4R2RSingle\-prompt baseline fails on every task \(0% success\), since it cannot detect and correct execution errors; run before I3v3 §3\.2\.2; order annotated, or traced and confirmed, by the authorsIN2INExecution succeeds through its feedback loopv3 §3\.2\.2L4LThe group’s idea\-generation method \(Case 3\): fine\-tuning, then reinforcement learning with novelty, feasibility, effectiveness reward modelsv3 §2\.1, fn\. 4I3IIdea agent trained \(Llama\): fine\-tuning on 1,000 papers, reinforcement learning with reward models from OpenReview feedback on 4,271 ICLR/NeurIPS papers, no steering at inference; experiment stages keptv3 §2\.1, §3\.2\.1, App\. BR3RIdeas: innovativeness 3\.9 vs ResearchAgent 3\.8 vs base LLM 3\.1; similarity 0\.13, 0\.15, 0\.32; execution: \+44\.2% \(Claude\-3\.7\), 50% of runs≥\\geq10%v3 Tables 1–4C2C \(open\)Stages run largely in sequence; transitions back from implementation to ideation are neededv3 §7\#edgetransition1RQ0, L0→\\rightarrowC1Prior systems either do not execute ideas or only make small edits \(annotated, or traced and confirmed, by the authors\)2C1→\\rightarrowI0Start execution from the paper’s own prototype code \(annotated, or traced and confirmed, by the authors\)3I0, L1, L2→\\rightarrowI1Take the ReAct framework from MLAgentBench’s agent and the iterative feedback from ResearchAgent \(annotated, or traced and confirmed, by the authors\)4I1→\\rightarrowR0→\\rightarrowIN1The agent drifts; a person can correct the direction \(annotated, or traced and confirmed, by the authors\)5L3, IN1→\\rightarrowI2Add retrieval from the Hub, on one researcher’s idea that it matches models to the task better, and human feedback to correct the drift \(annotated, or traced and confirmed, by the authors\)6I2→\\rightarrowR1Evaluate on five tasks7R1, R2→\\rightarrowIN2Execution feedback, not the prompt, makes the runs succeed \(stated in v3 §3\.2\.2\)8I2, L4→\\rightarrowI3Once the group’s ideation method exists, replace the prompted idea agent with it and keep the experiment stages \(annotated, or traced and confirmed, by the authors\)9I3→\\rightarrowR3Evaluate again10R3→\\rightarrowC2Record what remains open

### A\.5Case 5: on\-policy distillation from one training example

*Source: Fu, He, Zuo, Huang, Zhang, Xiao, Qian, Luo, Gao, Wang, Liu, Ding, Xiao, “Rethinking On\-Policy Distillation of Large Language Models II: One Training Example”, arXiv 2609\.04172, September 2026 \(Tsinghua, UCAS and collaborators\)\. Part I is Li et al\., “Rethinking On\-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe”, arXiv 2604\.13016, April 2026\.*A shorter account is in §[4\.5](https://arxiv.org/html/2610.27337#S4.SS5)\.

#### Full account\.

The second paper names one\-shot RLVR, not the first paper, as the source of its idea, and cites the first among studies of the training dynamics of OPD and for one of its metrics\. The initial idea was the transfer of an experimental lens rather than of a technique\. One\-shot RLVR\([Wang et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib56)\)had been used as an extreme controlled experiment to isolate the role of training data in RL with verifiable rewards, and the group brought the same lens to on\-policy distillation\. As an idea it is underspecified, since “train OPD on one example and see what happens” has no mechanism, no prediction, and no boundary\.

The first round is the direct experiment, training on a single query for hundreds of steps, with full\-data OPD as the reference \(17k queries in mathematics\), across four domains and three model families\. The result is the first piece of evidence, and it is sharper than anything the idea text contained\. One\-shot OPD keeps improving for hundreds of steps, recovering 69% of the teacher–student gap and 87% of full\-data OPD’s gain\. The insight from that result is about*where*supervision comes from\. Clustering the states the student visits shows that rollouts from one query reach 65\.9% of the state clusters by step 100 and 71\.5% by step 300, so a single input exposes the student to a large share of the teacher supervision that full\-data training exposes\. That insight raises a challenge that a text idea could not have raised, whether it is the query’s content that matters or only the states its rollouts generate\.

The second round answers it by ablation\. Content\-free templates and off\-domain chat prompts drive OPD nearly as effectively as the full training set, response diversity from one query improves accuracy monotonically, and sixteen diverse queries reach 98\.9% coverage and match full\-data OPD\. The insight is that an input is useful largely because it starts the student reasoning, so inputs are state generators rather than task instructions\. This produces the second challenge \(C2\), why absorption still takes hundreds of steps if states are cheap and broad\. The third round measures the absorption rate, the share of the remaining teacher–student distance that one update closes, in runs on one, four, sixteen, and 17k queries, and finds that it declines the same way at every size\. An ablation that holds the states fixed, reusing 64 trajectories sampled once from the initial student in a one\-shot run, still gains accuracy for about 200 steps, so a steady supply of fresh states is not what keeps a run going\. The final idea is a reframing with a recipe attached\. OPD is data\-overfed but algorithm\-starved, queries should be selected by the states they induce, and future work should go into the step efficiency of the algorithm rather than into data\.

The trajectory has three rounds and four states of evidence\. The decisive transitions do not combine existing components\. They sharpen a claim, propose a mechanism for it, and then ablate what the mechanism predicts\.

*What a whole\-idea regeneration loop would have lost\.*Here the final idea is a re\-scoped claim rather than a new idea, and each challenge is a question about one specific state\. The first challenge \(C1\) asks whether the state\-coverage mechanism, not the query content, explains the first result, and the second \(C2\) asks why the mechanism leaves absorption slow\. An agent that passes back “68\.5 against 69\.8”, one\-shot against full\-data accuracy at step 300, as a score would either stop, because the score is good, or generate an unrelated next experiment\. The ablation of content against states exists because the insight predicted it, and that justification is recorded on one transition and nowhere else\. Without attribution to states, the final claim reads as an interpretation of a results table rather than as the conclusion of two ablations that a stated mechanism predicted\.

#### Listings\.

idtypecontent \(short\)RQ0RQHow much training data does on\-policy distillation need, and whyL0LOne\-shot RLVR as a controlled experiment isolating the role of data\([Wang et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib56)\); Part I\([Li et al\., 2026](https://arxiv.org/html/2610.27337#bib.bib7)\), cited in Part II for OPD’s training dynamics and the top\-kkoverlap metricI0IBring the one\-shot lens to OPD: train on a single query \(no mechanism, no prediction, no boundary yet\)CE0CEOne\-shot OPD on one query per domain for hundreds of steps; full\-data OPD as reference \(17k queries in math\); 4 domains \(math, code, instruction following, agentic tool use\)×\\times3 families \(Qwen, Llama, OLMo\)R0ROne\-shot keeps improving for hundreds of steps; 68\.5 vs 69\.8 at step 300 \(three\-benchmark average\); 69% of the teacher–student gap and 87% of full\-data gain recoveredIN0INState coverage \(fraction of 200 state clusters reached\): one query’s rollouts reach 65\.9% by step 100 and 71\.5% by step 300; supervision breadth comes from rollouts, not from promptsC1CDoes the query’s*content*matter, or only the states its rollouts generate?CE1CEAblations: content\-free templates; off\-domain WildChat prompts; response diversity 1/4/16/64 trajectories; query diversity 1/4/16 queriesR1RTemplates and off\-domain prompts nearly match the full set; accuracy rises with trajectory diversity; 16 diverse queries reach 98\.9% coverage and match full\-data OPDIN1INAn input is useful because it starts the student reasoning: inputs are state generatorsC2CIf states are cheap and broad, why does absorption still take hundreds of steps?CE2CEAbsorption rate \(share of the remaining teacher–student distance closed per update\) on 1, 4, 16 and 17k queries; separately, a frozen\-state one\-shot run on 64 trajectories sampled once from the initial student \(accuracy, top\-16 overlap, overlap\-token advantage\)R2RThe absorption rate declines the same way at every training\-set size; the frozen\-state run keeps gaining accuracy for about 200 steps, so fresh states are not what keeps a run goingIN2INThe bottleneck is the algorithm’s absorption rate, not the quantity of supervisionI1IClaim \+ recipe: “OPD is data\-overfed but algorithm\-starved”; select queries by the states they induce; future work on step efficiency of OPD\#edgetransition1L0→\\rightarrowI0Transfer an experimental lens from RLVR to OPD2I0→\\rightarrowCE0→\\rightarrowR0Sharpen “how much data” into a measurable one\-shot experiment3R0→\\rightarrowIN0Mechanize: explain the gain by state coverage4IN0→\\rightarrowC1Derive a discriminating question from the mechanism5C1→\\rightarrowCE1→\\rightarrowR1→\\rightarrowIN1Ablate content vs states; confirm inputs are state generators6IN1→\\rightarrowC2The mechanism leaves a residual: slow absorption7C2→\\rightarrowCE2→\\rightarrowR2→\\rightarrowIN2Isolate the algorithmic bottleneck8IN0, IN1, IN2→\\rightarrowI1Reframe the field’s assumption and attach a recipe

### A\.6Case 6: on\-policy distillation as told by its authors

Initial RPG \(round 1\)Research Question \#0: cheap reasoning for a small modelIdea Update \#0: off\-policy distillation from the teacherCode & Exp\. Plan \#0: SFT on 400k teacher promptsExp\. Results \#0: 60% AIME’24 at 400k;∼\\sim2M needed for 70%Literature \#0: Qwen3 report: OPD 1,800 GPU\-h vs 17,920 for RLLiterature \#1: DAgger, GKD, MiniLLM, process rewardsInsight \#0: off\-policy compounds errors; RL is sparseCode & Exp\. Plan \#3: one prompt reused for 20 stepsExp\. Results \#6: one prompt nearly matches the teacherInsight \#3: reverse KL learns the distribution, not an answerChallenge \#3: how far does data efficiency go, and why \(studied later in Case 5\)Idea Update \#1: student samples, teacher grades each tokenCode & Exp\. Plan \#1: OPD from the 400k SFT checkpoint of 8B\-BaseExp\. Results \#1: 70% in∼\\sim150 steps; 9–30x cheaperChallenge \#1: why is dense supervision so much cheaper?Code & Exp\. Plan \#2: RL\-train 8B\-Base, distill it back into the baseExp\. Results \#2: 7–10x faster than RL; 50–100x less computeInsight \#1: RL is search over strategies; distillation skips itIdea Update \#3: old checkpoint as teacher; OPD after mid\-trainingExp\. Results \#5: IF\-eval 79% to 83%; internal QA 36% to 41%Research Question \#1: assistant that knows company docs, keeps IFIdea Update \#2: mid\-train \(SFT\) on documents mixed with chat dataExp\. Results \#3: QA up, IF\-eval down at any mixing weightExp\. Results \#4: SFT on the model’s own samples still degrades itInsight \#2: finite batches drift though KL is 0; fix the teacherChallenge \#2: add knowledge, keep the post\-trained behavior12345678910Reasoning on transition 9: no mixing weight preserves IF\-eval \(Exp\. Results \#3\), and the post’s discussion shows that even SFT on a model’s own samples degrades it \(Exp\. Results \#4\), so the training target must be a fixed teacher\. Reuse the on\-policy distillation objective of Idea Update \#1 with the earlier post\-trained checkpoint as the teacher\.Transitions 2 and 9 have several sources\. Research Question \#1 opens a second strand \(bottom row\)\. One reasoning record is shown\.Figure 7:Case 6, on\-policy distillation, as an RPG\. Three and a half rounds, seven experimental and two literature states of evidence, three idea updates after the initial one\. Listing below\.*Source: Kevin Lu with others at Thinking Machines Lab, “On\-Policy Distillation”, blog post, October 27, 2025 \(section headings cited in the tables\)\. The prior work the post takes as input is DAgger\([Ross et al\., 2011](https://arxiv.org/html/2610.27337#bib.bib12)\), GKD\([Agarwal et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib10)\), MiniLLM\([Gu et al\., 2024](https://arxiv.org/html/2610.27337#bib.bib11)\), process rewards\([Lightman et al\., 2023](https://arxiv.org/html/2610.27337#bib.bib13)\), and the Qwen3 technical report\([Qwen Team, 2025](https://arxiv.org/html/2610.27337#bib.bib14)\)\. This is a single\-source case\. Every node is traceable to a section of the post\.*

#### Full account\.

The post reports its runs in separate sections and states the reason for most method choices, which is why it can be drawn as an RPG; where it presents an experiment after the result that experiment explains, we order the states by their logical dependence\. The research question is how to give a small model the reasoning ability of a large one at low cost\. The initial idea is off\-policy distillation, which samples solutions from the teacher and fine\-tunes the student on them\. The authors ran it\. Fine\-tuned on 400k reasoning prompts with responses generated by QwQ\-32B \(OpenThoughts\-3\), the student, Qwen3\-8B\-Base, reaches 60% on AIME’24, and extrapolating the log\-linear trend says roughly 2M prompts would be needed to reach 70%\. That result is the first injection\. Two pieces of literature enter alongside it\. One is the Qwen3 report, which had reached 74\.4% with on\-policy distillation in 1,800 GPU hours against 17,920 for RL, and the other is the imitation\-learning line \(DAgger, GKD, MiniLLM\)\. The post states the next insight in its untitled introduction, before it reports any run, so placing it after the first result is our ordering\. It is a diagnosis of both baselines at once\. Off\-policy training makes the student learn in contexts the teacher frequents rather than ones the student will find itself in, so errors compound; RL trains on the student’s own contexts but gives very sparse feedback\. The idea update combines the two\. It samples from the student, has the teacher grade every token, and minimizes the per\-token reverse KL, a one\-line change on top of an RL loop with a KL regularizer\. Run from the same checkpoint, it reaches 70% AIME’24 in about 150 steps\. That completes the first round\.

The second round is a why\-question about that number\. To isolate the effect of dense supervision the authors train Qwen3\-8B\-Base with RL and then distill the RL\-trained policy back into the same base model, and distillation learns the RL\-trained policy in 7 to 10 times fewer gradient steps than RL needed\. The insight, which the post argues in its discussion without a further experiment, reframes what RL is doing\. RL spends its compute on search over semantic strategies, not on the gradient updates, so once a strategy exists, distillation can skip the search\.

The third round starts from a different application\. Mid\-training Qwen3\-8B on internal company documents raises internal QA but lowers instruction following, no mixing weight of chat data preserves the original IF\-eval score, and LoRA learns less and still forgets\. The idea update is to use the earlier post\-trained checkpoint, Qwen3\-8B, as the teacher and run on\-policy distillation on Tulu3 prompts after mid\-training, and the reason the post states is that the instruction\-following behavior was originally trained with RL, which is expensive and fragile\. IF\-eval recovers from 79% to 83% \(85% before mid\-training\), and internal QA rises from 36% to 41%\. On\-policy distillation is now a tool for recovering post\-trained behavior, which it was not in the initial idea\. In its discussion of continual learning the post reports a separate experiment, in which fine\-tuning Qwen3\-32B on its own samples still degrades it, although the KL between the two is zero in expectation, because every finite batch has a slightly different distribution\. The post presents this experiment after the recovery result and argues there that on\-policy distillation avoids the drift because the teacher stays fixed, but it does not give the experiment as the reason for the recovery method\. It is placed before the idea update because it explains why mixing cannot be the fix\. A last half\-round raises the data\-efficiency question that Case 5 later studies independently\. One training prompt, reused for 20 steps, approximately matches the teacher’s AIME’24 performance\.

The trajectory in Figure[7](https://arxiv.org/html/2610.27337#A1.F7)has three and a half rounds, seven experimental and two literature states of evidence, and three idea updates after the initial one\. Two caveats apply\. The source is a blog post, not a peer\-reviewed paper, and an authorial narrative is itself a retrospective reconstruction, which is the confabulation risk that any author\-annotated trajectory carries\([Nisbett and Wilson, 1977](https://arxiv.org/html/2610.27337#bib.bib16)\), discussed in §[8](https://arxiv.org/html/2610.27337#S8)\.

*What a whole\-idea regeneration loop would have lost\.*The third round is the clearest\. A review loop would receive “instruction following dropped” and regenerate a data\-mixing recipe; the authors did try that, and it is a result node\. The change that worked replaced one component of the idea, the training target, with a fixed on\-policy teacher\. The result that ruled mixing out is R3, in which no mixing weight preserved instruction following, and transition 8 adds the later experiment \(R4\) showing that even the model’s own samples degrade a model\. Without nodes for these results and transitions from them to the new idea, the final method reads as “distill from an old checkpoint” with no record of the experiments that ruled out the alternatives\.

#### Listings\.

idtypecontent \(short\)section of the postRQ0RQGive a small model the reasoning ability of a large one at low costDistillation for reasoningL0LQwen3 report: on\-policy distillation reaches 74\.4% AIME’24 in 1,800 GPU hours vs 17,920 for RLDistillation for reasoning: Reinforcement learningI0IOff\-policy distillation: sample from the teacher, SFT the studentDistillation for reasoning: Off\-policy distillationCE0CESFT on 400k teacher promptsOff\-policy distillationR0R60% AIME’24 at 400k; extrapolated ~2M prompts for 70%Off\-policy distillationL1LDAgger; GKD; MiniLLM; process reward modelsOn\-policy distillation — best of both worldsIN0INOff\-policy: student learns in the teacher’s contexts, errors compound\. RL: on\-policy but sparse, a fixed number of bits per episodeIntroduction \(untitled\)I1IStudent samples, teacher grades every token, per\-token reverse KL; a one\-line change on an RL loopImplementation: Loss function; PseudocodeCE1CEOn\-policy distillation into the 400k SFT checkpoint of Qwen3\-8B\-Base \(teacher Qwen3\-32B in the text, Qwen3\-8B per footnote\)Distillation for reasoning: On\-policy distillationR1R70% AIME’24 in ~150 steps; 9x \(SFT data given\) to 30x cost reductionOn\-policy distillationC1CWhy is dense supervision so much cheaper?Discussion: Dense supervision greatly improves compute efficiencyCE2CERL\-train Qwen3\-8B\-Base on DeepMath; distill the RL policy back into the baseDense supervisionR2RTeacher level reached 7 to 10x faster than RL; 50 to 100x less computeDense supervisionIN1INRL spends its compute on search over semantic strategies; distillation skips the searchDiscussion: RL searches in the space of semantic strategiesRQ1RQTrain an internal assistant that knows company documents and keeps instruction followingDistillation for personalization: Training an internal assistantI2IMid\-train \(SFT\) on internal documents mixed with chat dataTraining an internal assistantR3RInternal QA up, IF\-eval down; no mixing weight preserves IF\-eval; LoRA learns lessTraining on new knowledge degrades learned behaviorR4RSFT of Qwen3\-32B on its own samples \(temperature 1\.0, Tulu3 prompts\) still degrades it; reported after R5Discussion: On\-policy learning as a tool for continual learningIN2INKL is 0 in expectation but every finite batch differs; only a fixed teacher keeps the update on\-policyContinual learningC2CHow to add knowledge without leaving the post\-trained behaviorTraining on new knowledge degrades learned behaviorI3IUse the earlier post\-trained checkpoint as teacher; on\-policy distillation on instruction prompts after mid\-trainingOn\-policy distillation recovers post\-training behaviorR5RIF\-eval 79% to 83% \(85% before mid\-training\); internal QA 36% to 41%On\-policy distillation recovers post\-training behaviorCE3CEOne prompt reused for 20 stepsDiscussion: Distillation can effectively reuse training data for data efficiencyR6ROne prompt reused for 20 steps approximately matches the teacher’s AIME’24 performanceData efficiencyIN3INReverse KL learns the teacher’s distribution, not a memorized answerData efficiencyC3CHow far does data efficiency go, and why \(studied later, independently, in Case 5\)Data efficiency\#edgetransition1RQ0→\\rightarrowI0→\\rightarrowCE0→\\rightarrowR0Run the default and extrapolate its cost2R0, L0, L1→\\rightarrowIN0Diagnose both baselines; off\-policy compounds error, RL is sparse \(the post states the diagnosis in its introduction\)3IN0→\\rightarrowI1Combine on\-policy sampling with dense teacher grading4I1→\\rightarrowCE1→\\rightarrowR1Run from the same checkpoint; measure cost5R1→\\rightarrowC1→\\rightarrowCE2→\\rightarrowR2Isolate dense supervision by distilling an RL policy back into its base6R2→\\rightarrowIN1Reframe RL as search7RQ1→\\rightarrowI2→\\rightarrowR3Try mixing; observe that nothing preserves IF\-eval8R3→\\rightarrowR4→\\rightarrowIN2→\\rightarrowC2Test SFT on own samples; attribute degradation to finite\-batch drift; the post reports R4 later, in its discussion, on Qwen3\-32B9IN2, I1→\\rightarrowI3→\\rightarrowR5Reuse the OPD objective with the old checkpoint as teacher; the stated reason is that the behavior came from expensive, fragile RL10I1→\\rightarrowCE3→\\rightarrowR6→\\rightarrowIN3→\\rightarrowC3Probe data reuse; open the data\-efficiency question

### A\.7Case 7: from judging research ideas to executing them

Research Question: can LLMs generate ideas as good as experts’?Literature \#0: ideation systems; small studies or LLM judgesIdea Update \#0: large blind human study; agent: RAG, 4,000 seeds, rankerCode & Exp\. Plan \#0: 100\+ NLP researchers write and review blindExp\. Results \#0: AI ideas more novel, 5\.64 vs 4\.84; feasibility slightly lower \(n\.s\.\)Exp\. Results \#1: 200 unique of 4,000; ranker 53\.3%, reviewers 56\.1% agreementInsight \#0: judging ideas without execution is subjective, even for expertsChallenge \#1: do ideas judged better produce better research when executed?Idea Update \#1: execution study; 43 researchers each implement one 2024 ideaCode & Exp\. Plan \#1: 24 AI, 19 human ideas; 93\.7 / 112\.6 h; 58 blind reviewersExp\. Results \#2: overall drop−\-1\.98 vs−\-0\.63; effectiveness lead reversed \(n\.s\.\)Exp\. Results \#3: executors changed only details; eval substitution explains littleInsight \#1: execution evaluation uses empirical outcomes; ideation evaluation speculatesChallenge \#2: proxy reward models; closed loops from executed outcomes to ideasInitial RPG \(2024 study\)123456Reasoning on transition 3: the study’s own numbers \(reviewer–reviewer agreement 56\.1%, ranker–human agreement 53\.3%, 200 unique ideas of 4,000\) are read as a limit of judging ideas as text, not as noise, so the next step changes the judge, from idea reviews to executed projects\.Transition 1 has two sources\. One round; Challenge \#2 leaves the next round open\. The idea update changes the judge, not the idea text, and Insight \#1 is again about the judge\. Same three authors across both papers\. One reasoning record is shown\.Figure 8:Case 7, from the ideation study to the ideation–execution gap, as an RPG\. One round and an open challenge, five states of evidence\. Listing below\.*Sources:[Si et al\. \(2025b\)](https://arxiv.org/html/2610.27337#bib.bib18)and[Si et al\. \(2025a\)](https://arxiv.org/html/2610.27337#bib.bib19), by the same three authors\. The second paper restates the first paper’s finding and open question in its introduction and executes the same ideas\. Every edge cites its section\.*

#### Full account\.

The research question is whether language models can generate research ideas as good as experts’\. The literature the first paper starts from is the ideation systems of 2023 and 2024 and their evaluations, which were small or done by LLM judges \(§1\)\. The initial idea is to settle the question with a study large enough to be believed\. More than a hundred NLP researchers write and blindly review ideas, against an LLM agent built from retrieval over Semantic Scholar, over\-generation of 4,000 seed ideas per topic, deduplication, and a Swiss\-tournament pairwise ranker, which is a prompted Claude\-3\.5\-Sonnet model validated on ICLR 2024 submissions \(§3, §3\.3\)\. The headline result is that AI ideas are judged more novel \(5\.64 vs 4\.84, §5\.1\), with feasibility slightly lower but not significantly so \(abstract, §5\)\. The paper’s own analyses are the second injection\. Only about 200 of the 4,000 generated ideas are unique \(§7\.1\), the ranker agrees with humans 53\.3% of the time \(§7\.2\), and the human reviewers agree with each other 56\.1% of the time \(§6\.3\)\. The insight the authors draw from their own numbers is about the judge, not the generator\. Judging an idea without executing it is subjective even for experts, and novelty judged at ideation may not hold once the idea is executed\. They propose that evaluation should move to the execution phase \(§6\.3, §10\)\. The challenge is stated as a question, whether the ideas judged better produce better research when executed\.

The second paper is the answer, and its introduction records the reason for the step in the authors’ own words\. The first study “found that LLM ideas are judged as significantly more novel than human ideas”, but “evaluating research ideas is difficult even for experts, leaving open the question of whether these ideas would translate into better research outcomes”, because “during execution, every single step has to be grounded in realistic execution constraints, which impose higher feasibility standards than the ideation stage” \(§1\)\. The idea update is an execution study\. Forty\-three researchers each implement one idea, randomly assigned from the human or the AI pool of the first study \(24 AI, 19 human\), spending on average 93\.7 and 112\.6 hours, and the finished projects are reviewed blind by 58 reviewers, four to five per project, on novelty, excitement, effectiveness, soundness, and overall quality \(§2 to §3\)\. The result is the third injection\. On the four metrics rated at both stages, AI ideas lose far more than human ideas from ideation to execution, novelty−\-1\.05 vs−\-0\.01, excitement−\-1\.76 vs \+0\.08, effectiveness−\-1\.88 vs−\-0\.05, overall−\-1\.98 vs−\-0\.63 \(Table 5, §4\.2; soundness was not rated at ideation\)\. On effectiveness the AI ideas lead at ideation \(6\.00 vs 4\.83\) and trail after execution \(4\.13 vs 4\.78\), a difference that is not significant when each idea is one data point \(Table 4, §4\.2\)\. Two further analyses attribute the gap\. Executors changed only experiment details, about three per project, “rather than altering any algorithms proposed in the original ideas” \(§5\.1, Table 6\), so the gap belongs to the ideas\. Six AI ideas had a proposed human evaluation replaced by an automatic one, and the authors conclude that this and the other changes “only explain a small fraction” of the gap \(§5\.1\)\. The insight is a statement about evaluation itself, “execution evaluation always considers empirical experiment outcomes, while the ideation evaluation often speculates” \(§5\.2\)\. The challenge the authors leave is the one this paper takes up, proxy reward models that predict execution outcomes, and “closed feedback loops where the outcomes of executed experiments inform iterative idea improvement” \(§7\.3\)\.

The trajectory in Figure[8](https://arxiv.org/html/2610.27337#A1.F8)has one round, which ends at the execution study \(I1\), and a second round left open \(C2\)\. It has five states of evidence, one from the literature and four results, two of them from studies with people\. The idea whose path this is was never a system\. It was a claim about evaluation, its one update changed the judge, and the insight that followed \(IN1\) is again about the judge\.

*What a whole\-idea regeneration loop would have lost\.*A loop that receives “5\.64 vs 4\.84” regenerates a more novel generator\. The update that mattered was a change to the judge, caused by two agreement numbers and the diversity result \(R1\) read as a limit of judging ideas as text \(IN0\), and the insight after it was again about the judge \(IN1\)\. Neither change touches the idea text, and both are recordable only as states with edges back to the results that caused them\. The case also shows what the two papers could not hold between them\. About nine months passed between the question \(C1\) and its answer, and the reason for the answer had to be restated in the second introduction because there was nowhere else to keep it\.

#### Listings\.

idtypecontent \(short\)sourceRQ0RQCan LLMs generate research ideas as good as experts’?Si et al\. 2025 §1L0LIdeation systems and their evaluations: small studies or LLM judgesSi et al\. 2025 §1, §2I0IA large blind human study; LLM agent = RAG \+ 4,000 seeds per topic \+ dedup \+ Swiss\-tournament rankerSi et al\. 2025 §3CE0CE100\+ NLP researchers write and review; 79 reviewers; blindSi et al\. 2025 §4R0RAI ideas more novel \(5\.64 vs 4\.84\); feasibility slightly lower, not significantSi et al\. 2025 abstract, §5\.1R1R \(self\-analysis\)about 200 unique of 4,000 ideas; ranker 53\.3% agreement with humans; reviewer–reviewer agreement 56\.1%Si et al\. 2025 §7\.1, §7\.2, §6\.3IN0INIdea\-level judgment without execution is subjective even for experts; judged novelty may not survive executionSi et al\. 2025 §6\.3, §10C1CDo ideas judged better produce better research when executed?Si et al\. 2025 §10; Si et al\. 2026 §1 \(quoted\)I1IExecution study: 43 researchers each implement one randomly assigned idea from the 2024 pools; blind review of the projectsSi et al\. 2026 §2, §3CE1CE24 AI and 19 human ideas; 93\.7 / 112\.6 hours per idea; 58 reviewers, 181 reviews, 4–5 per projectSi et al\. 2026 §2, §3R2RIdeation→\\rightarrowexecution drop on the four metrics rated at both stages: novelty−\-1\.05 vs−\-0\.01; excitement−\-1\.76 vs \+0\.08; effectiveness−\-1\.88 vs−\-0\.05; overall−\-1\.98 vs−\-0\.63; effectiveness AI ahead at ideation \(6\.00 vs 4\.83\), behind after execution \(4\.13 vs 4\.78, not significant per idea\)Si et al\. 2026 Tables 4, 5R3R \(attribution\)Executors changed only experiment details \(2\.9–3\.1 per project\), no algorithms; removing human evaluation \(6 AI ideas\) explains only a small fractionSi et al\. 2026 Table 6, §5\.1IN1IN“Execution evaluation always considers empirical experiment outcomes, while the ideation evaluation often speculates”; execution raises the standard of rigorSi et al\. 2026 §5\.2 \(quoted\)C2C \(open\)Proxy reward models for execution outcomes; “closed feedback loops where the outcomes of executed experiments inform iterative idea improvement”Si et al\. 2026 §7\.3 \(quoted\)\#edgetransition1RQ0, L0→\\rightarrowI0→\\rightarrowCE0→\\rightarrowR0Replace small or automatic evaluation by a large blind human study; measure2R0→\\rightarrowR1Analyze the study’s own generator and judges3R1→\\rightarrowIN0→\\rightarrowC1Read the agreement and diversity numbers as a limit of idea\-level judgment; ask the execution question4C1→\\rightarrowI1→\\rightarrowCE1→\\rightarrowR2Execute a subset of the same ideas; review blind; measure the drop5R2→\\rightarrowR3→\\rightarrowIN1Rule out executor changes and evaluation substitution; locate the gap in the ideas and the judge6IN1→\\rightarrowC2Leave the loop open: outcomes should feed back into idea improvement

### A\.8Case 8: from GRPO to DeepSeek\-R1, with recorded failed explorations

Research Question: LLM reasoning with RL at scaleLiterature \#0: DeepSeekMath GRPO after SFT on mathInsight \#0: prior RL relied on supervised data; try reasoning from pure RLChallenge \#1: can reasoning emerge from RL alone? How to supervise it?Idea Update \#1: R1\-Zero: GRPO on V3\-Base, rule\-based rewardsExp\. Results \#1: AIME’24 pass@1 15\.6%→\\to71\.0%; “aha moment”Exp\. Results \#2: poor readability; language mixingAbandoned Idea Update X1: process reward model, reward hackingAbandoned Idea Update X2: MCTS over tokens; value model hard to trainInsight \#1: cold start with human priors for readability and better performanceChallenge \#2: does cold\-start data help? A readable, general model?Idea Update \#2: R1: cold start→\\toRL→\\toSFT→\\toRL, all scenariosExp\. Results \#3: more readable, slight cost; comparable to o1\-1217Challenge \#3: small models: distill from R1, or RL directly?Idea Update \#3a: distill 800k R1 samples into Qwen/LlamaIdea Update \#3b: large\-scale RL on Qwen\-32B\-BaseExp\. Results \#4: distilled 32B beats RL\-trained 32BChallenge \#4 \(open\): generality, language mixingInitial RPG1122a2b33456Reasoning on transition 3, as the report states it: R1\-Zero’s results are promising, but its output is often not suitable for reading and mixes languages\. Cold\-start data designed with human priors should make the output readable and, the authors observe, give better performance than R1\-Zero\. The reinforcement learning is kept, and the cold\-start data is added before it\.Dashed transitions 2a and 2b are the two attempts the report says were abandoned in the early stages of the project, kept with the reasons they were dropped\. The report does not say when they were made, and attaching them to Challenge \#1 is our placement\. Transitions 1 and 3 have two sources; transition 5 is a fork resolved by experiment \(Exp\. Results \#4\)\. Three rounds; one reasoning record is shown\.Figure 9:Case 8, DeepSeek\-R1, as an RPG\. Abandoned updates are drawn as dashed transitions\. Three rounds, two recorded abandoned branches, one fork, five states of evidence\. The report places the abandoned attempts only in the early stages of the project\. Listing below\.*Sources:[Shao et al\. \(2024\)](https://arxiv.org/html/2610.27337#bib.bib53), which introduces GRPO, and the first arXiv version of[DeepSeek\-AI \(2025\)](https://arxiv.org/html/2610.27337#bib.bib54)\(January 2025\), from which all quotations, numbers, and section numbers below are taken\. The report states the reason for each stage and records two attempts that failed “in the early stages of developing DeepSeek\-R1”, with the reasons for abandoning them\. It does not say when they were made relative to R1\-Zero\.*

#### Full account\.

The research question is how to improve reasoning in language models with reinforcement learning at scale\. The literature the idea starts from is the group’s own\. DeepSeekMath’s GRPO drops the critic and uses a group\-relative baseline, and it was applied after supervised fine\-tuning on mathematics \(DeepSeekMath §4; R1 §2\.2\.1\)\. The first insight is stated at the start of the R1\-Zero section\. The group’s earlier reinforcement learning for reasoning “heavily depended on supervised data, which are time\-intensive to gather”, so the report explores “the potential of LLMs to develop reasoning capabilities without any supervised data, focusing on their self\-evolution through a pure reinforcement learning process” \(§2\.2\)\. The challenge that follows is whether reasoning can emerge from RL alone, with no supervised data at all\. The idea update is DeepSeek\-R1\-Zero, GRPO on the DeepSeek\-V3 base model with rule\-based rewards for accuracy and format and a think/answer template \(§2\.2\.1 to §2\.2\.3\)\. It uses no neural reward model, because the authors “find that the neural reward model may suffer from reward hacking in the large\-scale reinforcement learning process” \(§2\.2\.2\)\. The result has two halves\. AIME 2024 pass@1 rises from 15\.6% to 71\.0%, 86\.7% with majority voting, and the authors report the model lengthening its thinking, re\-checking its steps, and the “aha moment” of rethinking a problem, none of it explicitly programmed \(§2\.2\.4\)\. The other half is the drawback paragraph, in which R1\-Zero “struggles with challenges like poor readability, and language mixing” \(§2\.2\.4\), with several languages inside one response \(§2\.3\.1\)\.

The second round starts from both halves of that result\. The report says that the “promising results” of R1\-Zero raise two questions, whether a small amount of high\-quality cold\-start data can further improve reasoning or speed up convergence, and how to train a user\-friendly model with clear chains of thought and strong general capabilities \(§2\.3\)\. It gives two reasons for the cold\-start data \(§2\.3\.1\)\. The first is readability, since R1\-Zero’s “content is often not suitable for reading”\. The second is potential, since with cold\-start data designed with human priors the authors “observe better performance against DeepSeek\-R1\-Zero”\. The insight is therefore that a small amount of human\-designed data should fix the form of the output and also raise performance, while the reinforcement learning is kept\. The idea update is DeepSeek\-R1, a four\-stage pipeline \(§2\.3\.1 to §2\.3\.4\)\. It starts with cold start on thousands of long chains of thought, followed by reasoning\-oriented RL with a language\-consistency reward\. Rejection sampling and supervised fine\-tuning follow on about 600k reasoning and 200k non\-reasoning samples, and the non\-reasoning data is added “to enhance the model’s capabilities in writing, role\-playing” and other general tasks \(§2\.3\.3\)\. A final RL stage covers all scenarios \(§2\.3\.4\)\. The result records a trade\-off the authors chose to accept\. The language\-consistency reward causes “a slight degradation in the model’s performance” but “aligns with human preferences, making it more readable” \(§2\.3\.2\), and R1 performs comparably to OpenAI\-o1\-1217 on reasoning tasks \(abstract; for mathematics, §3\.1\)\.

The report also records two attempts that failed “in the early stages of developing DeepSeek\-R1” \(§4\.2\)\. Process reward models were dropped because it is hard to define a fine\-grained step in general reasoning and hard to judge whether an intermediate step is correct, and because a model\-based process reward “inevitably leads to reward hacking” and needs a retraining pipeline\. Monte Carlo tree search over tokens was dropped for two reasons\. The search space is exponentially larger than in chess, and the maximum extension limit per node that bounds it can leave the model stuck in local optima\. The fine\-grained value model that guides the search is hard to train, so the model cannot improve itself iteratively through search\. The report does not say when the two attempts were made, and we draw both as sibling edges out of the challenge of how to supervise reasoning \(C1\), each with the reason it was dropped\.

The third round is a fork on small models, resolved by experiment\. The two branches are to distill R1’s 800k samples into Qwen and Llama models from 1\.5B to 70B \(§2\.4\) or to run the same large\-scale RL on a 32B model directly\. The result is that the distilled 32B model beats the RL\-trained one, 72\.6 against 47\.0 pass@1 on AIME 2024 \(Table 6\), and the authors conclude that “distilling more powerful models into smaller ones yields excellent results, whereas smaller models relying on the large\-scale RL … may not even achieve the performance of distillation” \(§4\.1\)\. The conclusion lists the open challenges, which are general capabilities, language mixing beyond Chinese and English, prompt sensitivity, and RL for software engineering, where evaluation is too slow \(§5\)\.

The trajectory in Figure[9](https://arxiv.org/html/2610.27337#A1.F9)has three rounds, two recorded dead ends, and one fork, with five states of evidence, four of them experimental\. This is the only case whose source reports abandoned approaches as separate branches, each with the reason it was dropped, and a representation that holds only the current idea cannot hold them at all\.

*What a whole\-idea regeneration loop would have lost\.*A results loop receiving “71% on AIME” keeps training\. The update that produced DeepSeek\-R1 came from the drawback paragraph \(R2\) read together with the promising results \(R1\), and the report states the reason on that edge \(IN1\)\. Cold\-start data designed with human priors should make the output readable and also raise performance, so the authors added a small amount of it and kept the RL\. The two abandoned branches are the sharper point\. A loop that regenerates the whole idea has no node for “we tried a process reward model and here is why it failed”, and it would try it again\. The RPG keeps X1 and X2 as sibling edges with their reasons, which is the record a later group, or the same group a year later, needs\.

#### Listings\.

idtypecontent \(short\)sourceRQ0RQHow to improve LLM reasoning with reinforcement learning at scaleR1 §1L0LDeepSeekMath: GRPO \(no critic, group\-relative baseline\) after SFT on mathematics; the group’s own prior workDeepSeekMath §4; R1 §2\.2\.1IN0INEarlier RL for reasoning “heavily depended on supervised data”; explore reasoning “without any supervised data”, through pure RLR1 §2\.2 \(quoted\)C1CCan reasoning emerge from RL on the base model with no supervised data? How to supervise it?R1 §2\.2, §2\.2\.2I1IDeepSeek\-R1\-Zero: GRPO on DeepSeek\-V3\-Base; rule\-based accuracy \+ format rewards; think/answer template; no neural reward model \(reward hacking\)R1 §2\.2\.1–2\.2\.3R1RAIME 2024 pass@1 15\.6%→\\rightarrow71\.0%, 86\.7% majority voting; self\-evolution: longer thinking, reflection, “aha moment”R1 §2\.2\.4R2R \(drawbacks\)Poor readability; language mixing within one chain of thoughtR1 §2\.2\.4 \(quoted\)X1I \(abandoned\)Process reward model: hard to define steps, hard to judge intermediate correctness, reward hacking and retraining costR1 §4\.2X2I \(abandoned\)MCTS over tokens: exponential search space, node extension limit→\\rightarrowlocal optima; value model hard to train, no iterative self\-improvementR1 §4\.2IN1INCold\-start data designed with human priors should make the output readable and give better performance than R1\-Zero; keep the RLR1 §2\.3, §2\.3\.1 \(Readability, Potential\)C2CCan a little high\-quality cold\-start data improve reasoning or speed convergence? How to train a user\-friendly model with readable CoT and general capability?R1 §2\.3I2IDeepSeek\-R1: cold start \(thousands of long CoT\)→\\rightarrowreasoning RL with language\-consistency reward→\\rightarrowrejection\-sampling SFT \(600k \+ 200k\)→\\rightarrowRL over all scenariosR1 §2\.3\.1–2\.3\.4R3RLanguage\-consistency reward: slight degradation, more readable \(chosen\); R1 comparable to o1\-1217 on reasoning tasksR1 §2\.3\.2, abstract, §3\.1C3CSmall models: distill from R1, or run large\-scale RL directly?R1 §2\.4, §4\.1I3aIDistill 800k R1 samples into Qwen/Llama 1\.5B–70BR1 §2\.4I3bILarge\-scale RL on Qwen\-32B\-Base directlyR1 §4\.1R4RDistilled 32B beats RL\-trained 32B \(AIME 2024 72\.6 vs 47\.0\); “smaller models relying on the large\-scale RL … may not even achieve the performance of distillation”R1 §4\.1, Table 6 \(quoted\)C4C \(open\)General capability; language mixing beyond Chinese/English; prompt sensitivity; RL for software engineering \(slow evaluation\)R1 §5\#edgetransition1RQ0, L0→\\rightarrowIN0→\\rightarrowC1State the aim of developing reasoning without supervised data; ask whether RL alone suffices and how to supervise it2C1→\\rightarrowI1→\\rightarrowR1, R2Run pure RL with rule\-based rewards; observe the capability and the drawbacks2aC1⇢\\dashrightarrowX1Try a process reward model; abandon, reasons recorded2bC1⇢\\dashrightarrowX2Try MCTS; abandon, reasons recorded3R1, R2→\\rightarrowIN1→\\rightarrowC2Read the promising results and the drawbacks together; expect cold\-start data to fix readability and raise performance4C2→\\rightarrowI2→\\rightarrowR3Add cold start and three more stages; pay a small accuracy cost for readability5I2→\\rightarrowC3→\\rightarrowI3a, I3b→\\rightarrowR4Fork on small models; resolve by experiment in favor of distillation6R4→\\rightarrowC4Record what remains open

### A\.9Cross\-case summary

The first table gives the type, source, and size of each trajectory, and the second gives what changed in it and what a loop that regenerates the whole idea would keep\. Cases 1 to 5 are presented in §[4](https://arxiv.org/html/2610.27337#S4)and Cases 6 to 8 in this appendix\.

CaseTypeWho states the reasonsRoundsEvidence from experiments or studiesEvidence from literature1methodthe authors \(annotated, or traced and confirmed\)1222applicationthe authors \(annotated, or traced and confirmed\)43 \(formative interviews, user study, benchmark\)3 \(one is a data source\)3method \(ideation system\)the paper and the authors \(annotated, or traced and confirmed\), in the paper’s order of presentation3434system \(research agent\)the paper, two versions, and the authors \(annotated, or traced and confirmed\)3 \+ an open challenge4 \(one an observation of the agent drifting\)5 \(one an external resource, one the group’s own later paper\)5empirical insightthe papers \(shared title, eight shared authors\)3316methodthe authors \(blog\)3\.5727evaluation of ideationthe same three authors \(second intro\)1 \+ an open second round4 \(two with people\)18methodthe organization \(report, incl\. dead ends\)3 \+ 2 dead ends \+ 1 fork41 \(own prior work\)CaseDecisive cognitive patternNode changed / evidence / recorded reasonWhat a loop that regenerates the whole idea from a score or summary keeps1abstract structure, substitute one slotLM slot of the switch / R0 \+ re\-read L0 / pointer\-generator is a switch between LM and copythe final model, not the attribution2transfer a parameterized cognitive model; build the missing data; move from patient to therapistconditioning input, styles, then the object \(patient→\\rightarrowtherapist\) / R0 \+ L1, R0, R1 \+ IN2 / structured cognition; expert\-named styles; the cognitive model is central on both sidesa better prompt, not the conclusion about the judge or the move to the therapist3take a trade\-off from prior work as a property of the task; give each dimension its own handle; localizethe generator \(per\-dimension rewards→\\rightarrowper\-dimension controllers\), then where emphasis applies / L1 \(trade\-off in prior work\), for both / the dimensions are interdependent; parts of an idea need different emphasisa better reward model, not the prior\-work trade\-off that motivated the controllers and the decoder4take components from other papers one at a time; add a person where the agent drifts; adopt a better method once it existsthe experiment agent, then its resources and a human channel, then the idea agent \(prompted→\\rightarrowtrained\) / L1 \+ L2, then L3 \+ R0, then L4 / ReAct from MLAgentBench and feedback from ResearchAgent; retrieval from the Hub matches models better, and a person corrects the drift; the better ideation method now existsa better system, not which component came from which paper or why each was added5sharpen, mechanize, ablate what the mechanism predictsclaim scope, twice / R1, R2 / IN0 predicts content\-independence; residual slow absorptiona good score, not the two ablations the mechanism predicted6diagnose both baselines, combine; then rule out alternatives by experimentsampling source and grading, then the training target / R0 \+ L0 \+ L1, then R3 \+ R4 / compounding error and sparse reward; finite\-batch drifta data\-mixing recipe, not the experiment that ruled mixing out7change the judgethe judge \(idea reviews→\\rightarrowexecuted projects\), then an insight about the judge / R1 agreement and diversity numbers, then R2 \+ R3 / idea\-level judgment is subjective; execution evaluation does not speculatea more novel generator, not the moved judge8act on promising results and stated drawbacks together; record what failedthe data regime \(none→\\rightarrowcold start\), plus two abandoned branches / R1 \+ R2, §4\.2 / readability and expected performance; step definition and reward hacking; search space and value modela stronger model, not the two branches it must not retry

## Appendix BA Proposed Protocol for Comparing Improvement Processes

This appendix records the design of the controlled comparison referred to in §[6\.3](https://arxiv.org/html/2610.27337#S6.SS3), which this paper proposes and does not run, together with the details of the pilot from which the judge finding of §[6\.4](https://arxiv.org/html/2610.27337#S6.SS4)is taken\. Nothing in the paper depends on the comparison’s outcome\. The pipeline, seeds, prompts and judge code exist and were pilot\-tested, and are released with the paper so that the comparison can be run as specified\.

Question and what is held fixed\.The protocol isolates one variable\. It asks whether an idea is improved by localized, reason\-recorded edits on a typed graph or by whole\-idea regeneration, under the same generator, the same output\-token budget and the same retrieval tool\. It does not score ideas absolutely, and it does not compare against any system’s released output on equal footing\.

Models\.All five conditions run on the same generator, claude\-sonnet\-4\-5, chosen for its early training cutoff \(July 2025, per the vendor’s model overview\) so that the seed window can be wide; older models of the same family were not reachable, and a smaller model of the same generation failed a protocol test by writing one long draft instead of localized edits\. The judge is a newer model with retrieval, claude\-opus\-5\-5\. The two cutoffs may differ, because the judge sees only the final idea text and the evidence it retrieves and never reconstructs a target\.

Seeds\.The seeds are twenty\-six usable ideas of type method \(15\), empirical insight \(7\), and application \(4\)\. Every seed is paired with a hidden target that is the same authors’ own next version of the idea, so that seed and target form a within\-author update path in the scoping of §[4](https://arxiv.org/html/2610.27337#S4), and every target is an arXiv v1 submitted after 2025\-08\-01, the month after the generator’s cutoff, so that no condition can recover it from memory\. Twenty\-one are retro seeds\. Following the RPG\-retro procedure, a model is given the title and abstract of the target and asked to reconstruct an early version of the idea, keeping the question and general direction and removing the key contribution, which is stored as the hidden target and never shown to any improvement condition\. The starting point is reconstructed, but the step from it to the target is the authors’ own\. Five are same\-author pairs in which both papers exist\. The earlier paper, mapped to the schema without weakening, is the seed, and the later paper is the target\. One pair is the Part I to Part II step of Case 5, with eight shared authors, and the other four share three to six authors with their targets\. Four retro seeds whose targets coincided with those of the pairs were replaced by the pairs\. Three cross\-group pairs used in an earlier version of the protocol are archived, and their follow\-up papers became retro seeds\. Four screens apply\. \(i\) Date: the target’s first version postdates the cutoff\. \(ii\) Contamination: the generator is given the title alone and asked to write the abstract, and the judge model scores its coverage of the actual abstract from 0 to 4; a score of 3 or more excludes the seed, which removed three of 29 candidates whose titles state the mechanism\. \(iii\) Manual review that the seed is genuinely early and does not leak the target\. \(iv\) Retrieval leakage: because web retrieval can return the target paper itself, which happened for two of the three pilot seeds, the retrieval layer excludes the target by title and arXiv identifier and every run is scanned afterwards for the target among its literature nodes\. Applied to the eight update paths of §[4](https://arxiv.org/html/2610.27337#S4), the same screens leave one usable path\. Case 1 ends in a 2020 paper, Cases 2, 3, 4, 7, and 8 end in papers whose first versions are inside the generator’s training data, and the on\-policy distillation post of Case 6, although it postdates the cutoff, is reconstructed from its title alone with coverage 3\. Case 5 scores 1\. The case paths are therefore evidence for the framework and not data for the protocol\.

Conditions\.Each of the five conditions \(Table[11](https://arxiv.org/html/2610.27337#A2.T11)\) has the same budget of T = 12,000 output tokens, spent over as many calls as fit \(at most 24\), and each tool condition may run at most Q = 3 searches returning five items each\. The pilot of §[6\.3](https://arxiv.org/html/2610.27337#S6.SS3)used an earlier budget unit, K = 6 calls with a cap of 2,000 output tokens per call, and is the reason for the change\. The cap was stated in the prompt but not enforced by the interface, and single calls reached 3,700 output tokens, so the protocol counts the tokens actually generated\. All conditions produce a final idea in one schema \(title, problem, key insight, method, experiment plan, expected results, assumptions and risks\) and the judge sees only that schema, never a trajectory, a graph or an edit log\.

Table 11:The five equal\-budget conditions\.A result loop \(Dolphin\-style, real experimental results fed back\) is a sixth condition that this version does not run, because the seeds have no experiments; simulating results would be circular\. Its absence is a limitation and an agenda item\.

External one\-shot baselines\.Published outputs of one\-shot systems, e\.g\., ResearchTown\([Yu et al\., 2025a](https://arxiv.org/html/2610.27337#bib.bib24)\)and the Ramon Llull’s Thinking Machine\([Zhao et al\., 2025b](https://arxiv.org/html/2610.27337#bib.bib21), not to be confused with the Thinking Machines Lab blog post of Appendix[A\.6](https://arxiv.org/html/2610.27337#A1.SS6)\), enter the same pairwise judge in two roles\. Their outputs are the real\-system baseline of the one\-shot tier, and the same outputs serve as seeds on which the five conditions add a fixed budget\. The comparison of an external output with its improved versions is not equal\-budget and is reported as “gain after a fixed added budget”, never as “RPG beats system X”\.

Judging: three signals, reported side by side and never combined\.The first is blind pairwise preference between two final ideas from the same seed, on novelty, excitement, feasibility, effectiveness, and overall quality, the five criteria of[Si et al\. \(2025b\)](https://arxiv.org/html/2610.27337#bib.bib18)\. Each pair is judged twice with positions swapped and counts only if both orders agree; win rates carry 95% Wilson intervals\. The judge model differs from, and is newer than, the generation model\. Every judgment carries a written rationale that is stored for audit\. Node identifiers and version markers are stripped from all outputs before judging so that the judge cannot infer the condition\. The second is verified novelty\. Two or three novelty claims are extracted from each version, literature is retrieved for each with the same tool the conditions used, with the target excluded, and the pairwise novelty judgment is made with that evidence visible, a step that the pilot below motivated\. A claim that retrieval cannot match is recorded as unsupported rather than as unrefuted, because in the judge study of §[6\.4](https://arxiv.org/html/2610.27337#S6.SS4)such claims passed the check\. The third, reported in its own table, is alignment with the authors’ real next step\. Given the seed, the hidden target and two final ideas, the judge says which is closer to the step the authors actually took and scores each 0 to 4 for coverage of the target’s key idea, mechanism and expected result\. For the same\-author pairs the judge first writes down the axis along which the authors changed the idea and then says whether each condition’s main change lies on that axis, a check less sensitive to wording than coverage\. Disagreement between the first and third signals is itself a result, since a process preferred on alignment and dispreferred by the text judge is the judge problem of §[6\.4](https://arxiv.org/html/2610.27337#S6.SS4)\. Alignment is not quality, since an idea better than the target can score low, and outputs of the external one\-shot systems, which have no target, do not enter this table\. A human subset of 40 pairs, stratified by condition pair with hidden sources, is judged by PhD students, and its agreement with the model judge is reported; Case 2 is the reason this step is not optional\.

Hypotheses\.The protocol tests four hypotheses\. The first is that the RPG loop is preferred over the one\-shot rewrite and the review loop; the second, that it is preferred over the literature loop, which has the same tool, since parity would mean that retrieval and not the graph drives the improvement; the third, that removing outside information removes most of the advantage; and the fourth, that the RPG loop’s output lies closer to the authors’ actual next step than the output of whole\-idea regeneration\. These are the predictions the framework makes, stated before any run; §[8](https://arxiv.org/html/2610.27337#S8)says what their failure would mean\.

Two smaller studies that reuse the same components\.The first is a next\-state prediction on annotated update paths\. Given the prefix of a path up to a transition, an agent predicts the type and content of the next state in one call, with type accuracy needing no judge and content scored 0 to 4 against the authors’ real next node, and the recorded abandoned branches of Case 8 give negative next steps that a system should not re\-propose once their reasons are on the graph\. On the present case set only Case 5 passes the screens, so the study was run once, on that path, as a demonstration\. At nine prediction points the agent given the RPG prefix predicted the type of the next state in seven and a plain\-text baseline in three, with mean content coverage of 2\.00 and 1\.06\. One path is not evidence, and the study is the minimal form of learning from update paths \(§[7](https://arxiv.org/html/2610.27337#S7.SS0.SSS0.Px3)\)\. The second, a judge study on the seeds, was run and is reported in §[6\.4](https://arxiv.org/html/2610.27337#S6.SS4)\. For each of the 26 seeds, the authors’ real next version, written in the schema from the target abstract, is paired with a tool\-free one\-shot rewrite of the seed, and the text\-level and verified\-novelty judges each decide blind and position\-swapped, with retrieval for the verified judge restricted to work posted before the target\. It gives a second instance of the judge problem beside Case 2 and says nothing about the RPG against any system\.

What the design guards against\.The judge is independent of the RPG \(different and newer model, sees no graph, positions swapped, rationales auditable, human calibration\)\. Budgets are equal and actual spend is reported in both call and token units\. Seeds and targets are post\-cutoff and contamination\-screened, and the target is always the same authors’ next version\. Length and structure bias are limited by the shared schema and an explicit instruction\. Retrieval leakage is filtered and scanned\. Three weaknesses remain\. The retro seeds are model\-reconstructed starting points, the loop has no result nodes, and the graph conditions consume more input tokens, since their input is the linearized graph, a difference that is reported and not corrected for\.

Table 12:Pilot \(n = 3 seeds\): mean spend per run and overall wins against the RPG loop\.Pilot, used as a judge finding\.Three seeds were run through all five conditions and all thirty pairs judged \(generator claude\-sonnet\-5\-5, judge claude\-opus\-5\-5; direction only\); Table[12](https://arxiv.org/html/2610.27337#A2.T12)gives spend and outcomes\. The pilot used a call\-count budget of six calls and a newer generator than the design above, and both the budget unit and the novelty judgment were changed after it and before any run under the protocol, so its numbers are not comparable with such a run and are reported only for the judge finding\. The first flaw is the budget unit\. At equal*call*count, the review and literature loops spent about 13,000 to 14,000 output tokens per run, the RPG loop about 5,000, because four of its six transitions were insights, challenges or searches that produce no idea text, and its idea updates changed only a few fields\. The RPG condition therefore ran on about a third of the generation compute\. The remedy, recorded in the protocol’s change log, is an equal*output\-token*budget of 12,000 generated tokens per condition; the RPG loop then takes as many transitions as fit, roughly twelve to sixteen\.

The second flaw is the judge’s notion of novelty\. In the rationales, the recurring point is that the RPG version’s own citations count against it, for example “B’s main hypothesis, the passive\-adaptive gap, is already reported by The Distillation Game \(2026\), which B cites itself”, so novelty goes to the other side; the review loop, which has no tools, makes bolder claims that the judge cannot check and does not penalize\. The judge therefore rewards claims that look new without checking whether they are\. The remedy is the verified\-novelty judgment described under Judging above, and the text\-only and verified judgments are both reported\.