Tag
DeepInstructor is an agentic framework that constructs an Experience Graph from peer reviews to evaluate research ideas, demonstrating superior alignment with human judgments over baseline methods.
This paper explores teaching language models to forecast the empirical success of research ideas by comparing pairs of ideas. Using a dataset of 11,488 idea pairs from PapersWithCode, the authors show that fine-tuning (SFT) boosts accuracy to 77.1%, outperforming GPT-5, and reinforcement learning with verifiable rewards achieves 71.35% with interpretable reasoning.