Tag
This paper from Yale and the University of Chicago finds that the main difference between LLM-generated and human research ideas is not quality but range: LLMs produce narrower ideas, with 47-64% of their ideas focusing on connecting separate work, compared to only 12.1% for humans.
SoundnessBench is a benchmark of 1,099 machine-learning research proposals that evaluates LLMs' ability to assess methodological validity, finding a pervasive optimism bias in current models.