When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation

Hugging Face Daily Papers Papers
ai-peer-review scientific-judgment-collapse trust-reviewer llm-fine-tuning mitigation feedback-loop academic-evaluation

Summary

This paper studies the feedback loop where AI-generated reviews influence future training of AI reviewers, leading to reduced judgment diversity called 'scientific-judgment collapse,' and introduces TrustReviewer, an open-source system to mitigate this through curated training and activation steering.

Large language models (LLMs) increasingly participate in scientific evaluation, both as automated reviewers and as assistants to human reviewers. As model-generated reviews enter public data and future training corpora, AI peer review can become recursive: later reviewers learn from judgments produced by earlier models. We study one step of this feedback loop in a controlled setting. Starting from Llama 3.1 8B, we first fine-tune a reviewer on official ICLR reviews from 2018--2023 and then train four successor models on ICLR 2024 data with systematically varied mixtures of official and model-generated reviews. Our study shows that introducing synthetic reviews compresses rating distributions and reduces both same-paper and corpus-level semantic diversity. We call this pattern scientific-judgment collapse. To mitigate this failure mode, we introduce TrustReviewer, an open-source LLM-based system for generating peer reviews of AI and machine learning papers. TrustReviewer intervenes at two complementary stages. For training-time prevention, we train the core reviewer in a single stage on a curated corpus designed to reduce low-quality and semantically degenerate supervision. For test-time correction, paired activation steering aims to further mitigate residual tendencies toward collapsed judgments without further training or additional expert annotation. Together, these results characterize a concrete risk of recursive reviewer training and provide practical interventions for preserving judgment diversity and improving recommendation alignment in AI-assisted scientific evaluation.
Original Article
View Cached Full Text

Cached at: 09/21/26, 03:20 AM

Paper page - When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation

Source: https://huggingface.co/papers/2609.20942

Abstract

Largelanguagemodels(LLMs)increasinglyparticipateinscientificevaluation,bothasautomatedreviewersandasassistantstohumanreviewers.Asmodel-generatedreviewsenterpublicdataandfuturetrainingcorpora,AIpeerreviewcanbecomerecursive:laterreviewerslearnfromjudgmentsproducedbyearliermodels.Westudyonestepofthisfeedbackloopinacontrolledsetting.StartingfromLlama3.18B,wefirstfine-tunearevieweronofficialICLRreviewsfrom2018--2023andthentrainfoursuccessormodelsonICLR2024datawithsystematicallyvariedmixturesofofficialandmodel-generatedreviews.Ourstudyshowsthatintroducingsyntheticreviewscompressesratingdistributionsandreducesbothsame-paperandcorpus-levelsemanticdiversity.Wecallthispatternscientific-judgmentcollapse.Tomitigatethisfailuremode,weintroduceTrustReviewer,anopen-sourceLLM-basedsystemforgeneratingpeerreviewsofAIandmachinelearningpapers.TrustReviewerintervenesattwocomplementarystages.Fortraining-timeprevention,wetrainthecorereviewerinasinglestageonacuratedcorpusdesignedtoreducelow-qualityandsemanticallydegeneratesupervision.Fortest-timecorrection,pairedactivationsteeringaimstofurthermitigateresidualtendenciestowardcollapsedjudgmentswithoutfurthertrainingoradditionalexpertannotation.Together,theseresultscharacterizeaconcreteriskofrecursivereviewertrainingandprovidepracticalinterventionsforpreservingjudgmentdiversityandimprovingrecommendationalignmentinAI-assistedscientificevaluation.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2609\.20942

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.20942 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.20942 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.20942 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

AI-written critiques help humans notice flaws

OpenAI Blog

OpenAI trained language models to write critiques of text summaries, helping human evaluators spot flaws more effectively — a step toward scalable oversight of AI systems on difficult tasks. The work explores how AI-assisted feedback can improve human evaluation quality as a proof of concept for alignment research.

NeurIPS 2026 AI-generated reviews [D]

Reddit r/MachineLearning

Discussion about the use of AI-generated reviews at NeurIPS 2026, including concerns over prompt injection and lack of consequences for reviewers using LLMs without proper oversight.