NeurIPS 2026 AI-generated reviews [D]

Reddit r/MachineLearning 07/28/26, 11:34 AM News

Summary

Discussion about the use of AI-generated reviews at NeurIPS 2026, including concerns over prompt injection and lack of consequences for reviewers using LLMs without proper oversight.

I'm really confused about what the point of the prompt injection was (speaking as an author). Is it just a study? I would really prefer that they took action against the AI-generated reviews. Obviously, we cannot assume that the reviewers were copy-pasting the output from the LLM without having given it any look at all, but in some cases that does look to be the case. In fact, in some cases the meta-reviewer seems to have also largely used LLMs. What exactly is the consequence here for using an LLM for reviewing?

Original Article

Similar Articles

NeurIPS 2026 Reviewer: AI-Generated Rebuttals (and Paper) [D]

Reddit r/MachineLearning

A NeurIPS reviewer reports encountering a paper and rebuttals that appear entirely LLM-generated, expressing frustration and seeking advice on how to evaluate such submissions.

Prompt Injection in NeurIPS 2026? [D]

Reddit r/MachineLearning

A user reports finding a prompt injection in their paper PDF downloaded from OpenReview for NeurIPS 2026, suggesting it may have been added by the conference and warns about potential LLM-generated reviewer comments containing specific phrases.

Review Arcade: On the Human Alignment and Gameability of LLM Reviews

Hugging Face Daily Papers

This paper investigates the alignment of LLM-generated reviews with human judgment using 1k real ACL 2025 submissions, finding limited agreement, instability across models/prompts, and a method to artificially inflate scores without meaningful changes. The authors advise against relying solely on LLM reviews and call for discussion on their use in handling increasing submission volumes.

Gaming AI-Assisted Peer Reviews Poses New Risks to the Scientific Community

arXiv cs.CL

A new study demonstrates that AI-assisted peer review is vulnerable to low-cost manipulation via superficial rephrasing of paper abstracts, significantly inflating AI-generated review scores and potentially biasing human editorial decisions, highlighting the need for safeguards.

Six months of using AI for code review taught me that "review this" is a QA problem disguised as a prompt problem

Reddit r/AI_Agents

A developer reflects on six months of using AI for code review, finding that vague prompts produce plausible but useless feedback. The fix is treating review as a gated pipeline with explicit context, scoped passes, validation checklists, and adversarial self-critique.