What do you think about fully open review systems? [D]

Reddit r/MachineLearning News
open-review peer-review ai-ethics academic-publishing transparency accountability

Summary

The article argues for transitioning to a fully open review system in academia to reduce bias, prevent false claims, and increase transparency, particularly to address challenges from AI-generated submissions and reviews.

In the age of AI, I believe the era in which individuals can dominate others solely on the basis of background knowledge, theoretical expertise, academic affiliation, or reputation is coming to an end. If we move from a double-blind review process to a fully open review system, it would become significantly more difficult to present false or misleading claims as established facts. Moreover, even under the current double-blind system, there is a tendency to favor papers from close-knit academic communities, as authorship can often be inferred by tracking preprints on platforms such as arXiv. I believe that a fully open review system could help mitigate this form of bias as well. Ultimately, such a system could also help address the overwhelming volume of both AI-generated submissions and AI-generated review comments by increasing transparency and accountability throughout the review process.
Original Article

Similar Articles

The Paper Reviewing Process is Now in Critical Condition [D]

Reddit r/MachineLearning

An AAAI/ICLR reviewer argues that peer review is in critical condition, estimating that roughly half of the reviews at top AI conferences are now AI-generated or AI-edited, and that AI writing is also saturating paper submissions. The author proposes fully disallowing AI usage, even for polishing tone and style, since it is eroding the trust-based review system.

Peer review is overwhelmed—can it survive in the AI era?

Ars Technica

The article examines how the peer review system is struggling to cope with the exponential growth of research publications and AI-assisted papers, leaving volunteer reviewers overwhelmed and leading to errors and delays, prompting calls for reform.

How's ICLR reviewing going for you? Any observations? [D]

Reddit r/MachineLearning

A discussion post asking the community about their experience reviewing submissions for ICLR, focusing on the rising volume of AI-generated papers and the growing difficulty of detecting AI-written text and 'slop' submissions in OpenReview.