What's up with AAAI reviewers and organizers? [D]

Reddit r/MachineLearning News

Summary

A researcher rants about problems with AAAI peer review, including unblinded papers, AI-like reviews, and poor organizer communication.

My paper advanced to the second round...but... Out of the papers I reviewed. One did not follow the AAAI template and was unblinded. My review was two lines. The other "human reviewer" gave a list of pros and cons that were similar to the AI review. One was incomplete (missing paragraphs, figures, code, no details). My review was also two lines. The other "human reviewer" also gave a list of pros and cons, that were similar to the AI review. One was LLM math which I believe was actually correct, because it advanced to the second round, despite the references being at a different level of detail, and covering multiple fields of math, insufficient references for theorems / rules, and no exposition as to why the paper was actually useful / interesting. My review for that paper was the longest out of all the papers I reviewed, dotting the is and crossing the ts to make sure it wouldn't be seen as a lazy "reject" review. Yet it advanced to Phase 2. Also, none of the AAAI workflow chairs or similar apologized or even acknowledged a mistake for spamming my coauthors about: "Your coauthor is irresponsible", because I accepted an emergency review invitation (and received these emails a few hours after accepting that invitation). Ok rant over.
Original Article

Similar Articles

AAAI 2027 Reviewer Bidding and Assignment Integrity [D]

Reddit r/MachineLearning

The article discusses collusion in the AAAI 2027 review process, particularly in reviewer assignment cycles, and critiques the lack of code publication in accepted papers at top AI conferences.

AAAI 2027 Review: No code submission? [D]

Reddit r/MachineLearning

A reviewer for AAAI 2027 expresses surprise at the low number of paper submissions with code, despite the conference's emphasis on reproducibility, and asks for opinions on whether lack of code should affect review scores.

Peer review is overwhelmed—can it survive in the AI era?

Ars Technica

The article examines how the peer review system is struggling to cope with the exponential growth of research publications and AI-assisted papers, leaving volunteer reviewers overwhelmed and leading to errors and delays, prompting calls for reform.