It's time to desk reject papers that don't include code that can reproduce the results [D]
Summary
The author, after reviewing for three major conferences, argues that papers without code to reproduce results should be desk rejected, citing that only 1 of 12 papers reviewed provided full code and 7 provided none.
Similar Articles
AAAI 2027 Review: No code submission? [D]
A reviewer for AAAI 2027 expresses surprise at the low number of paper submissions with code, despite the conference's emphasis on reproducibility, and asks for opinions on whether lack of code should affect review scores.
Failure to Reproduce Modern Paper Claims [D]
A researcher reports failure to reproduce claims from modern papers, with 4 out of 7 checked claims being irreproducible and 2 having unresolved GitHub issues, raising concerns about research quality standards.
When I reject AI code even if it works
The author explains why they often reject AI-generated code even when it works, citing reasons like inability to explain the approach, overly large diffs, premature abstractions, and reduced system reasoning, and argues for mandatory human review.
TMLR reached out to the authors of 10 papers slated for desk rejection, in an attempt to understand if the authors could explain the paper they submitted [D]
TMLR contacted authors of ten desk-rejected papers to evaluate their ability to explain their work, with results indicating that most authors could not adequately respond to questions, highlighting concerns in academic publishing.
Asking Authors About Their Own Papers
An editor at Transactions on Machine Learning Research conducted meetings with authors of papers slated for desk rejection, revealing many authors could not adequately explain their own papers, raising concerns about authorship, research integrity, and the review process in AI academia.