Tag
The paper explores using large language models (Claude Opus 5) to automatically construct simple baselines for 36 deep learning tasks from Nature/Science/Cell-family studies, finding these baselines recover a median 95.5% of the performance gap and match or exceed the published deep learning model in 12 studies, suggesting automated baseline checking as a scalable safeguard for scientific peer review.
A call for 2 emergency reviewers for ICLR, requiring expertise in RL, agent memory, and/or information theory, with reviews due by Nov 2 AoE.
A poster expresses disgust that a friend fabricated metrics in a gap-analysis paper and still got accepted at AAAI, raising concerns about peer review rigor at top AI conferences.
IEEE Spectrum's Benjamin Skuse ran an experiment with IEEE Access to see if an amateur could use free LLMs (ChatGPT, Gemini) to generate a convincing fake scientific paper and slip it past a journal's defenses, highlighting how AI slop is eroding trust in academic publishing.
A study reports that large language models are significantly better at detecting mistakes in academic papers than trained human reviewers, suggesting a potential shift in how peer review quality assurance could be assisted by AI.
An AAAI/ICLR reviewer argues that peer review is in critical condition, estimating that roughly half of the reviews at top AI conferences are now AI-generated or AI-edited, and that AI writing is also saturating paper submissions. The author proposes fully disallowing AI usage, even for polishing tone and style, since it is eroding the trust-based review system.
AllenAI (Ai2) releases 'Old Ideas, Novel Problems', a study and benchmark showing that LLM novelty judges are highly unreliable: small prompt changes shift pairwise novelty judgments by over 50 percentage points, and retrieval/reasoning additions don't consistently help. The release includes a benchmark, data, code, and an automated pipeline built from ICLR 2026 peer reviews for building new novelty-evaluation sets.
AlphaIdeaBench introduces a benchmark of 100 research-ideation tasks paired with explicit resource budgets, using an LLM judge validated against real ICLR/NeurIPS/ICML reviewer scores to evaluate eight ideation harnesses and eight backbone LLMs for novelty and feasibility.
The paper introduces PaperScorer, a family of models that predict a paper's peer-review accept/reject decision from text or page screenshots, achieving 68% accuracy on ICLR papers and 77% on arXiv ML papers by fine-tuning on anonymized corpora and outperforming frontier models like GPT and Gemini.
ICLR 2027 continues to issue desk rejects for submissions, with the community being thanked for helping guard against low-quality AI-generated papers flooding the review process.
研究者Peter Henderson在ICLR审稿中表达对AI冲击学术同行评审体系的担忧，指出大量疑似AI生成的投稿正导致审稿质量下降与评审系统的死亡螺旋。
该论文提出一种作者参与的智能体协作复现性审计流程，将作者声明作为AI agent调查与评估的显式输入，并在某机器学习会议的36篇论文、133个实验表格上部署，人工复核后提出20个问题并全部闭环解决。
A Hacker News discussion帖吐槽 ICLR 审稿中泛滥的晦涩 AI 写作，并附上一段匿名论文的典型模糊表述，呼吁社区讨论如何在不简单指责“AI 写作”的前提下批评这类糟糕行文的实质问题。
A researcher shares their frustration after being desk rejected from ICLR because half a sentence spilled onto page 10, criticizing the process and questioning the value of the provided LLM feedback for catching formatting errors.
The paper introduces ReviewFact, a benchmark for labeling factual errors in human and AI-written peer review comments, and PremiseCheck, a framework combining LLM premise rewriting with decision models that matches GPT-5.5 accuracy at roughly half the cost across NLP, ML, and natural sciences.
Google Research publicly released Deep Review, an agentic AI system for rigorous technical document review across academic, legal, and finance domains, evolving from its Paper Assistant Tool pilots with major CS conferences. It offers tiered access with a scalable Gemini API and guarantees uploaded documents are not used for model training.
An ICLR reviewer reports that all three assigned papers appear to be unfiltered Claude-generated slop sharing the same core idea of comparing predictor errors via paired differences to derive tighter bounds, raising concerns about automated paper-generation at top conferences.
一个名为 Paper Review Harness 的开源 GitHub 项目，利用 30 个 AI 审稿人角色模拟 CVPR、ICLR、ICML、NeurIPS 等会议的同行评审反馈，帮助研究者在投稿前对论文进行压力测试。它提供分区评分、缺陷识别和修订追踪工作流，但不能替代真实同行评审。
Michał Wójcik, former neuroscientist turned research-integrity investigator, documents AI-generated and fabricated content — nonsense figures, hallucinated references, and unreadable book chapters — slipping through the editorial filters of major publishers like Springer Nature and Elsevier.
A discussion post asking the community about their experience reviewing submissions for ICLR, focusing on the rising volume of AI-generated papers and the growing difficulty of detecting AI-written text and 'slop' submissions in OpenReview.