A submission was desk-rejected from NeurIPS based on an uncalibrated AI detector (Pangram), raising concerns about circularity in the review process and unvalidated false-positive rates on the target distribution.
I recently had a submission desk-rejected from the NeurIPS 2026 Position Paper Track for an alleged AI-policy violation. After corresponding with the track leadership and reading their public blog post, I think the broader methodological issue is worth discussing here. The track used Pangram, a proprietary AI-text detector, as part of the desk-rejection process. I was told that the materials considered for desk rejection were: * the detector output * the authors’ AI-use attestation This creates a potential circularity problem. If a high detector score is used to judge the author’s attestation as inconsistent, and that inconsistency is then used to justify desk rejection, the detector is not just an aid. It becomes a decisive part of the adjudication process. The bigger issue is validation. The NeurIPS blog describes tests using Pangram audits, older ACM FAccT papers, synthetic AI-generated position papers, and manually edited samples. But the target population was NeurIPS 2026 Position Paper submissions, whose ground-truth authorship process is unknown. So the key question is: **What is the false-positive rate of the final decision procedure on the actual target distribution?** A false-positive rate measured on one distribution does not automatically transfer to another. If the actual submission pool produced a "surprisingly high flagged rate" (citation from NeurIPS blog post), that could indicate distribution shift / miscalibration. To sanity-check the detector’s behavior, I also ran Pangram on recent 2026 papers authored by NeurIPS Position Paper Track Chairs. Pangram returned scores including: * 69% AI * 45% AI * 36% AI * 24% AI I am **not** claiming those papers were AI-written. For me, Pangram’s outputs alone does not permit such a conclusion. And that is exactly the point.
NeurIPS 2026 used a proprietary AI-text detector to desk-reject papers for alleged AI policy violations without validating it on the target distribution; the same detector later flagged conference chairs' own papers as likely AI-written.
Discussion about the use of AI-generated reviews at NeurIPS 2026, including concerns over prompt injection and lack of consequences for reviewers using LLMs without proper oversight.
A NeurIPS reviewer reports encountering a paper and rebuttals that appear entirely LLM-generated, expressing frustration and seeking advice on how to evaluate such submissions.
This paper reveals that commercial AI detectors like GPTZero and Pangram judge text from base language models as overwhelmingly human, while instruction-tuned model outputs are flagged as AI-generated. The authors propose HIP, a detector-agnostic iterative paraphrasing pipeline that improves human-likeness while preserving semantics.
A writer questions the accuracy of AI detection tools after receiving wildly inconsistent results for her own human-written movie reviews, highlighting the unreliability of current AI checkers.