AI detectors simply don't work

Reddit r/ArtificialInteligence News

Summary

An informal experiment testing multiple AI detectors found they produce inconsistent and unreliable results, flagging human-written essays as AI while sometimes missing actual AI-generated text. The author argues this unreliability is a serious problem for students and submitters whose work may be judged by these tools.

I put a test agaisnt multiple software programs that detect AAI-generatedcontent and text. My experiment was writing multiple essays myself in a Word document and telling AI to write them again for me word for word. The results came back interesting. Some of my data showed the detectors flagging my own essays as AI but not the actual AI one; others did the opposite. I used to think that the checkers only checked for AI watermarks hidden in text, but clearly it doesn't. I told AI to rewrite an essay twice, same words and phrases. The first one came out as "Human-Written", whereas the second one came back "100% AI", which is both AI. During this study, I copied and pasted the actual AI responses straight to the detectors and the handwritten one straight to the detector This is a huge problem that needs to be fixed; students and submitters of work can get in serious trouble, especially if the reviewer relies on AI detectors. With research done, I read that these detectors work by flagging consistent artificial flow in text, which would make sense. The only problem is that it flags non-AI text as AI, or it flags AI work as human-written. Same words, same phrases, same work, different results.
Original Article

Similar Articles

AI detectors are a bad idea

Reddit r/ArtificialInteligence

This article argues that AI detectors and watermarking are problematic because they dismiss human effort behind work and degrade AI-generated text quality.

AI detectors are creating a new era of distrust

The Verge

This article examines the growing use of AI detectors in education and the resulting atmosphere of distrust, highlighting concerns about false positives and the subjective nature of AI-written text detection.

Why AI Detection Fails for Academic Integrity

arXiv cs.LG

This paper evaluates commercial AI detectors in academic settings, finding high false-positive rates on AI-assisted human writing and near-total evasion via humanizers, concluding detector scores should not be standalone misconduct evidence.

How accurate are AI checkers?

Reddit r/artificial

A writer questions the accuracy of AI detection tools after receiving wildly inconsistent results for her own human-written movie reviews, highlighting the unreliability of current AI checkers.