@panickssery: In addition to identifying LLM use, Pangram has identified many academics who don't know the difference between false p…
Summary
Pangram, a tool for detecting LLM use, reveals that many academics do not understand the difference between false positives and false negatives.
Similar Articles
@rohanpaul_ai: LLMs can accept the same false claim differently depending on its tone, certainty, and grammatical form. Small wording …
This paper presents EoBench, a benchmark testing LLMs' acceptance of false claims phrased in 19 different styles, finding that tone, certainty, and grammatical form significantly affect model responses, with larger and instruction-tuned models showing more resistance.
Distinguishing Artificial from Authentic: Evaluating LLMs for Detecting LLM-Generated Content
This paper investigates how well LLMs can detect their own generated content in educational contexts, finding that detection accuracy varies by task type and is unreliable for short-answer questions.
Interpreting Pangram
The article discusses an incident where David Sacks' tweet was detected as AI-generated by Pangram, sparking debate about AI detectors, and includes an experiment using an LLM to mimic human writing.
[P] We re-ran NeurIPS's pre-LLM vs 2025 paper check with a second AI-text detector. Neither flags a 2022 paper; on 2025 papers they disagree [P]
A ParaTrace developer shares a head-to-head comparison of two AI-text detectors (ParaTrace v7 vs Pangram 3.3.2/4) on pre-LLM 2022 papers versus 2025 papers, finding no false positives on 2022 texts but significant disagreement on 2025 papers, and benchmarking ParaTrace against Pangram's public test sets.
@rohanpaul_ai: LLMs often cannot tell when an attack made them say something unsafe. Asking an LLM whether its own previous answer was…
This paper investigates whether LLMs can reliably self-report when their outputs have been compromised by adversarial prefills, finding that models often cannot distinguish between compromised and intentional outputs, and their limited recognition stems from normal refusal behavior rather than true self-awareness.