@panickssery: In addition to identifying LLM use, Pangram has identified many academics who don't know the difference between false p…
Summary
Pangram, a tool for detecting LLM use, reveals that many academics do not understand the difference between false positives and false negatives.
Similar Articles
@rohanpaul_ai: LLMs often cannot tell when an attack made them say something unsafe. Asking an LLM whether its own previous answer was…
This paper investigates whether LLMs can reliably self-report when their outputs have been compromised by adversarial prefills, finding that models often cannot distinguish between compromised and intentional outputs, and their limited recognition stems from normal refusal behavior rather than true self-awareness.
It's Not Just X. It's Y
This essay explores how LLMs' post-training (RLHF and RLVR) produces linguistic tics like negative parallelism, and critiques the use of AI detectors (Grammarly, Pangram) that force writers to sound like machines to avoid false accusations.
@LM_Braswell: Confirmed LLMs now much better than room of avid Anagram players - can you figure out where to put the last I?
LLMs now outperform a room of proficient anagram players, as demonstrated in a recent evaluation.
LLMs know when they are wrong. I made a fix relating to Anthropic's new "global workspace" paper [R]
The author presents a method to make LLMs verbalize calibrated confidence by using a linear probe on mid-layer states and a small trained bridge to confidence logits, requiring only 200 labeled examples and no weight modification. This is linked to Anthropic's global workspace paper explaining the know-say gap.
@GaryMarcus: Am old enough to remember when @GeoffreyHinton told me I was stupid for saying that LLMs regurgitate training data. He …
Gary Marcus highlights recent DeepMind research confirming that LLMs frequently memorize and regurgitate training data, countering past criticism from Geoffrey Hinton. The post underscores ongoing debates about LLM limitations and their real-world capabilities.