This from @TuhinChakr is brilliant. That prize winning story from Granta? Turns out it’s just a bunch of random whole phrases taken directly from existing text on the internet. Tool allows you to trace those n-grams directly to their source, which is mostly random fanfiction.

https://tuhinchakrabarty.substack.com/p/ai-slop-grantagate-and-bad-writing…

AI-slop , GrantaGate and Bad Writing

Source: https://tuhinchakrabarty.substack.com/p/ai-slop-grantagate-and-bad-writing A few days ago, at a very fancy party in NYC, this young woman in her 20s walked up to me and asked, “Are you writing something?” “Academics are never not writing, my friend,” I said in a mildly sarcastic tone. “You know what, I’d like to read your stuff on Substack,” she said. “But that doesn’t get you tenure,” I replied. Since that exchange, I have wondered: should I write something non-academic that is also a bit academic? My hesitance can in part be attributed toGeorge SaundersandOttessa Moshfegh, whose brilliant Substack columns intimidated me. I am no writer. Who on earth would want to read my Substack? Yet here I am, writing my first Substack post.

The last two days have been a bit crazy, thanks to acertain short storypublished in Granta magazine that has now been awarded a prize by the Commonwealth Foundation. Nabeel Qureshi was the first on my feed tohighlight that it was AI-generated. As I write, his tweet has 1.5M views which is roughly 1.4M more than anything I have ever written. For us fortunate, or if I may say unfortunate, professors whose inboxes are inundated by AI-written emails, it is not exactly detective work to recognize that this short story was completely AI-generated.

Pangram’s AI detection confirmed this. For those who don’t know me, I should admit I am a bit of an AI-detection maximalist, so obviously I got into arguments on X with people who were quick to dismiss the detection results and insist detectors don’t work. Now putting that aside, a journalist from El País asked me yesterday, “Why did we suddenly decide, as in the case of this award, that a text is AI-generated when there’s no reliable detection tool available?” My reply was brief. I pointed to the stylistic tics and patterns prevalent in AI writing. Last year I won a best-paper honorable mention award forwriting a paper characterizing idiosyncrasiesin AI writing. It should not be surprising that I, of all people, have now been trained on an inordinate amount of AI slop.

A few days ago, myvery smart PhD studentcame up to me with a point that stuck with me. In her words, bad AI writing, or low-quality AI writing, can be attributed to how much text it memorized from pre-training. Kind of a brilliant hypothesis. After all, LLMs are not conscious. They do not have a perfect sense of embodiment. They are autoregressive models that generate text by sampling, more or less, from a very large pile of things other people wrote. In simpler terms, it is what my colleagueNajoung Kim(a linguistics professor at BU) calls word salad. Last year we wrote a paper, now accepted to ICLR, on how seemingly novel n-grams areoften nonsensical or non-pragmatic. There are perfect examples of this in the short story. Consider certain phrases: “she had the kind of walking that made benches become men” or “*the girl smiled like sunrise over a sink.”*Like, what does it even mean?

So after much deliberation, I asked my student to run the story throughInfinigram(an n-gram attribution engine, which is a fancy way of saying it tells you where a phrase was probably stolen from). Based on my anecdotal experience, LLMs don’t follow genre conventions; they cherry-pick expressions from wherever, genre be damned. Say I am writing literary fiction; I would avoid tropes or hackneyed metaphors like the plague (you see what I did there :P). That kind of writing tends to live on fan fiction sites and the dustier, lower-ranked corners of the web.

Let’s have a little fun. Below you can see a little demo. Don’t forget to click on it. This will show you how LLMs stitch together verbatim expressions from different sources. It’s not technically infringement, because these expressions are not super long, but they are not original either.

Now let’s decode them one at a time.

• For instance, if you read the sentence “but a belly sound, as if the earthswallows a shoutand holds it there,” it seems rather jarring. Not really what you find in good writing, is it? Don’t worry, this might have been potentially drawn from a fan-fiction webpage,The Fall of a Phoenix. (Sorry, the story is removed now, but the sentence was: “Heswallows a shoutof pain, breathing heavily, his whole body trembling.”)
• I was intrigued by the very purple sentence ending in a typical ChatGPT rule of three (x, y, z). Here, “damp earth, woodsmokeand the sour tang of fermentingcocoa” seems hyperspecific and very rare. Turns out there is a possibility it could have been drawn from anAO3 webpage. The sentence being: “She smelled cookingand the sour tang of fermentinggrain.”
• Consider the sentence “Hiseyes narrowed against the glare outsideand**the**darker glare**inside him.” Here, “eyes narrowed against the glare outside” sounds a bit vain and ominous in a way that’s unnecessary. Well, the reason could be being drawn fromjedifiction.com(that’s defunct now), or arandom horror blog(the sentence being: “Hiseyes narrowed against the glareof the halogen strip lights”). Speaking of “the darker glare” while pretty common also appears in anotherfan-fiction story Poison.

So many other instances of word salad that at this point my head hurts. Another brilliant example of bad plagiarism: “the air sweet withcane and forgetting.” What does it even mean?? Readers on X were quick to remind me thatcanehere refers to sugarcane in Trinidad. I am skeptical. There is a chance it was copied out of context from a Better Call Saul review (an active elderly relative suddenly having to use acane and forgettingyour name).

More bad writing?

“Vishnu began to plan withthe patience of a reptile.” It’s could have been drawn fromThe Mangled Spider(“She hasthe patience of a reptile, lying in wait.”)

“Something coiled inside herchest.” Chance of being drawn from anotherfanficwebpage (“Something coiled inside herstomach as she began to give herself over to sensations[...]”).

Disclaimer: we can’t actually peek inside an LLM’s brain (which is probably for the best). But it’s entirely possible that when you prompt it to write fiction, it’s quietly scouring through every abandoned Wattpad draft and fanfic written at 2 a.m. by someone who really shouldn’t have.

So next time you cringe at AI writing, remember: LLMs are not thinking twice. There is no reflection. They are drawing words from whatever source they can. They do not conform to genre conventions. A lot of bad writing happens because AI hasn’t learned aesthetics. It has simply memorized the whole internet and called it a day. So sure, maybe you don’t trust AI detectors. But you can trust your own eyes. Read closely. I am no expert but maybe this verbatim borrowing, these tonal mismatches, and the metaphors that mean nothing are precisely what AI detectors learn to recognize ?

Discussion about this post

Ready for more?