Regression to the Mean: on LLMs and the quiet death of the new
Summary
An essay arguing that LLMs, by returning the most probable continuation, may inadvertently suppress genuine novelty and deviation, leading to a cultural convergence toward the average rather than the truly new.
View Cached Full Text
Cached at: 07/06/26, 02:03 PM
Similar Articles
LLMs are stuck in a groupthink groove. This startup is trying to get them out.
A startup called Springboards has developed an LLM named Flint that aims to produce more varied and creative responses than mainstream models, addressing a widespread issue of homogeneity in AI outputs. The article highlights research showing that many LLMs converge on similar answers due to similar training data and methods.
Making LLMs Better at Creative Writing using Entropy
This article presents a technique to improve LLM creative writing by modifying the sampling process using entropy, aiming to reduce the generic 'LLM feel' in generated text.
Use Boring Languages with LLMs
An opinion piece arguing that LLMs perform better with boring, consistent languages and ecosystems (like Ruby on Rails) because the training corpus has lower variance, leading to more reliable agentic output, while fragmented ecosystems (like JavaScript) produce poor results.
LLMs know when they are wrong. I made a fix relating to Anthropic's new "global workspace" paper [R]
The author presents a method to make LLMs verbalize calibrated confidence by using a linear probe on mid-layer states and a small trained bridge to confidence logits, requiring only 200 labeled examples and no weight modification. This is linked to Anthropic's global workspace paper explaining the know-say gap.
More Aligned, Less Diverse? Analyzing the Grammar and Lexicon of Two Generations of LLMs
This academic paper analyzes the syntactic and lexical diversity of two generations of LLMs compared to human-authored news text, finding that newer, aligned models exhibit reduced diversity.