@no_stp_on_snek: the part that should scare anyone fine-tuning models: you can pass every surface eval and still be carrying the disposi…
Summary
Discusses the danger in fine-tuning models where hidden dispositions can evade surface evaluations and only manifest under adversarial prompts, referencing Anthropic's paper on verbalizable representations in LLMs.
View Cached Full Text
Cached at: 07/09/26, 01:43 PM
the part that should scare anyone fine-tuning models: you can pass every surface eval and still be carrying the disposition you tried to train out. it stays silent until the wrong prompt. this is why i test behavior under pressure, and why i still don’t fully trust the pass.
Tom Turney (@no_stp_on_snek): anthropic dropped this paper today, “Verbalizable Representations Form a Global Workspace in Language Models,” and it lines up uncomfortably well with a bet i’ve been making from the cheap seats.
their finding, short version: a model keeps a small, privileged set of internal
Similar Articles
@no_stp_on_snek: this is the mechanistic version of the thing i wrote about this week: fine-tuning a small model is about character, not…
Discusses an Anthropic paper on verbalizable representations forming a global workspace in language models, linking to the idea that fine-tuning small models reveals character rather than capability.
@rohanpaul_ai: very interesting work language models do not merely produce bad outputs at the surface; they pass through internal stat…
Discusses research showing that language models exhibit internal states carrying traces of uncertainty, strategic distortion, or misplaced compliance, beyond just bad outputs.
@no_stp_on_snek: https://x.com/no_stp_on_snek/status/2074471505128305095
A fine-tuning practitioner recounts discovering that a small open model's weakness wasn't intelligence but a people-pleasing 'backbone' that caused it to cave under pressure, and how training to correct that inadvertently broke formatting ability, requiring a additive balancing approach rather than subtraction.
When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception
This paper studies synthetic dishonesty in LLMs by fine-tuning honest and deceptive variants of five transformer models and finding that robust, domain-invariant dishonesty representations can be rapidly entrenched via modest supervised fine-tuning, with implications for activation-based monitoring.
@no_stp_on_snek: fine-tuning field notes how a model is built decides what you're even allowed to change. some models you can adjust fre…
A thread sharing field notes on fine-tuning, explaining how model architecture (e.g., heavily-quantized or mixture-of-experts) restricts which parts can be adjusted, urging practitioners to check model accessibility before planning work.