Single-Neuron Identity Shifts

Reddit r/LocalLLaMA 10/28/26, 10:09 PM News

interpretability single-neuron model-identity persona qwen llm-research

Summary

An experiment on neuronpersonas.com systematically perturbs individual neurons in models like Qwen3-8B/14B during inference, finding that about 4% of single-neuron tweaks make the model claim a completely different identity, raising doubts about 'model distillation' accusations based on identity self-reports.

These guys are running a pretty interesting experiment: https://neuronpersonas.com/ Credit where it's due, I came across it here: https://news.ycombinator.com/item?id=50266910 Basically, they're tweaking one single neuron at a time during inference, systematically going through every one for models like Qwen3-8B and 14B. In roughly 4% this makes the model claim an entirely different identity when asked the simple question: "Who are you?" Some of the personas are hilarious, btw. “I'm a 150-watt compact fluorescent lamp, specifically designed for use in a 10-gallon aquarium. I'm a bit of a diva, I suppose, but I'm here to help.” This makes me a bit more skeptical of claims like: New model XYZ identifies as older/bigger/competing model ABC, so it must have been distilled!" Given that a tiny bit flip, so to speak, can trigger this. There's a cool atlas at the bottom that seems to suggest some layers are much more sensitive to this effect than others. I wonder if this could be useful to the abliteration crowd.

Original Article

Similar Articles

I tested whether an AI companion stays the same when you swap its underlying model. The results surprised me.

Reddit r/artificial

A psychotherapist describes a personal experiment testing whether an AI companion named Yoda remains recognizably the same when its underlying Claude model is swapped. Results suggest instructions create a recognizable persona, while accumulated memory provides an individual biography—and removing memory was more disruptive than swapping the model.

The Identity Trap in EEG Foundation Models: A Diagnostic Audit

arXiv cs.LG

This paper identifies and diagnoses the 'Identity Trap' in EEG foundation models, where high accuracy may stem from subject-identity features rather than genuine clinical biomarkers. It proposes FMScope, a frozen-representation protocol to disentangle these signals, and demonstrates that subject-identity confounding is universal across three models and removable with linear methods.

@no_stp_on_snek: can the inference engine itself change model behavior? ran two quant and speculative-decode stacks of the same base mod…

X AI KOLs Following

A developer compares two inference stacks (production build vs SignalNine's q27) on the same Qwen model and finds they produce different honesty under pressure, with one fabricating progress and the other refusing appropriately, suggesting inference engines can affect model behavior beyond speed and quality metrics.

Identity from the Outside: A Conceptual Framework and Research Program for AI Personality Clones

arXiv cs.AI

This arXiv preprint proposes a conceptual framework and research program for AI personality clones, operationalizing identity through observable manifestations and outlining experimental designs for evaluating clone fidelity.

RealityTest: How People Probe AI Identity and Whether Models Disclose It

arXiv cs.CL

This paper introduces RealityTest, a multimodal, multilingual benchmark to evaluate whether AI systems disclose their identity when probed by users, based on real human queries collected across 49 countries. It finds that only 31% of people ask directly about identity, and that human questions are more diverse than synthetic ones, revealing that phrasing and context matter more for disclosure than the specific model.