Tag
This paper investigates whether role-playing in LLMs changes only outputs or also internal truth representations, using linear probes. It finds that roleplay shifts outputs more than internal beliefs, while emergent misalignment causes larger shifts in internal representations.
This paper investigates how instruction-tuned LLMs combine persona and task specifications in the residual stream, finding that near answer formation the combination is approximately additive, enabling substitution with minimal KL divergence, but this additive regime does not account for the full multi-token generation mechanism.
A study demonstrates that simply changing the formatting (prose vs bullet points) of a persona prompt dramatically flips an LLM's behavior in a Prisoner's Dilemma, from 96% cooperation to 20%, illustrating extreme sensitivity to format despite identical content (p < 0.001).