Tag
This paper uses Centered Kernel Alignment (CKA) and activation patching across 17 instruction-tuned models to show that system prompts are 'seen' at every layer but only deeply restructure representations for persona/formatting instructions — safety prompts barely alter computation, providing a mechanistic explanation for why system-prompt-based safety remains jailbreakable.
This paper investigates the Platonic Representation Hypothesis by examining 16 language models across 8 families on 800 reasoning problems. It finds that while models converge in internal representations, they diverge in reasoning processes, especially post-decision, and shared representations have minimal causal influence on predictions.