Steered LLM Activations are Non-Surjective

Hugging Face Daily Papers Papers
activation-steering interpretability safety-research surjectivity llm white-box-control

Summary

This paper proves that activation steering in LLMs produces internal states that cannot be replicated by any textual prompt, establishing a formal separation between white-box steerability and black-box prompting.

Activation steering is a popular white-box control technique that modifies model activations to elicit an abstract change in its behavior. It has also become a standard tool in interpretability (e.g., probing truthfulness, or translating activations into human-readable explanations) and safety research (e.g., jailbreakability). However, it is unclear whether steered behavior is realizable by any textual prompt. In this work, we cast this question as a surjectivity problem: for a fixed model, does every steered activation admit a preimage under the model's natural forward pass? Under practical assumptions, we prove that activation steering pushes the residual stream off the manifold of states reachable from discrete prompts. Almost surely, no prompt can reproduce the same internal behavior induced by steering. We also illustrate this finding empirically across three widely used LLMs. Our results establish a formal separation between white-box steerability and black-box prompting. We therefore caution against interpreting the ease and success of activation steering as evidence of prompt-based interpretability or vulnerability, and argue for evaluation protocols that explicitly decouple white-box and black-box interventions.
Original Article
View Cached Full Text

Cached at: 05/18/26, 02:26 PM

Paper page - Steered LLM Activations are Non-Surjective

Source: https://huggingface.co/papers/2604.09839

Abstract

Activation steering in language models creates internal states that cannot be replicated through standard textual prompts, demonstrating a fundamental distinction between white-box and black-box control methods.

Activation steeringis a popularwhite-box controltechnique that modifies model activations to elicit an abstract change in its behavior. It has also become a standard tool ininterpretability(e.g., probing truthfulness, or translating activations into human-readable explanations) andsafety research(e.g., jailbreakability). However, it is unclear whether steered behavior is realizable by any textual prompt. In this work, we cast this question as asurjectivityproblem: for a fixed model, does every steered activation admit apreimageunder the model’s natural forward pass? Under practical assumptions, we prove thatactivation steeringpushes theresidual streamoff the manifold of states reachable from discrete prompts. Almost surely, no prompt can reproduce the same internal behavior induced by steering. We also illustrate this finding empirically across three widely used LLMs. Our results establish a formal separation between white-box steerability andblack-box prompting. We therefore caution against interpreting the ease and success ofactivation steeringas evidence ofprompt-based interpretabilityor vulnerability, and argue for evaluation protocols that explicitly decouple white-box and black-box interventions.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2604\.09839

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2604.09839 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2604.09839 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2604.09839 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

When is Your LLM Steerable?

arXiv cs.CL

This paper investigates when activation steering succeeds or fails for LLMs by analyzing early decoding dynamics. The authors introduce ASTEER, a large testbed of steered generations, and train a GBDT classifier to predict steering outcomes from early hidden states, enabling efficient steering strength search.

When is Your LLM Steerable?

Hugging Face Daily Papers

This paper introduces a method to predict activation steering effectiveness in language models from early decoding states using a Gradient Boosting Decision Trees (GBDT) classifier, enabling efficient steering strength optimization without full rollouts.

Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning

Hugging Face Daily Papers

ACTS (Agentic Chain-of-Thought Steering) formulates LLM reasoning control as a Markov decision process where a controller agent adaptively steers a frozen reasoner during inference using reasoning strategies and steering phrases. The approach achieves comparable accuracy to full-thinking models with significant token savings, enabling controllable accuracy-efficiency trade-offs.

Steering the Language Axis: From Linear Decodability to Causal Control

arXiv cs.CL

This paper investigates whether language identity in LLMs is linearly decodable and causally controllable via compact activation directions. Through steering and ablation experiments across multiple model families, the authors show that language selection is direction-dependent, layer-specific, and reverts to English when the language signal is ablated.