Tag
This paper proposes an 'empirical contract' for LLM interpretability that distinguishes association, control, sufficiency, and use of internal features by testing interventions (interchange installation/removal and downstream rescue) at natural input values, showing that steering behavior alone does not prove the model mechanistically uses a feature.
The paper proposes HEST, a method that embeds LLM hidden states into the Poincaré ball with an entropy-trained probe and steers only high-entropy (hesitant) tokens along hyperbolic geodesics, improving math reasoning accuracy on Qwen2.5-Math and Llama-3.1 models where fixed-vector Euclidean steering fails.
The paper demonstrates a targeted bias injection attack on frozen diffusion language models (dLLMs) using a proportional-integral closed-loop controller that adapts steering strength during denoising, strongly increasing biased answers on benchmarks like BBQ and SocialStigmaQA, and argues bias audits must examine the serving stack rather than just the frozen model.
The paper introduces Activation-Conditioned Self-Distillation (ACSD), which extracts a steering vector from activation contrasts between a model's correct and incorrect self-generated trajectories to build a frozen self-teacher, achieving state-of-the-art mean accuracy across five models on mathematical benchmarks, including 71.9% on DeepSeek-R1-0528-Qwen3-8B.
The paper introduces MetaSteer, a method that learns nonlinear, context-dependent interventions applied to attention projection matrices in LLMs, trained once on pooled preference data and transferred zero-shot to unseen concepts and agentic settings, matching or outperforming task-specific steering baselines.
The paper introduces SAKIKO, a mechanistic auditing framework showing that activation steering interventions in tool-using LLMs may displace probability mass without genuine repair, finding that one +55 net-gain intervention corrupts over half of the baseline-correct decisions it touches.
A paper introducing PersonaDose, a calibrated activation-steering approach that controls language model persona expression via trait descriptions and requested mean intensity, evaluated across Llama-3.1-8B, Qwen3-8B, and Gemma-3-4B.
This paper introduces AIMES, a framework for adaptive multi-value activation steering in large language models that uses online observer feedback for state-aware adaptation without additional training, showing improved controllability over fixed methods.
This paper introduces a guarded gradient-based activation steering method to detect and correct shutdown-avoidance responses in the Qwen3.5-0.8B model, using a classifier and minimum-step policy for selective and limited effectiveness.
LocUS is a novel method for targeted activation steering in large language models that restricts interventions to specific attention heads and a subspace of the unembedding matrix, improving performance while preserving general capabilities.
The study reveals that chat templates control whether language models adopt a disclaimer voice (e.g., 'I'm just an AI') or an experiential voice (e.g., 'I feel'), and identifies an activation direction that can steer this behavior, impacting AI safety research.
This paper reveals that the optimal layer for linear probing to read concepts differs from the optimal layer for activation steering in omni-modal large language models, challenging common heuristics in representation engineering.
This study finds that activation steering in latent chain-of-thought reasoning is less effective than in explicit CoT, highlighting a transition gap where interventions in latent space fail to transfer to language generation.
The study explores steering LLMs towards human moral foundations using the Norwegian MFQ-30 questionnaire, evaluating prompt-level persona steering and activation-level interventions.
This paper explores the role of fine-grained category-specific harmfulness representations in LLMs for safety. It shows that category residuals, orthogonal to general harm, vary in encoding harmfulness and can induce refusal, with implications for understanding LLM safety mechanisms.
This study validates activation-steering claims for sycophancy in language models, finding no usable linear capitulation direction in two small LLMs and highlighting measurement hazards that underestimate sycophancy.
GeoSteer is an optimization-based method for norm-preserving activation steering in large language models, using geodesic updates on the representation manifold to improve control over model behavior while preserving activation norms.
This paper proposes a spillover-aware method for multi-value activation steering to achieve pluralistic alignment in LLMs, improving control over multiple value dimensions without fine-tuning or reward models.
Research article exploring the use of Jacobian space to extract steering vectors from concept tokens for steering LLM behaviors, with experiments on Qwen3-1.7B showing promise for simple tasks but limitations in complex scenarios.
This paper introduces a benchmark to validate that distribution-driven methods for activation steering in LLMs recover human value topologies aligned with Schwartz's theory, improving with model size but declining post-instruction tuning.