activation-steering

Tag

Cards List
#activation-steering

Does the Model Use the Feature? Separating Steering from Mechanism in LLMs

arXiv cs.AI ↗ · 2026-10-08 Cached

This paper proposes an 'empirical contract' for LLM interpretability that distinguishes association, control, sufficiency, and use of internal features by testing interventions (interchange installation/removal and downstream rescue) at natural input values, showing that steering behavior alone does not prove the model mechanistically uses a feature.

0 favorites 0 likes
#activation-steering

Hesitation Has a Geometry: Entropy-Trained Hyperbolic Probes for Sparse Activation Steering

arXiv cs.LG ↗ · 2026-10-05 Cached

The paper proposes HEST, a method that embeds LLM hidden states into the Poincaré ball with an entropy-trained probe and steers only high-entropy (hesitant) tokens along hyperbolic geodesics, improving math reasoning accuracy on Qwen2.5-Math and Llama-3.1 models where fixed-vector Euclidean steering fails.

0 favorites 0 likes
#activation-steering

Noise Out, Bias In: Targeted Bias Injection in Diffusion Language Models via Closed-Loop Activation Steering

Hugging Face Daily Papers ↗ · 2026-10-05 Cached

The paper demonstrates a targeted bias injection attack on frozen diffusion language models (dLLMs) using a proportional-integral closed-loop controller that adapts steering strength during denoising, strongly increasing biased answers on benchmarks like BBQ and SocialStigmaQA, and argues bias audits must examine the serving stack rather than just the frozen model.

0 favorites 0 likes
#activation-steering

Activation-Conditioned Self-Distillation

arXiv cs.LG ↗ · 2026-10-02 Cached

The paper introduces Activation-Conditioned Self-Distillation (ACSD), which extracts a steering vector from activation contrasts between a model's correct and incorrect self-generated trajectories to build a frozen self-teacher, achieving state-of-the-art mean accuracy across five models on mathematical benchmarks, including 71.9% on DeepSeek-R1-0528-Qwen3-8B.

0 favorites 0 likes
#activation-steering

MetaSteer: Context-Conditioned, nonlinear Steering via Attention-Projection Adaptation

arXiv cs.CL ↗ · 2026-10-01 Cached

The paper introduces MetaSteer, a method that learns nonlinear, context-dependent interventions applied to attention projection matrices in LLMs, trained once on pooled preference data and transferred zero-shot to unseen concepts and agentic settings, matching or outperforming task-specific steering baselines.

0 favorites 0 likes
#activation-steering

When Does Correction Become Repair? Mechanistic Auditing of Internal Interventions in Tool-Using LLMs

arXiv cs.CL ↗ · 2026-09-30 Cached

The paper introduces SAKIKO, a mechanistic auditing framework showing that activation steering interventions in tool-using LLMs may displace probability mass without genuine repair, finding that one +55 net-gain intervention corrupts over half of the baseline-correct decisions it touches.

0 favorites 0 likes
#activation-steering

Persona Dosing: Calibrated Activation Steering for Graded Trait Control

arXiv cs.AI ↗ · 2026-09-30 Cached

A paper introducing PersonaDose, a calibrated activation-steering approach that controls language model persona expression via trait descriptions and requested mean intensity, evaluated across Llama-3.1-8B, Qwen3-8B, and Gemma-3-4B.

0 favorites 0 likes
#activation-steering

Adaptive Multi-Value Control in LLMs via Causal Activation Steering

arXiv cs.LG ↗ · 2026-09-29 Cached

This paper introduces AIMES, a framework for adaptive multi-value activation steering in large language models that uses online observer feedback for state-aware adaptation without additional training, showing improved controllability over fixed methods.

0 favorites 0 likes
#activation-steering

Guarded Gradient-Based Activation Steering of Shutdown Responses in Qwen3.5-0.8B: A Minimum-Step Policy

arXiv cs.LG ↗ · 2026-09-29 Cached

This paper introduces a guarded gradient-based activation steering method to detect and correct shutdown-avoidance responses in the Qwen3.5-0.8B model, using a classifier and minimum-step policy for selective and limited effectiveness.

0 favorites 0 likes
#activation-steering

LocUS: Head Selection and Subspace Projection for Targeted Activation Steering

arXiv cs.CL ↗ · 2026-09-28 Cached

LocUS is a novel method for targeted activation steering in large language models that restricts interventions to specific attention heads and a subspace of the unembedding matrix, improving performance while preserving general capabilities.

0 favorites 0 likes
#activation-steering

"As a Language Model...": Chat Template Switches LLM Self-Referential Voice and Activation Steering Reproduces It

arXiv cs.LG ↗ · 2026-09-23 Cached

The study reveals that chat templates control whether language models adopt a disclaimer voice (e.g., 'I'm just an AI') or an experiential voice (e.g., 'I feel'), and identifies an activation direction that can steer this behavior, impacting AI safety research.

0 favorites 0 likes
#activation-steering

Read-Best Is Not Steer-Best: A Probing--Steering Layer Dissociation in Omni-Modal Large Language Models

arXiv cs.CL ↗ · 2026-09-22 Cached

This paper reveals that the optimal layer for linear probing to read concepts differs from the optimal layer for activation steering in omni-modal large language models, challenging common heuristics in representation engineering.

0 favorites 0 likes
#activation-steering

When Steering Fails in Latent Reasoning: A Latent-to-Language Transition Gap

arXiv cs.CL ↗ · 2026-09-21 Cached

This study finds that activation steering in latent chain-of-thought reasoning is less effective than in explicit CoT, highlighting a transition gap where interventions in latent space fail to transfer to language generation.

0 favorites 0 likes
#activation-steering

Steering LLMs Responses Towards Moral Foundations on the Norwegian MFQ-30

arXiv cs.CL ↗ · 2026-09-21 Cached

The study explores steering LLMs towards human moral foundations using the Norwegian MFQ-30 questionnaire, evaluating prompt-level persona steering and activation-level interventions.

0 favorites 0 likes
#activation-steering

The Role of Fine-grained Harm Signals in LLM Safety

arXiv cs.CL ↗ · 2026-09-18 Cached

This paper explores the role of fine-grained category-specific harmfulness representations in LLMs for safety. It shows that category residuals, orthogonal to general harm, vary in encoding harmfulness and can induce refusal, with implications for understanding LLM safety mechanisms.

0 favorites 0 likes
#activation-steering

No Usable Linear "Capitulation Direction" in Two Small LLMs: A Validation Protocol for Activation-Steering Claims, and a Cross-Family Behavioral Study of Sycophancy Under Pushback

arXiv cs.CL ↗ · 2026-09-17 Cached

This study validates activation-steering claims for sycophancy in language models, finding no usable linear capitulation direction in two small LLMs and highlighting measurement hazards that underestimate sycophancy.

0 favorites 0 likes
#activation-steering

GEOSTEER: Geodesic Optimization for Activation Steering in Large Language Models

arXiv cs.LG ↗ · 2026-09-11 Cached

GeoSteer is an optimization-based method for norm-preserving activation steering in large language models, using geodesic updates on the representation manifold to improve control over model behavior while preserving activation norms.

0 favorites 0 likes
#activation-steering

Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignment

arXiv cs.AI ↗ · 2026-09-10 Cached

This paper proposes a spillover-aware method for multi-value activation steering to achieve pluralistic alignment in LLMs, improving control over multiple value dimensions without fine-tuning or reward models.

0 favorites 0 likes
#activation-steering

Extracting Steering Vectors from J space

Hacker News Top ↗ · 2026-09-06 Cached

Research article exploring the use of Jacobian space to extract steering vectors from concept tokens for steering LLM behaviors, with experiments on Qwen3-1.7B showing promise for simple tasks but limitations in complex scenarios.

0 favorites 0 likes
#activation-steering

Steering Geometry: Validating Human Value Geometry in LLM Steering Space

Hugging Face Daily Papers ↗ · 2026-09-05 Cached

This paper introduces a benchmark to validate that distribution-driven methods for activation steering in LLMs recover human value topologies aligned with Schwartz's theory, improving with model size but declining post-instruction tuning.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback