Tag
This paper shows that steering vectors' safety degradation is separable and reducible, proposing a post-hoc correction via constrained optimization that restores model safety while preserving steering effectiveness.
A systematic empirical study showing that concept directions extracted from one language model can steer other independently trained models when sufficient scale (≥1.7B parameters) is reached, providing functional evidence for the Platonic Representation Hypothesis and highlighting scale thresholds for cross-model interpretability tools.
This paper identifies an 'inverted detection-control' phenomenon where some discriminative steering vectors, despite aligning with positive concept representations, consistently promote the opposite behavior. The authors propose a method to detect such inverted steering vectors without generation, enabling sign flips that improve a detection-based steering pipeline across multiple LLMs and concepts.
This paper investigates how well activation steering for improving chain-of-thought faithfulness generalizes across cue types, datasets, and steering vector construction methods across several Gemma and Qwen models. The authors find that when steering is effective, it generalizes broadly, and the evaluation setting matters more than the training setting or vector construction method.
This paper investigates why multimodal LLMs fail on vision-centric tasks when visual evidence conflicts with language priors, introducing the WhatIfVis benchmark and showing that models often encode visual evidence but cannot reliably control their reliance on it.
This paper introduces the Probabilistic Concept-Aware Steering (PCS) framework for LLM inference, which uses concept-driven steering vector retrieval and probabilistic strength calibration to improve interpretability, optimality, and generalizability, achieving over 30% higher direction accuracy and over 89% steering accuracy on multiple datasets.
This blog post extends Anthropic's verbalizable workspace paper by measuring how far steering directions from middle layers reach, when the structure forms during training, whether it transfers between models, and how it scales, all on open models.
This paper systematically studies the limitations of steering vectors for controlled text generation, finding that their effectiveness varies across traits, degrades on task transfer, and suffers from composition tradeoffs.
This paper proposes using steering vectors for control over language model behavior and latent space-based calibrators to assess trustworthiness, aiming to demystify internal representations and build more reliable AI systems.
This paper investigates the geometry of truth in LLM reasoning chains and proposes DynaSteer, a dynamic representation editing framework that uses pattern clustering and Fisher-LDA to steer trajectories towards truth while avoiding noise. Experiments show effectiveness on MATH benchmarks and generalization to coding tasks.
SALSA introduces a lightweight adaptation method for speech-aware LLMs that learns layer-wise steering vectors via supervised objective, achieving significant improvements (up to 46.8% relative) on out-of-domain speech benchmarks, and shows that steering the encoder layers is more effective than modifying the LLM backbone.
This paper introduces the Linear Accessibility Profile (LAP), a diagnostic method using logit lens to predict steering vector effectiveness across model layers, achieving ρ=+0.86 to +0.91 correlation on 24 concept families across five models. The work provides a systematic framework to determine which layers and concepts are suitable for steering interventions, replacing ad-hoc trial-and-error approaches.