Tag
This paper introduces the Probabilistic Concept-Aware Steering (PCS) framework for LLM inference, which uses concept-driven steering vector retrieval and probabilistic strength calibration to improve interpretability, optimality, and generalizability, achieving over 30% higher direction accuracy and over 89% steering accuracy on multiple datasets.
This blog post extends Anthropic's verbalizable workspace paper by measuring how far steering directions from middle layers reach, when the structure forms during training, whether it transfers between models, and how it scales, all on open models.
This paper systematically studies the limitations of steering vectors for controlled text generation, finding that their effectiveness varies across traits, degrades on task transfer, and suffers from composition tradeoffs.
This paper proposes using steering vectors for control over language model behavior and latent space-based calibrators to assess trustworthiness, aiming to demystify internal representations and build more reliable AI systems.
This paper investigates the geometry of truth in LLM reasoning chains and proposes DynaSteer, a dynamic representation editing framework that uses pattern clustering and Fisher-LDA to steer trajectories towards truth while avoiding noise. Experiments show effectiveness on MATH benchmarks and generalization to coding tasks.
SALSA introduces a lightweight adaptation method for speech-aware LLMs that learns layer-wise steering vectors via supervised objective, achieving significant improvements (up to 46.8% relative) on out-of-domain speech benchmarks, and shows that steering the encoder layers is more effective than modifying the LLM backbone.
This paper introduces the Linear Accessibility Profile (LAP), a diagnostic method using logit lens to predict steering vector effectiveness across model layers, achieving ρ=+0.86 to +0.91 correlation on 24 concept families across five models. The work provides a systematic framework to determine which layers and concepts are suitable for steering interventions, replacing ad-hoc trial-and-error approaches.