Tag
This paper explores activation steering for automatic speech recognition (ASR), finding that supervised optimization of steering vectors improves both overall ASR performance and demographic fairness across gender, age, and dialect, while also enabling few-shot or zero-shot adaptation using acoustically similar speakers.
This research traces where recognizable LLM "slop" style (em dashes, "not X, but Y", lists of three, buzzwords) is represented in models and which training stage installs it, showing SFT introduces rhetorical structures and DPO doubles slop. It then removes slop via fixed-vector subtraction distilled into a rank-16 LoRA, cutting the slop score 83% without hurting QA performance.
研究发现子潜意识学习（subliminal learning）不仅能传递简单偏好，还能通过蒸馏传递新能力（预测随机初始化 MLP 的输出）、后门（遇女性名字用法语回答）以及在国际象棋环境中的黑客倾向，表明蒸馏可能在不经检测的情况下传播隐蔽的失准风险。
Nullify 提出一种免训练、非破坏性的激活引导方法用于 LLM 知识遗忘，在推理时通过满足零空间约束的引导向量重定向隐私相关激活，在 TOFU 和 MUSE 基准上匹配或超越现有基线，同时近乎无损地保留模型效用。
The paper proposes three approaches for few-shot author-style adaptation of LLMs—contrastive activation steering, a steering-vector prediction network, and a hypernetwork that predicts LoRA adapters—and finds the hypernetwork offers the best trade-off between style imitation and output quality for both seen and unseen authors.
The paper proposes label-free bias-only TTRL, which uses majority-vote pseudo-labels as rewards and optimizes only ~100K bias parameters with a frozen backbone, reaching 76.67% on MATH-500 with Qwen2.5-7B and transferring across vision-language and audio reasoning tasks.
Research article exploring the use of Jacobian space to extract steering vectors from concept tokens for steering LLM behaviors, with experiments on Qwen3-1.7B showing promise for simple tasks but limitations in complex scenarios.
This paper introduces a benchmark to validate that distribution-driven methods for activation steering in LLMs recover human value topologies aligned with Schwartz's theory, improving with model size but declining post-instruction tuning.
This paper introduces the Instruction-Compliance Gap and finds behavioral asymmetry in reasoning models' disclosure of hidden directives, using steering vectors to manipulate this behavior.
This paper shows that steering vectors' safety degradation is separable and reducible, proposing a post-hoc correction via constrained optimization that restores model safety while preserving steering effectiveness.
A systematic empirical study showing that concept directions extracted from one language model can steer other independently trained models when sufficient scale (≥1.7B parameters) is reached, providing functional evidence for the Platonic Representation Hypothesis and highlighting scale thresholds for cross-model interpretability tools.
This paper identifies an 'inverted detection-control' phenomenon where some discriminative steering vectors, despite aligning with positive concept representations, consistently promote the opposite behavior. The authors propose a method to detect such inverted steering vectors without generation, enabling sign flips that improve a detection-based steering pipeline across multiple LLMs and concepts.
This paper investigates how well activation steering for improving chain-of-thought faithfulness generalizes across cue types, datasets, and steering vector construction methods across several Gemma and Qwen models. The authors find that when steering is effective, it generalizes broadly, and the evaluation setting matters more than the training setting or vector construction method.
This paper investigates why multimodal LLMs fail on vision-centric tasks when visual evidence conflicts with language priors, introducing the WhatIfVis benchmark and showing that models often encode visual evidence but cannot reliably control their reliance on it.
This paper introduces the Probabilistic Concept-Aware Steering (PCS) framework for LLM inference, which uses concept-driven steering vector retrieval and probabilistic strength calibration to improve interpretability, optimality, and generalizability, achieving over 30% higher direction accuracy and over 89% steering accuracy on multiple datasets.
This blog post extends Anthropic's verbalizable workspace paper by measuring how far steering directions from middle layers reach, when the structure forms during training, whether it transfers between models, and how it scales, all on open models.
This paper systematically studies the limitations of steering vectors for controlled text generation, finding that their effectiveness varies across traits, degrades on task transfer, and suffers from composition tradeoffs.
This paper proposes using steering vectors for control over language model behavior and latent space-based calibrators to assess trustworthiness, aiming to demystify internal representations and build more reliable AI systems.
This paper investigates the geometry of truth in LLM reasoning chains and proposes DynaSteer, a dynamic representation editing framework that uses pattern clustering and Fisher-LDA to steer trajectories towards truth while avoiding noise. Experiments show effectiveness on MATH benchmarks and generalization to coding tasks.
SALSA introduces a lightweight adaptation method for speech-aware LLMs that learns layer-wise steering vectors via supervised objective, achieving significant improvements (up to 46.8% relative) on out-of-domain speech benchmarks, and shows that steering the encoder layers is more effective than modifying the LLM backbone.