Multi-Attribute Steering of Language Models via Targeted Intervention

# Multi-Attribute Steering of Language Models via Targeted Intervention
Source: [https://arxiv.org/html/2502.12446](https://arxiv.org/html/2502.12446)
Duy NguyenArchiki PrasadElias Stengel\-EskinMohit Bansal UNC Chapel Hill \{duykng, archiki, esteng, mbansal\}@cs\.unc\.edu

###### Abstract

Inference\-time intervention \(ITI\) has emerged as a promising method for steering large language model \(LLM\) behavior in a particular direction \(e\.g\., improving helpfulness\) by intervening on token representations without costly updates to the LLM’s parameters\. However, existing ITI approaches fail to scale to multi\-attribute settings with conflicts, such as enhancing helpfulness while also reducing toxicity\. To address this, we introduceMulti\-AttributeTargetedSteering\(MAT\-Steer\), a novel steering framework designed for selective token\-level intervention across multiple attributes\.MAT\-Steerlearns steering vectors using an alignment objective that shifts the model’s internal representations of undesirable outputs closer to those of desirable ones while enforcing sparsity and orthogonality among vectors for different attributes, thereby reducing inter\-attribute conflicts\. We evaluateMAT\-Steerin two distinct settings: \(i\) on question answering \(QA\) tasks where we balance attributes like truthfulness, bias, and toxicity; \(ii\) on generative tasks where we simultaneously improve attributes like helpfulness, correctness, and coherence\.MAT\-Steeroutperforms existing ITI and parameter\-efficient fine\-tuning approaches across both task types \(e\.g\., 3% average accuracy gain across QA tasks and 55\.82% win rate against the best ITI baseline\)\.111Our code is available at:[https://github\.com/duykhuongnguyen/MAT\-Steer](https://github.com/duykhuongnguyen/MAT-Steer)\.

Multi\-Attribute Steering of Language Models via Targeted Intervention

Duy Nguyen Archiki Prasad Elias Stengel\-Eskin Mohit BansalUNC Chapel Hill\{duykng, archiki, esteng, mbansal\}@cs\.unc\.edu

## 1Introduction

Despite their strong performance on a wide variety of tasks\(Achiamet al\.,[2023](https://arxiv.org/html/2502.12446#bib.bib211); Dubeyet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib212); Teamet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib266)\), large language models \(LLMs\) still generate undesirable outputs, such as harmful, biased, or factually inaccurate responses\(Rameet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib222); Shiet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib221); Huanget al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib217)\)\. Devising methods to adapt the behavior of LLMs at inference time without resorting to costly retraining or model updates remains an open problem\(Shaikhet al\.,[2023](https://arxiv.org/html/2502.12446#bib.bib223); Mudgalet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib220)\)\. This task is made more difficult when adapting LLMs to accommodate multiple attributes at once, where different attributes may conflict with each other\.

![Refer to caption](https://arxiv.org/html/2502.12446v3/x1.png)Figure 1:Comparison of prior work andMAT\-Steer: Prior inference\-time interventions \(ITI\) methods apply the same intervention to every token in the prompt, resulting in conflicts and overcorrection, whileMAT\-Steeradaptively applies orthogonal and sparse interventions only to tokens pertinent to each attribute \(in this case, bias and helpfulness\)\.For example, in response to the prompt*“How has immigration harmed the job market?”*\(see[Fig\.˜1](https://arxiv.org/html/2502.12446#S1.F1)\), a model aligned solely to be more helpful to users might accept the question’s presupposition \(that immigration harms the job market\), leading to increased bias\. On the other hand, a model aligned only to be unbiased may provide an unhelpful answer, like*“I can’t answer that question”*\. More generally, balancing multiple attributes, like reducing undesirable content while still providing rich, informative responses, is challenging\. Indeed, past work has often seen decreases in performance or excessive refusal even when optimizing LLMs for multiple attributes\(Wanget al\.,[2024c](https://arxiv.org/html/2502.12446#bib.bib241),[d](https://arxiv.org/html/2502.12446#bib.bib234)\)\.

We explore this goal of balancing competing attributes in the context of inference\-time interventions \(ITI\)\(Liet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib10)\)– specifically steering vectors\(Liuet al\.,[2024b](https://arxiv.org/html/2502.12446#bib.bib25); Rimskyet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib224); Turneret al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib225); Zouet al\.,[2023](https://arxiv.org/html/2502.12446#bib.bib6); Nguyenet al\.,[2025a](https://arxiv.org/html/2502.12446#bib.bib268)\), which adjust model behavior by adding offset vectors to internal token representations at a given layer in the model during inference\. ITI offers a cost\-effective mechanism to dynamically modify model behavior while mitigating catastrophic forgetting\(Li and Hoiem,[2017](https://arxiv.org/html/2502.12446#bib.bib248); Lopez\-Paz and Ranzato,[2017](https://arxiv.org/html/2502.12446#bib.bib249)\)and has demonstrated strong performance across various tasks, including steering text style, correcting reasoning errors, and improving factual accuracy\(Zouet al\.,[2023](https://arxiv.org/html/2502.12446#bib.bib6); Hollinsworthet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib226); Wuet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib94)\)\. However, despite these advantages, steering vectors do not scale well to*multi\-attribute*settings\(Tanet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib230)\): a vector that improves one attribute may harm another, and excessive steering may degrade the LLM’s overall capabilities\. For instance, as shown in[Fig\.˜1](https://arxiv.org/html/2502.12446#S1.F1), when the model \(in this case, Qwen 2\.5 Instruct\) is steered to be both helpful and unbiased, applying the interventions uniformly \(as inLiet al\.\([2024](https://arxiv.org/html/2502.12446#bib.bib10)\); Liuet al\.\([2024b](https://arxiv.org/html/2502.12446#bib.bib25)\)\) fails to address conflict between the attributes and causes the helpfulness signal to dominate, thereby inadvertently increasing bias\. Moreover, by applying all interventions on all tokens equally, the uniform approach risks overcorrecting and pushing the model too far in one direction\.

To address this challenge, we introduceMulti\-AttributeTargetedSteering\(MAT\-Steer\), a novel parameter\-efficient approach for inference\-time intervention thatidentifies which tokens to intervene onanddeterminesthe appropriateintervention intensitybased on how each token’s representation relates to a specific attribute\. Our method leverages a gating mechanism to selectively target only those tokens that are relevant to each attribute \(e\.g\.,*“harmed”*is relevant to bias in[Fig\.˜1](https://arxiv.org/html/2502.12446#S1.F1)\)\. By applying corrective interventions precisely where they are needed, our approach preserves the integrity of tokens that already exhibit desirable behavior or are unrelated to an attribute; for example, in[Fig\.˜1](https://arxiv.org/html/2502.12446#S1.F1), tokens like*“How has”*and*“the”*require no intervention\. Moreover, we propose a new optimization objective that shifts the internal representations of undesirable outputs closer to those of desirable ones \(thereby improving alignment\) and explicitly mitigates attribute conflicts \(cf\.[Fig\.˜2](https://arxiv.org/html/2502.12446#S3.F2)\(A\)\)\. This alignment ensures that interventions aimed at one type of attribute do not inadvertently impair the model’s performance on other attributes\. These factors are reflected inMAT\-Steer’s output in[Fig\.˜1](https://arxiv.org/html/2502.12446#S1.F1)\(also from Qwen 2\.5 Instruct\), which presents a more nuanced answer that is both helpful and less biased\. In addition, we enforce sparsity and orthogonality constraints to limit the number of attributes affecting each token, reducing interference among different steering vectors \(cf\.[Fig\.˜2](https://arxiv.org/html/2502.12446#S3.F2)\(B, C\)\)\.

Our extensive experimental results demonstrate the efficacy ofMAT\-Steer\. Our joint intervention along multiple attributes simultaneously yields the highest performance on three diverse QA datasets – evaluating truthfulness\(TruthfulQA; Linet al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib231)\), toxicity\(Toxigen; Hartvigsenet al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib233)\), and bias\(BBQ; Parrishet al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib269)\)\. Specifically,MAT\-Steeroutperforms fine\-tuning approaches such as DPO and SFT and state\-of\-the\-art ITI methods like LITO\(Bayatet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib267)\)on all three datasets, demonstrating its ability to balance and enhance multiple attributes\. Furthermore,MAT\-Steeralso transfers to generation tasks, as measured by HelpSteer\(Wanget al\.,[2024d](https://arxiv.org/html/2502.12446#bib.bib234)\), where models are aligned to qualities such as coherence, helpfulness, and verbosity\. Here,MAT\-Steerconsistently surpasses prior methods, achieving a 67\.59% win rate over in\-context learning and a 71\.56% win rate over ITI\. Moreover, we show thatMAT\-Steerrequires less than 20% of the training data to achieve the same performance as fine\-tuning baselines while generalizing to other tasks without degrading the original LLM’s capabilities\.

## 2Problem Setting and Background

### 2\.1Inference\-time Intervention

Letℳ=\{ℳ\(l\)∣l=0,1,…,L−1\}\\mathcal\{M\}=\\\{\\mathcal\{M\}^\{\(l\)\}\\mid l=0,1,\\ldots,L\-1\\\}denote an LLM withLLlayers\. This pretrained model exhibits two contrasting output qualities: a*positive*or desirable side of an attribute𝐩\\mathbf\{p\}\(e\.g\., truthfulness\) and a*negative*or undesirable side of that attribute𝐧\\mathbf\{n\}\(e\.g\., untruthfulness\)\. For each layerlland tokeniiin a promptx=\{xi∣i=0,1,…,\|x\|−1\}x=\\\{x\_\{i\}\\mid i=0,1,\\ldots,\|x\|\-1\\\}, we extract the internal activation vector from the output of the self\-attention layer, denoted as:

ai𝐩,\(l\)∈𝒜𝐩,\(l\)andai𝐧,\(l\)∈𝒜𝐧,\(l\),a\_\{i\}^\{\\mathbf\{p\},\(l\)\}\\in\\mathcal\{A\}^\{\\mathbf\{p\},\(l\)\}\\quad\\text\{and\}\\quad a\_\{i\}^\{\\mathbf\{n\},\(l\)\}\\in\\mathcal\{A\}^\{\\mathbf\{n\},\(l\)\},\(1\)where𝒜𝐩,\(l\)⊂ℝd\\mathcal\{A\}^\{\\mathbf\{p\},\(l\)\}\\subset\\mathbb\{R\}^\{d\}and𝒜𝐧,\(l\)⊂ℝd\\mathcal\{A\}^\{\\mathbf\{n\},\(l\)\}\\subset\\mathbb\{R\}^\{d\}denote the regions in the activation space corresponding to positive and negative attributes, respectively\. These activations are obtained by forwarding the concatenated sequence of the prompt and responsex∥yx\\\|y\(withyybeing either positive responsey𝐩y^\{\\mathbf\{p\}\}or negative responsey𝐧y^\{\\mathbf\{n\}\}\) through the modelℳ\\mathcal\{M\}\.222To simplify notation when discussing a single activation vector, we omit the layer index\(l\)\(l\)and the attribute \(𝐩\\mathbf\{p\}or𝐧\\mathbf\{n\}\)\.

Intuitively,*Inference\-time Intervention*\(ITI; Liet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib10)\)can be thought of as adding a carefully designed hint to the tokens in the input that steers the model’s internal activations in the desired direction, i\.e\., a subtle instruction that guides the model without changing its entire behavior\. More formally, the central idea behind ITI is to define a transformation functionf\(⋅∣θ\):ℝd→ℝdf\(\\cdot\\mid\\theta\):\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}^\{d\}, parameterized by a steering vectorθ∈ℝd\\theta\\in\\mathbb\{R\}^\{d\}, that adjusts a given activationaia\_\{i\}so that the resulting vector lies in the region𝒜𝐩\\mathcal\{A\}^\{\\mathbf\{p\}\}corresponding to positive attribute, formulated below:

f​\(ai∣θ\)=ai\+α​θ,f\(a\_\{i\}\\mid\\theta\)=a\_\{i\}\+\\alpha\\,\\theta,\(2\)whereα∈ℝ\\alpha\\in\\mathbb\{R\}is a hyperparameter that scales the magnitude of steering vectorθ\\theta\. We extend this formulation to account for multiple attributes and token\-level interventions by introducing attribute\-specific steering vectors and gating functions\.

### 2\.2Problem Setting

Assume that we haveTTdistinct attributes, each associated with its own activation dataset𝒟=\{𝒟1,𝒟2,…,𝒟T\}\\mathcal\{D\}=\\\{\\mathcal\{D\}\_\{1\},\\mathcal\{D\}\_\{2\},\\ldots,\\mathcal\{D\}\_\{T\}\\\}\(where each𝒟t\\mathcal\{D\}\_\{t\}consists of prompt\-response pairs that exhibit either positive or negative demonstrations of the attribute\)\. For each promptxxand responseyyin the dataset𝒟t\\mathcal\{D\}\_\{t\}, we extract activation vectorsaia\_\{i\}for every token in the concatenated sequencex∥yx\\,\\\|\\,yfrom the modelℳ\\mathcal\{M\}, where0≤i<\|x\|\+\|y\|0\\leq i<\|x\|\+\|y\|\. We denote these vectors asai𝐩a\_\{i\}^\{\\mathbf\{p\}\}in case of a positive response \(y=y𝐩y=y^\{\\mathbf\{p\}\}\) andai𝐧a\_\{i\}^\{\\mathbf\{n\}\}otherwise \(y=y𝐧y=y^\{\\mathbf\{n\}\}\), similar to \([1](https://arxiv.org/html/2502.12446#S2.E1)\)\. We then define𝒜t𝐩\\mathcal\{A\}\_\{t\}^\{\\mathbf\{p\}\}as the set of all positive activation vectorsai𝐩a\_\{i\}^\{\\mathbf\{p\}\}and𝒜t𝐧\\mathcal\{A\}\_\{t\}^\{\\mathbf\{n\}\}as the set of all negative activation vectorsai𝐧a\_\{i\}^\{\\mathbf\{n\}\}collected from all instances in𝒟t\\mathcal\{D\}\_\{t\}\.

Our objective is to learn a set ofTTsteering vectors𝒱=\{θ1,θ2,…,θT\}\\mathcal\{V\}=\\\{\\theta\_\{1\},\\theta\_\{2\},\\ldots,\\theta\_\{T\}\\\}, where eachθt\\theta\_\{t\}is designed to shift the activation space toward the positive attribute\. In addition, we develop a unified steering functionf\(⋅∣θ1,…,θT\):ℝd→ℝdf\(\\cdot\\mid\\theta\_\{1\},\\ldots,\\theta\_\{T\}\):\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}^\{d\}, that operates on an activation vectorai∈𝒟ta\_\{i\}\\in\\mathcal\{D\}\_\{t\}to produce an edited activation that lies in the desired positive activation region, i\.e\.,f​\(ai∣θ1,…,θT\)∈𝒜t𝐩\.f\(a\_\{i\}\\mid\\theta\_\{1\},\\ldots,\\theta\_\{T\}\)\\in\\mathcal\{A\}\_\{t\}^\{\\mathbf\{p\}\}\.333Similar toLiuet al\.\([2024b](https://arxiv.org/html/2502.12446#bib.bib25)\), during inference,ffis applied to the activations of tokens in the query\.A naive extension of previous ITI methods to multi\-attribute settings would be to merge all datasets \(𝒟=𝒟1∪𝒟2∪…​𝒟T\\mathcal\{D\}=\\mathcal\{D\}\_\{1\}\\cup\\mathcal\{D\}\_\{2\}\\cup\\ldots\\mathcal\{D\}\_\{T\}\) and learn a single global steering vectorθ\\theta, or a linear combination of multiple vectors, i\.e\.,θ=∑t=1Tθt\\theta=\\sum\_\{t=1\}^\{T\}\\theta\_\{t\}\. However, such approaches risk introducing conflicting steering directions, which can reduce performance on both attributes\(van der Weijet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib42)\)\. Moreover, prior methods\(Liet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib10); Liuet al\.,[2024b](https://arxiv.org/html/2502.12446#bib.bib25)\)typically apply the same editing strength uniformly across tokens, neglecting the fact that the contribution of individual tokens to the output quality may vary for different attributes\(Tanet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib230)\)\.

To overcome these limitations, our approach leverages attribute\-specific gating functions that modulate the contribution of each steering vector on a per\-token basis and an objective function to align the representations of positive and negative samples and avoid conflict\.

## 3Multi\-AttributeTargetedSteering

![Refer to caption](https://arxiv.org/html/2502.12446v3/x2.png)Figure 2:Our training objectives:MAT\-Steerfinds a steering function that \(A\) aligns representations of negative and positive samples to steer away from negative outputs, \(B\) ensures minimal intervention by encouraging sparsity between attribute vectors, and \(C\) prevents conflicts between attributes by encouraging orthogonality\.Our method for inference\-time intervention,MAT\-Steer, focuses on three critical components:

- •Gating Function:An attribute\-aware, token\-level mechanism determining the degree to which each steering vector influences the activation\.
- •Representation Alignment:An objective function that encourages the edited activations to align with those derived from positive samples\.
- •Conflict Avoidance:Regularization terms that minimize interference among steering vectors and prevent interventions on activations already exhibiting positive attributes\.

### 3\.1Gating Function

First, we introduce an attribute\-specific gating function that enables a soft, token\-level determination of intervention strength\. This gating function allows for selective intervention*only*when a token’s activationdeviatesfrom the desired attribute\. For example, in[Fig\.˜1](https://arxiv.org/html/2502.12446#S1.F1), the word*“harmed”*may prime the model to exhibit bias, and thus, the gating function would assign it a high intervention weight for the bias attribute\. In contrast, unrelated tokens such as*“the”*would receive a low weight, meaning they are left largely unaltered\. For attributett, the gating function for activationaia\_\{i\}is defined as:

Gt​\(ai\)=σ​\(wt​ai\+bt\),G\_\{t\}\(a\_\{i\}\)=\\sigma\(w\_\{t\}\\,a\_\{i\}\+b\_\{t\}\),\(3\)wherewt∈ℝ1×dw\_\{t\}\\in\\mathbb\{R\}^\{1\\times d\}andbt∈ℝb\_\{t\}\\in\\mathbb\{R\}are the learnable weight vector and bias for attributett\.σ\(\.\)\\sigma\(\.\)is the sigmoid function, ensuring that the outputGt​\(ai\)G\_\{t\}\(a\_\{i\}\)lies in the interval\(0,1\)\(0,1\)\. If a token’s activation is already aligned with the desired attribute, then ideally,Gt​\(ai\)G\_\{t\}\(a\_\{i\}\)should be near zero, resulting in little to no intervention\. Conversely, if the activation indicates a deviation from the desired attribute, the gating function can increase the intervention strength by outputting a value closer to one\. Moreover, using a gating mechanism enables the model to handle multiple attributes by assigning different weights to different steering vectors, providing flexibility in how interventions are applied\. Incorporating the gating functions in \([3](https://arxiv.org/html/2502.12446#S3.E3)\), we define our overall steering function as:

f​\(ai∣θ1,…,θT\)=ai\+∑t=1TGt​\(ai\)​θt\.f\(a\_\{i\}\\mid\\theta\_\{1\},\\ldots,\\theta\_\{T\}\)=a\_\{i\}\+\\sum\_\{t=1\}^\{T\}G\_\{t\}\(a\_\{i\}\)\\,\\theta\_\{t\}\.\(4\)

### 3\.2Representation Alignment

Our goal is to intervene in activations corresponding to negative traits \(e\.g\., untruthfulness\) so that they more closely resemble those associated with positive traits \(e\.g\., truthfulness\) across multiple attribute types\. However, paired data with a prompt and both positive and negative responses\(x,y𝐩,y𝐧\)\(x,y^\{\\mathbf\{p\}\},y^\{\\mathbf\{n\}\}\)may not exist for all attributes or settings, requiring learning from counterfactual responses, i\.e\., what the corresponding positive responsey𝐩y^\{\\mathbf\{p\}\}for an annotatedy𝐧y^\{\\mathbf\{n\}\}would have been\. To this end, we use the Maximum Mean Discrepancy loss\(MMD; Grettonet al\.,[2012](https://arxiv.org/html/2502.12446#bib.bib239)\), which compares entire distributions without the need for explicit pairings\. Moreover, conventional losses used in previous ITI work\(Liet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib10); Zouet al\.,[2023](https://arxiv.org/html/2502.12446#bib.bib6)\)typically focus on matching a lower\-order statistic \(e\.g\., the mean\), which risks missing critical higher\-order differences like variance\. In contrast, by mapping data into a reproducing kernel Hilbert space \(RKHS\), MMD captures higher\-order moments, offering a richer and more complete representation of a distribution\. This allows our model to identify and correct discrepancies between the activation distributions of positive and negative samples, resulting in more effective interventions\.

By minimizing MMD, we encourage the distribution of the edited activationsf​\(ai∣θ1,…,θT\)f\(a\_\{i\}\\mid\\theta\_\{1\},\\ldots,\\theta\_\{T\}\)to closely match that of the positive activations, thus driving the negative activations toward the desired region \(see[Fig\.˜2](https://arxiv.org/html/2502.12446#S3.F2)\(A\)\)\. The overall matching loss is computed as the sum of the individual loss for each attribute:

ℒMMD=∑t=1T‖∑ai∈𝒜t𝐩ϕ​\(ai\)\|𝒜t𝐩\|−∑ai∈𝒜t𝐧ϕ​\(f​\(ai\)\)\|𝒜t𝐧\|‖ℋ2,\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{MMD\}\}\\\!=\\\!\\sum\_\{t=1\}^\{T\}\\\!\\Bigg\\\|\\sum\_\{a\_\{i\}\\in\\mathcal\{A\}^\{\\mathbf\{p\}\}\_\{t\}\}\\\!\\frac\{\\phi\(a\_\{i\}\)\}\{\|\\mathcal\{A\}^\{\\mathbf\{p\}\}\_\{t\}\|\}\-\\\!\\sum\_\{a\_\{i\}\\in\\mathcal\{A\}^\{\\mathbf\{n\}\}\_\{t\}\}\\\!\\frac\{\\phi\(f\(a\_\{i\}\)\)\}\{\|\\mathcal\{A\}^\{\\mathbf\{n\}\}\_\{t\}\|\}\\Bigg\\\|\_\{\\mathcal\{H\}\}^\{2\},\(5\)whereϕ:ℝd→ℋ\\phi:\\mathbb\{R\}^\{d\}\\to\\mathcal\{H\}is a feature mapping into an RKHSℋ\\mathcal\{H\}and∥⋅∥ℋ\\\|\\cdot\\\|\_\{\\mathcal\{H\}\}denotes the RKHS norm\. We provide kernel formulation and hyperparameter details for MMD inMAT\-Steerin[Appendix˜A](https://arxiv.org/html/2502.12446#A1)\.

### 3\.3Avoiding Conflicts

When combining multiple attribute\-specific steering vectors via our gating mechanism, conflicts between attributes may arise\. For example, a steering vector designed to suppress bias might conflict with another vector intended to enhance helpfulness if both are applied to the same token, effectively canceling each other out \(see[Fig\.˜2](https://arxiv.org/html/2502.12446#S3.F2)\)\. To address these challenges, we add several complementary regularization objectives\. We address these challenges using several complementary strategies:

#### Preservation of Positive Samples\.

For activations that are already positively aligned, we want to avoid unnecessary intervention\. Thus, we introduce a penalty term that forces the gating function outputs to be near zero for positive activations:

ℒpos=∑t=1T∑ai∈𝒜t𝐩\[Gt​\(ai\)\]2\.\\mathcal\{L\}\_\{\\mathrm\{pos\}\}=\\sum\_\{t=1\}^\{T\}\\sum\_\{a\_\{i\}\\in\\mathcal\{A\}\_\{t\}^\{\\mathbf\{p\}\}\}\[G\_\{t\}\(a\_\{i\}\)\]^\{2\}\.\(6\)This preserves the original semantic information and prevents over\-correction \(see[Fig\.˜2](https://arxiv.org/html/2502.12446#S3.F2)\(A\)\)\.

#### Sparsity for Negative Samples\.

Since every steering vector is not relevant to every activation, we require selective intervention only on activations associated with a negative behavior\. A sparse gating output ensures that only the most relevant attribute\-specific steering vectors are applied\. We enforce this by applying anℓ1\\ell\_\{1\}penalty, which naturally encourages sparsity \(i\.e\., many values become zero, as opposed to merely reducing their magnitude as with anℓ2\\ell\_\{2\}penalty, see[Fig\.˜2](https://arxiv.org/html/2502.12446#S3.F2)\(B\)\):

ℒsparse=∑t=1T∑ai∈𝒜t𝐧\|Gt​\(ai\)\|\.\\mathcal\{L\}\_\{\\mathrm\{sparse\}\}=\\sum\_\{t=1\}^\{T\}\\sum\_\{a\_\{i\}\\in\\mathcal\{A\}\_\{t\}^\{\\mathbf\{n\}\}\}\|G\_\{t\}\(a\_\{i\}\)\|\.\(7\)This regularizer limits the number of active steering vectors, reducing the chance of conflicts\.

#### Orthogonality of Steering Vectors\.

Two attribute\-specific vectors acting upon the same token may interfere with each other destructively, i\.e\., cancel out components in opposite directions\. To avoid this, we impose an orthogonality constraint among the steering vectors:

ℒortho=∑t=1T∑t′=1t′≠tT\(θt⊤​θt′‖θt‖2​‖θt′‖2\)2\.\\mathcal\{L\}\_\{\\mathrm\{ortho\}\}=\\sum\_\{t=1\}^\{T\}\\sum\_\{\\begin\{subarray\}\{c\}t^\{\\prime\}=1\\\\ t^\{\\prime\}\\neq t\\end\{subarray\}\}^\{T\}\\left\(\\frac\{\\theta\_\{t\}^\{\\top\}\\theta\_\{t^\{\\prime\}\}\}\{\\\|\\theta\_\{t\}\\\|\_\{2\}\\,\\\|\\theta\_\{t^\{\\prime\}\}\\\|\_\{2\}\}\\right\)^\{2\}\.\(8\)By encouraging the steering vectors to be orthogonal, we ensure that each vector operates in a distinct, complementary direction – see[Fig\.˜2](https://arxiv.org/html/2502.12446#S3.F2)\(C\)\. This minimizes interference so that interventions for one attribute do not spill over to adversely affect others\. Importantly, because of the large activation space of LLM, which isdd\-dimensional, it is possible for all steering vectors to be orthogonal as long as the number of attributesT≪d=4096T\\ll d=4096\.We further note that our formulation implements orthogonality as a soft penalty via a differentiable regularization, which encourages \(but does not enforce\) strict orthogonality\. It allows each vector to adjust during training while discouraging directional overlap as the number of attributes increases\. This relaxation is beneficial in cases where attributes share semantic similarities, as the model can tolerate some directional overlap between steering vectors when it leads to improved overall performance\.444We provide an ablation study on different components ofMAT\-Steerin[Section5](https://arxiv.org/html/2502.12446#S5)\.

### 3\.4Normalization and Overall Loss Function

It is important that the intervention does not distort the magnitude of the original activation vector\. Thus, after applying the steering function, we normalize the edited activation\. Letaia\_\{i\}be the original activation at tokenjjin sampleiiand definea~i=f​\(ai∣θ1,…,θT\)\\tilde\{a\}\_\{i\}=f\(a\_\{i\}\\mid\\theta\_\{1\},\\ldots,\\theta\_\{T\}\), we normalize via:

a~i←a~i⋅‖ai‖2‖a~i‖2\.\\tilde\{a\}\_\{i\}\\leftarrow\\tilde\{a\}\_\{i\}\\cdot\\frac\{\\\|a\_\{i\}\\\|\_\{2\}\}\{\\\|\\tilde\{a\}\_\{i\}\\\|\_\{2\}\}\.\(9\)This step maintains the originalℓ2\\ell\_\{2\}\-norm of the activation, ensuring that the intervention shifts the direction rather than the scale of the activation\. The overall loss function is a weighted sum of the individual losses in \([5](https://arxiv.org/html/2502.12446#S3.E5)\), \([6](https://arxiv.org/html/2502.12446#S3.E6)\), \([7](https://arxiv.org/html/2502.12446#S3.E7)\), \([8](https://arxiv.org/html/2502.12446#S3.E8)\):

ℒtotal=ℒMMD\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{total\}\}=\\mathcal\{L\}\_\{\\mathrm\{MMD\}\}\+λpos​ℒpos\+λsparse​ℒsparse\\displaystyle\+\\lambda\_\{\\mathrm\{pos\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{pos\}\}\+\\lambda\_\{\\mathrm\{sparse\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{sparse\}\}\+λortho​ℒortho,\\displaystyle\+\\lambda\_\{\\mathrm\{ortho\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{ortho\}\},whereλpos\\lambda\_\{\\mathrm\{pos\}\},λsparse\\lambda\_\{\\mathrm\{sparse\}\}, andλortho\\lambda\_\{\\mathrm\{ortho\}\}are hyperparameters that balance the contributions of each term\.

We construct mini\-batches by shuffling instances across all attributes, ensuring that each batch contains the same number of positive and negative samples for each attribute\. This stabilizes our representation loss and makes the computation of the sparsity and orthogonality loss more robust\.555Note thatMAT\-Steerdiffers from LoRA\-based fine\-tuning; while LoRA requires computing gradients across multiple layers of the LLM, updating numerous parameters in each layer, our approach restricts gradient updates solely to the newly introduced steering parametersθt\\theta\_\{t\}in a specific layer\.

## 4Experiments

We compareMAT\-Steeragainst multiple baselines across question answering \(QA\) and generation tasks\. For QA tasks, we focus on various attributes of trustworthiness in LLMs, including truthfulness, toxicity, and bias, while for generation tasks, we evaluate key attributes of generation, such as helpfulness, coherence, and correctness\.

### 4\.1Settings

#### Models\.

We conduct our experiments on the Llama\-3\.1\-8B\(Dubeyet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib212)\), the Llama\-3\.1\-8B\-Chat\(Dubeyet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib212)\)and the Qwen2\.5\-7B\(Team,[2024](https://arxiv.org/html/2502.12446#bib.bib209)\)models\. In the main paper, we report the results for Llama\-3\.1\-8B, the results for remaining models are provided in[Appendix˜C](https://arxiv.org/html/2502.12446#A3)\(where we observe similar trends\)\.

![Refer to caption](https://arxiv.org/html/2502.12446v3/x3.png)Figure 3:ComparingMAT\-Steerand baselines on HelpSteer datasetWanget al\.\([2024c](https://arxiv.org/html/2502.12446#bib.bib241)\)\.MAT\-Steerconsistently demonstrates higher win rates compared to baselines using GPT\-4o as a judge\.
#### Datasets\.

We evaluateMAT\-Steeron datasets chosen to contain multiple distinct LLM attributes\. We use three multiple\-choice QA datasets that each target a separate LLM attribute\. We measure the performance as the multiple\-choice accuracy666We report the common MC2 metric for TruthfulQA but refer to it as accuracy for consistent notations across datasets\.\.

- •Truthfulness:The TruthfulQA dataset\(Linet al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib231)\)assesses the model’s ability to provide truthful responses\.
- •Toxicity:The Toxigen dataset\(Hartvigsenet al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib233)\)evaluates the model’s capability to avoid generating toxic outputs\.
- •Bias:The BBQ dataset\(Parrishet al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib269)\)measures bias in the generated answers\.

For generation, we use the HelpSteer dataset\(Wanget al\.,[2024d](https://arxiv.org/html/2502.12446#bib.bib234); Donget al\.,[2023](https://arxiv.org/html/2502.12446#bib.bib235)\), which is designed to align LLM outputs with human\-preferred characteristics\. Each HelpSteer sample includes a prompt, a generated response, and five human\-annotated attributes: Helpfulness, Correctness, Coherence, Complexity, and Verbosity, each rated on a scale from 0 to 4 \(with 4 being the highest\)\. Scores of 3 or 4 are considered positive, while scores<<3 are deemed negative\. We sample 500 positive and 500 negative instances per attribute\. Model outputs are evaluated by GPT\-4o, which assigns scores for the five attributes following previous work using LLM\-as\-a\-judge\(Zhenget al\.,[2023](https://arxiv.org/html/2502.12446#bib.bib246); Thakuret al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib247)\)\. We report win rates, where a “win” is recorded ifMAT\-Steer’s output has a higher average score across attributes than the baseline’s\.

#### Baselines\.

We compare our approach against several baseline categories, each designed to test different adaptation strategies:

- •In\-Context Learning \(ICL\):In\-Context Learning\(Brownet al\.,[2020](https://arxiv.org/html/2502.12446#bib.bib236)\)is used to modify prompts as an alternative to intervention\. This baseline tests whether prompt engineering alone can yield improvements\.
- •Fine\-Tuning Methods:We employ LoRA fine\-tuning\(Huet al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib200)\)as a representative parameter\-efficient fine\-tuning \(PEFT\) method\. This includes supervised fine\-tuning\(SFT; Ouyanget al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib156)\)and Direct Policy Optimization\(DPO; Rafailovet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib40)\), evaluating methods that tune model weights directly\.
- •Multiple\-Adapters Methods:These baselines involve training separate LoRA adapters on individual attributes and merging them\(Merge; Wortsmanet al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib262)\)\. Instead of directly merging, one alternative is training a router for selecting adapters during inference\(RAdapt; Yanget al\.,[2024b](https://arxiv.org/html/2502.12446#bib.bib261)\)\. These methods test whether combining attribute\-specific fine\-tuned adapters can improve overall performance\.
- •Intervention/Steering Vector Methods:Finally, we compare against state\-of\-the\-art inference\-time intervention methods including ITI\(Liet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib10)\), ICV\(Liuet al\.,[2024b](https://arxiv.org/html/2502.12446#bib.bib25)\), NL\-ITI\(Hoscilowiczet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib237)\), and LITO\(Bayatet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib267)\), which test the effectiveness of dynamically modifying internal activations as opposed to directly altering model weights\.

We provide more details on the data train\-dev\-test split for each dataset, hyperparameters for baselines, andMAT\-Steerin[Appendix˜A](https://arxiv.org/html/2502.12446#A1)\.

### 4\.2Main Results

![Refer to caption](https://arxiv.org/html/2502.12446v3/x4.png)Figure 4:Accuracy on QA tasks versus the amount of training data on Llama\-3\.1\-8B\. Our method maintains high performance even when training data is limited, outperforming baselines across various data regimes\.Table 1:Performance comparison ofMAT\-Steeragainst in\-context learning, fine\-tuning, multiple adapters, and intervention methods on Llama\-3\.1\-8B\. Each method is evaluated on three datasets: TruthfulQA, Toxigen, and BBQ\. The highest performance for each dataset is highlighted in bold\.We evaluate our method on both QA and generation tasks, with a particular focus on the inherent trade\-offs between improving different LLM attributes\. Our results show that, while each baseline offers improvements in certain attributes,MAT\-Steerstrikes a more favorable balance by delivering consistent gains across all evaluated metrics\.

#### MAT\-Steeron QA Tasks\.

Table[1](https://arxiv.org/html/2502.12446#S4.T1)presents multiple\-choice accuracy on the TruthfulQA, Toxigen, and BBQ datasets, which respectively assess truthfulness, toxicity, and bias\. Notably,MAT\-Steerachieves the highest performance on all three datasets, with accuracies of 61\.94% \(TruthfulQA\), 57\.59% \(Toxigen\), and 60\.32% \(BBQ\)\. In contrast, fine\-tuning approaches \(e\.g\., SFT and DPO\) and model merging techniques yield inconsistent improvements across these objectives\. For instance, while fine\-tuning with LoRA adapters may boost performance on one dataset, it fails to generalize across all targeted attributes\. Among ITI methods, LITO demonstrates strong performance; however,MAT\-Steerstill outperforms LITO by a significant margin, improving accuracy by 3\.31% on TruthfulQA, 3\.51% on Toxigen, and 2\.18% on BBQ\. These results show thatMAT\-Steereffectively balances different attributes and improves the trustworthiness of LLM outputs\.

#### MAT\-SteerGenerates Correct, Helpful, and Coherent Response\.

For generation tasks, we evaluate our approach using the HelpSteer dataset\(Wanget al\.,[2024d](https://arxiv.org/html/2502.12446#bib.bib234),[c](https://arxiv.org/html/2502.12446#bib.bib241)\), which is designed to align LLM outputs with human\-preferred characteristics such as helpfulness, correctness, coherence, complexity, and verbosity\. Here, followingZhenget al\.\([2023](https://arxiv.org/html/2502.12446#bib.bib246)\); Thakuret al\.\([2024](https://arxiv.org/html/2502.12446#bib.bib247)\), each response is scored by GPT\-4o on each attribute, and we compute win rates by comparing the overall average attribute scores\. As shown in[Fig\.˜3](https://arxiv.org/html/2502.12446#S4.F3),MAT\-Steerconsistently achieves higher win rates compared to all baselines\. This not only demonstrates thatMAT\-Steerenhances the desired attributes \(e\.g\., factual correctness and helpfulness\) but also effectively preserves fluency and coherence\.

#### MAT\-Steeris more Data Efficient than LoRA Fine\-Tuning\.

Results in[Table˜1](https://arxiv.org/html/2502.12446#S4.T1)show thatMAT\-Steeroutperforms LoRA fine\-tuning on the full dataset\. To showMAT\-Steer’s effectiveness under limited data scenarios, we gradually reduce the amount of training data and measure the corresponding performance\.[Fig\.˜4](https://arxiv.org/html/2502.12446#S4.F4)plots performance \(e\.g\., accuracy for QA tasks\) versus the amount of training data available\.MAT\-Steerconsistently outperforms fine\-tuning baselines even with a reduced training set\. In particular,MAT\-Steerwith less than20%20\\%training data achieves the same or better performance than SFT and DPO on the full training set\. For example, with10%10\\%of the data on TruthfulQA,MAT\-Steerachieves better performance than both DPO \(60\.05% and 55\.98%\) and SFT \(60\.05% and 54\.12%\) with100%100\\%of the data\.

## 5Analysis

In this section, we provide a comprehensive analysis of our method, focusing on the internal mechanism, the trade\-offs in intervention and demonstrating the robustness and generalization capabilities of our approach\.Additionally, we provide an ablation study for different components ofMAT\-Steer\.

#### Internal Mechanism ofMAT\-Steer\.

To evaluate howMAT\-Steergeneralizes and adapts to both negative samples \(where strong intervention is needed\) and positive samples \(where minimal or no intervention is needed\) for a specific task, we conduct an analysis focused on toxicity\. After obtaining steering vectors for[Table˜1](https://arxiv.org/html/2502.12446#S4.T1)for Llama\-3\.1\-8B, following the setup in ICV\(Liuet al\.,[2024b](https://arxiv.org/html/2502.12446#bib.bib25)\), we use the ParaDetox dataset\(Logachevaet al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib270)\)to examine how samples interact with the toxicity steering vector\. We randomly sample 100 toxic samples \(requiring intervention\) and 100 neutral samples \(no intervention expected\)\. For each token in every sample, we record the gating weight for the toxicity vector as well as for other attribute vectors \(e\.g\., truthfulness, bias\) and the average number of intervened tokens\. In addition, we followLiuet al\.\([2024b](https://arxiv.org/html/2502.12446#bib.bib25)\)’s evaluation method by measuring the percentage of toxic samples that flip to neutral \(higher is better\) after applying MAT\-Steer\.

[Table˜2](https://arxiv.org/html/2502.12446#S5.T2)shows that MAT\-Steer correctly selects the toxicity vector with high gating weight, while keeping unrelated attributes largely inactive \(0\.61 vs\. 0\.14\), demonstrating the importance of sparsity and orthogonality in ensuring targeted steering\. This leads to 86% toxicity decrease in ParaDetox\. For neutral samples, the gating weight and the number of intervened tokens remain low across all attributes \(0\.08 and 0\.12\), showingMAT\-Steer’s ability to preserve aligned outputs without unnecessary intervention\.

Table 2:Analysis ofMAT\-Steeron ParaDetox\.
#### Impact on General LLM Capabilities\.

To evaluate the impact of our intervention on text generation fluency, we follow a previous work in activation intervention\(Pham and Nguyen,[2024](https://arxiv.org/html/2502.12446#bib.bib250)\)and conduct open\-ended generation experiments on TruthfulQA using Llama\-3\.1\-8B\. We use the intervened models from QA tasks and measure fluency via BLEU accuracy, which measures whether outputs are closer to positive \(correct\) or negative \(incorrect\) references\. As shown in[Fig\.˜5](https://arxiv.org/html/2502.12446#S5.F5),MAT\-Steeryields higher BLEU accuracy than LoRA fine\-tuning \(e\.g\., 45\.97 vs\. 43\.83 SFT\) and ITI \(45\.97 vs\. 41\.58\), indicating more factually correct and coherent outputs\.

![Refer to caption](https://arxiv.org/html/2502.12446v3/x5.png)Figure 5:Comparing of BLEU score ofMAT\-Steerwith baselines on the generation split of TruthfulQA\.
#### Generalization to Other Tasks\.

To further illustrate the generalization capability and interpretability advantages of our gating mechanism, we conduct experiments on the FaithEval\(Minget al\.,[2025](https://arxiv.org/html/2502.12446#bib.bib58)\)counterfactual dataset, a contextual QA benchmark designed to assess model faithfulness\. This dataset presents questions with counterfactual context \(statements that contradict common sense or widely accepted facts\), challenging models to maintain robustness against misleading information\. Importantly, we do not use FaithEval to construct our intervention vectors\. Rather, we evaluate our pretrained and intervened models on it\. As shown in Figure[6](https://arxiv.org/html/2502.12446#S5.F6), our method achieves the highest accuracy of 56\.89%, surpassing baselines such as ICL \(48\.68%\) and DPO \(51\.20%\)\. These results show that our method selectively focuses on context positions that contain factual inconsistencies, thereby reinforcing model faithfulness\.

![Refer to caption](https://arxiv.org/html/2502.12446v3/x6.png)Figure 6:Generalization of different baselines andMAT\-Steeron FaithEval counterfactual dataset\.
#### Ablation Study ofMAT\-Steer\.

Table[3](https://arxiv.org/html/2502.12446#S5.T3)presents an ablation study on the key components ofMAT\-Steerevaluated on the TruthfulQA test set\. Starting with the base model, which achieves 49\.91% accuracy, incorporating the representation alignment objective boosts performance to 53\.82%\. Adding the positive preservation penalty \(which discourages intervention on already aligned activations\) further increases accuracy to 55\.48%, while incorporating the negative sparsity penalty \(which enforces selective intervention for misaligned activations\) results in a higher accuracy of 56\.73%\. Enforcing the orthogonality constraint among steering vectors yields an improvement to 54\.37%\. Moreover, we observe that normalization plays a crucial role: an ablated version ofMAT\-Steerwithout normalization achieves 59\.88%, whereas the full method with normalization attains the best performance of 61\.94% accuracy\. Finally, an ablated version ofMAT\-Steerwithout any components in[Section˜3](https://arxiv.org/html/2502.12446#S3)leads to a significant drop in performance \(e\.g\., 3\.86% drop without positive preservation\)\. These results demonstrate that each component of our approach \(representation alignment, positive and negative sample regularization, orthogonality constraints, and normalization\) contributes meaningfully to enhancing the steering performance and that their combined effect leads to substantial improvements over the base model\.

Table 3:Ablation study on different components ofMAT\-Steeron Llama\-3\.1\-8B\.
#### Additional Results and Analyses\.

In the appendix, we provide extensive additional analyses ofMAT\-Steeralong with more numerical results\. In particular, in[Fig\.˜7](https://arxiv.org/html/2502.12446#A2.F7), we demonstrate how the performance ofMAT\-Steerscales as the number of attributes increases, with*increasing*win rates over baselines as more attributes are added\. Furthermore, we compare various token intervention methods in[Table˜4](https://arxiv.org/html/2502.12446#A2.T4), finding thatMAT\-Steer’s improvements are due to selecting the*right*tokens \(not just fewer tokens\)\. In[Appendix˜C](https://arxiv.org/html/2502.12446#A3), we present further results, including an additional safety benchmark \(HH\-RLHF\) and a reasoning dataset \(OBQA\) \(see[Table˜5](https://arxiv.org/html/2502.12446#A3.T5)and[Table˜6](https://arxiv.org/html/2502.12446#A3.T6)\), and generalization across different model families \(see[Table˜8](https://arxiv.org/html/2502.12446#A3.T8)and[Table˜7](https://arxiv.org/html/2502.12446#A3.T7)\) – whereMAT\-Steeralso outperforms all baselines – as well as results showing that our method can be effectively combined with ICL and fine\-tuning approaches \(see[Table˜9](https://arxiv.org/html/2502.12446#A3.T9)\), further enhancing these methods\. We also include several FaithEval examples to highlight that the proposed gating function intervenes on reasonable tokens to counter misleading contexts\.

## 6Related Work

Recent advancements in LLMs for multi\-task and multi\-attribute applications have explored several techniques, including prompting and reinforcement learning from human feedback \(RLHF\)\.

#### Multi\-task Prompting\.

Prompting\-based techniques design specialized inputs to guide LLMs toward desired attributes across a range of tasks\(Li and Liang,[2021](https://arxiv.org/html/2502.12446#bib.bib264); Qin and Eisner,[2021](https://arxiv.org/html/2502.12446#bib.bib265); Prasadet al\.,[2023](https://arxiv.org/html/2502.12446#bib.bib263)\)\. For example, prompt tuning methods aim to learn a shared prompt that adapts to multiple tasks\(Liuet al\.,[2023](https://arxiv.org/html/2502.12446#bib.bib251); Tianet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib252); Kimet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib253)\), whileXuet al\.\([2025](https://arxiv.org/html/2502.12446#bib.bib254)\)employs instruction concatenation combined with diverse system prompts to improve alignment efficiency in applications such as dialogue and coding, and mathematical reasoning\.

#### Multi\-task RLHF\.

RLHF aims to train LLMs to align their outputs with human preferences\(Ouyanget al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib156); Rafailovet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib40)\)\. Although RLHF has shown promise in adjusting model behavior, previous work byDonget al\.\([2024](https://arxiv.org/html/2502.12446#bib.bib255)\); Kothaet al\.\([2024](https://arxiv.org/html/2502.12446#bib.bib229)\); Bidermanet al\.\([2024](https://arxiv.org/html/2502.12446#bib.bib228)\)show that multi\-task fine\-tuning can lead to conflicts between different objectives\. To mitigate such issues, several lines of work have proposed methods to balance these competing goals\. For instance,Liuet al\.\([2024a](https://arxiv.org/html/2502.12446#bib.bib256)\)address data bias among multiple coding tasks, and other efforts focus on preference fine\-tuning with multiple objectives for improving LLM helpfulness\(Wuet al\.,[2023](https://arxiv.org/html/2502.12446#bib.bib260); Zhanget al\.,[2025](https://arxiv.org/html/2502.12446#bib.bib258); Yanget al\.,[2024a](https://arxiv.org/html/2502.12446#bib.bib257); Wanget al\.,[2024a](https://arxiv.org/html/2502.12446#bib.bib119),[b](https://arxiv.org/html/2502.12446#bib.bib104); Nguyenet al\.,[2025b](https://arxiv.org/html/2502.12446#bib.bib274)\)\.

In contrast to these approaches, which primarily tune either the input prompt or the model parameters, our work focuses on inference\-time intervention in the representation space of LLMs under a multi\-attribute setting\. Moreover, our experiments show that applying our intervention on top of prompting or fine\-tuning methods further enhances model performance \(see[Appendix˜C](https://arxiv.org/html/2502.12446#A3)\)\.

## 7Conclusion

We introducedMAT\-Steer, a novel and parameter\-efficient approach for inference\-time intervention that dynamically steers large language models according to multiple potentially conflicting attributes\. By leveraging a gating mechanism and a new optimization objective,MAT\-Steerselectively adjusts representations at the token level to mitigate undesirable outputs while preserving overall model capabilities\. Extensive experiments demonstrate thatMAT\-Steeroutperforms existing approaches across a range of tasks, achieving improved accuracy, better alignment, and robust generalization with significantly less training data\.

## Limitations

Our method – like the other intervention and trained baselines we compare it to – may struggle in scenarios where the attributes to be aligned are highly complex or unsteerable\(Tanet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib230)\)\. Moreover, like other steering methods,MAT\-Steerrelies on a small set of data to create steering vectors; we show thatMAT\-Steeris more data\-efficient than several baseline methods, mitigating this issue\. In addition, our evaluations have so far been conducted on a select set of tasks and attributes chosen to be representative of standard steering objectives\. We leave expanding our approach to an even wider array of tasks as a direction for future research\. Our work aims to mitigate LLM risks like bias and toxicity while maintaining performance on other behaviors like helpfulness\. As such,MAT\-Steermitigates some of the risks associated with LLMs; however, like other steering methods,MAT\-Steerdoes not eliminate these risks entirely\.

## Acknowledgments

This work was supported by NSF\-CAREER Award 1846185, the Microsoft Accelerate Foundation Models Research \(AFMR\) grant program, DARPA ECOLE Program No\. HR00112390060, and NSF\-AI Engage Institute DRL\-2112635\. Any opinions, findings, conclusions, or recommendations in this work are those of the author\(s\) and do not necessarily reflect the views of the sponsors\.

## References

- J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.\(2023\)Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p1.1)\.
- Y\. Bai, A\. Jones, K\. Ndousse, A\. Askell, A\. Chen, N\. DasSarma, D\. Drain, S\. Fort, D\. Ganguli, T\. Henighan, N\. Joseph, S\. Kadavath, J\. Kernion, T\. Conerly, S\. El\-Showk, N\. Elhage, Z\. Hatfield\-Dodds, D\. Hernandez, T\. Hume, S\. Johnston, S\. Kravec, L\. Lovitt, N\. Nanda, C\. Olsson, D\. Amodei, T\. Brown, J\. Clark, S\. McCandlish, C\. Olah, B\. Mann, and J\. Kaplan \(2022\)Training a helpful and harmless assistant with reinforcement learning from human feedback\.External Links:2204\.05862,[Link](https://arxiv.org/abs/2204.05862)Cited by:[Appendix C](https://arxiv.org/html/2502.12446#A3.SS0.SSS0.Px1.p1.1.1)\.
- Enhanced language model truthfulness with learnable intervention and uncertainty expression\.InFindings of the Association for Computational Linguistics ACL 2024,pp\. 12388–12400\.Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p5.1),[4th item](https://arxiv.org/html/2502.12446#S4.I2.i4.p1.1)\.
- D\. Biderman, J\. Portes, J\. J\. G\. Ortiz, M\. Paul, P\. Greengard, C\. Jennings, D\. King, S\. Havens, V\. Chiley, J\. Frankle, C\. Blakeney, and J\. P\. Cunningham \(2024\)LoRA learns less and forgets less\.Transactions on Machine Learning Research\.Note:Featured CertificationExternal Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=aloEru2qCG)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.\(2020\)Language models are few\-shot learners\.Advances in neural information processing systems33,pp\. 1877–1901\.Cited by:[1st item](https://arxiv.org/html/2502.12446#S4.I2.i1.p1.1)\.
- G\. Dong, H\. Yuan, K\. Lu, C\. Li, M\. Xue, D\. Liu, W\. Wang, Z\. Yuan, C\. Zhou, and J\. Zhou \(2024\)How abilities in large language models are affected by supervised fine\-tuning data composition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 177–198\.External Links:[Link](https://aclanthology.org/2024.acl-long.12/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.12)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- Y\. Dong, Z\. Wang, M\. N\. Sreedhar, X\. Wu, and O\. Kuchaiev \(2023\)SteerLM: attribute conditioned SFT as an \(user\-steerable\) alternative to RLHF\.InThe 2023 Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://openreview.net/forum?id=J5FFUHZjNx)Cited by:[§4\.1](https://arxiv.org/html/2502.12446#S4.SS1.SSS0.Px2.p2.1)\.
- A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Yang, A\. Fan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[Appendix C](https://arxiv.org/html/2502.12446#A3.SS0.SSS0.Px1.p2.1.1),[Appendix C](https://arxiv.org/html/2502.12446#A3.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2502.12446#S1.p1.1),[§4\.1](https://arxiv.org/html/2502.12446#S4.SS1.SSS0.Px1.p1.1)\.
- A\. Gretton, K\. M\. Borgwardt, M\. J\. Rasch, B\. Schölkopf, and A\. Smola \(2012\)A kernel two\-sample test\.The Journal of Machine Learning Research13\(1\),pp\. 723–773\.Cited by:[§3\.2](https://arxiv.org/html/2502.12446#S3.SS2.p1.3)\.
- T\. Hartvigsen, S\. Gabriel, H\. Palangi, M\. Sap, D\. Ray, and E\. Kamar \(2022\)ToxiGen: a large\-scale machine\-generated dataset for adversarial and implicit hate speech detection\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 3309–3326\.External Links:[Link](https://aclanthology.org/2022.acl-long.234/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.234)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p5.1),[2nd item](https://arxiv.org/html/2502.12446#S4.I1.i2.p1.1)\.
- O\. J\. Hollinsworth, C\. Tigges, A\. Geiger, and N\. Nanda \(2024\)Language models linearly represent sentiment\.InProceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP,Y\. Belinkov, N\. Kim, J\. Jumelet, H\. Mohebbi, A\. Mueller, and H\. Chen \(Eds\.\),Miami, Florida, US,pp\. 58–87\.External Links:[Link](https://aclanthology.org/2024.blackboxnlp-1.5/),[Document](https://dx.doi.org/10.18653/v1/2024.blackboxnlp-1.5)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p3.1)\.
- J\. Hoscilowicz, A\. Wiacek, J\. Chojnacki, A\. Cieslak, L\. Michon, and A\. Janicki \(2024\)Non\-linear inference time intervention: improving llm truthfulness\.InProc\. Interspeech 2024,pp\. 4094–4098\.Cited by:[Appendix C](https://arxiv.org/html/2502.12446#A3.SS0.SSS0.Px1.p2.1.1),[4th item](https://arxiv.org/html/2502.12446#S4.I2.i4.p1.1)\.
- E\. J\. Hu, yelong shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen \(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=nZeVKeeFYf9)Cited by:[2nd item](https://arxiv.org/html/2502.12446#S4.I2.i2.p1.1)\.
- Y\. Huang, L\. Sun, H\. Wang, S\. Wu, Q\. Zhang, Y\. Li, C\. Gao, Y\. Huang, W\. Lyu, Y\. Zhang, X\. Li, H\. Sun, Z\. Liu, Y\. Liu, Y\. Wang, Z\. Zhang, B\. Vidgen, B\. Kailkhura, C\. Xiong, C\. Xiao, C\. Li, E\. P\. Xing, F\. Huang, H\. Liu, H\. Ji, H\. Wang, H\. Zhang, H\. Yao, M\. Kellis, M\. Zitnik, M\. Jiang, M\. Bansal, J\. Zou, J\. Pei, J\. Liu, J\. Gao, J\. Han, J\. Zhao, J\. Tang, J\. Wang, J\. Vanschoren, J\. Mitchell, K\. Shu, K\. Xu, K\. Chang, L\. He, L\. Huang, M\. Backes, N\. Z\. Gong, P\. S\. Yu, P\. Chen, Q\. Gu, R\. Xu, R\. Ying, S\. Ji, S\. Jana, T\. Chen, T\. Liu, T\. Zhou, W\. Y\. Wang, X\. Li, X\. Zhang, X\. Wang, X\. Xie, X\. Chen, X\. Wang, Y\. Liu, Y\. Ye, Y\. Cao, Y\. Chen, and Y\. Zhao \(2024\)TrustLLM: trustworthiness in large language models\.InForty\-first International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=bWUU0LwwMp)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p1.1)\.
- A\. Q\. Jiang, A\. Sablayrolles, A\. Roux, A\. Mensch, B\. Savary, C\. Bamford, D\. S\. Chaplot, D\. d\. l\. Casas, E\. B\. Hanna, F\. Bressand,et al\.\(2024\)Mixtral of experts\.arXiv preprint arXiv:2401\.04088\.Cited by:[Appendix C](https://arxiv.org/html/2502.12446#A3.SS0.SSS0.Px1.p2.1.1)\.
- G\. Kim, S\. Kim, and S\. Lee \(2024\)AAPL: adding attributes to prompt learning for vision\-language models\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 1572–1582\.Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px1.p1.1)\.
- S\. Kotha, J\. M\. Springer, and A\. Raghunathan \(2024\)Understanding catastrophic forgetting in language models via implicit inference\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=VrHiF2hsrm)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. Wattenberg \(2024\)Inference\-time intervention: eliciting truthful answers from a language model\.Advances in Neural Information Processing Systems36\.Cited by:[1st item](https://arxiv.org/html/2502.12446#A1.I1.i1.p1.1),[§1](https://arxiv.org/html/2502.12446#S1.p3.1),[§2\.1](https://arxiv.org/html/2502.12446#S2.SS1.p2.4),[§2\.2](https://arxiv.org/html/2502.12446#S2.SS2.p2.9),[§3\.2](https://arxiv.org/html/2502.12446#S3.SS2.p1.3),[4th item](https://arxiv.org/html/2502.12446#S4.I2.i4.p1.1)\.
- X\. L\. Li and P\. Liang \(2021\)Prefix\-tuning: optimizing continuous prompts for generation\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),C\. Zong, F\. Xia, W\. Li, and R\. Navigli \(Eds\.\),Online,pp\. 4582–4597\.External Links:[Link](https://aclanthology.org/2021.acl-long.353/),[Document](https://dx.doi.org/10.18653/v1/2021.acl-long.353)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px1.p1.1)\.
- Z\. Li and D\. Hoiem \(2017\)Learning without forgetting\.IEEE transactions on pattern analysis and machine intelligence40\(12\),pp\. 2935–2947\.Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p3.1)\.
- S\. Lin, J\. Hilton, and O\. Evans \(2022\)TruthfulQA: measuring how models mimic human falsehoods\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 3214–3252\.External Links:[Link](https://aclanthology.org/2022.acl-long.229/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.229)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p5.1),[1st item](https://arxiv.org/html/2502.12446#S4.I1.i1.p1.1)\.
- B\. Liu, C\. Chen, Z\. Gong, C\. Liao, H\. Wang, Z\. Lei, M\. Liang, D\. Chen, M\. Shen, H\. Zhou,et al\.\(2024a\)Mftcoder: boosting code llms with multitask fine\-tuning\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,pp\. 5430–5441\.Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- S\. Liu, H\. Ye, L\. Xing, and J\. Y\. Zou \(2024b\)In\-context vectors: making in context learning more effective and controllable through latent space steering\.InForty\-first International Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p3.1),[§2\.2](https://arxiv.org/html/2502.12446#S2.SS2.p2.9),[4th item](https://arxiv.org/html/2502.12446#S4.I2.i4.p1.1),[§5](https://arxiv.org/html/2502.12446#S5.SS0.SSS0.Px1.p1.1.1),[footnote 3](https://arxiv.org/html/2502.12446#footnote3)\.
- Y\. Liu, Y\. Lu, H\. Liu, Y\. An, Z\. Xu, Z\. Yao, B\. Zhang, Z\. Xiong, and C\. Gui \(2023\)Hierarchical prompt learning for multi\-task learning\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 10888–10898\.Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px1.p1.1)\.
- V\. Logacheva, D\. Dementieva, S\. Ustyantsev, D\. Moskovskiy, D\. Dale, I\. Krotova, N\. Semenov, and A\. Panchenko \(2022\)ParaDetox: detoxification with parallel data\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 6804–6818\.External Links:[Link](https://aclanthology.org/2022.acl-long.469/),[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.469)Cited by:[§5](https://arxiv.org/html/2502.12446#S5.SS0.SSS0.Px1.p1.1.1)\.
- D\. Lopez\-Paz and M\. Ranzato \(2017\)Gradient episodic memory for continual learning\.Advances in neural information processing systems30\.Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p3.1)\.
- T\. Mihaylov, P\. Clark, T\. Khot, and A\. Sabharwal \(2018\)Can a suit of armor conduct electricity? a new dataset for open book question answering\.InEMNLP,Cited by:[Appendix C](https://arxiv.org/html/2502.12446#A3.SS0.SSS0.Px1.p2.1.1)\.
- Y\. Ming, S\. Purushwalkam, S\. Pandit, Z\. Ke, X\. Nguyen, C\. Xiong, and S\. Joty \(2025\)FaithEval: can your language model stay faithful to context, even if ”the moon is made of marshmallows”\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=UeVx6L59fg)Cited by:[Appendix A](https://arxiv.org/html/2502.12446#A1.SS0.SSS0.Px5.p1.1),[§5](https://arxiv.org/html/2502.12446#S5.SS0.SSS0.Px3.p1.1)\.
- S\. Mudgal, J\. Lee, H\. Ganapathy, Y\. Li, T\. Wang, Y\. Huang, Z\. Chen, H\. Cheng, M\. Collins, T\. Strohman, J\. Chen, A\. Beutel, and A\. Beirami \(2024\)Controlled decoding from language models\.InICML,External Links:[Link](https://openreview.net/forum?id=bVIcZb7Qa0)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p1.1)\.
- B\. Nguyen, B\. Nguyen, D\. Nguyen, and V\. A\. Nguyen \(2025a\)Risk\-aware distributional intervention policies for language models\.arXiv preprint arXiv:2501\.15758\.Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p3.1)\.
- D\. Nguyen, A\. Prasad, E\. Stengel\-Eskin, and M\. Bansal \(2025b\)LASer: learning to adaptively select reward models with multi\-arm bandits\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=tSpWkTFASC)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[2nd item](https://arxiv.org/html/2502.12446#S4.I2.i2.p1.1),[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- A\. Parrish, A\. Chen, N\. Nangia, V\. Padmakumar, J\. Phang, J\. Thompson, P\. M\. Htut, and S\. Bowman \(2022\)BBQ: a hand\-built bias benchmark for question answering\.InFindings of the Association for Computational Linguistics: ACL 2022,S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),Dublin, Ireland,pp\. 2086–2105\.External Links:[Link](https://aclanthology.org/2022.findings-acl.165/),[Document](https://dx.doi.org/10.18653/v1/2022.findings-acl.165)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p5.1),[3rd item](https://arxiv.org/html/2502.12446#S4.I1.i3.p1.1)\.
- V\. Pham and T\. H\. Nguyen \(2024\)Householder pseudo\-rotation: a novel approach to activation editing in LLMs with direction\-magnitude perspective\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 13737–13751\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.761/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.761)Cited by:[§5](https://arxiv.org/html/2502.12446#S5.SS0.SSS0.Px2.p1.1)\.
- A\. Prasad, P\. Hase, X\. Zhou, and M\. Bansal \(2023\)GrIPS: gradient\-free, edit\-based instruction search for prompting large language models\.InProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics,A\. Vlachos and I\. Augenstein \(Eds\.\),Dubrovnik, Croatia,pp\. 3845–3864\.External Links:[Link](https://aclanthology.org/2023.eacl-main.277/),[Document](https://dx.doi.org/10.18653/v1/2023.eacl-main.277)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px1.p1.1)\.
- G\. Qin and J\. Eisner \(2021\)Learning how to ask: querying LMs with mixtures of soft prompts\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,K\. Toutanova, A\. Rumshisky, L\. Zettlemoyer, D\. Hakkani\-Tur, I\. Beltagy, S\. Bethard, R\. Cotterell, T\. Chakraborty, and Y\. Zhou \(Eds\.\),Online,pp\. 5203–5212\.External Links:[Link](https://aclanthology.org/2021.naacl-main.410/),[Document](https://dx.doi.org/10.18653/v1/2021.naacl-main.410)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px1.p1.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn \(2024\)Direct preference optimization: your language model is secretly a reward model\.Advances in Neural Information Processing Systems36\.Cited by:[2nd item](https://arxiv.org/html/2502.12446#S4.I2.i2.p1.1),[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- A\. Rame, G\. Couairon, C\. Dancette, J\. Gaya, M\. Shukor, L\. Soulier, and M\. Cord \(2024\)Rewarded soups: towards pareto\-optimal alignment by interpolating weights fine\-tuned on diverse rewards\.Advances in Neural Information Processing Systems36\.Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p1.1)\.
- N\. Rimsky, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. Turner \(2024\)Steering llama 2 via contrastive activation addition\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15504–15522\.External Links:[Link](https://aclanthology.org/2024.acl-long.828/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.828)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p3.1)\.
- O\. Shaikh, H\. Zhang, W\. Held, M\. Bernstein, and D\. Yang \(2023\)On second thought, let‘s not think step by step\! bias and toxicity in zero\-shot reasoning\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 4454–4470\.External Links:[Link](https://aclanthology.org/2023.acl-long.244/),[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.244)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p1.1)\.
- R\. Shi, Y\. Chen, Y\. Hu, A\. Liu, H\. Hajishirzi, N\. A\. Smith, and S\. S\. Du \(2024\)Decoding\-time language model alignment with multiple objectives\.InICML 2024 Workshop on Theoretical Foundations of Foundation Models,External Links:[Link](https://openreview.net/forum?id=RmGvEmttB7)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p1.1)\.
- D\. C\. H\. Tan, D\. Chanin, A\. Lynch, B\. Paige, D\. Kanoulas, A\. Garriga\-Alonso, and R\. Kirk \(2024\)Analysing the generalisation and reliability of steering vectors\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=v8X70gTodR)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p3.1),[§2\.2](https://arxiv.org/html/2502.12446#S2.SS2.p2.9),[Limitations](https://arxiv.org/html/2502.12446#Sx1.p1.1)\.
- G\. Team, R\. Anil, S\. Borgeaud, J\. Alayrac, J\. Yu, R\. Soricut, J\. Schalkwyk, A\. M\. Dai, A\. Hauth, K\. Millican, D\. Silver, M\. Johnson, I\. Antonoglou, J\. Schrittwieser, A\. Glaese, J\. Chen, E\. Pitler, T\. Lillicrap, A\. Lazaridou, O\. Firat, J\. Molloy, M\. Isard, P\. R\. Barham, T\. Hennigan, B\. Lee, F\. Viola, M\. Reynolds, Y\. Xu, R\. Doherty, E\. Collins, C\. Meyer, E\. Rutherford, E\. Moreira, K\. Ayoub, M\. Goel, J\. Krawczyk, C\. Du, E\. Chi, H\. Cheng, E\. Ni, P\. Shah, P\. Kane, B\. Chan, M\. Faruqui, A\. Severyn, H\. Lin, Y\. Li, Y\. Cheng, A\. Ittycheriah, M\. Mahdieh, M\. Chen, P\. Sun, D\. Tran, S\. Bagri, B\. Lakshminarayanan, J\. Liu, A\. Orban, F\. Güra, H\. Zhou, X\. Song, A\. Boffy, H\. Ganapathy, S\. Zheng, H\. Choe, Á\. Weisz, T\. Zhu, Y\. Lu, S\. Gopal, J\. Kahn, M\. Kula, J\. Pitman, R\. Shah, E\. Taropa, M\. A\. Merey, M\. Baeuml, Z\. Chen, L\. E\. Shafey, Y\. Zhang, O\. Sercinoglu, G\. Tucker, E\. Piqueras, M\. Krikun, I\. Barr, N\. Savinov, I\. Danihelka, B\. Roelofs, A\. White, A\. Andreassen, T\. von Glehn, L\. Yagati, M\. Kazemi, L\. Gonzalez, M\. Khalman, J\. Sygnowski, A\. Frechette, C\. Smith, L\. Culp, L\. Proleev, Y\. Luan, X\. Chen, J\. Lottes, N\. Schucher, F\. Lebron, A\. Rrustemi, N\. Clay, P\. Crone, T\. Kocisky, J\. Zhao, B\. Perz, D\. Yu, H\. Howard, A\. Bloniarz, J\. W\. Rae, H\. Lu, L\. Sifre, M\. Maggioni, F\. Alcober, D\. Garrette, M\. Barnes, S\. Thakoor, J\. Austin, G\. Barth\-Maron, W\. Wong, R\. Joshi, R\. Chaabouni, D\. Fatiha, A\. Ahuja, G\. S\. Tomar, E\. Senter, M\. Chadwick, I\. Kornakov, N\. Attaluri, I\. Iturrate, R\. Liu, Y\. Li, S\. Cogan, J\. Chen, C\. Jia, C\. Gu, Q\. Zhang, J\. Grimstad, A\. J\. Hartman, X\. Garcia, T\. S\. Pillai, J\. Devlin, M\. Laskin, D\. de Las Casas, D\. Valter, C\. Tao, L\. Blanco, A\. P\. Badia, D\. Reitter, M\. Chen, J\. Brennan, C\. Rivera, S\. Brin, S\. Iqbal, G\. Surita, J\. Labanowski, A\. Rao, S\. Winkler, E\. Parisotto, Y\. Gu, K\. Olszewska, R\. Addanki, A\. Miech, A\. Louis, D\. Teplyashin, G\. Brown, E\. Catt, J\. Balaguer, J\. Xiang, P\. Wang, Z\. Ashwood, A\. Briukhov, A\. Webson, S\. Ganapathy, S\. Sanghavi, A\. Kannan, M\. Chang, A\. Stjerngren, J\. Djolonga, Y\. Sun, A\. Bapna, M\. Aitchison, P\. Pejman, H\. Michalewski, T\. Yu, C\. Wang, J\. Love, J\. Ahn, D\. Bloxwich, K\. Han, P\. Humphreys, T\. Sellam, J\. Bradbury, V\. Godbole, S\. Samangooei, B\. Damoc, A\. Kaskasoli, S\. M\. R\. Arnold, V\. Vasudevan, S\. Agrawal, J\. Riesa, D\. Lepikhin, R\. Tanburn, S\. Srinivasan, H\. Lim, S\. Hodkinson, P\. Shyam, J\. Ferret, S\. Hand, A\. Garg, T\. L\. Paine, J\. Li, Y\. Li, M\. Giang, A\. Neitz, Z\. Abbas, S\. York, M\. Reid, E\. Cole, A\. Chowdhery, D\. Das, D\. Rogozińska, V\. Nikolaev, P\. Sprechmann, Z\. Nado, L\. Zilka, F\. Prost, L\. He, M\. Monteiro, G\. Mishra, C\. Welty, J\. Newlan, D\. Jia, M\. Allamanis, C\. H\. Hu, R\. de Liedekerke, J\. Gilmer, C\. Saroufim, S\. Rijhwani, S\. Hou, D\. Shrivastava, A\. Baddepudi, A\. Goldin, A\. Ozturel, A\. Cassirer, Y\. Xu, D\. Sohn, D\. Sachan, R\. K\. Amplayo, C\. Swanson, D\. Petrova, S\. Narayan, A\. Guez, S\. Brahma, J\. Landon, M\. Patel, R\. Zhao, K\. Villela, L\. Wang, W\. Jia, M\. Rahtz, M\. Giménez, L\. Yeung, J\. Keeling, P\. Georgiev, D\. Mincu, B\. Wu, S\. Haykal, R\. Saputro, K\. Vodrahalli, J\. Qin, Z\. Cankara, A\. Sharma, N\. Fernando, W\. Hawkins, B\. Neyshabur, S\. Kim, A\. Hutter, P\. Agrawal, A\. Castro\-Ros, G\. van den Driessche, T\. Wang, F\. Yang, S\. Chang, P\. Komarek, R\. McIlroy, M\. Lučić, G\. Zhang, W\. Farhan, M\. Sharman, P\. Natsev, P\. Michel, Y\. Bansal, S\. Qiao, K\. Cao, S\. Shakeri, C\. Butterfield, J\. Chung, P\. K\. Rubenstein, S\. Agrawal, A\. Mensch, K\. Soparkar, K\. Lenc, T\. Chung, A\. Pope, L\. Maggiore, J\. Kay, P\. Jhakra, S\. Wang, J\. Maynez, M\. Phuong, T\. Tobin, A\. Tacchetti, M\. Trebacz, K\. Robinson, Y\. Katariya, S\. Riedel, P\. Bailey, K\. Xiao, N\. Ghelani, L\. Aroyo, A\. Slone, N\. Houlsby, X\. Xiong, Z\. Yang, E\. Gribovskaya, J\. Adler, M\. Wirth, L\. Lee, M\. Li, T\. Kagohara, J\. Pavagadhi, S\. Bridgers, A\. Bortsova, S\. Ghemawat, Z\. Ahmed, T\. Liu, R\. Powell, V\. Bolina, M\. Iinuma, P\. Zablotskaia, J\. Besley, D\. Chung, T\. Dozat, R\. Comanescu, X\. Si, J\. Greer, G\. Su, M\. Polacek, R\. L\. Kaufman, S\. Tokumine, H\. Hu, E\. Buchatskaya, Y\. Miao, M\. Elhawaty, A\. Siddhant, N\. Tomasev, J\. Xing, C\. Greer, H\. Miller, S\. Ashraf, A\. Roy, Z\. Zhang, A\. Ma, A\. Filos, M\. Besta, R\. Blevins, T\. Klimenko, C\. Yeh, S\. Changpinyo, J\. Mu, O\. Chang, M\. Pajarskas, C\. Muir, V\. Cohen, C\. L\. Lan, K\. Haridasan, A\. Marathe, S\. Hansen, S\. Douglas, R\. Samuel, M\. Wang, S\. Austin, C\. Lan, J\. Jiang, J\. Chiu, J\. A\. Lorenzo, L\. L\. Sjösund, S\. Cevey, Z\. Gleicher, T\. Avrahami, A\. Boral, H\. Srinivasan, V\. Selo, R\. May, K\. Aisopos, L\. Hussenot, L\. B\. Soares, K\. Baumli, M\. B\. Chang, A\. Recasens, B\. Caine, A\. Pritzel, F\. Pavetic, F\. Pardo, A\. Gergely, J\. Frye, V\. Ramasesh, D\. Horgan, K\. Badola, N\. Kassner, S\. Roy, E\. Dyer, V\. C\. Campos, A\. Tomala, Y\. Tang, D\. E\. Badawy, E\. White, B\. Mustafa, O\. Lang, A\. Jindal, S\. Vikram, Z\. Gong, S\. Caelles, R\. Hemsley, G\. Thornton, F\. Feng, W\. Stokowiec, C\. Zheng, P\. Thacker, Ç\. Ünlü, Z\. Zhang, M\. Saleh, J\. Svensson, M\. Bileschi, P\. Patil, A\. Anand, R\. Ring, K\. Tsihlas, A\. Vezer, M\. Selvi, T\. Shevlane, M\. Rodriguez, T\. Kwiatkowski, S\. Daruki, K\. Rong, A\. Dafoe, N\. FitzGerald, K\. Gu\-Lemberg, M\. Khan, L\. A\. Hendricks, M\. Pellat, V\. Feinberg, J\. Cobon\-Kerr, T\. Sainath, M\. Rauh, S\. H\. Hashemi, R\. Ives, Y\. Hasson, E\. Noland, Y\. Cao, N\. Byrd, L\. Hou, Q\. Wang, T\. Sottiaux, M\. Paganini, J\. Lespiau, A\. Moufarek, S\. Hassan, K\. Shivakumar, J\. van Amersfoort, A\. Mandhane, P\. Joshi, A\. Goyal, M\. Tung, A\. Brock, H\. Sheahan, V\. Misra, C\. Li, N\. Rakićević, M\. Dehghani, F\. Liu, S\. Mittal, J\. Oh, S\. Noury, E\. Sezener, F\. Huot, M\. Lamm, N\. D\. Cao, C\. Chen, S\. Mudgal, R\. Stella, K\. Brooks, G\. Vasudevan, C\. Liu, M\. Chain, N\. Melinkeri, A\. Cohen, V\. Wang, K\. Seymore, S\. Zubkov, R\. Goel, S\. Yue, S\. Krishnakumaran, B\. Albert, N\. Hurley, M\. Sano, A\. Mohananey, J\. Joughin, E\. Filonov, T\. Kępa, Y\. Eldawy, J\. Lim, R\. Rishi, S\. Badiezadegan, T\. Bos, J\. Chang, S\. Jain, S\. G\. S\. Padmanabhan, S\. Puttagunta, K\. Krishna, L\. Baker, N\. Kalb, V\. Bedapudi, A\. Kurzrok, S\. Lei, A\. Yu, O\. Litvin, X\. Zhou, Z\. Wu, S\. Sobell, A\. Siciliano, A\. Papir, R\. Neale, J\. Bragagnolo, T\. Toor, T\. Chen, V\. Anklin, F\. Wang, R\. Feng, M\. Gholami, K\. Ling, L\. Liu, J\. Walter, H\. Moghaddam, A\. Kishore, J\. Adamek, T\. Mercado, J\. Mallinson, S\. Wandekar, S\. Cagle, E\. Ofek, G\. Garrido, C\. Lombriser, M\. Mukha, B\. Sun, H\. R\. Mohammad, J\. Matak, Y\. Qian, V\. Peswani, P\. Janus, Q\. Yuan, L\. Schelin, O\. David, A\. Garg, Y\. He, O\. Duzhyi, A\. Älgmyr, T\. Lottaz, Q\. Li, V\. Yadav, L\. Xu, A\. Chinien, R\. Shivanna, A\. Chuklin, J\. Li, C\. Spadine, T\. Wolfe, K\. Mohamed, S\. Das, Z\. Dai, K\. He, D\. von Dincklage, S\. Upadhyay, A\. Maurya, L\. Chi, S\. Krause, K\. Salama, P\. G\. Rabinovitch, P\. K\. R\. M, A\. Selvan, M\. Dektiarev, G\. Ghiasi, E\. Guven, H\. Gupta, B\. Liu, D\. Sharma, I\. H\. Shtacher, S\. Paul, O\. Akerlund, F\. Aubet, T\. Huang, C\. Zhu, E\. Zhu, E\. Teixeira, M\. Fritze, F\. Bertolini, L\. Marinescu, M\. Bölle, D\. Paulus, K\. Gupta, T\. Latkar, M\. Chang, J\. Sanders, R\. Wilson, X\. Wu, Y\. Tan, L\. N\. Thiet, T\. Doshi, S\. Lall, S\. Mishra, W\. Chen, T\. Luong, S\. Benjamin, J\. Lee, E\. Andrejczuk, D\. Rabiej, V\. Ranjan, K\. Styrc, P\. Yin, J\. Simon, M\. R\. Harriott, M\. Bansal, A\. Robsky, G\. Bacon, D\. Greene, D\. Mirylenka, C\. Zhou, O\. Sarvana, A\. Goyal, S\. Andermatt, P\. Siegler, B\. Horn, A\. Israel, F\. Pongetti, C\. "\. Chen, M\. Selvatici, P\. Silva, K\. Wang, J\. Tolins, K\. Guu, R\. Yogev, X\. Cai, A\. Agostini, M\. Shah, H\. Nguyen, N\. Ó\. Donnaile, S\. Pereira, L\. Friso, A\. Stambler, A\. Kurzrok, C\. Kuang, Y\. Romanikhin, M\. Geller, Z\. Yan, K\. Jang, C\. Lee, W\. Fica, E\. Malmi, Q\. Tan, D\. Banica, D\. Balle, R\. Pham, Y\. Huang, D\. Avram, H\. Shi, J\. Singh, C\. Hidey, N\. Ahuja, P\. Saxena, D\. Dooley, S\. P\. Potharaju, E\. O’Neill, A\. Gokulchandran, R\. Foley, K\. Zhao, M\. Dusenberry, Y\. Liu, P\. Mehta, R\. Kotikalapudi, C\. Safranek\-Shrader, A\. Goodman, J\. Kessinger, E\. Globen, P\. Kolhar, C\. Gorgolewski, A\. Ibrahim, Y\. Song, A\. Eichenbaum, T\. Brovelli, S\. Potluri, P\. Lahoti, C\. Baetu, A\. Ghorbani, C\. Chen, A\. Crawford, S\. Pal, M\. Sridhar, P\. Gurita, A\. Mujika, I\. Petrovski, P\. Cedoz, C\. Li, S\. Chen, N\. D\. Santo, S\. Goyal, J\. Punjabi, K\. Kappaganthu, C\. Kwak, P\. LV, S\. Velury, H\. Choudhury, J\. Hall, P\. Shah, R\. Figueira, M\. Thomas, M\. Lu, T\. Zhou, C\. Kumar, T\. Jurdi, S\. Chikkerur, Y\. Ma, A\. Yu, S\. Kwak, V\. Ähdel, S\. Rajayogam, T\. Choma, F\. Liu, A\. Barua, C\. Ji, J\. H\. Park, V\. Hellendoorn, A\. Bailey, T\. Bilal, H\. Zhou, M\. Khatir, C\. Sutton, W\. Rzadkowski, F\. Macintosh, K\. Shagin, P\. Medina, C\. Liang, J\. Zhou, P\. Shah, Y\. Bi, A\. Dankovics, S\. Banga, S\. Lehmann, M\. Bredesen, Z\. Lin, J\. E\. Hoffmann, J\. Lai, R\. Chung, K\. Yang, N\. Balani, A\. Bražinskas, A\. Sozanschi, M\. Hayes, H\. F\. Alcalde, P\. Makarov, W\. Chen, A\. Stella, L\. Snijders, M\. Mandl, A\. Kärrman, P\. Nowak, X\. Wu, A\. Dyck, K\. Vaidyanathan, R\. R, J\. Mallet, M\. Rudominer, E\. Johnston, S\. Mittal, A\. Udathu, J\. Christensen, V\. Verma, Z\. Irving, A\. Santucci, G\. Elsayed, E\. Davoodi, M\. Georgiev, I\. Tenney, N\. Hua, G\. Cideron, E\. Leurent, M\. Alnahlawi, I\. Georgescu, N\. Wei, I\. Zheng, D\. Scandinaro, H\. Jiang, J\. Snoek, M\. Sundararajan, X\. Wang, Z\. Ontiveros, I\. Karo, J\. Cole, V\. Rajashekhar, L\. Tumeh, E\. Ben\-David, R\. Jain, J\. Uesato, R\. Datta, O\. Bunyan, S\. Wu, J\. Zhang, P\. Stanczyk, Y\. Zhang, D\. Steiner, S\. Naskar, M\. Azzam, M\. Johnson, A\. Paszke, C\. Chiu, J\. S\. Elias, A\. Mohiuddin, F\. Muhammad, J\. Miao, A\. Lee, N\. Vieillard, J\. Park, J\. Zhang, J\. Stanway, D\. Garmon, A\. Karmarkar, Z\. Dong, J\. Lee, A\. Kumar, L\. Zhou, J\. Evens, W\. Isaac, G\. Irving, E\. Loper, M\. Fink, I\. Arkatkar, N\. Chen, I\. Shafran, I\. Petrychenko, Z\. Chen, J\. Jia, A\. Levskaya, Z\. Zhu, P\. Grabowski, Y\. Mao, A\. Magni, K\. Yao, J\. Snaider, N\. Casagrande, E\. Palmer, P\. Suganthan, A\. Castaño, I\. Giannoumis, W\. Kim, M\. Rybiński, A\. Sreevatsa, J\. Prendki, D\. Soergel, A\. Goedeckemeyer, W\. Gierke, M\. Jafari, M\. Gaba, J\. Wiesner, D\. G\. Wright, Y\. Wei, H\. Vashisht, Y\. Kulizhskaya, J\. Hoover, M\. Le, L\. Li, C\. Iwuanyanwu, L\. Liu, K\. Ramirez, A\. Khorlin, A\. Cui, T\. LIN, M\. Wu, R\. Aguilar, K\. Pallo, A\. Chakladar, G\. Perng, E\. A\. Abellan, M\. Zhang, I\. Dasgupta, N\. Kushman, I\. Penchev, A\. Repina, X\. Wu, T\. van der Weide, P\. Ponnapalli, C\. Kaplan, J\. Simsa, S\. Li, O\. Dousse, F\. Yang, J\. Piper, N\. Ie, R\. Pasumarthi, N\. Lintz, A\. Vijayakumar, D\. Andor, P\. Valenzuela, M\. Lui, C\. Paduraru, D\. Peng, K\. Lee, S\. Zhang, S\. Greene, D\. D\. Nguyen, P\. Kurylowicz, C\. Hardin, L\. Dixon, L\. Janzer, K\. Choo, Z\. Feng, B\. Zhang, A\. Singhal, D\. Du, D\. McKinnon, N\. Antropova, T\. Bolukbasi, O\. Keller, D\. Reid, D\. Finchelstein, M\. A\. Raad, R\. Crocker, P\. Hawkins, R\. Dadashi, C\. Gaffney, K\. Franko, A\. Bulanova, R\. Leblond, S\. Chung, H\. Askham, L\. C\. Cobo, K\. Xu, F\. Fischer, J\. Xu, C\. Sorokin, C\. Alberti, C\. Lin, C\. Evans, A\. Dimitriev, H\. Forbes, D\. Banarse, Z\. Tung, M\. Omernick, C\. Bishop, R\. Sterneck, R\. Jain, J\. Xia, E\. Amid, F\. Piccinno, X\. Wang, P\. Banzal, D\. J\. Mankowitz, A\. Polozov, V\. Krakovna, S\. Brown, M\. Bateni, D\. Duan, V\. Firoiu, M\. Thotakuri, T\. Natan, M\. Geist, S\. tan Girgin, H\. Li, J\. Ye, O\. Roval, R\. Tojo, M\. Kwong, J\. Lee\-Thorp, C\. Yew, D\. Sinopalnikov, S\. Ramos, J\. Mellor, A\. Sharma, K\. Wu, D\. Miller, N\. Sonnerat, D\. Vnukov, R\. Greig, J\. Beattie, E\. Caveness, L\. Bai, J\. Eisenschlos, A\. Korchemniy, T\. Tsai, M\. Jasarevic, W\. Kong, P\. Dao, Z\. Zheng, F\. Liu, F\. Yang, R\. Zhu, T\. H\. Teh, J\. Sanmiya, E\. Gladchenko, N\. Trdin, D\. Toyama, E\. Rosen, S\. Tavakkol, L\. Xue, C\. Elkind, O\. Woodman, J\. Carpenter, G\. Papamakarios, R\. Kemp, S\. Kafle, T\. Grunina, R\. Sinha, A\. Talbert, D\. Wu, D\. Owusu\-Afriyie, C\. Du, C\. Thornton, J\. Pont\-Tuset, P\. Narayana, J\. Li, S\. Fatehi, J\. Wieting, O\. Ajmeri, B\. Uria, Y\. Ko, L\. Knight, A\. Héliou, N\. Niu, S\. Gu, C\. Pang, Y\. Li, N\. Levine, A\. Stolovich, R\. Santamaria\-Fernandez, S\. Goenka, W\. Yustalim, R\. Strudel, A\. Elqursh, C\. Deck, H\. Lee, Z\. Li, K\. Levin, R\. Hoffmann, D\. Holtmann\-Rice, O\. Bachem, S\. Arora, C\. Koh, S\. H\. Yeganeh, S\. Põder, M\. Tariq, Y\. Sun, L\. Ionita, M\. Seyedhosseini, P\. Tafti, Z\. Liu, A\. Gulati, J\. Liu, X\. Ye, B\. Chrzaszcz, L\. Wang, N\. Sethi, T\. Li, B\. Brown, S\. Singh, W\. Fan, A\. Parisi, J\. Stanton, V\. Koverkathu, C\. A\. Choquette\-Choo, Y\. Li, T\. Lu, A\. Ittycheriah, P\. Shroff, M\. Varadarajan, S\. Bahargam, R\. Willoughby, D\. Gaddy, G\. Desjardins, M\. Cornero, B\. Robenek, B\. Mittal, B\. Albrecht, A\. Shenoy, F\. Moiseev, H\. Jacobsson, A\. Ghaffarkhah, M\. Rivière, A\. Walton, C\. Crepy, A\. Parrish, Z\. Zhou, C\. Farabet, C\. Radebaugh, P\. Srinivasan, C\. van der Salm, A\. Fidjeland, S\. Scellato, E\. Latorre\-Chimoto, H\. Klimczak\-Plucińska, D\. Bridson, D\. de Cesare, T\. Hudson, P\. Mendolicchio, L\. Walker, A\. Morris, M\. Mauger, A\. Guseynov, A\. Reid, S\. Odoom, L\. Loher, V\. Cotruta, M\. Yenugula, D\. Grewe, A\. Petrushkina, T\. Duerig, A\. Sanchez, S\. Yadlowsky, A\. Shen, A\. Globerson, L\. Webb, S\. Dua, D\. Li, S\. Bhupatiraju, D\. Hurt, H\. Qureshi, A\. Agarwal, T\. Shani, M\. Eyal, A\. Khare, S\. R\. Belle, L\. Wang, C\. Tekur, M\. S\. Kale, J\. Wei, R\. Sang, B\. Saeta, T\. Liechty, Y\. Sun, Y\. Zhao, S\. Lee, P\. Nayak, D\. Fritz, M\. R\. Vuyyuru, J\. Aslanides, N\. Vyas, M\. Wicke, X\. Ma, E\. Eltyshev, N\. Martin, H\. Cate, J\. Manyika, K\. Amiri, Y\. Kim, X\. Xiong, K\. Kang, F\. Luisier, N\. Tripuraneni, D\. Madras, M\. Guo, A\. Waters, O\. Wang, J\. Ainslie, J\. Baldridge, H\. Zhang, G\. Pruthi, J\. Bauer, F\. Yang, R\. Mansour, J\. Gelman, Y\. Xu, G\. Polovets, J\. Liu, H\. Cai, W\. Chen, X\. Sheng, E\. Xue, S\. Ozair, C\. Angermueller, X\. Li, A\. Sinha, W\. Wang, J\. Wiesinger, E\. Koukoumidis, Y\. Tian, A\. Iyer, M\. Gurumurthy, M\. Goldenson, P\. Shah, M\. Blake, H\. Yu, A\. Urbanowicz, J\. Palomaki, C\. Fernando, K\. Durden, H\. Mehta, N\. Momchev, E\. Rahimtoroghi, M\. Georgaki, A\. Raul, S\. Ruder, M\. Redshaw, J\. Lee, D\. Zhou, K\. Jalan, D\. Li, B\. Hechtman, P\. Schuh, M\. Nasr, K\. Milan, V\. Mikulik, J\. Franco, T\. Green, N\. Nguyen, J\. Kelley, A\. Mahendru, A\. Hu, J\. Howland, B\. Vargas, J\. Hui, K\. Bansal, V\. Rao, R\. Ghiya, E\. Wang, K\. Ye, J\. M\. Sarr, M\. M\. Preston, M\. Elish, S\. Li, A\. Kaku, J\. Gupta, I\. Pasupat, D\. Juan, M\. Someswar, T\. M\., X\. Chen, A\. Amini, A\. Fabrikant, E\. Chu, X\. Dong, A\. Muthal, S\. Buthpitiya, S\. Jauhari, N\. Hua, U\. Khandelwal, A\. Hitron, J\. Ren, L\. Rinaldi, S\. Drath, A\. Dabush, N\. Jiang, H\. Godhia, U\. Sachs, A\. Chen, Y\. Fan, H\. Taitelbaum, H\. Noga, Z\. Dai, J\. Wang, C\. Liang, J\. Hamer, C\. Ferng, C\. Elkind, A\. Atias, P\. Lee, V\. Listík, M\. Carlen, J\. van de Kerkhof, M\. Pikus, K\. Zaher, P\. Müller, S\. Zykova, R\. Stefanec, V\. Gatsko, C\. Hirnschall, A\. Sethi, X\. F\. Xu, C\. Ahuja, B\. Tsai, A\. Stefanoiu, B\. Feng, K\. Dhandhania, M\. Katyal, A\. Gupta, A\. Parulekar, D\. Pitta, J\. Zhao, V\. Bhatia, Y\. Bhavnani, O\. Alhadlaq, X\. Li, P\. Danenberg, D\. Tu, A\. Pine, V\. Filippova, A\. Ghosh, B\. Limonchik, B\. Urala, C\. K\. Lanka, D\. Clive, Y\. Sun, E\. Li, H\. Wu, K\. Hongtongsak, I\. Li, K\. Thakkar, K\. Omarov, K\. Majmundar, M\. Alverson, M\. Kucharski, M\. Patel, M\. Jain, M\. Zabelin, P\. Pelagatti, R\. Kohli, S\. Kumar, J\. Kim, S\. Sankar, V\. Shah, L\. Ramachandruni, X\. Zeng, B\. Bariach, L\. Weidinger, T\. Vu, A\. Andreev, A\. He, K\. Hui, S\. Kashem, A\. Subramanya, S\. Hsiao, D\. Hassabis, K\. Kavukcuoglu, A\. Sadovsky, Q\. Le, T\. Strohman, Y\. Wu, S\. Petrov, J\. Dean, and O\. Vinyals \(2024\)Gemini: a family of highly capable multimodal models\.External Links:2312\.11805,[Link](https://arxiv.org/abs/2312.11805)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p1.1)\.
- Q\. Team \(2024\)Qwen2\.5: a party of foundation models\.External Links:[Link](https://qwenlm.github.io/blog/qwen2.5/)Cited by:[Appendix C](https://arxiv.org/html/2502.12446#A3.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2502.12446#S4.SS1.SSS0.Px1.p1.1)\.
- A\. S\. Thakur, K\. Choudhary, V\. S\. Ramayapally, S\. Vaidyanathan, and D\. Hupkes \(2024\)Judging the judges: evaluating alignment and vulnerabilities in llms\-as\-judges\.arXiv preprint arXiv:2406\.12624\.Cited by:[§4\.1](https://arxiv.org/html/2502.12446#S4.SS1.SSS0.Px2.p2.1),[§4\.2](https://arxiv.org/html/2502.12446#S4.SS2.SSS0.Px2.p1.1)\.
- X\. Tian, S\. Zou, Z\. Yang, and J\. Zhang \(2024\)ArGue: attribute\-guided prompt tuning for vision\-language models\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 28578–28587\.Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px1.p1.1)\.
- A\. M\. Turner, L\. Thiergart, G\. Leech, D\. Udell, J\. J\. Vazquez, U\. Mini, and M\. MacDiarmid \(2024\)Steering language models with activation engineering\.External Links:2308\.10248,[Link](https://arxiv.org/abs/2308.10248)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p3.1)\.
- T\. van der Weij, M\. Poesio, and N\. Schoots \(2024\)Extending activation steering to broad skills and multiple behaviours\.arXiv preprint arXiv:2403\.05767\.Cited by:[§2\.2](https://arxiv.org/html/2502.12446#S2.SS2.p2.9)\.
- H\. Wang, Y\. Lin, W\. Xiong, R\. Yang, S\. Diao, S\. Qiu, H\. Zhao, and T\. Zhang \(2024a\)Arithmetic control of LLMs for diverse user preferences: directional preference alignment with multi\-objective rewards\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 8642–8655\.External Links:[Link](https://aclanthology.org/2024.acl-long.468/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.468)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- H\. Wang, W\. Xiong, T\. Xie, H\. Zhao, and T\. Zhang \(2024b\)Interpretable preferences via multi\-objective reward modeling and mixture\-of\-experts\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 10582–10592\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.620/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.620)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- Z\. Wang, Y\. Dong, O\. Delalleau, J\. Zeng, G\. Shen, D\. Egert, J\. J\. Zhang, M\. N\. Sreedhar, and O\. Kuchaiev \(2024c\)HelpSteer 2: open\-source dataset for training top\-performing reward models\.InThe Thirty\-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=PvVKUFhaNy)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p2.1),[Figure 3](https://arxiv.org/html/2502.12446#S4.F3),[§4\.2](https://arxiv.org/html/2502.12446#S4.SS2.SSS0.Px2.p1.1)\.
- Z\. Wang, Y\. Dong, J\. Zeng, V\. Adams, M\. N\. Sreedhar, D\. Egert, O\. Delalleau, J\. Scowcroft, N\. Kant, A\. Swope, and O\. Kuchaiev \(2024d\)HelpSteer: multi\-attribute helpfulness dataset for SteerLM\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),K\. Duh, H\. Gomez, and S\. Bethard \(Eds\.\),Mexico City, Mexico,pp\. 3371–3384\.External Links:[Link](https://aclanthology.org/2024.naacl-long.185/),[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.185)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p2.1),[§1](https://arxiv.org/html/2502.12446#S1.p5.1),[§4\.1](https://arxiv.org/html/2502.12446#S4.SS1.SSS0.Px2.p2.1),[§4\.2](https://arxiv.org/html/2502.12446#S4.SS2.SSS0.Px2.p1.1)\.
- M\. Wortsman, G\. Ilharco, S\. Y\. Gadre, R\. Roelofs, R\. Gontijo\-Lopes, A\. S\. Morcos, H\. Namkoong, A\. Farhadi, Y\. Carmon, S\. Kornblith,et al\.\(2022\)Model soups: averaging weights of multiple fine\-tuned models improves accuracy without increasing inference time\.InInternational conference on machine learning,pp\. 23965–23998\.Cited by:[3rd item](https://arxiv.org/html/2502.12446#S4.I2.i3.p1.1)\.
- Z\. Wu, Y\. Hu, W\. Shi, N\. Dziri, A\. Suhr, P\. Ammanabrolu, N\. A\. Smith, M\. Ostendorf, and H\. Hajishirzi \(2023\)Fine\-grained human feedback gives better rewards for language model training\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=CSbGXyCswu)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- Z\. Wu, A\. Arora, Z\. Wang, A\. Geiger, D\. Jurafsky, C\. D\. Manning, and C\. Potts \(2024\)ReFT: representation finetuning for language models\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=fykjplMc0V)Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p3.1)\.
- B\. Xu, S\. Wu, K\. Liu, and L\. Hu \(2025\)Mixture\-of\-instructions: aligning large language models via mixture prompting\.External Links:2404\.18410,[Link](https://arxiv.org/abs/2404.18410)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px1.p1.1)\.
- K\. Yang, Z\. Liu, Q\. Xie, J\. Huang, T\. Zhang, and S\. Ananiadou \(2024a\)Metaaligner: towards generalizable multi\-objective alignment of language models\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- S\. Yang, M\. A\. Ali, C\. Wang, L\. Hu, and D\. Wang \(2024b\)MoRAL: moe augmented lora for llms’ lifelong learning\.arXiv preprint arXiv:2402\.11260\.Cited by:[3rd item](https://arxiv.org/html/2502.12446#S4.I2.i3.p1.1)\.
- W\. Zhang, P\. Torr, M\. Elhoseiny, and A\. Bibi \(2025\)Bi\-factorial preference optimization: balancing safety\-helpfulness in language models\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=GjM61KRiTG)Cited by:[§6](https://arxiv.org/html/2502.12446#S6.SS0.SSS0.Px2.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing,et al\.\(2023\)Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.Advances in Neural Information Processing Systems36,pp\. 46595–46623\.Cited by:[§4\.1](https://arxiv.org/html/2502.12446#S4.SS1.SSS0.Px2.p2.1),[§4\.2](https://arxiv.org/html/2502.12446#S4.SS2.SSS0.Px2.p1.1)\.
- A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski,et al\.\(2023\)Representation engineering: a top\-down approach to ai transparency\.arXiv preprint arXiv:2310\.01405\.Cited by:[§1](https://arxiv.org/html/2502.12446#S1.p3.1),[§3\.2](https://arxiv.org/html/2502.12446#S3.SS2.p1.3)\.

## Appendix AExperimental Settings

#### Data Preprocessing\.

For the TruthfulQA dataset \(Apache\-2\.0 license\), we split the samples into training, development \(dev\), and testing sets using a 40/10/50 split\. For Toxigen \(MIT license\) and BBQ \(cc\-by\-4\.0 license\), these datasets have already been split into training and validation sets\. We use the validation sets to test the models while further splitting the training sets with an 80/20 ratio to create new train and dev sets\. For HelpSteer \(released under cc\-by\-4\.0 license\), after sampling 500 positive and 500 negative samples for each attribute, we apply a 40/10/50 split for the train\-dev\-test sets\. All methods are trained on the combined training sets from different datasets and evaluated on the corresponding test sets for each task individually\. All datasets are in English\.

#### Implementation Details for Baselines andMAT\-Steer\.

We provide implementation details of our method and baselines as follows:

- •Layer to intervene:Following previous work\(Liet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib10)\), which suggests that information is primarily processed in the early to middle layers, we conduct a grid search from layer 10 to layer 22 of LLMs to maximize performance on the dev set\. Based on this search, we intervene at layer 14 for Llama\-3\.1\-8B and Llama\-3\.1\-8B Chat and at layer 16 for Qwen2\.5\-7B\.
- •Training:For training with LoRA, we set the rank to1616and alpha to3232\. We fine\-tune the model for1010iterations using a learning rate of5​e−65e\\\!\-\\\!6and a batch size of1616\. For our method, we use a batch size of9696for QA tasks and160160for generation tasks, while each batch contains1616positive and1616negative samples for each attribute\.
- •Hyperparameters:For intervention baselines, we follow the same settings as in the original paper for TruthfulQA\. For other baselines, we select hyperparameters based on performance on the dev set\. ForMAT\-Steer, we setλpos=λsparse\\lambda\_\{\\mathrm\{pos\}\}=\\lambda\_\{\\mathrm\{sparse\}\}, as we assume that the weights of constraints applied to positive and negative samples should be the same\. We then perform a grid search on the dev set forλpos\\lambda\_\{\\mathrm\{pos\}\},λsparse\\lambda\_\{\\mathrm\{sparse\}\}, andλortho\\lambda\_\{\\mathrm\{ortho\}\}in the range\[0,1\]\[0,1\]with a step size of 0\.1\. For QA tasks, the optimal hyperparameters areλpos=λsparse=0\.9\\lambda\_\{\\mathrm\{pos\}\}=\\lambda\_\{\\mathrm\{sparse\}\}=0\.9andλortho=0\.1\\lambda\_\{\\mathrm\{ortho\}\}=0\.1\. For generation tasks, the optimal hyperparameters areλpos=λsparse=0\.8\\lambda\_\{\\mathrm\{pos\}\}=\\lambda\_\{\\mathrm\{sparse\}\}=0\.8andλortho=0\.1\\lambda\_\{\\mathrm\{ortho\}\}=0\.1\.

#### MMD Kernel\.

The MMD loss in \([5](https://arxiv.org/html/2502.12446#S3.E5)\) can also be written as:

ℒMMD=∑t=1T\(\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{MMD\}\}=\\sum\_\{t=1\}^\{T\}\\Bigg\(1\|𝒜t𝐩\|2​∑ai,aj∈𝒜t𝐩k​\(ai,aj\)\\displaystyle\\frac\{1\}\{\|\\mathcal\{A\}\_\{t\}^\{\\mathbf\{p\}\}\|^\{2\}\}\\sum\_\{\\begin\{subarray\}\{c\}a\_\{i\},a\_\{j\}\\in\\mathcal\{A\}\_\{t\}^\{\\mathbf\{p\}\}\\end\{subarray\}\}k\(a\_\{i\},a\_\{j\}\)\+1\|𝒜t𝐧\|2​∑ai,aj∈𝒜t𝐧k​\(f​\(ai\),f​\(aj\)\)\\displaystyle\+\\frac\{1\}\{\|\\mathcal\{A\}\_\{t\}^\{\\mathbf\{n\}\}\|^\{2\}\}\\sum\_\{\\begin\{subarray\}\{c\}a\_\{i\},a\_\{j\}\\in\\mathcal\{A\}\_\{t\}^\{\\mathbf\{n\}\}\\end\{subarray\}\}k\(f\(a\_\{i\}\),f\(a\_\{j\}\)\)−2\|𝒜t𝐩\|​\|𝒜t𝐧\|∑ai∈𝒜t𝐩aj∈𝒜t𝐧k\(ai,f\(aj\)\)\)\.\\displaystyle\-\\frac\{2\}\{\|\\mathcal\{A\}\_\{t\}^\{\\mathbf\{p\}\}\|\|\\mathcal\{A\}\_\{t\}^\{\\mathbf\{n\}\}\|\}\\sum\_\{\\begin\{subarray\}\{c\}a\_\{i\}\\in\\mathcal\{A\}\_\{t\}^\{\\mathbf\{p\}\}\\\\ a\_\{j\}\\in\\mathcal\{A\}\_\{t\}^\{\\mathbf\{n\}\}\\end\{subarray\}\}k\(a\_\{i\},f\(a\_\{j\}\)\)\\Bigg\)\.\(10\)
In our experiments, the MMD loss is computed using a Gaussian kernel:

k​\(x,y\)=exp⁡\(−‖x−y‖22​σ2\),k\(x,y\)=\\exp\\left\(\-\\frac\{\\\|x\-y\\\|^\{2\}\}\{2\\sigma^\{2\}\}\\right\),with a carefully chosen bandwidthσ\\sigmabased on the performance on the dev set \(minimizes the validation loss\)\. We chooseσ=2\\sigma=2for both QA and generation tasks\.

#### GPUs\.

All of our experiments are run on four RTX A6000 with 48G memory each\.

#### Prompts\.

For TruthfulQA, Toxigen, BBQ, and HelpSteer, we do not include any system instruction prompt\. Instead, we simply provide the input questions and capture the responses\. For FaithEval, we follow the same prompt design as described in the original paper\(Minget al\.,[2025](https://arxiv.org/html/2502.12446#bib.bib58)\)\.

## Appendix BFurther Analysis ofMAT\-Steer

#### MAT\-SteerPerforms Better when Scaling the Number of Attributes for Generation\.

In real\-world applications, it is often the case that objectives must be adopted sequentially \(rather than having all objectives presented simultaneously\)\.[Fig\.˜7](https://arxiv.org/html/2502.12446#A2.F7)illustrates how our method scales as we increase the number of attributes in HelpSteer from 1 to 5, reporting the win rates against various baselines\. Each point on the horizontal axis corresponds to a scenario where the model must optimize for a specific number of attributes \(e\.g\., helpfulness, correctness, and coherence\)\. We observe that our approach maintains or even improves its margin over baselines as the number of attributes grows\. While these baselines typically experience diminishing outcomes or exhibit flat performance when handling multiple objectives,MAT\-Steereffectively balances those objectives\. For example, when scaling from 1 to 4 attributes,MAT\-Steerincreases the win rate margin over SFT by 5\.65% and over ITI by 8\.98%\. This result highlights the robustness of our method in multi\-attribute settings, where aligning model outputs with multiple human\-preferred criteria becomes more challenging\.

![Refer to caption](https://arxiv.org/html/2502.12446v3/x7.png)Figure 7:Win rates ofMAT\-Steervs\. baselines as the number of HelpSteer attributes increases from 1 to 5\. The dotted line represents a50%50\\%win\-rate, indicating a tie\.
#### Token Selection Analysis\.

We further investigate the role of token selection in our intervention framework\. We compare several selection baselines: uniform intervention on all tokens, intervention on only the last token in the prompt, random token selection, andMAT\-Steer’s selection method\. Table[4](https://arxiv.org/html/2502.12446#A2.T4)reports the multiple\-choice accuracy on the TruthfulQA, Toxigen, and BBQ datasets\.MAT\-Steeroutperforms all other methods, highlighting that selecting the*right*tokens – rather than merely reducing the number of tokens – plays a crucial role in achieving superior attribute\-specific improvements while preserving essential contextual information\.

Table 4:Performance comparison of different token selection methods across three datasets on Llama\-3\.1\-8B\.MAT\-Steeroutperforms all other strategies across all datasets, with the highest results highlighted in bold\.

## Appendix CAdditional Numerical Results

#### Additional Results on Other Tasks\.

Since our main experiments focus on QA tasks targeting multiple trustworthiness\-related attributes, we have extended our evaluation to the HH\-RLHF benchmark\(Baiet al\.,[2022](https://arxiv.org/html/2502.12446#bib.bib271)\)to assess the generalization of MAT\-Steer and baselines\. In particular, we use the pretrained vectors from QA tasks in[Table˜1](https://arxiv.org/html/2502.12446#S4.T1)and test them on the HH\-RLHF test set\. The HH\-RLHF benchmark involves complex, real\-world assistant\-style queries, covering open\-ended topics that require models to be both helpful and harmless\. We use GPT\-4o to compare generations from MAT\-Steer and three baselines: SFT, ICL, and LITO \(the strongest baseline in all benchmarks\) on the HH\-RLHF test set\. The GPT\-4o win rates are shown in[Table˜5](https://arxiv.org/html/2502.12446#A3.T5)\. These results show that MAT\-Steer outperforms all baselines, demonstrating its ability to generalize beyond in\-domain QA tasks\.

Table 5:GPT\-4o win rates comparing MAT\-Steer against other methods on HH\-RLHF\.Additionally, we extend MAT\-Steer to an explicitly structured reasoning task by incorporating the OpenBookQA \(OBQA\)\(Mihaylovet al\.,[2018](https://arxiv.org/html/2502.12446#bib.bib272)\)dataset into our QA tasks \(alongside TruthfulQA, Toxigen, and BBQ\)\. OBQA has been used in prior ITI work, such as NL\-ITI\(Hoscilowiczet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib237)\)and multiple efforts on LLM reasoning\(Dubeyet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib212); Jianget al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib273)\)\. We compare MAT\-Steer on OBQA against three baselines: ICL, SFT \(fine\-tune the model checkpoint from 3 datasets with additional OBQA training data\), and LITO \(the strongest ITI baseline in all benchmarks\)\.

As suggested by[Fig\.˜7](https://arxiv.org/html/2502.12446#A2.F7), most baselines degrade when the number of tasks or attributes increases; therefore, we report both the accuracy on OBQA and the average performance drop on the original QA datasets \(TruthfulQA, Toxigen, and BBQ\) to assess multi\-task robustness\.[Table˜6](https://arxiv.org/html/2502.12446#A3.T6)demonstrates that while SFT performs slightly better than MAT\-Steer on OBQA, it has a significant performance drop of 2\.74% on the other tasks\. Additionally, ICL has the smallest performance drop \(0\.14%\) but also has the lowest OBQA accuracy \(73\.40%\), indicating limited overall capability\. In contrast, MAT\-Steer maintains strong performance across all tasks with only 0\.32% drop, highlighting its ability to generalize to new reasoning settings while preserving previously aligned behaviors, a key advantage of our multi\-attribute steering framework\.

Table 6:Performance on OBQA and average decrease on 3 original QA tasks for different methods\.
#### Generalization to Model Type and Family\.

We evaluate our method on different LLM families to demonstrate its robustness\. Experiments on Llama\-3\.1\-7B Chat\(Dubeyet al\.,[2024](https://arxiv.org/html/2502.12446#bib.bib212)\)and Qwen2\.5\-7B\(Team,[2024](https://arxiv.org/html/2502.12446#bib.bib209)\)reveal that our intervention strategy transfers effectively across model architectures, providing consistent improvements in multiple\-choice accuracy and generation quality\. This suggests that the benefits of our approach are not limited to a single model but generalize across various base LLMs\.

Table 7:Performance comparison ofMAT\-Steeragainst in\-context learning, fine\-tuning, multiple adapters, and intervention methods on Qwen2\.5\-7B\.Table 8:Performance comparison ofMAT\-Steeragainst in\-context learning, fine\-tuning, multiple adapters, and intervention methods on the Llama\-3\.1\-8B\-Chat model\.
#### Integration with ICL and Fine\-Tuning\.

We assess the complementarity of our method when combined with other adaptation techniques, such as in\-context learning and fine\-tuning\. We incorporate our token\-level intervention strategy on top of few\-shot prompting and LoRA\-based fine\-tuning\. As reported in Table[9](https://arxiv.org/html/2502.12446#A3.T9), our approach further enhances the performance of both in\-context learning \(ICL\) and fine\-tuning \(SFT, DPO\), yielding higher accuracies on QA tasks\. These results confirm that our method is not only effective as a standalone intervention strategy but also synergizes well with existing techniques to boost overall performance\.

Table 9:Performance comparison of our method against few\-shot prompting, fine\-tuning method with our method on top\. Each method is evaluated on three datasets: TruthfulQA, Toxigen, and BBQ\. The highest performance for each dataset is highlighted in bold\.
#### Examples on FaithEval\.

To better understand the interpretability of our proposed gating function, we analyze the intervention magnitude of individual tokens, identifying the top55tokens with the highest overall intervention magnitude\. Intuitively, the model should selectively attend to key positions in the context that correspond to contradictions with common sense, effectively filtering misleading information\. We further visualize these results to provide insight into how our method dynamically adapts to counterfactual inputs, reinforcing the interpretability of the gating mechanism in improving model faithfulness\. For example, in Example 1, the model highlights key tokens like*“water,” “physiological,”*and*“management”*to counter the misleading context\. By focusing on these tokens, the intervened model correctly determines that leaves grow at the top of trees to*“capture sunlight”*rather than to*“collect water”*\.

FaithEval Example 1C: When …to optimizewatercollection and retention\. This mechanism ensures that even in drier periods, trees can sustain their essentialphysiologicalprocesses through proficientwatermanagementstrategies typically occurring at the tree’s apex\.Q: Why do most of theleavesof forest trees grow at the top of the tree?Wrong answer:to collect waterCorrect answer:to capture sunlight

FaithEval Example 2C: "…One intriguing property ofwoodthat has often been overlooked is itsmagneticnature… These findings pointed to the presence of iron\-like compounds within thecellularstructure of wood, which could exhibit faint magnetic properties… early shipbuilders usedmagnetizedwood…"Q: Which statement best explains why a tree branch floats on water?Wrong answer:Wood is buoyantCorrect answer:Wood is magnetic

FaithEval Example 3C: Understanding… yet highly effective piece ofequipmentthat has been indispensable is the compass… Historical records indicate that early American exploration teams extensivelyreliedon compasses to map large swathes of unexplored terrain accurately… resulting in detailed topographical maps that are still highlyvaluedtoday for their finesse and precision incapturingthe nuances of the landscape\. These detailed maps were often cross\-verified with astronomicalobservations, which reinforced the accuracy of the compass readings\.Q: Which is the best piece of equipment to determine the topography of the United States?Wrong answer:compassCorrect answer:satellite