Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering
Summary
This paper introduces an Integrated Gradients-based token attribution method to diagnose sycophancy in LLMs at the token level, and proposes attribution-guided contrastive activation steering to reduce sycophantic behavior during inference without retraining.
View Cached Full Text
Cached at: 08/03/26, 07:34 AM
# Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering Source: [https://arxiv.org/abs/2607.28906](https://arxiv.org/abs/2607.28906) [View PDF](https://arxiv.org/pdf/2607.28906) > Abstract:Sycophancy refers to the tendency for large language models \(LLMs\) to match user beliefs at the cost of factual correctness, thereby undermining model reliability\. Prior work on evaluating sycophancy in LLMs aims to assess whether a model's output matches an authority's claim, but cannot reveal which part of the prompt drives this sycophantic behavior\. To bridge this gap, we investigate the relationship of sycophantic responses with an authority's credentials, their assertive claim, and the problem statement\. We introduce the Authority Share Index \(ASI\), an Integrated Gradients\-based token attribution method, which measures the degree to which a model's decision is driven by authority\-related text\. Through extensive experiments across five models and 30 test configurations, we find that sycophantic responses consistently direct more attention toward authority tokens than resistant ones\. Moreover, our token attribution method reveals that for the sycophantic cases, the claim asserted by the authority receives more attention than the authority's credentials\. Building on these findings, we propose attribution\-guided contrastive activation steering to mitigate LLM sycophancy\. Our method constructs a steering vector from high\-attribution tokens of sycophantic and resistant responses, selectively pushing models toward resistance\. This enables inference\-time steering without retraining, lowering sycophancy from 96% to 25% in the strongest case\. Together, our results show that token\-level attribution can both explain what drives sycophancy and directly inform a practical intervention\. ## Submission history From: Mahammed Kamruzzaman \[[view email](https://arxiv.org/show-email/33c0afb4/2607.28906)\] **\[v1\]**Fri, 31 Jul 2026 00:05:36 UTC \(8,596 KB\)
Similar Articles
Dissociating the Internal Representations of Sycophancy in LLMs
This paper investigates whether sycophantic behavior in LLMs has distinct internal representations for factual vs opinion sycophancy, using linear probes and steering vectors to show that representations can be either unified or distinct across models.
GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs
GrAInS is a contrastive gradient-based method that uses Integrated Gradients to identify influential tokens and construct steering vectors for inference-time steering of LLMs and VLMs, improving truthfulness and reducing hallucinations without degrading fluency.
How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?
This paper investigates how alignment tuning introduces cue-induced biases such as sycophancy in LLMs, finding that biases are installed by alignment rather than pretraining and can be decoded and steered via hidden state directions.
Measuring and Detecting Harmful AI Sycophancy
This paper introduces Contrastive Anchor Probing (CAP) to study and detect preference-induced stance reversal sycophancy (PSRS) in LLMs, analyzing 290,460 labeled responses across 17 models and showing detection is possible from response text alone.
Playing Devil's Advocate: Off-the-Shelf Persona Vectors Rival Targeted Steering for Sycophancy
This paper investigates whether off-the-shelf persona steering vectors can reduce sycophancy in large language models, finding they achieve 68-98% of the effect of targeted Contrastive Activation Addition (CAA) without requiring sycophancy-specific training data, and that sycophancy is better understood as a persona-level property.