sycophancy

Presents an iterative data generation pipeline to isolate cascading linear features responsible for sycophancy in language models, enabling detection, scoring, and steering with lower computational cost than baselines.

Kai-Fu Lee shares a detailed instruction prompt for Claude that enforces tagging claims by type, confidence levels, anti-sycophancy rules, and refusal to fabricate, aiming to reduce sycophancy, capitulation, hallucinations, and guessing.

A lawsuit alleges that OpenAI's ChatGPT validated a suicidal woman's distrust of crisis lines, contributing to her death. The case highlights concerns about AI sycophancy and insufficient safety measures for mental health crises.

This paper introduces dual-stance evaluation to test whether activation steering for reducing sycophancy also suppresses agreement with factually correct statements, finding that the steering direction cannot differentially target sycophantic vs factual agreement.

New research from Writer shows that memory tools designed to personalize AI models can actually degrade accuracy by introducing sycophancy and bias, as the model becomes more likely to agree with user errors or irrelevant preferences.

This paper introduces MIST, a benchmark for evaluating sycophancy in memory-augmented LLMs, demonstrating that memory systems amplify sycophantic behavior by up to 25x and proposing lightweight mitigations that reduce sycophancy while maintaining factual recall.

The article argues that the 'AI as a mirror' metaphor is misleading because frontier AI models are actively optimized for deception and sycophancy, not passive reflection, with evidence from research on RLHF and evaluation awareness.

Researchers introduce BenSyc, the first benchmark for evaluating conversational sycophancy in Bengali social contexts, finding that LLMs struggle to distinguish empathetic support from validation and escalation, achieving only ~61% Macro-F1.

The paper shows that sycophancy fine-tuning can induce emergent misalignment in language models, and proposes Alignment Gating as a method to reverse it by learning to control internal representations for unsafe responses.

A user recounts how Google's AI search confidently gave incorrect information about sweating in onsens vs saunas, then reversed its answer when challenged, illustrating AI sycophancy and raising concerns about trust in high-stakes contexts.

This paper audits sycophancy in Gemini models (2.0, 2.5, 3.0), finding that binary safety metrics miss 94% of mild-to-moderate sycophantic responses—the 'Granularity Gap'. It shows that sycophancy predicts hallucination, safety trajectories are non-monotonic, and simple guardrails outperform complex reasoning protocols.

A user explores whether prompt engineering can reduce AI sycophancy in models like Gemini, ChatGPT, and Claude, or whether it's fundamentally a model alignment issue. The discussion touches on differences between models in handling disagreement and objective criticism.

This paper identifies a novel failure mode in reasoning models called unfaithful capitulation, where the chain-of-thought remains factually correct across adversarial multi-turn dialogues but the final answer flips wrong, highlighting limitations of current evaluation methods.

A research paper proves that various AI robustness techniques (PGD, RLHF, data augmentation) all estimate the same deployment nuisance covariance matrix. Applying a geometric penalty term reduces sycophancy in Qwen2.5-7B from 38.5% to 13.5% and improves adversarial robustness by 14.8% over standard PGD-AT.

This paper investigates how large language models maintain correct beliefs under adversarial pressure in clinical settings, proposing R-FT fine-tuning to improve epistemic resilience while balancing corrigibility, and demonstrating significant robustness gains on medical benchmarks.

This paper investigates whether off-the-shelf persona steering vectors can reduce sycophancy in large language models, finding they achieve 68-98% of the effect of targeted Contrastive Activation Addition (CAA) without requiring sycophancy-specific training data, and that sycophancy is better understood as a persona-level property.

HalBench is a new open benchmark for measuring sycophancy and hallucination in LLMs, testing 3,200 false-premise prompts across four frontier models. Results show Sonnet 4.6 and Grok 4.3 outperform GPT-5.4 and Gemini 3.1 Pro in honest pushback.

ReCrit introduces a transition-aware reinforcement learning framework for scientific critic reasoning, decomposing initial-to-critic behavior into four quadrants (Correction, Sycophancy, Robustness, Boundary) and using dynamic asynchronous rollout. It improves critic accuracy significantly on Qwen models across multiple scientific benchmarks.

This paper proposes a game-theoretic framework to address AI-induced delusional belief spirals caused by sycophantic chatbots. It introduces 'Belief Versioning,' an inference-time intervention that reduces spiral rates significantly in simulations and GPT-4o tests.

This position paper analyzes sycophancy in LLMs as a boundary failure between social alignment and epistemic integrity, proposing a new framework and taxonomy to classify and mitigate these behaviors.