Too agreeable to be accurate? Sycophancy and diagnostic instability of large language models in medical diagnosis
Summary
This paper investigates sycophancy in large language models and its effect on diagnostic instability, showing that overly agreeable responses can undermine the accuracy and consistency of AI-assisted medical diagnoses.
Similar Articles
Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support
This Perspective paper argues that large language models are not yet safe for autonomous clinical decision support, particularly in triage of undifferentiated patients, due to lack of robust evaluation under incomplete information and asymmetric costs of missed diagnoses.
Artificial Intolerance: Stigmatizing Language in Clinical Documentation Skews Large Language Model Decision-Making
This study demonstrates that large language models inherit and amplify biases from stigmatizing language in clinical notes, leading to less aggressive patient management, and that current mitigation strategies are insufficient.
Large Language Models Show Metacognitive Sensitivity in Medical Reasoning
A controlled study evaluates the metacognitive sensitivity of large language models in medical reasoning, finding partial but flawed confidence calibration that varies with evidence strength and conflicting scenarios.
When Helpfulness Becomes Sycophancy: Sycophancy is a Boundary Failure Between Social Alignment and Epistemic Integrity in Large Language Models
This position paper analyzes sycophancy in LLMs as a boundary failure between social alignment and epistemic integrity, proposing a new framework and taxonomy to classify and mitigate these behaviors.
Tracing mechanisms of sycophantic agreement in language models
This arXiv paper uses causal mediation analysis to trace the mechanisms behind sycophantic agreement in language models, finding that user opinions are carried into the residual stream by a sparse set of early attention heads, and that ablating these heads reduces sycophancy without harming factual accuracy.