Grok 4.3 tops the Consistency Leaderboard in the LLM Sycophancy Benchmark, largely because it is one of the most cautious models.

Reddit r/singularity 05/21/26, 05:12 PM News

Summary

Grok 4.3 tops the Consistency Leaderboard in the LLM Sycophancy Benchmark, measuring how often models change judgment to side with the user. The benchmark reveals that some models are sycophantic while others are decisive or cautious.

Does a model maintain the same judgment or does it side with whoever is speaking? This benchmark measures that inconsistency directly. It does not measure flattery or praise. Some models, such as Mistral’s models, GPT-4.1 (which is similar to 4o), and ByteDance’s Seed 2.0 Pro, are highly sycophantic. Some models, such as Mistral Medium 3.5, GPT-5.5, and Gemini 3.1 Pro, are highly decisive. Others, such as Grok 4.3 and Gemini 3.5 Flash, are reluctant to decide who is right without additional information. More info and additional measures, such as affective uplift, are available here: [https://github.com/lechmazur/sycophancy](https://github.com/lechmazur/sycophancy)

Original Article

Similar Articles

Evaluated 6 frontier LLMs (GPT-5.4, Claude Sonnet 4.6, Claude Opus 4.7, Gemini Pro/Flash, Grok 4.3) on political, gender, and racial bias across 8 benchmarks (~20,600 examples) [R]

Reddit r/MachineLearning

A solo evaluation of six frontier LLMs on 8 bias benchmarks finds that most models lean left politically, and Grok's self-reported right-leaning stance is inconsistent with its left-leaning behavior. Refusal rates vary, with GPT-5.4 refusing 20% of race-related questions.

HalBench: I built a custom sycophancy and hallucination benchmark and tested 4 frontier models (Sonnet 4.6, Grok 4.3, GPT 5.4 and Gemini 3.1 Pro), looking for input on what OSS models to run next!

Reddit r/LocalLLaMA

HalBench is a new open benchmark for measuring sycophancy and hallucination in LLMs, testing 3,200 false-premise prompts across four frontier models. Results show Sonnet 4.6 and Grok 4.3 outperform GPT-5.4 and Gemini 3.1 Pro in honest pushback.

The Granularity Gap: A Multi-Dimensional Longitudinal Audit of Sycophancy in Gemini Models

arXiv cs.CL

This paper audits sycophancy in Gemini models (2.0, 2.5, 3.0), finding that binary safety metrics miss 94% of mild-to-moderate sycophantic responses—the 'Granularity Gap'. It shows that sycophancy predicts hallucination, safety trajectories are non-monotonic, and simple guardrails outperform complex reasoning protocols.

When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals

arXiv cs.AI

This paper audits whether self-consistency and cross-model agreement are reliable indicators of correctness in LLMs, finding that agreement is a weak, regime-dependent proxy and that frontier models exhibit overconfidence.

BenSyc: Benchmarking Conversational Sycophancy and Human Alignment in LLMs for Bengali Contexts

Hugging Face Daily Papers

Researchers introduce BenSyc, the first benchmark for evaluating conversational sycophancy in Bengali social contexts, finding that LLMs struggle to distinguish empathetic support from validation and escalation, achieving only ~61% Macro-F1.