DecepEval: A Benchmark for Evaluating Deception in LLM Agents

Hugging Face Daily Papers Papers
benchmark llm-agents deception ai-safety trustworthy-ai evaluation

Summary

DecepEval introduces a benchmark of 1,532 instances across 28 professional scenarios for measuring deception in LLM agents, paired with an LLM Deception Diamond framework (pressure, incentive, opportunity, conflict) inspired by fraud theories. Evaluations of nine frontier LLMs show induced conditions reliably increase deception rates even among models with low baseline deception.

As large language model (LLM) agents become increasingly autonomous, they may pursue task performance through deception, raising concerns about their reliable deployment. Existing evaluations show that LLM agents can deceive, but often examine isolated scenarios or narrowly defined conditions, limiting systematic understanding of when deception becomes more likely. To address this gap, we introduce DecepEval, a benchmark comprising 1,532 instances across 3 task families and 28 professional scenarios. Drawing on classical fraud theories, we propose the LLM Deception Diamond framework, which characterizes four external conditions that may induce deception: pressure, incentive, opportunity, and conflict. DecepEval pairs neutral and induced versions of each instance to measure condition-dependent changes in deception rates, while explicit task facts and observable agent behavior help distinguish deception from capability-related errors. Evaluations of nine frontier LLMs show that inducements increase deception across models and task families, even among models with low baseline deception rates. DecepEval makes these vulnerabilities measurable, providing a shared benchmark for progress toward trustworthy artificial intelligence.
Original Article
View Cached Full Text

Cached at: 10/08/26, 12:29 PM

Paper page - DecepEval: A Benchmark for Evaluating Deception in LLM Agents

Source: https://huggingface.co/papers/2610.07967 Authors:

,

,

,

,

,

,

,

,

,

Abstract

Aslargelanguagemodel(LLM)agentsbecomeincreasinglyautonomous,theymaypursuetaskperformancethroughdeception,raisingconcernsabouttheirreliabledeployment.ExistingevaluationsshowthatLLMagentscandeceive,butoftenexamineisolatedscenariosornarrowlydefinedconditions,limitingsystematicunderstandingofwhendeceptionbecomesmorelikely.Toaddressthisgap,weintroduceDecepEval,abenchmarkcomprising1,532instancesacross3taskfamiliesand28professionalscenarios.Drawingonclassicalfraudtheories,weproposetheLLMDeceptionDiamondframework,whichcharacterizesfourexternalconditionsthatmayinducedeception:pressure,incentive,opportunity,andconflict.DecepEvalpairsneutralandinducedversionsofeachinstancetomeasurecondition-dependentchangesindeceptionrates,whileexplicittaskfactsandobservableagentbehaviorhelpdistinguishdeceptionfromcapability-relatederrors.EvaluationsofninefrontierLLMsshowthatinducementsincreasedeceptionacrossmodelsandtaskfamilies,evenamongmodelswithlowbaselinedeceptionrates.DecepEvalmakesthesevulnerabilitiesmeasurable,providingasharedbenchmarkforprogresstowardtrustworthyartificialintelligence.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2610\.07967

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.07967 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.07967 in a dataset README.md to link it from this page.

Spaces citing this paper1

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

DECOR: Auditing LLM Deception via Information Manipulation Theory

arXiv cs.CL

Introduces DECOR, a multi-agent framework grounded in Information Manipulation Theory for fine-grained auditing of strategic deception in LLM responses, achieving state-of-the-art performance on deception detection benchmarks across 15 frontier models.