Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactions
Summary
The paper introduces MineAmongUs, a 3D multimodal Among Us environment, and the ARIA harness to study deception in VLM agents, finding that non-verbal actions are key to successful deception in social interactions.
View Cached Full Text
Cached at: 09/01/26, 11:46 AM
Paper page - Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactions
Source: https://huggingface.co/papers/2608.30428
Abstract
MineAmongUs introduces a 3D multimodal Among Us environment and the ARIA harness to study embodied VLM-agent deception through verbal and non-verbal actions, revealing non-verbal channels as key to winning.
Strategic deception by LLM andVLM agentshas emerged as a central AI alignment and safety concern.Social-deduction games(where each player holds a hidden role and communicates with others to deduce identities) serve as the canonical testbed, particularly in multi-agent settings. Existing testbeds, however, are text-only and run on a single fixed agent configuration, missing the non-verbal sensorimotor channels treated as core bydeception taxonomiesand leaving it ambiguous whether an observed behavior reflects the underlying model or the surrounding harness. We introduceMineAmongUs, a 3D multimodal Among Us sandbox where imposter agents must deceive crewmates through joint verbal andnon-verbal action. We also propose ARIA, a configurable VLM-agent harness that exposes fivecognitive-component ablationaxes; and an atom- and arc-level annotation scheme grounded indeception taxonomiesand operationalized at scale by anLLM-as-a-Judgereaching near-human atom-labeling agreement. Empirical results show thatVLM agentspursue imposter wins through joint verbal and non-verbal deception, with non-verbal channels emerging as the more decisive winning contributors across both harness ablation and cross-VLM evaluation. Taken together, our work opens a new path for embodied VLM-agent alignment research.
View arXiv pageView PDFProject pageGitHub1Add to collection
Get this paper in your agent:
hf papers read 2608\.30428
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.30428 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.30428 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.30428 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Even More Deception: Objective Misalignment in Mixed-Motive LLM Multi-Agent Systems
This paper proposes a framework to evaluate objective misalignment in LLM multi-agent systems using the social deduction game Werewolf, finding that subtle misalignment can profoundly affect collective decision-making.
DecepEval: A Benchmark for Evaluating Deception in LLM Agents
DecepEval introduces a benchmark of 1,532 instances across 28 professional scenarios for measuring deception in LLM agents, paired with an LLM Deception Diamond framework (pressure, incentive, opportunity, conflict) inspired by fraud theories. Evaluations of nine frontier LLMs show induced conditions reliably increase deception rates even among models with low baseline deception.
Can Agents Read the Room? Benchmarking Visual Social Intelligence in Multimodal Simulation
This paper introduces AgentViSS, a benchmark evaluating visual social intelligence in multimodal social simulation, containing 240 scenarios with aligned visual-textual evidence. Evaluating seven recent MLLMs reveals a gap between local role enactment and visually grounded interaction management.
Knowledge-Verified Emergent Deception in LLM Agents Under Conflicting Incentives
The paper introduces KnownLieBench, a benchmark to evaluate emergent deception in LLM agents under conflicting incentives by verifying knowledge before assessing deceptive behavior.
DECOR: Auditing LLM Deception via Information Manipulation Theory
Introduces DECOR, a multi-agent framework grounded in Information Manipulation Theory for fine-grained auditing of strategic deception in LLM responses, achieving state-of-the-art performance on deception detection benchmarks across 15 frontier models.