Hallucination in World Models is Predictable and Preventable
Summary
This paper demonstrates that hallucinations in world models are a data coverage issue, and proposes data-centric signals to detect them and coverage-aware sampling techniques to mitigate them, using the new MMBench2 dataset.
View Cached Full Text
Cached at: 06/26/26, 06:05 AM
Paper page - Hallucination in World Models is Predictable and Preventable
Source: https://huggingface.co/papers/2606.27326
Abstract
World models exhibit hallucinations in low-data regions of state-action space, which can be detected and mitigated using data-centric signals and coverage-aware sampling techniques.
Modern generativeworld modelsrender increasingly realistic action-controllable futures, yet they frequently hallucinate: rollouts remain visually fluent while drifting from the ground-truth dynamics. We hypothesize thathallucinationconcentrates in low-coverage regions of thestate-action space, where lightweightdata-centric signalscan both detect it and guide mitigation. To test this, we introduce MMBench2, a 427-hour, 210-task dataset forvisual world modelingwithground-truth actions,rewards, and live simulators, and train a 350M-parameter world model on it. We identify three distincthallucinationmodes: perceptual, action-marginalized, and scene-diverging -- each anchored to a different stage of the pipeline, and develop three signals that accurately predict where the model will fail. To close coverage gaps at training time, we develop acoverage-aware samplingtechnique; to close them online, ourhallucinationpredictors serve ascuriosity rewardsfor targeted data collection, yielding a data-efficient finetuning recipe that adapts the pretrained world model to entirely unseen environments with as few as 50 real environment trajectories. Overall, our findings reveal thathallucinationinworld modelsis inherently a data coverage issue, and that the same signals used to detect it can also be used for mitigation. An interactive web version of our paper is available at https://www.nicklashansen.com/mmbench2
View arXiv pageView PDFProject pageGitHub5Add to collection
Get this paper in your agent:
hf papers read 2606\.27326
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.27326 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.27326 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.27326 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models
HalluWorld is a controlled benchmark framework for evaluating hallucination in large language models using explicit reference world models across synthetic environments like gridworlds, chess, and realistic terminal tasks. It enables fine-grained analysis of failure modes such as perceptual hallucination, multi-step state tracking, and causal simulation, revealing that frontier models still struggle with complex reasoning not solved by extended thinking.
From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data
This paper analyzes hallucination in large language models as a structural consequence of three architectural decisions: self-attention's co-occurrence learning, maximum likelihood estimation training objective, and autoregressive decoding's left-to-right commitment. It maps each mechanism to specific hallucination types and argues that dataset pathologies amplify but do not cause these vulnerabilities.
PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts
This paper reveals that much of the reported progress in LLM hallucination detection is due to benchmark construction artifacts, where ground-truth answers are embedded in prompts, allowing a simple text-similarity baseline to achieve near-perfect scores. Through a large-scale controlled evaluation, the authors show that most methods perform near chance under proper controls, except for supervised probes on upper-layer hidden states such as SAPLMA and their proposed DRIFT.
Hallucination Detection via Activations of Open-Weight Proxy Analyzers
This paper introduces a proxy-analyzer framework that detects hallucinations in large language models by analyzing internal activations of small, open-weight models rather than the generator itself. The method achieves superior performance on benchmarks like RAGTruth compared to existing methods like ReDeEP, demonstrating that model size is less critical than the analysis approach.
Quickest Detection of Hallucination Onset: Delay Bounds and Learned CUSUM Statistics
Reformulates token-level hallucination detection as a quickest change detection problem, establishing theoretical lower bounds on detection delay and showing that causal recurrent models achieve near-optimal performance, outperforming linear baselines.