Paper page - HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation

Source: https://huggingface.co/papers/2604.28196

Abstract

HERMES++ combines 3D scene understanding and future geometry prediction through BEV representation, LLM-enhanced queries, temporal linking, and joint geometric optimization for autonomous driving applications.

Drivingworld modelsserve as a pivotal technology for autonomous driving by simulating environmental dynamics. However, existing approaches predominantly focus on future scene generation, often overlooking comprehensive3D scene understanding. Conversely, whileLarge Language Models(LLMs) demonstrate impressive reasoning capabilities, they lack the capacity to predict future geometric evolution, creating a significant disparity between semantic interpretation and physical simulation. To bridge this gap, we propose HERMES++, a unified driving world model that integrates3D scene understandingandfuture geometry predictionwithin a single framework. Our approach addresses the distinct requirements of these tasks through synergistic designs. First, aBEV representationconsolidates multi-view spatial information into a structure compatible with LLMs. Second, we introduceLLM-enhanced world queriesto facilitate knowledge transfer from the understanding branch. Third, aCurrent-to-Future Linkis designed to bridge the temporal gap, conditioning geometric evolution on semantic context. Finally, to enforce structural integrity, we employ aJoint Geometric Optimizationstrategy that integrates explicitgeometric constraintswith implicitlatent regularizationto align internal representations with geometry-aware priors. Extensive evaluations on multiple benchmarks validate the effectiveness of our method. HERMES++ achieves strong performance, outperforming specialist approaches in both future point cloud prediction and3D scene understandingtasks. The model and code will be publicly released at https://github.com/H-EmbodVis/HERMESV2.

View arXiv pageView PDFProject pageGitHub40Add to collection

Get this paper in your agent:

hf papers read 2604\.28196

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2604.28196 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2604.28196 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2604.28196 in a Space README.md to link it from this page.

Collections including this paper4