Paper page - Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Driving

Source: https://huggingface.co/papers/2605.22809 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Sensor2Sensor generates high-fidelity multi-modal sensor data from in-the-wild dashcam videos using diffusion models and 4D Gaussian Splatting for autonomous driving system training and validation.

Robust training and validation ofAutonomous Driving Systems(ADS) require massive, diverse datasets. Proprietary data collected by Autonomous Vehicle (AV) fleets, while high-fidelity, are limited in scale, diversity of sensor configurations, as well as geographic and long-tail-behavioral coverage. In contrast, in-the-wild data from sources like dashcams offers immense scale and diversity, capturing critical long-tail scenarios and novel environments. However, this unstructured,in-the-wild video datais incompatible with ADS expecting structured, multi-modal sensor inputs for validation and training. To bridge this data gap, we propose Sensor2Sensor, a novelgenerative modelingparadigm that translates in-the-wild monocular dashcam videos into a high-fidelity,multi-modal sensor suite(AV logs) comprisingmulti-view camera imagesandLiDAR point clouds. A core challenge is the lack of paired training data. We address this by converting real AV logs into dashcam-style videos via4D Gaussian Splatting(4DGS) reconstruction and novel-view rendering. Sensor2Sensor then utilizes adiffusion architectureto perform the generative conversion. We perform comprehensive quantitative evaluations on the fidelity and realism of the generated sensor data. We demonstrate Sensor2Sensor’s practical utility by converting challenging in-the-wild internet and dashcam footage into realistic, multi-modal data formats, further unlocking vast external data sources for AV development.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2605\.22809

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.22809 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.22809 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.22809 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.