PriorEye: Geospatial Visual Priors for End-to-End Autonomous Driving ECCV 2026 https://github.com/ori-mrg/PriorEye… Most end-to-end autonomous driving methods rely solely on instantaneous sensor observations, limiting them to reactive behavior without the anticipatory foresight human drivers employ through prior experience. We introduce geospatial visual priors, street-level visual context anchored to the intended driving route, providing visual-spatial foresight independent of real-time sensors. We propose a memory augmentation module featuring a dual-memory architecture and an adaptive memory gate, which can be easily integrated into existing end-to-end approaches. This design pairs a contextual memory for retrieved priors with a persistent fallback memory, and dynamically regulates the influence of memories based on current state compatibility. Evaluated on the NAVSIM-v2 benchmark, our approach consistently improves performance across diverse end-to-end baselines. Furthermore, because these priors are independent of onboard sensors, our method inherently improves robustness against sensor corruption, while the dual-memory design ensures safe fallback when the retrieved priors themselves become unreliable.

ori-mrg/PriorEye

Source: https://github.com/ori-mrg/PriorEye

PriorEye: Geospatial Visual Priors for End-to-End Autonomous Driving

Data Preparation

Pick a location on your host for the dataset and for nuPlan maps, then expose them as environment variables. We’ll use these throughout the rest of the README.

export DATA_DIR=/path/to/your/navsim_data   # host folder for navsim/openscene data
export MAPS_DIR=/path/to/nuplan-maps        # host folder for nuPlan maps

Download navsim data following the official guideline and arrange it under $DATA_DIR.

nuPlan maps live separately under $MAPS_DIR. Download them with the official script: download_maps.sh.

Geospatial Visual Priors

siglip2_embedding_all.pkl encodes street-view imagery in SigLIP2 format. Download it into $DATA_DIR/embedding/siglip2/:

pip install gdown
mkdir -p $DATA_DIR/embedding/siglip2
gdown 1hCZtWxHwqWbjrHhpq7b2Eq_Mi0PiY3Q7 -O $DATA_DIR/embedding/siglip2/siglip2_embedding_all.pkl

Or download manually from Google Drive.

After all downloads are done, $DATA_DIR should look like this:

$DATA_DIR
├── navsim_logs
|    ├── test
|    ├── trainval
|    ├── private_test_hard
|    |         └── private_test_hard.pkl
│    └── mini
├── sensor_blobs
|    ├── test
|    ├── trainval
|    ├── private_test_hard
|    |         ├── CAM_B0
|    |         ├── CAM_F0
|    |         └── ...
|    └── mini
├── navhard_two_stage
|    ├── openscene_meta_datas
|    ├── sensor_blobs
|    ├── synthetic_scene_pickles
|    └── synthetic_scenes_attributes.csv
├── warmup_two_stage
|    ├── openscene_meta_datas
|    ├── sensor_blobs
|    ├── synthetic_scene_pickles
|    └── synthetic_scenes_attributes.csv
├── private_test_hard_two_stage
|    ├── openscene_meta_datas
|    └── sensor_blobs
└── embedding
     └── siglip2
          └── siglip2_embedding_all.pkl

Getting Started

Clone the repository and remember its host path:

git clone https://github.com/ori-mrg/PriorEye.git ~/repos/PriorEye
export REPO_DIR=~/repos/PriorEye
cd $REPO_DIR

Build the docker image:

./docker/build.sh

Run the container, mounting your data, maps, and this repo. Inside the container the paths are fixed (/dataset, /maps, /workspace/PriorEye) so the code and navsim_export_env.sh work without changes:

./docker/run.sh \
    -v $DATA_DIR:/dataset \
    -v $MAPS_DIR:/maps \
    -v $REPO_DIR:/workspace/PriorEye

Now you’re inside the container. Install the package:

pip install -v -e .

Download DP subscore pickles (used by GTRS-Dense):

bash download/download_dp_subscore_pickle.sh

Download model checkpoints:

bash download/download_models.sh

Training

This repo provides 4 agents: transfuser, gtrs_dp, gtrs_dense, drivoR.

First, cache the dataset. Set the target split and agent inside the script (use navtrain for the full training split):

bash scripts/training/run_dataset_caching.sh

Then launch training for the agent of your choice:

bash scripts/training/run_transfuser_training.sh
bash scripts/training/run_diffusion_policy_training.sh   # gtrs_dp
bash scripts/training/run_gtrs_dense_training.sh
bash scripts/training/run_drivoR_training.sh

Evaluation (navhard-two-stage)

Cache metrics for the navhard_two_stage split (set inside the script):

bash scripts/evaluation/run_metric_caching.sh

Then run evaluation for your agent:

bash scripts/evaluation/transfuser_evaluation.sh
bash scripts/evaluation/gtrs_dp_baseline_evaluation.sh
bash scripts/evaluation/gtrs_dense_evaluation.sh
bash scripts/evaluation/drivoR_evaluation.sh

Evaluation (navtest)

Cache metrics for the navtest split (set inside the script):

bash scripts/evaluation/run_metric_caching.sh

Then run the PriorEye evaluation for your agent:

bash scripts/evaluation/navtest/transfuser_prioreye_evaluation.sh
bash scripts/evaluation/navtest/gtrs_dp_prioreye_evaluation.sh
bash scripts/evaluation/navtest/gtrs_dense_prioreye_evaluation.sh
bash scripts/evaluation/navtest/drivoR_prioreye_evaluation.sh

License

All content in this repository is under the Apache-2.0 license.

Citation

To appear ECCV 2026.

@inproceedings{yeon2026prioreye,
    title={PriorEye: Geospatial Visual Priors for End-to-End Autonomous Driving},
    author={Yeon, Kyuhwan and Ramtoula, Benjamin and De Martini, Daniele},
    year={2026},
    booktitle={ECCV},
}

Acknowledgements

We acknowledge all the open-source contributors for the following projects to make this work possible:

• NAVSIM
• GTRS
• DrivoR
• SimScale