instruction-grounded

SleepWalk is a three-tier benchmark for evaluating vision-language models' ability to predict spatially coherent trajectories in 3D environments from textual instructions and visual observations, revealing systematic failures in grounded spatial reasoning under occlusions and multi-step instructions.