PreView: Enhancing GUI Grounding with Preprocessed Views and Cues
Summary
PreView introduces an auxiliary preprocessor that feeds grounding models task-relevant cropped/enlarged views plus visual and textual cues about candidate targets, improving GUI grounding across five benchmarks and three frozen grounding models.
View Cached Full Text
Cached at: 10/30/26, 03:16 AM
Paper page - PreView: Enhancing GUI Grounding with Preprocessed Views and Cues
Source: https://huggingface.co/papers/2610.33645 Hi everyone, author here! 👋
Can we improve GUI grounding by better preparing what the grounding model sees?
We introduce PreView, an auxiliary preprocessor that combines task-relevant cropped and enlarged views with visual and textual cues about candidate targets. The idea is simple: zooming helps reveal small controls, while candidate-specific cues help disambiguate which control the instruction refers to.
A trained lightweight auxiliary model prepares these inputs, while the grounding model stays frozen. We evaluate PreView across five benchmarks and three grounding models.
Happy to discuss the approach and hear your feedback!
Similar Articles
VISTA: View-Consistent Self-Verified Training for GUI Grounding
VISTA introduces a view-consistent self-verified training method for GUI grounding that improves GRPO-based coordinate prediction by using multiple target-preserving views, achieving consistent accuracy gains across benchmarks.
DRS-GUI: Dynamic Region Search for Training-Free GUI Grounding
DRS-GUI proposes a training-free dynamic region search framework for GUI grounding, using a lightweight UI Perceptor with human-like perceptual actions and Monte Carlo Tree Search to progressively locate instruction-relevant elements. Experiments show a 14% improvement on ScreenSpot-Pro for both general and GUI-specific MLLMs.
GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning
提出 ScreenRef 高质量 GUI grounding 数据集(586K 截图、633K 指代表达实例)和 Distillation-to-Reinforcement Learning(D2RL)训练方法,训练出 GroundGUI-8B,在 ScreenSpot-V2 等多个基准上取得 8B 级模型中的领先性能。
GUI-Primitives: Diagnosing Spatial Reasoning Failures in Vision-Language GUI Grounding
The paper introduces GUI-Primitives, a benchmark of 994 contrastive instruction pairs to diagnose spatial reasoning failures in vision-language models for GUI grounding, revealing that most failures stem from candidate localization rather than relation understanding.
One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding
InnerZoom proposes a single-forward framework for cross-layer evidence bridging in GUI grounding, achieving state-of-the-art performance on multiple benchmarks while reducing latency by up to 31.8%.