PreView: Enhancing GUI Grounding with Preprocessed Views and Cues

Hugging Face Daily Papers Papers

Summary

PreView introduces an auxiliary preprocessor that feeds grounding models task-relevant cropped/enlarged views plus visual and textual cues about candidate targets, improving GUI grounding across five benchmarks and three frozen grounding models.

GUI grounding requires locating small controls in cluttered screenshots and identifying which control a natural-language instruction refers to. Cropping and enlargement can expose local detail, but the resulting image may still leave ambiguity about the intended target. We propose PreView, an auxiliary preprocessor that improves the inputs to an existing grounding model without further training that model. PreView prepares two complementary types of input features: task-relevant views for localization and candidate-specific cues for target understanding. A trained lightweight auxiliary model predicts and refines a center, crops the screenshot into a local view, and enlarges that view for zoomed-in grounding. It also prepares a visual cue (i.e., a red-circle marker with a location hint) and a textual cue describing the candidate's identity or function. The frozen grounding model uses the prepared inputs to predict the final click. Across five benchmarks, PreView improves the average grounding accuracy of Qwen3-VL-2B, OS-Atlas-4B, and Qwen3-VL-4B without further training these grounding models. Extensive analyses examine the contributions of views and cues, center refinement, and visual input budgets.
Original Article
View Cached Full Text

Cached at: 10/30/26, 03:16 AM

Paper page - PreView: Enhancing GUI Grounding with Preprocessed Views and Cues

Source: https://huggingface.co/papers/2610.33645 Hi everyone, author here! 👋

Can we improve GUI grounding by better preparing what the grounding model sees?

We introduce PreView, an auxiliary preprocessor that combines task-relevant cropped and enlarged views with visual and textual cues about candidate targets. The idea is simple: zooming helps reveal small controls, while candidate-specific cues help disambiguate which control the instruction refers to.

A trained lightweight auxiliary model prepares these inputs, while the grounding model stays frozen. We evaluate PreView across five benchmarks and three grounding models.

Happy to discuss the approach and hear your feedback!

Similar Articles

VISTA: View-Consistent Self-Verified Training for GUI Grounding

Hugging Face Daily Papers

VISTA introduces a view-consistent self-verified training method for GUI grounding that improves GRPO-based coordinate prediction by using multiple target-preserving views, achieving consistent accuracy gains across benchmarks.

DRS-GUI: Dynamic Region Search for Training-Free GUI Grounding

arXiv cs.AI

DRS-GUI proposes a training-free dynamic region search framework for GUI grounding, using a lightweight UI Perceptor with human-like perceptual actions and Monte Carlo Tree Search to progressively locate instruction-relevant elements. Experiments show a 14% improvement on ScreenSpot-Pro for both general and GUI-specific MLLMs.