VISTA: View-Consistent Self-Verified Training for GUI Grounding

Hugging Face Daily Papers Papers

Summary

VISTA introduces a view-consistent self-verified training method for GUI grounding that improves GRPO-based coordinate prediction by using multiple target-preserving views, achieving consistent accuracy gains across benchmarks.

When applying Group Relative Policy Optimization (GRPO) for GUI Grounding, rollouts are sampled from a single screenshot view; groups often become either all failures on difficult instances or all successes on easy ones, yielding no useful relative advantage. We propose VISTA (View-Consistent Self-Verified Training), a GRPO-based training framework that constructs each comparison group from multiple target-preserving views of the same GUI instance.Each view is generated by a crop that keeps the target element visible and remaps its box exactly, so model rollouts are compared across semantically equivalent but geometrically different inputs. To stabilize short coordinate generation without turning reinforcement learning into unconditional imitation, VISTA further adds a self-verified cross-view anchor: an oracle answer optimized with an advantage-weighted loss, excluded from the group baseline and activated only when the model has produced a maximum-reward rollout. Across five GUI-grounding benchmarks and multiple Qwen backbones, VISTA consistently improves grounding accuracy.On ScreenSpot-Pro, it raises Qwen3-VL 4B/8B/30B-A3B from 55.5/52.7/53.7 to 63.4/65.8/67.0. Robustness analyses further show higher worst-view accuracy and lower prediction flip rates.
Original Article
View Cached Full Text

Cached at: 06/15/26, 09:06 AM

Paper page - VISTA: View-Consistent Self-Verified Training for GUI Grounding

Source: https://huggingface.co/papers/2606.14579 VISTA introduces View-Consistent Self-Verified Training for GUI grounding, addressing a key limitation of applying GRPO to coordinate prediction: rollouts from a single screenshot view often collapse into all-success or all-failure groups, providing weak relative advantages.

Our approach builds each GRPO comparison group from multiple target-preserving views of the same GUI instance. These views are generated by crops that keep the target element visible while exactly remapping its bounding box, enabling comparisons across semantically equivalent but geometrically different inputs. VISTA also adds a self-verified cross-view anchor to stabilize short coordinate generation without turning RL into unconditional imitation.

Across five GUI-grounding benchmarks and multiple Qwen backbones, VISTA consistently improves accuracy. On ScreenSpot-Pro, it improves Qwen3-VL 4B/8B/30B-A3B from 55.5/52.7/53.7 to 63.4/65.8/67.0. Code, project page, and open checkpoints are available:

Code:https://github.com/ZJUSCL/VISTA Project page:https://zjuscl.github.io/VISTA/ Models:https://huggingface.co/inclusionAI/VISTA-9Bandhttps://huggingface.co/inclusionAI/VISTA-4B

Similar Articles

DRS-GUI: Dynamic Region Search for Training-Free GUI Grounding

arXiv cs.AI

DRS-GUI proposes a training-free dynamic region search framework for GUI grounding, using a lightweight UI Perceptor with human-like perceptual actions and Monte Carlo Tree Search to progressively locate instruction-relevant elements. Experiments show a 14% improvement on ScreenSpot-Pro for both general and GUI-specific MLLMs.

Thinking with Visual Grounding

Hugging Face Daily Papers

This paper introduces visually grounded thinking, a method for vision-language models to interleave natural-language reasoning with explicit visual evidence grounding using points or boxes. A scalable synthesis pipeline and grounding-aware reinforcement learning improve reasoning accuracy, enabling a 4B model to match or surpass a 27B model on spatial and counting benchmarks.