GridProbe: Posterior-Probing for Adaptive Test-Time Compute in Long-Video VLMs
Summary
GridProbe is a training-free inference paradigm for Long-Video VLMs that adaptively selects relevant frames using posterior probing, achieving sub-quadratic attention costs with minimal accuracy loss.
View Cached Full Text
Cached at: 05/13/26, 12:19 AM
Paper page - GridProbe: Posterior-Probing for Adaptive Test-Time Compute in Long-Video VLMs
Source: https://huggingface.co/papers/2605.10762
Abstract
GridProbe enables efficient long-video understanding by adaptively selecting relevant frames using a frozen VLM’s reasoning, achieving sub-quadratic attention cost with minimal accuracy loss through shape-adaptive selection and interpretable importance maps.
Long-video understanding in VLMs is bottlenecked by a single monolithic forward pass over thousands of frames at quadraticattention cost. A common mitigation is to first select a small subset of informative frames before the forward pass; common for training-free selectors via auxiliary encoder-space similarities. Such signals are capped by contrastive pretraining, which usually fails on reasoning-heavy queries (negation, cross-frame counting, holistic summarization). We propose GridProbe, an efficient training-freeposterior-probing inferenceparadigm that scores evidence in answer space using a frozen VLM’s own reasoning and then selectsquestion-relevant framesadaptively, resulting in sub-quadraticattention costwith little to no accuracy loss. We arrange frames on a K{times}K grid and run lightweight row R and column C probes, where each probe reads its peak posterior as a query-conditioned confidence. The outer product of R and C yields an interpretable importance map whose skewness and kurtosis driveShape-Adaptive Selection, a closed-form rule that reliably replaces the fixed frame budget M with a per-question M_{eff}. We show empirically that M_{eff} tracks intrinsic question difficulty without ever seeing the answer, a sign oftest-time adaptive compute. On Video-MME-v2, GridProbe matches the monolithic baseline within 1.6 pp Avg Acc at 3.36times TFLOPs reduction, while on LongVideoBench it Pareto-dominates the baseline (+0.9 pp at 0.35times compute). Because the selector and QA models can be decoupled, pairing a small 2B selector with a stronger 4B or 8B QA is strictly Pareto-dominant over the 2B monolithic baseline (up to +4.0 pp at 0.52times compute, on average), with no retraining. Finally, theinterpretabilityof theimportance mapsopens future avenues for behavioral diagnostics, grounding, and frame-selection distillation.
View arXiv pageView PDFGitHub0Add to collection
Get this paper in your agent:
hf papers read 2605\.10762
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.10762 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.10762 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.10762 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimization
This paper introduces a paradigm where Vision-Language Models (VLMs) act as test-time teachers to guide Video Generation Models (VGMs) via differentiable rewards and LoRA optimization, achieving a 16.7-point average improvement on video reasoning benchmarks.
AnyGroundBench: A Specialized-Domain Benchmark for Video Grounding in Vision-Language Models
Introduces AnyGroundBench, a domain-adaptation benchmark for spatio-temporal video grounding, evaluating 15 VLMs across five specialized domains and finding current models fail in zero-shot and in-context learning adaptation.
Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction
This paper introduces VG-GUIBench, a benchmark to evaluate MLLM-based GUI agents' ability to follow video tutorials, and proposes TASKER, a keyframe extraction method that improves performance on VideoQA and video-guided agentic tasks.
Which Pretraining Paradigm Better Serves Spatial Intelligence? An Empirical Comparison of Vision-Language and Video Generation Models
This paper presents a systematic frozen-feature probing study comparing vision-language models (VLMs) and video generation models (VGMs) on spatial intelligence tasks. It finds that VLMs excel at semantic tagging and instance grouping, while VGMs provide better dense geometry and camera motion signals, and a naive fusion of both yields strong performance across all axes.
LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation
LongLive-2.0 introduces an NVFP4-based parallel infrastructure for long video generation, achieving up to 2.15x training speedup and 1.84x inference speedup with a 5B model reaching 45.7 FPS.