vlms

This survey paper reviews decoding methods for large language models and vision-language models, focusing on inference-time approaches to enhance efficiency and control generation. It identifies paradigms, highlights challenges, and discusses future research directions.

This paper introduces Spatial Memory Agent (SMA), a runtime framework that improves frozen vision-language models' spatial reasoning through verifier-guided reflection and reusable memory without parameter updates or external tools, achieving strong results across five benchmarks and four base VLMs.

Microsoft Research introduces MindTopo, a benchmark that tests whether multimodal AI models understand topological relationships like connectivity, enclosure, and knots. Current models perform well on static recognition but struggle with interactive planning tasks, revealing a key gap for robotics and interactive environments.

A benchmark comparing 8 PDF parsers across 14 capabilities, finding Chandra the most accurate while noting trade-offs like speed; LightOnOCR-1B impresses for its size but hallucinates on illegible text.

GST-Bench is a new VQA benchmark for evaluating global spatial awareness in video understanding, testing whether VLMs can build coherent global scene representations from long-horizon egocentric video. Evaluation of 22 state-of-the-art VLMs shows a large gap versus humans, with the best model scoring 42.68 vs 79.08.

OSReward introduces a standardized benchmark for evaluating VLM judges on computer-use agent trajectories, revealing that even state-of-the-art models have systematic leniency bias. The authors release OS-Shepherd-9B and 35B reward models that provide reliable, low-cost rewards for CUA tasks.

LlamaIndex announces improved routing in LlamaParse that selects lightweight parsing for simple text-heavy PDFs and heavier VLM-based parsing for complex pages with tables or charts, optimizing cost and accuracy.

Z-Reward is a teacher-student framework that decouples complex reasoning from efficient reward deployment for text-to-image training. It achieves 89.6% human preference accuracy with a 27B teacher and 88.6% with a 9B student, outperforming prior methods.

Jerry Liu's team is presenting ParseBench, a comprehensive document understanding benchmark for VLMs, at CVPR 2026. The benchmark includes 2,000 pages of real-world enterprise documents with evaluation metrics for tables, charts, and visual grounding.

This paper introduces a nested geometric decomposition framework to analyze how prompting reorganizes internal representations in large language and vision-language models. The authors show that affine transformations, particularly cross-dimensional linear mixing, are key to explaining prompt-induced behavioral changes.

Fast-dDrive is a block-diffusion VLA model for end-to-end autonomous driving that achieves state-of-the-art trajectory accuracy while delivering over 12x throughput speedup over autoregressive baselines, addressing the trade-off between high-fidelity planning and efficient inference for edge deployment.

This paper presents SpaceNum, a unified framework to evaluate how vision-language models (VLMs) understand numerical values in spatial contexts, finding that current models largely fail to ground numbers spatially and often perform close to random guessing.

GridProbe is a training-free inference paradigm for Long-Video VLMs that adaptively selects relevant frames using posterior probing, achieving sub-quadratic attention costs with minimal accuracy loss.

GeoStack introduces a geometric framework to compose independently trained domain experts in Vision-Language Models without catastrophic forgetting, achieving constant-time inference and a 10x reduction in geometric error.

Researchers from MIT, WPI, and Google propose WRING, a novel post-processing debiasing method for Vision-Language Models that avoids the 'Whac-a-mole dilemma' of amplifying other biases when removing specific ones.