vqa

Introduces DrawingVQA, the first benchmark for evaluating multimodal large language models on real-world construction drawings, with 33 drawings and 92 QA pairs across three reasoning depths, revealing a gap between model and expert performance.

Promotes Gemini 3.5 Flash as a faster, cheaper, and more accurate model for OCR and VQA tasks.

This paper audits knowledge-based VQA benchmarks, revealing systematic violations of assumptions that make accuracy a misleading metric. It introduces a repair protocol and multi-entity augmentation to restore answer derivability and question clarity, showing that corrected settings yield markedly different model rankings.

This paper studies how self-driving car systems and humans perform on visual question answering tasks across different geographic locations (Lima and New York City), finding that both humans and VLMs show similar performance regardless of location but diverge based on question type.

OpenMedQ is a fully-open medical vision-language model pretrained on 14 datasets (~3.35M samples), achieving state-of-the-art results on medical VQA and classification benchmarks.

This paper presents a method for distilling answer-set programming rules from large language models to enhance neurosymbolic visual question answering, showing that only a few examples are needed to generate correct rules.

This paper introduces UniKE, the first benchmark for cross-modal knowledge editing in unified multimodal models (UMMs), revealing a significant modality gap where text edits achieve 92% efficacy but only 18.5% transfer to image generation. It proposes Reasoning-augmented Parameter Editing to improve cross-modal transfer, with gains up to 18.6 percentage points.

HakushoBench is a Japanese chart and table VQA benchmark built from governmental white papers to evaluate vision-language models' understanding of complex visual data, challenging open-weight models with a 58.6% accuracy and a 34.9-point gap to proprietary models.

This paper presents a systematic study of long-context continued pre-training for vision-language models, achieving generalization beyond 128K context with an efficient data mixture design and introducing the MMProLong model.

SGOCR is an open-source dataset pipeline for generating spatially-grounded, OCR-focused visual question answering (VQA) tuples with rich metadata to support diverse VLM training. The pipeline uses a multi-stage approach combining models like Nvidia's nemotron-ocr-v2, Gemma4, Qwen3-VL, and Gemini-2.5-Flash, along with an agentic optimization loop.