MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding
Summary
This paper introduces MechVQA, a dataset with 3.3k high-density mechanical engineering drawings and 21k question-answer pairs, along with the MechVL model that outperforms existing baselines by 7.57 percentage points on the MechVQA total score, advancing multimodal LLM understanding of mechanical drawings.
View Cached Full Text
Cached at: 06/05/26, 10:07 AM
Paper page - MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding
Source: https://huggingface.co/papers/2605.30794
Abstract
Mechanical engineering drawing understanding is improved through a specialized dataset and domain-specific model that outperforms existing baselines by leveraging multi-stage training and high-density visual question answering annotations.
Multimodal Large Language Models(MLLMs) have demonstrated significant achievements in generalvisual question answering(VQA) tasks. However, they remain brittle onmechanical engineering drawings, where high annotation density and weakdomain knowledge, compounded by unreliablespatial relation reasoningunder strictprojection rulesandgeometric constraints, make decisive cues easy to miss and frequently lead to wrong answers. To bridge this gap, we introduce the first comprehensive mechanical drawing understanding dataset, MechVQA, created through a semi-automated construction and quality-control pipeline. MechVQA contains 3.3k high-density pictures with 21K question-answer pairs, spanning 10 different fine-grained tasks across three capability levels: Recognition, Reasoning, and Judging, providing a testbed to evaluate and improve MLLM understanding on real-world mechanical drawings. On top of MechVQA, we then develop theMechVL modelthrough amulti-stage training paradigm, building a strong domain-specialized baseline. Extensive experimental results demonstrate that MechVL outperforms the strongest closed-source baseline by 7.57 percentage points on the MechVQA total score, significantly enhancing mechanical drawing understanding ability and providing a reusable foundation for deploying MLLMs in mechanical design and inspection scenarios.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2605\.30794
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.30794 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.30794 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.30794 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Do VLMs Reason Like Engineers? A Benchmark and a Stage-wise Evaluation
This paper introduces EngVQA, a multimodal benchmark for evaluating engineering reasoning in vision-language models, along with an 8-stage automatic evaluation framework that enables fine-grained analysis of reasoning failures. It reveals substantial limitations in current VLMs' engineering reasoning capabilities.
Benchmarking Visual State Tracking in Multimodal Video Understanding
Introduces VSTAT, a benchmark for evaluating visual state tracking in multimodal large language models (MLLMs) using 834 clips and 1,500 questions. Current MLLMs perform poorly compared to humans, failing at visual perception rather than reasoning.
VAMPS: Visual-Assisted Mathematical Problem Solving Benchmark
VAMPS is a new benchmark of 1,168 multimodal bilingual math problems designed to evaluate whether LLMs can benefit from constructing and reasoning over graphs/visualizations. Key finding: direct analytical solving surprisingly outperforms tool-enabled visual solving even on problems where plotting is a natural strategy.
Mind's Eye: A Benchmark of Visual Abstraction, Transformation and Composition for Multimodal LLMs
Researchers introduce Mind’s Eye, a benchmark of eight visual-cognitive tasks that reveals top multimodal LLMs score under 50% while humans reach 80%, exposing major gaps in visual abstraction, relation mapping and mental transformation.
BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs
BilliardPhys-Bench is a new benchmark that tests multimodal LLMs on physical reasoning using synthetic billiards scenarios, requiring predictions of collisions and final ball positions. The paper finds that current models struggle with longer simulations and exhibit a 'stasis bias' of predicting no interaction when uncertain.