Zero-Shot Mission-Level Evaluation for Aerial MLLM Agents
Summary
MissionBench is a new benchmark for evaluating multimodal large language models (MLLMs) on long-horizon embodied tasks in aerial 3D environments, revealing that even the best models succeed on fewer than 35% of missions compared to 84.4% human performance.
View Cached Full Text
Cached at: 07/27/26, 07:42 AM
# Zero-Shot Mission-Level Evaluation for Aerial MLLM Agents Source: [https://arxiv.org/abs/2607.22014](https://arxiv.org/abs/2607.22014) [View PDF](https://arxiv.org/pdf/2607.22014) > Abstract:Multimodal Large Language Models \(MLLMs\) are emerging as core reasoning modules for embodied agents, yet it remains unclear how well general\-purpose models can solve long\-horizon embodied tasks from a single high\-level instruction\. We introduce MissionBench, a benchmark for mission\-level evaluation of MLLMs in aerial 3D environments\. It comprises 120 missions across five simulated 3D environments and four task families\. Agents must autonomously plan, navigate, and report outcomes using only egocentric observations and its action history, without aerial\-specific fine\-tuning\. Across 22 open\- and closed\-source MLLMs, the strongest model succeeds on fewer than 35% of missions compared to 84\.4% human performance, highlighting the difficulty of multi\-step embodied tasks\. Despite large variations between model families, we observe gains from scaling, indicating that larger general\-purpose models possess stronger zero\-shot embodied capabilities\. Our analysis shows that mission\-level competence requires coordinating multiple capabilities beyond spatial perception, including multi\-step planning and adaptive reasoning\. This motivates closed\-loop evaluation and highlights both the promise and risk of scaling\-driven improvements for embodied AI\. ## Submission history From: Jona Ruthardt \[[view email](https://arxiv.org/show-email/66a745b9/2607.22014)\] **\[v1\]**Fri, 24 Jul 2026 06:22:50 UTC \(17,719 KB\)
Similar Articles
MuseBench: Benchmarking Intent-Level Audiovisual Arts Understanding in MLLMs
MuseBench is a comprehensive benchmark introduced to evaluate multimodal large language models on nuanced, intent-level understanding of audiovisual arts, revealing that even the best model achieves only 48.29% accuracy compared to 87.18% for human experts.
MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs
MLUBench is a large-scale benchmark for lifelong unlearning in multimodal large language models (MLLMs), featuring 127 entities across 9 classes. The paper identifies that existing unlearning methods suffer from cumulative degradation and proposes LUMoE to mitigate this, showing significant improvements.
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
OpenAI introduces MLE-bench, a benchmark of 75 Kaggle ML competitions to evaluate AI agents on real-world ML engineering tasks. The best setup, o1-preview with AIDE scaffolding, achieves at least a Kaggle bronze medal in 16.9% of competitions.
PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
PerceptionBench is a benchmark designed to evaluate atomic visual perception capabilities of Multimodal Large Language Models (MLLMs), using a bottom-up taxonomy of ten atomic perceptual capabilities. Results across 16 frontier MLLMs show no model reaches 60% accuracy, indicating visual perception remains largely unsolved.
Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videos
Artifact-Bench is a comprehensive benchmark that evaluates multimodal large language models on detecting and analyzing artifacts in AI-generated videos, revealing significant limitations and misalignment with human perception.