Tag
Introduces Blind-Spots-Bench, a benchmark designed to expose persistent failures in modern multimodal AI models on tasks that are trivial for humans. Evaluates a range of models, revealing performance gaps and that no single model dominates across all task types.
AI is enabling the collection and processing of previously inaccessible data from the physical world through cheaper sensors, robotics, and multimodal models, creating new data flywheels in infrastructure, healthcare, and industrial automation.
Introduces SciDraw-Bench, a benchmark for evaluating scientific figure generation by text-to-image and multimodal models, with a four-dimensional evaluation protocol. Findings show domain-specific systems outperform general-purpose models, with text fidelity remaining the hardest challenge.
AQuaUI is a training-free inference-time token reduction method for GUI agent models that uses adaptive quadtrees to reduce spatial redundancy in screenshots, achieving up to 13.22% speedup and 29.52% fewer visual tokens while retaining 99.06% of performance.
The Visual Aesthetic Benchmark (VAB) evaluates multimodal models' ability to judge aesthetics through comparative selection, revealing significant gaps versus human experts and showing that fine-tuning on expert examples improves accuracy.
This paper introduces UNO, an Understanding-Oriented Post-Training framework that uses comprehension tasks as supervisory signals to enhance image generation and editing in unified multimodal models.
Researchers introduce GSI-Bench, the first benchmark to quantify generative spatial intelligence in multimodal models by evaluating 3D spatial constraint compliance during image generation. Fine-tuning on their synthetic dataset boosts both spatial editing fidelity and downstream spatial understanding, showing generative training can strengthen spatial reasoning.