Physics-IQ Verified
Summary
This paper presents a systematic audit of the Physics-IQ benchmark for evaluating physical understanding in video generative models, proposing improvements to prompts and scoring to enhance reliability.
View Cached Full Text
Cached at: 06/18/26, 03:55 AM
Paper page - Physics-IQ Verified
Source: https://huggingface.co/papers/2606.18943
Abstract
A systematic evaluation of the Physics-IQ benchmark reveals limitations in measuring physical understanding of video generative models, leading to improvements in prompt quality and sample-level scoring that enhance reliability for assessing physically accurate video generation.
Video generative models( VGMs) have become a new frontier that can be used not just for video generation but for a multitude of downstream tasks, includingworld modeling. To advance these tasks, a good video model must understand the physical reality of the world. Evaluating this understanding is an emerging field and has led to thePhysics-IQ benchmark, which quantifies this explicitly by comparing model-generated videos to real-world videos of physical experiments. In this work, we present a systematic audit of thePhysics-IQ benchmark, expose shortcomings and propose three solutions that sharpen how we can measurephysical understandingof VGMs. Specifically, we improve prompt and ground-truth quality to reduce the influence of confounding factors and further introduce asample-level scoringsystem that weights each sample and metric equally. Our resulting benchmark, Physics-IQ Verified, refines 57.6\% of all samples and improves over 34.8\% of prompts. In a comparison study using six image-to-video generative models, we observe moderate but meaningful ranking changes (Kendall’s τ= 0.46). We hope Physics-IQ Verified advances the community by providing a more reliable signal toward physically accurate VGMs. The code for the benchmark can be accessed at https://github.com/google-deepmind/physics-iq-benchmark
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2606\.18943
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.18943 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.18943 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.18943 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Physics-R1: An Audited Olympiad Corpus and Recipe for Visual Physics Reasoning
This paper audits multimodal physics evaluation pipelines, revealing issues like train-eval contamination, translation drift, and MCQ saturation. It releases new datasets (PhysCorp-A, PhysR1Corp, PhysOlym-A) and a training recipe (Physics-R1) that significantly improves performance on held-out olympiad problems.
Physics Question Scene Graph: Fine-grained Evaluation of Physical Plausibility in Text-to-Video Generation
Physics Question Scene Graph (PQSG) is a hierarchical question-based pipeline using VLMs to evaluate video generation models' physical plausibility with fine-grained violation detection. It introduces the FinePhyEval dataset and shows higher correlation with human judgments than prior work.
Apple-π: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligence
Apple-π is a benchmark that evaluates video generation models on their ability to reason about physical laws through a three-stage protocol: perception, formulation, deduction. It includes 400 videos covering classical mechanics tasks and reveals current models fall short of reliable law-grounded world simulation.
Physics-Audited Agentic Discovery in Scientific Machine Learning
Introduces Physics-Audited Agentic SciML (PA-SciML), a verification-first workflow where LLM agents discover surrogate models and validate them against physics requirements such as boundary conditions and causality, not just error metrics. Numerical examples show improved trustworthiness over error-only baselines.
Physics-informed generative AI for semiconductor manufacturing: Enforcing hard physical constraints in generative models by construction
This paper argues that generative AI for semiconductor manufacturing must enforce hard physical constraints by construction, not via post-hoc filtering, and surveys architectural approaches like physics-informed diffusion and neural-operator priors to achieve physics fidelity.