Tag
Principia is a benchmark that evaluates video models on Newtonian physics using relational consistency between paired objects, revealing significant gaps in current models' physical reasoning.
Code-as-World has been open-sourced, introducing a framework that uses code as executable representations for the physical world to enhance AI physical reasoning, along with pre-trained models and evaluation tools.
Code-as-World introduces a paradigm that represents physical environments as executable code to enable quantitative reasoning for vision-language models, using an agentic discovery loop for scalable supervision and achieving state-of-the-art performance.
This paper proposes Answer-Level Trust Selection (ATS), a post-hoc, model-agnostic framework for assessing the reliability of individual predictions from vision-language models in quantitative physical reasoning tasks.
VAORA proposes a novel reward design for vision-language models that aligns reasoning with visual contexts and action outcomes, improving physical reasoning and generalization across unseen tasks and environments.
Introduces EVLA, a framework that enhances vision-language driving assistants with real-time awareness of electrified powertrain states, enabling energy-optimal and physically grounded decisions.
This paper identifies a failure mode called PhysHack in LLM-based LEGO assembly generation and proposes PVPO, a sample-efficient reinforcement learning method with model-based data selection that improves physical and semantic alignment using only a small fraction of training data.
WorldOlympiad presents a comprehensive benchmark for evaluating video-based world models across physical faithfulness, geometric consistency, and interaction fidelity, revealing significant gaps in current generative models.
Dream.exe proposes an evaluation framework that uses robotic manipulation tasks to assess video generation models' understanding of physical reality, finding that visual quality does not predict executable motion accuracy.
BilliardPhys-Bench is a new benchmark that tests multimodal LLMs on physical reasoning using synthetic billiards scenarios, requiring predictions of collisions and final ball positions. The paper finds that current models struggle with longer simulations and exhibit a 'stasis bias' of predicting no interaction when uncertain.
This paper introduces MM-CreativityBench, a benchmark for evaluating creative tool use in large multimodal models under physically constrained environments, and proposes affordance-grounded alignment using Direct Preference Optimization to reduce hallucination and improve grounded reasoning.
KinDER is a new open-source benchmark for physical reasoning in robotics, featuring procedurally generated environments and baselines to evaluate kinematic and dynamic constraint challenges.