Does AI Understand Imaging? A Systematic Benchmark of Agentic AI for Computational Imaging Tasks

# Does AI Understand Imaging? A Systematic Benchmark of Agentic AI for Computational Imaging Tasks
Source: [https://arxiv.org/html/2607.07189](https://arxiv.org/html/2607.07189)
Ethan Chung1,∗, Chuanjun Zheng1,∗, Jasper Tan2, Jingxi Li2, Haopeng Zhang1, and Huaijin Chen1∗Equal contribution\. 1University of Hawaii at Manoa\. 2Glass Imaging\.Preprint / work in progress\. Subject to revision; not the final version\.

###### Abstract

Vision–language models \(VLMs\) and agentic AI have shown strong performance on semantic visual tasks, but it remains unclear whether they can handle the physics and inverse problems that underlie computational imaging\. We presentImagingBench, a benchmark of 20 computational imaging tasks spanning five categories: ray and wave optics, image signal processing, inverse reconstruction, computational sensing, and calibration\. ImagingBench evaluates three complementary settings:Expert, fixed expert\-guided inverse reconstruction;Planner, planner\-guided inverse reconstruction; andForward, forward\-system simulation for consistency checking\. We benchmark leading proprietary and open\-source image\-centric multimodal systems, including Gemini, GPT, and Qwen, and compare them with representative task\-specific non\-agentic baselines\. Across tasks, agentic models remain consistently weaker than specialized methods, especially on computational sensing problems such as lensless imaging, event\-based reconstruction, time\-of\-flight imaging, and holography\. Planner guidance provides only modest and inconsistent gains over the fixed\-prompt Expert baseline\. Although the models often generate visually plausible outputs, their reference\-based fidelity remains poor, revealing a substantial gap between semantic visual competence and physically grounded imaging performance\. ImagingBench provides a unified testbed for measuring this gap and tracking progress in agentic AI for computational imaging\.

###### Index Terms:

Computational Photography, Vision\-Language Models, Generative AI, Inverse Problems, Benchmark\.

![Refer to caption](https://arxiv.org/html/2607.07189v1/x1.png)Figure 1:Overview of ImagingBench:Our benchmark comprises 20 subtasks spanning five computational imaging categories\. Each panel shows representative tasks in the corresponding category, illustrating how the benchmark spans the computational imaging pipeline from scene acquisition and optical encoding to sensing, image processing, reconstruction, and calibration\.## 1Introduction

Benchmarks have repeatedly reshaped progress in machine learning by turning broad capabilities into reproducible evaluation programs\. In vision and multimodal reasoning, widely used suites now measure recognition, segmentation, captioning, visual question answering, tool use, physical reasoning, and controllable generation\[[1](https://arxiv.org/html/2607.07189#bib.bib1),[2](https://arxiv.org/html/2607.07189#bib.bib2),[3](https://arxiv.org/html/2607.07189#bib.bib3),[4](https://arxiv.org/html/2607.07189#bib.bib4),[5](https://arxiv.org/html/2607.07189#bib.bib5),[6](https://arxiv.org/html/2607.07189#bib.bib6),[7](https://arxiv.org/html/2607.07189#bib.bib7),[8](https://arxiv.org/html/2607.07189#bib.bib8),[9](https://arxiv.org/html/2607.07189#bib.bib9),[10](https://arxiv.org/html/2607.07189#bib.bib10),[11](https://arxiv.org/html/2607.07189#bib.bib11)\]\. Yet these benchmarks do not directly test a central form of visual competence: whether agentic AI systems can reason about image formation and solve computational imaging problems\.

Computational imaging differs from semantic vision because success depends on optics, sensing, sampling, calibration, degradation, and reconstruction\. Tasks such as denoising, deblurring, demosaicking, HDR recovery, lensless reconstruction, time\-of\-flight restoration, holography, calibration, and imaging\-system design require models to reason about forward operators and solve ill\-posed inverse problems, rather than only assign semantic labels\. Classical and learning\-based computational imaging has made substantial progress through physics\-based modeling, task\-specific optimization, and learned reconstruction pipelines\[[12](https://arxiv.org/html/2607.07189#bib.bib12),[13](https://arxiv.org/html/2607.07189#bib.bib13),[14](https://arxiv.org/html/2607.07189#bib.bib14),[15](https://arxiv.org/html/2607.07189#bib.bib15),[16](https://arxiv.org/html/2607.07189#bib.bib16),[17](https://arxiv.org/html/2607.07189#bib.bib17),[18](https://arxiv.org/html/2607.07189#bib.bib18),[19](https://arxiv.org/html/2607.07189#bib.bib19)\], but benchmark design has not kept pace with the emergence of multimodal foundation models and agents\.

This gap matters because visually plausible outputs can be wrong when performance is governed by sensor physics, optical aberrations, noise statistics, or undersampling\. It also matters for agentic systems: a useful imaging agent should not only produce an image, but also diagnose degradations, propose physically sensible restoration strategies, and remain consistent with the forward imaging process\. Existing general\-vision, medical\-imaging, multimodal\-reasoning, and agent benchmarks do not jointly evaluate these abilities in a unified setting\.

To address this gap, we introduceImaging Benchmark \(ImagingBench\), a benchmark for evaluating agentic AI systems across the computational imaging pipeline\. ImagingBench spans image signal processing, inverse reconstruction, computational sensing, optics\-aware reasoning, and calibration, and evaluates direct execution, planner\-guided execution, and forward\-system simulation under a unified protocol\. Our goal is not to replace broad capability suites or task\-specific leaderboards, but to provide a specialized diagnostic benchmark for measuring whether modern multimodal agents exhibit coherent and transferable imaging competence\. The main contributions of this work are:

- •We introduce a unified benchmark for imaging\-AI competence that spans 20 task groups across five categories, covering both reconstruction fidelity and physics\-aware reasoning\.
- •We define three complementary evaluation settings—Expert \(expert\-guided inversion\), Planner \(planner\-guided inversion\), and Forward \(forward\-consistency evaluation\)—that disentangle execution ability, per\-instance planning, and forward\-model consistency\.
- •We formalize task\-specific evaluation using both raw metrics and normalized aggregate scores\.
- •We benchmark proprietary systems \(Gemini and GPT\) and an open\-source system \(Qwen\), together with representative task\-specific non\-agentic baselines, and analyze where foundation\-model competence transfers or fails under imaging\-specific constraints\.
- •We provide cost, robustness, and failure analyses, including noise sweeps, sampling sweeps, safety\-event logging, and agent\-failure cases, to support reproducible future comparisons\.

TABLE I:Positioning ofImagingBenchrelative to prior benchmark families\.Benchmark familyRepresentative resourcesWhat they evaluate wellWhat they miss for this paperGeneral visionImageNet\[[1](https://arxiv.org/html/2607.07189#bib.bib1)\], PASCAL VOC\[[2](https://arxiv.org/html/2607.07189#bib.bib2)\], COCO\[[3](https://arxiv.org/html/2607.07189#bib.bib3)\], VQA\[[20](https://arxiv.org/html/2607.07189#bib.bib20)\], nocaps\[[21](https://arxiv.org/html/2607.07189#bib.bib21)\]recognition, localization, segmentation, captioning, visual QAlittle or no image\-formation physics, inverse problems, calibration/design, planner–executor evaluationMedical imagingMedMNIST\[[22](https://arxiv.org/html/2607.07189#bib.bib22)\], CheXpert\[[23](https://arxiv.org/html/2607.07189#bib.bib23)\], MIMIC\-CXR\[[24](https://arxiv.org/html/2607.07189#bib.bib24)\], VinDr\-CXR\[[25](https://arxiv.org/html/2607.07189#bib.bib25)\], BraTS\[[26](https://arxiv.org/html/2607.07189#bib.bib26)\], VQA\-RAD\[[27](https://arxiv.org/html/2607.07189#bib.bib27)\], SLAKE\[[28](https://arxiv.org/html/2607.07189#bib.bib28)\], PathVQA\[[29](https://arxiv.org/html/2607.07189#bib.bib29)\], PMC\-VQA\[[30](https://arxiv.org/html/2607.07189#bib.bib30)\], GMAI\-MMBench\[[31](https://arxiv.org/html/2607.07189#bib.bib31)\]diagnosis, report\-aligned learning, segmentation, lesion localization, medical VQAmostly semantic/clinical understanding; limited forward\-model reasoning and low\-level imaging evaluationMultimodal reasoningGLUE\[[4](https://arxiv.org/html/2607.07189#bib.bib4)\], BIG\-bench\[[5](https://arxiv.org/html/2607.07189#bib.bib5)\], MMBench\[[6](https://arxiv.org/html/2607.07189#bib.bib6)\], MMMU\[[7](https://arxiv.org/html/2607.07189#bib.bib7)\]broad capability profiling, reasoning breadth, cross\-task comparisonnot imaging\-specific; limited quantitative image reconstruction evaluationAgents and physical reasoningAgentBench\[[8](https://arxiv.org/html/2607.07189#bib.bib8)\], GAIA\[[9](https://arxiv.org/html/2607.07189#bib.bib9)\], PhysBench\[[10](https://arxiv.org/html/2607.07189#bib.bib10)\]tool use, multi\-step planning, physical understandingnot end\-to\-end imaging pipelines; little task\-specific image fidelity scoringMultimodal generationMMMG\[[11](https://arxiv.org/html/2607.07189#bib.bib11)\]generation controllability and human\-aligned auto\-evaluationnot focused on inverse imaging or imaging\-system reasoningThis workImagingBenchphysical image formation, inverse reconstruction, calibration/design, and planner–executor evaluation in one framework—

## 2Related Work

Benchmarking has repeatedly driven progress by standardizing evaluation across broad capability families\. Vision benchmarks such as ImageNet, PASCAL VOC, COCO, VQA, and nocaps established common protocols for semantic perception and visual question answering\[[1](https://arxiv.org/html/2607.07189#bib.bib1),[2](https://arxiv.org/html/2607.07189#bib.bib2),[3](https://arxiv.org/html/2607.07189#bib.bib3),[20](https://arxiv.org/html/2607.07189#bib.bib20),[21](https://arxiv.org/html/2607.07189#bib.bib21)\]; medical\-imaging benchmarks extended evaluation toward clinical interpretation\[[22](https://arxiv.org/html/2607.07189#bib.bib22),[23](https://arxiv.org/html/2607.07189#bib.bib23),[24](https://arxiv.org/html/2607.07189#bib.bib24),[25](https://arxiv.org/html/2607.07189#bib.bib25),[26](https://arxiv.org/html/2607.07189#bib.bib26),[27](https://arxiv.org/html/2607.07189#bib.bib27),[28](https://arxiv.org/html/2607.07189#bib.bib28),[29](https://arxiv.org/html/2607.07189#bib.bib29),[30](https://arxiv.org/html/2607.07189#bib.bib30),[31](https://arxiv.org/html/2607.07189#bib.bib31)\]; and recent multimodal and agentic suites evaluate reasoning, tool use, physical understanding, and controllable generation\[[4](https://arxiv.org/html/2607.07189#bib.bib4),[5](https://arxiv.org/html/2607.07189#bib.bib5),[6](https://arxiv.org/html/2607.07189#bib.bib6),[7](https://arxiv.org/html/2607.07189#bib.bib7),[8](https://arxiv.org/html/2607.07189#bib.bib8),[9](https://arxiv.org/html/2607.07189#bib.bib9),[10](https://arxiv.org/html/2607.07189#bib.bib10),[32](https://arxiv.org/html/2607.07189#bib.bib32)\]\. However, these benchmark families still leave an important gap for computational imaging\. Our setting requires models to reason jointly about image formation, optics, sensing, inverse reconstruction, calibration, and agentic planning\. High performance on semantic recognition or multimodal QA does not imply competence on physically grounded imaging tasks, where success depends on correctly handling forward operators, degradations, and reconstruction objectives\.

### 2\.1Computational Imaging, Physics\-Aware Restoration, and Optical Design

Classical and learning\-based Computational Imaging \(CI\) has advanced primarily through physics\-grounded forward modeling, inverse methods, and task\-specific neural architectures\[[12](https://arxiv.org/html/2607.07189#bib.bib12),[13](https://arxiv.org/html/2607.07189#bib.bib13),[14](https://arxiv.org/html/2607.07189#bib.bib14),[33](https://arxiv.org/html/2607.07189#bib.bib33)\]\. Recent work has also begun to explore how foundation\-model priors can assist low\-level imaging\. For example, VLMIR injects vision\-language priors into a diffusion restoration pipeline through VLM\-based feature extraction and cross\-attention conditioning\[[34](https://arxiv.org/html/2607.07189#bib.bib34)\]\. In optical engineering, OPTIAGENT introduces an agentic framework for automated optical design, combining an LLM, a curated optical\-design dataset, physics\-driven rewards, and downstream optimization routines\[[35](https://arxiv.org/html/2607.07189#bib.bib35)\]\. These works are important steps toward integrating multimodal foundation models with imaging tasks, but they remain method\-centric and confined to relatively narrow task families\. They do not provide a unified benchmark spanning image formation, inverse recovery, calibration, and planner–executor evaluation\.

### 2\.2General Vision and Medical Imaging Benchmarks

General vision benchmarks have been highly successful at standardizing evaluation for semantic perception tasks\. ImageNet emphasized large\-scale classification and detection; PASCAL VOC standardized recognition and localization; COCO expanded toward instance segmentation and captioning in natural scenes; VQA added language\-conditioned reasoning over images; and nocaps stressed captioning under novel\-object generalization\[[1](https://arxiv.org/html/2607.07189#bib.bib1),[2](https://arxiv.org/html/2607.07189#bib.bib2),[3](https://arxiv.org/html/2607.07189#bib.bib3),[20](https://arxiv.org/html/2607.07189#bib.bib20),[21](https://arxiv.org/html/2607.07189#bib.bib21)\]\. These resources evaluate semantic understanding well, but they provide little or no coverage of forward image formation, inverse imaging, calibration, or design\.

Medical imaging benchmarks similarly focus on clinically meaningful interpretation\. MedMNIST provides a lightweight multi\-dataset biomedical classification suite; CheXpert, MIMIC\-CXR, and VinDr\-CXR support chest X\-ray classification and localization; BraTS is a major benchmark for tumor segmentation; and VQA\-RAD, SLAKE, PathVQA, PMC\-VQA, and GMAI\-MMBench extend evaluation to medical multimodal reasoning and visual question answering\[[22](https://arxiv.org/html/2607.07189#bib.bib22),[23](https://arxiv.org/html/2607.07189#bib.bib23),[24](https://arxiv.org/html/2607.07189#bib.bib24),[25](https://arxiv.org/html/2607.07189#bib.bib25),[26](https://arxiv.org/html/2607.07189#bib.bib26),[27](https://arxiv.org/html/2607.07189#bib.bib27),[28](https://arxiv.org/html/2607.07189#bib.bib28),[29](https://arxiv.org/html/2607.07189#bib.bib29),[30](https://arxiv.org/html/2607.07189#bib.bib30),[31](https://arxiv.org/html/2607.07189#bib.bib31)\]\. Yet these benchmarks are still mostly centered on semantic or clinical interpretation, rather than on reasoning about sensing operators, optical degradation, inverse reconstruction fidelity, or physically consistent forward simulation\.

### 2\.3Multimodal Reasoning, Agents, and Physical Understanding

A parallel line of work studies increasingly general multimodal or agentic capability\. GLUE and BIG\-bench emphasize broad cross\-task evaluation in language and reasoning\[[4](https://arxiv.org/html/2607.07189#bib.bib4),[5](https://arxiv.org/html/2607.07189#bib.bib5)\]\. MMBench and MMMU probe the breadth of multimodal reasoning and perception\[[6](https://arxiv.org/html/2607.07189#bib.bib6),[7](https://arxiv.org/html/2607.07189#bib.bib7)\]\. AgentBench and GAIA evaluate LLMs and multimodal systems as agents in interactive or tool\-using settings\[[8](https://arxiv.org/html/2607.07189#bib.bib8),[9](https://arxiv.org/html/2607.07189#bib.bib9)\]\. PhysBench focuses specifically on physical\-world understanding, showing that current VLMs still struggle with many physically grounded scenarios\[[10](https://arxiv.org/html/2607.07189#bib.bib10)\]\. MMMG studies controllable multimodal generation with human\-aligned automatic evaluation across multiple modality combinations\[[11](https://arxiv.org/html/2607.07189#bib.bib11)\]\. Complementarily, Luo et al\. introduce a text\-to\-image reasoning benchmark centered on knowledge\-image generation across disciplines and educational levels\[[32](https://arxiv.org/html/2607.07189#bib.bib32)\]\. These efforts are highly relevant to our study, especially for planner–executor design and physical reasoning, but they are not imaging\-specific and do not evaluate quantitative inverse reconstruction quality across a diverse computational imaging pipeline\.

### 2\.4How Our Benchmark Differs

Table[I](https://arxiv.org/html/2607.07189#S1.T1)summarizes the distinction between prior benchmark families and our setting\. Relative to general vision benchmarks, our benchmark evaluates physical image formation and inverse recovery rather than only semantic perception\. Relative to medical imaging benchmarks, it emphasizes low\-level imaging physics in addition to multimodal reasoning\. Relative to multimodal reasoning and agent benchmarks, it provides quantitative image\-fidelity evaluation and forward–inverse consistency checks\. Relative to generation benchmarks, it focuses on physically grounded imaging tasks rather than free\-form image generation alone\.

ImagingBench is intended as a specialized diagnostic benchmark rather than a replacement for broad capability suites or task\-specific leaderboards\. Its purpose is to isolate whether frontier multimodal agents can transfer into operator\-aware computational imaging under a unified, reproducible evaluation protocol\.

To the best of our knowledge,ImagingBenchis the first benchmark to jointly evaluate physical image formation, inverse reconstruction, calibration/design, and planner–executor behavior in a single computational imaging framework\.

## 3Benchmark Construction

### 3\.1Evaluated Tasks

TABLE II:Summary of benchmark subtasks, task descriptions, input/output modalities, and evaluation metrics\.TaskDescriptionIn→\\toOutInst\.Eval\.Ray and Wave OpticsLens System DesignDesign a multi\-element optical lens system from textual specifications\.T→\\toT—RMSComp\. Gen\. HolographyReconstruct an image from its computer\-generated hologram \(amplitude and phase\)\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SImage Signal ProcessingDemosaicking \(Hexadeca\)Reconstruct a full\-color image from a hexadeca \(8×88\{\\times\}8super\-pixel\) Bayer mosaic\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SDemosaicking \(Quad\)Reconstruct a full\-color image from a quad \(4×44\{\\times\}4super\-pixel\) Bayer mosaic\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SDemosaicking \(Regular\)Reconstruct a full\-color image from a quad \(2×22\{\\times\}2super\-pixel\) Bayer mosaic\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SHigh Dynamic RangeRecover a well\-exposed image from underexposed raw burst frames\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SWhite BalanceCorrect the color cast of an image captured under a non\-neutral illuminant\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SDenoisingRemove real sensor noise from smartphone camera images\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/S/LInverse ReconstructionSpherical AberrationRemove blur caused by spherical aberration at varying wavefront\-error levels\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SLongitudinal ChromaticCorrect per\-channel defocus from wavelength\-dependent focal\-length shifts\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SMotion BlurRecover a sharp image from real camera or object motion blur\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SRandomly Masked Sub\-SamplingReconstruct a full image from randomly sampled pixels at various sampling ratios\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SComp\. Sensing \(block\)Reconstruct a full image from block\-wise compressed measurements at various ratios\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SSuper\-ResolutionRecover a high\-resolution image from a downsampled low\-resolution input\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SInpaintingFill in masked \(missing\) image regions with plausible content\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SComputational SensingLensless ImagingRecover the scene from a lensless camera measurement\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SLightfield View ExtrapolationSynthesize an extreme off\-center view given clustered sub\-aperture center views\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/S/LEvent\-based Intensity ImagingReconstruct an intensity image from event streams\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/STime\-of\-Flight Depth ImagingReconstruct a depth map from raw time\-of\-flight sensor measurements\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\to![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)100P/SCalibrationCamera CalibrationEstimate intrinsic camera parameters from checkerboard calibration images\.T,![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)→\\toT—P/S
T = text prompt;![[Uncaptioned image]](https://arxiv.org/html/2607.07189v1/x2.png)= image; Inst\. = number of test instances\. P/S = PSNR/SSIM; P/S/L = PSNR/SSIM/LPIPS\.

CI extends conventional digital imaging by jointly designing image formation and reconstruction\. Rather than treating image formation as a fixed front\-end followed by generic post\-processing, CI integrates physics\-driven acquisition \(optics and sensors\) with computational reconstruction and inference\[[12](https://arxiv.org/html/2607.07189#bib.bib12)\]\. In ImagingBench, we evaluate whether Vision\-Language Models \(VLMs\) can meaningfully assist CI workflows\. The benchmark comprises 20 subtasks organized into five categories \(Fig\.[1](https://arxiv.org/html/2607.07189#S0.F1)\), covering the computational imaging pipeline from optical design and computational sensing to image signal processing, inverse reconstruction, and calibration\. The categories are summarized below and detailed in Table[II](https://arxiv.org/html/2607.07189#S3.T2):

- •Ray and Wave Optics:ray\-optics\-based lens design and Computer Generated Holography \(CGH\)\[[13](https://arxiv.org/html/2607.07189#bib.bib13),[14](https://arxiv.org/html/2607.07189#bib.bib14),[36](https://arxiv.org/html/2607.07189#bib.bib36)\]\.
- •Computational sensing:event\-camera intensity image reconstruction\[[37](https://arxiv.org/html/2607.07189#bib.bib37)\], Time of Flight \(ToF\) imaging\[[38](https://arxiv.org/html/2607.07189#bib.bib38)\], lensless reconstruction\[[39](https://arxiv.org/html/2607.07189#bib.bib39)\], and CGH\[[36](https://arxiv.org/html/2607.07189#bib.bib36)\]\.
- •Image signal processing:white balance\[[40](https://arxiv.org/html/2607.07189#bib.bib40)\], demosaicking\[[41](https://arxiv.org/html/2607.07189#bib.bib41)\], denoising\[[18](https://arxiv.org/html/2607.07189#bib.bib18)\], and HDR reconstruction\[[42](https://arxiv.org/html/2607.07189#bib.bib42)\]\.
- •Inverse reconstruction:deconvolution \(optical\-blur removal such as spherical/chromatic aberrations\) and undersampled reconstruction \(compressive sensing\[[43](https://arxiv.org/html/2607.07189#bib.bib43)\], super\-resolution\[[19](https://arxiv.org/html/2607.07189#bib.bib19)\], and inpainting\[[44](https://arxiv.org/html/2607.07189#bib.bib44)\]\)\.
- •Calibration:intrinsic camera\-parameter estimation from checkerboard calibration images\.

For the evaluation, each task familyttcan be represented as

𝒟t=\{\(xi,qi,yi,mi\)\}i=1Nt,\\mathcal\{D\}\_\{t\}=\\\{\(x\_\{i\},q\_\{i\},y\_\{i\},m\_\{i\}\)\\\}\_\{i=1\}^\{N\_\{t\}\},\(1\)wherexix\_\{i\}is the model input,qiq\_\{i\}is an optional text instruction,yiy\_\{i\}is the target output, andmim\_\{i\}contains task\-specific metadata such as sensor parameters, exposure settings, wavelength, sampling ratio, or calibration geometry\. To avoid overloadingxix\_\{i\}, we useziz\_\{i\}to denote the latent scene, clean image, or underlying physical quantity before measurement\. The benchmark inputxix\_\{i\}is generated fromziz\_\{i\}by a task\-dependent forward model, whileyiy\_\{i\}is the desired reconstruction, inferred parameter, or design output\. Fig\.[2](https://arxiv.org/html/2607.07189#S3.F2)shows representative tasks and outputs; the supplementary material provides the detailed formulation for each task\.

![Refer to caption](https://arxiv.org/html/2607.07189v1/figures/input_prompt_results_figure.png)Figure 2:Representative qualitative examples from ImagingBench: For each example, we show the task prompt and representative outputs produced by three frontier agentic AI systems \(GPT, Gemini, and Qwen\), highlighting the breadth of ImagingBench across physically grounded inverse problems and structured imaging\-reasoning tasks\. The examples also illustrate substantial variation in output fidelity, physical plausibility, and task adherence across models\.
### 3\.2Evaluation Protocol

Building on Eq\.[1](https://arxiv.org/html/2607.07189#S3.E1), a modelft\(k\)f\_\{t\}^\{\(k\)\}with parametersθk\\theta\_\{k\}produces

y^i=ft\(k\)​\(xi,qi,mi;θk\)\.\\hat\{y\}\_\{i\}=f\_\{t\}^\{\(k\)\}\(x\_\{i\},q\_\{i\},m\_\{i\};\\theta\_\{k\}\)\.
This notation covers both inverse and forward computational imaging tasks\. For inverse problems,xix\_\{i\}is typically a degraded measurement andyiy\_\{i\}is the desired reconstruction\. For forward tasks,xix\_\{i\}may instead be a clean image or latent signal, andyiy\_\{i\}is the corresponding simulated measurement under a known image\-formation model\. For agentic systems, the mapping from input to output may also be mediated by a planner interacting with tools over multiple steps\. We write this planner–executor process as

aτ∼πϕ​\(aτ∣sτ\),sτ\+1=T​\(sτ,aτ\),y^=gψ​\(s0:K\),a\_\{\\tau\}\\sim\\pi\_\{\\phi\}\(a\_\{\\tau\}\\mid s\_\{\\tau\}\),\\quad s\_\{\\tau\+1\}=T\(s\_\{\\tau\},a\_\{\\tau\}\),\\quad\\hat\{y\}=g\_\{\\psi\}\(s\_\{0:K\}\),whereπϕ\\pi\_\{\\phi\}is the planner policy,TTis the tool or environment transition, andgψg\_\{\\psi\}is the final executor or answer generator\.

The evaluation pipeline is illustrated in Fig\.[3](https://arxiv.org/html/2607.07189#S3.F3)\. To probe different aspects of imaging competence, we define three complementary paradigms based on the computational imaging formalism

x=𝒜​\(z;m\)\+n,x=\\mathcal\{A\}\(z;m\)\+n,wherezzdenotes the latent clean signal,𝒜\\mathcal\{A\}is the task\-dependent forward operator, andnnrepresents noise or corruption\. These paradigms are designed to isolate direct execution ability, per\-image planning ability, and conceptual consistency between the forward model and its inverse\. In this paper, “agentic” refers specifically to the observe–plan–execute structure of the Planner protocol: the system observes the degraded input, plans an image\-specific restoration instruction, and then executes it through the editor\. The resulting protocols evaluate operator\-conditioned behavior, but they do not by themselves prove that the model causally uses the provided forward model rather than generic image priors\.

#### 3\.2\.1Expert Protocol: Fixed\-Prompt Inverse Execution

In Expert, we evaluate the model’s ability to solve the inverse problem under a fixed, domain\-informed expert instruction\. For inverse tasks, this corresponds to recovering the desired targetyiy\_\{i\}from the degraded inputxix\_\{i\}using a shared promptqexpq^\{\\mathrm\{exp\}\}across all samples in the task:

y^i=ft\(k\)​\(xi,qexp,mi;θk\)\.\\hat\{y\}\_\{i\}=f\_\{t\}^\{\(k\)\}\(x\_\{i\},q^\{\\mathrm\{exp\}\},m\_\{i\};\\theta\_\{k\}\)\.This setting serves as a standardized baseline for execution quality\. The expert prompt encodes task\-relevant restoration guidance while imposing constraints intended to reduce structural hallucination, artificial texture synthesis, and other undesired generative artifacts\. As such, Expert isolates the core execution capability of the underlying image\-editing or multimodal model under strong but fixed human supervision\.

#### 3\.2\.2Planner Protocol: Adaptive Inverse Solution

In Planner, we replace the fixed expert instruction with a per\-image adaptive plan generated by a VLM planner\. The planner first analyzes the degraded input and associated metadata, then produces a tailored restoration prompt for the executor:

qiplan=πϕ​\(xi,mi\),y^i=ft\(k\)​\(xi,qiplan,mi;θk\)\.q\_\{i\}^\{\\mathrm\{plan\}\}=\\pi\_\{\\phi\}\(x\_\{i\},m\_\{i\}\),\\qquad\\hat\{y\}\_\{i\}=f\_\{t\}^\{\(k\)\}\(x\_\{i\},q\_\{i\}^\{\\mathrm\{plan\}\},m\_\{i\};\\theta\_\{k\}\)\.In practice, the planner receives the input image together with a minimal instruction such as “Analyze and restore this,” and outputs an image\-specific prompt describing the likely degradation and a suitable restoration strategy\. This prompt is then passed to the corresponding editing model\. By comparing Planner against Expert, we test whether adaptive multimodal planning provides a measurable prompting gain over a strong generalized expert prompt\.

#### 3\.2\.3Forward Protocol: System Simulation \(Consistency Check\)

While Expert and Planner evaluate inverse execution, Forward probes whether the model exhibits a coherent understanding of the forward image\-formation process itself\. In this setting, the model is given a clean signal or image and asked to synthesize the degraded measurement predicted by the task\-specific forward model:

y^i=ft\(k\)​\(zi,qi,mi;θk\),\\hat\{y\}\_\{i\}=f\_\{t\}^\{\(k\)\}\(z\_\{i\},q\_\{i\},m\_\{i\};\\theta\_\{k\}\),where the target corresponds to the known forward output associated with the task\. For example, the model may be asked to add stochastic noise, optical blur, or undersampling artifacts consistent with the degradations encountered in Expert and Planner\.

Conceptually, Forward reverses the direction of the inverse tasks: rather than removing degradation, the model must generate it in a physically meaningful way\. Strong performance in this setting indicates not only pattern\-matching ability, but also consistency with the underlying sensing or imaging process\.

For reproducibility, the supplementary material includes representative Expert/Planner/Forward prompts together with paired input/output examples\.

![Refer to caption](https://arxiv.org/html/2607.07189v1/x3.png)Figure 3:Overview of the ImagingBench evaluation pipeline:Each input task is paired with a selected dataset and prompt template, then evaluated under three protocols: the Expert protocol \(expert\-guided inverse reconstruction\), the Planner protocol \(planner\-guided inverse reconstruction\), and the Forward protocol \(forward\-system simulation\)\. In Planner and Forward, a frontier agentic AI planner generates image\-specific instructions that are executed by a VLM image editor\. Outputs are scored using reference metrics \(PSNR, SSIM, LPIPS\) and the no\-reference metric NIQE, then aggregated into a unified category\-level score for final comparison across models and protocols\.

### 3\.3Evaluation Metrics

For each task familyttand sampleii, the model produces a predictiony^i=ft\(k\)​\(xi,qi,mi;θk\)\\hat\{y\}\_\{i\}=f\_\{t\}^\{\(k\)\}\(x\_\{i\},q\_\{i\},m\_\{i\};\\theta\_\{k\}\), which is compared against the task targetyiy\_\{i\}using task\-appropriate metrics\. In line with the protocol above, we report*raw task metrics first*, and use a normalized aggregate score only as a secondary summary across heterogeneous tasks\.

Reconstruction\-based tasks\.For image reconstruction and restoration tasks, bothyiy\_\{i\}andy^i\\hat\{y\}\_\{i\}are images defined on the same valid supportΩi\\Omega\_\{i\}Let\|Ωi\|\|\\Omega\_\{i\}\|denote the number of valid pixels, and let the dynamic range beLL\(we useL=1L=1for normalized images\)\. We evaluate four complementary metrics: PSNR, SSIM, LPIPS, and NIQE\.

Normalized reconstruction score\.Because these metrics have different scales and directions, we additionally report a normalized per\-instance reconstruction score for cross\-task summarization\. For sampleii, we define

sipsnr\\displaystyle s^\{\\mathrm\{psnr\}\}\_\{i\}=clip​\(PSNR​\(yi,y^i\)−1540−15,0,1\),\\displaystyle=\\mathrm\{clip\}\\\!\\left\(\\frac\{\\mathrm\{PSNR\}\(y\_\{i\},\\hat\{y\}\_\{i\}\)\-15\}\{40\-15\},\\,0,\\,1\\right\),\(2\)sissim\\displaystyle s^\{\\mathrm\{ssim\}\}\_\{i\}=clip​\(SSIM​\(yi,y^i\),0,1\),\\displaystyle=\\mathrm\{clip\}\\\!\\left\(\\mathrm\{SSIM\}\(y\_\{i\},\\hat\{y\}\_\{i\}\),\\,0,\\,1\\right\),\(3\)silpips\\displaystyle s^\{\\mathrm\{lpips\}\}\_\{i\}=clip​\(1−LPIPS​\(yi,y^i\),0,1\),\\displaystyle=\\mathrm\{clip\}\\\!\\left\(1\-\\mathrm\{LPIPS\}\(y\_\{i\},\\hat\{y\}\_\{i\}\),\\,0,\\,1\\right\),\(4\)siniqe\\displaystyle s^\{\\mathrm\{niqe\}\}\_\{i\}=clip​\(20−NIQE​\(y^i\)17,0,1\)\.\\displaystyle=\\mathrm\{clip\}\\\!\\left\(\\frac\{20\-\\mathrm\{NIQE\}\(\\hat\{y\}\_\{i\}\)\}\{17\},\\,0,\\,1\\right\)\.\(5\)We then combine them into an instance\-level weighted score

sirecon=wpsnr​sipsnr\+wssim​sissim\+wlpips​silpips\+wniqe​siniqewpsnr\+wssim\+wlpips\+wniqe,s\_\{i\}^\{\\mathrm\{recon\}\}=\\frac\{w\_\{\\mathrm\{psnr\}\}s\_\{i\}^\{\\mathrm\{psnr\}\}\+w\_\{\\mathrm\{ssim\}\}s\_\{i\}^\{\\mathrm\{ssim\}\}\+w\_\{\\mathrm\{lpips\}\}s\_\{i\}^\{\\mathrm\{lpips\}\}\+w\_\{\\mathrm\{niqe\}\}s\_\{i\}^\{\\mathrm\{niqe\}\}\}\{w\_\{\\mathrm\{psnr\}\}\+w\_\{\\mathrm\{ssim\}\}\+w\_\{\\mathrm\{lpips\}\}\+w\_\{\\mathrm\{niqe\}\}\},with base weights

wpsnr=0\.3,wssim=0\.3,wlpips=0\.3,wniqe=0\.1\.w\_\{\\mathrm\{psnr\}\}=0\.3,\\quad w\_\{\\mathrm\{ssim\}\}=0\.3,\\quad w\_\{\\mathrm\{lpips\}\}=0\.3,\\quad w\_\{\\mathrm\{niqe\}\}=0\.1\.We weight PSNR, SSIM, and LPIPS equally because they provide complementary reference\-based fidelity views, while NIQE receives a smaller weight because it is no\-reference and is intended to inform the aggregate without dominating it\. If a metric is unavailable for a particular sample, its weight is set to zero and the remaining weights are renormalized\.

For each reconstruction tasktt, we report both the raw metric means and the averaged normalized score

Strecon=1Nt​∑i=1Ntsirecon\.S\_\{t\}^\{\\mathrm\{recon\}\}=\\frac\{1\}\{N\_\{t\}\}\\sum\_\{i=1\}^\{N\_\{t\}\}s\_\{i\}^\{\\mathrm\{recon\}\}\.
Task\-specific metrics beyond reconstruction\.Not all tasks in ImagingBench are image\-to\-image restoration problems\. For example, lens\-design tasks are evaluated by optical validity and downstream image\-quality criteria such as spot size, MTF, and Strehl ratio, while forward\-simulation tasks are evaluated by consistency with the known image\-formation process\. We denote such task\-specific scores generically by

Metrict​\(yi,y^i;mi\),\\mathrm\{Metric\}\_\{t\}\\\!\\left\(y\_\{i\},\\hat\{y\}\_\{i\};m\_\{i\}\\right\),to emphasize that the exact metric depends on the task\.

Aggregate score across tasks\.To summarize performance across heterogeneous task families, we use a two\-level macro\-average\. Each task contributes a task\-level score

St\(k\)=\{Strecon,\(k\),reconstruction\-based task,Normt​\(Metrict\(k\)\),otherwise,S\_\{t\}^\{\(k\)\}=\\begin\{cases\}S\_\{t\}^\{\\mathrm\{recon\},\(k\)\},&\\text\{reconstruction\-based task\},\\\\\[3\.99994pt\] \\mathrm\{Norm\}\_\{t\}\\\!\\left\(\\mathrm\{Metric\}\_\{t\}^\{\(k\)\}\\right\),&\\text\{otherwise\},\\end\{cases\}whereStrecon,\(k\)S\_\{t\}^\{\\mathrm\{recon\},\(k\)\}is the averaged normalized reconstruction score andNormt​\(⋅\)\\mathrm\{Norm\}\_\{t\}\(\\cdot\)maps a non\-reconstruction metric to a common higher\-is\-better\[0,1\]\[0,1\]scale\. Camera calibration usesNormt=clip​\(1−MRE,0,1\)\\mathrm\{Norm\}\_\{t\}=\\mathrm\{clip\}\(1\-\\mathrm\{MRE\},0,1\)over its intrinsic and distortion relative errors \(parse failures score0\), and lens design usesNormt=clip​\(Strehl/0\.8,0,1\)\\mathrm\{Norm\}\_\{t\}=\\mathrm\{clip\}\(\\mathrm\{Strehl\}/0\.8,0,1\)against the diffraction\-limited threshold \(unanalyzable targets score0\)\. We then average within each of the five task categoriesccand across categories:

Sc\(k\)=1\|𝒯c\|​∑t∈𝒯cSt\(k\),AggScore​\(k\)=15​∑cSc\(k\)\.S\_\{c\}^\{\(k\)\}=\\frac\{1\}\{\|\\mathcal\{T\}\_\{c\}\|\}\\sum\_\{t\\in\\mathcal\{T\}\_\{c\}\}S\_\{t\}^\{\(k\)\},\\qquad\\mathrm\{AggScore\}\(k\)=\\frac\{1\}\{5\}\\sum\_\{c\}S\_\{c\}^\{\(k\)\}\.Calibration and lens design enter the Calibration and Ray\-and\-Wave\-Optics categories, respectively\. We use a category macro\-average rather than a flat per\-task average so that families with many subtasks \(e\.g\. image signal processing\) do not dominate the score; the model ranking is unchanged under flat1/\|𝒯\|1/\|\\mathcal\{T\}\|weighting \(supplement\)\. The aggregate score is used only for cross\-task summarization; the primary analysis remains the raw task metrics reported for each subtask\.

### 3\.4Evaluated Models

We evaluate representative state\-of\-the\-art image\-centric multimodal systems spanning both proprietary and open\-source families\. For our base editing execution \(Expert\), we utilizeNano Banana 2\(Gemini 3\.1 Flash Image Preview\)\[[45](https://arxiv.org/html/2607.07189#bib.bib45)\],GPT\-Image\-1\.5\[[46](https://arxiv.org/html/2607.07189#bib.bib46)\], andQwen\-Image\-Edit\-2511\[[47](https://arxiv.org/html/2607.07189#bib.bib47)\]\. Nano Banana 2 is accessed via Google Vertex AI \(checkpoint: 2026\-02\-26\), and GPT\-Image\-1\.5 is accessed via Azure Foundry \(version: 2025\-12\-16\)\. Qwen\-Image\-Edit\-2511 is hosted on 1xNVIDIA H200 GPU and run with 28\-step diffusion\.

To support the planner\-guided setting \(Planner\), we pair these base editing models with their respective frontier multimodal reasoning models \(acting as the planner\):Gemini 3\.1 Pro Preview,GPT\-5, andQwen3\.5\-35B\-A3B\.

For Gemini and GPT, we use default API inference settings\. For Qwen\-Image\-Edit\-2511, we follow the official HuggingFace implementation with native 28\-step diffusion while keeping all other parameters at default values\.

To reduce model\-specific auto\-resizing artifacts, all benchmark inputs are standardized to a1024×10241024\\times 1024canvas before inference\. In practice, this avoids unintended scale changes from provider\-side resolution bucketing and keeps output size consistent for downstream scoring\. For tasks where resizing would remove task\-defining structure, we instead pad images to1024×10241024\\times 1024and evaluate predictions only on the unpadded crop\.

## 4Data Collection and Quality Assurance

As shown in Fig\.[4](https://arxiv.org/html/2607.07189#S4.F4), ImagingBench combines existing datasets with newly constructed data\. It contains three components: \(1\) the main benchmark, \(2\) an ablation suite for controlled sensitivity analysis, and \(3\) a hidden test set for fair evaluation on unseen examples\. We summarize the main curation procedure here and defer additional preprocessing details to the supplementary material\. We target roughly 100 examples per subtask because ImagingBench is intended as a diagnostic zero\-shot benchmark spanning many computational\-imaging operators rather than as a large\-scale training resource\.

### 4\.1Main Benchmark Data Collection

DatasetsTasksCGHLens DesignDem\-HDem\-QDem\-RHDRWBDenoiseSph\. Aber\.Long\. Chrom\.MotionRand\. MaskBlock CSSRInpaintLenslessLF ViewEventToFCalib\.OpticsImage Signal ProcessingInverse ReconstructionComp\. SensingMIT\-CGH\-4K†\[13\]Ours – Ray OpticsKodak†\[25\]DIV2K†\[28\]HDR\+ \[20\]WB Set1†\[17\]SIDD \[24\]Adobe5k†\[29\]GOPRO \[26\]DLMD†\[16\]LF Archive 2016†\[27\]DAVIS \[14\]UD\-ToF \[15\]Ours – Calibration

Figure 4:Datasets used for the benchmarking tasks\.†denotes degradations applied synthetically to clean images from the source dataset\.The main benchmark spans five categories \(Table[II](https://arxiv.org/html/2607.07189#S3.T2)\)\. For all subtasks except lens design and calibration, we use a two\-stage collection pipeline; data sources are summarized in Fig\.[4](https://arxiv.org/html/2607.07189#S4.F4)\. First, we adopt established datasets when they match the task definition\. When an official test split exists, we randomly sample 100 images\. Specifically, we use SIDD\[[48](https://arxiv.org/html/2607.07189#bib.bib48)\]for denoising, HDR\+\[[42](https://arxiv.org/html/2607.07189#bib.bib42)\]for HDR reconstruction, MIT\-CGH\-4K\[[36](https://arxiv.org/html/2607.07189#bib.bib36)\]for CGH, WB Set\-1\[[40](https://arxiv.org/html/2607.07189#bib.bib40)\]for white balance, Kodak\[[49](https://arxiv.org/html/2607.07189#bib.bib49)\]for demosaicking \(Regular/Hexa/Quad\), GoPro\[[50](https://arxiv.org/html/2607.07189#bib.bib50)\]for motion deblurring, DLMD\[[39](https://arxiv.org/html/2607.07189#bib.bib39)\]for lensless reconstruction, DAVIS\-color\[[37](https://arxiv.org/html/2607.07189#bib.bib37)\]for event\-based intensity reconstruction, a real under\-display ToF dataset for ToF imaging, and the Stanford Light Field Dataset\[[51](https://arxiv.org/html/2607.07189#bib.bib51)\]for light\-field view extrapolation\. Second, when no suitable dataset exists, we synthesize measurements using controlled forward models applied to high\-quality clean images\. For most such subtasks, we build each 100\-image set by evenly sampling from DIV2K\[[52](https://arxiv.org/html/2607.07189#bib.bib52)\]and Adobe FiveK\[[53](https://arxiv.org/html/2607.07189#bib.bib53)\], then applying the task\-specific forward operator\.

For lens design, we construct task instances from target specifications and physical constraints\[[13](https://arxiv.org/html/2607.07189#bib.bib13),[14](https://arxiv.org/html/2607.07189#bib.bib14),[33](https://arxiv.org/html/2607.07189#bib.bib33)\]\. Each optical system is defined by the half\-diagonal field of view \(HFOV\), F\-number \(FNUM\), sensor diagonal \(DIAG\), number of lens elements, candidate materials, and surface parameterization\[[14](https://arxiv.org/html/2607.07189#bib.bib14)\]\. We fix the sensor diagonal to 6\.0 mm, as well as the number of elements and material list, and parameterize each aspheric surface by one conic constant and six aspheric coefficients\. The same aspherical\-surface search space is used for all compared lens\-design methods\. We then vary HFOV and FNUM to control difficulty, yielding benchmark groups that span narrow\-to\-wide fields of view and slow\-to\-fast apertures\. Calibration task data are collected separately\.

### 4\.2Ablation Study Data Collection

For the ablation suite, we generate controlled degradations from clean DIV2K images to isolate specific corruption factors\. We consider \(i\) image noise at multiple levels and \(ii\) spherical\-aberration blur at multiple strengths\. For spherical aberration, we inject the Zernike modeZ40Z\_\{4\}^\{0\}\[[54](https://arxiv.org/html/2607.07189#bib.bib54)\]into the pupil function and vary the aberration strengthcsphc\_\{\\mathrm\{sph\}\}at a fixed wavelengthλ=550​nm\\lambda=550\\,\\mathrm\{nm\}\. We usecsph∈\{0\.1​λ,0\.25​λ,0\.5​λ,1\.0​λ,1\.5​λ\}c\_\{\\mathrm\{sph\}\}\\in\\\{0\.1\\lambda,0\.25\\lambda,0\.5\\lambda,1\.0\\lambda,1\.5\\lambda\\\}to synthesize five Point Spread Function \(PSF\) kernels, which are then applied to clean images\. For denoising, we use a Poisson–Gaussian noise model\[[55](https://arxiv.org/html/2607.07189#bib.bib55)\]and sweep ten degradation levels under three settings: Shot\-only \(vary photon countpp, fix read noiseσ\\sigma\), Read\-only \(varyσ\\sigma, fixpp\), and Mixed \(vary both\)\.

### 4\.3Statistical Analysis of the Dataset

ImagingBench contains 20 subtasks in total \(Table[II](https://arxiv.org/html/2607.07189#S3.T2)\)\. To characterize dataset diversity, we analyze both semantic scene composition and low\-level image statistics\. We estimate scene distributions over 13 semantic categories—indoor, outdoor, street, landscape, city, portrait, people, night, text, food, animals, sports, and close\-up—and measure brightness and contrast using mean pixel intensity and pixel\-intensity standard deviation\. As shown in Fig\.[5](https://arxiv.org/html/2607.07189#S4.F5), the semantic composition is broadly balanced across subtasks, with no single scene type dominating\. The brightness–contrast analysis in the supplementary material likewise shows substantial variation across tasks and samples\. Together, these results indicate that ImagingBench covers diverse semantic content and photometric characteristics, supporting evaluation under varied imaging scenarios\.

![Refer to caption](https://arxiv.org/html/2607.07189v1/x4.png)Figure 5:Semantic scene distribution across ImagingBench subtasks, shown as the average probability over 13 scene categories\. The distribution is broadly balanced, with no single scene type dominating the benchmark\.

## 5Results

![Refer to caption](https://arxiv.org/html/2607.07189v1/x5.png)Figure 6:Unified\-score radar comparison across representative image\-reconstruction subtasks\. Each axis reports task\-level unified score in\[0,1\]\[0,1\]\(higher is better\), computed from normalized PSNR/SSIM/LPIPS/NIQE and aggregated with per\-instance weighted averaging followed by macro\-averaging at task level\.![Refer to caption](https://arxiv.org/html/2607.07189v1/x6.png)Figure 7:Per\-task distributions of the unified score in\[0,1\]\[0,1\]\(higher is better\) for each model under the Expert and Planner protocols, shown as ridgeline density plots\. The unified score is computed from normalized PSNR/SSIM/LPIPS/NIQE with per\-instance weighted averaging\. Colored dots denote the mean of each distribution\.TABLE III:Quantitative results across benchmark tasks for Expert, Planner, and, where available, Forward\. In each agent metric cell, the Expert result is shown on the first line, the Planner result is shown on the second line, and the Forward result is shown on the third line only for tasks with Forward evaluation results\. Missing Non\-agent SoTA phases are shown as –\. Cell colors follow the metric\-specific normalization, with green denoting generally considered good and red denoting bad performance for each metric; for stacked cells, each line is colored independently from its own value\.TaskPhaseGemini\-3\.1\-Flash\-ImageGPT\-Image\-1\.5Qwen\-Image\-Edit\-2511Non\-agent SoTAPSNR↑\\uparrowSSIM↑\\uparrowLPIPS↓\\downarrowNIQE↓\\downarrowPSNR↑\\uparrowSSIM↑\\uparrowLPIPS↓\\downarrowNIQE↓\\downarrowPSNR↑\\uparrowSSIM↑\\uparrowLPIPS↓\\downarrowNIQE↓\\downarrowPSNR↑\\uparrowSSIM↑\\uparrowLPIPS↓\\downarrowNIQE↓\\downarrowCiteRay and Wave OpticsComp\. Gen\. HolographyExpertPlannerForward6\.73906\.73905\.60945\.609410\.354510\.35450\.06010\.06010\.00550\.00550\.10560\.10561\.11791\.11791\.12381\.12380\.76320\.76325\.68005\.68003\.47893\.478910\.840410\.84049\.58909\.58908\.18928\.18929\.98969\.98960\.28350\.28350\.18910\.18910\.13290\.13290\.84640\.84640\.91000\.91000\.90780\.90787\.77257\.77255\.31645\.31649\.93689\.936817\.308017\.308017\.330017\.33009\.21789\.21780\.38670\.38670\.33540\.33540\.15970\.15970\.36200\.36200\.27490\.27490\.75180\.75186\.46246\.46246\.57486\.57486\.00356\.0035\[[36](https://arxiv.org/html/2607.07189#bib.bib36)\]Image Signal ProcessingDemosaicking \(Hexadeca\)ExpertPlanner18\.843018\.843018\.876318\.87630\.66750\.66750\.66820\.66820\.27970\.27970\.29040\.29043\.55473\.55473\.57653\.576513\.677013\.677013\.807213\.80720\.36350\.36350\.34050\.34050\.49950\.49950\.53650\.53654\.25214\.25214\.51204\.512012\.198012\.198014\.520614\.52060\.27210\.27210\.36350\.36350\.75350\.75350\.64690\.64698\.08488\.08487\.19767\.1976–Demosaicking \(Quad\)ExpertPlanner18\.928018\.928018\.723118\.72310\.67820\.67820\.67050\.67050\.28160\.28160\.29900\.29903\.41913\.41913\.45983\.459813\.838013\.838013\.971913\.97190\.36290\.36290\.33750\.33750\.48360\.48360\.52490\.52494\.04794\.04794\.64824\.648214\.744014\.744015\.020615\.02060\.41030\.41030\.39620\.39620\.64570\.64570\.61630\.61637\.24037\.24036\.90226\.9022\[[41](https://arxiv.org/html/2607.07189#bib.bib41)\]Demosaicking \(Regular\)ExpertPlanner18\.851018\.851018\.838918\.83890\.68440\.68440\.67700\.67700\.28310\.28310\.28980\.28983\.52523\.52523\.56163\.561614\.185014\.185014\.150614\.15060\.37630\.37630\.37070\.37070\.44990\.44990\.46230\.46234\.11784\.11784\.16654\.166513\.299013\.299013\.476013\.47600\.26270\.26270\.26990\.26990\.79050\.79050\.78200\.782014\.987014\.987014\.854314\.8543\[[41](https://arxiv.org/html/2607.07189#bib.bib41)\]High Dynamic RangeExpertPlannerForward18\.505018\.505017\.757817\.757817\.926617\.92660\.58440\.58440\.57510\.57510\.54160\.54160\.20920\.20920\.21450\.21450\.31580\.31583\.77703\.77703\.77683\.77684\.10864\.108615\.726015\.726015\.237315\.237313\.801413\.80140\.44060\.44060\.43020\.43020\.39340\.39340\.39440\.39440\.40710\.40710\.51800\.51804\.12854\.12854\.22784\.22784\.03774\.037713\.411013\.411013\.406113\.406113\.375913\.37590\.38780\.38780\.39780\.39780\.47390\.47390\.47820\.47820\.48310\.48310\.47120\.47125\.68905\.68906\.01856\.01854\.64754\.6475–White BalanceExpertPlanner18\.997018\.997019\.316419\.31640\.75100\.75100\.74600\.74600\.31080\.31080\.31910\.31914\.21184\.21184\.33464\.334611\.961011\.961011\.844311\.84430\.43520\.43520\.43920\.43920\.51200\.51200\.51690\.51693\.98453\.98454\.05294\.052915\.234015\.234015\.455015\.45500\.65890\.65890\.66270\.66270\.42980\.42980\.42340\.42345\.05505\.05505\.01375\.0137\[[56](https://arxiv.org/html/2607.07189#bib.bib56)\]DenoisingExpertPlannerForward25\.433025\.433023\.815423\.815419\.577419\.57740\.79500\.79500\.77790\.77790\.44100\.44100\.31180\.31180\.31430\.31430\.56930\.56936\.93546\.93546\.97806\.97806\.05626\.056218\.002018\.002017\.984117\.984117\.716817\.71680\.51760\.51760\.54240\.54240\.25560\.25560\.55260\.55260\.53910\.53910\.58990\.58998\.99558\.99559\.56509\.56507\.40437\.404320\.189020\.189020\.924720\.924717\.907817\.90780\.68800\.68800\.66890\.66890\.41670\.41670\.47050\.47050\.44280\.44280\.88890\.88898\.15778\.15778\.35898\.35898\.75368\.7536\[[57](https://arxiv.org/html/2607.07189#bib.bib57)\]Inverse ReconstructionSpherical AberrationExpertPlanner22\.692022\.692022\.634922\.63490\.64540\.64540\.64590\.64590\.20490\.20490\.21060\.21063\.81533\.81533\.88843\.888414\.216014\.216014\.435414\.43540\.38540\.38540\.39000\.39000\.45640\.45640\.45760\.45764\.00164\.00164\.22094\.220920\.610020\.610019\.547419\.54740\.66520\.66520\.60550\.60550\.29650\.29650\.47910\.47915\.15925\.15928\.34018\.3401–Longitudinal ChromaticExpertPlanner21\.519021\.519021\.479421\.47940\.67200\.67200\.67390\.67390\.23170\.23170\.24440\.24443\.97533\.97534\.07074\.070715\.707015\.707015\.531515\.53150\.46740\.46740\.45370\.45370\.45280\.45280\.47010\.47014\.81804\.81804\.82204\.822019\.620019\.620018\.991818\.99180\.66120\.66120\.59340\.59340\.37450\.37450\.45850\.45855\.25465\.25466\.20806\.2080–Motion BlurExpertPlanner21\.725021\.725021\.671721\.67170\.65570\.65570\.65740\.65740\.37130\.37130\.36650\.36653\.67233\.67233\.96623\.966214\.655014\.655015\.155515\.15550\.42770\.42770\.46360\.46360\.54430\.54430\.52180\.52184\.20634\.20634\.91404\.914020\.697020\.697020\.421420\.42140\.76200\.76200\.73610\.73610\.42450\.42450\.45720\.45729\.24419\.24419\.91189\.9118\[[58](https://arxiv.org/html/2607.07189#bib.bib58)\]Randomly Masked Sub\-SamplingExpertPlanner13\.960013\.960013\.601013\.60100\.47330\.47330\.46510\.46510\.46210\.46210\.47470\.47474\.41474\.41474\.85924\.859212\.990012\.990011\.972911\.97290\.36560\.36560\.33680\.33680\.55540\.55540\.61230\.61234\.80114\.80115\.47645\.47649\.40009\.40008\.81988\.81980\.08880\.08880\.11540\.11540\.97420\.97420\.93150\.931516\.352016\.352011\.258111\.2581–Comp\. Sensing \(Block\)ExpertPlannerForward22\.450022\.450021\.309121\.309110\.550510\.55050\.62060\.62060\.61800\.61800\.24520\.24520\.25440\.25440\.26350\.26350\.79410\.79413\.77153\.77153\.74473\.744711\.757211\.757214\.927014\.927013\.765013\.76509\.91449\.91440\.42280\.42280\.39440\.39440\.21770\.21770\.44660\.44660\.47840\.47840\.81970\.81974\.08584\.08584\.22304\.223020\.230420\.230420\.934020\.934021\.668421\.66849\.08259\.08250\.62740\.62740\.63960\.63960\.18200\.18200\.60560\.60560\.58160\.58160\.70190\.701910\.128010\.128010\.040010\.040017\.582117\.5821–Super\-ResolutionExpertPlanner21\.745021\.745021\.473921\.47390\.60490\.60490\.59940\.59940\.24000\.24000\.24240\.24244\.01564\.01563\.82873\.828714\.365014\.365013\.771713\.77170\.39150\.39150\.36580\.36580\.46020\.46020\.50210\.50214\.01484\.01484\.02044\.020420\.156020\.156019\.819319\.81930\.61220\.61220\.59900\.59900\.32290\.32290\.30470\.30474\.83014\.83014\.64284\.6428\[[59](https://arxiv.org/html/2607.07189#bib.bib59)\]InpaintingExpertPlanner23\.777023\.777023\.109523\.10950\.77610\.77610\.76670\.76670\.12420\.12420\.13650\.13653\.84333\.84333\.80503\.805015\.180015\.180014\.271814\.27180\.43470\.43470\.41860\.41860\.40690\.40690\.45410\.45414\.23094\.23094\.19964\.199612\.813012\.813013\.981513\.98150\.47440\.47440\.53370\.53370\.49340\.49340\.40690\.40695\.33945\.33944\.99254\.9925\[[44](https://arxiv.org/html/2607.07189#bib.bib44)\]Computational SensingLensless CameraExpertPlannerForward7\.47707\.47707\.28337\.28338\.83168\.83160\.21120\.21120\.18870\.18870\.15300\.15300\.78750\.78750\.79170\.79170\.94810\.948111\.390011\.390016\.879516\.87955\.88615\.88617\.90407\.90408\.01578\.015710\.059010\.05900\.23490\.23490\.24290\.24290\.32910\.32910\.83730\.83730\.88750\.88750\.78980\.789811\.068011\.068016\.277916\.27799\.15359\.15358\.97408\.97408\.80988\.80987\.91387\.91380\.25180\.25180\.24150\.24150\.26100\.26100\.79310\.79310\.76270\.76270\.82800\.828016\.259016\.259013\.450613\.45068\.76118\.7611\[[39](https://arxiv.org/html/2607.07189#bib.bib39)\]Lightfield View ExtrapolationExpertPlanner16\.419316\.419316\.384016\.38400\.53150\.53150\.52170\.52170\.45110\.45110\.45440\.45445\.48935\.48935\.12785\.127812\.301512\.301514\.257914\.25790\.35140\.35140\.36980\.36980\.66900\.66900\.60430\.60434\.66964\.66964\.35474\.354711\.897311\.897315\.400315\.40030\.43360\.43360\.50570\.50570\.62690\.62690\.52560\.52566\.65866\.65866\.70526\.7052–Lightfield View InterpolationExpertPlanner21\.297521\.297520\.827020\.82700\.63660\.63660\.61440\.61440\.32790\.32790\.34920\.34925\.87815\.87815\.63655\.636515\.460615\.460615\.511215\.51120\.46790\.46790\.44970\.44970\.50540\.50540\.50900\.50905\.64465\.64464\.63444\.634416\.564216\.564218\.356318\.35630\.53380\.53380\.55310\.55310\.52940\.52940\.48250\.48258\.71828\.71828\.69028\.6902–Event\-based Intensity ImagingExpertPlanner9\.95109\.951010\.058510\.05850\.29430\.29430\.26770\.26770\.73450\.73450\.73920\.73929\.46599\.46599\.33359\.333510\.030010\.030010\.229510\.22950\.23050\.23050\.14910\.14910\.79780\.79780\.88960\.88969\.58759\.587511\.570411\.57047\.11707\.11707\.52607\.52600\.11240\.11240\.06820\.06820\.86020\.86020\.77410\.774128\.304028\.304022\.273122\.2731–Time\-of\-Flight Depth ImagingExpertPlanner4\.99004\.990010\.498710\.49870\.07040\.07040\.52510\.52510\.67960\.67960\.43410\.43419\.13029\.13029\.79179\.79179\.41509\.415012\.647812\.64780\.35180\.35180\.63690\.63690\.55190\.55190\.35790\.357911\.779011\.779011\.244211\.244223\.925023\.925023\.894823\.89480\.79560\.79560\.78090\.78090\.07800\.07800\.07540\.075415\.659015\.659015\.495715\.4957–CalibrationCamera Distortion ScoreExpertPlanner19\.129719\.129719\.030319\.03030\.56060\.56060\.56220\.56220\.13170\.13170\.13050\.13053\.23473\.23473\.25953\.259511\.031911\.031911\.283611\.28360\.29710\.29710\.30440\.30440\.55260\.55260\.53850\.53853\.89713\.89713\.95903\.959014\.542314\.542317\.012017\.01200\.43320\.43320\.47760\.47760\.41940\.41940\.29480\.29484\.45074\.45074\.27994\.2799–

TABLE IV:ImagingBench leaderboard under the Expert protocol\.Overallis the category\-weighted score \(macro\-average of the five task categories\);Aggis the aggregate score under a flat per\-task \(1/\|𝒯\|1/\|\\mathcal\{T\}\|\) weighting\. Reconstruction tasks contribute through their categories; camera calibration enters the Calibration category and lens design the Ray\-and\-Wave\-Optics category\. Higher is better; the per\-category breakdown is in the supplement\.ModelOverallAggGemini\-3\.1\-Flash\-Image0\.4130\.471GPT\-Image\-1\.50\.3050\.336Qwen\-Image\-Edit\-25110\.3410\.362### 5\.1Overall Quantitative Trends

Table[III](https://arxiv.org/html/2607.07189#S5.T3)reports raw task\-level results for Expert and Planner, Fig\.[7](https://arxiv.org/html/2607.07189#S5.F7)summarizes the corresponding metric distributions, and Fig\.[6](https://arxiv.org/html/2607.07189#S5.F6)shows task\-level unified scores\. Together, they reveal several consistent trends\.

Agentic vs\. non\-agentic\.Where specialized non\-agentic baselines are available, they generally outperform the agentic multimodal systems\. This is clear in Table[III](https://arxiv.org/html/2607.07189#S5.T3), where the baseline columns are often stronger, and in the radar plots, where the non\-agentic curve frequently lies outside the agentic ones\. Thus, general agentic multimodal capability still falls short of specialized non\-agentic methods\.

Fidelity vs\. plausibility\.The metrics show a clear mismatch between visual plausibility and reconstruction fidelity\. PSNR is often low for agentic models, and LPIPS is also generally worse than the specialized baselines, indicating poor agreement with the target\. SSIM is usually less degraded, suggesting that coarse scene structure is often preserved\. By contrast, NIQE is sometimes relatively favorable even when PSNR and LPIPS are poor, implying that the outputs may look natural without being correct\.

Task\-family differences\.Agentic models are more competitive on conventional image signal processing and inverse reconstruction tasks, such as denoising, white balance, inpainting, super\-resolution, and some blur\-removal problems\. They perform much worse on computational sensing tasks, including lensless imaging, event\-based intensity imaging, time\-of\-flight depth imaging, and holography\. This trend appears in both the raw table and the unified\-score radar plots, indicating that structured sensing operators remain substantially harder than photographic degradations\.

Expert vs\. Planner\.Planner guidance provides only modest and inconsistent gains\. In our setup, Expert uses a fixed expert prompt, whereas Planner instantiates the observe–plan–execute pipeline by having a planner generate an image\-specific restoration instruction before execution\. In some tasks, Planner slightly improves SSIM, LPIPS, or NIQE over Expert, suggesting that image\-specific prompting can help when the degradation is correctly diagnosed\. However, these gains are usually small and do not change the overall ranking\. The average unified\-score delta from Expert to Planner is also small across model families:−0\.0009\-0\.0009for Gemini,−0\.0103\-0\.0103for GPT, and\+0\.0109\+0\.0109for Qwen\. These near\-zero means suggest that prompt sensitivity is modest on average rather than dominant, although task\-wise gains and losses can cancel\. The radar plots for Expert and Planner therefore retain similar shapes, showing that planner guidance helps only in selected cases\.

Model\-family differences\.Across both Expert and Planner, Gemini is generally the strongest and most consistent family on standard restoration tasks, GPT is usually weaker, and Qwen is the most variable\. Qwen occasionally becomes competitive on sensing\-oriented tasks, especially time\-of\-flight depth imaging, but is less stable overall\. This suggests that base\-model capability matters more than planner design alone\.

Overall score\.Table[IV](https://arxiv.org/html/2607.07189#S5.T4)is the ImagingBench leaderboard under the Expert protocol\. It reports two aggregate scores per model: the category\-weighted*Overall*\(macro\-average of the five task\-category scores\) and the*Agg*score under a flat per\-task \(1/\|𝒯\|1/\|\\mathcal\{T\}\|\) weighting\. Reconstruction tasks contribute through their categories, while camera calibration \(scored by intrinsic/distortion parameter error\) and lens design \(scored by normalized Strehl ratio\) enter the Calibration and optics categories, so both scores reflect non\-reconstruction competence as well\. The ranking mirrors the per\-task trends and is identical under either weighting: Gemini leads \(0\.410\.41/0\.470\.47\), followed by Qwen \(0\.340\.34/0\.360\.36\) and GPT \(0\.310\.31/0\.340\.34\), confirming that stronger base multimodal capability, rather than protocol choice or aggregation scheme, drives benchmark standing\. A per\-category breakdown is given in the supplement\.

Summary\.Overall, the results support three conclusions: specialized non\-agentic methods remain substantially stronger than current agentic multimodal systems; computational sensing tasks are markedly harder than conventional restoration tasks; and visually plausible outputs do not necessarily correspond to physically accurate solutions\.

### 5\.2Individual Task Results

Due to space constraints, we show representative individual task results here and provide additional results in the supplementary material\.

Lens DesignFig\.[8](https://arxiv.org/html/2607.07189#S5.F8)compares ChatGPT with AutoLens\[[14](https://arxiv.org/html/2607.07189#bib.bib14)\]on three representative lens design targets\. The corresponding results for Gemini and Qwen are provided in the supplementary material\. Both methods are evaluated under the same aspherical\-surface parameterization and search space\. From both the spot diagrams and the MTF curves, AutoLens outperforms ChatGPT on most targets\. Specifically, AutoLens generally produces tighter and better\-corrected spot patterns with less chromatic spread, while ChatGPT more often generates broader spot distributions, stronger color separation, and weaker off\-axis aberration correction\. The MTF curves further confirm that AutoLens preserves higher and more stable modulation transfer\. Although ChatGPT can generate optically valid lens layouts that satisfy the basic system specifications, its final optical performance still requires further improvement\. Accordingly, this task should be interpreted as testing whether a general Vision\-Language Model \(VLM\) can propose a useful one\-shot initial optical design, not whether it can replace a full iterative optical\-design pipeline\. Overall, these results indicate that current LLM\-based lens design is better suited to generating coarse initial solutions, which may serve as useful starting points for subsequent refinement, rather than directly producing high\-quality final optical designs\. Quantitatively, the peak on\-axis Strehl ratio \(where1\.01\.0denotes the ideal diffraction\-limited value\) averaged over the five targets is0\.0230\.023for AutoLens—the highest among all methods and the leader on four of the five targets—versus0\.0140\.014for GPT\-5\.4 and0\.0130\.013for Gemini, while Qwen produces an analyzable prescription for only a single target \(0\.0050\.005\)\. All designs remain far below the0\.80\.8diffraction\-limited threshold, reinforcing that current VLM lens prescriptions are at best coarse initializations; per\-target Strehl values are reported in the supplementary material\.

Representative visual reconstruction results\.Fig\.[9](https://arxiv.org/html/2607.07189#S5.F9)shows three representative image tasks: compressive sensing, demosaicking, and denoising\. The examples mirror the overall quantitative trends\. On demosaicking and denoising, the models usually recover the global scene structure, but still exhibit color errors, texture loss, or oversmoothing relative to the ground truth\. On compressive sensing, missing information is often filled in with visually plausible but incorrect content, indicating weak operator\-aware inversion\. Across the three examples, Gemini is generally the most faithful, GPT is weaker, and Qwen is more variable, consistent with the broader benchmark statistics\.

![Refer to caption](https://arxiv.org/html/2607.07189v1/x7.png)Figure 8:Comparison of representative ray\-optics lens designs under different target specifications\. From left to right, the three columns correspond to benchmark groups with decreasing design difficulty, moving from wider fields of view, faster apertures, and larger sensor formats to narrower fields of view, slower apertures, and smaller sensor formats\. For each design, we show the optical layout, spot diagrams at multiple field angles, and the corresponding MTF curves\. The top and bottom rows present the results ofGPT\-5\.4andAutoLens\[[14](https://arxiv.org/html/2607.07189#bib.bib14)\], respectively, highlighting the gap between a general VLM and a state\-of\-the\-art learning\-based lens design method in both structural plausibility and optical performance\.![Refer to caption](https://arxiv.org/html/2607.07189v1/x8.png)Figure 9:Visual comparison of outputs from GPT, Gemini, and Qwen on three representative ImagingBench tasks: compressive sensing \(top\), demosaicking \(middle\), and denoising \(bottom\)\. The examples highlight differences in reconstruction fidelity across models in terms of structural recovery, color restoration, and noise suppression\. Additional visual results are provided in the supplementary material\.

## 6Discussion and Analysis

### 6\.1Sampling\-Ratio Ablation

Table[V](https://arxiv.org/html/2607.07189#S6.T5)studies how performance changes as the sampling ratio varies\. As expected, performance improves as more information is retained, but the gain is not linear and remains model dependent\. At aggressive undersampling, the models often produce visually plausible outputs that are inconsistent with the target, indicating that current agentic systems still require substantially more information than specialized inverse methods to achieve reliable recovery\.

TABLE V:Reconstruction performance under pixel\-space compressive sensing with different sampling ratios \(fraction of retained pixels\)\.ModelMetric10%20%40%60%80%GeminiPSNR↑\\uparrow8\.923511\.06613\.45916\.65219\.650SSIM↑\\uparrow0\.29090\.39240\.48780\.57070\.6228LPIPS↓\\downarrow0\.71580\.54960\.43280\.33470\.2800NIQE↓\\downarrow6\.23494\.75833\.92463\.64353\.5308GPTPSNR↑\\uparrow10\.45011\.37413\.76414\.54514\.936SSIM↑\\uparrow0\.31800\.33120\.38270\.39280\.4053LPIPS↓\\downarrow0\.69600\.61440\.52500\.48040\.4549NIQE↓\\downarrow5\.20855\.04344\.68284\.49914\.5530QwenPSNR↑\\uparrow6\.42098\.080211\.1059\.973811\.422SSIM↑\\uparrow0\.04320\.10370\.13180\.07700\.0885LPIPS↓\\downarrow0\.97210\.93201\.00961\.05140\.9060NIQE↓\\downarrow18\.93110\.07210\.19218\.00724\.556
### 6\.2Noise Ablation

For sensing tasks, noise tolerance is a basic empirical question because reconstruction quality depends on how much signal survives the measurement process\. We therefore perform an ablation study across ten degradation levels under three noise settings, as detailed in Table[VI](https://arxiv.org/html/2607.07189#S6.T6)\.Case 1 \(Shot Noise\)fixesσ=0\.2157\\sigma=0\.2157while decreasingppfrom 1000 to 5 to isolate signal\-dependent uncertainty;Case 2 \(Read Noise\)fixesp=70\.7p=70\.7while increasingσ\\sigmafrom 0\.0392 to 0\.3922 to test resilience against constant electronic interference; andCase 3 \(Mixed Noise\)varies bothppandσ\\sigmasimultaneously to simulate realistic low\-light conditions\. Gemini is substantially more robust than GPT and Qwen under moderate to extreme noise\.

TABLE VI:Denoising performance under three noise\-setting cases\. Cell colors follow the same higher\-is\-better / lower\-is\-better convention as Table[III](https://arxiv.org/html/2607.07189#S5.T3), but are normalized to the range of this ablation study\.Levelppσ\\sigmaGemini\-3\.1Flash\-ImageGPT\-Image\-1\.5Qwen\-ImageEdit\-2511PSNR↑\\uparrowSSIM↑\\uparrowLPIPS↓\\downarrowNIQE↓\\downarrowPSNR↑\\uparrowSSIM↑\\uparrowLPIPS↓\\downarrowNIQE↓\\downarrowPSNR↑\\uparrowSSIM↑\\uparrowLPIPS↓\\downarrowNIQE↓\\downarrowCase 1 \(Shot varies, Read fixed\):σ=0\.2157\\sigma=$0\.2157$fixed,ppvaries01000\.00\.215722\.45422\.4540\.63450\.63450\.26470\.26473\.5773\.57714\.95114\.9510\.34390\.34390\.46590\.46594\.4914\.49118\.51618\.5160\.60450\.60450\.39300\.39304\.7444\.7442308\.00\.215721\.82021\.8200\.62420\.62420\.27300\.27303\.8273\.82715\.25015\.2500\.34400\.34400\.45990\.45994\.7684\.76819\.15219\.1520\.60860\.60860\.38920\.38925\.0515\.051494\.90\.215722\.09322\.0930\.61420\.61420\.26590\.26593\.7633\.76314\.88414\.8840\.35490\.35490\.46810\.46814\.7694\.76918\.62718\.6270\.61620\.61620\.37240\.37244\.9014\.901629\.20\.215721\.62921\.6290\.60480\.60480\.27740\.27743\.7683\.76814\.90514\.9050\.33890\.33890\.46040\.46044\.6104\.61018\.10418\.1040\.55330\.55330\.47150\.47154\.9474\.94789\.00\.215720\.04720\.0470\.57470\.57470\.30330\.30333\.5173\.51714\.86314\.8630\.33770\.33770\.47160\.47164\.5484\.54815\.85315\.8530\.50070\.50070\.54240\.54244\.9964\.996Case 2 \(Read varies, Shot fixed\):p=70\.7p=$70\.7$fixed,σ\\sigmavaries070\.70\.039225\.04425\.0440\.74310\.74310\.16500\.16503\.7533\.75315\.35515\.3550\.35430\.35430\.45980\.45984\.6294\.62921\.13121\.1310\.74130\.74130\.21080\.21084\.1684\.168270\.70\.117623\.90523\.9050\.67450\.67450\.20870\.20873\.5593\.55915\.49115\.4910\.36130\.36130\.46430\.46434\.7464\.74620\.37820\.3780\.69770\.69770\.26370\.26374\.7554\.755470\.70\.196122\.45922\.4590\.63680\.63680\.25890\.25893\.6293\.62915\.27015\.2700\.34600\.34600\.46630\.46634\.5854\.58519\.30019\.3000\.61900\.61900\.37650\.37654\.5854\.585670\.70\.274521\.27021\.2700\.58990\.58990\.30120\.30123\.5863\.58615\.32515\.3250\.33920\.33920\.48300\.48304\.5504\.55016\.47716\.4770\.56280\.56280\.50000\.50005\.0245\.024870\.70\.352919\.88419\.8840\.55780\.55780\.32490\.32493\.6593\.65914\.60914\.6090\.34640\.34640\.47680\.47684\.1234\.12315\.22515\.2250\.52200\.52200\.55570\.55575\.4005\.400Case 3 \(Both vary\):ppandσ\\sigmavary together \(same level index\)01000\.00\.039226\.93826\.9380\.77270\.77270\.12630\.12633\.7883\.78815\.57115\.5710\.37240\.37240\.43000\.43004\.5674\.56721\.64221\.6420\.75890\.75890\.18260\.18264\.3844\.3842308\.00\.117624\.14624\.1460\.68410\.68410\.20020\.20023\.8593\.85915\.10515\.1050\.35710\.35710\.46140\.46144\.6164\.61621\.45921\.4590\.70310\.70310\.25010\.25014\.8344\.834494\.90\.196122\.56322\.5630\.63610\.63610\.27070\.27073\.5443\.54415\.17515\.1750\.33820\.33820\.48010\.48014\.7914\.79120\.13120\.1310\.65260\.65260\.34150\.34154\.9084\.908629\.20\.274520\.05820\.0580\.57130\.57130\.30650\.30653\.6793\.67915\.11615\.1160\.35020\.35020\.46340\.46344\.5384\.53818\.11418\.1140\.57250\.57250\.44960\.44965\.4635\.46389\.00\.352919\.11119\.1110\.54900\.54900\.33830\.33833\.9973\.99714\.60814\.6080\.33450\.33450\.48420\.48424\.3784\.37813\.91913\.9190\.48330\.48330\.61040\.61045\.9075\.907

### 6\.3Spherical\-Aberration Ablation

For spherical aberration, we perform an ablation study by varying the aberration coefficientcsphc\_\{\\mathrm\{sph\}\}over multiple degradation levels\. We synthesize five PSF kernels representing progressively severe degradation:csph∈\{0\.1​λ,0\.25​λ,0\.5​λ,1\.0​λ,1\.5​λ\}c\_\{\\mathrm\{sph\}\}\\in\\\{0\.1\\lambda,0\.25\\lambda,0\.5\\lambda,1\.0\\lambda,1\.5\\lambda\\\}\. Ascsphc\_\{\\mathrm\{sph\}\}increases, energy diffuses from the central peak into peripheral diffraction rings, leading to significant blur and contrast loss\. The performance of each model under these varying degrees of degradation is summarized in Table[VII](https://arxiv.org/html/2607.07189#S6.T7)\. Results suggest that agentic AI models are more reliable on mild degradations than on severe physics\-driven blur, where inversion requires a more faithful understanding of the forward operator\.

TABLE VII:Deblurring performance under spherical aberration with two peak wavefront error levelswerrw\_\{\\mathrm\{err\}\}\.Modelwerrw\_\{\\mathrm\{err\}\}\(waves\)PSNR↑\\uparrowSSIM↑\\uparrowLPIPS↓\\downarrowNIQE↓\\downarrowGemini0\.1023\.2330\.67530\.17313\.82850\.2022\.1530\.61570\.23643\.8021GPT0\.1014\.2090\.38250\.44653\.92180\.2014\.2240\.38830\.46634\.0804Qwen0\.1020\.6100\.66520\.29655\.15920\.20––––
### 6\.4API Cost

We estimate the cloud\-API cost of the benchmark by pairing the per\-call token usage logged for every run with each provider’s published prices; a full per\-model breakdown is given in the supplementary material\. The roughly 100\-example\-per\-subtask scale keeps this diagnostic benchmark practical while still covering many computational\-imaging operators\. To compare the families on equal terms, all per\-image figures use standard \(non\-batch\) list pricing\. At these rates the end\-to\-end cost per reconstructed image ranges from $0\.0480\.048to $0\.0790\.079depending on protocol and model \(Table[VIII](https://arxiv.org/html/2607.07189#S6.T8)\): the Expert protocol runs a single editor call, whereas the Planner and Forward protocols add a planner call\. The two hosted pipelines are close at standard rates—OpenAI is marginally lower per image—but Gemini becomes the cheaper pipeline once its50%50\\%batch discount is applied\.

TABLE VIII:End\-to\-end API cost per reconstructed image by protocol, at standard list pricing \(Gemini normalized from the50%50\\%\-off batch rates actually billed\)\. Self\-hosted Qwen is excluded as it incurs no per\-call API charge\.ProtocolGemini \($/img\)OpenAI \($/img\)Expert \(edit only\)0\.0690\.048Planner \(plan \+ edit\)0\.0770\.062Forward \(plan \+ edit\)0\.0790\.075The cost picture was not simply proprietary versus open\. In our setup Gemini was the strongest overall image agent and also the best turnkey cost–performance option, while OpenAI was the least favorable among the hosted options\. The self\-hosted Qwen models incur no per\-call API charge at all and remain attractive when local hosting or tighter deployment control matters, though they trail both hosted families in quality\. The main tradeoff for Gemini was latency, typically a 10–20 minute batch turnaround rather than immediate responses, whereas Qwen was the most deployable, being locally hostable on commodity GPUs\. Overall, Gemini was the best practical choice in our deployment setting, Qwen the most portable, and OpenAI the weakest on cost–performance\.

### 6\.5Failure Cases

The main benchmark failure mode was reconstruction error on tasks that require explicit inversion of a sensing operator\. Under the Expert protocol, the weakest families were lensless deconvolution, event reconstruction, and CGH, with mean PSNR values of 7\.96, 9\.03, and 11\.21 and LPIPS around 0\.78–0\.81\. As discussed in Sec\.[5](https://arxiv.org/html/2607.07189#S5), the models often produce plausible\-looking images while missing the physically correct inverse solution\.

At the same time, we also observed operational failures that were not captured by image metrics alone: some hosted models occasionally triggered safety filters or refused to return an answer, even when the prompt was part of a benign computational\-imaging benchmark\. These refusals were not the dominant scientific failure mode, but they still mattered in practice because they reduced reliability and added rerun overhead\.

Figure[10](https://arxiv.org/html/2607.07189#S6.F10)shows a compact CGH example preview\. All three agents recover coarse structure, but none reconstructs the diffraction\-consistent fine detail\. The broader lesson is simple: current frontier imaging agents are competitive on familiar Image Signal Processing \(ISP\) or restoration tasks, but they still break down when the task requires phase recovery, sparse inversion, or operator\-aware reasoning, and proprietary systems can additionally fail at the API layer through filtering or refusal\.

![Refer to caption](https://arxiv.org/html/2607.07189v1/figures/cgh_preview_3895.png)Figure 10:Representative CGH failure case\. The agents recover coarse structure, but none reconstructs the diffraction\-consistent fine detail required by the task\.
### 6\.6Scope and Future Work

This benchmark is intentionally a first version\. We currently focus on image\-centric agentic pipelines rather than more complex tool\-using systems, so that the benchmark isolates imaging competence itself\. We plan to open\-source the evaluation infrastructure, host an online leaderboard, and expand the benchmark as frontier models evolve\. Given the pace of progress in AI, we view ImagingBench as a living benchmark rather than a fixed one\-time release\. A future version will focus on imaging\-related coding tasks\.

## 7Conclusion

We introduced ImagingBench, the first benchmark for evaluating agentic AI systems on computational imaging tasks spanning image reconstruction, sensing, optics, and calibration\. Across 20 subtasks and three evaluation protocols, the results show a clear gap between high\-level multimodal competence and physically grounded imaging understanding\. Current agentic systems can often generate visually plausible outputs, but they remain consistently weaker than specialized non\-agentic methods, especially on computational sensing tasks and other problems that require explicit inversion of a structured forward operator\. Planner\-guided prompting provides only modest gains over a strong fixed\-prompt baseline, suggesting that the main limitation is not prompting alone but the underlying lack of low\-level imaging competence\. We hope ImagingBench provides a useful foundation for measuring this gap and for guiding future work toward agentic models that not only see images, but also understand how images are formed and reconstructed\.

## Acknowledgments

This work used the Delta advanced computing and data resource at the National Center for Supercomputing Applications, supported by the U\.S\. National Science Foundation under award OAC 2005572 and by the State of Illinois\. Delta is a joint effort of the University of Illinois Urbana\-Champaign and its National Center for Supercomputing Applications\.

This work also used Google Cloud Platform and Microsoft Azure through CloudBank under allocation CIS260120 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support \(ACCESS\) program, supported by U\.S\. National Science Foundation grants \#2138259, \#2138286, \#2138307, \#2137603, and \#2138296\. CloudBank is additionally supported by U\.S\. National Science Foundation grant \#1925001\.

## References

- \[1\]O\. Russakovsky, J\. Deng, H\. Su, J\. Krause, S\. Satheesh, S\. Ma, Z\. Huang, A\. Karpathy, A\. Khosla, M\. Bernstein, A\. C\. Berg, and L\. Fei\-Fei, “Imagenet large scale visual recognition challenge,”*International Journal of Computer Vision*, vol\. 115, no\. 3, pp\. 211–252, 2015\.
- \[2\]M\. Everingham, L\. Van Gool, C\. K\. I\. Williams, J\. Winn, and A\. Zisserman, “The PASCAL visual object classes \(VOC\) challenge,”*International Journal of Computer Vision*, vol\. 88, no\. 2, pp\. 303–338, 2010\.
- \[3\]T\.\-Y\. Lin, M\. Maire, S\. Belongie, J\. Hays, P\. Perona, D\. Ramanan, P\. Dollár, and C\. L\. Zitnick, “Microsoft COCO: Common objects in context,” in*European Conference on Computer Vision \(ECCV\)*, 2014, pp\. 740–755\.
- \[4\]A\. Wang, A\. Singh, J\. Michael, F\. Hill, O\. Levy, and S\. R\. Bowman, “GLUE: A multi\-task benchmark and analysis platform for natural language understanding,” in*Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP*, 2018, pp\. 353–355\.
- \[5\]A\. Srivastava, A\. Rastogi, A\. Rao*et al\.*, “Beyond the imitation game: Quantifying and extrapolating the capabilities of language models,”*Transactions on Machine Learning Research*, 2023\. \[Online\]\. Available:https://openreview\.net/forum?id=uyTL5Bvosj
- \[6\]Y\. Liu, H\. Duan, Y\. Zhang, B\. Li, S\. Zhang, W\. Zhao, Y\. Yuan, J\. Wang, C\. He, Z\. Liu, K\. Chen, and D\. Lin, “MMBench: Is your multi\-modal model an all\-around player?” in*European Conference on Computer Vision \(ECCV\)*, 2024, pp\. 216–233\.
- \[7\]X\. Yue, Y\. Ni, K\. Zhang, T\. Zheng, R\. Liu, G\. Zhang, S\. Stevens, D\. Jiang, W\. Ren, Y\. Sun, C\. Wei, B\. Yu, R\. Yuan, R\. Sun, M\. Yin, B\. Zheng, Z\. Yang, Y\. Liu, W\. Huang, H\. Sun, Y\. Su, and W\. Chen, “MMMU: A massive multi\-discipline multimodal understanding and reasoning benchmark for expert AGI,” in*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, 2024, pp\. 9556–9567\.
- \[8\]X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang, S\. Zhang, X\. Deng, A\. Zeng, Z\. Du, C\. Zhang, S\. Shen, T\. Zhang, Y\. Su, H\. Sun, M\. Huang, Y\. Dong, and J\. Tang, “AgentBench: Evaluating LLMs as agents,” 2023\. \[Online\]\. Available:https://arxiv\.org/abs/2308\.03688
- \[9\]G\. Mialon, C\. Fourrier, T\. Wolf, Y\. LeCun, and T\. Scialom, “GAIA: A benchmark for general AI assistants,” in*International Conference on Learning Representations \(ICLR\)*, 2024\. \[Online\]\. Available:https://openreview\.net/forum?id=fibxvahvs3
- \[10\]W\. Chow, J\. Mao, B\. Li, D\. Seita, V\. Guizilini, and Y\. Wang, “Physbench: Benchmarking and enhancing vision\-language models for physical world understanding,” in*ICLR*, 2025\. \[Online\]\. Available:https://arxiv\.org/abs/2501\.16411
- \[11\]J\. Yao, Y\. Hu, Y\. Yi, B\. Han, S\. Feng, G\. Yang, B\. Wen, R\. Krishna, L\. L\. Wang, Y\. Tsvetkov, N\. A\. Smith, and B\. Zhu, “Mmmg: a comprehensive and reliable evaluation suite for multitask multimodal generation,” 2025\. \[Online\]\. Available:https://arxiv\.org/abs/2505\.17613
- \[12\]J\. Suo, W\. Zhang, J\. Gong, X\. Yuan, D\. J\. Brady, and Q\. Dai, “Computational imaging and artificial intelligence: The next revolution of mobile vision,”*Proceedings of the IEEE*, vol\. 111, no\. 12, pp\. 1607–1639, 2023\.
- \[13\]C\. Wang, N\. Chen, and W\. Heidrich, “do: A differentiable engine for deep lens design of computational imaging systems,”*IEEE Transactions on Computational Imaging*, vol\. 8, pp\. 905–916, 2022\.
- \[14\]X\. Yang, Q\. Fu, and W\. Heidrich, “Curriculum learning for ab initio deep learned refractive optics,”*Nature communications*, vol\. 15, no\. 1, p\. 6572, 2024\.
- \[15\]W\. H\. Richardson, “Bayesian\-based iterative method of image restoration,”*Journal of the Optical Society of America*, vol\. 62, no\. 1, pp\. 55–59, 1972\.
- \[16\]L\. B\. Lucy, “An iterative technique for the rectification of observed distributions,”*The Astronomical Journal*, vol\. 79, pp\. 745–754, 1974\.
- \[17\]L\. I\. Rudin, S\. Osher, and E\. Fatemi, “Nonlinear total variation based noise removal algorithms,”*Physica D: Nonlinear Phenomena*, vol\. 60, no\. 1\-4, pp\. 259–268, 1992\.
- \[18\]K\. Zhang, W\. Zuo, Y\. Chen, D\. Meng, and L\. Zhang, “Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising,”*IEEE transactions on image processing*, vol\. 26, no\. 7, pp\. 3142–3155, 2017\.
- \[19\]K\. Zhang, W\. Zuo, and L\. Zhang, “Learning a single convolutional super\-resolution network for multiple degradations,” in*Proceedings of the IEEE conference on computer vision and pattern recognition*, 2018, pp\. 3262–3271\.
- \[20\]S\. Antol, A\. Agrawal, J\. Lu, M\. Mitchell, D\. Batra, C\. L\. Zitnick, and D\. Parikh, “VQA: Visual question answering,” in*Proceedings of the IEEE International Conference on Computer Vision \(ICCV\)*, 2015, pp\. 2425–2433\.
- \[21\]H\. Agrawal, K\. Desai, Y\. Wang, X\. Chen, R\. Jain, M\. Johnson, D\. Batra, D\. Parikh, S\. Lee, and P\. Anderson, “nocaps: Novel object captioning at scale,” in*Proceedings of the IEEE/CVF International Conference on Computer Vision \(ICCV\)*, 2019, pp\. 8948–8957\.
- \[22\]J\. Yang, R\. Shi, D\. Wei, Z\. Liu, L\. Zhao, B\. Ke, H\. Pfister, and B\. Ni, “MedMNIST v2: A large\-scale lightweight benchmark for 2d and 3d biomedical image classification,”*Scientific Data*, vol\. 10, no\. 1, p\. 41, 2023\.
- \[23\]J\. Irvin, P\. Rajpurkar, M\. Ko, Y\. Yu, S\. Ciurea\-Ilcus, C\. Chute, H\. Marklund, B\. Haghgoo, R\. Ball, K\. Shpanskaya, J\. Seekins, D\. A\. Mong, S\. S\. Halabi, J\. K\. Sandberg, R\. Jones, D\. B\. Larson, C\. P\. Langlotz, B\. N\. Patel, M\. P\. Lungren, and A\. Y\. Ng, “CheXpert: A large chest radiograph dataset with uncertainty labels and expert comparison,” in*Proceedings of the AAAI Conference on Artificial Intelligence*, vol\. 33, no\. 1, 2019, pp\. 590–597\.
- \[24\]A\. E\. W\. Johnson, T\. J\. Pollard, S\. J\. Berkowitz, N\. R\. Greenbaum, M\. P\. Lungren, C\.\-y\. Deng, R\. G\. Mark, and S\. Horng, “MIMIC\-CXR, a de\-identified publicly available database of chest radiographs with free\-text reports,”*Scientific Data*, vol\. 6, p\. 317, 2019\.
- \[25\]H\. Q\. Nguyen, K\. Lam, L\. T\. Le, H\. H\. Nguyen, H\. H\. Pham, H\. Tong, D\. Dinh, D\. Nguyen, M\. Dao, V\. Vu*et al\.*, “VinDr\-CXR: An open dataset of chest x\-rays with radiologist’s annotations,”*Scientific Data*, vol\. 9, no\. 1, p\. 429, 2022\.
- \[26\]B\. H\. Menze, A\. Jakab, S\. Bauer, J\. Kalpathy\-Cramer, K\. Farahani, J\. Kirby, Y\. Burren, N\. Porz, J\. Slotboom, R\. Wiest*et al\.*, “The multimodal brain tumor image segmentation benchmark \(BRATS\),”*IEEE Transactions on Medical Imaging*, vol\. 34, no\. 10, pp\. 1993–2024, 2015\.
- \[27\]J\. J\. Lau, S\. Gayen, A\. Ben Abacha, and D\. Demner\-Fushman, “A dataset of clinically generated visual questions and answers about radiology images,”*Scientific Data*, vol\. 5, p\. 180251, 2018\.
- \[28\]B\. Liu, L\.\-M\. Zhan, L\. Xu, L\. Ma, Y\. Yang, and X\.\-M\. Wu, “SLAKE: A semantically\-labeled knowledge\-enhanced dataset for medical visual question answering,” in*2021 IEEE 18th International Symposium on Biomedical Imaging \(ISBI\)*, 2021, pp\. 1650–1654\.
- \[29\]X\. He, Y\. Zhang, L\. Mou, E\. Xing, and P\. Xie, “PathVQA: 30000\+ questions for medical visual question answering,” 2020\. \[Online\]\. Available:https://arxiv\.org/abs/2003\.10286
- \[30\]X\. Zhang, C\. Wu, Z\. Zhao, W\. Lin, Y\. Zhang, Y\. Wang, and W\. Xie, “PMC\-VQA: Visual instruction tuning for medical visual question answering,” 2023\. \[Online\]\. Available:https://arxiv\.org/abs/2305\.10415
- \[31\]P\. Chen, J\. Ye, G\. Wang, Y\. Li, Z\. Deng, W\. Li, T\. Li, H\. Duan, Z\. Huang, Y\. Su, B\. Wang, S\. Zhang, B\. Fu, J\. Cai, B\. Zhuang, E\. J\. Seibel, Y\. Qiao, and J\. He, “GMAI\-MMBench: A comprehensive multimodal evaluation benchmark towards general medical ai,” in*Advances in Neural Information Processing Systems*, vol\. 37, 2024\. \[Online\]\. Available:https://proceedings\.neurips\.cc/paper\_files/paper/2024/hash/ab7e02fd60e47e2a379d567f6b54f04e\-Abstract\-Datasets\_and\_Benchmarks\_Track\.html
- \[32\]Y\. Luo, Y\. Yuan, J\. Chen, H\. Cai, Z\. Yue, Y\. Yang, F\. Z\. Daha, J\. Li, and Z\. Lian, “MMMG: A massive, multidisciplinary, multi\-tier generation benchmark for text\-to\-image reasoning,” 2025\. \[Online\]\. Available:https://arxiv\.org/abs/2506\.10963
- \[33\]J\. Chang, V\. Sitzmann, X\. Dun, W\. Heidrich, and G\. Wetzstein, “Hybrid optical\-electronic convolutional neural networks with optimized diffractive optics for image classification,”*Scientific reports*, vol\. 8, no\. 1, p\. 12324, 2018\.
- \[34\]C\. Yang, R\. Dong, and K\.\-M\. Lam, “Vision\-language model guided image restoration,” 2025\. \[Online\]\. Available:https://arxiv\.org/abs/2512\.17292
- \[35\]Y\. Geng, L\. Sun, Y\. Gao, X\. Hu, Z\. Yi, X\. Qian, W\. Hu, J\. Bai, and K\. Wang, “Optiagent: A physics\-driven agentic framework for automated optical design,” 2026\. \[Online\]\. Available:https://arxiv\.org/abs/2602\.23761
- \[36\]L\. Shi, B\. Li, C\. Kim, P\. Kellnhofer, and W\. Matusik, “Towards real\-time photorealistic 3d holography with deep neural networks,”*Nature*, vol\. 591, no\. 7849, pp\. 234–239, 2021\.
- \[37\]Q\. Gao, P\. Duan, H\. Lou, M\. Teng, Z\. Cai, X\. Chen, and B\. Shi, “Unified reconstruction of static and dynamic scenes from events,” in*Proceedings of the Computer Vision and Pattern Recognition Conference*, 2025, pp\. 27 914–27 923\.
- \[38\]X\. Qiao, C\. Ge, P\. Deng, H\. Wei, M\. Poggi, and S\. Mattoccia, “Depth restoration in under\-display time\-of\-flight imaging,”*IEEE transactions on pattern analysis and machine intelligence*, vol\. 45, no\. 5, pp\. 5668–5683, 2022\.
- \[39\]K\. Monakhova, J\. Yurtsever, G\. Kuo, N\. Antipa, K\. Yanny, and L\. Waller, “Learned reconstructions for practical mask\-based lensless imaging,”*Optics express*, vol\. 27, no\. 20, pp\. 28 075–28 090, 2019\.
- \[40\]M\. Afifi, B\. Price, S\. Cohen, and M\. S\. Brown, “When color constancy goes wrong: Correcting improperly white\-balanced images,” in*Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2019, pp\. 1535–1544\.
- \[41\]S\. A Sharif, R\. A\. Naqvi, and M\. Biswas, “Beyond joint demosaicking and denoising: An image processing pipeline for a pixel\-bin image sensor,” in*Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2021, pp\. 233–242\.
- \[42\]S\. W\. Hasinoff, D\. Sharlet, R\. Geiss, A\. Adams, J\. T\. Barron, F\. Kainz, J\. Chen, and M\. Levoy, “Burst photography for high dynamic range and low\-light imaging on mobile cameras,”*ACM Transactions on Graphics \(Proc\. SIGGRAPH Asia\)*, vol\. 35, no\. 6, 2016\.
- \[43\]H\. Chen, M\. Salman Asif, A\. C\. Sankaranarayanan, and A\. Veeraraghavan, “Fpa\-cs: Focal plane array\-based compressive imaging in short\-wave infrared,” in*Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, 2015, pp\. 2358–2366\.
- \[44\]R\. Suvorov, E\. Logacheva, A\. Mashikhin, A\. Remizova, A\. Ashukha, A\. Silvestrov, N\. Kong, H\. Goka, K\. Park, and V\. Lempitsky, “Resolution\-robust large mask inpainting with fourier convolutions,” in*Proceedings of the IEEE/CVF winter conference on applications of computer vision*, 2022, pp\. 2149–2159\.
- \[45\]Google DeepMind, “Gemini 3\.1 flash\-image model card,”https://deepmind\.google/models/model\-cards/gemini\-3\-1\-flash\-image/, 2026, accessed: 2026\-02\-26\.
- \[46\]OpenAI,*GPT\-Image 1\.5 Model Documentation*, OpenAI, San Francisco, CA, 2026, accessed: 2026\-02\-26\. \[Online\]\. Available:https://developers\.openai\.com/api/docs/models/gpt\-image\-1\.5
- \[47\]C\. Wu, J\. Li, J\. Zhou, J\. Lin, K\. Gao, K\. Yan, S\. ming Yin, S\. Bai, X\. Xu, Y\. Chen, Y\. Chen, Z\. Tang, Z\. Zhang, Z\. Wang, A\. Yang, B\. Yu, C\. Cheng, D\. Liu, D\. Li, H\. Zhang, H\. Meng, H\. Wei, J\. Ni, K\. Chen, K\. Cao, L\. Peng, L\. Qu, M\. Wu, P\. Wang, S\. Yu, T\. Wen, W\. Feng, X\. Xu, Y\. Wang, Y\. Zhang, Y\. Zhu, Y\. Wu, Y\. Cai, and Z\. Liu, “Qwen\-image technical report,” 2025\. \[Online\]\. Available:https://arxiv\.org/abs/2508\.02324
- \[48\]A\. Abdelhamed, S\. Lin, and M\. S\. Brown, “A high\-quality denoising dataset for smartphone cameras,” in*IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\)*, June 2018\.
- \[49\]X\. Li, B\. Gunturk, and L\. Zhang, “Image demosaicing: A systematic survey,” in*Visual Communications and Image Processing 2008*, vol\. 6822\. SPIE, 2008, pp\. 489–503\.
- \[50\]S\. Nah, T\. H\. Kim, and K\. M\. Lee, “Deep multi\-scale convolutional neural network for dynamic scene deblurring,” in*CVPR*, July 2017\.
- \[51\]Stanford Computer Graphics Laboratory, “The stanford light field archive \(2016\),”https://lightfields\.stanford\.edu/LF2016\.html, 2016, accessed: 2026\-02\-26\.
- \[52\]E\. Agustsson and R\. Timofte, “Ntire 2017 challenge on single image super\-resolution: Dataset and study,” in*Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\) Workshops*, July 2017\.
- \[53\]V\. Bychkovsky, S\. Paris, E\. Chan, and F\. Durand, “Learning photographic global tonal adjustment with a database of input / output image pairs,” in*The Twenty\-Fourth IEEE Conference on Computer Vision and Pattern Recognition*, 2011\.
- \[54\]Y\. Wu, V\. Boominathan, H\. Chen, A\. Sankaranarayanan, and A\. Veeraraghavan, “Phasecam3d—learning phase masks for passive single view depth estimation,” in*2019 IEEE International Conference on Computational Photography \(ICCP\)*\. IEEE, 2019, pp\. 1–12\.
- \[55\]K\. Wei, Y\. Fu, J\. Yang, and H\. Huang, “A physics\-based noise formation model for extreme low\-light raw denoising,” in*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2020, pp\. 2758–2767\.
- \[56\]M\. Afifi and M\. S\. Brown, “Deep white\-balance editing,” in*Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, 2020\.
- \[57\]K\. Zhang, W\. Zuo, and L\. Zhang, “Ffdnet: Toward a fast and flexible solution for cnn\-based image denoising,”*IEEE Transactions on Image Processing*, vol\. 27, no\. 9, pp\. 4608–4622, 2018\.
- \[58\]F\.\-J\. Tsai, Y\.\-T\. Peng, C\.\-C\. Tsai, Y\.\-Y\. Lin, and C\.\-W\. Lin, “Banet: A blur\-aware attention network for dynamic scene deblurring,”*IEEE Transactions on Image Processing*, vol\. 31, pp\. 6789–6799, 2022\.
- \[59\]X\. Wang, K\. Yu, S\. Wu, J\. Gu, Y\. Liu, C\. Dong, Y\. Qiao, and C\. Change Loy, “Esrgan: Enhanced super\-resolution generative adversarial networks,” in*European Conference on Computer Vision \(ECCV\)*, 2018, pp\. 0–0\.