Paper page - Agentic Visual Generation: From Generative Models to Agentic Control

Source: https://huggingface.co/papers/2609.06758

Abstract

The abstract proposes a four-level framework classifying agentic visual generation by the controller’s direct decision scope over generation operations, from fixed support to experience-adaptive control.

Visual generationis evolving from generative models used through a single invocation intoagentic controlprocesses that can plan, select tools, inspect intermediate synthesized outputs, revise failures, and reuse prior experience. In most existing systems, the controller is an LLM or VLM, whilevisual generationmodels serve as tools or executors. However, existing work lacks a consistent criterion for determining when a generation system becomes agentic. Planning depth, tool use, multi-role collaboration, andreinforcement learningare often treated as evidence of agenticity, even though none of them necessarily determines which generation decisions the controller can make. We organize the field according to what the controller can directly control in the generation process. At L1Conditioning Control, the controller prepares the input to a predetermined generator but does not control which visual operation is executed. At L2Execution Control, it selects and invokes actual generation, editing, rendering, or other content-modifying operations. At L3Outcome-Adaptive Control, it observes an intermediate outcome and uses that observation to change a subsequent operation within the current task. At L4Experience-Adaptive Control, it retains experience from completed tasks and uses that experience to change decisions on future tasks. L0 Fixed Support separately denotes generators, editors, evaluators, reward models, benchmarks, and fixed pipelines without a deployed controller that makes generation-level decisions. These levels describe a progressively broader decision-making scope rather than model size, system complexity, output quality, tool or role count, or training method. Applying this framework across image, video, editing, 3D, world, slide, and user-interface generation reveals how controller capabilities have evolved and how their mechanisms are distributed across levels.

View arXiv pageView PDFGitHub24Add to collection

Get this paper in your agent:

hf papers read 2609\.06758

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.06758 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.06758 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.06758 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.