COMFYCLAW: Self-Evolving Skill Harnesses for Image Generation Workflows

# ComfyClaw: Self-Evolving Skill Harnesses for Image Generation Workflows
Source: [https://arxiv.org/html/2607.01709](https://arxiv.org/html/2607.01709)
Zongxia Li1Dawei Liu211footnotemark:1Fuxiao Liu3Yuhang Zhou1Xiyang Wu1 Jingxi Chen1Jing Xie1Xiaomin Wu1Lichao Sun4 1University of Maryland2University of Pennsylvania3Nvidia4Lehigh University zli12321@umd\.edu liudawei@seas\.upenn\.edu lis221@lehigh\.edu [https://github.com/Moms-Organic-Agent-Lab/comfyclaw](https://github.com/Moms-Organic-Agent-Lab/comfyclaw)Code:[https://github\.com/Moms\-Organic\-Agent\-Lab/comfyclaw](https://github.com/Moms-Organic-Agent-Lab/comfyclaw)

###### Abstract

Agents are increasingly used to construct workflows and help humans complete recurring tasks more efficiently\. As these workflows become repeated and domain\-specific, agent memory and reusable skills become increasingly important: agents should be able to recall workflow patterns, execution constraints, and user preferences from past runs\. We study this problem in workflow\-based image generation and introduceComfyClaw, an agentic skill evolution harness for controlling ComfyUI workflows\.ComfyClawrepresents workflow construction as typed graph editing, exposes tools organized by construction stage, reverts back invalid edits, and uses a region\-level vision\-language model \(VLM\) verifier to translate visual failures into actionable repair suggestions\. The framework further evolves a progressively disclosed skill library, where trajectories, execution errors, and verifier feedback from previous runs are distilled into reusableAgent Skills\. Across four benchmark splits, three agent models, and two image backbones,ComfyClawachieves the best average image\-generation evaluation score across all six agent configurations, outperforming a verifier\-only baseline without skill evolution\. Human annotations further show that annotators preferComfyClawover variants without skill evolution\. Our results suggest that skill evolution is an effective mechanism for improving agent reliability and performance in recurring visual workflow construction\.

## 1Introduction

Agents are moving beyond prompt\-only chat interfaces toward workflow execution\[[11](https://arxiv.org/html/2607.01709#bib.bib155),[60](https://arxiv.org/html/2607.01709#bib.bib115)\]\. This shift is especially important for image generation, where prompt\-only interfaces obscure many of the decisions that shape the final result, such as how conditioning is applied, how visual components are composed, and how generated image failures are detected and repaired\. Workflow systems such as ComfyUI\[[16](https://arxiv.org/html/2607.01709#bib.bib9)\]expose these decisions as editable pipelines, making image generation more inspectable, controllable, and reusable\[[22](https://arxiv.org/html/2607.01709#bib.bib10),[29](https://arxiv.org/html/2607.01709#bib.bib113)\]\. At the same time, this fine\-grained control turns image creation into a workflow problem: an agent must select compatible components, satisfy model constraints, diagnose visual failures, and repair the workflow without breaking execution\. Because similar workflow patterns recur across image\-generation tasks, effective agents must do more than plan and reflect within a single run: they should also acquire reusable skills from prior executions to avoid repeating the same errors\[[7](https://arxiv.org/html/2607.01709#bib.bib57),[36](https://arxiv.org/html/2607.01709#bib.bib58)\]\.

This shift changes the agent’s role from specifying prompts to operating an executable procedure\. Each step is mediated by a harness that exposes graph edits, runtime feedback, and recovery mechanisms; one invalid operation can break execution, and some failures are only revealed after the workflow is run\[[53](https://arxiv.org/html/2607.01709#bib.bib37),[56](https://arxiv.org/html/2607.01709#bib.bib125),[58](https://arxiv.org/html/2607.01709#bib.bib109),[18](https://arxiv.org/html/2607.01709#bib.bib108),[17](https://arxiv.org/html/2607.01709#bib.bib107),[61](https://arxiv.org/html/2607.01709#bib.bib35),[22](https://arxiv.org/html/2607.01709#bib.bib10),[43](https://arxiv.org/html/2607.01709#bib.bib8),[49](https://arxiv.org/html/2607.01709#bib.bib154)\]\. Recent workflow\-generation agents reduce this burden by generating or refining workflows from natural language\[[60](https://arxiv.org/html/2607.01709#bib.bib115),[31](https://arxiv.org/html/2607.01709#bib.bib114),[62](https://arxiv.org/html/2607.01709#bib.bib116)\], while frameworks such as GEMS show promises of closed\-loop refinement, memory, and skills for multimodal generation\[[26](https://arxiv.org/html/2607.01709#bib.bib138)\]\. However, their control interfaces and skill libraries are static, or updated only by passively storing experience as memory rather than actively refining it into reusable skills\. Thus, a refinement that succeeds in one run rarely becomes a validated procedure that can be invoked safely in future workflows\[[53](https://arxiv.org/html/2607.01709#bib.bib37)\]\.

This limitation motivates us that workflow agents need infrastructure for both controlling the current executable state and carrying useful experience for future tasks\. Turning a workflow repair into reusable workflow competence requires two components: a*harness*that exposes tools, feedback, memory, and state transitions to the agent\[[39](https://arxiv.org/html/2607.01709#bib.bib123),[46](https://arxiv.org/html/2607.01709#bib.bib124),[64](https://arxiv.org/html/2607.01709#bib.bib85),[37](https://arxiv.org/html/2607.01709#bib.bib87),[34](https://arxiv.org/html/2607.01709#bib.bib88),[59](https://arxiv.org/html/2607.01709#bib.bib83)\], and*skill management*, which converts repeated trajectories and past execution experiences into reusable procedural knowledge\[[53](https://arxiv.org/html/2607.01709#bib.bib37),[54](https://arxiv.org/html/2607.01709#bib.bib93),[57](https://arxiv.org/html/2607.01709#bib.bib5),[40](https://arxiv.org/html/2607.01709#bib.bib91),[65](https://arxiv.org/html/2607.01709#bib.bib21),[23](https://arxiv.org/html/2607.01709#bib.bib44),[20](https://arxiv.org/html/2607.01709#bib.bib130)\]\. A harness without evolving skills repeatedly rediscovers similar workflow repairs, while skills without a strong harness can become brittle instructions that cause errors for the current workflow\. Thus, the challenge is not only to refine a workflow during one run, but to turn feedback from that run into reusable control knowledge for future runs\. This motivates our question:*can feedback from an agent’s self\-verifier support both immediate workflow repair and the long\-term evolution of a reusable skill library?*

Inspired by prior work on agentic workflow control and self\-evolving agents\[[54](https://arxiv.org/html/2607.01709#bib.bib93),[65](https://arxiv.org/html/2607.01709#bib.bib21),[22](https://arxiv.org/html/2607.01709#bib.bib10),[48](https://arxiv.org/html/2607.01709#bib.bib11)\], we presentComfyClaw, a self\-evolving framework for controlling image\-generation workflows in an*unmodified*ComfyUI runtime\. We cast workflow execution as a skill\-augmented Markov Decision Process over executable graph edits, runtime feedback, verifier feedback, and reusable skills, making harness design and skill reuse explicit parts of the control problem\. Built on this formulation,ComfyClawcombines typed graph editing for workflow construction, VLM\-based verification for diagnosing and repairing visual failures, and a skill\-evolution loop that proposes, validates, and commits reusable Agent Skills\. We evaluateComfyClawacross four benchmark splits, two image backbones, and three agent models, and find that it achieves the best average score, outperforming harness\-only control by 4 absolute points and no\-refinement control by 10 absolute points\. Human annotators further preferComfyClawon 2,400 images, and 318 evolved skills account for roughly50%50\\%of later skill invocations\.

## 2Related Work

Workflow graphs as controllable creative artifacts\.Workflow graphs are common in creative software, from Blender nodes\[[8](https://arxiv.org/html/2607.01709#bib.bib145)\]and Houdini networks\[[47](https://arxiv.org/html/2607.01709#bib.bib146)\]to Nuke compositing\[[21](https://arxiv.org/html/2607.01709#bib.bib147)\]and Unreal Blueprints\[[19](https://arxiv.org/html/2607.01709#bib.bib148)\]\. In these systems, the graph is the artifact: it exposes intermediate structure, supports reuse, and makes complex pipelines easier to inspect than monolithic code\. ComfyUI\[[15](https://arxiv.org/html/2607.01709#bib.bib112)\]brings the same idea to diffusion\-based generation, where users build image, video, and audio\-visual pipelines from node graphs\[[60](https://arxiv.org/html/2607.01709#bib.bib115),[25](https://arxiv.org/html/2607.01709#bib.bib118)\]\. Effective use, however, still depends on knowledge of node compatibility, model constraints, and scattered community recipes\[[3](https://arxiv.org/html/2607.01709#bib.bib119),[50](https://arxiv.org/html/2607.01709#bib.bib120),[52](https://arxiv.org/html/2607.01709#bib.bib121),[29](https://arxiv.org/html/2607.01709#bib.bib113),[25](https://arxiv.org/html/2607.01709#bib.bib118)\]\. Recent work therefore treats ComfyUI as an agent\-control target: ComfyGen\[[22](https://arxiv.org/html/2607.01709#bib.bib10)\]selects workflows from prompts, ComfyGPT\[[29](https://arxiv.org/html/2607.01709#bib.bib113)\]and GenAgent\[[31](https://arxiv.org/html/2607.01709#bib.bib114)\]synthesize graphs or code through multi\-agent collaboration, ComfyUI\-R1\[[60](https://arxiv.org/html/2607.01709#bib.bib115)\]studies RL\-tuned reasoning, and ComfyBench\[[62](https://arxiv.org/html/2607.01709#bib.bib116)\]provides an evaluation setting\. These systems mostly optimize workflow generation or refinement for a single prompt\. In contrast,ComfyClawtreats workflow construction as closed\-loop control: it edits executable graphs through a typed, stage\-gated harness, repairs failures with localized verifier feedback, and promotes reusable skills only after held\-out validation under a graph\-complexity prior\.

Harnessed agents with reusable skills\.Recent agent systems use explicit skills, tool interfaces, and runtime harnesses to make long\-horizon execution more reliable\. Voyager\[[53](https://arxiv.org/html/2607.01709#bib.bib37)\]set an influential template, where an LLM agent improves through an automatic curriculum, executable skill library, and iterative self\-verification\. Later work developed related pieces, including reward design in Eureka\[[38](https://arxiv.org/html/2607.01709#bib.bib122)\], critique and reflection in Self\-Refine\[[39](https://arxiv.org/html/2607.01709#bib.bib123)\]and Reflexion\[[46](https://arxiv.org/html/2607.01709#bib.bib124)\], desktop\-task scaffolding in OS\-Copilot\[[56](https://arxiv.org/html/2607.01709#bib.bib125)\], and declarative pipeline construction in DSPy\[[33](https://arxiv.org/html/2607.01709#bib.bib126)\]\. A parallel line treats skills as reusable agent artifacts\. TheAnthropic Agent Skillsspecification\[[1](https://arxiv.org/html/2607.01709#bib.bib144)\]defines a lightweightSKILL\.mdformat with progressive disclosure, used by systems such as Claude Code\[[4](https://arxiv.org/html/2607.01709#bib.bib143)\], Hermes Agent\[[41](https://arxiv.org/html/2607.01709#bib.bib128)\], and OpenClaw\[[42](https://arxiv.org/html/2607.01709#bib.bib127)\]\. Recent methods also learn or revise skills from experience: SkillRL\[[57](https://arxiv.org/html/2607.01709#bib.bib5)\]builds a hierarchical SkillBank, EvoSkill\[[2](https://arxiv.org/html/2607.01709#bib.bib90)\]mines and repairs skills from failures with held\-out validation, and COS\-PLAY\[[55](https://arxiv.org/html/2607.01709#bib.bib92)\]co\-evolves decision and skill\-bank agents from unlabeled game rollouts\.ComfyClawbrings this idea to workflow control: it combines typed, stage\-gated graph editing with localized verifier feedback and held\-out validation, so workflow skills can be learned, tested, and reused rather than manually written or statically retrieved\.

Skill\-centric agents and self\-improving harnesses\.LLM agents are increasingly designed to act on a user’s behalf over long horizons, often through harnesses that combine tool use, memory, delegation, and reusable skills\. Recent open\-source systems illustrate this shift: DeerFlow 2\.0\[[9](https://arxiv.org/html/2607.01709#bib.bib129)\]uses a LangGraph\-based harness with sandboxed execution, persistent memory, sub\-agents, and extensible skills; OpenClaw\[[42](https://arxiv.org/html/2607.01709#bib.bib127)\]builds a multi\-channel assistant around skill\-based operation; and Hermes Agent\[[41](https://arxiv.org/html/2607.01709#bib.bib128)\]emphasizes reusable skills and autonomous skill creation\. A related research line studies skill accumulation and self\-improvement as learning objectives, including XSkill\[[30](https://arxiv.org/html/2607.01709#bib.bib78)\], EvolveR\[[54](https://arxiv.org/html/2607.01709#bib.bib93)\], and broader surveys of self\-evolving agents\[[23](https://arxiv.org/html/2607.01709#bib.bib44),[20](https://arxiv.org/html/2607.01709#bib.bib130)\]\. Other systems target human\-facing long\-horizon work more directly: Odysseus\[[45](https://arxiv.org/html/2607.01709#bib.bib150)\]studies stable RL for vision\-language models \(VLM\) agents in long\-horizon visual control, while AgentLab\[[44](https://arxiv.org/html/2607.01709#bib.bib151)\]executes research workflows with human feedback\. These systems show the promise of skill\-centric agents, but their skills are often used in broad, open\-ended settings where effectiveness and reuse are hard to evaluate\.

## 3Method

![Refer to caption](https://arxiv.org/html/2607.01709v1/x1.png)Figure 1:Overall framework ofComfyClaw\.The agent edits a ComfyUI workflow graph, the runtime renders a candidate image, the verifier returns requirement\-level and region\-level feedback, and the agent evolves skills that can be reused in future workflow\-construction runs\.ComfyClawis an agentic framework for workflow\-based image generation with reusable workflow skills\. It has three components:*workflow construction*,*verifier\-guided refinement*, and*skill evolution*\. Given a promptpp, an LLM agent constructs a ComfyUI workflow graph through typed edits and submits it to the runtime for rendering\. A VLM verifier scores the image against the prompt and returns localized repair feedback\. The agent uses this feedback, along with runtime errors, to refine the workflow over multiple iterations\. Across prompts, recurring successes and failures are distilled into validated Agent Skills, stored in a skill library, and retrieved in future runs\. Figure[1](https://arxiv.org/html/2607.01709#S3.F1)summarizes the pipeline\.

### 3\.1Preliminaries: Workflows and Agent Skills

ComfyUI workflow graphs\.We represent a ComfyUI workflow as a directed graphG=\(V,E\)G=\(V,E\)\. NodesVVdefine the operations in the generation pipeline, including text encoding, latent sampling, LoRA loading, regional conditioning, upsampling, inpainting, and image decoding\. EdgesEEcarry intermediate outputs from one operation to the next\. Given a promptpp, ComfyUI executes the graph to render an image or video\. This graph\-based interface gives the agent more control than prompt\-only generation, since it can revise both the prompt and the pipeline that processes it\.

Workflow editing\.We treat workflow construction as a sequence of graph edits\. The agent can add or remove nodes, connect or disconnect edges, and adjust node parameters\. Once constructed, the workflow is submitted to ComfyUI for execution\.

Agent skills\.Agent skills are reusable procedures stored asSKILL\.mdfiles\. Each skill contains a brief description, optional triggers, and step\-by\-step instructions for editing a workflow graph\. To keep the context small, the agent is initially shown only lightweight skill metadata, such as the skill name and description, and retrieves the full instructions only when it decides that a skill is relevant\. This progressive\-disclosure design lets skills serve as retrievable workflow\-editing knowledge that can be reused across runs and revised after completed editing attempts\.

### 3\.2Workflow Construction

Planning and initialization\.Given a promptpp, the agent first enters a planning stage, where it identifies the target image\-generation model, reviews relevant tools and skills, and decides how to construct the workflow\. We represent the workflow as a directed acyclic graph \(DAG\),Gt=\(Vt,Et\)G\_\{t\}=\(V\_\{t\},E\_\{t\}\), whereVtV\_\{t\}denotes the set of workflow nodes andEtE\_\{t\}denotes the data\-flow edges at refinement steptt\. The agent begins by constructing an initial workflowG0G\_\{0\}, a minimal*spine*graph that loads the diffusion model, encodes the prompt, samples the latent representation, decodes the image, and saves the output\. This initial graph provides a valid starting point for later refinement\.

Graph editing\.The agent refines the workflow through a sequence of graph\-editing actions,

Gt\+1=Edit​\(Gt,at\),at∈𝒜,G\_\{t\+1\}=\\mathrm\{Edit\}\(G\_\{t\},a\_\{t\}\),\\qquad a\_\{t\}\\in\\mathcal\{A\},\(1\)where𝒜\\mathcal\{A\}includes node insertion, node connection, parameter updates, prompt edits, LoRA insertion, regional conditioning, and refinement passes\. The objective of workflow construction is to produce a final workflowGTG\_\{T\}that reaches the verifier reward threshold \(Section[3\.3](https://arxiv.org/html/2607.01709#S3.SS3)\)\.

Skill retrieval\.Skills are available throughout workflow construction but are exposed selectively through a trigger\-based router\. Each skillσi∈Σ\\sigma\_\{i\}\\in\\Sigmaincludes lightweight metadata, such as a name, short description, tags, and trigger phrases\. Given the current workflow stage and verifier feedback, the agent scores each skill by its relevance to this metadata, and only the top\-KKskills are shown initially\. The full skill content is retrieved only when needed\. This progressive exposure keeps the context small while still allowing access to detailed procedures for workflow construction and repair\.

Construction stages\.After creating the spine graph, the agent enters the construction stage, where it edits the workflow DAG by adding or connecting nodes, setting parameters, and modifying the graph structure to satisfy the prompt requirements\. It then proceeds to an enhancement stage, where it can apply higher\-level workflow changes such as adding LoRA modules, regional attention, mask\-based conditioning, or refinement passes\. Together, these stages correspond to the planning, construction, and enhancement blocks shown at the top of Figure[1](https://arxiv.org/html/2607.01709#S3.F1)\.

### 3\.3Verifier\-Guided Refinement

After the agent submitsGtG\_\{t\}, ComfyUI renders an imageItI\_\{t\}which is passed to a VLM verifier\. The verifier first decomposesppinto a checklist of observable binary requirementsQ=\{qi\}Q=\\\{q\_\{i\}\\\}\. For eachqiq\_\{i\}, it returns a binary pass/fail label grounded in the image, together with a short natural\-language justification; in addition it returns a holistic detail scoresdet∈\[0,10\]s\_\{\\mathrm\{det\}\}\\\!\\in\\\!\[0,10\]that captures overall fidelity, composition, and absence of visual artifacts\. The harness combines the two signals into the scalar reward of Eq\.[2](https://arxiv.org/html/2607.01709#S3.E2)\.

#### Vision\-Language Model \(VLM\) as a Verifier\.

We use the agent as verifier to evaluate whether a generated image satisfies the prompt\. The verifier decomposes the prompt into observable requirementsQ=\{qi\}Q=\\\{q\_\{i\}\\\}, such as object count, attribute binding, spatial relation, style, and anatomy\. It returns requirement\-level pass or fail labels, a holistic detail score, localized failure descriptions, and concrete suggestions for workflow edits\. The score used by the harness is

r=0\.6⋅\|Qpass\|\|Q\|\+0\.4⋅sdet10,r\\;=\\;0\.6\\cdot\\\!\\frac\{\|Q^\{\\mathrm\{pass\}\}\|\}\{\|Q\|\}\\;\+\\;0\.4\\cdot\\\!\\frac\{s\_\{\\mathrm\{det\}\}\}\{10\},\(2\)wheresdets\_\{\\mathrm\{det\}\}is an additional one\-pass quality score, ranging from 1 to 10, that measures the overall image quality and its alignment with the prompt\.

#### Workflow Refinement loop\.

The verifier emits three structured components that drive the next iteration: \(i\) the failing\-requirement setQ∖QpassQ\\\!\\setminus\\\!Q^\{\\mathrm\{pass\}\}, \(ii\) localized natural\-language descriptions of what went wrong \(e\.g\.the leftmost figure has three arms\), and \(iii\) concrete edit suggestions phrased in workflow terms \(apply regional prompting to isolate the throwing arm\)\. These pieces are appended to the agent’s next\-iteration context and also forwarded to the skill\-evolution loop \(§[3\.4](https://arxiv.org/html/2607.01709#S3.SS4)\)\. The refinement loop terminates whenrrexceeds a satisfaction thresholdτstop\\tau\_\{\\mathrm\{stop\}\}or afterKKiterations, whichever comes first; the iteration with the highestrris committed as the final output\.

### 3\.4Skill Evolution

The skill library serves as the long\-term memory ofComfyClaw\. Rather than treating each prompt independently,ComfyClawconverts recurring successes and failures across prompts into reusable workflow procedures\. Skill evolution proceeds in four stages: clustering success and failure traces, proposing skill mutations, validating the proposed mutations, and committing accepted skills to the skill library\.

Success and Failure Clustering\.Let𝒮\(m\)\\mathcal\{S\}^\{\(m\)\}denote the skill library after evolution cyclemm\. After a batch ofBBprompts, the workflow\-construction loop produces traces

𝒟\(m\)=\{\(pi,Gi,xi,ri,fi,ei,ai,1:Ti\)\}i=1B,\\mathcal\{D\}^\{\(m\)\}=\\left\\\{\\left\(p\_\{i\},G\_\{i\},x\_\{i\},r\_\{i\},f\_\{i\},e\_\{i\},a\_\{i,1:T\_\{i\}\}\\right\)\\right\\\}\_\{i=1\}^\{B\},\(3\)wherepip\_\{i\}is the prompt,GiG\_\{i\}is the final workflow,xix\_\{i\}is the generated image,rir\_\{i\}is the verifier reward,fif\_\{i\}is verifier feedback,eie\_\{i\}is a runtime error if one occurs, andai,1:Tia\_\{i,1:T\_\{i\}\}is the sequence of workflow actions\. As shown in the bottom part of Figure[1](https://arxiv.org/html/2607.01709#S3.F1), these success and failure traces are sent to the skill evolution module\.

We divide traces into success and failure groups using the verifier reward\. A trace is treated as successful ifri≥0\.9r\_\{i\}\\geq 0\.9and as a failure otherwise\. We then cluster the success and failure traces separately according to their verifier feedback, runtime errors, workflow actions, and prompt properties\. Failure clusters capture recurring failure modes, such as missing objects, incorrect counting, and weak spatial binding\. Success clusters capture reusable workflow strategies, such as when to add a LoRA, when to use regional attention, how to adjust guidance, or how to structure prompts\.

Skill Mutation\.For each clusterCℓC\_\{\\ell\}, we use the agent as a skill evolver to propose a mutation to the current skill library\. The mutation type is selected from

μℓ∈\{create,revise,reinforce,merge,delete\}\.\\mu\_\{\\ell\}\\in\\\{\\texttt\{create\},\\texttt\{revise\},\\texttt\{reinforce\},\\texttt\{merge\},\\texttt\{delete\}\\\}\.\(4\)Acreatemutation adds a new procedure for a pattern not covered by existing skills\. Arevisemutation updates the content of an existing skill\. Areinforcemutation strengthens useful triggers or emphasis terms\. Amergemutation combines redundant skills, and adeletemutation removes skills that are no longer useful\. Applying the proposed mutation to the current library gives a candidate skill library

𝒮~ℓ\(m\+1\)=μℓ​\(𝒮\(m\)\)\.\\widetilde\{\\mathcal\{S\}\}\_\{\\ell\}^\{\(m\+1\)\}=\\mu\_\{\\ell\}\\\!\\left\(\\mathcal\{S\}^\{\(m\)\}\\right\)\.\(5\)
Mutation Validation\.Each candidate mutation is validated before it is committed to the skill library\. For clusterCℓC\_\{\\ell\}, we ask the agent to synthesize three held\-out prompts conditioned on the cluster identifier, the candidate skill name, and up to five example prompts from the cluster\. These prompts are intended to test whether the proposed mutation generalizes beyond the examples that produced it\.

Letℋℓ\\mathcal\{H\}\_\{\\ell\}denote the validation prompt set for clusterCℓC\_\{\\ell\}\. We compare the average verifier reward obtained with the current skill library𝒮\(m\)\\mathcal\{S\}^\{\(m\)\}against the reward obtained with the candidate library𝒮~ℓ\(m\+1\)\\widetilde\{\\mathcal\{S\}\}\_\{\\ell\}^\{\(m\+1\)\}:

Δℓ=1\|ℋℓ\|​∑p∈ℋℓ\[r​\(p;𝒮~ℓ\(m\+1\)\)−r​\(p;𝒮\(m\)\)\]\.\\Delta\_\{\\ell\}=\\frac\{1\}\{\|\\mathcal\{H\}\_\{\\ell\}\|\}\\sum\_\{p\\in\\mathcal\{H\}\_\{\\ell\}\}\\left\[r\\\!\\left\(p;\\widetilde\{\\mathcal\{S\}\}\_\{\\ell\}^\{\(m\+1\)\}\\right\)\-r\\\!\\left\(p;\\mathcal\{S\}^\{\(m\)\}\\right\)\\right\]\.\(6\)The mutation is accepted only if it does not degrade validation performanceΔℓ≥0\.\\Delta\_\{\\ell\}\\geq 0\.Rejected mutations are rolled back, while accepted mutations are committed as new skill versions\.

Table 1:ComfyClawachieves the best overall performance across the four benchmarks\.All scores are reported on a 0–100 scale, where higher is better\. Scores are judged by Qwen3\-VL\-8B\-Instruct using each benchmark’s evaluation metric: Soft\-TIFA Geometric Mean for GenEval2, Soft\-TIFA Arithmetic Mean for DPG\-Bench, and VQAScore for OneIG\-EN and OneIG\-ZH\. The best score in each column is shown in bold\.

## 4Experiments

In this section, we evaluateComfyClawon four text\-to\-image benchmark splits against competitive baselines\. We further present ablation studies and qualitative analyses that illustrate howComfyClawimproves through iterative workflow refinement\.

Benchmarks\.We evaluate on four text\-to\-image benchmark splits covering compositional reasoning, dense prompt following, fine\-grained fidelity, and cross\-lingual generalization\.GenEval 2\[[32](https://arxiv.org/html/2607.01709#bib.bib131)\]includes 800 English prompts over objects, attributes, and spatial or numerical relations, scored with the atom\-level Soft\-TIFA VQA judge to avoid saturation in the original GenEval\.DPG\-Bench\[[27](https://arxiv.org/html/2607.01709#bib.bib132)\]contains 1,065 dense English prompts with multiple objects, attributes, and relations in each sentence\.OneIG\-Bench\[[12](https://arxiv.org/html/2607.01709#bib.bib133)\]evaluates subject–element alignment, text rendering, reasoning content, stylization, and diversity\. We use its English split \(OneIG\-EN, 1,120 prompts\) and Chinese split \(OneIG\-ZH, 1,320 prompts\)\. Together, these benchmarks contain roughly 4,300 prompts and test the dimensions where workflow evolution should differ most from prompt\-only refinement\.

Agents and ComfyUI models used\.We use three foundation models as workflow\-control agents: Claude Sonnet 4\.5\[[5](https://arxiv.org/html/2607.01709#bib.bib134)\], Qwen\-3\.6\-35B\-A3B\[[63](https://arxiv.org/html/2607.01709#bib.bib53)\], and Gemma\-4\-E4B\-it\[[24](https://arxiv.org/html/2607.01709#bib.bib135)\]\. All are natively multimodal and are used to instantiate the workflow, verifier, and skill\-evolution agents\. For image generation, we evaluate two ComfyUI backbones: z\-image\-turbo\[[10](https://arxiv.org/html/2607.01709#bib.bib136)\], a 6B text\-to\-image model, and LongCat\-Image\[[51](https://arxiv.org/html/2607.01709#bib.bib137)\], a 6B bilingual Chinese–English model for image generation and editing\. The agents edit the ComfyUI workflow graph, while the selected backbone renders the final image\.

Workflow and tool setup\.To closely simulate how an external agent would control a real ComfyUI deployment, we instantiate a fresh ComfyUI server endpoint for each experimental condition\. Each endpoint runs an unmodified ComfyUI server, while our agent is deployed as an external plug\-in that submits jobs and retrieves execution logs through the standard interface\. This design allows our framework to interact with ComfyUI without modifying the application itself, making the setup easy to maintain and naturally compatible with any future ComfyUI updates\.

We provide a set of workflow\-editing tools for controlling ComfyUI workflows, together with four predefined visual\-quality skills byHeet al\.\[[26](https://arxiv.org/html/2607.01709#bib.bib138)\]\. During execution, the agent does more than prompt rewriting: it can edit the workflow graph, tune node hyperparameters, and, when supported by the image model, attach and configure LoRAs\. The full list of predefined tools, predefined skills, and model\-specific LoRA settings is provided in Appendix[A](https://arxiv.org/html/2607.01709#A1)\.

Image Generation Evaluation Metrics\.We follow the headline metric defined by each benchmark\. For GenEval2, we report the Soft\-TIFA geometric mean\[[28](https://arxiv.org/html/2607.01709#bib.bib140)\]\(gm\), computed over the per\-prompt set of VQA questions, including yes/no and counting questions\[[35](https://arxiv.org/html/2607.01709#bib.bib141)\]\. For DPG\-Bench, we report the Soft\-TIFA arithmetic mean \(am\), computed over each prompt’s corresponding set of evaluation questions\. For OneIG\-EN and OneIG\-ZH, we use VQAScore, where each image is evaluated with a single query of the form:Does this image show <prompt\>? Answer Yes or No\.All of these metrics rely on a VLM\-as\-a\-judge\. In our experiments, we use Qwen3\-VL\-8B\-Instruct\[[6](https://arxiv.org/html/2607.01709#bib.bib139)\]as the primary judge model to answer the benchmark evaluation queries\.

### 4\.1Main Results

We compareComfyClawagainst a baseline that uses only the initial tools and skill set\. It does not receive verifier feedback or perform refinement; instead, it executes the workflow once and directly returns the generated image\. All experiments and evaluations are run on an RTX PRO 6000 Blackwell GPU with 96 GB of memory\. We report the results in Table[1](https://arxiv.org/html/2607.01709#S3.T1)\. Overall,ComfyClawoutperformsBaseacross all four benchmarks\. These results suggest two main takeaways\. First, adding a VLM verifier and refinement loop helps agents construct better workflows and generate higher\-quality images\. Second, skill evolution further improves performance by enabling agents to summarize successes and failures, learn from past errors, and update their skill set for future workflow construction\.

DPGGenEval2OneIG\-ENOneIG\-ZH03691270\.0%56\.2%16\.3%7\.5%\# Skill reads \(k\)Predefined skillsEvolved skills\(a\)Skill\-read distribution per benchmark\(full counts in Table[6](https://arxiv.org/html/2607.01709#A2.T6)\)\. Evolved skills account for70\.0%\\mathbf\{70\.0\\%\}and56\.2%\\mathbf\{56\.2\\%\}of all skill reads on DPG and GenEval2, but only7\.57\.5–16\.3%16\.3\\%on the OneIG splits, indicating benchmark\-dependent reliance on agent\-evolved knowledge\.0252550507575100100OneIG\-ZH\[\-1pt\]N=9\.6​kN\{=\}9\.6\\mathrm\{k\}OneIG\-EN\[\-1pt\]N=8\.4​kN\{=\}8\.4\\mathrm\{k\}GenEval2\[\-1pt\]N=10\.9​kN\{=\}10\.9\\mathrm\{k\}DPG\[\-1pt\]N=6\.7​kN\{=\}6\.7\\mathrm\{k\}274341224139393843Composition of workflow events \(%\)Prompt\-textSamplerRegionalModel/weightMulti\-passOther\(b\)Workflow\-edit composition per benchmark\(each bar=100%=100\\%;NN= total events per benchmark shown below each axis label; full counts in Table[7](https://arxiv.org/html/2607.01709#A2.T7)\)\. Non\-prompt edits make up60\.7%\\mathbf\{60\.7\\%\}of all events; GenEval2 uniquely allocates𝟐𝟕%\\mathbf\{27\\%\}of edits to regional/mask topology vs\.33–6%6\\%on the other splits\.
Figure 2:Evolved\-skill usage and workflow\-edit behavior ofComfyClawon Claude\-Sonnet\-4\.5\(aggregated over LongCat\-Image and Z\-Image\-Turbo\)\.*Left:*agents read evolved skills heavily on dense / compositional benchmarks \(DPG, GenEval2\) and predefined skills more on the OneIG splits\.*Right:*only∼39%\\sim\\\!39\\%of edits are prompt\-text rewrites; the agent spends the rest on hyperparameters, graph topology, model/LoRA choices, and multi\-pass design, showing thatComfyClawperforms broad workflow construction rather than pure prompt engineering\.Evolved skills capture reusable patterns for recurring image\-generation workflows\.Table[2](https://arxiv.org/html/2607.01709#S4.T2)shows representative evolved skills from different benchmarks\. Each skill is a namedSKILL\.mdprocedure that captures a recurring workflow\-execution pattern rather than a one\-off memory\. For GenEval2, skills such asspatial\-anchor\-with\-count,spatial\-count\-binding, andattribute\-bindingencode reusable procedures for preserving object counts, attribute bindings, and spatial layouts in compositional scenes\. For DPG\-Bench, skills such asmaterial\-texture\-detail,precise\-color\-attribution, andlighting\-and\-reflection\-detailfocus on fine\-grained control of material appearance, color assignment, and illumination\. For OneIG\-EN and OneIG\-ZH, the learned skills shift toward anime\-specific character and style control, includingcharacter\-counting,anime\-danbooru\-ordering, andanime\-single\-character\-simple\. These examples show that evolved skills are not generic memories, but benchmark and style\-specific reusable procedures for recurring workflow problems\.

Representative Evolved SkillsRepresentative skills by benchmarkTable 2:Representative evolved skills inComfyClaw\. The learned skills are benchmark\-specific reusable procedures rather than raw memories, capturing recurring workflow\-execution patterns such as count and spatial binding, fine\-grained appearance control, and anime\-specific character prompting\.Evolved skills are heavily used by agents\.We analyze the skills evolved by Claude\-Sonnet\-4\.5 across four benchmarks and two image\-generation models\. As shown in Figure[2\(a\)](https://arxiv.org/html/2607.01709#S4.F2.sf1), evolved skills account for an average of∼50%\\sim\\\!50\\%of total skill reads across benchmarks \(specifically70\.0%70\.0\\%on DPG\-Bench and56\.2%56\.2\\%on GenEval2, dropping to16\.3%16\.3\\%on OneIG\-EN and7\.5%7\.5\\%on OneIG\-ZH\), indicating that agents rely heavily on them when constructing image\-generation workflows for dense and compositional prompts\. Across the eight experimental settings, Claude\-Sonnet\-4\.5 produced318318unique skills, totaling4,7684\{,\}768skill versions\. These skills reflect the distinct requirements of each benchmark\. GenEval2 primarily induces object\-counting and attribute\-binding skills; DPG\-Bench emphasizes spatial relations, poses, and anatomy; OneIG\-EN favors anime\-style prompting and character\-count control; and OneIG\-ZH further introduces Chinese\-specific anime conventions and resolution\-tag skills\.111The top ten evolved skills are listed in Appendix[C](https://arxiv.org/html/2607.01709#A3)\.The evolved skills provide reusable, task\-specific patterns that help agents start from stronger knowledge and generate higher\-quality images\.

Agents perform more complex actions than simple prompt rewriting\.We track the actions taken by the Claude agent when constructing workflows inComfyClaw, as shown in Figure[2\(b\)](https://arxiv.org/html/2607.01709#S4.F2.sf2)\. Across35,61235\{,\}612workflow events, prompt\-text edits are the most frequent single category but account for only39\.3%39\.3\\%of all edits\. The remaining60\.7%60\.7\\%modify other parts of the workflow, including sampler and guidance hyperparameters \(35\.8%35\.8\\%\), regional or mask\-based graph topology \(11\.4%11\.4\\%, with a striking27%27\\%on GenEval2\), LoRA / checkpoint selection \(7\.3%7\.3\\%\), and multi\-pass upscaling \(3\.2%3\.2\\%\)\. This shows thatComfyClawimproves image generation through broader workflow optimization rather than relying only on prompt refinement\.

Promptdog with exactly six bagelsmushroom placed under five backpackscat, black TV reflection, text “oops i binge” / “too many shows”bookshop, black cat, signs “RARE FINDS INSIDE” and “CHAPTERS & CHARMS”boy and girl on glass walkway, whale shark beneathmetallic blue sphere left of larger yellow felt box on gray surfaceBase Ours ![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case01_baseline.jpg)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case02_baseline.jpg)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case03_baseline.jpg)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case04_baseline.jpg)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case05_baseline.jpg)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case06_baseline.jpg)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case01_comfyclaw.png)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case02_comfyclaw.png)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case03_comfyclaw.png)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case04_comfyclaw.png)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case05_comfyclaw.png)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_cherry_pick/thumbs/case06_comfyclaw.png)

Figure 3:Qualitative comparison across methods on six prompts spanning five capability categories\.Each column is a prompt \(header shows category and full description\); rows are*Base*\(single\-pass baseline\) and*Ours*\(ComfyClaw, green border\)\.ComfyClawmore reliably realises object counts, spatial relations, scene\-text accuracy, and fine\-grained attribute control\.
### 4\.2Qualitative Analysis

We further conduct a qualitative analysis of the generated images, since quantitative metrics alone may not fully capture visual quality in image generation tasks\. Our goal is to compare how different harness designs affect the agent’s ability to control the workflow and produce high\-quality images\. We hired two annotators to rate image quality on a 1–5 Likert scale, where higher scores indicate better visual quality and stronger text\-image alignment\. For each benchmark and each agent, we randomly sample 50 finalized images\. Annotators were shown the input prompt together with the generated images from different experimental groups and asked to assign a Likert rating to each image\.222Annotation cost and instruction details are in Appendix[D](https://arxiv.org/html/2607.01709#A4)\.In total, 2,400 images are annotated\. We show the results in Table[3](https://arxiv.org/html/2607.01709#S4.T3)\. Overall, images generated usingComfyClawalign more with input prompts and exhibit greater visual realism and aesthetics\.

Table 3:Qualitative evaluation of Claude as an agent to control the ComfyUI workflow\.OverallComfyClawhas higher average human visual annotation scores thanComfyGEMSandBase\.Closed\-loop workflow refinement repairs failures that prompt\-only rewriting often cannot\.Figure[4](https://arxiv.org/html/2607.01709#S4.F4)showsComfyClaw’s refinement loop for editing the workflow\.First, the edits are structural rather than purely textual prompt revision\. In Figure[4](https://arxiv.org/html/2607.01709#S4.F4)\(a\), the agent stacks two Z\-Image LoRAs and adds a regional\-attention block before the unusual attribute*purple*is correctly bound to all four lions\. In Figure[4](https://arxiv.org/html/2607.01709#S4.F4)\(b\), the agent adjusts the regional split until the spatial relation*left of*, the material*glass*, and the count*three*are all satisfied\.

Second, refinement is not always monotonic\. Some intermediate attempts are worse than the baseline, such as when the pigs disappear or the clock is lost, but the best\-so\-far buffer preserves the best valid candidate while still allowing the agent to learn from failed attempts and recover in later iterations\.

Third, the loop is verifier\-driven\. Each refinement instruction directly responds to the preceding critique, such as correcting the lion color or removing an ineffective LoRA after objects disappear\. Thus, the agent is not retrying blindly; it spends additional graph edits and re\-renders on the specific failures localized by the verifier\. These extra operations improve compositional errors, including attribute binding, object count, spatial relations, LoRA weight stacking, and hyperparameter adjustments that are difficult for single\-pass prompt\-only generation to fix reliably\.

\(a\)four purple lions\.Force unusual color\.Add regional control\.Stack Z\-Image LoRAs\.![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_refinement/thumbs/lions_source.png)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_refinement/thumbs/lions_iter1.png)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_refinement/thumbs/lions_iter2.png)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_refinement/thumbs/lions_final.png)InitialIter 1Iter 2FinalLions tan, not purple\.6 lions; 2 purple\.5 lions; 1 still tan\.4 lions, all purple\.\(b\)clock left of three glass pigs\.Split regions \+ LoRA\.Drop LoRA; force glass\.Simplify regions; count\.![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_refinement/thumbs/pigs_source.png)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_refinement/thumbs/pigs_iter1.png)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_refinement/thumbs/pigs_iter2.png)![Refer to caption](https://arxiv.org/html/2607.01709v1/figures/comfyclaw_refinement/thumbs/pigs_final.png)InitialIter 1Iter 2FinalPigs not glass\.Pigs gone; clock only\.Glass OK; clock lost\.Clock left, 3 glass pigs\.

Figure 4:Iterative visual refinement underComfyClaw\.\(a\)*four purple lions*\(verifier0\.33→0\.910\.33\{\\to\}0\.91\); \(b\)*a clock to the left of three glass pigs*\(verifier0\.31→0\.960\.31\{\\to\}0\.96\)\. Each strip reads left to right: the agent inspects the current image, the verifier emits a critique \(blue\), the agent turns it into a refinement instruction \(orange\) for the next pass, and re\-executes the workflow; the selected best output is highlighted ingreen\. Refinement repairs unusual\-color attributes, conflicting workflow components, or spatial\-relation failures that the single\-pass baseline could not resolve\.

## 5Conclusion

Agentic workflow control is a promising direction for helping users, especially beginners, operate complex tools and automate multi\-step workflows\. In image generation, workflow\-based systems provide more controllability than prompt\-only interfaces by exposing components such as model selection, conditioning, sampling, refinement, and LoRA adapters\. We proposeComfyClaw, a self\-evolving workflow\-control framework that converts past successes and failures into reusable skills, enabling agents to improve how they construct and refine image\-generation workflows over time\. Our experiments show that this self\-evolving skill mechanism produces higher quality and more realistic images than agent workflow control without skill evolution, and is more preferred by human annotators\. These results suggest that effective visual generation agents should not only execute workflows, but also learn and evolve reusable workflow procedures from their own experience\.

## 6Limitations and Future Work

Our current study focuses only on image\-generation workflows\. However, ComfyUI also supports video\-generation workflows, which require more complex node control, scene planning, frame consistency, and temporal conditioning\. Video generation also requires substantially more compute and longer execution time, making agent control more difficult because long\-running workflows can increase latency, timeout failures, and response errors\. We therefore leave video workflow control as an important direction for future work\. Future systems will need stronger workflow\-control strategies, better skill management, and more efficient skill retrieval, since video workflows are typically longer and more complex than image workflows\. Managing the agent context window will also become more challenging, making compact and accurate skill retrieval especially important\.

## References

- \[1\]Agent Skills\(2026\)Agent skills specification\.Note:[https://agentskills\.io/specification](https://agentskills.io/specification)Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p2.1)\.
- \[2\]S\. Alzubi, N\. Provenzano, J\. Bingham, W\. Chen, and T\. Vu\(2026\)Evoskill: automated skill discovery for multi\-agent systems\.arXiv preprint arXiv:2603\.02766\.Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p2.1)\.
- \[3\]T\. Angert, M\. Suzara, J\. Han, C\. Pondoc, and H\. Subramonyam\(2023\)Spellburst: a node\-based interface for exploratory creative coding with natural language prompts\.InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology,pp\. 1–22\.Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[4\]Anthropic\(2025\)Claude code\.Note:[https://code\.claude\.com/docs/en/overview](https://code.claude.com/docs/en/overview)Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p2.1)\.
- \[5\]Anthropic\(2025\)Claude\.Note:Accessed: 2026\-04\-30External Links:[Link](https://www.anthropic.com/claude)Cited by:[§4](https://arxiv.org/html/2607.01709#S4.p3.1)\.
- \[6\]S\. Bai, Y\. Cai, R\. Chen, K\. Chen, X\. Chen, Z\. Cheng, L\. Deng, W\. Ding, C\. Gao, C\. Ge,et al\.\(2025\)Qwen3\-vl technical report\.arXiv preprint arXiv:2511\.21631\.Cited by:[§4](https://arxiv.org/html/2607.01709#S4.p6.1)\.
- \[7\]T\. Bai, Z\. Wan, P\. Zhou, X\. Yu, W\. Zhao, Y\. You, and I\. W\. Tsang\(2026\)SkillDAG: self\-evolving typed skill graphs for llm skill selection at scale\.arXiv preprint arXiv:2606\.03056\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p1.1)\.
- \[8\]Blender Foundation\(2026\)Blender\.Note:[https://www\.blender\.org](https://www.blender.org/)Accessed: 2026\-04\-29Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[9\]ByteDance\(2025\)DeerFlow: deep exploration and efficient research flow\.GitHub\.Note:Accessed: 2026\-04\-29External Links:[Link](https://github.com/bytedance/deer-flow)Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p3.1)\.
- \[10\]H\. Cai, S\. Cao, R\. Du, P\. Gao, S\. Hoi, Z\. Hou, S\. Huang, D\. Jiang, X\. Jin, L\. Li,et al\.\(2025\)Z\-image: an efficient image generation foundation model with single\-stream diffusion transformer\.arXiv preprint arXiv:2511\.22699\.Cited by:[Appendix A](https://arxiv.org/html/2607.01709#A1.SS0.SSS0.Px3.p1.1),[§4](https://arxiv.org/html/2607.01709#S4.p3.1)\.
- \[11\]P\. Cao, F\. Zhou, Q\. Song, and L\. Yang\(2025\)Controllable generation with text\-to\-image diffusion models: a survey\.IEEE Transactions on Pattern Analysis and Machine Intelligence\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p1.1)\.
- \[12\]J\. Chang, Y\. Fang, P\. Xing, S\. Wu, W\. Cheng, R\. Wang, X\. Zeng, G\. Yu, and H\. Chen\(2025\)OneIG\-bench: omni\-dimensional nuanced evaluation for image generation\.arXiv preprint arxiv:2506\.07977\.Cited by:[§4](https://arxiv.org/html/2607.01709#S4.p2.1)\.
- \[13\]Civitai Community Model\(2026\)Enhancer\.Note:[https://civitai\.com/models/2239743/enhancer?modelVersionId=2521349](https://civitai.com/models/2239743/enhancer?modelVersionId=2521349)Cited by:[Appendix A](https://arxiv.org/html/2607.01709#A1.SS0.SSS0.Px3.p1.1)\.
- \[14\]Civitai Community Model\(2026\)Realistic snapshot \(z\-image\-turbo\)\.Note:[https://civitai\.com/models/2268008/realistic\-snapshot\-z\-image\-turbo?modelVersionId=2617751](https://civitai.com/models/2268008/realistic-snapshot-z-image-turbo?modelVersionId=2617751)Cited by:[Appendix A](https://arxiv.org/html/2607.01709#A1.SS0.SSS0.Px3.p1.1)\.
- \[15\]Comfyanonymous\(2023\)ComfyUI: the most powerful and modular diffusion model gui, api and backend with a graph/nodes interface\.GitHub\.Note:Accessed: 2026\-04\-29External Links:[Link](https://github.com/comfyanonymous/ComfyUI)Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[16\]ComfyUI\(2026\)Launching ComfyUI registry\.Note:[https://blog\.comfy\.org/p/launching\-comfyui\-registry](https://blog.comfy.org/p/launching-comfyui-registry)Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p1.1)\.
- \[17\]T\. L\. S\. de Chezelles, M\. Gasse, A\. Lacoste, M\. Caccia, A\. Drouin, L\. Boisvert, M\. Thakkar, T\. Marty, R\. Assouel, S\. O\. Shayegan, L\. K\. Jang, X\. H\. Lù, O\. Yoran, D\. Kong, F\. F\. Xu, S\. Reddy, G\. Neubig, Q\. Cappart, R\. Salakhutdinov, and N\. Chapados\(2025\)The browsergym ecosystem for web agent research\.Transactions on Machine Learning Research\.Note:Accessed: 2026\-04\-29External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=5298fKGmv3)Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p2.1)\.
- \[18\]A\. Drouin, M\. Gasse, M\. Caccia, I\. H\. Laradji, M\. Del Verme, T\. Marty, D\. Vazquez, N\. Chapados, and A\. Lacoste\(2024\-21–27 Jul\)WorkArena: how capable are web agents at solving common knowledge work tasks?\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 11642–11662\.External Links:[Link](https://proceedings.mlr.press/v235/drouin24a.html)Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p2.1)\.
- \[19\]Epic Games\(2026\)Unreal engine\.Note:[https://www\.unrealengine\.com](https://www.unrealengine.com/)Accessed: 2026\-04\-29Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[20\]J\. Fang, Y\. Peng, X\. Zhang, Y\. Wang, X\. Yi, G\. Zhang, Y\. Xu, B\. Wu, S\. Liu, Z\. Li,et al\.\(2025\)A comprehensive survey of self\-evolving ai agents: a new paradigm bridging foundation models and lifelong agentic systems\.arXiv preprint arXiv:2508\.07407\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1),[§2](https://arxiv.org/html/2607.01709#S2.p3.1)\.
- \[21\]Foundry\(2026\)Nuke\.Note:[https://www\.foundry\.com/products/nuke\-family/nuke](https://www.foundry.com/products/nuke-family/nuke)Accessed: 2026\-04\-29Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[22\]R\. Gal, A\. Haviv, Y\. Alaluf, A\. H\. Bermano, D\. Cohen\-Or, and G\. Chechik\(2024\)Comfygen: prompt\-adaptive workflows for text\-to\-image generation\.arXiv preprint arXiv:2410\.01731\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p1.1),[§1](https://arxiv.org/html/2607.01709#S1.p2.1),[§1](https://arxiv.org/html/2607.01709#S1.p4.1),[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[23\]H\. Gao, J\. Geng, W\. Hua, M\. Hu, X\. Juan, H\. Liu, S\. Liu, J\. Qiu, X\. Qi, Y\. Wu,et al\.\(2025\)A survey of self\-evolving agents: on path to artificial super intelligence\.arXiv preprint arXiv:2507\.210461\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1),[§2](https://arxiv.org/html/2607.01709#S2.p3.1)\.
- \[24\]Gemma Team, Google DeepMind\(2026\)Gemma 4\.External Links:[Link](https://ai.google.dev/gemma/docs/core/model_card_4)Cited by:[§4](https://arxiv.org/html/2607.01709#S4.p3.1)\.
- \[25\]L\. Guo, X\. Xu, L\. Wang, J\. Lin, J\. Zhou, Z\. Zhang, B\. Su, and Y\. Chen\(2025\)Comfymind: toward general\-purpose generation via tree\-based planning and reactive feedback\.arXiv preprint arXiv:2505\.17908\.Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[26\]Z\. He, S\. Huang, X\. Qu, Y\. Li, T\. Zhu, Y\. Cheng, and Y\. Yang\(2026\)GEMS: agent\-native multimodal generation with memory and skills\.arXiv preprint arXiv:2603\.28088\.Cited by:[Appendix A](https://arxiv.org/html/2607.01709#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2607.01709#S1.p2.1),[§4](https://arxiv.org/html/2607.01709#S4.p5.1)\.
- \[27\]X\. Hu, R\. Wang, Y\. Fang, B\. Fu, P\. Cheng, and G\. Yu\(2024\)Ella: equip diffusion models with llm for enhanced semantic alignment\.arXiv preprint arXiv:2403\.05135\.Cited by:[§4](https://arxiv.org/html/2607.01709#S4.p2.1)\.
- \[28\]Y\. Hu, B\. Liu, J\. Kasai, Y\. Wang, M\. Ostendorf, R\. Krishna, and N\. A\. Smith\(2023\)Tifa: accurate and interpretable text\-to\-image faithfulness evaluation with question answering\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 20406–20417\.Cited by:[§4](https://arxiv.org/html/2607.01709#S4.p6.1)\.
- \[29\]O\. Huang, Y\. Ma, Z\. Zhao, M\. Wu, J\. Ji, R\. Zhang, Z\. Hu, X\. Sun, and R\. Ji\(2025\)Comfygpt: a self\-optimizing multi\-agent system for comprehensive comfyui workflow generation\.arXiv preprint arXiv:2503\.17671\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p1.1),[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[30\]G\. Jiang, Z\. Su, X\. Qu,et al\.\(2026\)XSkill: continual learning from experience and skills in multimodal agents\.arXiv preprint arXiv:2603\.12056\.Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p3.1)\.
- \[31\]K\. Jiang, Y\. Wang, J\. Zhou, P\. Li, Z\. Liu, C\. Xie, Z\. Chen, Y\. Zheng, and W\. Zhang\(2026\)GenAgent: scaling text\-to\-image generation via agentic multimodal reasoning\.arXiv preprint arXiv:2601\.18543\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p2.1),[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[32\]A\. Kamath, K\. Chang, R\. Krishna, L\. Zettlemoyer, Y\. Hu, and M\. Ghazvininejad\(2025\)GenEval 2: addressing benchmark drift in text\-to\-image evaluation\.arXiv preprint arXiv:2512\.16853\.Cited by:[§4](https://arxiv.org/html/2607.01709#S4.p2.1)\.
- \[33\]O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. Haq, A\. Sharma, T\. T\. Joshi, H\. Moazam, H\. Miller,et al\.\(2023\)DSPy: compiling declarative language model calls into state\-of\-the\-art pipelines\.InThe Twelfth International Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p2.1)\.
- \[34\]Y\. Lee, R\. Nair, Q\. Zhang, K\. Lee, O\. Khattab, and C\. Finn\(2026\)Meta\-harness: end\-to\-end optimization of model harnesses\.arXiv preprint arXiv:2603\.28052\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1)\.
- \[35\]Z\. Lin, D\. Pathak, B\. Li, J\. Li, X\. Xia, G\. Neubig, P\. Zhang, and D\. Ramanan\(2024\)Evaluating text\-to\-visual generation with image\-to\-text generation\.InEuropean Conference on Computer Vision,pp\. 366–384\.Cited by:[§4](https://arxiv.org/html/2607.01709#S4.p6.1)\.
- \[36\]D\. Liu, Z\. Li, H\. Du, X\. Wu, S\. Gui, Y\. Kuang, and L\. Sun\(2026\)Graph of skills: dependency\-aware structural retrieval for massive agent skills\.arXiv preprint arXiv:2604\.05333\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p1.1)\.
- \[37\]X\. Lou, M\. Lázaro\-Gredilla, A\. Dedieu, C\. Wendelken, W\. Lehrach, and K\. P\. Murphy\(2026\)AutoHarness: improving llm agents by automatically synthesizing a code harness\.arXiv preprint arXiv:2603\.03329\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1)\.
- \[38\]Y\. J\. Ma, W\. Liang, G\. Wang, D\. Huang, O\. Bastani, D\. Jayaraman, Y\. Zhu, L\. Fan, and A\. Anandkumar\(2023\)Eureka: human\-level reward design via coding large language models\.arXiv preprint arXiv:2310\.12931\.Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p2.1)\.
- \[39\]A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang,et al\.\(2023\)Self\-refine: iterative refinement with self\-feedback\.Advances in neural information processing systems36,pp\. 46534–46594\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1),[§2](https://arxiv.org/html/2607.01709#S2.p2.1)\.
- \[40\]J\. Ni, Y\. Liu, X\. Liu, Y\. Sun, M\. Zhou, P\. Cheng, D\. Wang, X\. Jiang, and G\. Jiang\(2026\)Trace2Skill: distill trajectory\-local lessons into transferable agent skills\.arXiv preprint arXiv:2603\.25158\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1)\.
- \[41\]Nous Research\(2026\)Hermes agent: the self\-improving ai agent\.GitHub\.Note:Accessed: 2026\-04\-29External Links:[Link](https://github.com/NousResearch/hermes-agent)Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p2.1),[§2](https://arxiv.org/html/2607.01709#S2.p3.1)\.
- \[42\]OpenClaw Contributors\(2026\)OpenClaw\-personal ai assistant\.GitHub\.Note:Accessed: 2026\-04\-29External Links:[Link](https://github.com/openclaw/openclaw)Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p2.1),[§2](https://arxiv.org/html/2607.01709#S2.p3.1)\.
- \[43\]L\. Pan, L\. Zou, S\. Guo, J\. Ni, and H\. Zheng\(2026\)Natural\-language agent harnesses\.arXiv preprint arXiv:2603\.25723\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p2.1)\.
- \[44\]S\. Schmidgall, Y\. Su, Z\. Wang, X\. Sun, J\. Wu, X\. Yu, J\. Liu, M\. Moor, Z\. Liu, and E\. Barsoum\(2025\)Agent laboratory: using llm agents as research assistants\.Findings of the Association for Computational Linguistics: EMNLP 2025,pp\. 5977–6043\.Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p3.1)\.
- \[45\]C\. Shi, W\. Li, X\. Liang, Y\. Lu, W\. Yang, R\. Feng, S\. Karten, Z\. Yang, Z\. Ding, G\. Sarch, D\. Chen, K\. Narasimhan, and C\. Jin\(2026\)Odysseus: scaling vlms to 100\+ turn decision\-making in games via reinforcement learning\.External Links:2605\.00347,[Link](https://arxiv.org/abs/2605.00347)Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p3.1)\.
- \[46\]N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. Yao\(2023\)Reflexion: language agents with verbal reinforcement learning\.Advances in neural information processing systems36,pp\. 8634–8652\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1),[§2](https://arxiv.org/html/2607.01709#S2.p2.1)\.
- \[47\]SideFX\(2026\)Houdini\.Note:[https://www\.sidefx\.com](https://www.sidefx.com/)Accessed: 2026\-04\-29Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[48\]D\. Sobania, M\. Briesch, and F\. Rothlauf\(2024\)Comfygi: automatic improvement of image generation workflows\.arXiv preprint arXiv:2411\.14193\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p4.1)\.
- \[49\]J\. Su, Q\. Lan, Z\. Wang, Y\. Xia, H\. Wen, Y\. Duan, X\. Xiao, T\. Shi, Y\. Jingsong, and L\. He\(2026\)ComfySearch: autonomous exploration and reasoning for comfyui workflows\.arXiv preprint arXiv:2601\.04060\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p2.1)\.
- \[50\]S\. G\. Tamilselvam, N\. Panwar, S\. Khare, R\. Aralikatte, A\. Sankaran, and S\. Mani\(2019\)A visual programming paradigm for abstract deep learning model development\.InProceedings of the 10th Indian Conference on Human\-Computer Interaction,pp\. 1–11\.Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[51\]M\. L\. Team, H\. Ma, H\. Tan, J\. Huang, J\. Wu, J\. He, L\. Gao, S\. Xiao, X\. Wei, X\. Ma,et al\.\(2025\)Longcat\-image technical report\.arXiv preprint arXiv:2512\.07584\.Cited by:[§4](https://arxiv.org/html/2607.01709#S4.p3.1)\.
- \[52\]S\. K\. R\. Vuruma, A\. Margetts, J\. Su, F\. Ahmed, and B\. Srivastava\(2024\)From cloud to edge: rethinking generative ai for low\-resource design challenges\.arXiv preprint arXiv:2402\.12702\.Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[53\]G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. Anandkumar\(2023\)Voyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p2.1),[§1](https://arxiv.org/html/2607.01709#S1.p3.1),[§2](https://arxiv.org/html/2607.01709#S2.p2.1)\.
- \[54\]R\. Wu, X\. Wang, J\. Mei, P\. Cai, D\. Fu, C\. Yang, L\. Wen, X\. Yang, Y\. Shen, Y\. Wang,et al\.\(2025\)Evolver: self\-evolving llm agents through an experience\-driven lifecycle\.arXiv preprint arXiv:2510\.16079\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1),[§1](https://arxiv.org/html/2607.01709#S1.p4.1),[§2](https://arxiv.org/html/2607.01709#S2.p3.1)\.
- \[55\]X\. Wu, Z\. Li, G\. Shi, A\. Duffy, T\. Marques, M\. L\. Olson, T\. Zhou, and D\. Manocha\(2026\)Co\-evolving llm decision and skill bank agents for long\-horizon tasks\.External Links:2604\.20987,[Link](https://arxiv.org/abs/2604.20987)Cited by:[§2](https://arxiv.org/html/2607.01709#S2.p2.1)\.
- \[56\]Z\. Wu, C\. Han, Z\. Ding, Z\. Weng, Z\. Liu, S\. Yao, T\. Yu, and L\. Kong\(2024\)Os\-copilot: towards generalist computer agents with self\-improvement\.arXiv preprint arXiv:2402\.07456\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p2.1),[§2](https://arxiv.org/html/2607.01709#S2.p2.1)\.
- \[57\]P\. Xia, J\. Chen, H\. Wang, J\. Liu, K\. Zeng, Y\. Wang, S\. Han, Y\. Zhou, X\. Zhao, H\. Chen,et al\.\(2026\)SkillRL: evolving agents via recursive skill\-augmented reinforcement learning\.arXiv preprint arXiv:2602\.08234\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1),[§2](https://arxiv.org/html/2607.01709#S2.p2.1)\.
- \[58\]T\. Xie, D\. Zhang, J\. Chen, X\. Li, S\. Zhao, R\. Cao, T\. J\. Hua, Z\. Cheng, D\. Shin, F\. Lei,et al\.\(2024\)Osworld: benchmarking multimodal agents for open\-ended tasks in real computer environments\.Advances in Neural Information Processing Systems37,pp\. 52040–52094\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p2.1)\.
- \[59\]R\. Xu and Y\. Yan\(2026\)Agent skills for large language models: architecture, acquisition, security, and the path forward\.arXiv preprint arXiv:2602\.12430\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1)\.
- \[60\]Z\. Xu, Y\. Wang, X\. Yang, L\. Wang, W\. Luo, K\. Zhang, B\. Hu, and M\. Zhang\(2025\)Comfyui\-r1: exploring reasoning models for workflow generation\.arXiv preprint arXiv:2506\.09790\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p1.1),[§1](https://arxiv.org/html/2607.01709#S1.p2.1),[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[61\]Z\. Xu, X\. Wang, S\. Li, T\. Yu, L\. Wang, Q\. Fu, and W\. Yang\(2025\)Agents play thousands of 3d video games\.arXiv preprint arXiv:2503\.13356\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p2.1)\.
- \[62\]X\. Xue, Z\. Lu, D\. Huang, Z\. Wang, W\. Ouyang, and L\. Bai\(2025\)Comfybench: benchmarking llm\-based agents in comfyui for autonomously designing collaborative ai systems\.InProceedings of the computer vision and pattern recognition conference,pp\. 24614–24624\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p2.1),[§2](https://arxiv.org/html/2607.01709#S2.p1.1)\.
- \[63\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§4](https://arxiv.org/html/2607.01709#S4.p3.1)\.
- \[64\]J\. Young\(2025\-11\-26\)Effective harnesses for long\-running agents\.Note:AnthropicCited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1)\.
- \[65\]Y\. Zhai, S\. Tao, C\. Chen, A\. Zou, Z\. Chen, Q\. Fu, S\. Mai, L\. Yu, J\. Deng, Z\. Cao,et al\.\(2025\)Agentevolver: towards efficient self\-evolving agent system\.arXiv preprint arXiv:2511\.10395\.Cited by:[§1](https://arxiv.org/html/2607.01709#S1.p3.1),[§1](https://arxiv.org/html/2607.01709#S1.p4.1)\.

## Appendix APredefined Tools, Skills, and LoRA Settings

#### Workflow tools\.

We expose 17 basic tools for controlling ComfyUI workflows\. These tools allow the agent to inspect the current workflow, add and remove nodes, connect nodes, edit node inputs, set prompts, configure model\-specific parameters, validate the workflow, and submit the final graph for rendering\. Examples includeadd\_node,set\_prompt, andset\_lora\. Together, these tools allow the agent to construct and repair workflows at the graph level rather than only rewriting the text prompt\.

#### Predefined skills\.

In addition to evolved skills, we provide four predefined skills inspired byHeet al\.\[[26](https://arxiv.org/html/2607.01709#bib.bib138)\]:photorealistic,creative,high\-quality, andprompt\-artist\. These skills provide general image\-generation guidance, such as improving photorealism, increasing visual creativity, enhancing overall quality, and rewriting prompts for stronger visual composition\. They serve as the initial reusable skill library before benchmark\-specific skills are evolved\.

#### Model\-specific LoRA settings\.

ForZ\-Image\-Turbo, we include two LoRAs\. The first is a realistic snapshot LoRA\[[10](https://arxiv.org/html/2607.01709#bib.bib136),[14](https://arxiv.org/html/2607.01709#bib.bib152)\], which encourages more photorealistic and real\-life imagery\. The second is an enhancer LoRA\[[13](https://arxiv.org/html/2607.01709#bib.bib153)\], which improves generation quality across a range of visual styles\.LongCat\-Imagedoes not currently support LoRAs in our setup, so LoRA\-based tools are disabled for that model\.

GroupToolPurposeInspectioninspect\_workflowSummarise nodes, IDs, classes, inputs of the current graph\.query\_available\_
modelsList installed checkpoints / LoRAs / UNETs / VAEs / upscalers\.explore\_nodesQuery/object\_infoand classify nodes by pipeline stage\.Graph editsadd\_nodeAppend a node; return its new ID\.connect\_nodesWire*src*\.slot→\\to*dst*\.input\.delete\_nodeRemove a node and its incident links\.set\_paramSet a scalar input on a specific node\.Composite macrosset\_promptSet positive/negative text on every encoder feeding a sampler\.add\_lora\_loaderInsert a LoRA between model source and downstream consumers\.add\_regional\_
attentionSplit conditioning into foreground/background regional prompts\.add\_hires\_fixAdd latent upscale \+ secondKSampler\+VAEDecode\.add\_inpaint\_passAdd a targeted inpaint pass for a specific region\.Skillsread\_skillLoad aSKILL\.mdbody by name \(progressive disclosure\)\.Control / metareport\_evolution\_
strategyDeclare the iteration plan and top issue before edits\.validate\_workflowCheck the graph for dangling refs, wrong slots, missing outputs\.finalize\_workflowSignal completion \(auto\-validates; blocks if errors remain\)\.transition\_stageAdvance through planning→⋯→\\to\\cdots\\tofinalization\.Table 4:The 17 predefined tools the agent uses to construct and repair ComfyUI workflows, grouped by function\. Composite macros bundle a sequence of base graph edits with the correct rewiring and slot binding\.Table 5:The four general\-purpose skills loaded into every run before any benchmark\-specific evolution\. Each is aSKILL\.mdpackage with YAML frontmatter exposing only the description at startup; the full body is loaded on demand viaread\_skill\.

## Appendix BSkill Read Statistics and Workflow Modification Breakdown

Tables[6](https://arxiv.org/html/2607.01709#A2.T6)and[7](https://arxiv.org/html/2607.01709#A2.T7)characterize the agent’s workflow\-construction behavior from two complementary angles\. Table[6](https://arxiv.org/html/2607.01709#A2.T6)measures how much the agent relies on evolved skills versus the four predefined base skills, broken down by benchmark\. Table[7](https://arxiv.org/html/2607.01709#A2.T7)then dissects what kinds of graph edits the agent actually performs when it constructs or repairs a workflow\.

Table 6:Per\-benchmark evolved skill read statistics aggregated overLongCatandZ\-Image\-Turboruns with Claude\-Sonnet\-4\.5\.*Evolved Skills*counts the number of distinct skills synthesized by the skill\-evolution loop;*Total Reads*is the aggregate number ofread\_skillcalls across all prompts;*Base Reads*and*Evolved Reads*partition those calls by skill origin\. Benchmarks with richer compositional demands \(DPG\-Bench, GenEval 2\) draw far more heavily on evolved skills, whereas the OneIG splits—which emphasize single\-character stylization—are served adequately by the base library\.Table[7](https://arxiv.org/html/2607.01709#A2.T7)further disaggregates the agent’s edits into six semantic event categories\. Prompt\-text changes and sampler/guidance adjustments together account for∼75%\{\\sim\}75\\%of all events across benchmarks, confirming that the agent primarily operates at the level of conditioning rather than graph topology\. Structural modifications—regional masking, model swaps, and multi\-pass upscaling—are more frequent on GenEval 2, which contains the most demanding multi\-object layout tasks\.

Workflow EventDPG\-BenchGenEval2OneIG\-ENOneIG\-ZHTotal%Prompt\-text changes2,8624,1393,2773,71413,99239\.3%Sampler / guidance hyper\-params2,7762,4153,4084,16112,76035\.8%Regional / mask graph topology2142,9584844004,05611\.4%Model / weight changes \(LoRA, ckpt\)5705187218062,6157\.3%Multi\-pass / upscale topology1952583283581,1393\.2%Other \(class swaps, removes, misc\)1145991701671,0502\.9%Total6,73110,8878,3889,60635,612100\.0%

Table 7:Action\-event breakdown for workflow construction inComfyClaw\. Each*event*denotes a node\-level workflow change: adding a node, removing a node, or editing a node’sinputsfield\. Counts are aggregated over the two image\-generation models,LongCatandZ\-Image\-Turbo, for each benchmark\.
## Appendix CEvolved Skill Usage

Table[8](https://arxiv.org/html/2607.01709#A3.T8)provides a usage\-level view of the skills produced by the Claude\-Sonnet\-4\.5\. Rather than treating these counts as direct measures of skill quality, we use them to show which reusable recipes the agent actually consults when solving each benchmark\. The resulting distribution separates benchmark\-specific needs: GenEval 2 emphasizes spatial binding and counting, DPG\-Bench emphasizes material, lighting, and scene detail, and the OneIG splits emphasize character\-count and anime\-style control\.

\#OneIG\-ENOneIG\-ZHSkillReadsSkillReads1character\-counting166anime\-single\-character\-simple442anime\-danbooru\-ordering126anime\-multi\-character333anime\-character\-counting124detailed\-character\-design334anime\-style\-declaration95anime\-solo\-simple215presentation\-slide\-text61anime\-character\-state\-verification196presentation\-chart\-internal\-labels20chinese\-group\-counting187presentation\-diagram\-structure17anime\-direct\-tag\-format168rate\-limit\-mitigation15resolution\-quality\-tags159anime\-style\-tag—anime\-square\-resolution—10character\-state\-control—chinese\-stylized\-prompt—

Table 8:Top\-10 evolved skills per benchmark for Claude\-Sonnet\-4\.5\. Counts are totalread\_skillcalls aggregated over all prompts and both image backbones \(Z\-Image\-Turbo and LongCat\-Image\)\. We exclude two run\-level aggregator skills so the table reflects topical recipes rather than bookkeeping\.We omit the run\-level aggregator skillslearned\-successesandlearned\-errors, which summarize each cycle’s success and failure clusters rather than encoding topical recipes\. Their raw read counts are high by construction: 3135/649/1096/226 and 379/—/53/49 for GenEval 2, DPG\-Bench, OneIG\-EN, and OneIG\-ZH, respectively\.

Table 9:Model\-specific LoRA settings used in our experiments\. Z\-Image\-Turbo is an S3\-DiT architecture, so all LoRAs are injected viaLoraLoaderModelOnly\(model\-only, no CLIP weights\)\. LongCat\-Image exposes no model tensor compatible with our LoRA tools, so they are disabled for that backbone\. Strengths follow the recipe in thez\-image\-turboskill\.
## Appendix DQualitative Annotations

For the user study, each participant was given the instruction template in Box[D](https://arxiv.org/html/2607.01709#A4)and asked to annotate generated images\. The images were randomly shuffled so that annotators were blind to the corresponding method groups\. Annotators were paid$​0\.10\\mathdollar 0\.10per image, for a total of 2,400 annotations and$​240\\mathdollar 240in compensation\. The annotation task involved no anticipated risk to participants, and our protocol was reviewed and exempted by the Institutional Review Board \(IRB\)\.

Evaluation InstructionsRate the overall result on a1–5 scalethat combines three criteria:\(a\)Prompt–image alignment— Does the image faithfully depict the objects, attributes, count, relationships, and style specified in the prompt?\(b\)Visual aesthetic— Composition, lighting, colour, and balance\.\(c\)Image quality— Sharpness, absence of artifacts, anatomical and structural correctness, no distorted text/hands/faces\.

## Appendix ELLM Prompts

This section lists the exact prompts used in the implementations of the agent loop \(§[3\.2](https://arxiv.org/html/2607.01709#S3.SS2)\) and the region\-level VLM verifier \(§[3\.3](https://arxiv.org/html/2607.01709#S3.SS3)\)\. All prompts are drawn verbatim fromcomfyclaw/agent\.py,comfyclaw/verifier\.py, andcomfyclaw/harness\.py\.

### E\.1Agent System Prompt \(§[3\.2](https://arxiv.org/html/2607.01709#S3.SS2)\)

The following string \(\_SYSTEM\_PROMPT\_BASE\) is prepended to every agent conversation as thesystemrole message\. When a model is pinned thePinned image modelparagraph is appended\. The<available\_skills\>XML block \(skill names and one\-line descriptions\) is injected between the base prompt and the pinned\-model paragraph\.

YouareComfyClaw,anexpertComfyUIworkflowengineer\.YourjobistoBUILD

andGROWComfyUIworkflowtopologies\-\-constructingcompletepipelinesfrom

scratchwhentheworkflowisempty,andevolvingexistingonesinresponseto

theverifier’sregion\-levelfeedback\.

Iterationstrategy

\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-

1\.Callreport\_evolution\_strategyfirst:stateyourplanandthetopissue\.

2\.Callinspect\_workflowtoseethecurrenttopology\.

3\.\*\*Iftheworkflowisempty\*\*\(nonodes\):

a\.Callread\_skill\("workflow\-builder"\)toloadarchitecturerecipes\.

b\.Callquery\_available\_models\("checkpoints"\)andquery\_available\_models\("diffusion\_models"\)

todiscoveravailablemodels\-\-NEVERguessfilenames\.

c\.Matchthemodelfilenametoanarchitecture\(SD1\.5,SDXL,Flux,Qwen,etc\.\)

usingthepatternsintheworkflow\-builderskill\.

d\.Buildthefullpipelinenode\-by\-nodeusingadd\_node,followingthematchingrecipe\.

e\.UseONLYexactfilenamesfromqueryresults\.

f\.SetdetailedpromptsontheCLIPTextEncodenodes\.

g\.Callvalidate\_workflowtocatchwiringerrorsbeforesubmitting\.

h\.Callfinalize\_workflow\(itauto\-validatesandblocksiferrorsremain\)\.

4\.\*\*Iftheworkflowalreadyhasnodes\*\*,followtheevolutionstrategy:

a\.Callset\_prompt\-\-craftadetailed,professionalpositivepromptANDastrong

negativepromptbasedontheuser’sgoal\(see"Promptengineering"below\)\.

DothisEVERYiteration,evenifyoualsoplanstructuralchanges\.

b\.Ifarelevantskillislistedin<available\_skills\>,callread\_skilltoload

itsfullinstructionsBEFOREapplyingthatupgrade\.

c\.Callquery\_available\_modelsBEFOREaddinganyLoRAnode\.

d\.Applystructuralupgrades\(LoRA/regional/hires/inpaint\)\.

e\.Tunesamplerparameters\(steps,CFG,seed\)asneeded\.

f\.Callvalidate\_workflowtocatchwiringerrors\.

g\.Callfinalize\_workflowwhendone\(itauto\-validates\)\.

Promptengineering\(step3\)

\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-

Theworkflow’spositivepromptispre\-seededwiththeuser’srawgoaltext\.

YouMUSTreplaceitwithaprofessional\-qualityprompteveryiteration\.

Positiveprompt\-\-structure:

\[subject&scene\],\[style\],\[lighting\],\[camera/lens\],\[qualityboosters\]

\*Expandeverymeaningfulconcept:vaguenouns\-\>vividadjectives\+nouns\.

\*Addartistic/photographicstyle:"cinematic","conceptart","photorealistic",

"watercolorpainting","isometric",etc\.

\*Addlighting:"goldenhour","dramaticrimlighting","neonglow","softdiffuse"\.

\*Addqualityboosters:"8k","ultradetailed","sharpfocus","raytracing","awardwinning"\.

\*Iftheimagehasmultiplesubjects/regions,describeeachclearly\.

Negativeprompt\-\-alwaysincludethesebaselineentries,thenaddscene\-specificones:

"blurry,outoffocus,lowquality,lowresolution,noisy,grainy,jpegartifacts,

watermark,text,signature,ugly,badanatomy,deformed,disfigured,

poorlydrawnhands,extrafingers,mutatedlimbs,clonedface,plasticskin"

Example\(input:"acyberpunkcityatnight"\):

positive:"afuturisticcyberpunkcityskylineatnight,toweringneon\-litskyscrapers,

wetreflectivestreets,holographicadvertisements,denserain,cinematiccomposition,

dramaticvolumetriclighting,wideanglelens24mm,8k,photorealistic,ultradetailed,

sharpfocus,raytracing,bladerunneraesthetic"

negative:"blurry,lowquality,noisy,watermark,text,badanatomy,deformed,ugly,

cartoon,anime,daytime,sunny,emptystreet"

Structuralupgradepriority\(iteration2\+\)

\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-

WhentheworkflowalreadyhasnodesANDverifierfeedbackispresent:

\*DoNOTjustrefinetheprompt\-\-prompt\-onlychangesplateauquickly\.

\*PREFERstructuralupgrades:LoRA,hires\-fix,regional,inpaint\.

\*IfANYregion\_issuehasfix\_strategiescontaining"inject\_lora\_\*",

youMUSTattemptthatstructuralupgrade,notfallbacktoprompttweaking\.

\*Combine:alwaysrefinethepromptANDaddastructuralupgradetogether\.

\*Onlyfallbacktoprompt\-onlywhennoLoRA/inpaintmodelsareinstalled

orthefixstrategiesareexclusivelyprompt\-related\.

Human\-in\-the\-loopfeedback

\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-

Whentheverifierfeedbacksectionstartswith"\#\#HumanReviewerFeedback",

thefeedbackcomesfromahumanreviewer,notanautomatedVLM\.

Humanfeedbackexpressessubjectivepreferences\-\-style,mood,composition,

colorpalette,artisticdirection\.Prioritizetheseoverstructural/technical

changes\.Itemsprefixedwith\[HUMAN\]aredirecthumanrequests\-\-address

eachonespecifically\.Donotsecond\-guessoroverridehumanpreferences\.

\[Pinnedimagemodelparagraph\-\-appendedwhenamodelislocked\]

Theimage\-generationmodelforthissessionisLOCKEDto:\{model\_name\}

\*DONOTchangetheckpt\_name/unet\_nameofanyloadernode\.

\*YouMAYaddLoRAloadersontopofthepinnedmodel\.

\*Iftheserverhasnomodelsavailable\(offline/dry\-run\),skipanyaction

thatrequiresmodeldiscoveryandfocusonprompt/samplertuning\.

### E\.2Agent User Message Template \(§[3\.2](https://arxiv.org/html/2607.01709#S3.SS2)\)

Each agent invocation receives a dynamically assembledusermessage \(built by\_build\_user\_messageinagent\.py\)\. The template below shows all sections that may appear; sections are joined with double newlines and omitted when their data are absent\.

\#\#ImageGoal\(user’soriginalrequest\)

\{original\_prompt\}

\#\#Iteration

\{iteration\}

\#\#CurrentPositivePrompt\(baseline\-\-needsrefinement\)

\{current\_positive\_prompt\_from\_workflow\}

Use‘set\_prompt‘toreplacethiswithadetailed,high\-qualityversion\.

\#\#ActiveModel

‘\{checkpoint\_or\_unet\_filename\}‘\(\{architecturedescriptionfromarch\.yaml\}\)

\#\#Pre\-loadedSkill:\{model\_skill\_name\}\[whenamodel\-specificskillexists\]

\{fullskillbody\-\-agentdoesNOTneedtocallread\_skillforthis\}

\#\#LearnedSkills\(frompastexperienceonthismodel\)\[whenevolvedskillsexist\]

\{descriptionofevolvedskill\}\-\>read\_skill\("\{evolved\_skill\_name\}"\)

\#\#SuggestedSkills\[whenkeyword\-matchedbuilt\-inskillsexist\]

Theseskillsmayberelevant:\{comma\-separatedskillnames\}

Callread\_skill\(<name\>\)toloadfullinstructionsbeforeapplying\.

\#\#VerifierFeedback\(previousiteration\)\[fromiteration2onward\]

\{VerifierResult\.format\_feedback\(\)output\}

REQUIREDstructuralupgrades\(fromverifierfix\_strategies\):

\*‘inject\_lora\_detail‘\-\>read\_skill\("lora\-enhancement"\)thenadd\_lora\_loader

\*‘add\_hires\_fix‘\-\>read\_skill\("hires\-fix"\)thenadd\_hires\_fix

\.\.\.

\#\#HumanReviewerFeedback\(previousiteration\)\[hybrid/humanverifiermode\]

\{humanfeedbacktextwith\[HUMAN\]prefixeditems\}

Prioritizetheirsubjectivepreferences\(style,mood,composition,color\)\.

\#\#Memory/PastAttempts

Attempt\{n\}\(score=\{s:\.2f\}\):

Passed:\{passedrequirements\}

Failed:\{failedrequirements\}

Experience:\{one\-linelessonfromthatattempt\}

\#\#CRITICAL\-\-EmptyWorkflow\-\-YouMUSTBuildFromScratch\[whenworkflowisempty\]

TheworkflowisCOMPLETELYEMPTY\.YouCANNOTfinalizewithoutaddingnodes\.

Step1:Call‘read\_skill\("workflow\-builder"\)‘\.\.\.

Step2:Call‘query\_available\_models\(’checkpoints’\)‘\.\.\.

\.\.\.

Beginwithreport\_evolution\_strategy,theninspect\_workflow,

applyyourchanges,thenfinalize\_workflow\.

### E\.3Workflow Repair Prompt \(§[3\.2](https://arxiv.org/html/2607.01709#S3.SS2)\)

When ComfyUI rejects a workflow submission \(HTTP 4xx, validation failure, or execution error\), the harness re\-invokes the agent with the following feedback string \(\_build\_repair\_feedbackinharness\.py\) as theverifier\_feedbackargument\.

\#\#ComfyUIRejectedtheWorkflow\-\-RepairRequired

Yourlastworkflowsubmissionwasrejectedwiththefollowingerror:

‘‘‘

\{verbatimComfyUIerrorstring\}

‘‘‘

\*\*Repairprotocol\(followinorder\):\*\*

1\.Call‘inspect\_workflow‘toseetheFULLcurrenttopologyandallconnections\.

2\.Call‘validate\_workflow‘togetalistofgrapherrors\(danglingrefs,wrongslots\)\.

3\.Foreacherror:

\-Ifanodereferencesanonexistentsource\-\>fixwith‘connect\_nodes‘or‘delete\_node‘

\-Ifaslotindexiswrong\-\>‘delete\_node‘thebrokennodeand‘add\_node‘anewone

withcorrectwiring

\-Ifamodel/filenameiswrong\-\>use‘query\_available\_models‘togetexactnames,

then‘set\_param‘

\-Ifanodeclassdoesn’texist\-\>‘delete\_node‘itanduseadifferentclass\_type

4\.Call‘validate\_workflow‘againtoconfirmallissuesareresolved\.

5\.Call‘finalize\_workflow‘\(itwillauto\-validateandblockifstillbroken\)\.

\*\*IMPORTANT:\*\*DoNOTjustaddnewnodesontopofbrokenones\-\-‘delete\_node‘the

brokennodefirst,then‘add\_node‘areplacementwithcorrectconnections\.

\*\*Outputslotreference:\*\*

CheckpointLoaderSimple\-\>slot0:MODEL,slot1:CLIP,slot2:VAE

UNETLoader/CLIPLoader/VAELoader\-\>slot0only

KSampler\-\>slot0:LATENT

VAEDecode\-\>slot0:IMAGE

CLIPTextEncode\-\>slot0:CONDITIONING

\[WhenapreviousVerifierResultisavailable:\]

\-\-PreviousVerifierFeedback\(forcontext\)\-\-

\{VerifierResult\.format\_feedback\(\)output\}

### E\.4Verifier Prompts \(§[3\.3](https://arxiv.org/html/2607.01709#S3.SS3)\)

The verifier \(comfyclaw/verifier\.py\) uses three distinct prompts corresponding to the two\-pass pipeline described in §[3\.3](https://arxiv.org/html/2607.01709#S3.SS3)\.

#### Pass 1 – Requirement decomposition \(\_DECOMPOSE\_PROMPT\)\.

A text\-only call \(no image\) that breaks the user prompt into a list of yes/no questions\. In the default batched mode \(batch\_mode=True\) this step runs once per unique prompt and is cached for all subsequent iterations, so each unique prompt pays the decomposition cost only once\.

Analyzethefollowingimagegenerationpromptandbreakitdownintospecific,

observablevisualrequirements\.Foreach,writeayes/noquestionanswerablefromtheimage\.

RespondONLYwithaJSONarrayofquestionstrings\.

Prompt:\{prompt\}

#### Pass 2 – Batched unified verification \(\_UNIFIED\_VERIFY\_PROMPT\)\.

The default path\. A single vision call answers all decomposed yes/no questions and simultaneously produces region\-level issues, evolution suggestions, and a holistic 1–10 score\. The image is uploaded exactly once per verify call\.

YouareanexpertimagequalityanalystandComfyUIworkflowengineer\.

Youaregivenonegeneratedimageandtheintendedprompt\.DoBOTHtasksina

SINGLEpassandreturnONEJSONobject\.Donotemitanyprose,markdown,or

codefences\-\-justtheJSON\.

TASK1\-\-Requirementchecks:Answereachyes/noquestionbelowbasedonthe

image\.Usestrict"yes"/"no"answers\(lower\-case\)\.EveryquestionMUSTbe

answered;neverskip\.

Questions\(answerall\):

\{questions\_block\}

TASK2\-\-Holisticanalysis:Produceashortoverallassessment,aninteger

score\(1\-10\),alistofregion\-levelissues,andalistofconcreteworkflow

evolutionsuggestions\.

Scorerubric\(integer1\-10\):

1\-2:Completelywrong\-\-unrecognizable,norelationtoprompt

3\-4:Majorfailures\-\-wrongsubject,severeartifacts,missingkeyelements

5\-6:Partialmatch\-\-rightsubjectbutsignificantqualityoraccuracyissues

7\-8:Good\-\-matchespromptwellwithminorissues\(slightartifacts,softdetails\)

9\-10:Excellent\-\-faithfultoprompt,highquality,minimalornoissues

Fixstrategyvocabulary\(usetheseexactstrings\):

inject\_lora\_detail\|inject\_lora\_style\|inject\_lora\_anatomy\|inject\_lora\_lighting

add\_regional\_prompt\|add\_hires\_fix\|add\_inpaint\_pass\|add\_ip\_adapter

refine\_positive\_prompt\|refine\_negative\_prompt\|increase\_steps\|adjust\_cfg\|adjust\_sampler

ReturnexactlythisJSONschema:

\{

"requirements":\[

\{"question":"<verbatimquestiontext\>","answer":"yes"or"no"\}

\],

"overall\_assessment":"<1\-2sentenceoverallqualitysummary\>",

"score":<integer1\-10\>,

"region\_issues":\[

\{

"region":"<foregroundsubject\|background\|face\|hands\|sky\|\.\.\.\>",

"issue\_type":"<anatomy\|texture\|lighting\|artifact\|composition\|detail\|color\|proportion\>",

"description":"<specificproblemdescription\>",

"severity":"<low\|medium\|high\>",

"fix\_strategies":\["<workflowaction1\>","<workflowaction2\>"\]

\}

\],

"evolution\_suggestions":\[

"<concreteworkflowchange1\>",

"<concreteworkflowchange2\>"

\]

\}

CRITICAL:requirementsMUSTcontainexactly\{n\_questions\}entries,inthesame

orderasthequestionsabove\.

Intendedprompt:\{prompt\}

#### Pass 2 – Legacy detailed analysis \(\_DETAILED\_ANALYSIS\_PROMPT\)\.

Used as a fallback when the unified call fails to parse, or whenbatch\_mode=False\. This call receives the image and returns the holistic analysis only; requirement checks are handled by separate per\-question calls in that mode\.

YouareanexpertimagequalityanalystandComfyUIworkflowengineer\.

Analyzethisgeneratedimageagainsttheintendedprompt,thenreturnaJSONobjectwith:

\{

"overall\_assessment":"<1\-2sentenceoverallqualitysummary\>",

"score":<integer1\-10\>,

"region\_issues":\[

\{

"region":"<specificarea:foregroundsubject\|background\|face\|hands\|sky\|\.\.\.\>",

"issue\_type":"<anatomy\|texture\|lighting\|artifact\|composition\|detail\|color\|proportion\>",

"description":"<specificproblemdescription\>",

"severity":"<low\|medium\|high\>",

"fix\_strategies":\["<workflowaction1\>","<workflowaction2\>"\]

\}

\],

"evolution\_suggestions":\[

"<concreteworkflowchange1:whattoadd/modifyandwhy\>",

"<concreteworkflowchange2\>"

\]

\}

Scorerubric\(integer1\-10\):

1\-2:Completelywrong\-\-unrecognizable,norelationtoprompt

3\-4:Majorfailures\-\-wrongsubject,severeartifacts,missingkeyelements

5\-6:Partialmatch\-\-rightsubjectbutsignificantqualityoraccuracyissues

7\-8:Good\-\-matchespromptwellwithminorissues\(slightartifacts,softdetails\)

9\-10:Excellent\-\-faithfultoprompt,highquality,minimalornoissues

Fixstrategyvocabulary\(usetheseexactstrings\):

inject\_lora\_detail\|inject\_lora\_style\|inject\_lora\_anatomy\|inject\_lora\_lighting

add\_regional\_prompt\|add\_hires\_fix\|add\_inpaint\_pass\|add\_ip\_adapter

refine\_positive\_prompt\|refine\_negative\_prompt\|increase\_steps\|adjust\_cfg\|adjust\_sampler

Intendedprompt:\{prompt\}