Paper page - Chat2Workflow: A Benchmark for Generating Executable Visual Workflows with Natural Language

Source: https://huggingface.co/papers/2604.19667

Abstract

Chat2Workflow presents a benchmark and agentic framework for automating executable visual workflow generation from natural language, revealing significant challenges in achieving industrial-grade automation despite advances in language models.

At present,executable visual workflowshave emerged as a mainstream paradigm in real-worldindustrial deployments, offering strong reliability and controllability. However, in current practice, such workflows are almost entirely constructed through manual engineering: developers must carefully design workflows, write prompts for each step, and repeatedly revise the logic as requirements evolve-making development costly, time-consuming, and error-prone. To study whether large language models can automate this multi-round interaction process, we introduceChat2Workflow, a benchmark for generatingexecutable visual workflowsdirectly from natural language, and propose a robustagentic frameworkto mitigate recurrent execution errors.Chat2Workflowis built from a large collection of real-world business workflows, with each instance designed so that the generated workflow can be transformed and directly deployed to practical workflow platforms such as Dify and Coze. Experimental results show that while state-of-the-art language models can often capture high-level intent, they struggle to generate correct, stable, and executable workflows, especially under complex or changing requirements. Although ouragentic frameworkyields up to 5.34% resolve rate gains, the remaining real-world gap positionsChat2Workflowas a foundation for advancing industrial-grade automation. Code is available at https://github.com/zjunlp/Chat2Workflow.

View arXiv pageView PDFGitHub7Add to collection

Get this paper in your agent:

hf papers read 2604\.19667

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2604.19667 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2604.19667 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2604.19667 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.