EvoGenUI-Bench: Evaluating LLMs as Multi-Turn Generative UI Assistants
Summary
EvoGenUI-Bench introduces a benchmark for evaluating LLMs as multi-turn generative UI assistants, focusing on interface maintenance across 150 tasks with challenges in state propagation and external grounding.
View Cached Full Text
Cached at: 09/01/26, 11:41 AM
Paper page - EvoGenUI-Bench: Evaluating LLMs as Multi-Turn Generative UI Assistants
Source: https://huggingface.co/papers/2608.29387
Abstract
Largelanguagemodelscangenerateinteractivewebinterfaces,butreliablegenerativeUIrequiresmaintaininganexecutableartifactasuserrequestsevolve.WeintroduceEvoGenUI-Bench,abenchmarkformulti-turninterfacemaintenancecomprising150five-turntasksand750turnsacrossthreescenarios:informationpresentation,executableinteraction,andtool-groundedexternalstate.Weexecutegeneratedartifactsinabrowserandevaluatethemusingscreenshots,sourceandDOMevidence,actortraces,andruntimelogs.Beyondturn-levelandepisode-levelsuccess,wemeasurecross-turnretentionwithAdjacentPassRetention.Acrosseightmodels,eventhestrongestachieves74.9%TurnPasswhilecompletingonly37.3%offive-turnepisodes;APRfurtherfallsto52.4%ontool-groundedtasks.Diagnosticanalysisshowsthatpresentationfailurescenteroninformationarchitecture,interactionfailuresonderived-statepropagationandaffordancebinding,andtool-groundedfailuresadditionallyinvolveexternal-stategroundingandrequirementdecomposition.TheseresultsreframegenerativeUIevaluationfromjudgingisolatedoutputstotestingwhetherinterfacebehavior,derivedstate,externalstate,andassistantclaimsremainsynchronizedastheartifactevolves.
View arXiv pageView PDFGitHub0Add to collection
Get this paper in your agent:
hf papers read 2608\.29387
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.29387 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.29387 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.29387 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Reasoning for Mobile User Experience with Multimodal LLMs: Task, Benchmark, and Approach
The paper introduces UXBench, a multimodal benchmark for evaluating MLLMs on mobile UX reasoning tasks, and presents UI-UX, a fine-tuned MLLM based on Qwen3-VL-4B-Thinking that achieves state-of-the-art performance on this benchmark.
OmniAssistBench: Assistant-style Interaction Benchmark for Omni-LLMs
OmniAssistBench is a benchmark for evaluating omni-modal large language models as real-time video assistants, revealing that current models struggle with visual prompts, context retention, and timely responses.
DashArena: Benchmarking LLMs on Interactive Analytic Dashboard Generation
Introduces DashArena, the first benchmark for open-ended, task-grounded generation of interactive analytic dashboards by LLMs. It uses a browser executor to replay interaction trajectories and a VLM judge to evaluate results, with human studies confirming its effectiveness.
Looks Right, Works Right: A Project-Level Benchmark for Multi-Screen Mobile App Generation
MobileForge is a benchmark for project-level multi-screen mobile app generation, evaluating multimodal LLMs on build success, cross-page navigation, visual fidelity, maintainability, and efficiency. Experiments on six frontier multimodal LLMs show current models can compile and reach pages but still struggle with interactive navigation and visual quality.
UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation
UI2App presents a benchmark for evaluating vision-language models on generating executable web applications from UI screenshots, focusing on interaction inference beyond visual fidelity, and finds significant gaps in current models' ability to infer complete interaction behavior.