Text2Dashboard: A Governed Agent Architecture for Natural-Language Dashboard Generation over Enterprise DataBrain

# Text2Dashboard: A Governed Agent Architecture for Natural-Language Dashboard Generation over Enterprise DataBrain
Source: [https://arxiv.org/html/2610.06914](https://arxiv.org/html/2610.06914)
Yiou WuDepartment of Computer Science University College London London, United Kingdom [yiou\.wu\.24@ucl\.ac\.uk](mailto:[email protected])Zezhi TangDepartment of Computer Science University College London London, United Kingdom [zezhi\.tang@ucl\.ac\.uk](mailto:[email protected])Ningwei BaiDepartment of Computer Science University College London London, United Kingdom [ningwei\.bai\.23@ucl\.ac\.uk](mailto:[email protected])Liuhaichen YangDepartment of Computer Science University College London London, United Kingdom [liuhaichen\.yang\.23@ucl\.ac\.uk](mailto:[email protected])††thanks:Zezhi Tang, Ningwei Bai and Liuhaichen Yang are co\-corresponding authors\.

###### Abstract

Text2Dashboard is a DataBrain\-specific prototype that turns natural\-language analytic requests into inspectable dashboards\. An installable Codex plugin and standalone Agent Runtime combine schema\-constrained model decisions with typed tools, persistent state, and deterministic Hooks for approval, audit, checkpointing, recovery, and failure handling\. The pipeline resolves entities, discovers metadata, enforces read\-only SQL, composes dashboards, and applies static checks, dynamic preflight, and browser inspection\. The model proposes actions while deterministic software controls execution and records state transitions\.

We evaluate the workflow on frozen real\-DataBrain tasks and controlled Hook faults\. Strict success was 6/8 on metadata and SQL tasks: metadata selection passed 4/4, all four SQL tasks met semantic criteria, and 2/4 met the exact output\-column contract\. The final release passed 4/4 single\-panel dashboard tasks, one two\-panel task, and one existing\-dashboard refinement; a parameterised task exceeded its step limit\. All ten fault scenarios met their specified outcomes without unapproved external side effects\. Model inference accounted for over 97% of observed runtime in every reported group\. These small, DataBrain\-specific results do not establish production readiness, general text\-to\-SQL accuracy, or an efficiency advantage over manual dashboard construction\.

###### Index Terms:

large language model agents, natural\-language interfaces, text\-to\-SQL, dashboards, human oversight, DataBrain

## IIntroduction

### I\-AProblem and motivation

A dashboard request usually starts as a business question rather than a query\. Producing a useful dashboard from that question requires several linked tasks: identifying the relevant entity and data table, inspecting the schema, writing SQL, choosing a visual encoding, defining filters, and checking the rendered result\. These tasks are difficult for users who understand the decision they want to make but do not know the organisation’s database structure\.

Natural\-language generation can reduce this authoring burden, but fluent output is not sufficient in an enterprise setting\. A plausible query may still use a non\-existent field, select the wrong data domain, or request an operation that the user did not approve\. A generated report may also pass a structural check while failing in the browser\. The system therefore needs to control execution as well as generate content\. Its actions must be grounded in current metadata, restricted to read\-only access, recorded for inspection, and recoverable after an interruption\.

Prior work characterises the difficulty of this generation problem\. Spider introduced a cross\-domain text\-to\-SQL benchmark that tests generalisation to unseen database schemas, while the Big Bench for Large\-Scale Database Grounded Text\-to\-SQLs \(BIRD\) increased the scale and realism of database\-grounded tasks\[[1](https://arxiv.org/html/2610.06914#bib.bib1),[2](https://arxiv.org/html/2610.06914#bib.bib2)\]\. Natural\-language visualisation research adds another stage because a request must be mapped to an analytic specification and a visual form\[[3](https://arxiv.org/html/2610.06914#bib.bib9),[4](https://arxiv.org/html/2610.06914#bib.bib10),[5](https://arxiv.org/html/2610.06914#bib.bib11)\]\. These studies provide methods and evaluation settings for query and chart generation\. They do not define the permissions, persistent state, approval rules, or recovery behaviour of this company’s dashboard workflow\.

### I\-BProject aim and boundary

The aim of Text2Dashboard is to turn a natural\-language analytics request into an inspectable and dynamically verified dashboard over the company’s DataBrain platform\. The system is designed around DataBrain entity identifiers, DataLab metadata, explicit data domains, BigQuery\-oriented SQL, and the existing dashboard artifact format\.

The implemented endpoint is a validated local dashboard preview\. Online DataBrain publication is available in the plugin workflow but is not exposed as a standalone Runtime tool\. A standalone web chat interface is also outside the evaluated contribution\. These boundaries keep the study focused on the Agent control loop, company data access, and the quality of the generated artifact\.

### I\-CSystem approach

Text2Dashboard has two entry surfaces\. The Codex plugin presents the workflow through a set of progressively disclosed skills\. The standalone Python Agent Runtime obtains schema\-constrained decisions from an authenticated Codex command\-line interface \(CLI\) session and executes registered typed tools\. Both surfaces reuse the same DataBrain adapters, dashboard state machine, panel format, HyperText Markup Language \(HTML\) renderer, and validation pipeline\.

The standalone Agent follows a decision, action, and observation loop related to the Reasoning and Acting \(ReAct\) interaction pattern\[[6](https://arxiv.org/html/2610.06914#bib.bib4)\]\. In each iteration, the model may request one registered tool, ask a necessary question, or finish\. The Runtime remains responsible for argument validation, approval, tool execution, state persistence, and failure handling\. This separation is important because model decisions are probabilistic, while database policy and artifact state transitions require deterministic enforcement\.

The workflow first resolves a game entity and discovers candidate tables\. It then inspects the selected schema before generating bounded, read\-only SQL\. A dashboard is built from explicit panel artifacts rather than free\-form HTML\. The output passes static validation, an approved dynamic preflight against DataBrain, and browser inspection\. Lifecycle Hooks run at model, tool, checkpoint, and completion boundaries\. They provide exact\-call approval, duplicate\-call control, audit logging, checkpointing, dashboard recovery, and bounded handling of external service failures\.

### I\-DResearch objectives

The study has one principal objective: to design and empirically assess a governed natural\-language\-to\-dashboard agent for the company’s DataBrain environment\. Four subsidiary objectives make this aim operational\. The first is to define an architecture that separates probabilistic model decisions from tool authority, approval, and persistent state\. The second is to measure task correctness for real metadata discovery and approved read\-only SQL under criteria fixed before confirmatory execution\. The third is to examine whether deterministic lifecycle Hooks produce the specified outcomes for approval, failure handling, checkpointing, audit, and bounded external actions\. The fourth is to assess end\-to\-end dashboard completion, refinement, execution cost, and convergence within the implemented prototype boundary\.

### I\-EContributions and evidence

The central technical contribution is an implemented execution architecture in which the model proposes schema\-constrained actions while deterministic software retains execution authority through typed tools, lifecycle Hooks, and durable state\. The Codex plugin and standalone Runtime share the same DataBrain and dashboard capability layer\. The system also provides an inspectable artifact pipeline from verified metadata and read\-only SQL to panel configuration, dynamic HTML, preflight, and local preview\. The methodological contribution is an evaluation procedure that preserves frozen task criteria, failed runs, staged validation results, timing records, and controlled fault scenarios\.

The evidence characterises a research prototype rather than a production deployment\. Six of eight frozen metadata and SQL tasks met every strict criterion\. All four metadata tasks passed\. All four SQL tasks met the semantic rubric and executed successfully under the read\-only policy, while two failed the exact output\-column contract\. The final release completed all four frozen single\-panel dashboard tasks\. The ten controlled Hook and Runtime fault scenarios produced their specified outcomes without an unapproved external side effect\. An extension task nevertheless exposed failure to terminate within the declared step budget\. The results are reported by task group because the small, heterogeneous samples do not justify a pooled reliability estimate\.

## IIBackground and Related Work

### II\-ATool\-using language\-model agents

A language\-model agent differs from a single text\-generation call because it selects actions, receives observations from an environment, and uses those observations in later decisions\. ReAct demonstrated a prompting pattern that interleaves model reasoning with actions and observations\[[6](https://arxiv.org/html/2610.06914#bib.bib4)\]\. Toolformer studied how a language model can learn to decide when and how to call external APIs\[[7](https://arxiv.org/html/2610.06914#bib.bib5)\]\. These approaches establish how a model can select and use actions, but they do not by themselves specify application\-level authority over approvals, persistent artifacts, or recovery\. Text2Dashboard uses the same broad interaction pattern, but it does not train a new model\. A separate Runtime requests one structured decision at a time and retains authority over the available tools and their execution\.

This distinction changes what should be evaluated\. AgentBench evaluates models in interactive environments where success depends on a sequence of actions rather than a final answer alone\[[8](https://arxiv.org/html/2610.06914#bib.bib6)\]\. API\-Bank tests planning, retrieval, and calling of external APIs\[[9](https://arxiv.org/html/2610.06914#bib.bib32)\]\. ToolEmu studies risks that arise when language\-model agents can use tools with real consequences\[[10](https://arxiv.org/html/2610.06914#bib.bib7)\]\. WebArena measures task completion in a reproducible web environment\[[11](https://arxiv.org/html/2610.06914#bib.bib20)\]\. Theτ\\tau\-bench benchmark evaluates agents interacting with users and domain\-specific API policies, including the resulting database state\[[12](https://arxiv.org/html/2610.06914#bib.bib21)\]\. AgentDojo examines prompt injection through untrusted tool outputs\[[13](https://arxiv.org/html/2610.06914#bib.bib22)\]\. InjecAgent similarly benchmarks indirect prompt injection through external content encountered by tool\-integrated agents\[[14](https://arxiv.org/html/2610.06914#bib.bib33)\]\. SkillTV\-Bench evaluates whether judges can verify skill\-augmented agent executions using their trajectories and inspectable artifacts\[[15](https://arxiv.org/html/2610.06914#bib.bib31)\]\. These works motivate trajectory\-level evidence\. Their benchmark and emulation settings, however, do not constitute evidence that a particular enterprise workflow enforces its declared control boundaries\. The Text2Dashboard evaluation therefore records tool order, terminal state, approval events, latency, and prohibited actions in addition to the generated SQL and Dashboard artifact\.

### II\-BFrom language to data and visualisations

Text\-to\-SQL research addresses the mapping from a question to an executable query\. Spider tests cross\-domain generalisation to database schemas that were not seen during training\[[1](https://arxiv.org/html/2610.06914#bib.bib1)\]\. BIRD extends this setting with larger databases, database values, external knowledge, and query efficiency\[[2](https://arxiv.org/html/2610.06914#bib.bib2)\]\. Decomposed In\-Context Learning of Text\-to\-SQL with Self\-Correction \(DIN\-SQL\) divides the task into schema linking, classification, query generation, and correction\[[16](https://arxiv.org/html/2610.06914#bib.bib3)\]\. RAT\-SQL explicitly models schema relations and links between questions and database fields\[[17](https://arxiv.org/html/2610.06914#bib.bib23)\]; a separate schema\-linking study finds that this alignment is a major source of text\-to\-SQL errors\[[18](https://arxiv.org/html/2610.06914#bib.bib24)\]\. RESDSQL separates schema linking from SQL skeleton parsing\[[19](https://arxiv.org/html/2610.06914#bib.bib34)\]\. PICARD constrains decoding to reject invalid SQL prefixes\[[20](https://arxiv.org/html/2610.06914#bib.bib25)\], while Dr\.Spider tests robustness under changes to questions, schemas, and queries\[[21](https://arxiv.org/html/2610.06914#bib.bib26)\]\. Together, these studies establish schema grounding, constrained generation, and robustness as components of text\-to\-SQL evaluation\. They do not cover the exact result\-column contract, approval state, and persistent visual artifact required by the present workflow\.

Text2Dashboard has a narrower domain and a wider output\. It does not attempt to measure generalisation across unrelated databases\. It operates within DataBrain, where entity identity, data domain, table availability, and permissions must be resolved before SQL is accepted\. The confirmatory benchmark evaluates both metadata routing and read\-only SQL behaviour\. It does not support a claim of a new state\-of\-the\-art text\-to\-SQL model\.

Natural\-language visualisation adds analytic and presentation choices\. SeeDB recommends candidate visualisations by scoring patterns in the underlying data\[[22](https://arxiv.org/html/2610.06914#bib.bib35)\]\. Data2Vis framed visualisation generation as sequence translation to Vega\-Lite specifications\[[23](https://arxiv.org/html/2610.06914#bib.bib8)\]; Vega\-Lite defines a grammar for interactive visual encodings and selections\[[24](https://arxiv.org/html/2610.06914#bib.bib27)\]\. Natural Language for Data Visualization \(NL4DV\) maps natural\-language requests to attributes, analytic tasks, and candidate charts\[[3](https://arxiv.org/html/2610.06914#bib.bib9)\]\. nvBench provides a cross\-domain benchmark for language\-to\-visualisation tasks\[[4](https://arxiv.org/html/2610.06914#bib.bib10)\]\. LIDA separates data summarisation, goal exploration, visualisation generation, and infographic generation\[[5](https://arxiv.org/html/2610.06914#bib.bib11)\]\. MatPlotAgent combines code generation with visual feedback for scientific figures\[[25](https://arxiv.org/html/2610.06914#bib.bib36)\]\. Data Formulator lets authors specify data concepts for an AI\-assisted visualisation workflow\[[26](https://arxiv.org/html/2610.06914#bib.bib28)\]\. Data Formulator 2 extends this line of work to iterative authoring with both graphical and natural\-language input\[[27](https://arxiv.org/html/2610.06914#bib.bib29)\]\. DynaVis turns natural language editing requests into persistent visual controls\[[28](https://arxiv.org/html/2610.06914#bib.bib37)\]\. VisEval evaluates generated visualisations beyond code generation alone\[[29](https://arxiv.org/html/2610.06914#bib.bib30)\]\. Together, these works address data representation, analytic intent, visual specification, and chart evaluation\. A valid specification alone does not show that live company data can be queried safely or that an interactive report remains valid after parameters and state transitions are applied\.

Text2Dashboard applies that principle to a company dashboard lifecycle\. Each panel retains its SQL template, parameters, chart configuration, and state\. The final artifact is dynamic HTML rather than a static image\. Static contract validation, DataBrain\-backed preflight, and browser inspection remain separate gates because success at one level does not guarantee success at the next\.[TableI](https://arxiv.org/html/2610.06914#S2.T1)summarises the relationship between these research areas and the application\-specific requirements of Text2Dashboard\.

TABLE I:Relationship between the closest research areas and the project gap\.
### II\-CHuman control and lifecycle governance

Human\-AI interaction research emphasises communicating system capabilities, supporting correction, and maintaining appropriate user control\[[30](https://arxiv.org/html/2610.06914#bib.bib12)\]\. AI Chains shows how decomposing language\-model tasks can expose intermediate outputs and local control points\[[31](https://arxiv.org/html/2610.06914#bib.bib13)\]\. The United States National Institute of Standards and Technology \(NIST\) AI Risk Management Framework places validity, safety, resilience, accountability, transparency, and privacy within a wider risk\-management process\[[32](https://arxiv.org/html/2610.06914#bib.bib14)\]\.

These sources motivate human control and inspectability at different levels, but they do not prescribe the execution semantics of the DataBrain workflow\. In particular, a visible confirmation message is weaker than an approval bound to the exact arguments of one pending external action\.

Text2Dashboard implements a limited engineering interpretation of these ideas\. Approval is bound to the exact tool name and canonical arguments\. Checkpoint, approval, and policy failures stop execution\. Audit failure is recorded but is not allowed to disable the main workflow\. These mechanisms address observable failure modes in this application\. They do not demonstrate compliance with a complete organisational governance framework, and they do not replace the company’s access controls\.

### II\-DPosition relative to orchestration frameworks and retrieval\-augmented generation \(RAG\)

LangGraph documents durable execution, persistence, and human\-in\-the\-loop interrupts as features of its orchestration runtime\[[33](https://arxiv.org/html/2610.06914#bib.bib16),[34](https://arxiv.org/html/2610.06914#bib.bib17)\]\. Text2Dashboard implements overlapping concepts in a small custom Runtime\. This choice makes the event order, state representation, and approval authority directly inspectable for the dissertation\. It is not evidence that a custom Runtime is generally better\. No controlled performance or correctness comparison with LangGraph was performed\.

The standalone Agent does not implement retrieval\-augmented generation\. Its runtime dependencies do not include LangChain, LangGraph, an embedding model, or a vector database\. Instead, a deterministic function injects a compact, source\-labelled DataBrain routing summary into the model context, while physical tables and columns must still be verified through live metadata tools\.

### II\-EEngineering problem addressed

Prior work provides methods for tool use, schema\-grounded SQL, visual specification, and human intervention\. The application\-specific engineering problem addressed here is their controlled integration within DataBrain\. The model must be able to propose metadata and dashboard actions while authority over permissions, approval, persistence, and validation remains in deterministic software\. The following methodology examines one implementation of that boundary\. It does not claim that the architecture resolves a general gap across databases or orchestration frameworks\.

## IIIMethodology

### III\-AStudy Design and System Requirements

#### III\-A1Operational context

The target users are analysts and product managers who can specify an analytic need without necessarily knowing DataBrain entity identifiers, table names, SQL, or dashboard configuration\. The studied workflow begins with a natural\-language request and proceeds through metadata selection, approval of real data execution, artifact validation, and local browser inspection\. The system also supports a constrained revision of an existing dashboard while preserving unrelated panels\.

Data owners and administrators impose independent requirements concerning table permissions and company\-data transfer\. Runtime recovery and diagnostic traces are required for implementation review\. Experimental assessment further requires criteria fixed before confirmatory execution, retained failures, and a separation between implemented mechanisms and observed task outcomes\. These constraints determine the functional and non\-functional requirements below\.

#### III\-A2System requirements

[TableII](https://arxiv.org/html/2610.06914#S3.T2)summarises the requirements used to assess the implementation\. Functional requirements describe the user\-visible workflow\. Non\-functional requirements constrain how the workflow is executed\. Functional requirements \(FR\) and non\-functional requirements \(NFR\) use the corresponding prefixes in[TableII](https://arxiv.org/html/2610.06914#S3.T2)\.

TABLE II:Evidence\-backed system requirements\.Approval is grounded in Runtime state rather than model text\. It applies to the fingerprint of one tool name and one canonical argument object\. A changed SQL query, date range, or preflight request therefore requires new approval\. A successful preflight receipt is similarly bound to the hash of one generated HTML report\. Regenerating the report invalidates the receipt\.

#### III\-A3Domain boundary

The DataBrain\-specific scope allows the project to define explicit rules for game identity, public and business data domains, DataLab metadata, BigQuery\-oriented SQL, permission failures, and the company’s dashboard format\.

The project does not train a language model or claim general text\-to\-SQL accuracy across unrelated databases\. The standalone evaluated boundary ends at an approved, dynamically preflighted local preview\. Online DataBrain publishing is described by the plugin workflow but is not a registered standalone tool\. A separate web chat interface, a user study, and a comparison with LangGraph are also outside the completed evaluation\.

#### III\-A4Privacy, ethics, and company data handling

The model provider delegates authentication to the official Codex CLI using a ChatGPT account sign\-in flow\[[35](https://arxiv.org/html/2610.06914#bib.bib18)\]\. Text2Dashboard does not read or offer an OpenAI application programming interface \(API\) key path\. It also removes credential\-like environment variables from the model subprocess\. DataBrain credentials remain outside that model process and are used by the outer tool Runtime and credentialed DataBrain, preflight, or preview paths\.

Real DataBrain observations may enter later model context\. Each real run and cross\-process resume therefore requires the explicit–allow\-company\-data\-to\-codexacknowledgement\. The flag records a user decision but does not grant company permission\. Raw runs, SQL results, audit files, and screenshots remain in ignored local evidence directories\. Only manually reviewed and redacted derivatives may enter the dissertation\.

The evaluation did not recruit participants or collect new personal data\. Consequently, no user\-study or usability claim is made\. Raw SQL results, audit traces, internal identifiers, credentials, and unredacted screenshots are not included in the submitted report\. The numerical analysis uses locally retained structured summaries that do not expose row\-level company data\. This procedure reduces disclosure in the dissertation but does not replace the company’s access\-control and information\-governance processes\.

#### III\-A5Evaluation design

The study follows an iterative systems\-engineering method\. Requirements were implemented as typed tool schemas, state transitions, Hook policies, and artifact contracts\. The reported evaluation concerns the final frozen release\. Task criteria were fixed before confirmatory execution, and failed outcomes were retained rather than removed after inspection\.

The evaluation comprises four evidence groups\. First, the real\-DataBrain core benchmark contains eight valid tasks, split evenly between metadata and read\-only SQL\. Second, four frozen dashboard tasks, labelled D01 to D04, evaluate key performance indicator \(KPI\), line, category, and stacked visualisations against structured artifact criteria and a predefined browser\-inspection rubric\. Third, the extension evidence contains one two\-panel task, one joint parameter task, and one existing\-dashboard refinement\. Fourth, ten controlled fault scenarios examine permission errors, gateway retries, schema mismatch, approval, duplicate calls, search limits, checkpoint failure, changed arguments, and fail\-open audit behaviour\. The resulting evidence sequence is summarised in[Figure1](https://arxiv.org/html/2610.06914#S3.F1)\.

![Refer to caption](https://arxiv.org/html/2610.06914v1/Figures/evaluation-pipeline.png)Fig\. 1:Evaluation evidence pipeline\. Frozen criteria and append\-only run records precede structured checks, rubric\-based browser inspection where required, and final adjudication against a declared denominator\.Only tasks with frozen criteria and valid execution records enter a reported denominator\. Infrastructure failures and benchmark\-construction errors remain in the local evidence record but are not analysed as Agent outcomes\.

#### III\-A6Outcome measures and analysis

The primary outcome is strict task success against the frozen task rubric\. For a complete dashboard, success requires correct entity and data scope, acceptable table and fields, semantically valid read\-only SQL, a valid panel, successful static validation and preflight, a usable browser render, and no prohibited or unapproved action\. Stage\-level results are reported separately so that a correct SQL query cannot hide a failed terminal state or an unusable dashboard\.

Each criterion was evaluated from a specified evidence source\. Structured run records established tool order, approvals, terminal state, timing, and prohibited actions\. Persisted panel and report artifacts established field, parameter, and lifecycle contracts\. Dynamic preflight established compatibility with live DataBrain responses\. Browser inspection assessed visible content and console state using the predefined rubric\. These evidence types were kept separate so that a favourable observation at one stage could not override a failure at another\.

Efficiency measures include wall time, model time, tool time, model decisions, tool calls, token counts, and approval interventions\. Reliability measures include error classification, bounded retries, loop prevention, recovery state, and prohibited side effects\. Success proportions use Wilson 95% confidence intervals because the task samples are small\[[36](https://arxiv.org/html/2610.06914#bib.bib15)\]\. Raw denominators and individual failures remain visible\. No significance test is reported, and no manual\-workflow speed comparison or user study was completed\.

These requirements and evaluation rules define what the implementation must enforce and what the reported observations may support\. The following system design maps them to explicit decision, governance, state, and artifact boundaries\.

### III\-BSystem Architecture and Design

#### III\-B1Design principles

Text2Dashboard is organised around a simple division of authority\. The language model proposes the next action, while deterministic code decides whether that action may run and how its result is recorded\. This prevents a fluent model response from becoming equivalent to approval, valid SQL, or a verified Dashboard\.

Four principles shaped the architecture\. First, probabilistic reasoning and deterministic enforcement are separate\. Second, the Codex plugin and the standalone Agent share domain capabilities rather than maintaining two implementations\. Third, recovery depends on files that survive both model and Python process failure\. Fourth, report generation is staged so that static correctness, live\-data compatibility, and browser inspection remain distinct claims\.

#### III\-B2Dual\-entry architecture

[Figure2](https://arxiv.org/html/2610.06914#S3.F2)summarises the two entry points and their shared DataBrain and Dashboard capability layer\.

![Refer to caption](https://arxiv.org/html/2610.06914v1/Figures/system-architecture.png)Fig\. 2:Dual\-entry architecture with deterministic authority retained by the Runtime, Hooks, tools, and artifact validators\.The plugin is the embedded entry point\. Codex provides the surrounding conversation and invokes skills that impose the Text2Dashboard workflow and load domain guidance when needed\. The standalone package supplies its own manager loop, typed tools, state, Hooks, command\-line resume, and experiment runner\. Both entries reuse the same DataBrain scripts, Dashboard state utility, composer, validator, and report builder\. They are therefore two orchestration surfaces over one capability layer, not two separate Dashboard products\.

The evaluated standalone boundary ends at an approved local preview\. Online DataBrain publication belongs to the plugin publishing workflow and is not registered as a standalone Agent tool\. The shared Dashboard state still recognisespublishedso that artifacts remain compatible when control returns to the plugin\.

#### III\-B3Structured decision loop

The provider interface contains one operation,decide\(state, tool\_schemas\)\. Its result is anAgentDecisionwith exactly one of three meanings: call one tool, ask one question, or finish\. This narrow protocol keeps ChatGPT authentication and Codex process management outside the core Runtime and makes each proposed transition inspectable\.

For each bounded iteration,AgentRuntimeemitsbefore\_model, requests one schema\-constrained decision, records the decision, and emitsafter\_model\. A tool decision then passes throughbefore\_tool, the registry, and eitherafter\_toolortool\_error\.ToolRegistryvalidates required fields, rejects additional or mistyped arguments, and converts exceptions intoToolResultobservations\. The next model turn therefore receives a structured outcome instead of gaining direct Python, shell, file, or DataBrain access\. A maximum\-step limit provides a final bound if other stopping rules do not terminate the run\.

The Codex subscription adapter reuses one Codex session within a run\. Later calls send only new Runtime events while the session retains the earlier instructions and schemas\. Session reuse reduces repeated context transfer, but the session is a performance aid rather than the source of truth\. The Runtime remains authoritative for the trace, allowed tools, approvals, stopping, and persisted state\.

#### III\-B4Hook governance layer

Hooks place policy at stable lifecycle boundaries instead of relying on the manager prompt\. The normal event order is:

run\_start\-\>checkpoint\-\>before\_model\-\>after\_model

\-\>before\_tool\-\>after\_tool\|tool\_error\-\>checkpoint

\-\>\.\.\.\-\>run\_complete\|run\_failed\-\>checkpoint

Each Hook can returncontinue,pause,block\_external, orblock\. A pause creates a resumable user question\. An external block records a temporary dependency failure separately from an internal Agent failure\. Hook implementations also choose a failure policy\. Audit is fail\-open because losing one telemetry write should not silently become a business decision\. Approval, Dashboard\-state loading, circuit breaking, loop protection, and checkpoint persistence are fail\-closed because their failure removes a safety or recovery guarantee\. The responsibilities and execution boundaries of the built\-in Hooks are summarised in[TableIII](https://arxiv.org/html/2610.06914#S3.T3)\.

TABLE III:Responsibilities of the built\-in governance Hooks\.Approval is bound to a Secure Hash Algorithm 256\-bit \(SHA\-256\) fingerprint of the tool name and canonical arguments\. Fields that could imitate confirmation are excluded from the canonical object\. The Runtime adds the internal confirmation flag only after matching an approval already held inAgentState\. Changing SQL, a date range, or another argument changes the fingerprint and requires a new approval\. The approval is removed after success or failure, so it cannot authorise a later call\.

The loop guard combines a call fingerprint with a hash of\.dashboard\. This distinction matters during iterative construction\. Repeating the same validator after changing a panel is legitimate, while repeating the same metadata call with no new evidence is not\. The circuit breaker is also deliberately narrow\. Metadata operations retry selected timeouts and Hypertext Transfer Protocol \(HTTP\) 502, 503, and 504 responses up to three attempts\. SQL execution is not automatically retried because an uncertain execution outcome should not be duplicated\.

#### III\-B5Two\-layer state and recovery

Runtime and Dashboard state have different lifetimes\.AgentStaterecords the request, phase, iteration, trace, open question, pending and granted approvals, final response, error, and provider session information held incontext\_data\.JsonStateStorevalidates run identifiers and writes through a temporary file followed by atomic replacement\. A waiting or externally blocked run can therefore be loaded by a later command and resumed with new user input\.

The second authority is\.dashboard/\_state\.json\. It records Dashboard identity, panel ordering and status, and publication binding\.DASHBOARD\.md,ROADMAP\.md, andSTATE\.mdare readable views generated by the Dashboard state utility\. A panel directory holdsCONTEXT\.md,sql\.sql, andpanel\.json\. Keeping these artifacts outsideAgentStateallows a new run, the plugin, or a human reviewer to continue the same Dashboard without replaying an old conversation\.

On run start,DashboardResumeHookcopies validated Dashboard state into the new Agent context\. Corrupt state fails closed\. Cross\-process resume rebuilds the provider and tool registry, and real company\-data runs require acknowledgement again\. The special failed\-run retry path accepts only provider failures\. It removes the uncertain Codex session before trying the model turn again because the provider may have completed a decision that the outer Runtime did not receive\.

#### III\-B6Data and artifact pipelines

The DataBrain boundary exposes five typed tools\.inspect\_databrain\_metadataperforms the common read\-only sequence of entity resolution, up to three keyword searches, candidate selection, and schema retrieval\. The four fine\-grained tools retain explicit access to entity search, table discovery, schema retrieval, and SQL execution when targeted refinement is needed\.

Entity identity does not determine data scope\. Public intelligence and opinion requests are routed togame\_code=databrain, even when entity search returns a game\-specific code\. A business code is retained only when business scope is explicit or no public metric rule applies\. HTTP 401 is represented as invalid or expired authentication\. HTTP 403 means that the credential may be valid but lacks access to the selected scope or table\. This separation avoids treating every permission response as a failed login\.

The shared artifact lifecycle recognisesplanned,drafted,built,verified, andpublished\. Standalone construction can move directly from a planned entry to a built panel, while the fuller plugin discussion workflow may use the drafted stage\.build\_dashboard\_panelnormalises the panel contract, checks panel identity, chart axes, KPI fields, parameters, and read\-only SQL, then writes the panel artifacts\.validate\_dashboard\_panelchanges the panel to verified only when validator output is accepted\.

Strict report generation collects verified panels in order, merges parameters, createsdashboard\.json, and renders SQL templates intodashboard\.html\. Static report validation checks the local contract without contacting DataBrain\. Approved dynamic preflight then executes bounded option and chart queries against real DataBrain, including default, single, multiple, and empty selection cases where applicable\. It checks returned fields and unresolved placeholders\.

A successful preflight receipt stores the DataBrain host, completion time, and SHA\-256 of the exact HTML report\. It stores no credential\. Regenerating the report deletes the receipt, and preview refuses a report whose hash no longer matches\. The managed preview proxy binds to localhost, holds a random shutdown token, and exposes only non\-secret status through the Agent tool\.

#### III\-B7Security trade\-offs

The Codex subprocess receives a sanitised environment that removes API billing variables and credential\-like names\. It runs with a read\-only sandbox and a strict decision schema\. A token read from the recommended project\.envis explicitly passed only to DataBrain, preflight, or preview paths\. An operating\-system token can also be inherited by general Dashboard child processes, although the compose and static validation scripts do not use it\. Panel identifiers and Dashboard paths are constrained to the configured workspace\.

These measures provide defence in depth, not a formal security proof\. SQL validation is lexical and accepts only one statement beginning withSELECTorWITHwhile rejecting known mutation keywords\. It is not a full SQL parser\. Redaction depends on recognised secret patterns, and the localhost proxy remains a credential\-bearing process while active\. These limits define the evaluated prototype boundary and motivate further security assessment before wider deployment\.

The architecture defines where authority resides\. The implementation described next shows how those boundaries are realised in the Runtime, provider adapter, Hooks, DataBrain tools, artifact lifecycle, and evaluation records\.

### III\-CImplementation

The standalone system is implemented as a Python package undersrc/text2dashboard\_agent\. The package translates the architectural boundaries in[SectionIII\-B](https://arxiv.org/html/2610.06914#S3.SS2)into a bounded Runtime, a model\-provider adapter, deterministic governance Hooks, DataBrain and Dashboard tools, and evaluation instrumentation\. The main modules are summarised in[TableIV](https://arxiv.org/html/2610.06914#S3.T4)\. The implementation uses Python data classes, protocols, subprocess management, JavaScript Object Notation \(JSON\) persistence, and a typed\-tool layer\. It does not depend on LangChain, LangGraph, or an OpenAI software development kit\.

TABLE IV:Principal standalone implementation modules\.#### III\-C1Runtime implementation

The Runtime implements one model decision per iteration\. A decision has exactly one of three forms: a request to call one registered tool, a necessary question for the user, or a final response\. Tool requests are represented separately from execution results, and every transition appends a structured trace event\. This representation allows a proposed action to be inspected before the corresponding handler is reached and avoids reconstructing experimental trajectories from terminal text\.

Tools are exposed through a registry with JSON\-compatible argument schemas\. The registry rejects unknown names, missing fields, unexpected fields, and basic type mismatches before invoking a handler\. Domain\-specific checks remain inside the handlers because the local schema validator implements only the subset needed by the registered tools\. A maximum\-step limit bounds the complete loop\. Questions, approval pauses, external dependency blocks, internal failures, and successful completion are represented as distinct persisted phases rather than as free\-form model messages\.

The resulting implementation preserves the principal authority boundary\. The model chooses among advertised actions, but it cannot register a new tool, bypass argument checks, add approval, alter a checkpoint, or assign a terminal phase directly\. These transitions remain functions of Runtime and Hook logic\.

#### III\-C2Model\-provider integration

The provider adapter delegates authentication and model execution to the official Codex command\-line client\. This supports ChatGPT account sign\-in, non\-interactive structured output, a read\-only execution sandbox, and session resume\[[35](https://arxiv.org/html/2610.06914#bib.bib18),[37](https://arxiv.org/html/2610.06914#bib.bib19)\]\. Text2Dashboard verifies that the client is authenticated through the account flow and does not read or copy its credential files\. An API\-key billing path is not exposed by the standalone Agent\.

The first model request contains the manager instructions, a redacted Runtime state, registered tool schemas, and compact DataBrain routing guidance\. A required output schema limits the result to the three decision forms accepted by the Runtime\. Physical tables and fields are not accepted from the guidance alone and must be established through metadata tools\. Invalid JSON, an unknown decision type, or malformed tool arguments is returned as a provider failure rather than being treated as an executable action\.

Within a run, the provider records the model\-session identifier and a trace cursor\. Subsequent decisions resume the session and transmit only Runtime events added since the previous turn\. This reduces repeated context transfer, while persisted Runtime state remains the source of truth for tools, approvals, observations, and stopping conditions\. Session history is therefore a context optimisation rather than a recovery or governance mechanism\.

Before starting the model subprocess, the adapter removes API\-billing variables and inherited variables whose names indicate tokens, keys, secrets, passwords, credentials, or authorisation values\. This includes DataBrain credentials\. Only non\-secret model configuration is stored in the workspace\.

#### III\-C3Governance and persistence

Hooks are registered at stable lifecycle boundaries around model decisions, tool execution, errors, checkpoints, and completion\. Each Hook returns a directive that either permits progress, pauses for user input, records an external dependency block, or blocks the run\. Hook order is deterministic, and processing stops at the first directive that prevents continuation\.

Failure policy depends on the guarantee provided by the Hook\. Audit is fail\-open because a telemetry\-write failure should remain visible without silently changing a permitted benign action\. Approval, checkpointing, Dashboard\-state loading, circuit breaking, and loop control are fail\-closed because failure of these mechanisms removes a safety or recovery condition\. Hook errors and directives are written into the same trace used for later inspection\.

Approval is attached to an exact tool call rather than to natural\-language confirmation\. The tool name and canonical arguments are serialised and hashed\. Confirmation\-like fields are excluded from that canonical object\. A matching grant is consumed after the call, whether execution succeeds or fails\. Changing the SQL, date range, report, or another argument changes the fingerprint and requires another approval\. Duplicate\-call control combines the call fingerprint with Dashboard state, permitting a repeated validation after a material artifact change while stopping an unchanged loop\.

Agent state and Dashboard state are persisted separately\. Agent state stores the request, phase, trace, questions, approvals, errors, and provider\-session metadata\. It is written through a temporary file followed by atomic replacement\. Dashboard state stores artifact identity, panel order, panel lifecycle, and publication binding\. This separation permits a later Runtime process, the plugin, or a reviewer to inspect and continue the artifact without replaying the original model conversation\. Corrupt Dashboard state blocks resume rather than being reconstructed speculatively\.

#### III\-C4DataBrain integration

The DataBrain adapter reuses the existing project capability for entity search, table discovery, schema retrieval, and SQL execution\. The standalone tool layer adds typed inputs, error classification, retry bounds, data\-scope routing, and approval enforcement around that capability\. A composite metadata tool performs the common read\-only sequence of entity resolution, bounded keyword searches, candidate selection, and schema retrieval\. Fine\-grained metadata tools remain available when a targeted correction is required\.

Entity identity and data scope are treated as separate decisions\. Explicit public\-intelligence or opinion requests use the public DataBrain scope, whereas explicit business requests retain the appropriate game code\. This rule is enforced by the adapter rather than relying only on the model prompt\. Metadata operations retry selected connection failures, timeouts, and HTTP 502, 503, and 504 responses within a fixed bound\. HTTP 401 is classified as an authentication failure, while HTTP 403 is classified as insufficient permission for the selected scope or table\. Real SQL execution is not retried automatically\.

The SQL policy removes comments, permits one statement beginning withSELECTorWITH, masks string literals for inspection, and rejects recognised mutation or administrative keywords\. The same policy is applied to direct execution and to SQL templates stored in panels\. A real, non\-dry\-run request additionally requires the Runtime\-supplied approval marker\. This lexical policy provides a deterministic prototype boundary but is not equivalent to parser\-based SQL verification\.

#### III\-C5Dashboard artifact lifecycle

Dashboard construction is based on persistent panel artifacts rather than model\-generated HTML\. A panel contains source context, a SQL template, parameters, chart configuration, and lifecycle state\. Panel identifiers and workspace paths are constrained before files are written\. The builder normalises supported input variants, checks chart and parameter contracts, and applies the read\-only SQL policy to main, tab, and option queries\. A panel is marked as verified only after its artifact validator succeeds\.

Report composition collects verified panels in their recorded order, merges parameters by name, and renders the shared dynamic HTML template\. Static validation checks chart contracts, parameter declarations, SQL placeholders, API construction, and credential handling without contacting DataBrain\. This stage is intentionally distinct from dynamic preflight because local structural validity does not establish compatibility with live results\.

Approved preflight executes bounded parameter\-option and chart queries against DataBrain\. Where applicable, it covers default, single, multiple, and empty selections and checks returned fields and unresolved placeholders\. Success creates a receipt bound to the hash of the exact HTML report and containing no credential\. Regenerating the report invalidates the receipt\. Local preview is permitted only when the current report hash matches a successful preflight and is served through a localhost process with workspace\-local lifecycle state\. These rules implement the transition from built artifact to verified dynamic preview without allowing model text to assert that a stage has passed\.

#### III\-C6Evaluation instrumentation

The experiment runner loads frozen task definitions, rejects inconsistent task policies, and creates an isolated workspace and persisted state for each run\. A manifest records task order, configuration, random seed, and the applicable policy\. A dedicated Hook blocks any tool declared forbidden for a task before its handler can execute\.

Outcome derivation uses terminal phase, selected metadata, required tool order, SQL fragments, panel count, artifact contracts, and prohibited attempts\. Timing records distinguish model duration, tool duration, and wall time\. Model attempts, tool calls, token usage, session reuse, approvals, and Hook interventions are also retained\. Raw run results and browser\-inspection records are append\-only, while tabular and summary outputs are regenerated from those records\. Browser inspection can reject a structurally successful Dashboard but cannot promote a run that failed its structured criteria\.

#### III\-C7Implementation limitations

The local argument validator implements a shallow schema subset, and SQL policy is lexical rather than parser\-based\. Tool execution is synchronous, secret redaction is pattern\-based, and managed preview depends on explicit subprocess lifecycle control\. A DataBrain token exported as an operating\-system variable may be inherited by operational Dashboard subprocesses, although the model subprocess remains sanitised\. The standalone Runtime does not expose online publication or a browser chat interface\. These are implementation limits of the evaluated prototype rather than completed or evaluated capabilities\.

## IVResults and Analysis

### IV\-AEvidence and adjudication

The final evaluation uses the four evidence groups defined in[SectionIII\-A](https://arxiv.org/html/2610.06914#S3.SS1)\. The real core benchmark evaluates metadata and read\-only SQL\. The final single\-panel group evaluates four dashboard archetypes\. Extension and refinement tasks examine multi\-panel construction, parameters, termination, and preservation of an existing Dashboard\. A controlled fault experiment evaluates governance outcomes\.

Every reported result is assigned to its own declared denominator\. Only final release runs with valid task definitions and experiment infrastructure enter the analysis\. The predefined browser rubric can reject a run that passes the structured artifact criteria, but it cannot promote a run that fails those criteria\. Wilson 95% confidence intervals are reported for small binomial task groups\[[36](https://arxiv.org/html/2610.06914#bib.bib15)\]\. No significance test, ablation claim, or manual\-workflow speed comparison is made\.[TableV](https://arxiv.org/html/2610.06914#S4.T5)reports the resulting task\-group outcomes\.

TABLE V:Strict task outcomes\. Groups are reported separately\.The groups are not pooled because they cover different task families and step budgets\. The reported proportions therefore describe only their declared task sets\.[Figure3](https://arxiv.org/html/2610.06914#S4.F3)presents the success proportions and observed execution times without combining the groups\.

![Refer to caption](https://arxiv.org/html/2610.06914v1/Figures/evaluation-summary.png)Fig\. 3:Strict success by declared task group and observed execution time\. Bars use the denominators shown on the figure\. Core time is the median over eight runs, the final single\-panel group uses a group mean, and D06 and D05/D07 are single\-run observations\.
### IV\-BMetadata and read\-only SQL

The valid core denominator contains four metadata and four SQL tasks\. Metadata strict success was 4/4\. All selected top\-1 entity identifier, entity type, and game\-code checks passed\. The six tasks that required table and schema evidence also found the frozen table and field criteria\. No prohibited tool was attempted\.

Strict SQL success was 2/4\. However, all four SQL tasks passed the semantic rubric and eventually executed an approved read\-only query successfully\. The two strict failures were output\-contract failures\. One returnedrevenue\_usdrather than the frozen aliasrevenue\. The other returnedtotal\_mentionsrather thanmentions\. That task also produced an invalid fully qualified identifier on its first attempt, observed the DataBrain error, changed the SQL, obtained a new fingerprinted approval, and succeeded on the repaired query\.

The disaggregated results distinguish semantic query correctness from interface compatibility\. Metadata routing passed in this four\-task subset and the SQL semantic rubric passed in 4/4 cases\. Exact result\-interface compliance was 2/4, yielding overall strict success of 6/8\. The alias failures affect downstream Dashboard bindings even when the query is semantically acceptable\. They therefore remain strict failures under the frozen rubric\.

### IV\-CFinal single\-panel Dashboard correctness

The final release was evaluated on four frozen single\-panel archetypes: KPI, line, category bar, and sentiment stack\. All four runs passed the declared entity, metadata, panel, SQL, tool\-order, validation, preflight, and prohibited\-action criteria\. Each local browser preview was then inspected against the frozen visual rubric\. Every task displayed one visible non\-empty result of the requested form, and no page\-console warning or error was observed\. Strict success was 4/4\.

Every single\-panel task also made one rejectedbuild\_dashboard\_panelcall usingsqlrather thansql\_template\. The typed tool prevented an invalid artifact, and the Agent repaired the call on its next decision\. This preserved correctness but added one model round trip per task, exposing a remaining mismatch between model tool affordance and the formal panel contract\.

### IV\-DExtension, parameters, and refinement

D06 required two coherent panels over one data source and time window\. It passed the structured checks, one approved real preflight, and browser inspection\. The output contained exactly two non\-empty panels with the required metric and sentiment\-series contracts\. This observation establishes successful multi\-panel composition for the evaluated task without manual file editing\. It does not estimate success across other multi\-panel requests\.

The joint D05/D07 task tested an interactive filter and its default, single, multiple, and empty selection paths\. Its entity, schema, panel, SQL, safety, and tool\-order criteria were observed as correct\. Three separately approved dynamic preflights succeeded\. The manager loop then rebuilt the panel after each valid chain, invalidated the latest report receipt, and did not return a final decision within 30 steps\. The formal outcome therefore remains a failure\. This distinction is important: the parameter and preflight mechanisms produced the required intermediate states, but the Agent did not recognise that the task was complete\. The observation identifies convergence and termination policy, rather than parameter execution, as the failed system property in this run\.

R03 started from a copied final\-release Dashboard and requested only a shorter time window\. The source artifact remained unchanged\. The working copy preserved panel count, panel ID, chart type, metric, entity filter, grouping, and ordering, while changing the 30\-day and 31\-row settings to 14 days and 15 rows\. Real preflight and browser inspection passed, and no initialise, add\-panel, preview, or publication action was attempted\. This single observation supports state\-preserving refinement, not general multi\-turn reliability\.

### IV\-EEfficiency

The observed duration and interaction counts are reported in[TableVI](https://arxiv.org/html/2610.06914#S4.T6)\.

TABLE VI:Observed end\-to\-end efficiency\. Aggregate and single\-run rows are labelled separately\.Core median wall time was 430\.852 seconds, with an interquartile range of 385\.413 seconds and p95 of 700\.430 seconds\. Metadata median time was 242\.255 seconds, compared with 631\.793 seconds for SQL\. The mean tool time was 9\.141 seconds per task, while mean model time was 438\.834 seconds\.

The same pattern became stronger for Dashboard work\. Final single\-panel tools averaged 16\.31 seconds in total per run, while model decisions averaged 28\.65 minutes\. D06 spent 23\.45 seconds in tools and 39\.52 minutes in model decisions\. The failed D05/D07 run spent 56\.32 seconds in tools and 60\.19 minutes in model decisions\. R03 used 13 model attempts, 11 tool calls, and about 26\.1 minutes of active time excluding explicit approval wait\. Its cumulative input usage reached 2\.36 million tokens, of which 1\.72 million were cached\.

These measurements do not establish that the Agent is faster than a human because no manual baseline was collected\. Within the recorded runs, model time accounted for the largest share of wall time, while DataBrain and local tool execution accounted for a small share\. The observed efficiency constraints were repeated sequential model decisions and, in D05/D07, failure to terminate within the step budget\. The latency is therefore primarily an orchestration bottleneck in the evaluated architecture\. Each additional repair, validation, or completion decision incurs another model round trip even when the associated local or DataBrain tool completes quickly\.

### IV\-FControlled Hook fault evaluation

Ten controlled component scenarios were executed\. All ten produced the specified governance outcome, and no unapproved external side effect occurred\.[TableVII](https://arxiv.org/html/2610.06914#S4.T7)groups the observed outcomes by governance function\.

TABLE VII:Grouped outcomes from the controlled Hook fault experiment\.Within the registered tools and injected fault classes, the matrix provides direct evidence for deterministic approval, bounded metadata failure, observable fail\-open telemetry, and fail\-closed recovery controls\. It does not establish resistance to arbitrary prompt injection, security against an adversarial bypass of the lexical SQL policy, or an end\-to\-end reliability rate under live service outages\.

## VDiscussion

### V\-AInterpretation of findings

The implementation satisfies the architectural objective of separating model proposals from execution authority\. Model output is constrained to one typed decision, while the Runtime and Hooks retain control over tool registration, argument validation, approval, checkpointing, and terminal state\. Agent state and Dashboard state are persisted separately\. Consequently, recovery and artifact continuation do not rely exclusively on the model provider’s conversation history\. The plugin and standalone entry points reuse the same DataBrain and artifact capabilities, although their orchestration mechanisms remain distinct\.

The core benchmark indicates that metadata selection and semantic SQL were more reliable than exact result\-interface compliance in the evaluated sample\. Strict success was 6/8, comprising 4/4 metadata passes and 2/4 strict SQL passes\. All four SQL tasks met the semantic rubric, but two returned column aliases that did not satisfy the frozen downstream contract\. This difference is operationally relevant because a semantically valid query may still be unusable by a panel that expects a specific field name\.

The controlled fault matrix provides evidence that the implemented Hook boundaries produced the specified outcomes for ten declared scenarios\. The observations include exact\-call approval, changed\-argument reapproval, bounded metadata retries, loop interruption, checkpoint failure, and fail\-open audit behaviour\. Zero unapproved external side effects were recorded in these controlled scenarios\. This finding is limited to registered tools and injected fault classes\. It is not a general security assurance\.

End\-to\-end dashboard evidence was mixed\. The final release passed four frozen single\-panel tasks\. One two\-panel task and one existing\-dashboard refinement also passed their structured and browser criteria\. In contrast, the joint parameter task failed because the manager loop did not terminate within 30 steps, despite producing three successful intermediate preflights\. Across the recorded task groups, model decisions accounted for more than 97% of wall time\. These observations locate the principal remaining limitations in exact contract adherence, sequential model interaction, and termination control\.

Taken together, the evidence supports the architectural claim that model proposals can be separated from deterministic execution authority, and it documents successful completion of the specified tasks\. It does not justify a claim of production readiness, a general reliability rate, or autonomous suitability for unattended daily reporting\.

### V\-BThreats to validity

Internal validity is limited by model and service variability and by one confirmatory execution per frozen task\. The system was developed and evaluated by the same researcher, and the browser\-based visual assessment was performed by a single evaluator using a predefined rubric\. This design introduces potential experimenter and evaluator bias\. The R03 infrastructure correction was confined to process search paths and preserved the initial artifact hash, but it still required an exclusion decision\.

Construct validity is affected by exact alias requirements, browser inspection, and step\-budget definitions\. Alias failures correctly measure interface compatibility, but they make semantic SQL success look different from strict success\. The frozen visual rubric provides consistent criteria but does not measure usability as a user study would\. A maximum\-step failure operationalises autonomous convergence, even when an intermediate artifact satisfies the dynamic checks\. Other definitions of completion could produce a different classification\.

External validity is limited by a DataBrain\-only system, a small set of evaluated game entities, public intelligence and opinion tables, and a small set of chart archetypes\. Business\-table permissions vary independently from entity visibility\. Results therefore do not transfer automatically to other DataBrain domains, arbitrary databases, schemas, or business intelligence \(BI\) platforms\. Prior text\-to\-SQL evaluation work similarly shows that test\-set construction changes what generalisation claims an experiment can support\[[38](https://arxiv.org/html/2610.06914#bib.bib38)\]\.

Conclusion validity is limited by small denominators and one confirmatory execution per task\. The wide Wilson intervals make this uncertainty explicit, and single observations are reported without treating them as rate estimates\. No hypothesis test is reported, no ablation identifies the causal effect of the composite tool or session reuse, and no manual baseline supports a comparative efficiency claim\. No user study supports a usability claim\. Raw company\-data traces and screenshots remain local and are not used as submitted dissertation figures\. The numerical results derive from structured run summaries and append\-only browser\-inspection records\.

## VIFuture Work

The most immediate engineering priorities are therefore targeted improvements to output\-contract adherence and deterministic convergence\. The Runtime should recognise when the current artifact already satisfies a successful preflight and prevent unnecessary rebuilds before finalisation\. Tool descriptions and argument contracts should also reduce the repeatedsqlversussql\_templaterepair\. Further implementation work should reduce sequential model calls, adopt parser\-backed SQL validation, minimise credential inheritance by operational subprocesses, and improve preview\-process cleanup\. Because model decisions accounted for more than 97% of observed runtime in the reported groups, reducing unnecessary model round trips is likely to be an important direction for improving practical responsiveness\.

Future evaluation should strengthen the evidence base rather than only expand functionality\. Representative tasks should be repeated across multiple runs to measure variability and convergence more convincingly\. A small governance ablation could compare the full Runtime with reduced variants that remove, for example, approval or loop\-control mechanisms, allowing their contribution to redundant calls, unapproved or prohibited actions, and termination behaviour to be examined more directly\. Broader evaluation should also include additional entity and schema families, independent visual reviewers, and a predefined manual baseline where comparative efficiency is a research question\.

A further research direction is to examine how much of the current governed execution design transfers beyond DataBrain\. The present study deliberately uses application\-specific metadata, permissions, artifact formats, and validation rules, so it does not establish portability to other databases or business\-intelligence platforms\. Future work could separate reusable governance mechanisms from DataBrain\-specific adapters and evaluate whether the same authority, state, and validation principles remain effective in other enterprise analytics environments\. Online publication and a standalone browser interface should likewise be evaluated only after they are implemented as explicit, approval\-gated capabilities\.

## VIIConclusion

This dissertation presented and evaluated Text2Dashboard, a natural\-language Dashboard Agent in which the language model proposes actions while deterministic software retains authority over DataBrain access, approved read\-only SQL, persistent visual artifacts, and lifecycle governance\. The architecture combines schema\-constrained model decisions with typed tools, exact\-call approval, persistent Agent and Dashboard state, staged validation, and controlled recovery\.

Within the evaluated task set, metadata routing passed in 4/4 tasks, the final single\-panel release passed 4/4 tasks, one two\-panel task passed, and one state\-preserving refinement passed\. The ten controlled Hook and Runtime fault scenarios produced their specified outcomes without an unapproved external side effect\. At the same time, exact SQL output contracts passed in only 2/4 SQL tasks, the joint parameterised task did not terminate within its step budget, and new Dashboard tasks required approximately 29 to 61 minutes\. These values describe the declared tasks and should not be interpreted as general reliability or efficiency rates\.

The evaluation highlights two broader lessons from the prototype\. First, semantic correctness alone is insufficient for an artifact\-producing agent: although all four SQL tasks satisfied the semantic rubric, two failed because their output aliases did not match the downstream panel contract\. Evaluation of this type of system should therefore consider machine\-readable interface compatibility in addition to query semantics\. Second, successful intermediate tool execution does not guarantee agent\-level convergence\. In the parameterised task, valid panels and successful preflights were produced, but the manager loop continued rebuilding the artifact and eventually reached the step limit\. This suggests that completion and termination should be treated as explicit system properties rather than left entirely to model judgement\.

## References

- \[1\]T\. Yu, R\. Zhang, K\. Yang, M\. Yasunaga, D\. Wang, Z\. Li, J\. Ma, I\. Li, Q\. Yao, S\. Roman, Z\. Zhang, and D\. Radev\(2018\)Spider: A Large\-Scale Human\-Labeled Dataset for Complex and Cross\-Domain Semantic Parsing and Text\-to\-SQL Task\.InProc\. EMNLP,pp\. 3911–3921\.Cited by:[§I\-A](https://arxiv.org/html/2610.06914#S1.SS1.p3.1),[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p1.1)\.
- \[2\]J\. Li, B\. Hui, G\. Qu, J\. Yang, B\. Li, B\. Li, B\. Wang, B\. Qin, R\. Geng, N\. Huo, X\. Zhou, C\. Ma, G\. Li, K\. Chang, F\. Huang, R\. Cheng, and Y\. Li\(2023\)Can LLM Already Serve as A Database Interface? A BIg Bench for Large\-Scale Database Grounded Text\-to\-SQLs\.InAdv\. Neural Inf\. Process\. Syst\. 36,pp\. 42330–42357\.Cited by:[§I\-A](https://arxiv.org/html/2610.06914#S1.SS1.p3.1),[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p1.1)\.
- \[3\]\(2021\)NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queries\.IEEE Trans\. Vis\. Comput\. Graph\.27\(2\),pp\. 369–379\.Cited by:[§I\-A](https://arxiv.org/html/2610.06914#S1.SS1.p3.1),[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p3.1)\.
- \[4\]Y\. Luo, N\. Tang, G\. Li, C\. Chai, W\. Li, and X\. Qin\(2021\)Synthesizing Natural Language to Visualization \(NL2VIS\) Benchmarks from NL2SQL Benchmarks\.InProc\. ACM SIGMOD,pp\. 1235–1247\.Cited by:[§I\-A](https://arxiv.org/html/2610.06914#S1.SS1.p3.1),[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p3.1)\.
- \[5\]V\. Dibia\(2023\)LIDA: A Tool for Automatic Generation of Grammar\-Agnostic Visualizations and Infographics using Large Language Models\.InProc\. ACL System Demonstrations,pp\. 113–126\.Cited by:[§I\-A](https://arxiv.org/html/2610.06914#S1.SS1.p3.1),[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p3.1)\.
- \[6\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\(2023\)ReAct: Synergizing Reasoning and Acting in Language Models\.InProc\. ICLR,Cited by:[§I\-C](https://arxiv.org/html/2610.06914#S1.SS3.p2.1),[§II\-A](https://arxiv.org/html/2610.06914#S2.SS1.p1.1)\.
- \[7\]T\. Schick, J\. Dwivedi\-Yu, R\. Dessi, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. Scialom\(2023\)Toolformer: Language Models Can Teach Themselves to Use Tools\.InAdv\. Neural Inf\. Process\. Syst\. 36,pp\. 68539–68551\.Cited by:[§II\-A](https://arxiv.org/html/2610.06914#S2.SS1.p1.1)\.
- \[8\]X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang, S\. Zhang, X\. Deng, A\. Zeng, Z\. Du, C\. Zhang, S\. Shen, T\. Zhang, Y\. Su, H\. Sun, M\. Huang, Y\. Dong, and J\. Tang\(2024\)AgentBench: Evaluating LLMs as Agents\.InProc\. ICLR,Cited by:[§II\-A](https://arxiv.org/html/2610.06914#S2.SS1.p2.1)\.
- \[9\]M\. Li, Y\. Zhao, B\. Yu, F\. Song, H\. Li, H\. Yu, Z\. Li, F\. Huang, and Y\. Li\(2023\)API\-Bank: A Comprehensive Benchmark for Tool\-Augmented LLMs\.InProc\. EMNLP,pp\. 3102–3116\.Cited by:[§II\-A](https://arxiv.org/html/2610.06914#S2.SS1.p2.1)\.
- \[10\]Y\. Ruan, H\. Dong, A\. Wang, S\. Pitis, Y\. Zhou, J\. Ba, Y\. Dubois, C\. J\. Maddison, and T\. Hashimoto\(2024\)Identifying the Risks of LM Agents with an LM\-Emulated Sandbox\.InProc\. ICLR,Cited by:[§II\-A](https://arxiv.org/html/2610.06914#S2.SS1.p2.1)\.
- \[11\]S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. Neubig\(2024\)WebArena: A Realistic Web Environment for Building Autonomous Agents\.InProc\. ICLR,Cited by:[§II\-A](https://arxiv.org/html/2610.06914#S2.SS1.p2.1)\.
- \[12\]S\. Yao, N\. Shinn, P\. Razavi, and K\. Narasimhan\(2025\)τ\\tau\-bench: A Benchmark for Tool\-Agent\-User Interaction in Real\-World Domains\.InProc\. ICLR,Cited by:[§II\-A](https://arxiv.org/html/2610.06914#S2.SS1.p2.1)\.
- \[13\]E\. Debenedetti, J\. Zhang, M\. Balunovic, L\. Beurer\-Kellner, M\. Fischer, and F\. Tramèr\(2024\)AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents\.InAdv\. Neural Inf\. Process\. Syst\. 37,Cited by:[§II\-A](https://arxiv.org/html/2610.06914#S2.SS1.p2.1)\.
- \[14\]Q\. Zhan, Z\. Liang, Z\. Ying, and D\. Kang\(2024\)InjecAgent: Benchmarking Indirect Prompt Injections in Tool\-Integrated Large Language Model Agents\.InFindings of ACL,pp\. 10471–10506\.Cited by:[§II\-A](https://arxiv.org/html/2610.06914#S2.SS1.p2.1)\.
- \[15\]Z\. Han, C\. Zeng, L\. Yang, Z\. Guo, M\. Zhou, and Y\. Li\(2026\)SkillTV\-Bench: Benchmarking How Well Judges Perform on Skill\-Augmented Agentic Execution\.Note:arXiv preprint arXiv:2608\.05573Cited by:[§II\-A](https://arxiv.org/html/2610.06914#S2.SS1.p2.1)\.
- \[16\]M\. Pourreza and D\. Rafiei\(2023\)DIN\-SQL: Decomposed In\-Context Learning of Text\-to\-SQL with Self\-Correction\.InAdv\. Neural Inf\. Process\. Syst\. 36,Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p1.1)\.
- \[17\]B\. Wang, R\. Shin, X\. Liu, O\. Polozov, and M\. Richardson\(2020\)RAT\-SQL: Relation\-Aware Schema Encoding and Linking for Text\-to\-SQL Parsers\.InProc\. ACL,pp\. 7567–7578\.Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p1.1)\.
- \[18\]W\. Lei, W\. Wang, Z\. Ma, T\. Gan, W\. Lu, M\. Kan, and T\. Chua\(2020\)Re\-examining the Role of Schema Linking in Text\-to\-SQL\.InProc\. EMNLP,pp\. 6943–6954\.Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p1.1)\.
- \[19\]H\. Li, J\. Zhang, C\. Li, and H\. Chen\(2023\)RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text\-to\-SQL\.Proc\. AAAI Conf\. Artif\. Intell\.37\(11\),pp\. 13067–13075\.Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p1.1)\.
- \[20\]T\. Scholak, N\. Schucher, and D\. Bahdanau\(2021\)PICARD: Parsing Incrementally for Constrained Auto\-Regressive Decoding from Language Models\.InProc\. EMNLP,pp\. 9895–9901\.Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p1.1)\.
- \[21\]S\. Chang, J\. Wang, M\. Dong, L\. Pan, H\. Zhu, A\. H\. Li, W\. Lan, S\. Zhang, J\. Jiang, J\. Lilien, S\. Ash, W\. Y\. Wang, Z\. Wang, V\. Castelli, P\. Ng, and B\. Xiang\(2023\)Dr\.Spider: A Diagnostic Evaluation Benchmark Towards Text\-to\-SQL Robustness\.InProc\. ICLR,Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p1.1)\.
- \[22\]M\. Vartak, S\. Rahman, S\. Madden, A\. Parameswaran, and N\. Polyzotis\(2015\)SeeDB: Efficient Data\-Driven Visualization Recommendations to Support Visual Analytics\.Proc\. VLDB Endow\.8\(13\),pp\. 2182–2193\.Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p3.1)\.
- \[23\]V\. Dibia and Çağatay Demiralp\(2019\)Data2Vis: Automatic Generation of Data Visualizations Using Sequence\-to\-Sequence Recurrent Neural Networks\.IEEE Comput\. Graph\. Appl\.39\(5\),pp\. 33–46\.Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p3.1)\.
- \[24\]A\. Satyanarayan, D\. Moritz, K\. Wongsuphasawat, and J\. Heer\(2017\)Vega\-Lite: A Grammar of Interactive Graphics\.IEEE Trans\. Vis\. Comput\. Graph\.23\(1\),pp\. 341–350\.Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p3.1)\.
- \[25\]Z\. Yang, Z\. Zhou, S\. Wang, X\. Cong, X\. Han, Y\. Yan, Z\. Liu, Z\. Tan, P\. Liu, D\. Yu, Z\. Liu, X\. Shi, and M\. Sun\(2024\)MatPlotAgent: Method and Evaluation for LLM\-Based Agentic Scientific Data Visualization\.InFindings of ACL,pp\. 11789–11804\.Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p3.1)\.
- \[26\]C\. Wang, J\. Thompson, and B\. Lee\(2024\)Data Formulator: AI\-Powered Concept\-Driven Visualization Authoring\.IEEE Trans\. Vis\. Comput\. Graph\.30\(1\),pp\. 1128–1138\.Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p3.1)\.
- \[27\]C\. Wang, B\. Lee, S\. Drucker, D\. Marshall, and J\. Gao\(2025\)Data Formulator 2: Iterative Creation of Data Visualizations, with AI Transforming Data Along the Way\.InProc\. CHI,Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p3.1)\.
- \[28\]P\. Vaithilingam, E\. L\. Glassman, J\. P\. Inala, and C\. Wang\(2024\)DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing\.InProc\. CHI,pp\. 1–17\.Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p3.1)\.
- \[29\]N\. Chen, Y\. Zhang, J\. Xu, K\. Ren, and Y\. Yang\(2025\)VisEval: A Benchmark for Data Visualization in the Era of Large Language Models\.IEEE Trans\. Vis\. Comput\. Graph\.31\(1\),pp\. 1301–1311\.Cited by:[§II\-B](https://arxiv.org/html/2610.06914#S2.SS2.p3.1)\.
- \[30\]S\. Amershi, D\. Weld, M\. Vorvoreanu, A\. Fourney, B\. Nushi, P\. Collisson, J\. Suh, S\. Iqbal, P\. N\. Bennett, K\. Inkpen, J\. Teevan, R\. Kikin\-Gil, and E\. Horvitz\(2019\)Guidelines for Human\-AI Interaction\.InProc\. CHI,pp\. 1–13\.Cited by:[§II\-C](https://arxiv.org/html/2610.06914#S2.SS3.p1.1)\.
- \[31\]T\. Wu, M\. Terry, and C\. J\. Cai\(2022\)AI Chains: Transparent and Controllable Human\-AI Interaction by Chaining Large Language Model Prompts\.InProc\. CHI,pp\. 1–22\.Cited by:[§II\-C](https://arxiv.org/html/2610.06914#S2.SS3.p1.1)\.
- \[32\]E\. Tabassi\(2023\)Artificial Intelligence Risk Management Framework \(AI RMF 1\.0\)\.Technical reportTechnical ReportNIST AI 100\-1,National Institute of Standards and Technology\.Cited by:[§II\-C](https://arxiv.org/html/2610.06914#S2.SS3.p1.1)\.
- \[33\]LangChain\(n\.d\.\)LangGraph Overview\.Note:Official LangGraph documentationAccessed 2 October 2026Cited by:[§II\-D](https://arxiv.org/html/2610.06914#S2.SS4.p1.1)\.
- \[34\]LangChain\(n\.d\.\)Interrupts\.Note:Official LangGraph documentationAccessed 2 October 2026Cited by:[§II\-D](https://arxiv.org/html/2610.06914#S2.SS4.p1.1)\.
- \[35\]OpenAI\(n\.d\.\)Authentication\.Note:Official OpenAI documentationAccessed 2 October 2026Cited by:[§III\-A4](https://arxiv.org/html/2610.06914#S3.SS1.SSS4.p1.1),[§III\-C2](https://arxiv.org/html/2610.06914#S3.SS3.SSS2.p1.1)\.
- \[36\]E\. B\. Wilson\(1927\)Probable Inference, the Law of Succession, and Statistical Inference\.J\. Amer\. Statist\. Assoc\.22\(158\),pp\. 209–212\.Cited by:[§III\-A6](https://arxiv.org/html/2610.06914#S3.SS1.SSS6.p3.1),[§IV\-A](https://arxiv.org/html/2610.06914#S4.SS1.p2.1)\.
- \[37\]OpenAI\(n\.d\.\)Developer Commands\.Note:Official OpenAI documentationAccessed 2 October 2026Cited by:[§III\-C2](https://arxiv.org/html/2610.06914#S3.SS3.SSS2.p1.1)\.
- \[38\]C\. Finegan\-Dollak, J\. K\. Kummerfeld, L\. Zhang, K\. Ramanathan, S\. Sadasivam, R\. Zhang, and D\. Radev\(2018\)Improving Text\-to\-SQL Evaluation Methodology\.InProc\. ACL,pp\. 351–360\.Cited by:[§V\-B](https://arxiv.org/html/2610.06914#S5.SS2.p3.1)\.