FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation

# Measuring and Improving Institution-Grade Financial Report Generation
Source: [https://arxiv.org/html/2608.04374](https://arxiv.org/html/2608.04374)
###### Abstract

Large language models can produce fluent financial analysis, but fluency does not establish whether a report is suitable for institutional delivery\. Existing evaluations use broad dimensions that do not identify the specific defects that cause professional rejection\. We introduceFinReportBench, an expert\-grounded benchmark for measuring and improving institution\-grade financial report generation\. In a pilot review, three experts assign all 75 reports from three models the lowest readiness score despite generally fluent financial prose; the recurring gaps concern report identity, institutional components, source discipline, and visual delivery\. Pilot interviews further identify a hierarchy of report criteria\. We then derive the rubric through two rounds: expert partial orders over a small output set guide item induction with multimodal evidence, and experts audit the resulting decision boundaries\. The final instrument contains 35 observable items across deliverability, report identity, and institutional completeness\. Starting from 10,000 balanced Chinese and English financial\-research source records, we curate 244 bilingual tasks across three research objects and two input tiers\. Each task separates the public query, reconstructed research trajectory, and hidden source packet\. Three independent judge families reproduce the expert partial order at near\-ceiling rates, showing that the rubric’s bounded, observable criteria support reliable evaluation\. Across nine model families, basic deliverability is nearly saturated, while report identity and institutional completeness remain the primary bottlenecks\. The largest cross\-model gaps arise in generation\-trace control, information density, and data discipline rather than basic report framing\. Finally, we use benchmark\-guided skill distillation to convert these recurrent failures into reusable generation and self\-review constraints\. Across five model families, the evolved skill improves mean G1 by 33\.85 points and mean G2 by 13\.83 points over the paired no\-skill condition, while preserving G0 for every pair\. Code and benchmark artifacts are available athttps://github\.com/MisterBrookT/finreportbench\.

## Introduction

Financial institutions communicate research through professional reports\. These reports combine analytical claims, data, charts, source attribution, institutional identity, and a stable document structure\(Jinet al\.[2026](https://arxiv.org/html/2608.04374#bib.bib84); Lifanet al\.[2026](https://arxiv.org/html/2608.04374#bib.bib85)\)\. They are reviewed, distributed, and archived as accountable research artifacts\. We refer to reports that satisfy this delivery standard as*institution\-grade financial reports*\.

Language models can produce fluent financial analysis, but their outputs often fail to convince professional readers\. In preliminary interviews with three financial experts, all three assigned a satisfaction score of 1, the lowest value on the five\-point scale, to all 75 reviewed model\-generated artifacts\. They found that an output could contain plausible or locally accurate statements while still falling far short of an acceptable research deliverable\. The largest deficits concerned professional report conventions and visual presentation, including front\-page framing, institutional identity, page systems, compliance components, and source presentation\. These shortcomings call for a systematic evaluation that can identify specific problems in generated financial reports\.

Recent work has begun to benchmark financial report generation, but its evaluation criteria remain broad and coarse\-grained\. FinSight evaluates factual accuracy, information effectiveness, and presentation quality, while Cogito evaluates data quality, analytical quality, and presentation quality\(Jinet al\.[2026](https://arxiv.org/html/2608.04374#bib.bib84); Lifanet al\.[2026](https://arxiv.org/html/2608.04374#bib.bib85)\)\. These dimensions support overall system comparison, but they do not identify specific problems that prevent institutional use\. For example, a webpage\-like report with polished charts may receive a high presentation score even when it lacks a publisher, analyst identity, and a stable page system\.

To fill this gap, we presentFinReportBench, the first benchmark designed to measure whether generated financial reports meet institutional delivery standards\. Building such a benchmark requires addressing three challenges\. First, converting expert knowledge into a reliable rubric is difficult\. Professionals rely on conventions learned through practice and can often express relative preferences more reliably than an exhaustive scoring rule\. Interviews and preference labels must therefore be turned into observable criteria, clear boundaries, and rules that another annotator can reproduce\. Full\-report review is also costly, which limits the amount of expert annotation available for rubric construction\. Second, a published report shows only the final artifact\. The original client request and the research steps that produced it are usually unavailable, so realistic benchmark inputs cannot be collected directly\. Third, a broad score can rank systems but does not tell a developer what to fix\. Actionable evaluation requires specific failure evidence that can guide improvement without exposing test answers or benchmark content\.

![Refer to caption](https://arxiv.org/html/2608.04374v1/x1.png)Figure 1:Metadata overview of FinReportBench\. \(a\) A two\-dimensional projection of the shared multilingual embedding space places the 244 selected tasks across the density of the 10,000\-task reference pool; color denotes research object and marker shape is a language diagnostic\. Coverage is computed in the original embedding space, where the suite covers 95\.65% of reference tasks at cosine similarity≥0\.60\\geq 0\.60\. \(b1\) Model inputs are substantially longer than query\-only financial\-report benchmarks; \(b2\) component profiles show the anatomy of the 244 scripted research trajectories\. \(c1\) Each task contains multiple retrieved sources and structured evidence facts\. \(c2\) Counts across the diagnostic domain taxonomy show the breadth of selected research topics\.We make five contributions:

- •Expert\-grounded gap diagnosis\.We provide an expert\-grounded diagnosis of why fluent model outputs remain unsuitable as institutional reports\. Reviews with three senior financial professionals identify the gap and reveal a hierarchy of professional requirements \(§ 2\)\.
- •Financial\-report generation dataset\.We introduce a 244\-task bilingual dataset curated from a balanced 10,000\-task reference space, covering three research objects, two input tiers, and 95\.65% of the multilingual semantic space\. Each task includes a tool\-use research trajectory with separated public queries, retrieved evidence, and hidden source packets \(§ 3\)\.
- •Hierarchical evaluation framework\.We propose a hierarchical framework derived from expert reviews and preference\-guided rubric induction\. It evaluates institutional readiness through 35 observable criteria and an evidence\-based multimodal judging protocol \(§ 4\)\.
- •Benchmark\-guided skill evolution\.We propose a method that converts recurrent item\-level failures across models into a compact, reusable production skill\. The method develops and selects skill revisions on external tasks before a locked cross\-model transfer test \(§ 5\)\.
- •Evaluation and findings\.In the completed evaluation, we characterize nine model families through item\-level analyses on FinReportBench\. The same task set is used for every model, and visible output defects remain part of the measured result\. We also test whether the evolved skill transfers across five model families \(§ 6\)\.

## Related Work and Pilot Interviews

### Related Work

Financial NLP benchmarks largely evaluate question answering and multimodal reasoning over tables, filings, and conversations\(Chenet al\.[2021](https://arxiv.org/html/2608.04374#bib.bib1),[2022](https://arxiv.org/html/2608.04374#bib.bib2); Zhuet al\.[2021](https://arxiv.org/html/2608.04374#bib.bib3); Zhaoet al\.[2022](https://arxiv.org/html/2608.04374#bib.bib4); Islamet al\.[2023](https://arxiv.org/html/2608.04374#bib.bib5); Reddyet al\.[2024](https://arxiv.org/html/2608.04374#bib.bib6); Liu and others[2025](https://arxiv.org/html/2608.04374#bib.bib7); Xie and others[2024](https://arxiv.org/html/2608.04374#bib.bib8); Xieet al\.[2023](https://arxiv.org/html/2608.04374#bib.bib9); Wuet al\.[2023](https://arxiv.org/html/2608.04374#bib.bib10)\), whereas agent benchmarks and training methods emphasize web interaction, retrieval, and multi\-step information seeking\(Mialonet al\.[2024](https://arxiv.org/html/2608.04374#bib.bib14); Zhouet al\.[2024](https://arxiv.org/html/2608.04374#bib.bib16); Wu and others[2025](https://arxiv.org/html/2608.04374#bib.bib21); Nakanoet al\.[2021](https://arxiv.org/html/2608.04374#bib.bib13); Jin and others[2025](https://arxiv.org/html/2608.04374#bib.bib20); Wei and others[2025](https://arxiv.org/html/2608.04374#bib.bib15); Denget al\.[2023](https://arxiv.org/html/2608.04374#bib.bib17); Gou and others[2025](https://arxiv.org/html/2608.04374#bib.bib18); Baeket al\.[2024](https://arxiv.org/html/2608.04374#bib.bib19); Liet al\.[2026](https://arxiv.org/html/2608.04374#bib.bib31); Bianet al\.[2025](https://arxiv.org/html/2608.04374#bib.bib33)\)\. Recent financial\-report systems add broad factual, analytical, and presentation dimensions\(Jinet al\.[2026](https://arxiv.org/html/2608.04374#bib.bib84); Lifanet al\.[2026](https://arxiv.org/html/2608.04374#bib.bib85); Wenget al\.[2025](https://arxiv.org/html/2608.04374#bib.bib22); Tanget al\.[2026b](https://arxiv.org/html/2608.04374#bib.bib23); Xieet al\.[2024](https://arxiv.org/html/2608.04374#bib.bib28),[2026](https://arxiv.org/html/2608.04374#bib.bib30); Tan and others[2024a](https://arxiv.org/html/2608.04374#bib.bib34); Wu and others[2024](https://arxiv.org/html/2608.04374#bib.bib35); Kim and others[2024](https://arxiv.org/html/2608.04374#bib.bib37); Asai and others[2024](https://arxiv.org/html/2608.04374#bib.bib39)\), but these scores do not identify the specific defects that cause professional rejection\. FinReportBench instead evaluates complete rendered reports through fine\-grained observable items organized by an expert\-derived hierarchy\.

Recent work also extracts reusable agent skills from resources or execution failures and selects revisions on held\-out tasks\(Li and others[2026](https://arxiv.org/html/2608.04374#bib.bib86); Huang and others[2026](https://arxiv.org/html/2608.04374#bib.bib87); Alzubi and others[2026](https://arxiv.org/html/2608.04374#bib.bib88); Shen and others[2026](https://arxiv.org/html/2608.04374#bib.bib89); Zhang and others[2026](https://arxiv.org/html/2608.04374#bib.bib90)\)\. We adapt this idea to institutional reporting: recurrent item\-level failures become compact production rules, and external validation guards against negative transfer\.

### Pilot Expert Interviews

We conducted multi\-round pilot interviews with three senior financial professionals, each with more than ten years of experience\. They reviewed 75 anonymized reports produced by three models for 25 queries and rated complete\-report readiness on a five\-point scale\.All 75 reports received the lowest score of 1\.The interviews yielded three design findings\. First, fluent outputs still lacked professional structure, institutional identity, source discipline, risk disclosure, and consistent visual delivery\(Qu and others[2025](https://arxiv.org/html/2608.04374#bib.bib63); Wenet al\.[2026](https://arxiv.org/html/2608.04374#bib.bib25); Tanget al\.[2026c](https://arxiv.org/html/2608.04374#bib.bib27)\)\. Second, review was hierarchical: experts checked deliverability, then report identity, then detailed institutional components\. Third, research preceded writing; analysts expected a model to continue from an organized evidence state rather than generate directly from a short query\.

These findings motivate, respectively, fine\-grained item evaluation, the G0–G1–G2 hierarchy, and the reconstructed research trajectory used by FinReportBench\. Interview questions, review instructions, and additional protocol details appear in the supplementary material\.

## FinReportBench Dataset

### Dataset Design Objectives

FinReportBench targets four properties:authenticity, by deriving tasks from real analyst or institutional sources;diversity, across languages, research objects, domains, and query specificity;efficiency, by selecting a compact suite that preserves the semantic coverage of a much larger pool; and arealistic generation pipeline, in which report writing continues from an evidence\-rich research trajectory rather than a query alone\.

### Construction Pipeline

Figure[2](https://arxiv.org/html/2608.04374#Sx3.F2)summarizes five stages: source collection, task synthesis, query validation, coverage\-oriented curation, and trajectory construction\.

![Refer to caption](https://arxiv.org/html/2608.04374v1/figures/fig_dataset_pipeline.png)Figure 2:The FinReportBench dataset construction pipeline\. We synthesize and validate source\-grounded tasks, curate the benchmark for broad coverage, and construct a research trajectory for each selected task\.#### Source\-grounded task synthesis\.

We assemble a balanced multilingual reference pool of 5,000 Chinese analyst reports from Eastmoney and 5,000 English institutional publications from the World Bank\. Because original client requests are rarely available, reverse synthesis recovers a plausible request from each source’s research object, context, and analytical angle\. An LLM\-based audit rejects unnatural requests, taxonomy errors, and publisher, source, or answer leakage\. Each accepted task separates a public query card from a hidden source mapping and construction metadata; the source artifact is never provided to the report model\.

#### Coverage\-oriented curation\.

We embed all source\-derived research tasks in one multilingual semantic space and select a compact suite while balancing language, research object, domain, and query specificity\. Language is a diagnostic slice, not a separate benchmark\. The resulting 244 tasks contain 122 Chinese and 122 English requests; 61 are company, 92 industry, and 91 macro tasks\. Eighty\-seven are open\-ended T0 requests and 157 are thesis\-guided T1 requests\. At cosine similarity 0\.60, the selected set covers 95\.65% of the 10,000\-task reference space, including 99\.46% of Chinese and 91\.84% of English references\.

#### Research trajectory construction\.

For every selected task, we reconstruct a scripted deep\-research trajectory containing the public request, search decisions, retrieved evidence, and a final textual research note\. The final user message asks for a self\-contained HTML report\. The trajectory does not replay the hidden source and excludes benchmark identifiers, internal evidence labels, and URL lists from the research note\. All 244 trajectories pass role\-order, provenance, final\-note, and leakage checks\.

The supplementary material provides source\-pool composition, semantic fingerprint fields, embedding and selection details, query\-audit criteria, the complete trajectory contract, and additional dataset statistics\.

## Evaluation Design

### Evaluation Design Objectives

Our evaluator targets three properties\.Low\-cost expert groundinguses sparse within\-query preferences instead of exhaustive item annotation\.Fine\-grained diagnosisreturns observable item decisions and supporting evidence rather than only an overall score\.Hierarchical validityfollows the professional review order found in the pilot study, preventing later strengths from fully compensating for an earlier\-stage failure\.

### Evaluation Construction Pipeline

Figure[3](https://arxiv.org/html/2608.04374#Sx4.F3)summarizes the process\. Experts first provide partial orders over small sets of anonymized reports generated for the same query\. A multimodal model contrasts preferred and dispreferred reports to mine candidate visual and textual evidence\(Tanget al\.[2026a](https://arxiv.org/html/2608.04374#bib.bib24); Panet al\.[2025](https://arxiv.org/html/2608.04374#bib.bib26); Masryet al\.[2022](https://arxiv.org/html/2608.04374#bib.bib40); Kantharajet al\.[2022](https://arxiv.org/html/2608.04374#bib.bib41); Methaniet al\.[2020](https://arxiv.org/html/2608.04374#bib.bib44); Masryet al\.[2023](https://arxiv.org/html/2608.04374#bib.bib46); Xieet al\.[2025](https://arxiv.org/html/2608.04374#bib.bib29); Chenet al\.[2025](https://arxiv.org/html/2608.04374#bib.bib32)\)\. Recurring, generalizable differences are converted into observable items withpass,partial,fail, and, where appropriate,not\-applicableboundaries\. Experts then remove, merge, split, or revise invalid and redundant items\. Rubric induction and final validation use disjoint report samples\.

![Refer to caption](https://arxiv.org/html/2608.04374v1/figures/fig_evaluation_design_v3.png)Figure 3:The FinReportBench evaluation pipeline\. Sparse expert preferences guide contrastive evidence mining and observable item induction\. Expert\-audited items form a hierarchical rubric that returns layer scores, item decisions, and supporting evidence from rendered pages and extracted text\.This two\-round process concentrates scarce expert effort on decision\-relevant contrasts: experts supply preference boundaries and audit induced items rather than label every report–criterion pair\. The supplementary material contains the annotation protocol, contrastive mining procedure, item\-induction prompts, and additional audit examples\.

### Hierarchical Rubric and Scoring

The frozen rubric contains 35 items in three ordered layers\.G0: deliverabilityuses four preflight checks for a visible, readable, continuous, and non\-broken artifact; any required failure sets the total to zero\.G1: report identityuses four items to test whether the artifact is recognizable as institutional research rather than a webpage, dashboard, slide deck, generic summary, or direct task answer\.G2: institutional completenessuses 27 items covering front\-page framing, institutional identity, compliance, page systems, information density, source and chart discipline, and generation\-artifact control\.

For active G1 and G2 items, pass receives 1, partial 0\.5, and fail 0; not applicable is removed from the denominator only when a criterion genuinely does not apply\. LetS1S\_\{1\}be the weighted G1 percentage, withg1,d1g\_\{1\},d\_\{1\}the weighted points obtained and available in G1, and defineg2,d2g\_\{2\},d\_\{2\}analogously for G2\. For a G0\-passing report,

S=100​g1\+\(S1/100\)​g2d1\+d2;S=0​if G0 fails\.S=100\\frac\{g\_\{1\}\+\(S\_\{1\}/100\)g\_\{2\}\}\{d\_\{1\}\+d\_\{2\}\};\\qquad S=0\\ \\text\{if G0 fails\}\.\(1\)Thus weak report identity discounts isolated G2 details without a discontinuous score cap\. We report G1 and G2 alongside this provisional total\. Complete item descriptors, weights, boundaries, and scoring examples appear in the supplementary material\.

### Itemized Multimodal Evaluation

We render each HTML report and extract its text\. The judge evaluates every applicable item using the appropriate visual, textual, or mixed evidence and returns an item decision with a short evidence citation\. This produces both comparable layer scores and an actionable failure profile for model or skill improvement\.

Held\-out validation compares automatic and expert within\-query partial orders over model outputs\. We also compare judge families while fixing reports, rubric, and scoring code\(Zhenget al\.[2023](https://arxiv.org/html/2608.04374#bib.bib57); Chiang and others[2024](https://arxiv.org/html/2608.04374#bib.bib60); Tan and others[2024b](https://arxiv.org/html/2608.04374#bib.bib61); Zhuge and others[2024](https://arxiv.org/html/2608.04374#bib.bib62); Zhenget al\.[2024](https://arxiv.org/html/2608.04374#bib.bib74)\)\. Section[Experiments](https://arxiv.org/html/2608.04374#Sx6)reports the main validation results; complete pair\-sampling instructions, prompts, and additional calibration matrices are supplied in the supplementary material\.

## Benchmark\-Grounded Skill Evolution

### Skill Design Objectives

Our method follows three objectives\.Diagnostic groundingrequires every revision to address recurring, observable item\-level failures rather than unconstrained self\-reflection\.Generalizable improvementfavors compact rules that transfer across queries and model families instead of fitting one output\.Leakage\-safe evolutionproposes and selects revisions only on external cases; the full FinReportBench test set is used after the skill is frozen\. Model weights remain unchanged throughout\.

### Evolution Pipeline

Figure[4](https://arxiv.org/html/2608.04374#Sx5.F4)shows the complete pipeline\. Starting from skillKtK\_\{t\}, several report models generate outputs for external financial\-report cases\. FinReportBench converts their rendered reports into item\-level decisions and observable evidence\. An optimizer turns recurring failures into a candidateKt\+1K\_\{t\+1\}, organized as planning, writing, and review rules\. Query\-disjoint external validation either accepts the candidate or retainsKtK\_\{t\}\. When improvement stops, the best accepted skill is frozen asK⋆K^\{\\star\}and evaluated once on a locked benchmark subset against a paired no\-skill condition\.

![Refer to caption](https://arxiv.org/html/2608.04374v1/figures/fig_skill_evolution_compact.png)Figure 4:Benchmark\-grounded skill evolution\. External cases support iterative diagnosis, revision, and validation; the selected skill is frozen before paired evaluation on FinReportBench\.The evolution and validation sets cover the same broad financial\-report setting but share no query, trajectory, or source document with the benchmark\. We use five external discovery cases and five disjoint external validation cases, balanced across Chinese and English and covering company, industry, and macro research\. Using multiple report models exposes failures that recur across model families while holding each model’s generation settings fixed\.

### Failure\-Grounded Skill Revision

For each external report, the evaluator returns a categorical decision and short visual or textual evidence for every applicable item\. We aggregate outcomes across cases and models and revise only failures with sufficient cross\-model support\. The optimizer receives item semantics, aggregate outcomes, and anonymized evidence, but not item identifiers, weights, judge prompts, benchmark queries, or hidden source packets\.

The optimizer contrasts failed and passed evidence and writes compact constraints with three parts: when a rule applies, what the generator should do, and how the output can be checked\. The resulting skill has*plan*,*write*, and*review*blocks\. A critic removes task\-specific facts, rubric wording, duplication, and conflicting instructions, and rejects rules that require unsupported analyst identities, qualifications, ratings, or target prices\. This keeps the skill actionable without copying the rubric into a long prompt\.

### External Validation and Freezing

Each candidate is tested on held\-out external cases\. It is accepted only when targeted items improve without reducing deliverability, report identity, institutional completeness, or materially regressing non\-target items; otherwise the previous skill remains active\. This failure\-driven update follows the general pattern ofAlzubi and others \([2026](https://arxiv.org/html/2608.04374#bib.bib88)\), but uses rendered financial reports and hierarchical item evidence as feedback\.

Evolution stops after a fixed round budget or a patience window without validation improvement\. The best accepted revision, rather than the last proposal, becomesK⋆K^\{\\star\}\. Only then does each report model receive the locked FinReportBench evaluation under paired no\-skill andK⋆K^\{\\star\}conditions, with query, trajectory, generation budget, and inference settings held fixed\. Test queries and their order are frozen before generation; selection does not use report or judge outcomes\. We measure changes in G0, G1, G2, item pass rates, the provisional total, non\-target regressions, generation cost, and skill length\.

Exact external\-case composition, support and acceptance thresholds, stopping parameters, skill budget, optimizer and critic prompts, and pseudocode are provided in the supplementary material\.

## Experiments

### Setup

The completed run evaluates nine recent model families from international and Chinese providers: DeepSeek V4 Flash, DeepSeek V4 Pro, Qwen 3\.7 Max, GLM\-5\.2, Kimi K2\.6, MiniMax M2\.5, and MiniMax M2\.7\. The run additionally includes MiniMax M3 and Qwen 3\.6 27B\. Every system is evaluated on FinReportBench in theno\_skillcondition\. We record model identifiers, run date, and inference settings in the supplementary material\. We render every output and evaluate extracted text together with up to four rendered pages using GPT\-5\.6 Luna at medium reasoning effort and the frozen 35\-item rubric\. All systems share the same judge prompt, rubric, and scoring code\. Tasks, trajectories, and the output contract are also fixed before the run\.

### Main Benchmark Results

Every system is evaluated on the same benchmark tasks, while visible generation and encoding defects remain system errors\. The provisional hierarchical\-score ordering is MiniMax M2\.7 \(22\.6\), GLM\-5\.2 \(21\.8\), MiniMax M2\.5 \(21\.1\), DeepSeek V4 Flash \(20\.2\), Qwen 3\.7 Max \(19\.1\), DeepSeek V4 Pro \(18\.8\), MiniMax M3 \(18\.6\), Kimi K2\.6 \(17\.7\), and Qwen 3\.6 27B \(15\.1\)\. MiniMax M2\.7 has the highest G1 score \(40\.0\) and G2 score \(46\.6\)\. Because the total remains provisional, we report G1 and G2 alongside it and do not interpret small score gaps as stable ability differences\. Table[1](https://arxiv.org/html/2608.04374#Sx6.T1)presents every rubric item and the provisional total\.

Table 1:Item\-level FinReportBench results across all 35 rubric items\. Every system is evaluated on the same benchmark tasks, and visible output defects remain included\. Cells report mean item credit \(pass=100, partial=50, fail=0\)\. Overall is the provisional hierarchical score\.G0: DeliverabilityG1: Report identity G2: Institutional completenessOverallSystemD01 D02 D03 D04 R01 R02 R03 R04 C01 C02 C03 C04 C05 C06 C07 C08 C09 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19 C20 C21 C22 C23 C24 C25 C26 C27 All MiniMax M2\.710099100100100151225610086519676500004706603181001009082100675085311822\.6GLM\-5\.2100991001001000512121009151985850000510770410100100958910051508823321\.8MiniMax M2\.51001001001001000491954998451882350000470670091001008577100455097464121\.1DeepSeek V4 Flash1001001001001000521078998350967250000500860011100100848410051509237520\.2Qwen 3\.7 Max10010099100990501219998750955352000490610113100100878310048497532119\.1DeepSeek V4 Pro1001001001001000489659982509611500004908300310010083889965506127618\.8MiniMax M3100100971001001501121008850962350000510520091001009288100525079421518\.6Kimi K2\.61001001001001000504321008952911150000480600031001007589100465095392517\.7Qwen 3\.6 27B949582100970471048987947874050000430470039791828695525080473915\.1

### Human–Evaluator Alignment

We conduct a calibration\-scale expert validation using relative judgments rather than exhaustive item\-level gold labels\. Three senior financial experts jointly review 18 anonymized reports from three queries and establish a consensus three\-tier partial order: three authentic source reports rank above two generated reports judged to have recognizable report form, which in turn rank above the remaining 13 generations\. Expanding only cross\-tier relations yields 71 directed pairwise constraints\. For each evaluator run, a constraint is satisfied when the hierarchical score of the expert\-preferred report is strictly higher\.

![Refer to caption](https://arxiv.org/html/2608.04374v1/x2.png)Figure 5:Alignment with the three\-expert consensus partial order\. GPT5\.6\-Luna and GPT5\.6\-Terra satisfy all 71 constraints, while Grok\-4\.5 satisfies 70\. The axis is truncated to distinguish the near\-ceiling results\.All three evaluator families preserve 98\.6–100\.0% of the expert partial\-order constraints\. A separate Terra repeat agrees with the reference run on 97\.0% of item decisions, while Luna and Grok agree with the Terra runs on 92\.1–93\.8%; corresponding score\-ranking correlations range from 0\.949 to 0\.981\. Together, the expert alignment and cross\-family agreement support the criterion validity and reliability of the automatic evaluator\.

### Hierarchical Diagnostic Profiles

The leaderboard establishes ranking; the hierarchical profile explains how systems differ\. We aggregate the 35 items into deliverability, report identity, front\-page framing, institutional identity, compliance, page systems, source and chart discipline, and generation\-trace control\. Figure[6](https://arxiv.org/html/2608.04374#Sx6.F6)shows five representative models in the main paper; profiles for all models will appear in the supplementary material\.

![Refer to caption](https://arxiv.org/html/2608.04374v1/x3.png)Figure 6:Hierarchical component profiles for five representative models\. The radial axes aggregate related rubric items while preserving the G0–G1–G2 interpretation\.Cross\-model gaps are largest for generation\-trace control \(31\.1 points\), followed by information density \(13\.4\) and data discipline \(13\.2\)\. The report\-frame gap is only 4\.6 points\.

### Performance Across Benchmark Slices

We next test whether conclusions depend on one language or research setting\. We report Chinese versus English, company versus industry versus macro, and open\-ended T0 versus thesis\-guided T1 requests\. Figure[7](https://arxiv.org/html/2608.04374#Sx6.F7)displays each slice as a deviation from the same model’s overall score, preventing globally strong systems from dominating the visual scale\. Absolute scores, sample sizes, and confidence intervals remain available in the supplementary tables\.

![Refer to caption](https://arxiv.org/html/2608.04374v1/x4.png)Figure 7:Performance variation across language, research object, and query\-specificity slices\. Cells show deviation from each model’s overall score\.We use these slices diagnostically rather than define separate benchmarks\. The largest model\-relative slice deviation is 4\.1 points\. Rankings are highly stable across T0 and T1 requests \(ρ=0\.983\\rho=0\.983and1\.0001\.000\) and remain similar across company, industry, and macro tasks \(ρ=0\.833\\rho=0\.833,0\.8330\.833, and0\.9000\.900\)\. Language is the largest source of rank variation \(ρ=0\.650\\rho=0\.650for Chinese and0\.9170\.917for English\)\.

### Item Difficulty and Discrimination

A useful benchmark should contain neither only trivial checks nor only unattainable requirements\. We therefore measure each item’s pass rate and model\-discrimination strength, then cluster item failures by their co\-occurrence across reports\. Figure[8](https://arxiv.org/html/2608.04374#Sx6.F8)separates item difficulty from discrimination and reveals whether recurring failures form coherent institutional components\.

![Refer to caption](https://arxiv.org/html/2608.04374v1/x5.png)Figure 8:Item analysis\. Left: pass rate and model discrimination for all 35 items\. Right: failure co\-occurrence aggregated by rubric component\.Mean item credit spans 0\.0–99\.9%, confirming that the rubric contains both near\-universal delivery checks and consistently absent institutional requirements\. The maximum between\-model standard deviation is 25\.9 points\. Table[1](https://arxiv.org/html/2608.04374#Sx6.T1)provides the complete auditable item matrix\.

### Skill\-Evolution Evaluation

Finally, we test whether benchmark diagnostics can improve generation\. Five external discovery cases produce the initial skillK0K\_\{0\}and candidate revisions; five disjoint external validation cases determine which candidates are retained\. AfterK⋆K^\{\\star\}is frozen, a locked FinReportBench evaluation compares no skill andK⋆K^\{\\star\}across Qwen 3\.7 Max, DeepSeek V4 Pro, DeepSeek V4 Flash, MiniMax M3, and GLM\-5\.2 under 100 paired task–model comparisons\.

![Refer to caption](https://arxiv.org/html/2608.04374v1/x6.png)Figure 9:Validation performance across skill\-evolution stages for five report models\. Filled markers are retained versions and open markers are rejected candidates\.On the locked FinReportBench evaluation,K⋆K^\{\\star\}improves mean G1 by 33\.85 points \(95% CI: 31\.17–36\.63\) and mean G2 by 13\.83 points \(12\.65–15\.02\), while preserving G0 for every pair\. All five models improve: G1 gains range from 26\.39 to 41\.86 points and G2 gains from 11\.22 to 16\.98 points\. The larger G1 gain shows that the skill primarily repairs report identity and document structure, while the consistent G2 gain shows that it also strengthens institutional completeness\. The shared direction across model families demonstrates transfer beyond a single generator\.

## Discussion and Conclusion

FinReportBench connects measurement, diagnosis, and improvement for institution\-grade financial report generation\. Its results show that professional quality is a staged decision: an artifact must first be deliverable, then recognizable as the target document genre, and finally complete in its institutional components\. This hierarchy explains why fluent outputs can remain unusable in practice, while the itemized rubric turns recurrent failures into concrete targets for skill distillation\. The locked evaluation further shows that these diagnostics can be converted into reusable constraints that improve report identity and institutional completeness across model families\.

## References

- A\. Alzubiet al\.\(2026\)EvoSkill: automated skill discovery for multi\-agent systems\.arXiv preprint arXiv:2603\.02766\.External Links:[Link](https://arxiv.org/abs/2603.02766)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p2.1),[External Validation and Freezing](https://arxiv.org/html/2608.04374#Sx5.SSx4.p1.1)\.
- A\. Asaiet al\.\(2024\)OpenScholar: synthesizing scientific literature with retrieval\-augmented language models\.arXiv preprint arXiv:2411\.14199\.External Links:[Link](https://arxiv.org/abs/2411.14199)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- J\. Baek, S\. K\. Jauhar, S\. Cucerzan, and S\. J\. Hwang \(2024\)ResearchAgent: iterative research idea generation over scientific literature with large language models\.arXiv preprint arXiv:2404\.07738\.External Links:[Link](https://arxiv.org/abs/2404.07738)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Y\. Bian, X\. Lin, Y\. Xie, T\. Liu, M\. Zhuge, S\. Lu, H\. Tang, J\. Wang, J\. Zhang, J\. Chen,et al\.\(2025\)You don’t know until you click: automated GUI testing for production\-ready software evaluation\.arXiv preprint arXiv:2508\.14104\.Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Y\. Chen, Y\. Wu, S\. Shen, Y\. Xie, L\. Shen, H\. Xiong, and Y\. Luo \(2025\)ChartMark: a structured grammar for chart annotation\.In2025 IEEE Visualization and Visual Analytics \(VIS\),pp\. 311–315\.Cited by:[Evaluation Construction Pipeline](https://arxiv.org/html/2608.04374#Sx4.SSx2.p1.1)\.
- Z\. Chen, W\. Chen, C\. Smiley, S\. Shah, I\. Borova, D\. Langdon, R\. Moussa, M\. Beane, T\. Huang, B\. Routledge, and W\. Y\. Wang \(2021\)FinQA: a dataset of numerical reasoning over financial data\.InProceedings of EMNLP,External Links:[Link](https://arxiv.org/abs/2109.00122)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Z\. Chen, S\. Li, C\. Smiley, Z\. Ma, S\. Shah, and W\. Y\. Wang \(2022\)ConvFinQA: exploring the chain of numerical reasoning in conversational finance question answering\.InProceedings of EMNLP,External Links:[Link](https://arxiv.org/abs/2210.03849)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- W\. Chianget al\.\(2024\)Chatbot arena: an open platform for evaluating llms by human preference\.InProceedings of ICML,External Links:[Link](https://arxiv.org/abs/2403.04132)Cited by:[Itemized Multimodal Evaluation](https://arxiv.org/html/2608.04374#Sx4.SSx4.p2.1)\.
- X\. Deng, Y\. Gu, B\. Zheng, S\. Chen, S\. Stevens, B\. Wang, H\. Sun, and Y\. Su \(2023\)Mind2Web: towards a generalist agent for the web\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://arxiv.org/abs/2306.06070)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- B\. Gouet al\.\(2025\)Mind2Web 2: evaluating agentic search with agent\-as\-a\-judge\.arXiv preprint arXiv:2506\.21506\.External Links:[Link](https://arxiv.org/abs/2506.21506)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Z\. Huanget al\.\(2026\)From raw experience to skill consumption: a systematic study of model\-generated agent skills\.arXiv preprint arXiv:2605\.23899\.External Links:[Link](https://arxiv.org/abs/2605.23899)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p2.1)\.
- P\. Islam, A\. Kannappan, D\. Kiela, R\. Qian, N\. Scherrer, and B\. Vidgen \(2023\)FinanceBench: a new benchmark for financial question answering\.arXiv preprint arXiv:2311\.11944\.External Links:[Link](https://arxiv.org/abs/2311.11944)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- B\. Jinet al\.\(2025\)Search\-r1: training llms to reason and leverage search engines with reinforcement learning\.arXiv preprint arXiv:2503\.09516\.External Links:[Link](https://arxiv.org/abs/2503.09516)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- J\. Jin, Y\. Zhang, Y\. Xu, H\. Qian, Y\. Zhu, and Z\. Dou \(2026\)FinSight: towards real\-world financial deep research\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),External Links:[Link](https://aclanthology.org/2026.acl-long.265/)Cited by:[Introduction](https://arxiv.org/html/2608.04374#Sx1.p1.1),[Introduction](https://arxiv.org/html/2608.04374#Sx1.p3.1),[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- S\. Kantharaj, X\. L\. Do, R\. T\. K\. Leong, J\. Q\. Tan, E\. Hoque, and S\. Joty \(2022\)OpenCQA: open\-ended question answering with charts\.InProceedings of EMNLP,External Links:[Link](https://arxiv.org/abs/2210.06628)Cited by:[Evaluation Construction Pipeline](https://arxiv.org/html/2608.04374#Sx4.SSx2.p1.1)\.
- S\. Kimet al\.\(2024\)The biggen bench: a principled benchmark for fine\-grained evaluation of language models with language models\.arXiv preprint arXiv:2406\.05761\.External Links:[Link](https://arxiv.org/abs/2406.05761)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- B\. Li, Y\. Peng, Y\. Xie, S\. Lu, Y\. Zhu, X\. Mu, X\. Liu, and Y\. Luo \(2026\)DeepEye: a steerable self\-driving data agent system\.InCompanion of the International Conference on Management of Data,pp\. 74–77\.Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- X\. Liet al\.\(2026\)SkillsBench: benchmarking how well agent skills work across diverse tasks\.arXiv preprint arXiv:2602\.12670\.External Links:[Link](https://arxiv.org/abs/2602.12670)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p2.1)\.
- C\. Lifan, W\. Ding, J\. Yang, X\. Zhou, J\. Chen, and F\. Lin \(2026\)Cogito: a cognitive agentic framework driven by dynamic graph of thoughts for financial report generation\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 8846–8860\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.430),[Link](https://aclanthology.org/2026.findings-acl.430/)Cited by:[Introduction](https://arxiv.org/html/2608.04374#Sx1.p1.1),[Introduction](https://arxiv.org/html/2608.04374#Sx1.p3.1),[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- J\. Liuet al\.\(2025\)FinMME: benchmark dataset for financial multi\-modal reasoning evaluation\.arXiv preprint arXiv:2505\.24714\.External Links:[Link](https://arxiv.org/abs/2505.24714)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- A\. Masry, P\. Kavehzadeh, X\. L\. Do, E\. Hoque, and S\. Joty \(2023\)UniChart: a universal vision\-language pretrained model for chart comprehension and reasoning\.arXiv preprint arXiv:2305\.14761\.External Links:[Link](https://arxiv.org/abs/2305.14761)Cited by:[Evaluation Construction Pipeline](https://arxiv.org/html/2608.04374#Sx4.SSx2.p1.1)\.
- A\. Masry, D\. X\. Long, J\. Q\. Tan, S\. Joty, and E\. Hoque \(2022\)ChartQA: a benchmark for question answering about charts with visual and logical reasoning\.InFindings of ACL,External Links:[Link](https://aclanthology.org/2022.findings-acl.177/)Cited by:[Evaluation Construction Pipeline](https://arxiv.org/html/2608.04374#Sx4.SSx2.p1.1)\.
- N\. Methani, P\. Ganguly, M\. M\. Khapra, and P\. Kumar \(2020\)PlotQA: reasoning over scientific plots\.InProceedings of WACV,External Links:[Link](https://arxiv.org/abs/1909.00997)Cited by:[Evaluation Construction Pipeline](https://arxiv.org/html/2608.04374#Sx4.SSx2.p1.1)\.
- G\. Mialon, C\. Fourrier, C\. Swift, T\. Wolf, Y\. LeCun, and T\. Scialom \(2024\)GAIA: a benchmark for general ai assistants\.InProceedings of ICLR,External Links:[Link](https://arxiv.org/abs/2311.12983)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- R\. Nakano, J\. Hilton, S\. Balaji, J\. Wu, L\. Ouyang, C\. Kim, C\. Hesse, S\. Jain, V\. Kosaraju, W\. Saunders,et al\.\(2021\)WebGPT: browser\-assisted question\-answering with human feedback\.arXiv preprint arXiv:2112\.09332\.External Links:[Link](https://arxiv.org/abs/2112.09332)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- B\. Pan, Y\. Fu, K\. Wang, J\. Lu, L\. Pan, Z\. Qian, Y\. Chen, G\. Wang, Y\. Zhou, L\. Zheng, Y\. Tang, Z\. Wen, Y\. Wu, J\. Lu, B\. Zhu, M\. Zhu, B\. Zhang, and W\. Chen \(2025\)VIS\-Shepherd: constructing critic for LLM\-based data visualization generation\.arXiv preprint arXiv:2506\.13326\.External Links:[Link](https://arxiv.org/abs/2506.13326)Cited by:[Evaluation Construction Pipeline](https://arxiv.org/html/2608.04374#Sx4.SSx2.p1.1)\.
- Y\. Quet al\.\(2025\)FinFlier: automating graphical overlays for financial visualizations with knowledge\-grounding large language models\.IEEE Transactions on Visualization and Computer Graphics \(TVCG\)\.External Links:[Link](https://arxiv.org/abs/2412.06821)Cited by:[Pilot Expert Interviews](https://arxiv.org/html/2608.04374#Sx2.SSx2.p1.1)\.
- V\. Reddy, R\. Koncel\-Kedziorski, V\. D\. Lai, M\. Krumdick, C\. Lovering, and C\. Tanner \(2024\)DocFinQA: a long\-context financial reasoning dataset\.InProceedings of ACL,External Links:[Link](https://arxiv.org/abs/2401.06915)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- S\. Shenet al\.\(2026\)SKILLFOUNDRY: building self\-evolving agent skill libraries from heterogeneous scientific resources\.arXiv preprint arXiv:2604\.03964\.External Links:[Link](https://arxiv.org/abs/2604.03964)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p2.1)\.
- H\. Tanet al\.\(2024a\)PROXYQA: an alternative framework for evaluating long\-form text generation\.InProceedings of ACL,External Links:[Link](https://arxiv.org/abs/2401.15042)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- S\. Tanet al\.\(2024b\)JudgeBench: a benchmark for evaluating llm\-based judges\.arXiv preprint arXiv:2410\.12784\.External Links:[Link](https://arxiv.org/abs/2410.12784)Cited by:[Itemized Multimodal Evaluation](https://arxiv.org/html/2608.04374#Sx4.SSx4.p2.1)\.
- Y\. Tang, X\. Liu, B\. Zhang, T\. Lan, Y\. Xie, J\. Lao, Y\. Wang, H\. Li, T\. Gao, B\. Pan, L\. Weng, X\. Huang, M\. Zhu, Y\. Feng, Y\. Luo, and W\. Chen \(2026a\)IGenBench: benchmarking the reliability of text\-to\-infographic generation\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics,External Links:[Link](https://arxiv.org/abs/2601.04498)Cited by:[Evaluation Construction Pipeline](https://arxiv.org/html/2608.04374#Sx4.SSx2.p1.1)\.
- Y\. Tang, Y\. Xie, Y\. Feng, T\. Lan, J\. Lao, Y\. Cheng, and W\. Chen \(2026b\)ViviDoc: generating interactive documents through human\-agent collaboration\.arXiv preprint arXiv:2603\.27991\.External Links:[Link](https://arxiv.org/abs/2603.27991)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Y\. Tang, Y\. Xie, Y\. Feng, J\. Lao, T\. Lan, and W\. Chen \(2026c\)Demonstrating chart\-plot: closing the last mile of academic chart generation\.arXiv preprint arXiv:2606\.09174\.External Links:[Link](https://arxiv.org/abs/2606.09174)Cited by:[Pilot Expert Interviews](https://arxiv.org/html/2608.04374#Sx2.SSx2.p1.1)\.
- J\. Weiet al\.\(2025\)BrowseComp: a simple yet challenging benchmark for browsing agents\.arXiv preprint arXiv:2504\.12516\.External Links:[Link](https://arxiv.org/abs/2504.12516)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Z\. Wen, L\. Weng, Y\. Tang, R\. Zhang, Y\. Liu, B\. Pan, M\. Zhu, and W\. Chen \(2026\)Exploring multimodal prompt for visualization authoring with large language models\.IEEE Transactions on Visualization and Computer Graphics\.External Links:[Document](https://dx.doi.org/10.1109/TVCG.2026.3701510),[Link](https://arxiv.org/abs/2504.13700)Cited by:[Pilot Expert Interviews](https://arxiv.org/html/2608.04374#Sx2.SSx2.p1.1)\.
- L\. Weng, Y\. Tang, Y\. Feng, Z\. Chang, R\. Chen, H\. Feng, C\. Hou, D\. Huang, Y\. Li, H\. Rao, H\. Wang, C\. Wei, X\. Yang, Y\. Zhang, Y\. Zheng, X\. Huang, M\. Zhu, Y\. Ma, B\. Cui, P\. Chen, and W\. Chen \(2025\)DataLab: a unified platform for LLM\-powered business intelligence\.InProceedings of the IEEE International Conference on Data Engineering,pp\. 4346–4359\.External Links:[Link](https://arxiv.org/abs/2412.02205)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- J\. Wuet al\.\(2025\)WebDancer: towards autonomous information seeking agency\.arXiv preprint arXiv:2505\.22648\.External Links:[Link](https://arxiv.org/abs/2505.22648)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- S\. Wu, O\. Irsoy, S\. Lu, V\. Dabravolski, M\. Dredze, S\. Gehrmann, P\. Kambadur, D\. Rosenberg, and G\. Mann \(2023\)BloombergGPT: a large language model for finance\.arXiv preprint arXiv:2303\.17564\.External Links:[Link](https://arxiv.org/abs/2303.17564)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Y\. Wuet al\.\(2024\)LongGenBench: benchmarking long\-form generation in long\-context llms\.arXiv preprint arXiv:2409\.02076\.External Links:[Link](https://arxiv.org/abs/2409.02076)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Q\. Xie, W\. Han, X\. Zhang, Y\. Lai, M\. Peng, A\. Lopez\-Lira, and J\. Huang \(2023\)PIXIU: a large language model, instruction data and evaluation benchmark for finance\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://arxiv.org/abs/2306.05443)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Q\. Xieet al\.\(2024\)FinBen: a holistic financial benchmark for large language models\.arXiv preprint arXiv:2402\.12659\.External Links:[Link](https://arxiv.org/abs/2402.12659)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Y\. Xie, Y\. Luo, G\. Li, and N\. Tang \(2024\)HaiChart: human and AI paired visualization system\.arXiv preprint arXiv:2406\.11033\.Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Y\. Xie, C\. Ma, Z\. Wang, L\. Wang, J\. Zhu, C\. Zeng, Z\. Shen, B\. Li, and Y\. Luo \(2026\)DataMagic: transforming tabular data into data insight video\.arXiv preprint arXiv:2606\.20388\.Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- Y\. Xie, Z\. Zhang, Y\. Wu, S\. Lu, J\. Zhang, Z\. Yu, J\. Wang, S\. Hong, B\. Liu, C\. Wu,et al\.\(2025\)VisJudge\-Bench: aesthetics and quality assessment of visualizations\.arXiv preprint arXiv:2510\.22373\.Cited by:[Evaluation Construction Pipeline](https://arxiv.org/html/2608.04374#Sx4.SSx2.p1.1)\.
- H\. Zhanget al\.\(2026\)CoEvoSkills: self\-evolving agent skills via co\-evolutionary verification\.arXiv preprint arXiv:2604\.01687\.External Links:[Link](https://arxiv.org/abs/2604.01687)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p2.1)\.
- Y\. Zhao, Y\. Li, C\. Li, and R\. Zhang \(2022\)MultiHiertt: numerical reasoning over multi hierarchical tabular and textual data\.InProceedings of ACL,External Links:[Link](https://arxiv.org/abs/2206.01347)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing,et al\.\(2023\)Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://arxiv.org/abs/2306.05685)Cited by:[Itemized Multimodal Evaluation](https://arxiv.org/html/2608.04374#Sx4.SSx4.p2.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2024\)Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://arxiv.org/abs/2306.05685)Cited by:[Itemized Multimodal Evaluation](https://arxiv.org/html/2608.04374#Sx4.SSx4.p2.1)\.
- S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried,et al\.\(2024\)WebArena: a realistic web environment for building autonomous agents\.InProceedings of ICLR,External Links:[Link](https://arxiv.org/abs/2307.13854)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- F\. Zhu, W\. Lei, Y\. Huang, C\. Wang, S\. Zhang, J\. Lv, F\. Feng, and T\. Chua \(2021\)TAT\-qa: a question answering benchmark on a hybrid of tabular and textual content in finance\.InProceedings of ACL,External Links:[Link](https://arxiv.org/abs/2105.07624)Cited by:[Related Work](https://arxiv.org/html/2608.04374#Sx2.SSx1.p1.1)\.
- M\. Zhugeet al\.\(2024\)Agent\-as\-a\-judge: evaluate agents with agents\.arXiv preprint arXiv:2410\.10934\.External Links:[Link](https://arxiv.org/abs/2410.10934)Cited by:[Itemized Multimodal Evaluation](https://arxiv.org/html/2608.04374#Sx4.SSx4.p2.1)\.