Earnings25: A Comprehensive 500-Hour Speech Benchmark for Finance

# Earnings25: A Comprehensive 500-Hour Speech Benchmark for Finance
Source: [https://arxiv.org/html/2607.23813](https://arxiv.org/html/2607.23813)
Jiang Zhou Wadhawan Fahy Ramesh Weisberg Derkachevskiy Sheehan Prasad Franceschini

HaoranAnshulBrendanVinayDavidDmitriyHelenSrivasMicheleBloomberg, United States[\{djiang108,hzhou245,awadhawan9,bfahy2,vramesh7,dweisberg6, dderkachevsk,hsheehan,sprasad60,mfrancesch10\}@bloomberg\.net](https://arxiv.org/html/2607.23813v1/mailto:%7Bdjiang108,hzhou245,awadhawan9,bfahy2,vramesh7,dweisberg6,%0Adderkachevsk,hsheehan,sprasad60,mfrancesch10%[email protected])

###### Abstract

We introduce Earnings25, a finance\-domain benchmark for evaluating automatic speech recognition \(ASR\) on English\-language earnings calls under realistic conditions\. Earnings25 comprises two complementary test sets: \(i\)testset\-full, 498 hours of full English\-language S&P 500 earnings calls from Q4 2025, and \(ii\)testset\-segmented, a 46\-hour industry\-balanced set of 290 segments sampled from English\-language U\.S\. earnings calls in 2025\. The benchmark provides aligned transcripts and structured metadata, including speaker roles, industry labels, and call structure, enabling speaker\- and industry\-aware evaluation beyond aggregate word error rate \(WER\)\. We report reproducible baselines for Whisper and Parakeet\-TDT using standardized scoring\.

###### keywords:

automatic speech recognition, ASR benchmark, financial speech, earnings calls, speech dataset, industry\-aware evaluation, long\-form speech, domain adaptation

## 1Introduction

### 1\.1Financial speech understanding is challenging

Earnings calls are a critical channel for corporate communication and pose unique challenges for automatic speech recognition \(ASR\)\. They combine spontaneous dialogue with scripted remarks, heavy financial jargon, company and product names, frequent numeric expressions, rapid turn\-taking, and overlapping speakers during Q&A—conditions that expose limitations of ASR systems trained primarily on read or general conversational speech\.

These challenges are amplified by domain shift\. Earnings calls often include operator boilerplate, variable recording quality due to telephony compression or background noise, and frequent speaker transitions\. Linguistically, the domain contains jargon and acronyms under\-represented in general ASR training data, as well as accented English from international executives\. Although self\-supervised and weakly supervised pretraining has improved general ASR performance \(e\.g\., wav2vec 2\.0\[baevski2020wav2vec2\]and Whisper\[radford2022whisper\]\), domain mismatch remains a major source of error, motivating the need for recent, domain\-specific speech resources\.

### 1\.2Toward a comprehensive benchmark for financial ASR

Despite growing interest in financial speech and NLP, few publicly available resources have been purpose\-built for*evaluating*ASR on earnings calls\. Existing datasets often emphasize training data without standardized evaluation protocols or lack the metadata required for detailed error analysis\. As a result, the field lacks a principled and reproducible benchmark for measuring progress on financial ASR\.

Beyond transcription accuracy, practical financial speech systems require evaluation of speaker attribution, diarization, role\-aware performance \(e\.g\., executives vs\. analysts\), and errors across industries and call structure\. A comprehensive benchmark must therefore pair realistic long\-form audio with rich metadata\.

Earnings25is designed to provide aligned transcripts together with speaker information, industry context, and structured metadata for fine\-grained evaluation\. Unlike prior resources that emphasize scale or aggregate metrics, it is explicitly constructed to reveal domain\-specific and long\-tail failure modes that are obscured by naturally distributed corpora\. By combining long\-form audio with industry\-balanced evaluation, the benchmark enables systematic study of industry\-aware ASR robustness and structured variation in financial speech\.

## 2Prior Work

### 2\.1Conversational and meeting speech benchmarks

Earnings calls share several conversational properties with meeting and telephone speech, including rapid turn\-taking, disfluencies, and speaker overlap\. Standard benchmarks such asSwitchboard\(1992\)\[godfrey1992switchboard\],CallHome\(1997\)\[canavan1997callhome\], andAMI\(2007\)\[carletta2007ami\]capture aspects of these phenomena, but generally lack the finance\-specific terminology, dense numeracy, and domain structure that significantly affect ASR performance on earnings calls\.

### 2\.2Existing financial speech benchmarks

Existing financial speech datasets fall short as comprehensive evaluation benchmarks in several key dimensions\.SPGISpeech, released in 2021 and expanded in 2025\[spgispeech,spgispeech2\], provides over 5,000 hours of professionally transcribed earnings\-call audio\. However, it was designed primarily as a*training corpus*: the dataset includes very large train, development, and test splits \(with approximately 2,000 hours in the test set\), making it computationally expensive for controlled and reproducible benchmark evaluation\.

TheEarnings\-21\(2021\) andEarnings\-22\(2022\) benchmarks\[delrio21\_interspeech,delrio2022earnings22\]focus on open evaluation with aligned transcripts and accent\- and country\-level metadata\. Earnings\-22 provides approximately 160 hours of long\-form global earnings calls, but lacks industry\-balanced sampling: high\-frequency sectors dominate evaluation metrics, obscuring performance on underrepresented domains\. Moreover, neither dataset provides the structured metadata—such as speaker roles, industry classification, or company identifiers—needed for fine\-grained error analysis and speaker\-aware evaluation\.

Overall, the financial speech domain lacks a comprehensive, balanced, and metadata\-rich benchmark designed specifically for evaluating ASR systems under realistic earnings\-call conditions\.

## 3Corpus Definition

Earnings25 comprises two complementary test sets designed to support both long\-form and segment\-level evaluation, as summarized in Table[1](https://arxiv.org/html/2607.23813#S3.T1)\.testset\-fullconsists of 498 hours of complete earnings\-call recordings from S&P 500 companies in 2025 Q4, preserving full conversational context with typical call durations of approximately one hour\.testset\-segmentedis a curated 46\-hour evaluation set sampled from over 2,000 U\.S\. earnings calls across 2025 \(Q1–Q4\), comprising 290 segments—one per industry—to ensure balanced domain coverage\. Segments intestset\-segmentedare 5–10 minutes long\.

Both test sets include aligned transcripts and structured metadata and are constructed to preserve realistic conversational flow, natural turn\-taking, and cross\-speaker dynamics\. In addition to transcription, Earnings25 provides rich metadata, including speaker segmentation, industry classification, and company identifiers, enabling fine\-grained analysis beyond aggregate word error rate \(WER\) and supporting speaker\- and structure\-aware evaluation\.

Table 1:Overview of the Earnings25 test sets\.
## 4Corpus Generation

We sample full earnings calls, apply CTC\-based forced alignment to obtain word\-level timestamps, and aggregate shorter speech segments into 5–10 minute evaluation units\. This section details the industry\-stratified sampling, alignment, and segmentation procedures\.

### 4\.1Industry\-stratified sampling of the full earnings\-call universe

To construct a representative and industry\-balanced evaluation set, we sample earnings calls from a pool of over 2,100 U\.S\. earnings calls spanning 2025 Q1–Q4\. Sampling is performed using a two\-stage, reproducible procedure designed to ensure broad industry coverage while preventing dominance by high\-frequency sectors\.

In the first stage, we filter the corpus to retain only U\.S\.\-domiciled companies within the target date range\. In the second stage, we apply*disproportionate stratified sampling*based on industry classification\. Calls are grouped by industry, and a fixed number of samples is drawn from each group, ensuring equal representation across industries regardless of their underlying frequency in the corpus\. When the number of industry groups exceeds the target evaluation size, a final random selection is applied\. All sampling steps are seeded for reproducibility\.

### 4\.2Alignment

We perform forced alignment to obtain word\-level timestamps, which enable the extraction of shorter segments from long earnings calls and thereby increase industry coverage\. These time spans are also used to support speaker diarization and speaker\-aware analysis\.

#### 4\.2\.1CTC\-based forced alignment

Forced alignment is performed using Connectionist Temporal Classification \(CTC\) models implemented in NVIDIA NeMo\[kuchaiev2019nemo\]\. CTC\[graves2006ctc\]defines a sequence\-level objective that marginalizes over all monotonic alignments between input acoustic frames and output label sequences, producing frame\-level posterior probabilities over output tokens and a blank symbol\. These posteriors are used to align reference transcripts to audio and recover word\-level timing information\.

Given a reference transcript tokenized into subword units, we compute frame\-level CTC log\-probabilities and apply Viterbi decoding to recover the most likely alignment path under the CTC lattice\. Token\-level timestamps are then aggregated into word boundaries\. To mitigate alignment errors caused by trailing silence or low\-confidence regions, we impose a maximum word\-duration constraint of 0\.5 seconds\.

The alignment pipeline consists of: \(1\) resampling audio to 16 kHz WAV format; \(2\) computing frame\-level CTC log\-probabilities; \(3\) Viterbi alignment between audio frames and reference tokens; and \(4\) word\-boundary extraction from aligned tokens\.

### 4\.3Segment extraction from full calls

After obtaining word\-level timestamps via forced alignment, we aggregate contiguous speech segments into non\-overlapping*blocks*with durations constrained to a predefined range \(e\.g\., 5–10 minutes\)\. Blocks are constructed greedily by accumulating consecutive segments until a minimum duration is reached and extending the block while the total duration remains below the maximum threshold\.

#### 4\.3\.1Quality filtering

We apply content\-based filtering to remove blocks whose merged transcripts contain undesired patterns such as operator boilerplate or non\-speech cues\. From the remaining candidates, one block is sampled uniformly at random per call to avoid over\-representation of individual calls\. Prior to audio extraction, a fixed padding of 0\.2 seconds is added to block boundaries to mitigate alignment jitter\. This procedure produces a compact and diverse set of segments with clean transcripts, balanced across calls and suitable for segment\-level ASR benchmarking\.

Speaker tags are used to further split the segments to ensure that segment boundaries do not bisect speaker turns\. This procedure yields transcripts with reliable timing suitable for segment\-level ASR evaluation, speaker\-aware analysis, and downstream speech processing tasks\.

## 5Corpus Analysis

### 5\.1Geographic coverage and accent distribution

testset\-fullincludes earnings calls from all S&P 500 companies and spans a broad geographic footprint, covering English earnings calls from 12 countries\. While U\.S\.\-domiciled companies dominate the corpus \(reflecting index composition\), the dataset also includes calls from companies headquartered in Europe, Asia, and other regions\. This geographic diversity introduces English accent variation and heterogeneous recording conditions that reflect real\-world earnings\-call audio\.

In contrast,testset\-segmentedrestricts evaluation to U\.S\.\-domiciled companies to reduce accent variability and provide a controlled evaluation setting\. This design allows baseline ASR performance to be assessed under consistent linguistic conditions, while accent\-robustness can be studied using the full\-call test set\.

Table 2:Geographic distribution oftestset\-full\.
### 5\.2Industry distribution

Industry coverage is a core design consideration ofEarnings25\.testset\-fullspans 284 distinct industry categories, reflecting the natural sector distribution of the S&P 500\. As summarized in Table[3](https://arxiv.org/html/2607.23813#S5.T3), utilities and financial services are among the most frequently represented sectors, while the majority of industries appear only a small number of times, preserving the long\-tail structure characteristic of financial data\.

Table 3:Industry distribution oftestset\-full\.To mitigate frequency bias in evaluation,testset\-segmentedis constructed via industry\-stratified sampling from over 2,000 U\.S\. earnings calls spanning 2025 \(Q1–Q4\)\. All 290 segments are drawn exclusively from U\.S\.\-domiciled companies to ensure consistent audio quality and reduce accent variability\. The segmented test set spans 290 unique industry categories, with exactly one segment per industry, ensuring broad coverage of domain\-specific vocabulary across diverse financial sub\-domains—from 3D Printers and Adult Nightclubs to Wind Turbines and Wireline Telecom Equipment\. This design enables fine\-grained, industry\-aware analysis of ASR performance while maintaining a controlled linguistic evaluation setting\.

### 5\.3Speaking\-style variation

Earnings calls exhibit substantial variation in speaking style and interaction structure\. Within a single call, speech alternates between scripted prepared remarks and spontaneous analyst Q&A, often with rapid turn\-taking and occasional overlap\.testset\-fullpreserves these dynamics across entire calls, whiletestset\-segmentedretains multi\-speaker conversational structure within each 5–10 minute segment\.

Speaker attribution metadata enables role\-aware analysis of ASR performance across operators, executives, and analysts\. Operators typically deliver formulaic announcements, executives present prepared remarks, and analysts contribute unscripted questions\. This variation allows evaluation of ASR robustness across speaking styles, roles, and interaction patterns common in financial speech\.

Table 4:Baseline ASR results on the full earnings\-call test set \(498h\) and the curated segmented test set \(46h\)\. Lower is better\.Table 5:Selected industry\-level ASR performance ofParakeet\-tdt\-0\.6b\-v2ontestset\-full\(2–3 calls per subsector based on industry tags\)\. This table is illustrative and not a comprehensive ranking\. Lower is better\.

## 6Transcription Experiments

### 6\.1Baseline Models and Evaluation Protocol

We benchmark representative ASR systems spanning sequence\-to\-sequence and transducer architectures: OpenAIWhispermodels\[radford2022whisper\]\(base, medium, large\-v2\) and NVIDIA NeMo’sParakeet\-TDT\-0\.6B\-v2\[xu2023efficient\]\. No external language model is used\. Whisper inference:Decoding options are loaded from a fixed model configuration\. We set a fixed random seed and use deterministic decoding settings\. The language is set to English\. No optional keyword prompting or boosting words are used during decoding\. Parakeet\-TDT inference:We use the pretrained checkpointnvidia/parakeet\-tdt\-0\.6b\-v2with greedy transducer decoding and no external language model\. Scoring and normalization:We report four consistent variants:WER\(raw\),WER\-N\(NeMo\-normalized\),WER\-nc\-np\(lowercased, punctuation removed\), andWER\-N\-nc\-np\(NeMo\-normalized, lowercased, punctuation removed\)\. For normalization\-based variants, the same NeMo English text normalization\[zhang2021nemo\]is applied to both reference and hypothesis\. Reproducibility:All sampling and segmentation procedures use a fixed random seed \(2025\)\. Experimental scripts setPYTHONHASHSEED=2025and framework RNG seeds, and log the full decoding configuration used for each run\.

### 6\.2Results

Table[4](https://arxiv.org/html/2607.23813#S5.T4)reports baseline performance ontestset\-full\(498 h\) and the industry\-balancedtestset\-segmented\(46 h\)\. Lower is better\. Across models, performance ontestset\-segmentedis slightly worse than ontestset\-full, despite shorter duration\. This reflects the effect of industry stratification: whiletestset\-fullfollows the natural frequency distribution of sectors, the segmented set enforces equal industry representation and increases exposure to long\-tail terminology\.

To illustrate domain variability, Table[5](https://arxiv.org/html/2607.23813#S5.T5)reports results for selected subsectors \(2–3 calls each\) using Parakeet\-tdt\-0\.6b\-v2\. This is an illustrative subset and not a comprehensive industry ranking\. Terminology\-dense domains such as biotech and pharma show substantially higher WER \(15\.3–15\.4%\) than the aggregate testset\-full WER \(10\.8%\)\. Because the corpus\-level metric is frequency\-weighted across industries, high\-volume and more repetitive sectors lower the overall average\. In contrast, the unweighted per\-industry results emphasize more challenging domains, demonstrating that aggregate WER can mask substantial domain\-dependent variation\.

## 7Limitations and Conclusion

Earnings25 provides value along three dimensions: \(1\)domain\-specific evaluation, offering a challenging benchmark based on S&P 500 earnings calls with broad industry coverage and finance\-specific terminology; \(2\)reproducible baselines, with standardized evaluations for contemporary ASR models, including Whisper and Parakeet\-TDT; and \(3\)rich metadata, including industry, call\-structure, and speaker annotations that enable stratified and speaker\-aware analysis\.

Regarding limitations, Earnings25 focuses on English\-language earnings calls and primarily reflects speech from U\.S\.\-domiciled companies\. While this design enables controlled and high\-quality evaluation, it does not capture the full linguistic diversity of global earnings calls\. Extending the benchmark to include multilingual earnings\-call data across a broader range of countries and languages remains an important direction for future work\.

## 8Data Access and Licensing

Earnings25 is released for research and benchmarking purposes\. We redistribute the audio recordings, transcripts, metadata, annotations, and evaluation splits through Zenodo:[https://doi\.org/10\.5281/zenodo\.18762168](https://doi.org/10.5281/zenodo.18762168)\. The transcripts, annotations, metadata, evaluation splits, and alignments are released under the Creative Commons Attribution 4\.0 International license \(CC BY 4\.0\)\. The redistributed audio recordings remain subject to any applicable terms of the original content providers, and users are responsible for ensuring compliance with those terms\.

## 9Generative AI Use Disclosure

During preparation of this manuscript, the authors used a generative AI assistant only for language editing and polishing \(e\.g\., grammar and phrasing\)\. The tool was not used to generate experimental results, analyses, or conclusions, and no generative AI system is listed as an author\.

## References