Tag
This comparative study evaluates encoder and decoder-based large language models for automatic speech recognition (ASR) assessment, finding that encoder metrics like BERTScore and SemDist are highly competitive, while generative models enhance hypothesis comparison and interpretability.
This paper presents a benchmark evaluating five commercial ASR systems on code-switching speech across Arabic-English, Persian-English, and German-English pairs, using a two-stage pipeline to select 300 samples per pair and assessing performance with WER and BERTScore. ElevenLabs Scribe v2 achieves the lowest overall WER (13.2%) and highest BERTScore (0.936), with public dataset available.