Extremely Fast and Accurate Transcription with Granite Speech 5.0 Turbo CTC

Source: https://huggingface.co/blog/ibm-granite/granite-speech-5-0-470m-turboctc

https://huggingface.co/blog/ibm-granite/granite-speech-5-0-470m-turboctc#tldrTL;DR

Today we’re delighted to announce the release of two new models in theGranite Speech family: compact, 470M-parameter English speech recognition models that pair strong accuracy with unprecedented speed — over 12,600 RTFx on an NVIDIA H200 GPU, meaning they can transcribe more than 3.5 hours of speech in one second using batched inference.

1. granite-speech-5.0-470m-turboctc
2. granite-speech-5.0-470m-turboctc-nc

To get a sense of the models’ responsiveness, check out ourWebGPU demoof streaming speech recognition.Note that the demo only runs on Chrome or Edge browsers.

The two models differ mainly in training data and licensing:granite\-speech\-5\.0\-470m\-turboctc\-nctrains on additional data and carries aCC-BY-NC-SA-4.0license, whilegranite\-speech\-5\.0\-470m\-turboctctrains on a smaller set and isApache 2.0licensed.

https://huggingface.co/blog/ibm-granite/granite-speech-5-0-470m-turboctc#performancePerformance

We report official results on thepublic, English short-form test sets from theOpenASR Leaderboardbelow. Please see the leaderboard for the results that include the private datasets.

We provide bar charts showing the performance ofibm\-granite/granite\-speech\-5\.0\-470m\-turboctc\-ncandibm\-granite/granite\-speech\-5\.0\-470m\-turboctcas well as Pareto plots for the speed vs. accuracy trade-off and the model size vs. accuracy trade-off. Both models offer high accuracy, with the noncommercial model scoring an aggregate 4.85% WER and the Apache 2.0 model scoring 5.00% WER, and unprecedented aggregate throughput in excess of 12,600 RTFx. The noncommercial model is slightly more accurate than the Apache 2.0 model on most test sets, with a more noticeable advantage on SPGI Speech and a noticeable disadvantage on the new chunked Earnings22 test.

***Figure 1:*Results for the public test sets used in the OpenASR leaderboard as of 25 August 2026.

***Figure 2:*Pareto plots (speed vs. accuracy and model size vs. accuracy) based on results for the OpenASR leaderboard public test sets as of 25 August 2026.

Both models also perform well on far-field speech recognition as measured by theFFASR Leaderboard. As of 25 August 2026,ibm\-granite/granite\-speech\-5\.0\-470m\-turboctcranked ninth in accuracy andibm\-granite/granite\-speech\-5\.0\-470m\-turboctc\-ncranked fifth while also being the fastest two models. Note that these are offical results.

https://huggingface.co/blog/ibm-granite/granite-speech-5-0-470m-turboctc#model-architectureModel Architecture

These new models are encoder-only models, unlikeprior Granite Speech modelswhich comprise an acoustic encoder, projector, and Granite LM with LoRA adapters. The encoder-only design provides strong transcription performance, a small memory footprint of only 470M parameters, and over 20x faster throughput than previous Granite Speech models. The new models give up some of the capabilities of the LM-equipped models, such as speech translation and keyword biasing, but they are ideal for speech-to-text tasks on edge devices.

The Granite 5.0 Speech models have many things in common with the encoders used in previous Granite Speech models. They all

• comprise a stack of 16Conformerblocks,
• useself-conditioningat the output of the 8th block,
• employchunkwiseattention to avoid the quadratic scaling with sequence length of standard dot product attention, and
• optimize the connectionist temporal classification(CTC)loss during training.

What’s new about the Granite 5.0 Speech models is that they operate at a much lower token rate than our previous encoders and use a different output tokenization. While our previous encoders generated 50 characters per second, the Granite 5.0 models generate 12.5 tokens per second, withibm\-granite/granite\-speech\-5\.0\-470m\-turboctc\-ncusing a SentencePiece tokenization andibm\-granite/granite\-speech\-5\.0\-470m\-turboctcusing a BPE tokenization. In both cases, the tokenizers are trained on speech transcripts.

To get from the 100 frames per second rate of the log Mel spectrogram front end down to a rate of 12.5 tokens per second, we rely on three stages of 2x subsampling, as shown below. The first stage, which is also used in the encoders of previous Granite Speech models, stacks successive log Mel feature vectors together via a simplereshape\(\)operation. The second and third stages are built into the first two Conformer blocks in the stack, which use strided convolutions to perform the temporal downsampling.

***Figure 3:*Placement of the subsampling operations in the stack of 16 Conformer blocks making up the encoder. Note that the self-conditioning is omitted in this figure for clarity.

The figure below illustrates the implementation of the Conformer block that performs temporal subsampling in comparison to a standard Conformer block. The two changes are that (1) the convolutional block performs temporal convolutions withstride= 2and (2) the residual for the convolutional block is temporally subsampled by taking the mean of successive pairs of positions. The standard Conformer block follows theimplementationfrom Phil Wang (lucidrains), except that we use chunkwise attention and rely on PyTorch’sscaled\_dot\_product\_attention\(\)instead of the originaleinsum\(\)implementation.

***Figure 4:*Comparison of a standard and a subsampling Conformer block.

https://huggingface.co/blog/ibm-granite/granite-speech-5-0-470m-turboctc#training-dataTraining Data

The Granite Speech 5.0 models are trained with a combination of natural and synthetic data. The table below lists the natural datasets used in training and specifies whether the dataset was used to train both models or only the non-commercial model.

Both models are also trained on three synthetic datasets:

1. 2000 hours of multi-speaker data generated by concatenating single-speaker segments from MLS, YODAS, CommonVoice-17, VoxPopuli, and AMI;
2. 500 hours of multi-speaker data generated by concatenating single-speaker segments from Earnings-22; and
3. 240 hours of utterances containing numbers, currencies, website names, phone numbers, addresses, and items containing decimal points or dots which were generated using eithergpt\-oss\-120borgpt\-oss\-20band synthesized usingStyleTTS2.

https://huggingface.co/blog/ibm-granite/granite-speech-5-0-470m-turboctc#usageUsage

Granite Speech 5.0 TurboCTC is supported natively in transformers. Until the next Transformers release, install from source:

pip install git+https://github.com/huggingface/transformers.git datasets

from datasets import Audio, load_dataset
from transformers import AutoModelForCTC, AutoProcessor

model_id = "ibm-granite/granite-speech-5.0-470m-turboctc"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForCTC.from_pretrained(model_id, device_map="auto")

ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate))
speech_samples = [el["array"] for el in ds["audio"][:5]]

# `device` computes the log-mel front-end on the model's accelerator, saving a host-to-device copy
inputs = processor(
    speech_samples, sampling_rate=processor.feature_extractor.sampling_rate, device=model.device
)
inputs.to(model.device, dtype=model.dtype)
outputs = model.generate(**inputs)
print(processor.batch_decode(outputs, skip_special_tokens=True))