granite-speech-5.0-470m-turboctc-nc

ibm-granite/granite-speech-5.0-470m-turboctc-nc

作者
ibm-granite
任务
语音识别
下载量
38,779
点赞数
47
趋势分
10
许可证
cc-by-nc-sa-4.0
收录时间
2026/11/4

license: cc-by-nc-sa-4.0 language:

  • en pipeline_tag: automatic-speech-recognition library_name: transformers

Granite Speech 5.0 470M TurboCTC NC

Disclaimer: This model is intended for research and non-commercial use-only. See Granite Speech 5.0 Turbo CTC if interested in commercial use.

Model Summary: Granite Speech 5.0 TurboCTC Non-commercial is a compact 470 million parameter English ASR model with very high inference speed that is released for research and noncommercial purposes only. We invite users to refer to granite-speech-5.0-470m-turboctc for other use cases.

The model consists of a conformer acoustic encoder with block self-attention, self-conditioning and temporal downsampling with an output layer corresponding to 16,384 BPE units. It was trained on approximately 75,000 hours of English audio from public corpora using Connectionist Temporal Classification (CTC) and inference is done non-autoregressively with greedy decoding.

Evaluations:

We evaluated granite-speech-5.0-470m-turboctc-nc on standard short-form English ASR benchmarks from the Open ASR leaderboard: granite-5.0-470m-turboctc-nc-wer Performance on the Open ASR leaderboard (official results as of October 2, 2026, RTFx measured on 1 H200): newplot-4-readable pareto-size-readable Performance on noisy and reverberant speech from the FFASR leaderboard (official results as of August 25, 2026, RTFx measured on 1 L4 GPU) ffasr_wer ffasr_rtfx

Release Date: August 25, 2026

License: CC-BY-NC-SA-4.0

Supported Languages: English

Intended Use: Use of this model is governed by the terms of the CC-BY-NC-SA-4.0 license.

Usage:

Granite Speech 5.0 TurboCTC NC is supported natively in transformers>=5.16.0:

pip install transformers>=5.16.0 datasets
from datasets import Audio, load_dataset
from transformers import AutoModelForCTC, AutoProcessor

model_id = "ibm-granite/granite-speech-5.0-470m-turboctc-nc"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForCTC.from_pretrained(model_id, device_map="auto")

ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate))
speech_samples = [el["array"] for el in ds["audio"][:5]]

# `device` computes the log-mel front-end on the model's accelerator, saving a host-to-device copy
inputs = processor(
    speech_samples, sampling_rate=processor.feature_extractor.sampling_rate, device=model.device
)
inputs.to(model.device, dtype=model.dtype)
outputs = model.generate(**inputs)
print(processor.batch_decode(outputs, skip_special_tokens=True))

Model Architecture:

The architecture of granite-speech-5.0-470m-turboctc-nc consists of 16 conformer blocks trained with Connectionist Temporal Classification (CTC) with a 16,384 BPE classification head (see configuration below). We perform temporal subsampling by a factor of 8 to reduce frame rates from 100Hz to 12.5Hz: first by stacking and skipping consecutive logmel+delta frames (2x) followed by strided convolutions and pooled residuals in the first two conformer blocks (4x) as shown in the figure below. In addition, the encoder uses block-attention with blocks of 128 frames and self-conditioned CTC from the middle layer.

Configuration parameterValue
Input dimension320 (80 logmels + 80 deltas) x 2
Nb. of layers16
Hidden dimension1024
Nb. of attention heads8
Attention head size128
Attention block size128
Convolution kernel size7
Output dimension (BPE)16384

conformer_blocks

Training Data:

Our training data is entirely comprised of publicly available datasets or of synthetic data generated from public corpora specifically targeting English ASR. A detailed description of the training datasets can be found in the table below:

NameNb. hoursSource
CommonVoice-172500https://huggingface.co/datasets/mozilla-foundation/common_voice_17_0
MLS44600https://huggingface.co/datasets/facebook/multilingual_librispeech
Librispeech960https://huggingface.co/datasets/openslr/librispeech_asr
VoxPopuli500https://huggingface.co/datasets/facebook/voxpopuli
YODAS8900https://huggingface.co/datasets/espnet/yodas
AMI150https://huggingface.co/datasets/edinburghcstr/ami
Earnings-22100https://huggingface.co/datasets/esb/datasets
GigaSpeech10000https://huggingface.co/datasets/speechcolab/gigaspeech
SPGI Speech4900https://huggingface.co/datasets/kensho/spgispeech

In addition, the model was trained on three synthetic datasets:

  1. 2000 hours of multi-speaker data generated by concatenating single-speaker segments from MLS, YODAS, CommonVoice-17, VoxPopuli, and AMI;
  2. 500 hours of multi-speaker data generated by concatenating single-speaker segments from Earnings-22; and
  3. 240 hours of utterances containing numbers, currencies, website names, phone numbers, addresses, and items containing decimal points or dots which were generated using either gpt-oss-120b or gpt-oss-20b and synthesized using StyleTTS2.

Infrastructure: We train Granite Speech TurboCTC NC using IBM's super computing cluster, Blue Vela, which is outfitted with NVIDIA H100 GPUs. This cluster provides a scalable and efficient infrastructure for training our models over thousands of GPUs. The training of this particular model was completed in 10 days on 8 H100 GPUs.

Resources

Citation

@misc{granite-speech-5.0-470m-turboctc,
  title={Granite Speech 5.0 TurboCTC},
  author={IBM Granite Speech Team},
  year={2026},
  url={https://huggingface.co/ibm-granite/granite-speech-5.0-470m-turboctc}
}

内容来源:Hugging Face 模型卡

在 Hugging Face 查看