zipformer-asr-turkish-v1

A ~20M-parameter streaming Turkish speech recognizer built for edge devices: a causal Zipformer2 encoder with a pruned RNN-T (transducer) head and an auxiliary CTC head, trained with icefall on ~2.1k hours of Turkish speech. It runs in real time on a single CPU core (ONNX int8) with 640 ms latency, and the same weights also decode with 320 ms chunks or with full (non-streaming) context.

With beam search + a small LSTM language model and internal-LM estimation (ILME), it reaches 5.28% WER on FLEURS Turkish (743 clips, balbal-normalizer; 6.50% with Damga) and 8.59% on MediaSpeech (Damga) while staying fully streaming at 640 ms.


Parameters	19.9M (ASR) + 10.2M (optional LSTM LM)
Input	16 kHz mono audio, 80-dim log-mel fbank
Output	lowercase spoken-form Turkish: no punctuation, numbers written as words, fillers kept
Latency	640 ms chunks (also 320 ms or full context)
Speed	RTF 0.020 on one desktop CPU thread (int8, greedy, 640 ms)
Vocabulary	SentencePiece unigram, 500 tokens

Results

WER (%) on FLEURS tr test (743 clips) and MediaSpeech tr (252 clips), scored with the two normalizations other open Turkish models report, so the numbers can be compared directly:

Damga (used e.g. by mihuai/turkish-stt-66m-v2): Damga written-to-spoken normalization (numbers, ordinals, abbreviations, units), Turkish lowercasing, apostrophe and circumflex removal, punctuation removal.
balbal-normalizer (used by balbal-v2): the balbal scoring normalizer (corpus_metrics_v2). On top of the above it ignores compound-spacing differences ("yurt dışı" = "yurtdışı"), compares numbers digit by digit (so "doksan" = "dokuz sıfır"), and treats "X ile" = "Xle".

dev/test are held-out in-house sets (~7.9 h each, not released). All rows are the same checkpoint (epoch 29, average of epochs 26-29).

Decoding	Runtime	FLEURS Damga	FLEURS balbal	MediaSpeech Damga	MediaSpeech balbal	dev	test
640 ms, greedy	PyTorch	7.99	6.75	10.34	10.31	4.24	4.21
640 ms, beam 4	PyTorch	7.82	6.57	9.77	9.73	4.00	4.00
640 ms, beam 4 + LSTM LM + LODR	PyTorch	6.86	5.77	8.94	8.89	3.67	3.59
640 ms, beam 8 + LSTM LM + ILME	PyTorch	6.50	5.28	8.59	8.44	3.47	3.51
640 ms, greedy	ONNX int8 (`streaming-640ms`)	8.05	6.80	10.22	10.23
320 ms, greedy	ONNX int8 (`streaming-320ms`)	8.66	7.42	10.87	10.81
full context, greedy	ONNX int8 (`offline-full-context`)	7.09	5.91	8.73	8.71

ONNX int8 speed (one desktop CPU thread, greedy): RTF 0.020 at 640 ms, 0.028 at 320 ms, 0.013 full context.

Other systems on the same clips. Rows marked "card" are copied from that model's own model card; the rest are our runs. As a check, our Whisper large-v3 run scores 3.77 with the balbal-normalizer vs 3.71 on the balbal-v2 card.

System	Params	Streaming	FLEURS Damga	FLEURS balbal	MediaSpeech Damga	MediaSpeech balbal
Google Chirp 3 (cloud)	—	no	2.34	1.72	—	—
ElevenLabs Scribe (cloud)	—	no	2.84	2.41	9.67	6.59
Whisper large-v3	1.55B	no	4.75	3.77	11.12	10.41
Whisper large-v3-turbo	809M	no	5.39	4.42	10.24	9.46
balbal-v2 (card; KenLM)	45.4M	offline eval	—	5.22	—	—
mihuai/turkish-stt-66m-v2 (card; 320 ms, beam 4)	66M	yes	6.21	5.26 (balbal card)	10.95	—
omniASR LLM 7B (balbal card)	7B	no	—	5.31	—	—
Qwen3-ASR 1.7B (balbal card)	1.7B	no	—	6.52	—	—
Duyu-small (balbal card)	49.3M	no	—	6.62	—	—
zipformer-asr-turkish-v1, beam 8 + LM + ILME, 640 ms	19.9M + 10.2M LM	yes	6.50	5.28	8.59	8.44
zipformer-asr-turkish-v1, full context, greedy, ONNX int8	19.9M	no	7.09	5.91	8.73	8.71
zipformer-asr-turkish-v1, 640 ms, greedy, ONNX int8	19.9M	yes	8.05	6.80	10.22	10.23

Usage

Streaming with sherpa-onnx (recommended for devices)

import numpy as np, soundfile as sf, sherpa_onnx

d = "streaming-640ms"   # or "streaming-320ms"
rec = sherpa_onnx.OnlineRecognizer.from_transducer(
    tokens=f"{d}/tokens.txt", encoder=f"{d}/encoder.int8.onnx",
    decoder=f"{d}/decoder.int8.onnx", joiner=f"{d}/joiner.int8.onnx",
    num_threads=1, sample_rate=16000, feature_dim=80, decoding_method="greedy_search",
    enable_endpoint_detection=True)

x, sr = sf.read("example.wav", dtype="float32")
s = rec.create_stream()
for i in range(0, len(x), 1600):          # feed 100 ms at a time, like a microphone
    s.accept_waveform(sr, x[i:i + 1600])
    while rec.is_ready(s):
        rec.decode_stream(s)
    print(rec.get_result(s), end="\r")
s.accept_waveform(sr, np.zeros(int(0.8 * sr), np.float32)); s.input_finished()
while rec.is_ready(s):
    rec.decode_stream(s)
print(rec.get_result(s))

Full context (non-streaming)

d = "offline-full-context"
rec = sherpa_onnx.OfflineRecognizer.from_transducer(
    tokens=f"{d}/tokens.txt", encoder=f"{d}/encoder.int8.onnx",
    decoder=f"{d}/decoder.int8.onnx", joiner=f"{d}/joiner.int8.onnx",
    num_threads=1, sample_rate=16000, feature_dim=80, decoding_method="greedy_search")
s = rec.create_stream(); s.accept_waveform(sr, x); rec.decode_stream(s); print(s.result.text)

Best accuracy: beam search + LM + ILME (PyTorch / icefall)

sherpa-onnx has no ILME, so the best setting runs in PyTorch with icefall. The script was tested with icefall at git commit cbc985f (git -C icefall checkout cbc985f); newer versions should work but are untested.

python inference/transcribe.py --icefall ~/icefall --method beam_lm_ilme --context 640ms example.wav
# --method greedy | beam | beam_lm_ilme   --context 640ms | 320ms | full   --beam 8 --lm-scale 0.45 --ilme-scale 0.2

Files

Path	Content
`exp/pretrained.pt`	PyTorch weights (icefall `{"model": state_dict}`), averaged epochs 26-29
`streaming-640ms/`, `streaming-320ms/`	streaming transducer ONNX (encoder/decoder/joiner, fp32 and int8) + `tokens.txt`
`offline-full-context/`	the same model exported for full-context decoding (fp32 and int8)
`lm/epoch-4.pt`, `lm/lstm768-with-state*.onnx`	2×768 LSTM token LM (PyTorch; ONNX for sherpa-onnx streaming LM)
`data/lang_bpe_500/`	SentencePiece model, vocab (for hotwords) and tokens
`inference/`	`transcribe.py` and `beam_search_ext.py` (beam search with LM shallow fusion, LODR and ILME; derived from icefall)

Model

Encoder: causal Zipformer2, 6 stacks (2 layers each, dims 192-256-256-256-256-192, feed-forward 512-640), trained with mixed chunk sizes {320 ms, 640 ms, 1.28 s, full}, so one model serves several latencies.
Decoder / joiner: stateless predictor (2-token context), joiner dim 320; auxiliary CTC head.
Training: pruned RNN-T + CTC, 29 epochs on one RTX 5080. Augmentation from epoch 20: synthetic room impulse responses, MUSAN noise and music, street/mall/factory recordings, Turkish multi-talker babble, device simulation (band-limiting, AGC, clipping) and codecs (8 kHz μ-law/GSM, Opus 7-32 kbps, MP3).
Language model (optional): 2-layer LSTM (768, tied embeddings) on Turkish Wikipedia + mC4 + transcripts, sentences overlapping the test sets removed. Fused with λ = 0.45 (shallow fusion).
ILME: the transducer's predictor already learns an internal language model from the training transcripts, so adding an external LM counts that knowledge twice. ILME estimates the internal LM by running the joiner with the encoder contribution removed and subtracts it (weight 0.2), so the external LM adds new information instead of repeating what the model already knows.

Training data

~2.1k hours of Turkish speech from publicly available online videos, labelled automatically: a segment is kept where Whisper and an in-house Turkish CTC model agree, and disagreement regions are recovered with a rule-based selector (+ LLM and a third ASR system as tie-breakers). Train/dev/test are split by channel. The data is not released.

Limitations

English and foreign words: code-switched words ("laptop", "deadline") and foreign names are often written phonetically or dropped. This is the main focus of the next version.
Very quiet or far-field speech degrades noticeably.
Output format: spoken form only (no capitalization, punctuation or digits); add inverse text normalization if you need written form.
Heavy noise: error rates rise quickly below 10 dB SNR.

License

Apache-2.0 for the model weights and code. inference/beam_search_ext.py is derived from icefall (Apache-2.0).

Acknowledgements

Built on icefall, k2, lhotse and sherpa-onnx. Zipformer: Yao et al., Zipformer: A faster and better encoder for automatic speech recognition, ICLR 2024.

Türkçe özet

Uç cihazlar için ~20M parametreli, akışlı (640 ms) Türkçe konuşma tanıma modeli. Tek CPU çekirdeğinde gerçek zamanlı çalışır (ONNX int8). Işın araması + LSTM dil modeli + ILME ile FLEURS Türkçe'de (743 klip) %5,28 (balbal-normalizer) / %6,50 (Damga), MediaSpeech'te %8,59 WER (Damga). Çıktı konuşulduğu gibi: küçük harf, noktalama yok, sayılar yazıyla.

Author: Ömer Aykılıç