speakwrite-azerbaijani

Azerbaijani speech recognition for CPU: FastConformer-CTC (18 layers, d512) with a 3-pass loop refiner, product-key acoustic memory (700 x 700 sub-keys) and a 6-gram token KenLM. Input is 16 kHz mono; output has case and . , ? !, with numbers written as words.

Results

LocalDoc/fleurs-azerbaijani-asr, not used for training. Reference: text_norm (numbers as words). WER and CER ignore case and punctuation. Clips with norm_status = failed are excluded (11 of 3977).

WER / CER:

split	clips	hours	PyTorch, no LM	PyTorch + KenLM	ONNX fast + KenLM
test	914	3.20	16.34 / 4.75	10.98 / 3.57	11.03 / 3.61
dev	397	1.34	16.52 / 4.75	11.05 / 3.50	11.25 / 3.59
train	2655	9.27	15.82 / 4.80	11.30 / 3.78	11.36 / 3.83
all	3966	13.81	16.01 / 4.79	11.20 / 3.70	11.27 / 3.75

Decoder weights (alpha 1.0, beta 2.0) were chosen on the FLEURS dev split from 8 pairs, so the test row is the one to quote.
The ONNX plain graph also gives 11.03 / 3.61 on FLEURS test; it was not run on the other splits.
Sentences of the FLEURS test split were excluded from the KenLM text; dev and train sentences were not.
Internal validation, no LM, on our own data mix: WER 13.48 / CER 6.01.

Speed on one i7-12700K, in Docker, times faster than real time, for the acoustic model plus KenLM:

runtime	threads	speed	weights	measured on
ONNX fast	8	84-87x	268 MB	this checkpoint, FLEURS test, dev and train
ONNX plain	8	~68x	548 MB	this checkpoint, FLEURS test
PyTorch float32	8	~45x	865 MB	this checkpoint, all FLEURS
PyTorch float32	1	12.7x	865 MB	an earlier checkpoint of this model, 100 clips

The machine had other load during the runs, so these are rough. KenLM takes about 1.5% of the time in the PyTorch runs and adds 1 GB to the download for every runtime. ONNX was not measured on 1 thread.

Use

pip install balbal

import balbal

model = balbal.Model("LocalDoc/speakwrite-azerbaijani", threads=8)
print(model.transcribe("speech.wav")["text"])

Tested with balbal 0.3.0 on Python 3.12 and Linux. The first run downloads about 1.3 GB: the fast graph and the KenLM. Pass runtime="onnx" for the plain graph, which uses standard ONNX operators only.

transcribe takes a file path or a waveform array with its sample rate: model.transcribe(wave, 16000). WAV, FLAC and OGG are read directly; other formats need FFmpeg. The result also has words, with word, start, end and confidence for each word.

Audio longer than 30 s is cut into 30 s windows with 10 s overlap. overlap_seconds=0 cuts fixed 30 s chunks instead, which is faster but gave a higher WER on our long recordings.

PyTorch

model.pt is the original checkpoint, for further training and experiments.

Needs balbal at commit 1be9f88b77c63fbafa7713c6e46e79847f6c390e (Python 3.11-3.12, Linux; install requirements/full.txt, then run inference/build_native.py). Copy this repository's refiner.json over balbal's configs/balbal-v2-manas/refiner.json: this model's memory size differs from balbal's default, and the weights will not load otherwise.

import json, sys
sys.path[:0] = ["balbal/inference", "balbal"]
from balbal.audio import read_audio
from runtime import Engine

engine = Engine("speakwrite-azerbaijani", runtime="torch", settings=json.load(open("speakwrite-azerbaijani/decoder.json")))
print(engine.transcribe(read_audio("speech.mp3"), itn=False)["text"])

Keep itn=False: balbal's number conversion is Turkish. Long audio is cut into fixed 30 s chunks.

To use the PyTorch model without the KenLM, take the argmax of the CTC output instead of Engine: without an LM, WER on FLEURS test is 16.34. The lm/ folder is then not needed.

Training

About 2800 hours of Azerbaijani speech. Encoder initialized from nvidia/stt_en_fastconformer_ctc_large, then modified and trained for 4.5 epochs (127552 updates) with the balbal-v2-manas recipe. balbal's own weights were not used.

Acknowledgements

The ONNX graphs and the pip install balbal setup were contributed by erayyapagci, the author of balbal.

License

CC BY-NC 4.0. Non-commercial use only; credit is required in any project using this model: "speakwrite-azerbaijani by LocalDoc" with a link to this page. For commercial use, contact LocalDoc.

Based on nvidia/stt_en_fastconformer_ctc_large (CC BY 4.0, NVIDIA) and on code from balbal (Apache 2.0).