VoiceTut-TTS-GGUF

mohammedaly22/VoiceTut-TTS-GGUF

作者
mohammedaly22
任务
语音合成
下载量
89,066
点赞数
8
趋势分
1
许可证
apache-2.0
收录时间
2026/11/12

license: apache-2.0 language:

  • ar
  • arz
  • en library_name: audio.cpp pipeline_tag: text-to-speech base_model:
  • mohammedaly22/VoiceTut-TTS tags:
  • text-to-speech
  • tts
  • egyptian-arabic
  • arabic
  • code-switching
  • voice-cloning
  • omnivoice
  • gguf
  • audio.cpp

𓋹 VoiceTut-TTS GGUF

Egyptian Arabic & code-switching TTS for audio.cpp: up to 6.6× faster than the PyTorch package on the same GPU, no Python

🤗 Original model audio.cpp 🎧 Samples Companion ASR

These are standalone GGUF packages of VoiceTut-TTS, an OmniVoice fine-tune trained on ~380 hours of Egyptian podcast speech. It speaks natural Egyptian Arabic, switches between Arabic and English mid-sentence (عندي meeting بكرة), and clones voices zero-shot.

VoiceTut has exactly the same architecture as OmniVoice, so it runs through audio.cpp's existing omnivoice family with every OmniVoice route: voice cloning, voice design, streaming, the long-text chunker, and the server.

📦 Files

FileTypeSizeNotes
voicetut-tts-q8_0.ggufQ8_01.26 GiBRecommended. Same tensor policy as the official OmniVoice Q8_0 package
voicetut-tts-f16.ggufF161.53 GiBHigher-precision option for GPUs with tensor cores (slow on GTX 16xx, see below)

Each file is self-contained. It holds the VoiceTut generator, the OmniVoice audio tokenizer (unchanged from the base model), the tokenizer and config files, and the audio.cpp model spec. One file is all you need.

🚀 Usage

Build audio.cpp (Linux · Windows · macOS), then:

# install through the audio.cpp model manager (or download the .gguf directly)
audiocpp_model_manager install voicetut_tts_q8_0 --models-dir models

# voice cloning with Egyptian Arabic + English code-switching
audiocpp_cli --task tts --family omnivoice --model models/VoiceTut-TTS-GGUF --backend cuda \
  --language arz \
  --text "عندي meeting بكرة الصبح، فهحاول أخلص الشغل بدري النهارده." \
  --voice-ref speaker.wav --reference-text "<transcript of speaker.wav>" \
  --out out.wav
  • Always pass --language arz. This is the Egyptian Arabic language id VoiceTut was trained with.
  • Long text: add --text-chunk-size 160. For streaming chunk events, add --mode streaming --out-dir chunks/.
  • --num-inference-steps (default 32) trades speed for quality; --guidance-scale defaults to 2.0.

🗣️ The 17 built-in voices as named server voices

The original repo ships 17 studio reference voices: Abdelrahman, Abdullah, Ahmed, Aly, Asmaa, Esraa, Essam, Hanan, Hossam, Kamal, Mohamed, Omar, Omnia, Sarah, Sayed, Yasmin, and Zaki. Put each one in an audio.cpp server voice_dir as <Name>.wav, and add a prompt_text file with one <Name>|<transcript> line per voice (the transcripts are in reference_speakers/references.json). Then request them by name:

{ "voice_dir": "voices", "models": [{ "id": "voicetut", "family": "omnivoice",
  "path": "models/VoiceTut-TTS-GGUF/voicetut-tts-q8_0.gguf", "task": "tts" }] }
curl http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' -o out.wav \
  -d '{"model":"voicetut","input":"ازيك؟ عامل ايه النهارده؟","voice":"Mohamed","language":"arz"}'

⚡ Speed: original PyTorch vs. audio.cpp

Same model, same GPU, same 16 Egyptian Arabic sentences (~75 s of speech: short lines, questions, code-switching, numbers, and a 22 s paragraph), the same 4 built-in voices, 32 diffusion steps, and guidance 2.0. Each runtime loaded the model once and ran one warm-up request, then timed every sentence.

  • PyTorch: the official voicetut-tts package (OmniVoice backend), torch 2.11 + CUDA 12.6.
  • audio.cpp: a long-lived audiocpp_server session.
  • Hardware: GTX 1660 Ti Laptop 6 GB (Turing, no tensor cores), WSL2.

RTF is generation time divided by audio length; below 1.0 is faster than real time.

RuntimeWeightsOverall RTFMean latency / sentence22 s paragraphvs PyTorch defaultPeak VRAM
PyTorch voicetut-ttsfp16 (package default)8.6840.8 s107.0 s1.0×~2.6 GB
PyTorch voicetut-ttsfp321.999.3 s24.4 s4.4×~3.9 GB
audio.cppF16 GGUF8.1938.5 s97.3 s1.1×~2.5 GB
audio.cppfp32 (original safetensors)2.1610.2 s28.0 s4.0×~4.3 GB
audio.cppQ8_0 GGUF2.7813.1 s37.2 s3.1×~2.5 GB
audio.cpp, GTX 16xx MMQ build ¹Q8_0 GGUF1.326.2 s20.0 s6.6×~2.5 GB

How to read this:

  • On this GPU, the fastest setup is audio.cpp Q8_0 with the MMQ build ¹. It is 6.6× faster than the PyTorch package default, 1.5× faster than PyTorch's best setting (fp32), and uses ~1.4 GB less VRAM. The 22 s paragraph is generated in 20 s, faster than real time.
  • Like-for-like precisions perform about the same. audio.cpp fp32 is within ~8% of PyTorch fp32, and F16 is slow in both runtimes because this card has no tensor cores. On GTX 16xx cards, use Q8_0 and avoid F16.
  • Short sentences have a higher RTF than long text in every runtime, because each diffusion step also processes the 5–12 s reference clip.
  • audio.cpp needs no Python stack, starts in seconds, and keeps speed and memory flat across a long-lived session.
  • ¹ -DCMAKE_CUDA_ARCHITECTURES="61-virtual;80-virtual" -DGGML_CUDA_FORCE_MMQ=ON. This is what ggml's startup log recommends for GTX 16xx cards, and it halves Q8_0 generation time there. On GPUs with tensor cores, both runtimes are much faster: audio.cpp's README reports OmniVoice at RTF 0.146 on its reference GPU.

🎯 Output parity with PyTorch

Every runtime's output for the same 16 sentences was transcribed with QwenCleo-ASR and scored against the input text, using Egyptian-aware normalization that ignores diacritics and أ/إ/آ, ى/ي, ة/ه variants:

VoiceTut outputCERWER
PyTorch fp163.7%7.9%
PyTorch fp323.8%8.8%
audio.cpp Q8_0 (two runs)3.6–4.8%9.3–10.2%
audio.cpp fp324.7%13.9%

Generation samples randomly, so WER on 16 clips moves several points between runs of the same runtime. Character error rate is steadier, and it lands at 3.6–4.8% for every runtime: the audio.cpp port produces equally intelligible speech. This is an automatic intelligibility check, not an accent score. For that, listen to the audio demos.

⚠️ Limitations

  • audio.cpp passes text through as-is. It does not include VoiceTut's Python Egyptian text normalizer, so spell out numbers, times, dates and currencies (الساعة تلاتة ونص, not 3:30).
  • Voice cloning quality depends on a clean 5–10 s reference with an accurate transcript.

📜 License & credits

  • VoiceTut-TTS fine-tune: Apache-2.0, by Mohammed Aly.
  • The base OmniVoice weights are distributed by k2-fsa under their own terms (CC-BY-NC since 2026-07-03). The audio tokenizer, bosonai/higgs-audio-v2-tokenizer, is under the Boson Higgs Audio 2 Community License. Review these before commercial use.
  • Runtime: audio.cpp by @0xShug0. Conversion: audiocpp_gguf --type q8_0|f16 --family omnivoice.
@software{voicetut_tts_2026,
  author  = {Mohammed Aly},
  title   = {VoiceTut-TTS: Egyptian Arabic & Code-Switching Text-to-Speech},
  year    = {2026},
  url     = {https://github.com/MohammedAly22/VoiceTuT-TTS},
  note    = {Fine-tuned from OmniVoice}
}

内容来源:Hugging Face 模型卡

在 Hugging Face 查看