license: other license_name: fish-audio-community-license license_link: https://huggingface.co/FashionFlora/ogg-tts/blob/main/LICENSE-FISH-AUDIO.md language: [en, pl, es, de, ru, fr, pt, it] pipeline_tag: text-to-speech library_name: pytorch base_model: k2-fsa/OmniVoice tags: [tts, text-to-speech, ogg-tts, block-diffusion, voice-cloning, expressive-speech, emotion, multilingual, fish-audio-codec]

OGG TTS

Expressive multilingual text-to-speech with block diffusion

Preview release. This is checkpoint 120k of a 400k-step pre-training run that is still in progress, with a new acoustic decoder (axial MTP). Updated checkpoints will be published as new revisions of this repository; the earlier 20k and 40k previews (previous architecture) stay available in the history.

Overview

OGG TTS is a 0.75B-parameter zero-shot text-to-speech model trained on about 8,200 hours of studio-quality speech in English, Polish, Spanish, German, Russian, French, Portuguese and Italian.

It generates discrete tokens of the Fish Audio 44.1 kHz codec with block diffusion:

each block of 0.74 s is produced in parallel by iterative unmasking;
blocks follow one another left to right, so audio can be streamed as soon as the first block is ready.

The backbone is initialised from OmniVoice (Qwen3-0.6B, masked discrete diffusion). An axial multi-token prediction (MTP) decoder then fills in the acoustic detail of the block: the 9 residual codebooks, one per step, for all frames at once.

The name comes from the trolls of The Witcher. OGG is the first of three architectures (OGG, ROGG, POGG) that we compare in an upcoming technical report.

Highlights

Expressive by default. The model reads with natural, context-appropriate emotion.
Optional inline tags. 24 tags for emotion, delivery and vocal events, placed exactly where they should take effect.
Zero-shot voice cloning. From a few seconds of reference audio, or one of the built-in speaker identities without any prompt.
Context-aware delivery. Neighbouring lines can be given as text context, so a line of dialogue or a sentence of a story fits its scene.
Streaming. Finished blocks never change and can be played immediately.

Emotion and expression tags

Tags are optional. Without them the model infers the delivery from the text and its context. With them it follows the requested emotion or sound at the given position:

[frustrated] Not again… [sighing] Fine, I'll do it myself.
[scared] [whispering] Someone is here…
[shouting] Run! They're right behind us!

Group	Tags
Emotion	`[angry]` `[frustrated]` `[excited]` `[tired]` `[playful]` `[sad]` `[scared]` `[shouting]`
Delivery	`[whispering]` `[softly]` `[voice breaking]` `[singing]` `[pause]` `[slowly]`
Vocal events	`[laughing]` `[sighing]` `[gasping]` `[panting]` `[coughing]` `[clears throat]` `[sniffling]` `[scoffing]` `[yawning]` `[humming]`

The labels come from an audio-language-model pass over 502,668 sentences, i.e. the audio with its transcript, under a closed list of 24 tags. The model was instructed to tag only what is audible in the recording, never what the text alone suggests. 57,016 sentences (11 %) received at least one tag; the rest carry a checked transcript without tags, so the model also learns when not to add an effect. Tags are dropped from part of the training samples, which keeps them optional at inference time.

Tag	Examples	Tag	Examples	Tag	Examples
`[shouting]`	19,407	`[laughing]`	2,621	`[tired]`	708
`[angry]`	10,287	`[scared]`	1,847	`[slowly]`	419
`[frustrated]`	5,565	`[whispering]`	1,502	`[clears throat]`	356
`[pause]`	4,065	`[playful]`	1,380	`[humming]`	346
`[excited]`	3,980	`[gasping]`	1,266	`[coughing]`	333
`[sighing]`	3,738	`[scoffing]`	1,010	`[panting]`	248
		`[sad]`	894	`[voice breaking]`	144
		`[softly]`	887	`[singing]`	95

Rare tags ([singing], [yawning] 59, [sniffling] 46) have too few examples to be reliable. Tag control is still weak in early checkpoints and strengthens as pre-training continues.

Model architecture

Component	Details
Backbone	OmniVoice / Qwen3-0.6B, 28 layers, d = 1024. Bidirectional attention within a block, block-causal across blocks; earlier blocks are seen with all 10 codebooks
Acoustic decoder	Axial MTP, 110M parameters: 6 layers over a (frame × codebook) grid, each with attention across the codebooks of a frame (causal) and across the frames of a codebook (the prompt, earlier blocks and the current block)
Total parameters	~0.75B (codec excluded)
Audio codec	Fish Audio Drama 3 / S2 codec, 44.1 kHz, 21.5 frames/s, 1 semantic codebook (4096) + 9 residual codebooks (1024)
Block size	16 frames (0.74 s)
Decoding	6 confidence-ordered unmasking iterations per block with classifier-free guidance, then 9 axial-MTP steps (one per residual codebook); KV cache
Length	predicted by the model (end-of-speech token)

Usage

Requires the code from GitHub, the OmniVoice weights (k2-fsa/OmniVoice) and the Fish Audio codec (codec.pth from fishaudio/drama-3).

git clone https://github.com/FashionFlora/ogg-tts && cd ogg-tts
git clone https://github.com/fishaudio/fish-speech ogg_tts/fish-speech
export OMNIVOICE_PATH=$(huggingface-cli download k2-fsa/OmniVoice)
export FISH_CODEC_PATH=$(huggingface-cli download fishaudio/drama-3 codec.pth)

import sys
sys.path.insert(0, "ogg_tts")
from huggingface_hub import hf_hub_download
import infer_omnifish as inf, fish_codec as fc

model, tok = inf.load_model(hf_hub_download("FashionFlora/ogg-tts", "model.safetensors"), "cuda")
codec = fc.load_codec("cuda")

codes = inf.generate_cached(model, tok, None, "", "[sighing] Hello! I am OGG, a text to speech model.", "en", cfg=1.5)
audio = fc.decode(codec, codes[None].cuda())[0].float().cpu().numpy()   # 44.1 kHz

Voice cloning: pass the reference audio's codes (fc.encode) and its exact transcript:

ref, _ = fc.encode(codec, wav_44k[None].cuda())          # wav_44k: torch tensor, mono, 44.1 kHz, up to ~15 s
codes = inf.generate_cached(model, tok, ref[0].cpu(), "Transcript of the reference.", "Text to speak.", "en", cfg=1.5)

Context: neighbouring lines (not spoken) set the delivery, e.g. for a line of dialogue or the next sentence of a story:

codes = inf.generate_cached(model, tok, ref[0].cpu(), ref_text, "[frustrated] I told you already. I was at work.", "en",
                            semantic_left=("Where were you last night?",), after=("At three in the morning?",), cfg=1.5)

Guidance and pace. cfg (classifier-free guidance) also sets the tempo: 1–1.5 gives the natural pace of the voice, 2 or more speaks faster and more pointedly.

Training data

About 8,200 hours of studio-quality speech. The data is a private collection and is not publicly available.

Language	Hours
English	~2,610
Polish	~1,300
Spanish	~820
German	~750
Russian	~645
French	~500
Portuguese	~300
Italian	~290

Training


Hardware	1× NVIDIA RTX PRO 6000 Blackwell (96 GB)
Schedule	400k steps, 16k tokens per step, AdamW, cosine decay, bf16

Compute and cost

All training ran on a single rented AWS instance with 1× NVIDIA RTX PRO 6000 (96 GB) at $3.36 per hour. The figures below cover the whole rental, including setup, data processing and evaluation, not only the GPU time of the final model. Hours come from the run logs' timestamps and are approximate (±1 h per phase).

GPU time

Phase	Period (UTC, 2026)	GPU hours	Cost (USD)
Setup: pilot runs, first OmniFish (NAR) run, audio tokenization, speaker embeddings	Oct 28 – Oct 30	~40	~$134
Baseline: OmniFish-BD with semantic left context (136k steps)	Oct 30 – Nov 1	~57	~$192
Ablations: context layouts, curricula, speaker modes, endpoint heads, MTP variants (v2–v25), axial-decoder smoke test	Nov 1 – Nov 3	~49	~$164
Fork A vs. fork B (axial decoder, prompt handling)	Nov 3 – Nov 4	~2.6	~$9
Final model: OmniFish v26 (axial MTP decoder), 400k steps	Nov 4 – ~Nov 11	~183	~$615
Total		~332	~$1,115

The final run trains at about 2,200–2,500 steps per hour, about 100–110× real time in audio seen per hour.

The ablations and the baseline account for ~$365, about one third of the total. The final run is about half.

Annotation (Gemini)

Emotion and style tags and transcript checks were produced with Gemini Flash.

	Tokens	Rate (USD per 1M)	Cost (USD)
Input	294.8M	$0.50–1.00	~$220–295
Output	87.3M	$3.00	~$262
Total	382.1M		~$480–560

Rates follow the published Gemini 3 Flash list prices. Requests went through a third-party gateway, so the amounts actually billed may differ. Another 8.9M tokens went to a fallback model and are not priced here.

Storage

Amazon S3 (us-east-2), about 1.18 TB: ~$27 per month. Upload and same-region transfer to training machines cost nothing.

Summary

	Cost (USD)
GPU (whole rental, including ablations)	~$1,115
Annotation (Gemini)	~$480–560
Storage (S3)	~$27 / month
Total, one-off	~$1,595–1,675

The total does not include the later Audio8 fine-tune, the stage-3 experiments on a second machine, or the cost of collecting the raw data.

Evaluation

Checkpoint 120k, cfg=1.5, one sample per item, no cherry-picking. WER with Whisper large-v3-turbo (tags removed from the reference), speaker similarity with ECAPA2 (cosine), pitch variation as the standard deviation of F0 in semitones.

Held-out recordings (validation set: recordings and texts never trained on, 136 items in English, Spanish and German; a voice may also appear elsewhere in training). Each item clones the voice from a 6–10 s prompt and speaks two consecutive sentences that also exist as a real recording. The real recording, passed through the same codec, is the reference.

	OGG TTS 120k	Real recording
WER	1.4 %	2.6 %
Speaker similarity to the prompt	0.773	0.737
Duration vs. the real recording (median)	0.99	1.00
Pitch variation (F0 std, semitones)	3.58	4.25

Language	Items	WER	Similarity to prompt	Duration ratio
English	54	1.8 %	0.777	0.98
Spanish	78	1.2 %	0.765	0.99
German	4	0.6 %	0.887	0.97

All 8 languages (new texts, 32 items: a neutral and an angry line per voice, voices from the training set): WER 0.8 % on average, speaker similarity 0.785.

	EN	PL	ES	DE	RU	FR	PT	IT
WER	0.0 %	0.8 %	0.0 %	0.0 %	0.0 %	0.0 %	3.3 %	6.7 %
Similarity	0.780	0.767	0.828	0.824	0.763	0.708	0.771	0.841

The tempo now matches real speech (duration ratio 0.99). Intonation is still about 15 % flatter than in the real recordings. Public benchmarks (Seed-TTS, CV3) follow with the final checkpoint.

Limitations

Preview checkpoint: speaker similarity is not yet final, and the model may still mispronounce rare names.
Italian and Portuguese have the least training data.
Rare tags are not reliable yet.
Intonation is somewhat flatter than in real speech; cfg above 2 speeds the speech up.
Multi-speaker scenes in one pass are not part of this run; generate a dialogue line by line with context.

License and responsible use

Fish Audio codec. The model generates Fish Audio codec tokens and needs the Fish Audio codec to produce audio. Use is subject to the Fish Audio Community License (see NOTICE).
Real voices. Do not use OGG TTS to imitate the voice of a real person without their consent, to impersonate, deceive, defraud or harass anyone, or to create sexual content involving minors.
Research use. The weights are released for research.

Citation

@misc{oggtts2026,
  title  = {OGG TTS: Expressive Multilingual Text-to-Speech with Block Diffusion},
  author = {FashionFlora},
  year   = {2026},
  url    = {https://huggingface.co/FashionFlora/ogg-tts}
}