Flow-matching voice cloning

F5-TTS is a zero-shot voice-cloning text-to-speech system from the X-LANCE lab at Shanghai Jiao Tong University, built on flow matching with a diffusion transformer. The practical pitch is that you give it a short reference clip and a transcript, and it speaks your target text in that voice, in Chinese and English, with quality that holds up against commercial systems. At 14,759 stars as of 2026-06, it is a well-regarded research-grade option that is also genuinely usable.

The architecture is part of why people reach for it: flow matching converges and runs faster than older diffusion approaches, and the design avoids a separate duration model, which simplifies the pipeline.

What it does

You provide a reference audio of a few seconds plus its transcript, then the text you want spoken. F5-TTS clones the reference voice for the new text, supports multi-speaker and code-switching between languages, and ships a Gradio web UI and a CLI for both quick use and scripting. Community fine-tunes have extended it to more languages beyond the base Chinese and English.

Install

pip install f5-tts

For training and fine-tuning, install from source with pip install -e . after cloning. There is a Gradio app (f5-tts_infer-gradio) and a CLI (f5-tts_infer-cli) for inference. A consumer GPU handles inference; training is far heavier on VRAM.

MIT code, non-commercial weights

This is the licensing nuance that matters and that people get wrong: the code is MIT, but the pretrained weights are not. The README states the code is released under MIT while the pretrained models are under CC-BY-NC, because they were trained on the Emilia dataset, an in-the-wild corpus. So you can use, modify, and ship the code commercially, but you cannot use the official released weights in a commercial product. The clean path to commercial use is to train your own weights on data you are licensed for, using the MIT code. That split makes F5-TTS more commercial-friendly than a fully restricted model, as long as you understand which half is which.

Where it fits, and where it doesn’t

Reach for F5-TTS when you want high-quality zero-shot voice cloning with a permissive code license, when Chinese and English cover your needs or a community fine-tune covers your language, and when you can supply a clean reference clip. It is a strong base for both research and a product, if you handle the weights question.

It is a weaker fit if you need a turnkey commercial model with no training step, since the official weights are non-commercial. Cloning quality depends heavily on the reference clip, very short or noisy references produce worse results, and reported inference speed varies from the headline benchmarks depending on hardware and setup. Training stability has also been a reported pain point for people fine-tuning.

F5-TTS versus the other open TTS models

F5-TTSfish-speechChatTTSVoxCPM
Stars14,75930,83239,45828,884
Strengthflow-matching cloningmultilingual cloningconversational prosodyvoice design and cloning
Code licenseMITFish Audio ResearchAGPLv3Apache-2.0
WeightsCC-BY-NCresearch, non-commercialCC-BY-NCApache-2.0
Commercial useretrain weights or licenseseparate license requirednot permittedpermitted

Counts are from GitHub as of June 2026. fish-speech needs a separate commercial license for any use. ChatTTS is research-only on both layers. VoxCPM under Apache-2.0 is fully commercial-friendly out of the box. F5-TTS sits in between: MIT code with a clear path to commercial use if you retrain the weights.

The star curve

The star curve climbed steadily after the paper and base models landed, as a flow-matching approach with strong zero-shot cloning drew both researchers and builders. The continued release activity into 2026 backs up the maintenance story, which is more active than some of its better-known peers.

What the issue tracker warns you about

The recurring issues are about reference quality and training. Cloning is sensitive to the reference clip, so short, noisy, or mismatched references degrade output, and Chinese tone and plosive artifacts show up in some cases. Real-world inference speed often differs from the headline benchmark numbers, which depend on specific hardware and acceleration setups that are hard to reproduce. People fine-tuning report training instability, including loss spikes after many steps. And the dependency chain (torch, attention kernels, Gradio versions) is fragile enough that environment pinning matters.

For multilingual cloning under a research license, see fish-speech. For conversational TTS, see ChatTTS. For an Apache-2.0, commercial-friendly model, see VoxCPM. For what is trending, see LLM tooling, the daily digest, and the weekly report.

FAQ

Can I use F5-TTS commercially? The code yes, the official weights no. The code is MIT, but the pretrained weights are CC-BY-NC because of the Emilia training data. To use it commercially, train your own weights on licensed data with the MIT code, or arrange separate licensing for the data.

F5-TTS vs fish-speech, which is more commercial-friendly? F5-TTS, with a caveat. Its code is MIT and you can retrain weights for commercial use, whereas fish-speech requires a separate commercial license for any use. For a fully permissive default, VoxCPM under Apache-2.0 is cleaner.

How long a reference clip does F5-TTS need? A few seconds is enough, but quality depends on the clip. Short, noisy, or mismatched references produce worse cloning, so use a clean reference of several seconds for the best result.

Which languages does F5-TTS support? The base models cover Chinese and English, with code-switching between them. Community fine-tunes extend it to other languages, so check for a fine-tune if your target is outside the base set.

Is F5-TTS fast enough for real-time use? It is faster than older diffusion TTS, but real-world speed varies from the published benchmarks depending on hardware and acceleration. Streaming first-packet latency can exceed real-time, so test for your specific setup before assuming real-time.