travis-moore/sema-tts-afro

Sema Afro TTS v1

Sema Afro TTS is a multilingual text-to-speech model covering 24 African language varieties with zero-shot voice cloning: give it a few seconds of any speaker and a transcript, and it speaks new text in that voice, in the target language.

Built under Sema by Angelo Asante as the speech layer of a voice-to-voice suite for African languages.

Hear it

Every clip below is one voice — the author's — cloned from a single 9-second reference, speaking languages the reference never spoke:

Language	Sample
Twi
English
Pidgin
Hausa
Yoruba
Swahili

Languages

Afrikaans	Amharic	English	Fula
Hausa	Igbo	Kabuverdianu	Kamba
Lingala	Luganda	Luo	Northern Sotho
Nyanja (Chichewa)	Oromo	Shona	Somali
Swahili	Twi	Umbundu	Wolof
Xhosa	Yoruba	Zulu	Ghanaian code-switch & Pidgin

Twi, English, Ghanaian code-switched speech and West African Pidgin come through the model's lineage and dedicated in-run data; the other 21 languages were added in this fine-tune, with the vocabulary extended to new scripts (Ge'ez) and diacritics (Yoruba tone marks and sub-dots).

Training

• Data: roughly 230 hours of curated read and conversational speech across the 24 varieties, resampled to 24 kHz mono.
• Schedule: 18 epochs / 197,640 updates on a single data-center GPU, with additional English, code-switch and Pidgin material introduced from update 100k.
• Vocabulary: the 3,567-token base vocab extended to 3,821 tokens (union of the base vocab and every character in the training text).
• Architecture: unchanged F5-TTS v1 Base (DiT + ConvNeXt V2, flow matching); only the text-embedding table was resized for the larger vocab.

Evaluation

Every language is scored with an ASR-in-the-loop protocol: a held-out native-speaker clip is cloned, the model reads an unseen sentence, and both the generated clip and the real human reference are transcribed by the same multilingual recognizer (MMS 1b-all). We report the character error rate gap = CER(generated) − CER(human reference). The subtraction cancels the recognizer's own error in each language; 0 means indistinguishable from the human baseline by this measure, negative means the ASR found the model easier to understand than the human clip.

16 of 21 scoreable languages finish within 10 points of their human baseline; 6 finish at or better than it. Luganda is excluded from the chart: the recognizer mis-hears even the real human reference there, so its numbers are not meaningful.

Language	Gap (pp)	Language	Gap (pp)
Lingala	−7.7	Kamba	+6.4
Wolof	−3.7	Oromo	+9.8
Nyanja	−2.2	Northern Sotho	+15.5
Somali	−1.9	Afrikaans	+17.4
Xhosa	−1.7	Hausa	+18.4
Swahili	−1.1	Yoruba	+30.9
Zulu	+0.8	Amharic	+78.8
Luo	+2.9
Igbo / Kabuverdianu	+3.6
Umbundu	+3.7
Shona	+3.9
Fula	+5.0

Audio samples from training milestones are in samples/.

Looking for stronger Hausa? See sema-tts-boost, a targeted continuation of this model that takes Hausa from +18.4 to +8.5.

Usage

pip install f5-tts==1.1.22

from huggingface_hub import hf_hub_download
from f5_tts.api import F5TTS

ckpt = hf_hub_download("travis-moore/sema-tts-afro", "sema_afro_v1.pt")
vocab = hf_hub_download("travis-moore/sema-tts-afro", "vocab.txt")

tts = F5TTS(model="F5TTS_v1_Base", ckpt_file=ckpt, vocab_file=vocab, use_ema=True)
tts.infer(
    ref_file="speaker.wav",            # 5-10s clean clip of any voice
    ref_text="what the speaker says in that clip",
    gen_text="Text to speak, in any supported language.",
    file_wave="out.wav",
    nfe_step=32, speed=0.9, seed=42,
)

Tips:

• A clean, calmly-paced 5–10 s reference gives the best clone; the model copies the reference's rhythm.
• The model honors full stops much more than commas — prefer short sentences.
• With f5-tts==1.1.22 and this extended vocab, patch one line before batch inference: in f5_tts/model/utils.py, change padding_value=-1 to padding_value=0.

Limitations

• Amharic remains far behind its human baseline; treat it as experimental.
• Yoruba and Hausa are usable but noticeably behind the strong languages in v1 — the boost model addresses both.
• Scores are single-sentence ASR probes: indicative, not exhaustive MOS studies.
• Best English/Twi/Pidgin results come from West African voice references.

Safety

This model clones voices from a few seconds of audio. The Apache 2.0 license permits broad use, but impersonating a real person without their consent, deception, and fraud are explicitly against the intent of this release. Only clone voices you have the right to use, and disclose synthetic audio where listeners could reasonably be misled.

Acknowledgements

Built on OpenF5-TTS and the F5-TTS codebase (paper). Scoring uses MMS.