yusasif/Yoruba_lang_tts

Yoruba F5-TTS — Zero-Shot Yoruba Voice Cloning

A fine-tuned F5-TTS model trained on 200 hours of Yoruba speech data, enabling high-quality text-to-speech synthesis and zero-shot voice cloning in the Yoruba language.

Given a short reference audio clip of any Yoruba speaker, the model clones that voice and synthesizes any Yoruba text in it — no speaker enrollment or fine-tuning required.

Model Details

Base model	F5-TTS (SWivid/F5-TTS)
Language	Yoruba (yo)
Training data	200 hours of Yoruba speech
Architecture	Flow matching (DiT)
Task	Text-to-speech, zero-shot voice cloning
Sample rate	24 kHz
License	CC-BY-NC-4.0

Capabilities

• Synthesize natural Yoruba speech from text
• Clone any Yoruba voice from a short reference audio clip
• Preserve tone, accent, and speaking style of the reference speaker
• Supports tonal markers in Yoruba text

Installation

pip install f5-tts

Usage

Basic voice cloning

from f5_tts.infer.utils_infer import (
    infer_process,
    load_model,
    load_vocoder,
    preprocess_ref_audio_text,
)
from f5_tts.model import DiT

# Load vocoder
vocoder = load_vocoder(vocoder_name="vocos", is_local=False, device="cuda")

# Load fine-tuned Yoruba model
ema_model = load_model(
    DiT,
    dict(dim=1024, depth=22, heads=16, ff_mult=2, text_dim=512, conv_layers=4),
    ckpt_path="path/to/model.pt",  # downloaded from this repo
    mel_spec_type="vocos",
    device="cuda",
)

# Provide a reference audio clip of the target Yoruba speaker
ref_audio, ref_text = preprocess_ref_audio_text(
    ref_audio_path="yoruba_speaker.wav",
    ref_text="Transcript of the reference audio in Yoruba.",  # or leave "" for auto-transcription
)

# Synthesize in the cloned voice
audio, sample_rate, _ = infer_process(
    ref_audio=ref_audio,
    ref_text=ref_text,
    gen_text="Ẹ káàárọ̀, báwo ni ẹ ṣe wà?",  # Yoruba text to synthesize
    model_obj=ema_model,
    vocoder=vocoder,
    mel_spec_type="vocos",
    speed=1.0,
    nfe_step=32,
    cfg_strength=2.0,
    device="cuda",
)

import soundfile as sf
sf.write("output.wav", audio, sample_rate)

CLI usage

f5-tts_infer-cli \
  --model path/to/model.pt \
  --ref_audio yoruba_speaker.wav \
  --ref_text "Transcript of reference audio." \
  --gen_text "Ẹ káàárọ̀, báwo ni ẹ ṣe wà?"

Tips for Best Results

• Use a clean, noise-free reference audio clip (3–10 seconds works well)
• Provide the correct transcript of the reference audio for best voice matching
• For tonal accuracy, include diacritical marks in the input text where possible

Citation

@article{chen-etal-2024-f5tts,
  title={F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching},
  author={Yushen Chen and Zhikang Niu and Ziyang Ma and Keqi Deng and Chunhui Wang and Jian Zhao and Kai Yu and Xie Chen},
  journal={arXiv preprint arXiv:2410.06885},
  year={2024}
}