YorubaEnglish-CodeSwitching-TTS

⚠️ Experimental research model — trained for only ~1 epoch. A full fine-tune of VoxCPM2 for Yoruba and Yoruba–English code-switched speech, trained on ~1,039 hours pooled from four sources, but stopped at 28,000 steps — only ~1.06 passes over the full dataset, and well before validation loss plateaued (see Training below). Promising and a clear step up from our first release, but this model has seen each training example only about once; expect it to keep improving substantially with further training.

A Yoruba text-to-speech model from LyngualLabs, full fine-tuned from openbmb/VoxCPM2 — a 2B, tokenizer-free, voice-cloning TTS. It speaks Yoruba in native script, including code-switching with English, with no G2P / espeak / language tags.

This is the second, larger-scale release in this line — the successor to LyngualLabs/VoxCPM2-Yoruba, which was trained on 100 hours. This model trains on **10x more data**.

Training data

Pooled from four independent sources into one ~1,039-hour corpus (dataset):

Source	Hours	Notes
DSN African Voices	309.0 h	spontaneous, multi-speaker
NaijaVoices	614.0 h	read + spontaneous, multi-speaker
YECS (LyngualLabs)	107.5 h	Yoruba-English code-switching, 140 speakers
WAXAL (yor_tts)	8.1 h	spontaneous
Total	1,038.7 h

Training

• Method: full fine-tuning (all 2B weights), not LoRA
• Base: openbmb/VoxCPM2 · Steps: 28,000 · Training time: 15h 42m 19s wall-clock
• Effective batch: 32 (bs 2 × grad-accum 16) · LR: 1e-5 cosine · bf16
• Hardware: single NVIDIA GPU, ~95 GB VRAM (Colab)
• Status: stopped at ~1.06 epochs. Validation loss (val/loss/diff) was still trending down with noise at the final checkpoint (0.730, a new low for the run) — not yet at a plateau, unlike our first (100h) release, which clearly plateaued and began overfitting. More training would likely improve this further.

Usage

A voice-cloning model: give a short reference clip + its transcript, and it speaks your text in that voice — or use voice-design (a text description, no reference).

# pip install git+https://github.com/OpenBMB/VoxCPM.git
from huggingface_hub import snapshot_download
from voxcpm import VoxCPM
import soundfile as sf

model_dir = snapshot_download("LyngualLabs/YorubaEnglish-CodeSwitching-TTS")
model = VoxCPM.from_pretrained(model_dir, load_denoiser=False)

# --- voice cloning: a reference clip + its exact transcript ---
wav = model.generate(
    text="Mo fẹ́ learn how to code, ṣùgbọ́n mi ò mọ programming language wo ló make sense.",
    prompt_wav_path="reference.wav",
    prompt_text="Ìròyìn ti sọ pé the government will ensure electricity tariff goes down ní January.",
    cfg_value=2.0,
    inference_timesteps=22,
)
sf.write("output.wav", wav, model.tts_model.sample_rate)

# --- or voice-design (no reference): prepend a voice description in parentheses ---
# NOTE: on this checkpoint voice-design is unreliable (low volume, weak gender/energy
# control) -- full fine-tuning on cloning-style data didn't reinforce this base-model
# skill. For controllable voices, prefer cloning (above) from a short reference clip.
wav = model.generate(
    text="(A young Nigerian woman, clear voice) Ẹ kú àárọ̀, ẹ jọ̀ọ́ ẹ jẹ́ ká bẹ̀rẹ̀ ìpàdé.",
    cfg_value=2.0,
    inference_timesteps=22,
)
sf.write("voice_design.wav", wav, model.tts_model.sample_rate)

Tips: raise inference_timesteps (20–25) for smoother audio; cfg_value higher sticks closer to the reference.

Audio samples

Synthesized by this model (step 28,000) — code-switched Yoruba–English, cloned from a single reference voice (same reference + sentences as our first release, for direct comparison).

If rain falls tomorrow, jọ̀ọ́ rántí láti mú umbrella àti raincoat.

Ìròyìn ti sọ pé the government will ensure electricity tariff goes down ní January.

Ẹ̀gbọ́n mi tó ń ṣiṣẹ́ ní bank ti gba promotion, we're all so so happy.

Mo fẹ́ learn how to code, ṣùgbọ́n mi ò mọ programming language wo ló make sense.

Wọ́n ní ìpàdé ní hotel tó wà ní Victoria Island, kì í ṣe ní office wa.

Long-form sample

A longer, multi-clause sentence — a tougher test of the ~1-epoch training budget:

Ní ọjọ́ Aje tó kọjá, mo lọ sí ilé-ìwòsàn pẹ̀lú ẹ̀gbọ́n mi nítorí pé ó ní severe headache tí kò dá dúró fún ọjọ́ mẹ́ta, àmọ́ dókítà tó yẹ̀ wá wò ní kíákíá sọ pé kò sí ohun tó le jù lọ, kìkì stress àti àìsùn dáadáa ni ó fà á, torí náà ó kọ̀wé àwọn oògùn díẹ̀ fún wa kí a sì rí i pé a sùn dáadáa, a mu omi tó pọ̀, a sì dín iye coffee tí a máa ń mu lójoojúmọ́ kù.

A second long-form sample with heavier Yoruba–English code-switching (full clauses alternating between the two languages, not just borrowed words):

Nígbà tí mo ń lọ síbi office ni òwúrọ̀ yìí, mo gba call kan from my manager tí ó sọ pé we need to reschedule the meeting nítorí pé ọ̀pọ̀ àwọn client ṣì wà ní traffic, torí náà dípò tí a ó ti bẹ̀rẹ̀ ní aago mẹ́wàá, a ó kàn bẹ̀rẹ̀ ní ọ̀sán yìí. But he also mentioned that everyone should come prepared with their reports, nítorí náà mo ní láti yára parí iṣẹ́ mi kí n tó dé síbẹ̀. Honestly, this kind of last-minute change máa ń fa wahala, ṣùgbọ́n mo mọ̀ pé we just have to adapt and keep moving forward. jẹ́jẹ́ láyé gbà.

A third long-form sample, English only — the Nigerian-English accent and cadence here were learned entirely from the YECS corpus (part of the pooled training data above), not from any English-specific data:

I was supposed to submit my report yesterday, but because of the network problems at the office, I could not send it on time, so I had to explain everything to my supervisor this morning, and thankfully, she was very understanding and gave me till tomorrow to complete it, so now I just need to focus and make sure I don't disappoint her this time.

Diverse voices

Voice-design (a text-only voice description, no reference clip) turned out to be unreliable on this checkpoint — full fine-tuning specializes the model on cloning-style data, and doesn't reinforce the base model's more abstract voice-design skill. Instead, these use cloning (the mechanism this model is actually strong at) from short reference clips:

Woman — Ìròyìn ti sọ pé the government will ensure electricity tariff goes down ní January.

Young man — Mo fẹ́ learn how to code, ṣùgbọ́n mi ò mọ programming language wo ló make sense.

Angry — Victor, are you serious right now?! Gbogbo wa la wà níbi wedding yìí, o kàn ń tẹ̀ laptop rẹ síbẹ̀ láìsimi!

Limitations

• Trained for only ~1 epoch (28,000 steps over ~1039h) — the model has seen each training example roughly once. Validation loss had not plateaued when training stopped; a longer run would very likely sound meaningfully better.
• Inconsistent quality across sentences — some inputs render cleanly, others (e.g. containing proper nouns or less common phrasing) come out noticeably weaker. This unevenness is consistent with the ~1-epoch training budget above.
• Multi-speaker training — use a reference clip for a consistent voice.
• Voice-design (text-only voice description, no reference) is unreliable on this checkpoint — low volume and weak gender/energy control. Use cloning instead.
• Specialized to Yoruba — full FT trades the base's other-language ability for Yoruba.
• Reflects the mixed spontaneous/read/code-switched style of its training sources.

Credits & license

Base: OpenBMB/VoxCPM2 (Apache-2.0). Training data: DSN African Voices (Data Science Nigeria), NaijaVoices (Chris Emezue, Busayo Awobade, Abraham Owodunni, Handel Emezue, Gloria Monica Tobechukwu Emezue, Nefertiti Nneoma Emezue, Sewade Ogun, Bunmi Akinremi, David Ifeoluwa Adelani, and the wider NaijaVoices community), YECS (LyngualLabs, 140 speakers), and WAXAL (Google / WAXAL Research Collective). Deep thanks to every team and speaker behind these corpora. Part of an independent Yoruba TTS effort by Victor Olufemi and LyngualLabs, alongside the WAXAL Research Collective's edge-ASR benchmark work. Apache-2.0.

@misc{datasciencenigeria_african_voices_2025,
  title        = {African Voices: Multilingual Speech Dataset for Low-Resource African Languages},
  author       = {DataScience Nigeria},
  year         = {2025},
  howpublished = {\url{https://www.africanvoices.ai}}
}

@article{emezue2025naijavoices,
  title   = {The NaijaVoices Dataset: Cultivating Large-Scale, High-Quality, Culturally-Rich Speech Data for African Languages},
  author  = {Emezue, Chris and Community, NaijaVoices and Awobade, Busayo and Owodunni, Abraham and Emezue, Handel and Emezue, Gloria Monica Tobechukwu and Emezue, Nefertiti Nneoma and Ogun, Sewade and Akinremi, Bunmi and Adelani, David Ifeoluwa and others},
  journal = {arXiv preprint arXiv:2505.20564},
  year    = {2025}
}

@misc{lynguallabs_yecs_2026,
  title        = {{YECS}: A 120-Hour Community-Curated Yoruba-English Code-Switching Corpus},
  author       = {{LyngualLabs}},
  year         = {2026},
  howpublished = {\url{https://lynguallabs.org/yecs}}
}

@article{waxal2026,
  title   = {WAXAL: A Large-Scale Multilingual African Language Speech Corpus},
  author  = {Anonymous},
  journal = {arXiv preprint arXiv:2602.02734},
  year    = {2026}
}

@article{waxalnet2026,
  title  = {The WAXAL ASR Benchmark: Fine-Tuned Edge Models Across 19 African Languages},
  author = {Olufemi, Victor Tolulope and Babatunde, Oreoluwa and Njema, Ramsey and
             Gbotemi, Bolarinwa and Yen, Wanchi Lucia and Uzodinma, John and
             Ajayi, Sunday and Williams, Oluwademilade and Moshood, Kausar and
             Anyaele, Innocent Elendu and Arefaine, Akebert Tesfahunegn and
             Hunzwi, Candace and Daniel, Wongel Dawit and Namuganga, Emmilly Immaculate and
             Kadima, Cleophas and Bahizire, Athanase Biluge and Ranaivoson, Onitsiky and
             Aaron, Emmanuel and Ladislaus, Nicholaus Dismas and Muhammed, Idris and
             Simenya, Jonathan Enoch and Koome, Martin and Endaylalu, Matewos Tegete and
             Adeyemo, Peter Ifeoluwa and Birindwa, Hondi Prisca and Eze-Mbey, Ukachi Agnes and
             Oduro-Yeboah, Yacoba and Aremu, Toluwani and Adjovi, Pericles and
             Ngueajio, Mikel K and Mitra, Prasenjit},
  year   = {2026},
  note   = {arXiv preprint arXiv:2606.02375}
}

Compute for training was generously provided by Open Token — thank you for making this possible.