waxal-benchmarking/voxcpm2-top5-full

WAXAL VoxCPM2 — Top-5 African TTS

A full fine-tune of VoxCPM2 (2B params, tokenizer-free, diffusion-AR TTS with voice cloning) on the 5 highest-resource languages in the WAXAL/AfroNet collection, produced by the WAXAL project at LyngualLabs.

This is a research checkpoint. It is not production-polished — see Known Limitations below before using it for anything beyond experimentation.

This is the first model in the collection trained on a cleaned version of the WAXAL/AfroNet pooled corpus (see Data Cleaning below) rather than the raw pooled data used for earlier releases like voxcpm2-multilang28-full. It's also the smallest of three sibling models (Top-5 / Top-10 / all-28) trained from the same cleaned data and methodology, to compare whether language count or data quality matters more at this scale.

Languages

VoxCPM2 has no built-in mechanism for specifying language at inference — the base model infers pronunciation purely from text orthography. We use a dedicated <lang:XX> tag per language, registered as a real token in the tokenizer and resized into the text embedding table (mean-initialized, then learned during fine-tuning). The tag is mandatory — always prepend it to the text you want synthesized, e.g.:

<lang:yo> Ẹ káàárọ̀! Báwo ni oorun rí lálẹ́ àná?

Language	Tag	Hours after cleaning	Hours after resampling
Swahili	<lang:sw>	922.8	833.2
Kinyarwanda	<lang:rw>	910.5	827.6
Yoruba	<lang:yo>	732.3	742.2
Igbo	<lang:ig>	620.2	683.1
Hausa	<lang:ha>	534.2	633.9

Source data: the AfroNet speech collection (Professor/<language>-speech-data), after the Phase 1 cleaning pass described below. These tags are a subset of the 10 registered in VoxCPM2_langtagged (the same tagged base used for the Top-10 sibling model); the other 5 tag embedding rows exist in this checkpoint's vocabulary but received no training signal here.

Data Cleaning

Earlier releases in this collection (voxcpm2-multilang28-full, voxcpm2-multilang10-full, voxcpm2-bantu8-full) trained directly on the pooled raw corpus. This model and its Top-10 / all-28 siblings are trained on data that first passed through a dedicated cleaning pipeline, applied per language before any resampling:

1. Clip/peak check — reject clips with a sustained near-full-scale amplitude (likely clipped/distorted audio).
2. Silence trim — trim leading/trailing silence (RMS-threshold based) before any downstream check, so later checks evaluate the audio that's actually used for training, not the untrimmed original.
3. Duration-vs-text-length ratio check — flag clips whose trimmed duration is a statistical outlier relative to their text length for that language (1st/99th percentile bounds). This caught a real upstream data bug: a small number of rows (worst case, a single 9-second clip in the Luganda corpus) had their text field corrupted with multiple concatenated Common Voice metadata records — up to 225,940 characters for one clip. This check strips that kind of corruption out before it reaches training.
4. ASR-CER gate — transcribe the trimmed audio with a small ASR model (omniASR_CTC_1B) and compare against the label text (CER threshold 0.3, diacritic/tone-mark-insensitive comparison so correct transcriptions aren't penalized for missing a tone mark the ASR doesn't produce).

Retention for this model's 5 languages after cleaning: Swahili 96.4%, Kinyarwanda 96.1%, Hausa 82.0%, Yoruba 74.0%, Igbo 69.6%. Yoruba and Igbo's lower retention is primarily genuine ASR-CER disagreement rather than bad data — both are tonal/diacritic-heavy languages where omniASR_CTC_1B performs less reliably; see Known Limitations.

Training

• Base: VoxCPM2_langtagged — openbmb/VoxCPM2 with 10 <lang:XX> tokens added and the text embedding table resized (mean-initialized rows) before any fine-tuning. This is the same tagged base reused for the Top-10 sibling model; this run only trains on data for 5 of its 10 registered tags.
• Mode: full fine-tuning (all parameters trainable except the frozen AudioVAE codec).
• Data balancing: sqrt-temperature resampling (weight ∝ √hours, renormalized to preserve total volume) across the 5 languages, same methodology as the other models in this collection.
• Scale: 2,217,874 training rows (clips) after resampling, 3,230 held-out validation rows.
• Steps: 34,000 steps (~1.01 epochs over the resampled training set), batch size 3 × grad-accum 11 × 2 GPUs (effective batch 66).
• Hardware: 2× NVIDIA H200, via torchrun DDP.
• Learning rate: 1e-5, 100 warmup steps.
• Final validation loss: total 0.838 (diff 0.822, stop 0.017) at step 33,999.

Usage

from huggingface_hub import snapshot_download
from voxcpm import VoxCPM
import soundfile as sf

model_dir = snapshot_download("waxal-benchmarking/voxcpm2-top5-full")
model = VoxCPM.from_pretrained(model_dir, load_denoiser=False, optimize=False)

wav = model.generate(
    text="<lang:sw> Habari ya asubuhi! Leo ni siku nzuri sana.",
    cfg_value=2.0,
    inference_timesteps=22,
)
sf.write("out.wav", wav, model.tts_model.sample_rate)

optimize=False is required — torch.compile (the default optimization path) conflicts with some inference environments (e.g. HF ZeroGPU Spaces); disabling it costs some speed but avoids crashes.

Voice cloning works the same way as base VoxCPM2 — pass prompt_wav_path and prompt_text (the literal transcript of what's spoken in the reference clip) alongside the tagged text.

Evaluation

Pending. The automated evaluation pipeline used for earlier releases (ASR-based CER against ground-truth text, a real-recording CER noise floor per language, and DNSMOS naturalness) has not yet been run against this checkpoint. Numbers from voxcpm2-multilang28-full are not applicable here — that model was trained on uncleaned data with a different language set and is not a valid proxy for this model's quality. This section will be updated once Stage A/B/C evaluation has run.

Known Limitations

• No evaluation results yet (see above) — training loss converging normally is not the same as verified downstream speech quality.
• Yoruba and Igbo retained less data than the other three languages (74.0% and 69.6% vs. 96%+ for Swahili/Kinyarwanda) due to the ASR-CER gate disagreeing more often on these tonal/diacritic-heavy languages. We did not disable the CER gate for these two (unlike Somali/Ikposo in the all-28 cleaning pass, where drop rates were far more extreme at 70-80%) — their elevated-but-not-extreme drop rates were judged plausible rather than clearly an ASR-model artifact, but this is a judgment call, not a certainty.
• Voice-design and cross-lingual flexibility are likely reduced, consistent with what we've seen in other full-fine-tune checkpoints in this collection — full-FT on read-speech-only data tends to degrade the base model's text-prompted style/voice-design control and non-trained-language generation.
• Non-terminating generation was an observed failure mode in earlier releases on this same base model (see voxcpm2-multilang28-full's model card); whether it recurs here is one of the things the pending evaluation will check.
• Only 5 of the 10 tag embedding rows present in this checkpoint's vocabulary (inherited from the shared VoxCPM2_langtagged base) received any training signal — do not use the other 5 tags with this checkpoint.

Related

• waxal-benchmarking/voxcpm2-multilang28-full and voxcpm2-multilang10-full — earlier releases trained on the uncleaned pooled corpus.
• Top-10 and all-28 (clean-data) sibling models — trained the same way as this one, for comparison; linked here once released.
• AfroNet speech collection — the source training data.