⚠️ Training methodology
The base moldel initialised and then trained further on a domain-specific Fulfulde corpus. The decision to fine-tune the model VITS on a single T4 GPU with ~2k utterances would converge to noise, whereas fine-tuning a multilingual base preserves phoneme coverage learned from language pretraining.
Training data
| Property | Value |
|---|---|
| Utterances | 1,924 |
| Format | 16 kHz mono WAV |
| Language | Fulfulde |
| Domain | Conversational / clinical phrases |
| Transcription | Human-verified, aligned to audio filenames |
| Metadata | metadata.csv with audio, text columns |
Training configuration
| Hyperparameter | Value |
|---|---|
| Base checkpoint | rnjema-unima/mms-tts-ful-baseline |
| Architecture | VITS (end-to-end TTS, stochastic duration predictor) |
| Optimiser | AdamW, β = (0.9, 0.999), ε = 1e-8 |
| Learning rate | 2e-5, linear warmup (100 steps) |
| Batch size | 4 per device, gradient accumulation = 1 |
| Precision | fp16 mixed |
| Epochs | 50 |
| Total steps | ~24,050 |
| GPU | NVIDIA T4 (15 GB) |
| Framework | finetune-hf-vits (HuggingFace), PyTorch, transformers==4.35.1 |
| Loss weights | mel 35.0, KL 1.5, dur 1.0, disc 3.0, gen 1.0, fmaps 1.0 |
Observed training curves
Generator loss fell from 40.9 → 30.8 within the first epoch; mel
reconstruction loss from 0.673 → 0.555; KL divergence from 6.65 → 3.11.
Full per-step traces are logged on Weights & Biases:
https://wandb.ai/yacksweb-yacksweb-tech/fulfulde_tts_finetune
Usage
from transformers import pipeline
from IPython.display import Audio, display
pipe = pipeline("text-to-speech", model="Qooladuntoye/tts-ful-model")
out = pipe("Ume waduman do ade miwargan")
display(Audio(out["audio"].squeeze(), rate=out["sampling_rate"]))
- Downloads last month
- 69