speecht5_tts-pld-hil-v2

sapinsapin/speecht5_tts-pld-hil continue-finetuned on 10,664 hil clips from sapinsapin/pld -- the language's full usable set, not a sample of it.

The published checkpoints were each trained on ~1,800 clips for 1,000 steps, a cap that was a default rather than a measurement of how much data existed. This run keeps the same recipe and gives the model the rest of its own language.

training clips	10,464 train + 200 eval
steps	4,000
batch	8 x 4 accumulation
learning rate	1e-05
precision	fp32 + gradient checkpointing (fp16 NaNs SpeechT5's mel loss)
eval loss (this run)	0.3681

Usage

SpeechT5 holds no voice of its own -- every call needs a 512-d x-vector. None ships with the original PLD checkpoints, which is why they are usually run with an American speaker from the HuggingFace tutorial. speaker.npy in this repo is a real hil speaker from the training data, so you can skip that.

import numpy as np, torch, soundfile as sf
from huggingface_hub import hf_hub_download
from transformers import SpeechT5ForTextToSpeech, SpeechT5HifiGan, SpeechT5Processor

REPO = "Splintir/speecht5_tts-pld-hil-v2"

processor = SpeechT5Processor.from_pretrained(REPO)
model = SpeechT5ForTextToSpeech.from_pretrained(REPO).eval()
vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan").eval()

# speaker.npy is stored as (512,); the model wants (1, 512). Without the
# unsqueeze this raises "The first dimension of speaker_embeddings must be
# either 1 or the same as batch size".
speaker = torch.from_numpy(np.load(hf_hub_download(REPO, "speaker.npy"))).float().unsqueeze(0)

ids = processor(text="Maayong buntag sa imong tanan.", return_tensors="pt")["input_ids"]
with torch.inference_mode():
    speech = model.generate_speech(ids, speaker, vocoder=vocoder)

sf.write("out.wav", speech.numpy(), 16000)   # 16 kHz mono

# In a notebook (Colab, Jupyter), play it inline instead of saving:
from IPython.display import Audio, display
display(Audio(speech.numpy(), rate=16000))

Runs unmodified on a stock Colab CPU runtime -- every dependency is preinstalled, including the sentencepiece the tokenizer needs -- at roughly 1.8x real time. For GPU, .to("cuda") the model, the vocoder and ids/speaker, then .cpu() the result before sf.write.

The display(Audio(...)) line renders a play button in a notebook cell. It produces no sound over a terminal or SSH session; there, write the wav and fetch it (from google.colab import files; files.download("out.wav")).

Spell numbers out: the tokenizer is character-level Latin and drops digits silently.

Trained with scripts/train_tts.py; preprocessing matches finetune_tts.py from the halohalo pipeline -- one x-vector per clip, never averaged.