Continue-finetune on 99 ceb clips

README.md

@@ -1,64 +1,63 @@
 ---
-library_name: transformers
+language: ceb
 license: mit
+library_name: transformers
+pipeline_tag: text-to-speech
 base_model: Splintir/speecht5_tts-pld-ceb-v2
+datasets:
+- sapinsapin/pld
 tags:
-- generated_from_trainer
-model-index:
-- name: speecht5_tts-pld-ceb-solo
-  results: []
+- text-to-speech
+- philippines
+- philippine-languages
+- ceb
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
 # speecht5_tts-pld-ceb-solo
 
-This model is a fine-tuned version of [Splintir/speecht5_tts-pld-ceb-v2](https://huggingface.co/Splintir/speecht5_tts-pld-ceb-v2) on an unknown dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.3093
-
-## Model description
-
-More information needed
-
-## Intended uses & limitations
-
-More information needed
+[`Splintir/speecht5_tts-pld-ceb-v2`](https://huggingface.co/Splintir/speecht5_tts-pld-ceb-v2) continue-finetuned on **99
+more `ceb` clips** from [`sapinsapin/pld`](https://huggingface.co/datasets/sapinsapin/pld).
 
-## Training and evaluation data
+The published checkpoint was trained on ~1,800 clips for 1,000 steps. Measuring
+it showed that speaker conditioning was the part that undertrained: most native
+speaker embeddings collapsed the decoder into a long, quiet ramble. This run
+keeps the same recipe and gives it the rest of the language.
 
-More information needed
+| | |
+|---|---|
+| training clips | 99 |
+| steps | 300 |
+| batch | 4 x 4 accumulation |
+| learning rate | 5e-06 |
+| precision | fp32 + gradient checkpointing (fp16 NaNs SpeechT5's mel loss) |
+| eval loss (this run) | 0.3060 |
 
-## Training procedure
 
-### Training hyperparameters
+## Usage
 
-The following hyperparameters were used during training:
-- learning_rate: 5e-06
-- train_batch_size: 4
-- eval_batch_size: 4
-- seed: 42
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 16
-- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: linear
-- lr_scheduler_warmup_steps: 30
-- training_steps: 300
+SpeechT5 holds no voice of its own -- every call needs a 512-d x-vector. None
+ships with the original PLD checkpoints, which is why they are usually run with
+an American speaker from the HuggingFace tutorial. **`speaker.npy` in this repo
+is a real ceb speaker from the training data**, so you can skip that.
 
-### Training results
+```python
+import numpy as np, torch, soundfile as sf
+from huggingface_hub import hf_hub_download
+from transformers import SpeechT5ForTextToSpeech, SpeechT5HifiGan, SpeechT5Processor
 
-| Training Loss | Epoch | Step | Validation Loss |
-|:-------------:|:-----:|:----:|:---------------:|
-| 1.2577        | 10.8  | 75   | 0.3122          |
-| 1.1890        | 21.48 | 150  | 0.3115          |
-| 1.1782        | 32.16 | 225  | 0.3082          |
-| 1.2334        | 42.96 | 300  | 0.3093          |
+processor = SpeechT5Processor.from_pretrained("microsoft/speecht5_tts")
+model = SpeechT5ForTextToSpeech.from_pretrained("Splintir/speecht5_tts-pld-ceb-solo")
+vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")
 
+speaker = torch.tensor(np.load(hf_hub_download("Splintir/speecht5_tts-pld-ceb-solo", "speaker.npy")))
+ids = processor(text="Maayong buntag sa imong tanan.", return_tensors="pt")["input_ids"]
+sf.write("out.wav", model.generate_speech(ids, speaker, vocoder=vocoder).numpy(), 16000)
+```
 
-### Framework versions
+Spell numbers out: the tokenizer is character-level Latin and drops digits
+silently.
 
-- Transformers 5.13.1
-- Pytorch 2.11.0+cu128
-- Datasets 4.0.0
-- Tokenizers 0.22.2
+Trained with [`scripts/train_tts.py`](https://github.com/SPlinterRed/finetune-sapin);
+preprocessing matches `finetune_tts.py` from the
+[halohalo](https://github.com/sapinsapin/halohalo) pipeline -- one x-vector per
+clip, never averaged.

added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "<ctc_blank>": 80,
+  "<mask>": 79
+}

eval_texts.txt

@@ -0,0 +1,2 @@
+He is not getting younger so , gikinahanglan nga mangita siya og trabaho alang kanimo ug sa iyang umaabot nga pamilya .
+Sa mga nagmabdos na, mahimong niwang (o gamay o daot) ang gisabak, hangtod pwede pud nga malulid o abnormal.

processor_config.json

@@ -0,0 +1,20 @@
+{
+  "feature_extractor": {
+    "do_normalize": false,
+    "feature_extractor_type": "SpeechT5FeatureExtractor",
+    "feature_size": 1,
+    "fmax": 7600,
+    "fmin": 80,
+    "frame_signal_scale": 1.0,
+    "hop_length": 16,
+    "mel_floor": 1e-10,
+    "num_mel_bins": 80,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "return_attention_mask": true,
+    "sampling_rate": 16000,
+    "win_function": "hann_window",
+    "win_length": 64
+  },
+  "processor_class": "SpeechT5Processor"
+}

speaker.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:099d84bd036ff9783c48f3129c96631b360f1195a12c14c0a6433d13c7bfdb96
+size 2176

spm_char.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7fcc48f3e225f627b1641db410ceb0c8649bd2b0c982e150b03f8be3728ab560
+size 238473

tokenizer_config.json

@@ -0,0 +1,65 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "79": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "80": {
+      "content": "<ctc_blank>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "backend": "sentencepiece",
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "model_max_length": 600,
+  "normalize": false,
+  "pad_token": "<pad>",
+  "processor_class": "SpeechT5Processor",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "SpeechT5Tokenizer",
+  "unk_token": "<unk>"
+}