Default back to model pauses, speed 1.0 (listening test); clause mode experimental

README.md

@@ -27,7 +27,7 @@ It is built on [FunAudioLLM/CosyVoice2-0.5B](https://huggingface.co/FunAudioLLM/
 
 This repo is a **private research preview** for testing. It is not a production model.
 
-Inference defaults to **prosody v2**: clause-level generation with anchor-matched pause gaps, and a 1.05 pitch-preserving speed-up. See the section below.
+Inference defaults to the **Phase 2 final behaviour** (`--pause_mode model --speed 1.0`): model-generated pauses. In a listening test the original clearly sounded more natural than prosody v2, so clause mode is kept only as an experimental option.
 
 ## What is in the repo
 
@@ -69,14 +69,14 @@ The 0.5B model should fit comfortably on a 16 GB T4. This build was tested on an
 cd dynamic-accent-preview
 python infer.py --cosyvoice_dir ../CosyVoice \
   --text "<eng>this is dynamic accent,</eng> <pcm>e dey sound nice abi?</pcm>" \
-  --seed 0 --out out.wav                     # defaults: --pause_mode clause --speed 1.05
-# Phase 2 final behaviour: add  --pause_mode model --speed 1.0
+  --seed 0 --out out.wav                     # defaults: --pause_mode model --speed 1.0
+# experimental prosody v2: add  --pause_mode clause --speed 1.05
 ```
 
 - **Language tags:** wrap each span in `<eng>`, `<pcm>`, `<yor>`, `<ibo>` or `<hau>`. Untagged text is read as English. Keep punctuation inside the span it belongs to.
 - **`--seed`:** the final system is a single sample at seed 0. Try seeds 1 and 2 (or more) and keep the take you like. Best-of-N picking lowered CER by about a third in evaluation.
-- **`--pause_mode clause`** (default, prosody v2): the text is cut at `, ; : . ? !` and at language switches. Each chunk is generated separately and its edge silence is trimmed (energy VAD, 30 ms kept). Chunks are then joined with set gaps taken from the real anchor's pause statistics: 0.40 s ± 0.05 at commas and `;:`, 0.60 s at sentence ends, and 0.12 s at an unpunctuated language switch, with 10 ms fades. A one-word chunk is merged into the next chunk of the same language.
-- **`--speed`** (default 1.05): a pitch-preserving tempo factor (WSOLA via `audiotsm`) applied to the speech only, so the gaps stay as set. The allowed range is 1.0–1.2.
+- **`--pause_mode clause`** (experimental, prosody v2; rejected in listening): the text is cut at `, ; : . ? !` and at language switches. Each chunk is generated separately and its edge silence is trimmed (energy VAD, 30 ms kept). Chunks are then joined with set gaps taken from the real anchor's pause statistics: 0.40 s ± 0.05 at commas and `;:`, 0.60 s at sentence ends, and 0.12 s at an unpunctuated language switch, with 10 ms fades. A one-word chunk is merged into the next chunk of the same language.
+- **`--speed`** (default 1.0): a pitch-preserving tempo factor (WSOLA via `audiotsm`) applied to the speech only, so the gaps stay as set. The allowed range is 1.0–1.2.
 - **`--pause_mode model --speed 1.0`** reproduces the Phase 2 final system: model-generated pauses, sentence-by-sentence generation, internal silences capped at 0.6 s, and 0.6 s between sentences. Both modes use anchor prompt clip 0 at the LM and flow (`--prompt_clip 0`).
 - **`--dtype`:** the LM ran under bf16 autocast. T4 has no bf16, so `auto` uses fp16 there. fp16 has not been tested yet; if a T4 run sounds wrong, compare against an A100 (bf16) run.
 - **Determinism:** generation is not bit-reproducible across runs on GPU. Sampling under bf16 can pick a different token on near-ties, so the same seed can give a slightly different take.
@@ -88,7 +88,7 @@ from infer import DynamicAccent
 tts = DynamicAccent("pretrained/CosyVoice2-0.5B", "../CosyVoice",
                     "checkpoints/p2_lm_B/best.pt", "checkpoints/p2_flow_sft/best.pt", "prompts/prompts.json")
 wav = tts.say("<hau>Ina kwana, yaya aiki?</hau> <eng>The meeting has been moved to Friday morning.</eng>",
-              seed=0, pause_mode="clause", speed=1.05)
+              seed=0, pause_mode="model", speed=1.0)
 import soundfile as sf; sf.write("hau_eng.wav", wav, tts.sr)
 ```
 
@@ -108,7 +108,7 @@ The Yoruba, Igbo and Hausa example lines are hand-written and have not been chec
 ## Samples
 
 - `samples/pcm_eng_dynamic_accent_seed{0,1,2}.mp3`: "this is dynamic accent, e dey sound nice abi?" (English then Pidgin), seeds 0, 1 and 2. These are Phase 2 final renders (`--pause_mode model --speed 1.0`).
-- `samples/prosody_v2/*.mp3`: the same custom sentence plus `para_ibo_pcm_eng`, `para_yor_eng`, `para_hau_eng_pcm`, `ibo_000` and `yor_000`, rendered with the defaults (`--pause_mode clause --speed 1.05`, seed 0).
+- `samples/prosody_v2/*.mp3`: the same custom sentence plus `para_ibo_pcm_eng`, `para_yor_eng`, `para_hau_eng_pcm`, `ibo_000` and `yor_000`, rendered with experimental prosody v2 (`--pause_mode clause --speed 1.05`, seed 0). The user preferred the original.
 
 ## Prosody v2 (inference-time pause and pace fix)
 
@@ -129,7 +129,7 @@ The median ratio is 1.06, so the default `--speed` is **1.05** (rounded to 0.05
 | real anchor | 0.417 | 0.543 | 0.137 | 4.45 | – | – | – | ceiling ≈0.84 |
 | before: Phase 2 final (`model`, 1.0) | 0.339 | 0.488 | 0.057 | 4.26 | 0.175 | 0.124 | 0.175 | 0.814 |
 | clause, speed 1.0 | 0.339 | 0.526 | 0.095 | 3.95 | 0.179 | 0.123 | 0.166 | 0.775 |
-| **after: clause, speed 1.05 (default)** | 0.275 (≈0.37 *) | 0.550 | 0.086 | 4.18 | 0.193 | 0.143 | 0.207 | 0.750 |
+| after: clause, speed 1.05 (experimental) | 0.275 (≈0.37 *) | 0.550 | 0.086 | 4.18 | 0.193 | 0.143 | 0.207 | 0.750 |
 | clause, speed 1.10 | 0.315 | 0.550 | 0.067 | 4.37 | 0.244 | 0.194 | 0.224 | 0.748 |
 | model pauses, speed 1.05 | 0.235 | 0.441 | 0.029 | 4.35 | 0.189 | 0.149 | 0.188 | 0.788 |