markmuller/TTS_POST_trainnig_1_emo

VoxCPM2 Persian — expressive post-training (emotion + paralinguistic tags)

A Persian TTS model built on VoxCPM2, post-trained to take a leading emotion caption and inline paralinguistic tags that steer how a line is delivered.

(warm)هر آدمی رنگ خودشو داره. می‌دونستی؟ [chuckles]
(fear)[breath] صدا رو شنیدی؟ یه چیزی اون پشت هست، مطمئنم.
(sad)[sighs] نمی‌دونم از کجا شروع کنم. همه‌چیز یه‌دفعه عوض شد.

The caption is optional — 25% of the post-training data had it stripped, so bare text still works and produces neutral delivery.

control	values
emotion caption	(warm) (happy) (excited) (sad) (fear) (anger)
inline tags	[breath] [sighs] [uhm] [chuckles] [yawns]

English inherited from an earlier stage uses different tag lemmas — [breathing], [laughing], [sighing] — because it came from a different corpus. The two conventions coexist rather than merge.

Training lineage

Each stage starts from the previous one's weights.

stage	data	steps	what it added
base	openbmb/VoxCPM2	—	multilingual TTS
round 1	2,137 h Persian audiobook, fully diacritized text	12,540 (1.5 ep)	Persian, but dependent on harakat
round 2	same audio duplicated — every clip once with harakat, once plain — plus 3× English NonverbalTTS replay	17,049 (1 ep)	reads plain Persian without diacritics; keeps English + nonverbal tags
this repo	78.5k Gemini-TTS conversational Persian clips (144 h, 7 voices, 6 emotions) + 30% replay of the round-2 corpus	9,168 (4 ep)	emotion captions and paralinguistic tags

The 30% Persian replay in this stage is load-bearing. The post-training corpus is studio-clean synthetic speech from only 7 voices; without replay it pulls the acoustic prior onto that signature and welds conversational style to those voices. The replay keeps the real, many-speaker audiobook prior — and both the plain and harakat spellings from round 2 — under load throughout.

Effective batch 48 (6 × 8 grad-accum, 1×B200), LR 1e-5, 300 warmup. 110,040 train rows / 1,001 val rows, split so no clip and no source dialogue crosses the train/val boundary.

Checkpoints

All eight saved checkpoints are included, each a complete folder (model.safetensors, audiovae.pth, tokenizer, config, and optimizer state so training can be resumed).

folder	step	epoch	val loss/total	val loss/stop
step_0002000	2,000	0.87	0.8357	0.0095
best_checkpoint_of_post_training	4,000	1.75	0.8282	0.0118
step_0006000	6,000	2.62	0.8404	0.0173
step_0008000	8,000	3.49	0.8389	0.0178
step_0008500	8,500	3.71	0.8341	0.0176
step_0009000	9,000	3.93	0.8356	0.0179
step_0009167	9,167	4.00	0.8360	0.0177
step_0009168	9,168	4.00	final save	—

best_checkpoint_of_post_training is best by validation loss, not by a listening test — read the honest caveat below before treating it as settled.

Usage

Every checkpoint lives in its own subfolder, so download the one you want and point from_pretrained at that local path:

from huggingface_hub import snapshot_download
from voxcpm.core import VoxCPM

CKPT = "best_checkpoint_of_post_training"   # or step_0009168, step_0002000, ...

path = snapshot_download(
    repo_id="markmuller/TTS_POST_trainnig_1_emo",
    allow_patterns=f"{CKPT}/*",             # ~26 GB, not the whole 207 GB repo
)
model = VoxCPM.from_pretrained(
    hf_model_id=f"{path}/{CKPT}",           # a local dir, so no second download
    load_denoiser=False,
    optimize=True,
)

wav = model.generate(
    text="(warm)سلام، وقتتون بخیر. چطور می‌تونم کمکتون کنم؟ [breath]",
    reference_wav_path="your_reference_voice.wav",
    cfg_value=2.0,
    inference_timesteps=10,
    max_len=4096,
    normalize=False,
    seed=1234,
)

Do not call VoxCPM.from_pretrained(hf_model_id="markmuller/TTS_POST_trainnig_1_emo") directly. It snapshots the entire repository — all 207 GB of checkpoints — and then fails anyway, because the repo root holds no model files. Use the allow_patterns form above.

Two parameters worth knowing:

• max_len=4096 — the library default. A smaller value silently truncates: at max_len=1000 output caps near 20 seconds, which cuts long passages mid-sentence.
• reference_wav_path is voice cloning. prompt_wav_path is a different mode — continuation — and will make the model read your prompt back to you.

Each folder also contains optimizer.pth and scheduler.pth, so training can be resumed from any of these checkpoints, not just restarted from the weights. If you only want to run inference, add ignore_patterns=[f"{CKPT}/optimizer.pth", f"{CKPT}/scheduler.pth"] to the download and it drops from 26 GB to 8.9 GB.

Honest limitations

The "best" checkpoint is not settled. Validation loss bottomed at epoch ~1.7 and the stop-head loss then doubled (0.0089 → 0.0179) through epoch 4, which reads as textbook overfitting on 7 voices. But a long-form listening test (7 passages of 179–330 characters, 14–27 s, rendered by all six checkpoints) found no length failures at epoch 4 — deviations from the per-passage median stayed under ±12% for every epoch-4 render. The loss metric and the audio disagree, and on this model the loss has been wrong-footed before: round 2's validation loss was completely flat while its audio audibly improved. Judge by ear.

The expressive data is synthetic. It was generated with Gemini TTS, so disfluency is performed rather than authentic, every clip is studio-clean with no recording texture, and there are only 7 source voices. excited is the thinnest emotion (2,759 clips, 3.5%) and is the most likely to render weakly.

Emotion range is bounded by the caption vocabulary. Six emotions, five tags. Anything outside that is unmodelled.

Persian only for the expressive controls. The emotion captions were trained on Persian; English is carried through from the earlier replay stage and has no caption training.

Data provenance

• Persian audiobook (rounds 1–2): Fidibo and Thomcles audiobook corpora.
• English (round 2 replay): NonverbalTTS, filtered.
• Expressive (this stage): dialogue mined from Persian ebooks, converted literary → colloquial by an LLM, then synthesized with Gemini TTS.

The apache-2.0 tag above is inherited from the VoxCPM2 base model and covers the modelling code, not the training corpora. The audiobook material and the Gemini-TTS-generated audio each carry their own terms, and anyone redistributing or using these weights commercially should satisfy themselves about those separately. If you are a rights-holder and want this taken down, open a discussion on the repo.