Add model card for GibbsTTS

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +56 -3
README.md CHANGED
@@ -1,3 +1,56 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ pipeline_tag: text-to-speech
4
+ ---
5
+
6
+ # GibbsTTS
7
+
8
+ GibbsTTS is a zero-shot text-to-speech model based on metric-induced discrete flow matching (MI-DFM). It incorporates a kinetic-optimal scheduler and finite-step CTMC moment correction to address heuristic scheduling and path-tracking errors in discrete generation.
9
+
10
+ - **Paper:** [Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speech](https://arxiv.org/abs/2605.09386)
11
+ - **Project Page:** [https://ydqmkkx.github.io/GibbsTTSProject](https://ydqmkkx.github.io/GibbsTTSProject)
12
+ - **Code:** [https://github.com/ydqmkkx/GibbsTTS](https://github.com/ydqmkkx/GibbsTTS)
13
+
14
+ ## Model Description
15
+ GibbsTTS addresses two primary limitations of MI-DFM:
16
+ 1. **Kinetic-Optimal Scheduler:** A training-free numerical schedule that traverses the probability path at constant Fisher-Rao speed, replacing heuristic schedulers.
17
+ 2. **Moment Correction:** A finite-step correction that adjusts jump probabilities to reduce errors while preserving the jump destination distribution.
18
+
19
+ The model was trained on the [Emilia](https://huggingface.co/datasets/amphion/Emilia-Dataset) dataset and supports English and Chinese Mandarin.
20
+
21
+ ## Sample Usage
22
+
23
+ To use the model, first clone the [official repository](https://github.com/ydqmkkx/GibbsTTS) and install the requirements.
24
+
25
+ ```python
26
+ from models import GibbsTTS
27
+ from config import ModelConfig
28
+ from IPython.display import Audio
29
+ import soundfile as sf
30
+
31
+ # Initialize model
32
+ configs = ModelConfig()
33
+ model = GibbsTTS(configs)
34
+
35
+ # English Synthesis
36
+ prompt_audio = "./prompt_examples/common_voice_en_188092-common_voice_en_188093.wav"
37
+ prompt_text = "This man looked exactly the same, except that now the roles were reversed."
38
+ target_text = "He also tried to remember some good stories to relate as he sheared the sheep."
39
+ language = "en"
40
+
41
+ audio = model.synthesize(prompt_audio=prompt_audio, prompt_text=prompt_text, target_text=target_text, language=language)
42
+
43
+ # Save and play
44
+ sf.write('target.wav', audio, 24000, 'PCM_24')
45
+ Audio(data=audio, rate=24000)
46
+ ```
47
+
48
+ ## Citation
49
+ ```bibtex
50
+ @article{GibbsTTS,
51
+ author = {Dong Yang and Yiyi Cai and Haoyu Zhang and Yuki Saito and Hiroshi Saruwatari},
52
+ title = {Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speech},
53
+ year = {2026},
54
+ journal = {arXiv preprint arXiv:2605.09386},
55
+ }
56
+ ```