Prompt48 commited on
Commit
d1ebcb6
·
verified ·
1 Parent(s): 0a801e6

Upload edit\Qwen3-TTS-test\finetuning\README.md with huggingface_hub

Browse files
edit//Qwen3-TTS-test//finetuning//README.md ADDED
@@ -0,0 +1,121 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## Fine Tuning Qwen3-TTS-12Hz-1.7B/0.6B-Base
2
+
3
+ The Qwen3-TTS-12Hz-1.7B/0.6B-Base model series currently supports single-speaker fine-tuning. Please run `pip install qwen-tts` first, then run the command below:
4
+
5
+ ```
6
+ git clone https://github.com/QwenLM/Qwen3-TTS.git
7
+ cd Qwen3-TTS/finetuning
8
+ ```
9
+
10
+ Then follow the steps below to complete the entire fine-tuning workflow. Multi-speaker fine-tuning and other advanced fine-tuning features will be supported in future releases.
11
+
12
+ ### 1) Input JSONL format
13
+
14
+ Prepare your training file as a JSONL (one JSON object per line). Each line must contain:
15
+
16
+ - `audio`: path to the target training audio (wav)
17
+ - `text`: transcript corresponding to `audio`
18
+ - `ref_audio`: path to the reference speaker audio (wav)
19
+
20
+ Example:
21
+ ```jsonl
22
+ {"audio":"./data/utt0001.wav","text":"其实我真的有发现,我是一个特别善于观察别人情绪的人。","ref_audio":"./data/ref.wav"}
23
+ {"audio":"./data/utt0002.wav","text":"She said she would be here by noon.","ref_audio":"./data/ref.wav"}
24
+ ```
25
+
26
+ `ref_audio` recommendation:
27
+ - Strongly recommended: use the same `ref_audio` for all samples.
28
+ - Keeping `ref_audio` identical across the dataset usually improves speaker consistency and stability during generation.
29
+
30
+
31
+ ### 2) Prepare data (extract `audio_codes`)
32
+
33
+ Convert `train_raw.jsonl` into a training JSONL that includes `audio_codes`:
34
+
35
+ ```bash
36
+ python prepare_data.py \
37
+ --device cuda:0 \
38
+ --tokenizer_model_path Qwen/Qwen3-TTS-Tokenizer-12Hz \
39
+ --input_jsonl train_raw.jsonl \
40
+ --output_jsonl train_with_codes.jsonl
41
+ ```
42
+
43
+
44
+ ### 3) Fine-tune
45
+
46
+ Run SFT using the prepared JSONL:
47
+
48
+ ```bash
49
+ python sft_12hz.py \
50
+ --init_model_path Qwen/Qwen3-TTS-12Hz-1.7B-Base \
51
+ --output_model_path output \
52
+ --train_jsonl train_with_codes.jsonl \
53
+ --batch_size 32 \
54
+ --lr 2e-6 \
55
+ --num_epochs 10 \
56
+ --speaker_name speaker_test
57
+ ```
58
+
59
+ Checkpoints will be written to:
60
+ - `output/checkpoint-epoch-0`
61
+ - `output/checkpoint-epoch-1`
62
+ - `output/checkpoint-epoch-2`
63
+ - ...
64
+
65
+
66
+ ### 4) Quick inference test
67
+
68
+ ```python
69
+ import torch
70
+ import soundfile as sf
71
+ from qwen_tts import Qwen3TTSModel
72
+
73
+ device = "cuda:0"
74
+ tts = Qwen3TTSModel.from_pretrained(
75
+ "output/checkpoint-epoch-2",
76
+ device_map=device,
77
+ dtype=torch.bfloat16,
78
+ attn_implementation="flash_attention_2",
79
+ )
80
+
81
+ wavs, sr = tts.generate_custom_voice(
82
+ text="She said she would be here by noon.",
83
+ speaker="speaker_test",
84
+ )
85
+ sf.write("output.wav", wavs[0], sr)
86
+ ```
87
+
88
+ ### One-click shell script example
89
+
90
+ ```bash
91
+ #!/usr/bin/env bash
92
+ set -e
93
+
94
+ DEVICE="cuda:0"
95
+ TOKENIZER_MODEL_PATH="Qwen/Qwen3-TTS-Tokenizer-12Hz"
96
+ INIT_MODEL_PATH="Qwen/Qwen3-TTS-12Hz-1.7B-Base"
97
+
98
+ RAW_JSONL="train_raw.jsonl"
99
+ TRAIN_JSONL="train_with_codes.jsonl"
100
+ OUTPUT_DIR="output"
101
+
102
+ BATCH_SIZE=2
103
+ LR=2e-5
104
+ EPOCHS=3
105
+ SPEAKER_NAME="speaker_1"
106
+
107
+ python prepare_data.py \
108
+ --device ${DEVICE} \
109
+ --tokenizer_model_path ${TOKENIZER_MODEL_PATH} \
110
+ --input_jsonl ${RAW_JSONL} \
111
+ --output_jsonl ${TRAIN_JSONL}
112
+
113
+ python sft_12hz.py \
114
+ --init_model_path ${INIT_MODEL_PATH} \
115
+ --output_model_path ${OUTPUT_DIR} \
116
+ --train_jsonl ${TRAIN_JSONL} \
117
+ --batch_size ${BATCH_SIZE} \
118
+ --lr ${LR} \
119
+ --num_epochs ${EPOCHS} \
120
+ --speaker_name ${SPEAKER_NAME}
121
+ ```