Use the fih banner; point quick start at setup_hebrew.sh
Browse files
README.md
CHANGED
|
@@ -16,7 +16,7 @@ library_name: fish-speech
|
|
| 16 |
pipeline_tag: text-to-speech
|
| 17 |
---
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
# Fish Audio S2-Pro โ Hebrew (LoRA + atomic IPA tokens)
|
| 22 |
|
|
@@ -46,33 +46,33 @@ Code lives in the fork the adapter was trained with:
|
|
| 46 |
```bash
|
| 47 |
git clone https://github.com/maxmelichov/fish-speech
|
| 48 |
cd fish-speech && uv sync --python 3.12 --extra cu129
|
| 49 |
-
pip install renikud-plus
|
| 50 |
|
| 51 |
-
|
| 52 |
-
hf download notmax123/Fish-Audio-S2-Pro-He --local-dir checkpoints/he
|
| 53 |
-
```
|
| 54 |
-
|
| 55 |
-
Build the IPA-extended base checkpoint (symlinks the S2-Pro weights, drops the
|
| 56 |
-
extended tokenizer and config on top), then synthesize:
|
| 57 |
-
|
| 58 |
-
```bash
|
| 59 |
-
python tools/hebrew/build_ipa_checkpoint.py \
|
| 60 |
-
--base checkpoints/s2-pro --output checkpoints/s2-pro-he-ipa
|
| 61 |
|
| 62 |
python tools/hebrew/infer_hebrew.py \
|
| 63 |
--text "ืฉืืื, ืื ืฉืืืื ืืืื?" \
|
| 64 |
-
--
|
| 65 |
-
--lora-checkpoint checkpoints/he/hebrew_lora_step2200.ckpt \
|
| 66 |
-
--lora-config r_32_alpha_16_core \
|
| 67 |
-
--ref-audio my_voice.wav --ref-text "..." \
|
| 68 |
--output out.wav
|
| 69 |
```
|
| 70 |
|
|
|
|
|
|
|
| 71 |
`infer_hebrew.py` runs plain unvocalized Hebrew through RenikudPlus G2P, maps the
|
| 72 |
IPA to the atomic tokens, and chunks long inputs on sentence boundaries.
|
| 73 |
`--lora-scale` scales the delta (0.0 = pure base model) if you want to dial the
|
| 74 |
adaptation down.
|
| 75 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
## How it works
|
| 77 |
|
| 78 |
**Atomic IPA tokens.** S2-Pro's BPE splits IPA into pieces that collide with
|
|
|
|
| 16 |
pipeline_tag: text-to-speech
|
| 17 |
---
|
| 18 |
|
| 19 |
+
<img src="fih.png" alt="Fish Audio S2-Pro Hebrew" width="420">
|
| 20 |
|
| 21 |
# Fish Audio S2-Pro โ Hebrew (LoRA + atomic IPA tokens)
|
| 22 |
|
|
|
|
| 46 |
```bash
|
| 47 |
git clone https://github.com/maxmelichov/fish-speech
|
| 48 |
cd fish-speech && uv sync --python 3.12 --extra cu129
|
| 49 |
+
pip install renikud-plus # Hebrew grapheme-to-phoneme
|
| 50 |
|
| 51 |
+
bash tools/hebrew/setup_hebrew.sh # base weights + this adapter + IPA checkpoint
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
|
| 53 |
python tools/hebrew/infer_hebrew.py \
|
| 54 |
--text "ืฉืืื, ืื ืฉืืืื ืืืื?" \
|
| 55 |
+
--lora-checkpoint checkpoints/hebrew/hebrew_lora_step2200.safetensors \
|
|
|
|
|
|
|
|
|
|
| 56 |
--output out.wav
|
| 57 |
```
|
| 58 |
|
| 59 |
+
Add `--ref-audio my_voice.wav --ref-text "..."` to clone a voice.
|
| 60 |
+
|
| 61 |
`infer_hebrew.py` runs plain unvocalized Hebrew through RenikudPlus G2P, maps the
|
| 62 |
IPA to the atomic tokens, and chunks long inputs on sentence boundaries.
|
| 63 |
`--lora-scale` scales the delta (0.0 = pure base model) if you want to dial the
|
| 64 |
adaptation down.
|
| 65 |
|
| 66 |
+
**Fine-tuning on your own Hebrew data** is one command โ a directory per speaker
|
| 67 |
+
of `*.wav` plus sibling `.lab` transcripts:
|
| 68 |
+
|
| 69 |
+
```bash
|
| 70 |
+
AUDIO_ROOT=my_audio tools/hebrew/run_hebrew_pipeline.sh
|
| 71 |
+
```
|
| 72 |
+
|
| 73 |
+
See [`tools/hebrew/README.md`](https://github.com/maxmelichov/fish-speech/blob/main/tools/hebrew/README.md)
|
| 74 |
+
for the full guide.
|
| 75 |
+
|
| 76 |
## How it works
|
| 77 |
|
| 78 |
**Atomic IPA tokens.** S2-Pro's BPE splits IPA into pieces that collide with
|