Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,76 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- text-to-speech
|
| 4 |
+
license: cc-by-nc-sa-4.0
|
| 5 |
+
language:
|
| 6 |
+
- zh
|
| 7 |
+
- en
|
| 8 |
+
- de
|
| 9 |
+
- ja
|
| 10 |
+
- fr
|
| 11 |
+
- es
|
| 12 |
+
- ko
|
| 13 |
+
- ar
|
| 14 |
+
- nl
|
| 15 |
+
- ru
|
| 16 |
+
- it
|
| 17 |
+
- pl
|
| 18 |
+
- pt
|
| 19 |
+
pipeline_tag: text-to-speech
|
| 20 |
+
inference: false
|
| 21 |
+
---
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
This is a reformat of the [official Fish Speech v1.5 weights](https://huggingface.co/fishaudio/fish-speech-1.5) to work with [fish-speech.rs](https://github.com/EndlessReform/fish-speech.rs/tree/main).
|
| 25 |
+
|
| 26 |
+
I've made the following changes, for better compatibility with [Candle.rs](https://github.com/huggingface/candle) and the HuggingFace ecosystem:
|
| 27 |
+
- DualAR transformer weights converted to `.safetensors` for safety and easier loading
|
| 28 |
+
- Tokenizer ported from [Tiktoken](https://github.com/openai/tiktoken) format and [custom wrapper](https://github.com/fishaudio/fish-speech/pull/696/files#diff-0db124559777c2399fda2a45cff039afa7eb1a53f2f98f628b02862c10b339ef) to HuggingFace Tokenizers for easier downstream use
|
| 29 |
+
- VQGAN is unchanged from v1.4, so copying the weight-norm merged safetensors and FireflyGAN config from [my previous conversion](https://huggingface.co/jkeisling/fish-speech-1.4)
|
| 30 |
+
|
| 31 |
+
NOTE:
|
| 32 |
+
- Please respect the original license and do not use this model for commercial purposes. You can support Fish Audio by using the official API at [fish.audio](https://fish.audio/auth/).
|
| 33 |
+
- These weights WILL NOT work with the [official Fish Speech inference code](https://github.com/fishaudio/fish-speech)!
|
| 34 |
+
|
| 35 |
+
---
|
| 36 |
+
# ORIGINAL README: Fish Speech V1.5
|
| 37 |
+
|
| 38 |
+
**Fish Speech V1.5** is a leading text-to-speech (TTS) model trained on more than 1 million hours of audio data in multiple languages.
|
| 39 |
+
|
| 40 |
+
Supported languages:
|
| 41 |
+
- English (en) >300k hours
|
| 42 |
+
- Chinese (zh) >300k hours
|
| 43 |
+
- Japanese (ja) >100k hours
|
| 44 |
+
- German (de) ~20k hours
|
| 45 |
+
- French (fr) ~20k hours
|
| 46 |
+
- Spanish (es) ~20k hours
|
| 47 |
+
- Korean (ko) ~20k hours
|
| 48 |
+
- Arabic (ar) ~20k hours
|
| 49 |
+
- Russian (ru) ~20k hours
|
| 50 |
+
- Dutch (nl) <10k hours
|
| 51 |
+
- Italian (it) <10k hours
|
| 52 |
+
- Polish (pl) <10k hours
|
| 53 |
+
- Portuguese (pt) <10k hours
|
| 54 |
+
|
| 55 |
+
Please refer to [Fish Speech Github](https://github.com/fishaudio/fish-speech) for more info.
|
| 56 |
+
Demo available at [Fish Audio](https://fish.audio/).
|
| 57 |
+
|
| 58 |
+
## Citation
|
| 59 |
+
|
| 60 |
+
If you found this repository useful, please consider citing this work:
|
| 61 |
+
|
| 62 |
+
```
|
| 63 |
+
@misc{fish-speech-v1.4,
|
| 64 |
+
title={Fish-Speech: Leveraging Large Language Models for Advanced Multilingual Text-to-Speech Synthesis},
|
| 65 |
+
author={Shijia Liao and Yuxuan Wang and Tianyu Li and Yifan Cheng and Ruoyi Zhang and Rongzhi Zhou and Yijin Xing},
|
| 66 |
+
year={2024},
|
| 67 |
+
eprint={2411.01156},
|
| 68 |
+
archivePrefix={arXiv},
|
| 69 |
+
primaryClass={cs.SD},
|
| 70 |
+
url={https://arxiv.org/abs/2411.01156},
|
| 71 |
+
}
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
## License
|
| 75 |
+
|
| 76 |
+
This model is permissively licensed under the BY-CC-NC-SA-4.0 license.
|