itriedcoding commited on
Commit
ecd462c
·
verified ·
1 Parent(s): 849b6b2

Add model card for Sol

Browse files
Files changed (1) hide show
  1. README.md +107 -0
README.md ADDED
@@ -0,0 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ license: apache-2.0
4
+ library_name: transformers
5
+ pipeline_tag: text-to-speech
6
+ base_model: microsoft/speecht5_tts
7
+ datasets:
8
+ - openslr/librispeech_asr
9
+ tags:
10
+ - text-to-speech
11
+ - tts
12
+ - speecht5
13
+ - sol
14
+ - speech-synthesis
15
+ ---
16
+
17
+ # Sol
18
+
19
+ **Sol** is a text-to-speech model that turns written text into natural sounding
20
+ speech. It is a `SpeechT5` acoustic model that was **fine-tuned on public voice
21
+ data** (a single-speaker subset of [LibriSpeech](https://www.openslr.org/12),
22
+ speaker `2277`) and paired with the open `speecht5_hifigan` neural vocoder.
23
+
24
+ Sol was trained end-to-end by this project: data collection, preprocessing,
25
+ fine-tuning, evaluation and packaging. Training ran on CPU — no GPU required to
26
+ reproduce.
27
+
28
+ ## Model details
29
+
30
+ | | |
31
+ |---|---|
32
+ | Architecture | SpeechT5 (Transformer encoder–decoder TTS + HiFi-GAN vocoder) |
33
+ | Base checkpoint | [`microsoft/speecht5_tts`](https://huggingface.co/microsoft/speecht5_tts) |
34
+ | Vocoder | [`microsoft/speecht5_hifigan`](https://huggingface.co/microsoft/speecht5_hifigan) |
35
+ | Training data | LibriSpeech `clean` / `validation`, speaker `2277` (public) |
36
+ | Utterances | 87 |
37
+ | Optimizer steps | ~250 (CPU, AdamW, lr 1e-5 → 5e-6) |
38
+ | Sampling rate | 16 kHz |
39
+ | Language | English |
40
+ | Voice | single speaker via a fixed x-vector (`speaker_embedding.npy`) |
41
+
42
+ ## Example audio
43
+
44
+ These clips were generated by Sol (`<audio>` elements play on the Hub):
45
+
46
+ <audio controls src="examples/hello.mp3"></audio>
47
+
48
+ > "hello, i am sol. i can turn any text you give me into natural sounding speech."
49
+
50
+ <audio controls src="examples/about.mp3"></audio>
51
+
52
+ > "sol is a small, open text to speech model, fine tuned on public domain voice recordings."
53
+
54
+ <audio controls src="examples/ai.mp3"></audio>
55
+
56
+ > "artificial intelligence is changing how we create and interact with sound."
57
+
58
+ More clips are in the [`examples/`](./examples) folder.
59
+
60
+ ## Usage
61
+
62
+ ```python
63
+ import numpy as np
64
+ import soundfile as sf
65
+ import torch
66
+ from transformers import SpeechT5ForTextToSpeech, SpeechT5HifiGan, SpeechT5Processor
67
+
68
+ repo = "itriedcoding/Sol"
69
+
70
+ processor = SpeechT5Processor.from_pretrained(repo)
71
+ model = SpeechT5ForTextToSpeech.from_pretrained(repo)
72
+ vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")
73
+
74
+ speaker = torch.tensor(np.load("speaker_embedding.npy")).unsqueeze(0) # shipped with the model
75
+
76
+ text = "hello, i am sol."
77
+ inputs = processor(text=text, return_tensors="pt")
78
+
79
+ with torch.no_grad():
80
+ speech = model.generate_speech(inputs["input_ids"], speaker, vocoder=vocoder)
81
+
82
+ sf.write("sol.wav", speech.numpy(), samplerate=16000)
83
+ ```
84
+
85
+ The `speaker_embedding.npy` file is included in this repository so you get Sol's
86
+ voice without needing a separate speaker-encoding model.
87
+
88
+ ## Intended use & limitations
89
+
90
+ - **Intended use:** research, prototyping and demos of English speech synthesis.
91
+ - **Limitations:** single English speaker; no multi-speaker or multilingual
92
+ support; may mispronounce rare names, numbers and unusual punctuation; can
93
+ occasionally cut a sentence short. Output is not a human voice and should not
94
+ be presented as one.
95
+ - Do not use for impersonation or deceptive audio.
96
+
97
+ ## Training data & credits
98
+
99
+ - Voice data: [LibriSpeech](https://www.openslr.org/12) (`openslr/librispeech_asr`),
100
+ released under CC BY 4.0.
101
+ - Base model & vocoder: Microsoft Research Asia, via 🤗 Transformers.
102
+ - Fine-tuned and packaged by the **Sol** project.
103
+
104
+ ## License
105
+
106
+ Apache-2.0, following the base model. Please respect the original dataset and
107
+ base-model licenses.