Instructions to use itriedcoding/Sol with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use itriedcoding/Sol with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-to-speech", model="itriedcoding/Sol")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForTextToSpectrogram processor = AutoProcessor.from_pretrained("itriedcoding/Sol") model = AutoModelForTextToSpectrogram.from_pretrained("itriedcoding/Sol", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add model card for Sol
Browse files
README.md
ADDED
|
@@ -0,0 +1,107 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language: en
|
| 3 |
+
license: apache-2.0
|
| 4 |
+
library_name: transformers
|
| 5 |
+
pipeline_tag: text-to-speech
|
| 6 |
+
base_model: microsoft/speecht5_tts
|
| 7 |
+
datasets:
|
| 8 |
+
- openslr/librispeech_asr
|
| 9 |
+
tags:
|
| 10 |
+
- text-to-speech
|
| 11 |
+
- tts
|
| 12 |
+
- speecht5
|
| 13 |
+
- sol
|
| 14 |
+
- speech-synthesis
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# Sol
|
| 18 |
+
|
| 19 |
+
**Sol** is a text-to-speech model that turns written text into natural sounding
|
| 20 |
+
speech. It is a `SpeechT5` acoustic model that was **fine-tuned on public voice
|
| 21 |
+
data** (a single-speaker subset of [LibriSpeech](https://www.openslr.org/12),
|
| 22 |
+
speaker `2277`) and paired with the open `speecht5_hifigan` neural vocoder.
|
| 23 |
+
|
| 24 |
+
Sol was trained end-to-end by this project: data collection, preprocessing,
|
| 25 |
+
fine-tuning, evaluation and packaging. Training ran on CPU — no GPU required to
|
| 26 |
+
reproduce.
|
| 27 |
+
|
| 28 |
+
## Model details
|
| 29 |
+
|
| 30 |
+
| | |
|
| 31 |
+
|---|---|
|
| 32 |
+
| Architecture | SpeechT5 (Transformer encoder–decoder TTS + HiFi-GAN vocoder) |
|
| 33 |
+
| Base checkpoint | [`microsoft/speecht5_tts`](https://huggingface.co/microsoft/speecht5_tts) |
|
| 34 |
+
| Vocoder | [`microsoft/speecht5_hifigan`](https://huggingface.co/microsoft/speecht5_hifigan) |
|
| 35 |
+
| Training data | LibriSpeech `clean` / `validation`, speaker `2277` (public) |
|
| 36 |
+
| Utterances | 87 |
|
| 37 |
+
| Optimizer steps | ~250 (CPU, AdamW, lr 1e-5 → 5e-6) |
|
| 38 |
+
| Sampling rate | 16 kHz |
|
| 39 |
+
| Language | English |
|
| 40 |
+
| Voice | single speaker via a fixed x-vector (`speaker_embedding.npy`) |
|
| 41 |
+
|
| 42 |
+
## Example audio
|
| 43 |
+
|
| 44 |
+
These clips were generated by Sol (`<audio>` elements play on the Hub):
|
| 45 |
+
|
| 46 |
+
<audio controls src="examples/hello.mp3"></audio>
|
| 47 |
+
|
| 48 |
+
> "hello, i am sol. i can turn any text you give me into natural sounding speech."
|
| 49 |
+
|
| 50 |
+
<audio controls src="examples/about.mp3"></audio>
|
| 51 |
+
|
| 52 |
+
> "sol is a small, open text to speech model, fine tuned on public domain voice recordings."
|
| 53 |
+
|
| 54 |
+
<audio controls src="examples/ai.mp3"></audio>
|
| 55 |
+
|
| 56 |
+
> "artificial intelligence is changing how we create and interact with sound."
|
| 57 |
+
|
| 58 |
+
More clips are in the [`examples/`](./examples) folder.
|
| 59 |
+
|
| 60 |
+
## Usage
|
| 61 |
+
|
| 62 |
+
```python
|
| 63 |
+
import numpy as np
|
| 64 |
+
import soundfile as sf
|
| 65 |
+
import torch
|
| 66 |
+
from transformers import SpeechT5ForTextToSpeech, SpeechT5HifiGan, SpeechT5Processor
|
| 67 |
+
|
| 68 |
+
repo = "itriedcoding/Sol"
|
| 69 |
+
|
| 70 |
+
processor = SpeechT5Processor.from_pretrained(repo)
|
| 71 |
+
model = SpeechT5ForTextToSpeech.from_pretrained(repo)
|
| 72 |
+
vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")
|
| 73 |
+
|
| 74 |
+
speaker = torch.tensor(np.load("speaker_embedding.npy")).unsqueeze(0) # shipped with the model
|
| 75 |
+
|
| 76 |
+
text = "hello, i am sol."
|
| 77 |
+
inputs = processor(text=text, return_tensors="pt")
|
| 78 |
+
|
| 79 |
+
with torch.no_grad():
|
| 80 |
+
speech = model.generate_speech(inputs["input_ids"], speaker, vocoder=vocoder)
|
| 81 |
+
|
| 82 |
+
sf.write("sol.wav", speech.numpy(), samplerate=16000)
|
| 83 |
+
```
|
| 84 |
+
|
| 85 |
+
The `speaker_embedding.npy` file is included in this repository so you get Sol's
|
| 86 |
+
voice without needing a separate speaker-encoding model.
|
| 87 |
+
|
| 88 |
+
## Intended use & limitations
|
| 89 |
+
|
| 90 |
+
- **Intended use:** research, prototyping and demos of English speech synthesis.
|
| 91 |
+
- **Limitations:** single English speaker; no multi-speaker or multilingual
|
| 92 |
+
support; may mispronounce rare names, numbers and unusual punctuation; can
|
| 93 |
+
occasionally cut a sentence short. Output is not a human voice and should not
|
| 94 |
+
be presented as one.
|
| 95 |
+
- Do not use for impersonation or deceptive audio.
|
| 96 |
+
|
| 97 |
+
## Training data & credits
|
| 98 |
+
|
| 99 |
+
- Voice data: [LibriSpeech](https://www.openslr.org/12) (`openslr/librispeech_asr`),
|
| 100 |
+
released under CC BY 4.0.
|
| 101 |
+
- Base model & vocoder: Microsoft Research Asia, via 🤗 Transformers.
|
| 102 |
+
- Fine-tuned and packaged by the **Sol** project.
|
| 103 |
+
|
| 104 |
+
## License
|
| 105 |
+
|
| 106 |
+
Apache-2.0, following the base model. Please respect the original dataset and
|
| 107 |
+
base-model licenses.
|