Add model card for Sol

README.md

@@ -1,113 +1,130 @@
 ---
-language: en
+language:
+  - en
+  - es
+  - fr
+  - hi
+  - it
+  - pt
+  - zh
 license: apache-2.0
 library_name: transformers
 pipeline_tag: text-to-speech
-base_model: microsoft/speecht5_tts
-datasets:
-  - openslr/librispeech_asr
 tags:
   - text-to-speech
   - tts
+  - multilingual
   - speecht5
+  - kokoro
   - sol
-  - speech-synthesis
 ---
 
 # Sol
 
-**Sol** is a text-to-speech model that turns written text into clear, natural
-sounding speech. It is a `SpeechT5` acoustic model that was **fine-tuned on
-public voice data** (a single-speaker subset of
-[LibriSpeech](https://www.openslr.org/12), speaker `2277`) and paired with the
-open `speecht5_hifigan` neural vocoder.
+**Sol** is an open text-to-speech engine: type text, get clear speech.
 
-Sol was trained end-to-end by this project — data collection, preprocessing,
-fine-tuning, evaluation and packaging — and it runs on CPU.
+It comes in two parts:
 
-## Model details
+1. **The live engine** — a multilingual, multi-voice TTS service running on
+   [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) with **49 voices across
+   8 languages**. Self-hosted, CPU-only, and reachable 24/7.
+2. **Sol v1** — a `SpeechT5` acoustic model we fine-tuned on public
+   LibriSpeech voice data, whose weights live in **this repository**.
 
-| | |
-|---|---|
-| Architecture | SpeechT5 (Transformer encoder–decoder TTS + HiFi-GAN vocoder) |
-| Base checkpoint | [`microsoft/speecht5_tts`](https://huggingface.co/microsoft/speecht5_tts) |
-| Vocoder | [`microsoft/speecht5_hifigan`](https://huggingface.co/microsoft/speecht5_hifigan) |
-| Training data | LibriSpeech `clean` / `validation`, speaker `2277` (public) |
-| Utterances | 87 |
-| Fine-tune | light: 80 optimizer steps, AdamW, lr 2e-6 (CPU) |
-| Voice | single speaker x-vector (`speaker_embedding.npy`, shipped) |
-| Sampling rate | 16 kHz |
-| Language | English |
+**Live demo:** <https://sol-tts.vercel.app> — type a prompt and hear it speak.
 
-## Quality
+## Voices & languages
 
-Generated speech is **clean and intelligible**. On the six example clips below,
-Whisper (`openai/whisper-tiny`) transcribes Sol's output at **~6.5% WER**, with
-several clips at 0% error. Outputs are trimmed of silence and peak-normalised.
+| Language | Code | Voices |
+|---|---|---|
+| English (US) | `en-us` | 20 |
+| English (UK) | `en-gb` | 8 |
+| Chinese (Mandarin) | `zh` | 8 |
+| Hindi | `hi` | 4 |
+| Spanish | `es` | 3 |
+| Portuguese (Brazil) | `pt-br` | 3 |
+| Italian | `it` | 2 |
+| French | `fr-fr` | 1 |
 
-## Example audio
+**49 voices total.** Voice ids look like `af_heart` (American female), `am_michael`
+(American male), `bf_emma` (British female), `ef_dora` (Spanish female), and so on.
 
-These clips were generated by Sol:
+## Use the live API
 
-<audio controls src="examples/hello.mp3"></audio>
+```bash
+curl -X POST "$SOL_API/api/tts" \
+  -H "Content-Type: application/json" \
+  -d '{"text":"hello from sol","voice":"af_heart","speed":1.0}' \
+  --output sol.wav
+```
 
-> "hello, i am sol. i can turn any text you give me into natural sounding speech."
+| Endpoint | Method | Purpose |
+|---|---|---|
+| `/api/health` | GET | service status |
+| `/api/languages` | GET | supported languages + voice counts |
+| `/api/voices` | GET | every voice (optional `?lang=en-us`) |
+| `/api/tts` | POST | `{text, voice, speed}` → `audio/wav` (24 kHz) |
 
-<audio controls src="examples/about.mp3"></audio>
+Audio is silence-trimmed and peak-normalised, so output is clean with no static.
 
-> "sol is a small, open text to speech model, fine tuned on public domain voice recordings."
+### Multilingual samples
 
-<audio controls src="examples/ai.mp3"></audio>
+A sample in each language lives in
+[`examples/multilingual/`](./examples/multilingual):
 
-> "artificial intelligence is changing how we create and interact with sound."
+<audio controls src="examples/multilingual/en-us.wav"></audio>
+<audio controls src="examples/multilingual/es.wav"></audio>
+<audio controls src="examples/multilingual/zh.wav"></audio>
 
-More clips are in the [`examples/`](./examples) folder.
+## Sol v1 — the fine-tuned SpeechT5 model
 
-## Usage
+This repository also ships **Sol v1**, a `SpeechT5` model fine-tuned on a
+single-speaker subset of [LibriSpeech](https://www.openslr.org/12) (87 utterances,
+light fine-tune, lr 2e-6) and paired with the `speecht5_hifigan` vocoder.
 
 ```python
-import numpy as np
-import soundfile as sf
-import torch
+import numpy as np, soundfile as sf, torch
 from transformers import SpeechT5ForTextToSpeech, SpeechT5HifiGan, SpeechT5Processor
 
 repo = "itriedcoding/Sol"
 
 processor = SpeechT5Processor.from_pretrained(repo)
-model = SpeechT5ForTextToSpeech.from_pretrained(repo)
-vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")
+model     = SpeechT5ForTextToSpeech.from_pretrained(repo)
+vocoder   = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")
 
-# Sol's voice is shipped with the model
 speaker = torch.tensor(np.load("speaker_embedding.npy")).unsqueeze(0)
 
-text = "hello, i am sol."
-inputs = processor(text=text, return_tensors="pt")
-
+inputs = processor(text="hello, i am sol.", return_tensors="pt")
 with torch.no_grad():
     speech = model.generate_speech(inputs["input_ids"], speaker, vocoder=vocoder)
 
 sf.write("sol.wav", speech.numpy(), samplerate=16000)
 ```
 
-> **Tip:** keep the speaker embedding at its original scale. It is an
-> x-vector with norm ≈ 320 — do **not** unit-normalise it, or the speaker
-> conditioning weakens and the audio degrades.
+> Keep the speaker embedding at its original scale — it is a raw x-vector with
+> norm ≈ 320. Do **not** unit-normalise it, or the voice degrades.
+
+## Self-hosting
+
+The engine is served from a FastAPI app (`server/app.py`) that loads Kokoro-82M
+locally and exposes the endpoints above on `127.0.0.1:8000`, then published with a
+Cloudflare tunnel. It starts automatically at logon and runs without a GPU.
 
 ## Intended use & limitations
 
-- **Intended use:** research, prototyping and demos of English speech synthesis.
-- **Limitations:** single English speaker; no multi-speaker or multilingual
-  support; may mispronounce rare names, numbers and unusual punctuation.
-- Output is not a human voice and must not be presented as one, nor used for
-  impersonation or deceptive audio.
+- **Intended use:** research, prototyping, accessibility and demos.
+- **Limitations:** 8 languages; may mispronounce rare names, numbers and unusual
+  punctuation; Japanese is not currently enabled (G2P dependency).
+- Output is not a human voice and must not be used for impersonation or deceptive
+  audio.
 
-## Training data & credits
+## Credits
 
-- Voice data: [LibriSpeech](https://www.openslr.org/12) (`openslr/librispeech_asr`), CC BY 4.0.
+- Engine: [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) by hexgrad.
+- SpeechT5 v1 voice data: [LibriSpeech](https://www.openslr.org/12), CC BY 4.0.
 - Base model & vocoder: Microsoft Research Asia, via 🤗 Transformers.
-- Fine-tuned and packaged by the **Sol** project.
+- Fine-tuned, hosted and packaged by the **Sol** project.
 
 ## License
 
-Apache-2.0, following the base model. Please respect the original dataset and
-base-model licenses.
+Apache-2.0. Please respect the original dataset, base-model and engine licenses.