Sol / README.md

Add model card for Sol

d77d328 verified 4 days ago

4.17 kB

language:
  - en
  - es
  - fr
  - hi
  - it
  - pt
  - zh
license: apache-2.0
library_name: transformers
pipeline_tag: text-to-speech
tags:
  - text-to-speech
  - tts
  - multilingual
  - speecht5
  - kokoro
  - sol

Sol

Sol is an open text-to-speech engine: type text, get clear speech.

It comes in two parts:

The live engine — a multilingual, multi-voice TTS service running on Kokoro-82M with 49 voices across 8 languages. Self-hosted, CPU-only, and reachable 24/7.
Sol v1 — a SpeechT5 acoustic model we fine-tuned on public LibriSpeech voice data, whose weights live in this repository.

Live demo: https://sol-tts.vercel.app — type a prompt and hear it speak.

Voices & languages

Language	Code	Voices
English (US)	`en-us`	20
English (UK)	`en-gb`	8
Chinese (Mandarin)	`zh`	8
Hindi	`hi`	4
Spanish	`es`	3
Portuguese (Brazil)	`pt-br`	3
Italian	`it`	2
French	`fr-fr`	1

49 voices total. Voice ids look like af_heart (American female), am_michael (American male), bf_emma (British female), ef_dora (Spanish female), and so on.

Use the live API

curl -X POST "$SOL_API/api/tts" \
  -H "Content-Type: application/json" \
  -d '{"text":"hello from sol","voice":"af_heart","speed":1.0}' \
  --output sol.wav

Endpoint	Method	Purpose
`/api/health`	GET	service status
`/api/languages`	GET	supported languages + voice counts
`/api/voices`	GET	every voice (optional `?lang=en-us`)
`/api/tts`	POST	`{text, voice, speed}` → `audio/wav` (24 kHz)

Audio is silence-trimmed and peak-normalised, so output is clean with no static.

Multilingual samples

A sample in each language lives in examples/multilingual/:

Sol v1 — the fine-tuned SpeechT5 model

This repository also ships Sol v1, a SpeechT5 model fine-tuned on a single-speaker subset of LibriSpeech (87 utterances, light fine-tune, lr 2e-6) and paired with the speecht5_hifigan vocoder.

import numpy as np, soundfile as sf, torch
from transformers import SpeechT5ForTextToSpeech, SpeechT5HifiGan, SpeechT5Processor

repo = "itriedcoding/Sol"

processor = SpeechT5Processor.from_pretrained(repo)
model     = SpeechT5ForTextToSpeech.from_pretrained(repo)
vocoder   = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")

speaker = torch.tensor(np.load("speaker_embedding.npy")).unsqueeze(0)

inputs = processor(text="hello, i am sol.", return_tensors="pt")
with torch.no_grad():
    speech = model.generate_speech(inputs["input_ids"], speaker, vocoder=vocoder)

sf.write("sol.wav", speech.numpy(), samplerate=16000)

Keep the speaker embedding at its original scale — it is a raw x-vector with norm ≈ 320. Do not unit-normalise it, or the voice degrades.

Self-hosting

The engine is served from a FastAPI app (server/app.py) that loads Kokoro-82M locally and exposes the endpoints above on 127.0.0.1:8000, then published with a Cloudflare tunnel. It starts automatically at logon and runs without a GPU.

Intended use & limitations

Intended use: research, prototyping, accessibility and demos.
Limitations: 8 languages; may mispronounce rare names, numbers and unusual punctuation; Japanese is not currently enabled (G2P dependency).
Output is not a human voice and must not be used for impersonation or deceptive audio.

Credits

Engine: Kokoro-82M by hexgrad.
SpeechT5 v1 voice data: LibriSpeech, CC BY 4.0.
Base model & vocoder: Microsoft Research Asia, via 🤗 Transformers.
Fine-tuned, hosted and packaged by the Sol project.

License

Apache-2.0. Please respect the original dataset, base-model and engine licenses.