| ---
|
| license: apache-2.0
|
| language:
|
| - bg
|
| - en
|
| pipeline_tag: text-to-speech
|
| tags:
|
| - tts
|
| - bulgarian
|
| - miocodec
|
| - encoder-decoder
|
| - voice-cloning
|
| - speech-synthesis
|
| library_name: pytorch
|
| ---
|
|
|
| # BgTTS-38M V2 — Bulgarian Text-to-Speech with Voice Cloning
|
|
|
| A lightweight **38M parameter** encoder-decoder TTS model for **Bulgarian and English** speech synthesis with **zero-shot voice cloning** via [MioCodec](https://huggingface.co/Aratako/MioCodec-25Hz-24kHz).
|
|
|
| **V2 improvements over V1:**
|
| - **Speaker normalization** — stable voice quality across all reference audio files
|
| - **Larger training dataset** — 1,537 hours (vs 1,172h in V1)
|
| - **BF16 training** — more stable gradients, no GradScaler needed
|
| - **Zero dropout** — better utilization of model capacity
|
| - **20 epochs** with careful LR scheduling
|
|
|
| ## Audio Samples
|
|
|
| ### Female Voice (Bulgarian)
|
|
|
| <audio controls src="https://huggingface.co/beleata74/BgTTS-38M-V2/resolve/main/samples/sample_female_bg1.wav"></audio>
|
|
|
| ### Female Voice (English)
|
|
|
| <audio controls src="https://huggingface.co/beleata74/BgTTS-38M-V2/resolve/main/samples/sample_female_en1.wav"></audio>
|
|
|
| ### Male Voice 1 (Bulgarian)
|
|
|
| <audio controls src="https://huggingface.co/beleata74/BgTTS-38M-V2/resolve/main/samples/sample_male_bg1.wav"></audio>
|
|
|
| ### Male Voice 1 (English)
|
|
|
| <audio controls src="https://huggingface.co/beleata74/BgTTS-38M-V2/resolve/main/samples/sample_male_en1.wav"></audio>
|
|
|
| ### Male Voice 2 (Bulgarian)
|
|
|
| <audio controls src="https://huggingface.co/beleata74/BgTTS-38M-V2/resolve/main/samples/sample_male2_bg1.wav"></audio>
|
|
|
| ### Male Voice 2 (English)
|
|
|
| <audio controls src="https://huggingface.co/beleata74/BgTTS-38M-V2/resolve/main/samples/sample_male2_en1.wav"></audio>
|
|
|
| ## Key Features
|
|
|
| - **Bilingual**: Native Bulgarian + English in a single model
|
| - **Voice cloning**: Zero-shot — just provide 3-10 seconds of reference audio
|
| - **Tiny footprint**: 146 MB inference checkpoint, runs on CPU
|
| - **Fast**: RTF ~0.3 on both GPU and CPU (3.3× faster than real-time)
|
| - **Speaker-stable**: V2's normalized speaker embedding ensures consistent quality regardless of reference audio
|
|
|
| ## 🎙️ Voice Cloning
|
|
|
| This model supports zero-shot voice cloning — it can generate speech in any voice given just a short reference audio clip. No fine-tuning needed.
|
|
|
| ### How it Works
|
|
|
| 1. Provide a reference audio (3-10 seconds of clear speech, WAV format, ideally 24kHz)
|
| 2. MioCodec extracts a 128-dimensional speaker embedding (`global_embedding`)
|
| 3. The embedding is **L2-normalized** and scaled by a learned parameter (`spk_scale`) before being added to the decoder
|
| 4. The same embedding is used for MioCodec waveform reconstruction
|
|
|
| ### V2 Improvement: Speaker Normalization
|
|
|
| In V1, the speaker embedding had 7× larger norm than content tokens, causing the model to over-rely on the reference audio for pronunciation quality. V2 normalizes the speaker vector to unit norm, ensuring:
|
| - **Consistent quality** across all reference voices
|
| - The model learns speech patterns from data, not from speaker shortcuts
|
| - Reference audio only affects **timbre**, not articulation
|
|
|
| ## Model Architecture
|
|
|
| | Component | Details |
|
| |---|---|
|
| | Text Encoder | 4-layer bidirectional Transformer (d=384, 6 heads, ff=1536) |
|
| | Audio Decoder | 8-layer causal Transformer (d=384, 6 heads, ff=1536) with cross-attention |
|
| | Speaker Injection | L2-normalized Linear(128 → 384) with learned scale, additive bias |
|
| | Audio Codec | [MioCodec](https://huggingface.co/Aratako/MioCodec-25Hz-24kHz) 25Hz, 1 codebook, 12800 codes, 24kHz output |
|
| | Total Parameters | 38.2M (Encoder: 9.6M, Decoder: 28.6M) |
|
| | Activations | SwiGLU |
|
| | Normalization | RMSNorm (pre-norm) |
|
| | Positional Encoding | Learned (encoder), RoPE (decoder) |
|
| | Embeddings | Tied decoder (lm_head = token_embedding) |
|
| | KV-Cache | Yes (for fast autoregressive inference) |
|
|
|
| ### Tokenizer
|
|
|
| Character-level tokenizer supporting 146 characters:
|
| - Bulgarian Cyrillic (А-Я, а-я)
|
| - English Latin (A-Z, a-z)
|
| - Digits, punctuation, whitespace
|
|
|
| Total vocabulary: **12,955 tokens** (9 special + 146 text + 12,800 audio codes)
|
|
|
| ## Training
|
|
|
| | Parameter | Value |
|
| |---|---|
|
| | **Data** | 728K samples, **1,537 hours** total |
|
| | Bulgarian | ~620K samples (~1,368 hours) |
|
| | English | ~108K samples (~169 hours) |
|
| | **Epochs** | 20 |
|
| | **LR Schedule** | Cosine decay, peak 7e-5, warmup 2 epochs, min 5e-6 |
|
| | **Batch Size** | 64 |
|
| | **Optimizer** | AdamW (betas=0.9, 0.999), weight decay 0.01 |
|
| | **Precision** | BF16 (no GradScaler) |
|
| | **Dropout** | 0.0 (unnecessary — model is 38M, data is 1,537h) |
|
| | **Final Loss** | 5.04 |
|
| | **Hardware** | NVIDIA RTX 5090 (32GB VRAM) |
|
|
|
| ### Why Zero Dropout?
|
|
|
| With only 38M parameters and 138M audio tokens (1,537 hours), the model has **0.28 parameters per token**. Overfitting is mathematically impossible — the model is severely underfitting the data. Dropout only slows convergence without providing any regularization benefit.
|
|
|
| ## Quick Start
|
|
|
| ### Requirements
|
|
|
| ```bash
|
| pip install torch torchaudio soundfile miocodec
|
| ```
|
|
|
| ### Python API
|
|
|
| ```python
|
| import torch
|
| from model import load_for_inference
|
| from tokenizer import TTSTokenizer
|
| from codec import CodecV6
|
| from inference import generate
|
|
|
| device = "cuda" # or "cpu"
|
|
|
| # Load model
|
| model = load_for_inference("checkpoint_inference.pt", device=device)
|
| tokenizer = TTSTokenizer()
|
| codec = CodecV6(device=device)
|
|
|
| # Get speaker embedding from reference audio
|
| ref = codec.encode("reference_speaker.wav")
|
| speaker_emb = ref["global_embedding"].to(device)
|
|
|
| # Generate
|
| codes = generate(
|
| model, tokenizer,
|
| text="Здравейте, как сте днес?",
|
| speaker_emb=speaker_emb,
|
| temperature=0.3,
|
| top_k=250,
|
| max_new_tokens=512,
|
| device=device,
|
| )
|
|
|
| # Decode to audio
|
| if codes is not None:
|
| wav = codec.tokens_to_wav(codes, speaker_emb, "output.wav")
|
| ```
|
|
|
| ### CLI
|
|
|
| ```bash
|
| python inference.py \
|
| --checkpoint checkpoint_inference.pt \
|
| --text "Здравейте, как сте днес?" \
|
| --speaker-wav reference.wav \
|
| --output output.wav \
|
| --temperature 0.3
|
| ```
|
|
|
| ### Web UI (Gradio)
|
|
|
| ```bash
|
| python server.py
|
| # Opens at http://localhost:7860
|
| ```
|
|
|
| ### Parameters
|
|
|
| | Parameter | Default | Description |
|
| |---|---|---|
|
| | `--temperature` | 0.3 | Sampling temperature (lower = stable, higher = expressive) |
|
| | `--top-k` | 250 | Top-k filtering |
|
| | `--top-p` | 0.95 | Nucleus sampling threshold |
|
| | `--rep-penalty` | 1.1 | Repetition penalty on recent tokens |
|
| | `--max-tokens` | 512 | Maximum decoder steps (~20 seconds) |
|
|
|
| **Recommended temperature: 0.3** for clean, stable output. Use 0.5-0.7 for more expressive/varied speech.
|
|
|
| ## ⚠️ Important: Sentence Length
|
|
|
| > The encoder supports up to **256 characters** (~18 seconds of audio). For longer texts, `inference.py` automatically splits by sentence boundaries and concatenates the audio. No manual splitting needed.
|
|
|
| ## Files
|
|
|
| ```
|
| checkpoint_inference.pt # Model weights only (146 MB)
|
| checkpoint.pt # Full checkpoint with optimizer state (438 MB, for continued training)
|
| config.py # Model configuration
|
| model.py # Architecture (TTSEncoderDecoder + speaker normalization)
|
| tokenizer.py # Character-level tokenizer
|
| codec.py # MioCodec wrapper
|
| inference.py # Inference pipeline with KV-cache + sentence splitting
|
| train.py # Training script (BF16)
|
| server.py # Gradio web UI
|
| samples/ # Audio samples (3 voices × 2 languages × 3 texts)
|
| ```
|
|
|
| ## Performance
|
|
|
| ### Benchmarks
|
|
|
| | Hardware | RTF | Speed | Notes |
|
| |---|---|---|---|
|
| | **Intel i3-9100F (CPU)** | **0.30** | **3.3× real-time** | **Windows 10, CPU-only, no GPU** |
|
|
|
| ### CPU-only Deployment (Tested on Windows 10)
|
|
|
| | Component | Disk Space |
|
| |---|---|
|
| | Python venv (PyTorch CPU + deps) | 654 MB |
|
| | BgTTS-38M-V2 (checkpoint + code) | 146 MB |
|
| | MioCodec (auto-downloaded, cached) | 499 MB |
|
| | WavLM base+ (auto-downloaded, cached) | 872 MB |
|
| | **Total** | **2.12 GB** |
|
|
|
| No NVIDIA GPU, no CUDA, no special drivers needed. Works on any x86-64 machine with Python 3.8+.
|
|
|
| ## Comparison with Other Models
|
|
|
| | Model | Parameters | Size | Languages | Voice Cloning | Open Source |
|
| |---|---|---|---|---|---|
|
| | **BgTTS-38M V2** | **38M** | **146 MB** | BG + EN | ✅ | ✅ |
|
| | Kokoro-82M | 82M | ~200 MB | Multi | ❌ | ✅ |
|
| | XTTS-v2 | ~467M | ~1.8 GB | 16 | ✅ | ✅ |
|
| | CSM-1B | 1B | ~4 GB | EN | ✅ | ✅ |
|
| | Dia-1.6B | 1.6B | ~6.4 GB | EN | ✅ | ✅ |
|
|
|
| BgTTS-38M V2 is the **smallest TTS model with voice cloning** we are aware of, and the **only** open-source TTS model with native Bulgarian language support.
|
|
|
| ## Limitations
|
|
|
| - Best with sentences up to ~18 seconds. Longer texts are auto-split by `inference.py`.
|
| - Bulgarian quality is superior to English (82% of training data is Bulgarian).
|
| - Voice cloning quality depends on reference audio clarity — use clean recordings without background noise.
|
| - No explicit prosody control (pitch, speed) — these are implicitly learned from data.
|
| - Character-level tokenizer may struggle with rare Unicode characters outside the supported set.
|
|
|
| ## License
|
|
|
| Apache 2.0
|
|
|
| ## Citation
|
|
|
| ```bibtex
|
| @misc{bgtts38mv2,
|
| title={BgTTS-38M V2: Bulgarian Text-to-Speech with Voice Cloning and Speaker Normalization},
|
| author={beleata74},
|
| year={2026},
|
| url={https://huggingface.co/beleata74/BgTTS-38M-V2}
|
| }
|
| ```
|
|
|
