Cloned from pnnbao-ump/VieNeu-TTS

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer_config.json filter=lfs diff=lfs merge=lfs -text
+VieNeu-TTS-F16.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,151 @@
+---
+license: apache-2.0
+datasets:
+- pnnbao-ump/VieNeu-TTS-1000h
+- pnnbao-ump/VieNeuCodec-dataset
+- pnnbao-ump/VieNeu-TTS-140h
+language:
+- vi
+base_model:
+- neuphonic/neutts-air
+pipeline_tag: text-to-speech
+---
+
+# VieNeu-TTS
+
+[![GitHub](https://img.shields.io/badge/GitHub-Repository-blue)](https://github.com/pnnbao97/VieNeu-TTS)
+[![Model](https://img.shields.io/badge/Hugging%20Face-Model-yellow)](https://huggingface.co/pnnbao-ump/VieNeu-TTS)
+
+
+![Untitled](https://cdn-uploads.huggingface.co/production/uploads/68b923a86c86c127a1975eda/vd7kW8h7ooSafcIhEQtyr.png)
+
+## Overview
+
+**VieNeu-TTS** is an advanced on-device Vietnamese Text-to-Speech (TTS) model with **instant voice cloning**.  
+
+Trained on ~1000 hours of high-quality Vietnamese speech, this model represents a significant upgrade from VieNeu-TTS-140h with the following improvements:
+
+- **Enhanced pronunciation**: More accurate and stable Vietnamese pronunciation
+- **Code-switching support**: Seamless transitions between Vietnamese and English
+- **Better voice cloning**: Higher fidelity and speaker consistency
+- **Real-time synthesis**: 24 kHz waveform generation on CPU or GPU
+
+VieNeu-TTS-1000h delivers production-ready speech synthesis fully offline.
+
+**Author:** Phạm Nguyễn Ngọc Bảo
+
+## Support This Project
+
+Training high-quality TTS models requires significant GPU resources and compute time. If you find this model useful, please consider supporting the development:
+
+[![Buy Me a Coffee](https://img.shields.io/badge/Buy%20Me%20a%20Coffee-Support-orange?logo=buy-me-a-coffee)](https://buymeacoffee.com/pnnbao)
+
+Your support helps maintain and improve VieNeu-TTS! 🙏
+
+---
+
+## Voice Cloning Inference
+
+**Reference Voice (Speaker Example):**  
+<audio controls src="https://cdn-uploads.huggingface.co/production/uploads/68b923a86c86c127a1975eda/Rpw1V6X1px59SWQKn_W9D.wav"></audio>
+
+**Input Text:**  
+> Trên bầu trời xanh thẳm, những đám mây trắng lửng lờ trôi như những chiếc thuyền nhỏ đang lướt nhẹ theo dòng gió. Dưới mặt đất, cánh đồng lúa vàng rực trải dài tới tận chân trời, những bông lúa nghiêng mình theo từng làn gió.
+
+**Generated Output (Cloned Voice):**  
+<audio controls src="https://cdn-uploads.huggingface.co/production/uploads/68b923a86c86c127a1975eda/f40t4ueGqmsGDmNIGcU3J.mpga"></audio>
+
+---
+
+## Installation
+
+```bash
+git clone https://github.com/pnnbao97/VieNeu-TTS.git
+cd VieNeu-TTS
+uv sync
+```
+
+## Gradio Demo
+
+```bash
+uv run gradio_app.py
+```
+Open your browser at `http://127.0.0.1:7860`.
+
+**Demo Video:**
+
+
+<video controls src="https://cdn-uploads.huggingface.co/production/uploads/68b923a86c86c127a1975eda/pRsUExICceCh47dgu4P8I.mp4" width="100%"></video>
+
+## Reference Voices
+
+| File                    | Gender | Accent | Description        |
+|-------------------------|--------|--------|--------------------|
+| Bình (nam miền Bắc)     | Male   | North  | Male voice, North accent |
+| Tuyên (nam miền Bắc)    | Male   | North  | Male voice, North accent |
+| Nguyên (nam miền Nam)   | Male   | South  | Male voice, South accent |
+| Sơn (nam miền Nam)      | Male   | South  | Male voice, South accent |
+| Vĩnh (nam miền Nam)     | Male   | South  | Male voice, South accent |
+| Hương (nữ miền Bắc)     | Female | North  | Female voice, North accent |
+| Ly (nữ miền Bắc)        | Female | North  | Female voice, North accent |
+| Ngọc (nữ miền Bắc)      | Female | North  | Female voice, North accent |
+| Đoan (nữ miền Nam)      | Female | South  | Female voice, South accent |
+| Dung (nữ miền Nam)      | Female | South  | Female voice, South accent |
+
+---
+
+## Model Architecture
+
+| Component | Description |
+|----------|-------------|
+| Backbone | Qwen 0.5B (chat-format LM) |
+| Codec    | NeuCodec (supports ONNX + quantization) |
+| Output   | 24 kHz waveform synthesis |
+| Context Window | 2048 tokens shared text + speech |
+| Watermark | Enabled |
+| Training Data | VieNeuCodec-dataset + Emilia dataset pretraining |
+
+## Features
+
+- High-quality Vietnamese speech
+- Instant **voice cloning** (3–5 second reference audio)
+- Fully **offline**
+- Runs real-time or faster
+- Multi-voice reference support
+- Python API + CLI + Gradio
+
+## Troubleshooting
+
+| Issue | Cause | Solution |
+|------|-------|----------|
+| Missing `libespeak` | System dependency | Install eSpeak NG |
+| GPU OOM | VRAM too small | Use CPU or quantized model |
+| Poor voice match | Bad reference sample | Try a clearer reference clip |
+
+## License
+
+Apache 2.0
+
+## Citation
+
+```bibtex
+@misc{vieneutts2025,
+  title        = {VieNeu-TTS: Vietnamese Text-to-Speech with Instant Voice Cloning},
+  author       = {Pham Nguyen Ngoc Bao},
+  year         = {2025},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/pnnbao-ump/VieNeu-TTS}}
+}
+```
+
+Please also cite the base model:
+
+```bibtex
+@misc{neuttsair2025,
+  title        = {NeuTTS Air: On-Device Speech Language Model with Instant Voice Cloning},
+  author       = {Neuphonic},
+  year         = {2025},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/neuphonic/neutts-air}}
+}
+```

config.json

@@ -0,0 +1,54 @@
+{
+  "architectures": [
+    "Qwen2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 896,
+  "initializer_range": 0.02,
+  "intermediate_size": 4864,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "max_window_layers": 21,
+  "model_type": "qwen2",
+  "num_attention_heads": 14,
+  "num_hidden_layers": 24,
+  "num_key_value_heads": 2,
+  "pad_token_id": 151645,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "4.56.0",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 217652
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151645,
+  "repetition_penalty": 1.1,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "4.56.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e46be7e7cea1a3431e55f2f2c988e5e531724ec63c1e585079126567fa4d5633
+size 1105860992

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74c466530bd698626a5b6a424d204711c58dfff0a6b3dd8b4dbac1e1e8c9aa87
+size 24140239

tokenizer_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:058efca33cdb0e871f3e4f46c162cf3231e709129c0a024fb83317bffccd32f0
+size 12063181