| # Usage guide | |
| This engine supports **GPT-SoVITS V5 only** — the `v5turbo` (4-step DMD) and `v5dev` | |
| (32-step) models. v2 / v3 / v4 checkpoints are not supported. | |
| ## 1. Get a build | |
| Prebuilt binaries are produced by CI for two platforms, both with the **Vulkan** ggml backend | |
| and the ggml HiFi-GAN vocoder: | |
| | platform | artifact | contents | | |
| |---|---|---| | |
| | Windows x64 | `gsv.cpp-windows-x64-vulkan.zip` | `gsv_server.exe` + ggml DLLs | | |
| | Linux x64 | `gsv.cpp-linux-x64-vulkan.tar.gz` | `gsv_server` + ggml shared libraries | | |
| Download them from the **Actions → latest workflow run → Artifacts** of this repository (or | |
| the Releases page once published). | |
| Runtime requirements: | |
| - A Vulkan 1.2+ driver. CPU-only works too: `--device cpu`. | |
| - The Vulkan backend needs a device with >= 32 KB compute shared memory and working f16 matmul | |
| pipelines. Some newer/experimental drivers (e.g. Moore Threads MUSA on the MTT S70: 16 KB | |
| shared memory, `createComputePipeline` fails for `matmul_f16_f32_f16acc`) cannot run it; use | |
| `--device cpu` there. | |
| - x86-64 with AVX2 for the CPU backend (the published Windows/Linux builds are compiled with | |
| an AVX2 baseline rather than `-march=native`, so they run on any AVX2 CPU). | |
| - Linux binaries are built on Ubuntu 22.04 (glibc 2.35+). | |
| - Python 3.10+ for the text sidecar (`tools/text_sidecar.py`). | |
| ### Reproducibility | |
| The text frontend is deterministic across platforms (identical phones / ert_ids). The AR | |
| sampler is not: the same seed on Windows and Linux yields different token trajectories because | |
| floating-point differences in the logits change the sampled codes, so the two wavs differ in | |
| length and content. Within one platform and build the output is bit-exact (same SHA-256). | |
| ## 2. Get models | |
| Converted models are published at **<https://huggingface.co/KakaruHayate/GSV.CPP>**: | |
| | file | role | notes | | |
| |---|---|---| | |
| | `gsv-ar-f16.gguf` | AR (s1v3) | fp16, reference precision | | |
| | `gsv-ar-q8.gguf` | AR (s1v3) | Q8_0 attention/FFN, ~2× smaller | | |
| | `gsv-cond-f16.gguf` | cond (enc_p / ref_enc / wns1 / rvq) | v5turbo | | |
| | `gsv-dit-f16all.gguf` | DiT / CFM | v5turbo | | |
| | `gsv-refcode.gguf` | reference encoder (CN-HuBERT projection) | v5turbo | | |
| | `gsv-v5dev-cond-f16.gguf` | cond | v5dev | | |
| | `gsv-v5dev-dit-f16all.gguf` | DiT / CFM | v5dev | | |
| | `gsv-v5dev-refcode.gguf` | reference encoder | v5dev | | |
| | `gsv-bert-f16.gguf` | BERT (Chinese RoBERTa) | shared | | |
| | `gsv-hubert-f16.gguf` | CN-HuBERT | shared | | |
| | `gsv-vocoder-f32.gguf` | HiFi-GAN vocoder (48 kHz, ggml, sub-pixel) | shared | | |
| | `resampy_kaiser_best.bin` | resampling filter table | shared | | |
| The vocoder is a GGUF file: the ONNX vocoder (`vocoder_fp32.onnx`) is no longer supported. | |
| `gsv-vocoder-f32.gguf` uses the sub-pixel layout (phase-major `voc.upsub.*` tensors); older | |
| exports with `voc.ups.*` tensors cannot be loaded. | |
| AR is published in two precisions; pick one (q8 is the default choice). The v5dev files are | |
| only needed if you want the 32-step model. All files go into the model directory: | |
| ``` | |
| models/ | |
| gsv-ar-q8.gguf | |
| gsv-cond-f16.gguf | |
| gsv-dit-f16all.gguf | |
| gsv-refcode.gguf | |
| gsv-bert-f16.gguf | |
| gsv-hubert-f16.gguf | |
| gsv-vocoder-f32.gguf | |
| resampy_kaiser_best.bin | |
| ``` | |
| Extra voices (e.g. v5dev or a fine-tuned checkpoint) can be described with a manifest: | |
| ```json | |
| { | |
| "name": "v5dev", | |
| "version": "v5dev", | |
| "ar": "gsv-ar-q8.gguf", | |
| "cond": "gsv-v5dev-cond-f16.gguf", | |
| "dit": "gsv-v5dev-dit-f16all.gguf", | |
| "refcode": "gsv-v5dev-refcode.gguf", | |
| "bert": "gsv-bert-f16.gguf", | |
| "hubert": "gsv-hubert-f16.gguf", | |
| "vocoder": "gsv-vocoder-f32.gguf" | |
| } | |
| ``` | |
| Put it in `models/voices/<name>/manifest.json`; relative paths resolve against that | |
| directory. Switch at runtime with `GET /set_gpt_weights?weights_path=<name>`. | |
| Licenses for the model files are those of the upstream GPT-SoVITS project (MIT); see | |
| [licenses/GPT-SoVITS-LICENSE.txt](../licenses/GPT-SoVITS-LICENSE.txt). | |
| ## 3. Convert your own trained models | |
| Training/fine-tuning happens in the upstream [GPT-SoVITS](https://github.com/RVC-Boss/GPT-SoVITS) | |
| repository. Point the exporter at the resulting checkpoints: | |
| ```bash | |
| # AR (s1v3.ckpt) — two precisions | |
| python tools/export_gguf/export.py --kind ar --ckpt s1v3.ckpt --out models/gsv-ar-f16.gguf | |
| python tools/export_gguf/export.py --kind ar --ckpt s1v3.ckpt --out models/gsv-ar-q8.gguf \ | |
| --spec attn=q8_0,ffn=q8_0 | |
| # cond / DiT / refcode from the SoVITS checkpoint (s2Gv5turbo.pth or s2Gv5dev.pth) | |
| python tools/export_gguf/export.py --kind cond --ckpt s2Gv5turbo.pth --out models/gsv-cond-f16.gguf | |
| python tools/export_gguf/export.py --kind dit --ckpt s2Gv5turbo.pth --out models/gsv-dit-f16all.gguf --spec lin=f16 | |
| python tools/export_gguf/export.py --kind refcode --ckpt s2Gv5turbo.pth --out models/gsv-refcode.gguf | |
| # BERT / HuBERT (or reuse the published files) | |
| python tools/export_gguf/export.py --kind bert --ckpt chinese-roberta-wwm-ext-large --out models/gsv-bert-f16.gguf | |
| python tools/export_gguf/export.py --kind hubert --ckpt chinese-hubert-base --out models/gsv-hubert-f16.gguf | |
| # vocoder (vocoder.pth -> GGUF; F32 weights, phase-major sub-pixel upsampling) | |
| python tools/export_gguf/export_vocoder_gguf.py --ckpt models/vocoder.pth --out models/gsv-vocoder-f32.gguf | |
| # resampling filter table | |
| python tools/export_resampy_filter.py --out models/resampy_kaiser_best.bin | |
| ``` | |
| Notes: | |
| - The exporter requires PyTorch + `gguf`; `--self-test` re-reads the written file and checks | |
| tensor shapes/counts: | |
| `python tools/export_gguf/export.py --kind dit --ckpt ... --out ... --self-test`. | |
| - For a fine-tuned v5 model, keep the same `version` in your voice manifest | |
| (`v5turbo` or `v5dev`) so the default sample steps / CFG match the training recipe. | |
| - Quantization of AR is the only aggressive one; cond/DiT/refcode stay F16/F32. See | |
| `tools/export_gguf/README.md` for the `--spec` syntax. | |
| - Do not convert v2/v3/v4 checkpoints: the graph layout differs and is not supported. | |
| ## 4. Run | |
| The text frontend (G2P) is vendored in `frontend/` (upstream layout, MIT). It needs **no | |
| torch/transformers**; see [frontend/README.md](../frontend/README.md) for the data files | |
| (`G2PWModel`, BERT tokenizer, `nltk_data`) and how to point the sidecar at them. | |
| ```bash | |
| # text frontend (zh only; add frontend/requirements-multi.txt for en/ja/ko/yue) | |
| python -m venv .venv-sidecar | |
| .venv-sidecar/bin/pip install -r frontend/requirements-zh.txt # Windows: .venv-sidecar\Scripts\pip | |
| export GSV_G2PW_MODEL=/path/to/G2PWModel # dir with g2pW.onnx | |
| export GSV_BERT_DIR=/path/to/chinese-roberta-wwm-ext-large # vocab.txt + tokenizer_config.json | |
| export NLTK_DATA=/path/to/.nltk_data # required by en (g2p_en) | |
| .venv-sidecar/bin/python tools/text_sidecar.py --port 9882 | |
| # engine + HTTP service | |
| ./gsv_server --device vulkan --port 9880 | |
| # CPU-only: ./gsv_server --device cpu | |
| ``` | |
| ### Choosing a GPU | |
| With more than one Vulkan device (multi-GPU, or an iGPU next to a dGPU), list them first: | |
| ```bash | |
| ./gsv_server --list-devices | |
| # ggml devices (2): | |
| # #0 GPU Vulkan0 NVIDIA GeForce RTX 2070 (7.2/7.9 GiB free) | |
| # -- CPU CPU Intel(R) Xeon(R) CPU @ 1.80GHz | |
| # use --device vulkan:N (or --device N) for GPU #N, --device cpu for CPU | |
| ``` | |
| Then pass the index: `--device 1` or `--device vulkan:1`. An index that does not exist fails | |
| with `[device] GPU N not found` instead of silently falling back to CPU. | |
| ### Windows one-click package | |
| `python scripts/package_windows.py --out dist/GSV.CPP-win64` assembles a portable folder: | |
| GAG sources, `runtime/gsv` (engine + DLLs + VC runtime), an embeddable Python with the GAG and | |
| frontend dependencies, `runtime/frontend` (sidecar + frontend code), `runtime/models` (GGUF, | |
| G2PW, tokenizer, nltk) and a `Start GAG.bat`. It expects a `GAG` checkout next to this repo. | |
| Useful flags: `--reuse-models` (keep an existing `runtime/models`), `--no-models`, | |
| `--skip-pip` (reuse the embedded Python). | |
| ```bash | |
| curl -X POST http://127.0.0.1:9880/tts -H "Content-Type: application/json" -d @request.json -o out.wav | |
| ``` | |
| `request.json`: | |
| ```json | |
| { | |
| "text": "你好，欢迎使用语音合成系统。", | |
| "text_lang": "zh", | |
| "ref_audio_path": "ref.wav", | |
| "prompt_text": "参考音频对应的文字。", | |
| "prompt_lang": "zh", | |
| "sample_steps": 4, | |
| "cfg_rate": 0.0, | |
| "seed": 20261014 | |
| } | |
| ``` | |
| If `sample_steps` / `cfg_rate` are omitted they follow the voice version (v5turbo 4 / 0.0, | |
| v5dev 32 / 1.30). Bind to loopback only unless `--token <secret>` is set. | |
| ## 5. License | |
| Engine code is MIT (see [LICENSE](../LICENSE)). Third-party components and the model weights | |
| are covered by their own licenses; see [THIRD_PARTY_NOTICES.md](../THIRD_PARTY_NOTICES.md). | |