Add model card (README.md)
Browse files
README.md
ADDED
|
@@ -0,0 +1,83 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
tags:
|
| 5 |
+
- coreml
|
| 6 |
+
- core-ml
|
| 7 |
+
- openvoice
|
| 8 |
+
- voice-conversion
|
| 9 |
+
- tone-conversion
|
| 10 |
+
- apple
|
| 11 |
+
- ios
|
| 12 |
+
- macos
|
| 13 |
+
license: mit
|
| 14 |
+
library_name: coremltools
|
| 15 |
+
pipeline_tag: text-to-speech
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
+
# OpenVoice V2 — ToneColorConverter (Core ML)
|
| 19 |
+
|
| 20 |
+
Core ML (`.mlpackage`) exports of the **ToneColorConverter** from [myshell-ai/OpenVoiceV2](https://huggingface.co/myshell-ai/OpenVoiceV2), for on-device inference on **macOS** and **iOS** with Apple’s Core ML runtime.
|
| 21 |
+
|
| 22 |
+
This repository is **private** and intended for TranslateBlue / internal workflows. It does **not** include the PyTorch checkpoints or MeloTTS—only the converted Core ML packages and small JSON sidecars describing tensor layouts.
|
| 23 |
+
|
| 24 |
+
## What’s included
|
| 25 |
+
|
| 26 |
+
| Artifact | Description |
|
| 27 |
+
|----------|-------------|
|
| 28 |
+
| `ToneColorConverter_ReferenceEncoder.mlpackage` | Reference mel → tone embedding `g` |
|
| 29 |
+
| `ToneColorConverter_PosteriorEncoder.mlpackage` | Base linear STFT → latent `z` |
|
| 30 |
+
| `ToneColorConverter_Flow_Forward.mlpackage` | Source tone removal (`z` → `z_p`) |
|
| 31 |
+
| `ToneColorConverter_Flow_Reverse.mlpackage` | Target tone injection (`z_p` → `z_hat`) |
|
| 32 |
+
| `ToneColorConverter_Decoder.mlpackage` | HiFi-GAN decoder (`z_hat` → waveform) |
|
| 33 |
+
| `ToneColorConverter_E2E.mlpackage` | Single-shot pipeline: STFT + `sid_src` / `sid_tgt` → waveform |
|
| 34 |
+
| `pipeline_info.json` | Shapes and execution order for the **5-split** path |
|
| 35 |
+
| `pipeline_info_e2e.json` | Inputs/outputs for the **E2E** model |
|
| 36 |
+
|
| 37 |
+
Sampling rate and STFT settings match the OpenVoice V2 converter (e.g. **22050 Hz**, linear STFT layout consistent with the PyTorch reference implementation). See the JSON files for exact ranks and notes.
|
| 38 |
+
|
| 39 |
+
## Intended use
|
| 40 |
+
|
| 41 |
+
- **Voice / tone conversion** on Apple Silicon or Apple Neural Engine–capable devices, using embeddings extracted from a short reference clip.
|
| 42 |
+
- **E2E** path: supply linear magnitude STFT `y`, frame count, and precomputed source/target embeddings (`sid_src`, `sid_tgt`).
|
| 43 |
+
- **5-split** path: orchestrate the five models in order (see `pipeline_info.json`) when you need intermediate tensors or tighter control.
|
| 44 |
+
|
| 45 |
+
**Base speech synthesis** (e.g. MeloTTS) is **not** part of these packages; generate or load a base waveform (or spectrogram) in your app or server, then run ToneColorConverter.
|
| 46 |
+
|
| 47 |
+
## Limitations
|
| 48 |
+
|
| 49 |
+
- Posterior path uses a **deterministic** (tau≈0 style) formulation suitable for Core ML export; behaviour may differ slightly from full stochastic sampling in PyTorch.
|
| 50 |
+
- You may see `MILCompilerForANE` / E5RT warnings on some macOS versions; inference can still fall back to CPU/GPU.
|
| 51 |
+
- Large `.mlpackage` weights are binary blobs—clone with Git LFS if you mirror this repo outside the Hub.
|
| 52 |
+
|
| 53 |
+
## Download
|
| 54 |
+
|
| 55 |
+
```bash
|
| 56 |
+
export HF_TOKEN="***" # read token with access to this private repo
|
| 57 |
+
huggingface-cli download aoiandroid/OpenVoiceV2-CoreML-TranslateBlue --local-dir ./OpenVoiceV2-CoreML-TranslateBlue
|
| 58 |
+
```
|
| 59 |
+
|
| 60 |
+
Or with Python:
|
| 61 |
+
|
| 62 |
+
```python
|
| 63 |
+
from huggingface_hub import snapshot_download
|
| 64 |
+
snapshot_download(
|
| 65 |
+
"aoiandroid/OpenVoiceV2-CoreML-TranslateBlue",
|
| 66 |
+
local_dir="OpenVoiceV2-CoreML-TranslateBlue",
|
| 67 |
+
token=os.environ["HF_TOKEN"],
|
| 68 |
+
)
|
| 69 |
+
```
|
| 70 |
+
|
| 71 |
+
## Source & license
|
| 72 |
+
|
| 73 |
+
- **Upstream model:** [myshell-ai/OpenVoiceV2](https://huggingface.co/myshell-ai/OpenVoiceV2) (research / demo use per upstream terms).
|
| 74 |
+
- **OpenVoice code & weights** are subject to the original **MIT** license (see [OpenVoice LICENSE](https://github.com/myshell-ai/OpenVoice/blob/main/LICENSE)).
|
| 75 |
+
- This Core ML conversion is a **derivative** packaging for Apple platforms; retain upstream notices when redistributing.
|
| 76 |
+
|
| 77 |
+
## Citation
|
| 78 |
+
|
| 79 |
+
If you use OpenVoice, cite the upstream project as in the [official repository](https://github.com/myshell-ai/OpenVoice).
|
| 80 |
+
|
| 81 |
+
---
|
| 82 |
+
|
| 83 |
+
*Model card maintained for the TranslateBlue OpenVoice V2 Core ML export (2026).*
|