| --- |
| language: |
| - en |
| tags: |
| - coreml |
| - core-ml |
| - openvoice |
| - voice-conversion |
| - tone-conversion |
| - apple |
| - ios |
| - macos |
| license: mit |
| library_name: coremltools |
| pipeline_tag: text-to-speech |
| --- |
| |
| # OpenVoice V2 — ToneColorConverter (Core ML) |
|
|
| Core ML (`.mlpackage`) exports of the **ToneColorConverter** from [myshell-ai/OpenVoiceV2](https://huggingface.co/myshell-ai/OpenVoiceV2), for on-device inference on **macOS** and **iOS** with Apple’s Core ML runtime. |
|
|
| This repository is **private** and intended for on-device iOS / internal workflows. It does **not** include the PyTorch checkpoints or MeloTTS—only the converted Core ML packages and small JSON sidecars describing tensor layouts. |
|
|
| ## What’s included |
|
|
| | Artifact | Description | |
| |----------|-------------| |
| | `ToneColorConverter_ReferenceEncoder.mlpackage` | Reference mel → tone embedding `g` | |
| | `ToneColorConverter_PosteriorEncoder.mlpackage` | Base linear STFT → latent `z` | |
| | `ToneColorConverter_Flow_Forward.mlpackage` | Source tone removal (`z` → `z_p`) | |
| | `ToneColorConverter_Flow_Reverse.mlpackage` | Target tone injection (`z_p` → `z_hat`) | |
| | `ToneColorConverter_Decoder.mlpackage` | HiFi-GAN decoder (`z_hat` → waveform) | |
| | `ToneColorConverter_E2E.mlpackage` | Single-shot pipeline: STFT + `sid_src` / `sid_tgt` → waveform | |
| | `pipeline_info.json` | Shapes and execution order for the **5-split** path | |
| | `pipeline_info_e2e.json` | Inputs/outputs for the **E2E** model | |
|
|
| Sampling rate and STFT settings match the OpenVoice V2 converter (e.g. **22050 Hz**, linear STFT layout consistent with the PyTorch reference implementation). See the JSON files for exact ranks and notes. |
|
|
| ## Audio samples |
|
|
| Example WAV outputs are in the [`samples/`](https://huggingface.co/aoiandroid/OpenVoiceV2-CoreML-mirror/tree/main/samples) folder (listening tests only): |
|
|
| | File | Description | |
| |------|-------------| |
| | `samples/clone_1_text_eng.wav` | MeloTTS English base + Core ML E2E tone conversion; timbre from a **Japanese reference** clip (~5.5 s, 22050 Hz mono). | |
| | `samples/tts_coreml_out.wav` | Python (`run_openvoice_tts_coreml.py`) + Core ML run. | |
| | `samples/swift_tts_out.wav` | Swift CLI (`openvoice-coreml-tts`) run. | |
|
|
| See `samples/SAMPLE_INFO.md` for a short legend. |
|
|
| ## Intended use |
|
|
| - **Voice / tone conversion** on Apple Silicon or Apple Neural Engine–capable devices, using embeddings extracted from a short reference clip. |
| - **E2E** path: supply linear magnitude STFT `y`, frame count, and precomputed source/target embeddings (`sid_src`, `sid_tgt`). |
| - **5-split** path: orchestrate the five models in order (see `pipeline_info.json`) when you need intermediate tensors or tighter control. |
|
|
| **Base speech synthesis** (e.g. MeloTTS) is **not** part of these packages; generate or load a base waveform (or spectrogram) in your app or server, then run ToneColorConverter. |
|
|
| ## Limitations |
|
|
| - Posterior path uses a **deterministic** (tau≈0 style) formulation suitable for Core ML export; behaviour may differ slightly from full stochastic sampling in PyTorch. |
| - You may see `MILCompilerForANE` / E5RT warnings on some macOS versions; inference can still fall back to CPU/GPU. |
| - Large `.mlpackage` weights are binary blobs—clone with Git LFS if you mirror this repo outside the Hub. |
|
|
| ## Download |
|
|
| ```bash |
| export HF_TOKEN="***" # read token with access to this private repo |
| huggingface-cli download aoiandroid/OpenVoiceV2-CoreML-mirror --local-dir ./OpenVoiceV2-CoreML-on-device iOS client |
| ``` |
|
|
| Or with Python: |
|
|
| ```python |
| from huggingface_hub import snapshot_download |
| snapshot_download( |
| "aoiandroid/OpenVoiceV2-CoreML-mirror", |
| local_dir="OpenVoiceV2-CoreML-on-device iOS client", |
| token=os.environ["HF_TOKEN"], |
| ) |
| ``` |
|
|
| ## Source & license |
|
|
| - **Upstream model:** [myshell-ai/OpenVoiceV2](https://huggingface.co/myshell-ai/OpenVoiceV2) (research / demo use per upstream terms). |
| - **OpenVoice code & weights** are subject to the original **MIT** license (see [OpenVoice LICENSE](https://github.com/myshell-ai/OpenVoice/blob/main/LICENSE)). |
| - This Core ML conversion is a **derivative** packaging for Apple platforms; retain upstream notices when redistributing. |
|
|
| ## Citation |
|
|
| If you use OpenVoice, cite the upstream project as in the [official repository](https://github.com/myshell-ai/OpenVoice). |
|
|
| --- |
|
|
| *Model card maintained for the on-device iOS client OpenVoice V2 Core ML export (2026).* |
|
|
