---
language:
  - en
tags:
  - coreml
  - core-ml
  - openvoice
  - voice-conversion
  - tone-conversion
  - apple
  - ios
  - macos
license: mit
library_name: coremltools
pipeline_tag: text-to-speech
---

# OpenVoice V2 — ToneColorConverter (Core ML)

Core ML (`.mlpackage`) exports of the **ToneColorConverter** from [myshell-ai/OpenVoiceV2](https://huggingface.co/myshell-ai/OpenVoiceV2), for on-device inference on **macOS** and **iOS** with Apple’s Core ML runtime.

This repository is **private** and intended for on-device iOS / internal workflows. It does **not** include the PyTorch checkpoints or MeloTTS—only the converted Core ML packages and small JSON sidecars describing tensor layouts.

## What’s included

| Artifact | Description |
|----------|-------------|
| `ToneColorConverter_ReferenceEncoder.mlpackage` | Reference mel → tone embedding `g` |
| `ToneColorConverter_PosteriorEncoder.mlpackage` | Base linear STFT → latent `z` |
| `ToneColorConverter_Flow_Forward.mlpackage` | Source tone removal (`z` → `z_p`) |
| `ToneColorConverter_Flow_Reverse.mlpackage` | Target tone injection (`z_p` → `z_hat`) |
| `ToneColorConverter_Decoder.mlpackage` | HiFi-GAN decoder (`z_hat` → waveform) |
| `ToneColorConverter_E2E.mlpackage` | Single-shot pipeline: STFT + `sid_src` / `sid_tgt` → waveform |
| `pipeline_info.json` | Shapes and execution order for the **5-split** path |
| `pipeline_info_e2e.json` | Inputs/outputs for the **E2E** model |

Sampling rate and STFT settings match the OpenVoice V2 converter (e.g. **22050 Hz**, linear STFT layout consistent with the PyTorch reference implementation). See the JSON files for exact ranks and notes.

## Audio samples

Example WAV outputs are in the [`samples/`](https://huggingface.co/aoiandroid/OpenVoiceV2-CoreML-mirror/tree/main/samples) folder (listening tests only):

| File | Description |
|------|-------------|
| `samples/clone_1_text_eng.wav` | MeloTTS English base + Core ML E2E tone conversion; timbre from a **Japanese reference** clip (~5.5 s, 22050 Hz mono). |
| `samples/tts_coreml_out.wav` | Python (`run_openvoice_tts_coreml.py`) + Core ML run. |
| `samples/swift_tts_out.wav` | Swift CLI (`openvoice-coreml-tts`) run. |

See `samples/SAMPLE_INFO.md` for a short legend.

## Intended use

- **Voice / tone conversion** on Apple Silicon or Apple Neural Engine–capable devices, using embeddings extracted from a short reference clip.
- **E2E** path: supply linear magnitude STFT `y`, frame count, and precomputed source/target embeddings (`sid_src`, `sid_tgt`).
- **5-split** path: orchestrate the five models in order (see `pipeline_info.json`) when you need intermediate tensors or tighter control.

**Base speech synthesis** (e.g. MeloTTS) is **not** part of these packages; generate or load a base waveform (or spectrogram) in your app or server, then run ToneColorConverter.

## Limitations

- Posterior path uses a **deterministic** (tau≈0 style) formulation suitable for Core ML export; behaviour may differ slightly from full stochastic sampling in PyTorch.
- You may see `MILCompilerForANE` / E5RT warnings on some macOS versions; inference can still fall back to CPU/GPU.
- Large `.mlpackage` weights are binary blobs—clone with Git LFS if you mirror this repo outside the Hub.

## Download

```bash
export HF_TOKEN="***"   # read token with access to this private repo
huggingface-cli download aoiandroid/OpenVoiceV2-CoreML-mirror --local-dir ./OpenVoiceV2-CoreML-on-device iOS client
```

Or with Python:

```python
from huggingface_hub import snapshot_download
snapshot_download(
    "aoiandroid/OpenVoiceV2-CoreML-mirror",
    local_dir="OpenVoiceV2-CoreML-on-device iOS client",
    token=os.environ["HF_TOKEN"],
)
```

## Source & license

- **Upstream model:** [myshell-ai/OpenVoiceV2](https://huggingface.co/myshell-ai/OpenVoiceV2) (research / demo use per upstream terms).
- **OpenVoice code & weights** are subject to the original **MIT** license (see [OpenVoice LICENSE](https://github.com/myshell-ai/OpenVoice/blob/main/LICENSE)).
- This Core ML conversion is a **derivative** packaging for Apple platforms; retain upstream notices when redistributing.

## Citation

If you use OpenVoice, cite the upstream project as in the [official repository](https://github.com/myshell-ai/OpenVoice).

---

*Model card maintained for the on-device iOS client OpenVoice V2 Core ML export (2026).*