File size: 4,411 Bytes
f08cce0 e42f808 f08cce0 83da555 e42f808 83da555 f08cce0 e42f808 f08cce0 e42f808 f08cce0 e42f808 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | ---
language:
- en
tags:
- coreml
- core-ml
- openvoice
- voice-conversion
- tone-conversion
- apple
- ios
- macos
license: mit
library_name: coremltools
pipeline_tag: text-to-speech
---
# OpenVoice V2 — ToneColorConverter (Core ML)
Core ML (`.mlpackage`) exports of the **ToneColorConverter** from [myshell-ai/OpenVoiceV2](https://huggingface.co/myshell-ai/OpenVoiceV2), for on-device inference on **macOS** and **iOS** with Apple’s Core ML runtime.
This repository is **private** and intended for on-device iOS / internal workflows. It does **not** include the PyTorch checkpoints or MeloTTS—only the converted Core ML packages and small JSON sidecars describing tensor layouts.
## What’s included
| Artifact | Description |
|----------|-------------|
| `ToneColorConverter_ReferenceEncoder.mlpackage` | Reference mel → tone embedding `g` |
| `ToneColorConverter_PosteriorEncoder.mlpackage` | Base linear STFT → latent `z` |
| `ToneColorConverter_Flow_Forward.mlpackage` | Source tone removal (`z` → `z_p`) |
| `ToneColorConverter_Flow_Reverse.mlpackage` | Target tone injection (`z_p` → `z_hat`) |
| `ToneColorConverter_Decoder.mlpackage` | HiFi-GAN decoder (`z_hat` → waveform) |
| `ToneColorConverter_E2E.mlpackage` | Single-shot pipeline: STFT + `sid_src` / `sid_tgt` → waveform |
| `pipeline_info.json` | Shapes and execution order for the **5-split** path |
| `pipeline_info_e2e.json` | Inputs/outputs for the **E2E** model |
Sampling rate and STFT settings match the OpenVoice V2 converter (e.g. **22050 Hz**, linear STFT layout consistent with the PyTorch reference implementation). See the JSON files for exact ranks and notes.
## Audio samples
Example WAV outputs are in the [`samples/`](https://huggingface.co/aoiandroid/OpenVoiceV2-CoreML-mirror/tree/main/samples) folder (listening tests only):
| File | Description |
|------|-------------|
| `samples/clone_1_text_eng.wav` | MeloTTS English base + Core ML E2E tone conversion; timbre from a **Japanese reference** clip (~5.5 s, 22050 Hz mono). |
| `samples/tts_coreml_out.wav` | Python (`run_openvoice_tts_coreml.py`) + Core ML run. |
| `samples/swift_tts_out.wav` | Swift CLI (`openvoice-coreml-tts`) run. |
See `samples/SAMPLE_INFO.md` for a short legend.
## Intended use
- **Voice / tone conversion** on Apple Silicon or Apple Neural Engine–capable devices, using embeddings extracted from a short reference clip.
- **E2E** path: supply linear magnitude STFT `y`, frame count, and precomputed source/target embeddings (`sid_src`, `sid_tgt`).
- **5-split** path: orchestrate the five models in order (see `pipeline_info.json`) when you need intermediate tensors or tighter control.
**Base speech synthesis** (e.g. MeloTTS) is **not** part of these packages; generate or load a base waveform (or spectrogram) in your app or server, then run ToneColorConverter.
## Limitations
- Posterior path uses a **deterministic** (tau≈0 style) formulation suitable for Core ML export; behaviour may differ slightly from full stochastic sampling in PyTorch.
- You may see `MILCompilerForANE` / E5RT warnings on some macOS versions; inference can still fall back to CPU/GPU.
- Large `.mlpackage` weights are binary blobs—clone with Git LFS if you mirror this repo outside the Hub.
## Download
```bash
export HF_TOKEN="***" # read token with access to this private repo
huggingface-cli download aoiandroid/OpenVoiceV2-CoreML-mirror --local-dir ./OpenVoiceV2-CoreML-on-device iOS client
```
Or with Python:
```python
from huggingface_hub import snapshot_download
snapshot_download(
"aoiandroid/OpenVoiceV2-CoreML-mirror",
local_dir="OpenVoiceV2-CoreML-on-device iOS client",
token=os.environ["HF_TOKEN"],
)
```
## Source & license
- **Upstream model:** [myshell-ai/OpenVoiceV2](https://huggingface.co/myshell-ai/OpenVoiceV2) (research / demo use per upstream terms).
- **OpenVoice code & weights** are subject to the original **MIT** license (see [OpenVoice LICENSE](https://github.com/myshell-ai/OpenVoice/blob/main/LICENSE)).
- This Core ML conversion is a **derivative** packaging for Apple platforms; retain upstream notices when redistributing.
## Citation
If you use OpenVoice, cite the upstream project as in the [official repository](https://github.com/myshell-ai/OpenVoice).
---
*Model card maintained for the on-device iOS client OpenVoice V2 Core ML export (2026).*
|