aoiandroid commited on
Commit
f08cce0
·
verified ·
1 Parent(s): bf0fc09

Add model card (README.md)

Browse files
Files changed (1) hide show
  1. README.md +83 -0
README.md ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - coreml
6
+ - core-ml
7
+ - openvoice
8
+ - voice-conversion
9
+ - tone-conversion
10
+ - apple
11
+ - ios
12
+ - macos
13
+ license: mit
14
+ library_name: coremltools
15
+ pipeline_tag: text-to-speech
16
+ ---
17
+
18
+ # OpenVoice V2 — ToneColorConverter (Core ML)
19
+
20
+ Core ML (`.mlpackage`) exports of the **ToneColorConverter** from [myshell-ai/OpenVoiceV2](https://huggingface.co/myshell-ai/OpenVoiceV2), for on-device inference on **macOS** and **iOS** with Apple’s Core ML runtime.
21
+
22
+ This repository is **private** and intended for TranslateBlue / internal workflows. It does **not** include the PyTorch checkpoints or MeloTTS—only the converted Core ML packages and small JSON sidecars describing tensor layouts.
23
+
24
+ ## What’s included
25
+
26
+ | Artifact | Description |
27
+ |----------|-------------|
28
+ | `ToneColorConverter_ReferenceEncoder.mlpackage` | Reference mel → tone embedding `g` |
29
+ | `ToneColorConverter_PosteriorEncoder.mlpackage` | Base linear STFT → latent `z` |
30
+ | `ToneColorConverter_Flow_Forward.mlpackage` | Source tone removal (`z` → `z_p`) |
31
+ | `ToneColorConverter_Flow_Reverse.mlpackage` | Target tone injection (`z_p` → `z_hat`) |
32
+ | `ToneColorConverter_Decoder.mlpackage` | HiFi-GAN decoder (`z_hat` → waveform) |
33
+ | `ToneColorConverter_E2E.mlpackage` | Single-shot pipeline: STFT + `sid_src` / `sid_tgt` → waveform |
34
+ | `pipeline_info.json` | Shapes and execution order for the **5-split** path |
35
+ | `pipeline_info_e2e.json` | Inputs/outputs for the **E2E** model |
36
+
37
+ Sampling rate and STFT settings match the OpenVoice V2 converter (e.g. **22050 Hz**, linear STFT layout consistent with the PyTorch reference implementation). See the JSON files for exact ranks and notes.
38
+
39
+ ## Intended use
40
+
41
+ - **Voice / tone conversion** on Apple Silicon or Apple Neural Engine–capable devices, using embeddings extracted from a short reference clip.
42
+ - **E2E** path: supply linear magnitude STFT `y`, frame count, and precomputed source/target embeddings (`sid_src`, `sid_tgt`).
43
+ - **5-split** path: orchestrate the five models in order (see `pipeline_info.json`) when you need intermediate tensors or tighter control.
44
+
45
+ **Base speech synthesis** (e.g. MeloTTS) is **not** part of these packages; generate or load a base waveform (or spectrogram) in your app or server, then run ToneColorConverter.
46
+
47
+ ## Limitations
48
+
49
+ - Posterior path uses a **deterministic** (tau≈0 style) formulation suitable for Core ML export; behaviour may differ slightly from full stochastic sampling in PyTorch.
50
+ - You may see `MILCompilerForANE` / E5RT warnings on some macOS versions; inference can still fall back to CPU/GPU.
51
+ - Large `.mlpackage` weights are binary blobs—clone with Git LFS if you mirror this repo outside the Hub.
52
+
53
+ ## Download
54
+
55
+ ```bash
56
+ export HF_TOKEN="***" # read token with access to this private repo
57
+ huggingface-cli download aoiandroid/OpenVoiceV2-CoreML-TranslateBlue --local-dir ./OpenVoiceV2-CoreML-TranslateBlue
58
+ ```
59
+
60
+ Or with Python:
61
+
62
+ ```python
63
+ from huggingface_hub import snapshot_download
64
+ snapshot_download(
65
+ "aoiandroid/OpenVoiceV2-CoreML-TranslateBlue",
66
+ local_dir="OpenVoiceV2-CoreML-TranslateBlue",
67
+ token=os.environ["HF_TOKEN"],
68
+ )
69
+ ```
70
+
71
+ ## Source & license
72
+
73
+ - **Upstream model:** [myshell-ai/OpenVoiceV2](https://huggingface.co/myshell-ai/OpenVoiceV2) (research / demo use per upstream terms).
74
+ - **OpenVoice code & weights** are subject to the original **MIT** license (see [OpenVoice LICENSE](https://github.com/myshell-ai/OpenVoice/blob/main/LICENSE)).
75
+ - This Core ML conversion is a **derivative** packaging for Apple platforms; retain upstream notices when redistributing.
76
+
77
+ ## Citation
78
+
79
+ If you use OpenVoice, cite the upstream project as in the [official repository](https://github.com/myshell-ai/OpenVoice).
80
+
81
+ ---
82
+
83
+ *Model card maintained for the TranslateBlue OpenVoice V2 Core ML export (2026).*