OpenVoice V2 ONNX (Melo EN/ZH + tone converter)

Private ONNX exports for OpenVoice V2 style inference: MeloTTS base speech (English and Chinese supported via separate Melo exports), tone-color speaker embedding from a reference clip, and tone color conversion to match the reference voice. Built from a local iOS project under Docs/OpenVoiceV2/ (convert_to_onnx.py, convert_onnx_precision.py).

This is not an official MyShell release; it is a derivative export for experimentation and on-device ONNX Runtime workflows.

Supported languages (this export)

Layer	English	Chinese (中文)	Notes
MeloTTS → ONNX	`melotts_en.onnx`	`melotts_zh.onnx`	Same tensor I/O contract; export both with `OPENVOICE_MELO_LANGS=EN,ZH` (see below).
`tone_color_converter.onnx`	✓	✓	Language-agnostic: linear spectrogram + `sid_src` / `sid_tgt` embeddings.
`tone_color_se_extractor.onnx`	✓	✓	Language-agnostic: reference mel (fixed time in export) → 256-d embedding.

Base speaker .pth (OpenVoiceV2 layout): e.g. base_speakers/ses/en-newest.pth for English-style base timbre, base_speakers/ses/zh.pth for Chinese-style base timbre — pick the file that matches your Melo language when doing full OpenVoice-style cloning.

Other Melo languages (ES / FR / JP / KR): same convert_to_onnx.py mechanism with OPENVOICE_MELO_LANGS; filenames follow melotts_{es|fr|jp|kr}.onnx.

Hub snapshot: may contain only melotts_en.onnx until you re-export and re-upload; once melotts_zh.onnx is present under fp32/ (and derived fp16/ / int8/), the same layout applies.

Exporting English + Chinese Melo ONNX

From Docs/OpenVoiceV2 (after checkpoints + deps are installed):

export OPENVOICE_MELO_LANGS=EN,ZH
python convert_to_onnx.py

This produces (alongside shared tone-color ONNX) melotts_en.onnx and melotts_zh.onnx. Then run convert_onnx_precision.py per directory as usual; precision scripts pick up every melotts_*.onnx in the input folder.

Repository layout

Path	Precision	Contents
`fp32/`	FP32	`tone_color_se_extractor.onnx`, `tone_color_converter.onnx`, and one or more `melotts_*.onnx` (e.g. `melotts_en.onnx`, `melotts_zh.onnx`).
`fp16/`	Mixed	Same filenames as `fp32/`. Global FP16 often breaks ORT on large VITS graphs; the script falls back to FP32 per file when ORT cannot load the graph. SE extractor is often the only true FP16 file.
`int8/`	Dynamic INT8 (ORT)	Same filenames. Weight-only dynamic quantization (`MatMul` / `Gemm`, with `MatMul`-only fallback).

Model I/O (summary)

Melo ONNX (`melotts_en.onnx`, `melotts_zh.onnx`, …)

Input	Shape (typical)	Dtype
`phone_ids`	`[1, S]`	int64
`phone_lens`	`[1]`	int64
`sid`	`[1]`	int64
`tones`	`[1, S]`	int64
`lang_ids`	`[1, S]`	int64
`bert_feat`	`[1, 1024, S]`	float32
`ja_bert`	`[1, 768, S]`	float32

Output	Shape	Dtype
`audio`	`[1, 1, T_wav]`	float32

Chinese exports use Melo’s Chinese text pipeline (e.g. jieba + multilingual BERT) at training/export time; ONNX inputs remain the same fields as English.

`tone_color_se_extractor.onnx`

Input	Shape	Dtype
`mel`	`[1, 513, 128]` (fixed time in export)	float32

Output	Shape	Dtype
`speaker_embedding`	`[1, 256]`	float32

`tone_color_converter.onnx`

Input	Shape	Dtype
`y`	`[1, 513, T_mel]`	float32
`y_lengths`	`[1]`	int64
`sid_src`	`[1, 256, 1]`	float32
`sid_tgt`	`[1, 256, 1]`	float32

Output	Shape	Dtype
`audio_out`	`[1, 1, T_audio]`	float32

Sampling rate and STFT hyperparameters follow OpenVoice V2 converter/config.json (typically 22050 Hz, filter_length=1024, hop_length=256, win_length=1024).

Intended use

Research and on-device prototyping with ONNX Runtime (CPU / optional CoreML EP on Apple platforms).
English or Chinese base speech from the matching melotts_*.onnx, then shared SE + converter ONNX.
Pairs with OpenVoice V2 checkpoints from myshell-ai/OpenVoiceV2 — not re-uploaded here.

Out-of-scope / limitations

Not a drop-in replacement for a separate CoreML OpenVoice bundle (different tensor contracts in the app).
No bundled text frontend: run Melo-compatible preprocessing or use tts_onnx_sentence.py (--melo-language EN_NEWEST or ZH, etc.).
FP16 / INT8: validate on your ORT version and EP.

Evaluation

Smoke: smoke_onnx_openvoice.py, smoke_onnx_precision.py (all melotts_*.onnx in the folder).
Sentence TTS (Python): tts_onnx_sentence.py with --melo-language matching the ONNX file.

How to download

pip install huggingface_hub
huggingface-cli download aoiandroid/OpenVoiceV2-ONNX-mirror --local-dir ./openvoice_onnx --include "fp32/*"

Use a token with read access to this private repo.

Citation (upstream)

Cite MyShell OpenVoice and MeloTTS. This Hub repo is a convenience export for on-device experimentation.

日本語サマリー

Melo ONNX: 英語（melotts_en.onnx）と中国語（melotts_zh.onnx）に対応。OPENVOICE_MELO_LANGS=EN,ZH で convert_to_onnx.py から両方エクスポート可能（I/O 形式は共通）。
コンバータ / SE: 言語非依存。参照音声で声色を寄せる用途。
fp32 / fp16 / int8: ディレクトリ内の melotts_*.onnx をすべて精度変換の対象に含めます。
公式ではありません。 upstream のライセンス・条件に従ってください。

Downloads last month: -; Downloads are not tracked for this model. How to track