Reverb-TTS (ONNX export)

Unofficial ONNX export of the transformer of software-mansion/Reverb-TTS (297M masked-diffusion TTS derived from k2-fsa/OmniVoice, Qwen3 layers), so it runs in ONNX Runtime (CPU or Vulkan/WebGPU) without PyTorch. Weights are exported from the fp16 checkpoint and stored as fp32 graphs. License: CC BY-NC 4.0 (non-commercial, attribution), as the original. Not affiliated with Software Mansion. See NOTICE.

The audio codec is not in this repo: it is the Higgs Audio v2 tokenizer, exported separately in melser/Higgs-Audio-2-Tokenizer-onnx under its own license.

Files (onnx/)

Graph Inputs -> outputs Size
core.onnx x[1,S,1024], past_k/past_v[24,1,6,Pp,128] (Pp may be 0), bias[S,Pp+S] -> hidden, new_k, new_v 1.06 GB
embed_prefix.onnx text_ids[1,Pt], ref_codes[1,8,R] -> x[1,P,1024] 98 MB
embed_target.onnx tokens[1,8,F] -> x[1,F,1024] 34 MB
heads.onnx hidden[1,F,1024] -> log_probs[1,8,F,1024] (fp32, mask id excluded) 34 MB

SHA256SUMS lists checksums. Tokenizer, preset voices (voices/*.json) and the Spanish/English prompt format come from the original repo; the sampler is reverb_tts/sampler.py (pip install reverb-tts).

How the graphs are used

  1. Prompt <|denoise|><|lang_start|>{lang}<|lang_end|><|instruct_start|>None<|instruct_end|><|text_start|>{reference_text} {text}<|text_end|> -> token ids; reference audio -> 8-codebook codes with the codec encoder.
  2. Prefix: embed_prefix -> core (empty past, zero bias) -> keep new_k/new_v.
  3. 8 main + 2 polish passes over the fully masked [8,F] target: embed_target -> core (past = prefix K/V; bias 0, or polish_bias=1.0 on the reference columns in polish passes) -> heads; unmask/remask per sampler.py.
  4. Codes -> waveform with the codec decoder (24 kHz).

Verification (CPU, 2026-11-20)

  • Driving the official reverb_tts.sampler.sample() through these graphs gives 100 % identical codes (all 8 codebooks, two prompts) and a bit-identical waveform versus the PyTorch path.
  • AMD BC-250 (Vulkan/RADV, ORT WebGPU EP): core/heads match CPU (argmax 100 %), 4-6x faster. Keep the codec decoder on CPU there (it returns near-silence on that GPU).
  • Spanish word error rate (Whisper small, 4 texts x 6 seeds): 0.000 with presets es_male_standard_1/2, 0.031 cloned from a clean 7 s clip.

Conversion scripts and parity checks are in scripts/ (requirements.txt pins the versions used).

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for melser/Reverb-TTS-onnx

Finetuned
Qwen/Qwen3-0.6B
Finetuned
k2-fsa/OmniVoice
Quantized
(3)
this model