jsbeaudry/Sidon-ONNX-web

Sidon vocoder for the browser

A 53 MB build of the Sidon v0.1 vocoder (the DAC decoder that turns cleaned speech features into 48 kHz audio). It runs with ONNX Runtime Web on WebGPU and on WebAssembly. It goes with the int8 predictor of soniqo/Sidon-ONNX, so the browser downloads 248 MB in total instead of 405 MB.

It is soniqo's fp32 vocoder (210 MB) with two changes, made by web_vocoder.py:

1. weight norm folded into plain conv weights (the original recomputes them from weight_v and weight_g on every run);
2. the 33 conv weights stored as int8, one scale per output channel, turned back into fp32 by DequantizeLinear. All the math stays fp32.

The fp16 vocoder in soniqo's int8/ folder computes in fp16, and on WebGPU its output comes out garbled. This one does not have that problem.

Checks

Seven 20-second Kreyòl speech clips went through the same pipeline (high-pass, w2v-BERT features, int8 predictor, vocoder, −16 LUFS), once with each vocoder:

vocoder	size	UTMOS	DNSMOS overall	speaker similarity	words heard by an ASR (284 in the originals)
fp32 (soniqo)	210 MB	2.341	3.286	0.794	277
this one	53 MB	2.340	3.281	0.797	281

Its waveform correlates 0.999 with the fp32 vocoder's (SNR 28 dB). On an M3 Pro, a 20-second clip takes about 3 s in the vocoder on WebGPU and about 12 s on WebAssembly with 4 threads, the same as fp32. The browser output matches ONNX Runtime on the CPU (correlation 1.000). UTMOS is trained on English, so read its numbers as a comparison between versions, not as an absolute grade.

Use

import * as ort from 'onnxruntime-web/webgpu';
const opts = { executionProviders: ['webgpu'] };   // or ['wasm']
const pred = await ort.InferenceSession.create('https://huggingface.co/soniqo/Sidon-ONNX/resolve/main/int8/sidon-predictor.onnx', opts);
const voc = await ort.InferenceSession.create('https://huggingface.co/jsbeaudry/Sidon-ONNX-web/resolve/main/sidon-vocoder-int8.onnx', opts);
// input_features [1, T, 160]: the w2v-BERT 2.0 (SeamlessM4T) log-mel features of 16 kHz audio
const p = await pred.run({ input_features });
const { audio } = await voc.run({ features: p[pred.outputNames[0]] });   // [1, M], 48 kHz

License

MIT, like Sidon v0.1 (sarulab-speech) and the ONNX export by soniqo it is built from.