WebASMR TTS (F5-TTS, browser-ready ONNX)
F5-TTS v1 Base fine-tuned on the DownFlow/WebAMSR corpus, exported for onnxruntime-web. This is the acoustic half of WebASMR; the director LLM is a separate repo.
Graphs
| file | inputs | output |
|---|---|---|
dit_step.onnx |
x [1,N,100], cond [1,N,100], text_ids [1,U] i64, time [1], mask [1,N] |
velocity [1,N,100] |
dit_step_fp16.onnx |
same | velocity [1,N,100] (+ auxiliary early outputs) |
vocoder.onnx |
mel [1,100,T] |
stft_planes [1,1026,T] |
vocoder_fp16.onnx |
same | stft_planes [1,1026,T] |
The flow-matching Euler loop is not in the graphs: run dit_step 32 times
with t = linspace(0,1,33), t += -1*(cos(pi/2*t)-1+t) (sway), then the
vocoder, then an ISTFT (n_fft=1024, hop=256, hann, center) on the complex
spectrogram implied by the planes. ONNX has no complex tensors, which is why the
vocoder stops at the linear head.
Zero-shot layout (matches F5-TTS base): cond is the reference mel for the
first R frames (reference length) followed by zeros; x starts as Gaussian
noise for all N = R + G frames; text_ids is ref_text + gen_text tokenized
together; after the flow, frames [R:] are the generated audio. CFG: run the
step graph twice, once as-is and once with cond = 0 and text_ids = -1
(filler token), combine pred + (pred - null) * 2.
dit_step_fp16.onnx is the WebGPU build of the same weights. Two known
onnxruntime-web JSEP issues make the plain fp16 export unusable: the exported
Split chain miscomputes, and a fused early text-mask pattern is miscompiled
(velocity cosine ~0.994 vs onnxruntime CPU, compounding over 32 Euler steps).
The fp16 build rewrites Split -> Slice and exposes the early auxiliary
tensors as graph outputs to disable the bad fusion (velocity cosine 0.99998 vs
CPU). Consumers only read velocity.
vocab.json maps ids for ASCII + numbered pinyin (there are no CJK glyphs;
Chinese must be converted first), pinyin.json is the char table, and
refs/<artist>.{bin,json} are precomputed 6 s reference mels for zero-shot
voice cloning. manifest.json carries the sampler/tail metadata the browser
runtime consumes.
Quantization
int8 weight-only (MatMulNBits, block 64) on both graphs: 680 MB fp16 ->
492 MB, output correlation 0.9998 vs fp16. int4 reaches 396 MB but at 0.988
correlation on the velocity field, which compounds over 32 Euler steps and is
audibly worse; proper int4 needs GPTQ with calibration (not done yet).
The int8 dit_step.onnx and the fp16 dit_step_fp16.onnx agree to 0.99998
cosine on CPU, so the fp16 build is the recommended fast path on WebGPU and the
int8 build the wasm fallback.
Provenance
- Base weights: F5-TTS v1 Base (
SWivid/F5-TTS), CC-BY-4.0 upstream terms. - Fine-tune data: DownFlow/WebAMSR, purchased corpus, non-commercial use; explicit content is included only in the genres that carry it.
- Reference voices are corpus clips used as style references. Voice identity is a reference-clip pack, so swapping in a consented pack is a file replacement.