Add an engine backend: the 22-block DiT runs on hand-written WebGPU
kernels (tiled/sgmat GEMM, fused LN+adaLN, RoPE, tiled QK/softmax/AV,
grouped convpos, GRN) with 4-bit packed weights and per-group scales
dequantized in-kernel. The worker loads only the ORT vocoder for this
path; ONNX Runtime remains the fallback.
- 229.7 MB int4 bundle, ~1.2 s per clip on WebGPU (was 2.8 s int4 ORT)
- flow: 4 steps, sway -1, cfg 0 (B=1, no null pass)
- tuning: tts.backend / tts.gemm switch kernels without touching the worker
- fixes found on the way: u32 vs f32 uniforms, second SiLU on the time MLP,
RoPE inv_freq exponent, AV score-tile column, f32 attention scores,
safe tanh in Mish
Switch the runtime to the reflow-distilled 4-step model:
- worker: cfgStrength from tuning/manifest, skip the null pass at cfg=0
- tuning: flowSteps 4, cfgStrength 0
- model repo: dit_step_4s_int4.onnx (Split->Slice + aux outputs so JSEP
does not miscompile it) and vocoder_4s_int4.onnx; manifest steps=4, cfg=0
- README: 435 MB download, few-seconds synthesis on WebGPU
Synthesis on this M3 Air: ~2.8-3.4 s per short clip on WebGPU (was ~22 s at
32 steps, ~8 s at 16), ~31 s on the wasm fallback.
Quantization is not a speed win on this pair of runtimes: 4-bit
MatMulNBits runs 3.7x slower on the WebGPU EP, 8-bit is unsupported
there, and int8/int4 wasm time is unchanged. The DiT is compute-bound,
so cutting sampler steps is the lever; 16 steps gives 8.2s vs 22.0s
with identical ASR transcripts (same-seed mel cosine 0.984).
