OmniVoice Web — bidirectional LLM re-export
Re-exported llm_decoder sub-model for OmniVoice Web,
a fully in-browser (WebGPU) TypeScript runtime for k2-fsa/OmniVoice.
Why this repo exists
The public onnx-community/OmniVoice-Onnx llm_decoder uses causal attention
(GroupQueryAttention + 2-D causal mask). OmniVoice is a bidirectional masked-diffusion LM, so
with causal attention the model ignores the text prompt and emits generic speech. This repo hosts
the Qwen3-0.6B core re-exported with a 4-D all-zero (non-causal) attention mask, which restores
exact text conditioning.
Files
| file | dtype | size | notes |
|---|---|---|---|
llm_decoder_int4.onnx (+.data) |
int4 MatMulNBits | ~280 MB | browser default; f32 activations, no shader-f16 needed |
llm_decoder_fp16.onnx (+.data) |
fp16 | ~885 MB | needs a WebGPU adapter with shader-f16 |
llm_decoder.onnx (+.data) |
fp32 | ~1.77 GB | reference-faithful; CUDA/WebGPU |
All three reproduce the PyTorch reference with WER 0 (round-trip Whisper) on the evaluation
sentence. The embeddings/heads/tokenizer/codec sub-models remain on
onnx-community/OmniVoice-Onnx.
Export
tools/export_llm.py --dtype fp32|fp16 then tools/quantize_llm_legacy.py (ORT 1.20). See the
project repository for commands.
License
Model weights are CC-BY-NC-4.0 (non-commercial), inherited from OmniVoice.