rumble-2-flash-web / README.md
kaptionai's picture
Rumble-2-Flash: WebGPU package (int8 encoder, 4-bit decoder)
5b3dde6 verified
|
Raw History Blame Contribute Delete
5.4 kB
metadata
license: apache-2.0
base_model: Reson8/Rumble-2-Flash
pipeline_tag: automatic-speech-recognition
library_name: onnxruntime
language:
  - af
  - ar
  - bn
  - bs
  - ca
  - ceb
  - cs
  - da
  - de
  - el
  - en
  - es
  - eu
  - fa
  - fi
  - fil
  - fr
  - hi
  - hr
  - hu
  - id
  - is
  - it
  - ja
  - ko
  - lt
  - lv
  - mk
  - ms
  - mt
  - nb
  - nl
  - pl
  - pt
  - ro
  - ru
  - sq
  - sr
  - sv
  - th
  - tr
  - uk
  - vi
  - yue
  - zh
tags:
  - onnx
  - onnxruntime-web
  - webgpu
  - qwen3-asr
  - kaption

Rumble-2-Flash for the browser (WebGPU), converted by Kaption

ONNX files to run Reson8/Rumble-2-Flash in a browser with onnxruntime-web on WebGPU. 494 MB in total. The layout is the same as kaptionai/qwen3-asr-web.

  • Base model: Reson8/Rumble-2-Flash, Apache-2.0, revision a0c34287229103b706c670d70791683e34379732. It is a fine-tune of Qwen/Qwen3-ASR-0.6B (Apache-2.0) with the decoder cut to 14 layers and the vocabulary cut to 99,630 tokens.
  • GGUF quantisation of the same model: AtomicChat/Rumble-2-Flash-GGUF. No file here comes from that repository; their model card is where the relation to Qwen3-ASR-0.6B is documented.
  • These files: all converted by Kaption from the base model's model.safetensors, except tokenizer.json (Reson8's file, unchanged).
  • Not endorsed by Reson8, AtomicChat or Alibaba Qwen. All credit for the model belongs to its authors.

Files

File Bytes SHA-256 Origin
encoder.int8f16.onnx 202,340,869 f6fb2942f0d8048967f9d60b76d531642d18e44e6e431c289aaaa7ad9d51339b Kaption conversion
decoder.gqa-q4f16.onnx 182,951 07d102f48cb4ed993cb78c3112ec0f49b4423aab167fb96e26026bca13ae3525 Kaption conversion
decoder_weights.q4f16.data 181,774,208 b1cc416f9ec33741e2f4b35bb6f7bd4ef07de92cd2828534d92c6184d4b20b07 Kaption conversion (external data of the decoder graph)
embed_tokens.int8.bin 102,021,120 be3ebc783a5b42edda8de10fce666b10960eb2ae25e42ff6565399aa46d17953 Kaption conversion
embed_scales.f32.bin 398,520 b192623ed103834f0ada43b5ce2a47bb84b6b0bfda979dc9fe63cd90c57f082e Kaption conversion
tokenizer.json 7,422,837 60d63d6d615abcf61470a29a74b8c7434a96f87f2e97bca710cb767e076c0040 Reson8/Rumble-2-Flash, unchanged
mel_filters.json 132,427 dcdaf139353616855b4f85e304c075f5f3442573d40637ae0f031db705626181 The same file as in kaptionai/qwen3-asr-web (128 mel bins, 16 kHz)
prompt_config.json 1,528 df4482b2b51d485481bbac1fcd3469e06bc6f07492e8a77e61e478f0d79b6ac1 Written by Kaption: prompt token ids under the pruned vocabulary
LICENSE 11,358 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 Apache License 2.0 text

How the files were made

  • encoder.int8f16.onnx: the audio encoder and projector exported to ONNX (18 layers, checked against the Transformers model), every MatMul weight quantised to ONNX Runtime MatMulNBits (8-bit, block 32), then converted to fp16 activations inside with fp32 inputs and outputs. It takes a batch of 8 s windows: mel [W, 128, 800] and valid [W, 104] in, emb [W, 104, 1024] out.
  • decoder.gqa-q4f16.onnx + decoder_weights.q4f16.data: the 14-layer text decoder built with onnxruntime-genai's model builder (4-bit weights, WebGPU target, embeddings left out), then rewritten as one graph for the prefill and every step: GroupQueryAttention with rotary and q/k norm inside, only the last position reaches the output head, and the greedy pick runs on the GPU, so next_token is the only small output. Float inputs and outputs are fp16.
  • embed_tokens.int8.bin + embed_scales.f32.bin: the token embedding table (99,630 rows of 1,024), int8 per row with one fp32 scale per row (largest error 0.001). The embedding lookup is done outside the graph.
  • prompt_config.json: token ids of the fixed prompt. The system turn carries Reson8's control tokens <digits><nofiller><verbatim> (the defaults of the model's chat template), then one audio turn, then the assistant turn. It also holds the language <Name> ids used to force a language.

Runtime

onnxruntime-web with the WebGPU execution provider. The GPU must support shader-f16. The decoder graph does not run on ONNX Runtime's CPU provider. Audio is 16 kHz mono, log-mel with 128 bins, cut into 8 s encoder windows; decoding is greedy.

Known limits (measured by Kaption)

  • Chinese is weaker than with Qwen3-ASR-0.6B. On 15 FLEURS clips: character error rate 8.3 against 5.3 for the Qwen3-ASR WebGPU package. Spanish and Portuguese are better (FLEURS, 240 clips in en/es/pt: word error rate 4.08 against 6.50).
  • It invents text on digital silence. On 36 synthetic clips without speech it wrote text on 4, each time one sentence repeated about 10 times. Put a voice-activity gate in front and do not send silence to the model.
  • No Hebrew. The model's 45 languages are the ones listed in the metadata above.
  • No word timestamps. The output is text only.
  • Tested in Chrome on Apple silicon and on an Intel UHD 620 laptop. Other browsers were not tested.

License

Apache-2.0 (LICENSE), the same as Reson8/Rumble-2-Flash and Qwen/Qwen3-ASR-0.6B. Copyright of the model weights stays with their authors.