mlboydaisuke commited on
Commit
743aa9e
·
verified ·
1 Parent(s): e9f7582

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +27 -13
README.md CHANGED
@@ -24,7 +24,7 @@ token_embeddings input_ids (1, L) int64 -> rows (1, L, 10
24
  text_model rows, positions, slots -> logits (1, L, 151936)
25
  ```
26
 
27
- - **File**: `qwen3_asr_0_6b_xnnpack_8da4w.pte` — **697.5 MB**, fifteen methods
28
  - **Source**: [Qwen/Qwen3-ASR-0.6B](https://huggingface.co/Qwen/Qwen3-ASR-0.6B) — 782.4M parameters
29
  - **License**: apache-2.0
30
  - **Languages**: 30, auto-detected — the model emits `language <Name>` before the text
@@ -88,7 +88,9 @@ tipping rather than silence gradually swamping the signal. The errors look like
88
 
89
  **Hence the ladder.** Thirteen encoder methods cost **3.8 MB** in total, because
90
  methods in one `.pte` share their constants: one window is 3753.4 MB in fp32 and
91
- thirteen are 3757.2 MB. There is no reason to make a caller over-pad.
 
 
92
 
93
  ## Verification
94
 
@@ -99,7 +101,7 @@ and window. Nine clips, Japanese / Portuguese / Russian, windows 2–4 s:
99
  | bundle | worst CER | ignoring case and terminal punctuation |
100
  |---|---|---|
101
  | fp32 (3757.2 MB, not published) | **0.0000** | 0.0000 |
102
- | 8da4w (697.5 MB, this file) | 0.1333 | **0.0667** |
103
 
104
  Both arms run at the same window on purpose, so this measures the conversion and not
105
  the window choice, which is measured separately above.
@@ -140,16 +142,28 @@ reproduces the fp32 model, and nothing about word error rate on real speakers.
140
  Measured standalone at the 30-second window, swapping only the audio half and reading
141
  the transcript:
142
 
143
- | build | size (MB) | Mac ms* | corr vs eager | worst CER |
144
- |---|---|---|---|---|
145
- | XNNPACK fp32 | 746.1 | 170.0 | 0.999998 | 0.0000 |
146
- | XNNPACK fp16 | 381.9 | 320.5 | 0.999991 | 0.0000 |
147
- | XNNPACK int8 | 200.7 | 149.4 | 0.998460 | **0.0000** |
148
- | Core ML | 374.1 | **40.9** | 0.973094 | **0.0690** |
149
-
150
- \*Mac arm64, median of 5 — a reference point for relative cost, not a device number.
151
- Torch eager fp32 on the same machine is 558.1 ms, so the XNNPACK build is already
152
- 3.3x faster than eager.
 
 
 
 
 
 
 
 
 
 
 
 
153
 
154
  Two things worth taking from that table. **int8 on the audio tower is free** in the
155
  only unit that matters, which is why the shipped bundle uses it there. And **the Core
 
24
  text_model rows, positions, slots -> logits (1, L, 151936)
25
  ```
26
 
27
+ - **File**: `qwen3_asr_0_6b_xnnpack_8da4w.pte` — **697.0 MB**, fifteen methods
28
  - **Source**: [Qwen/Qwen3-ASR-0.6B](https://huggingface.co/Qwen/Qwen3-ASR-0.6B) — 782.4M parameters
29
  - **License**: apache-2.0
30
  - **Languages**: 30, auto-detected — the model emits `language <Name>` before the text
 
88
 
89
  **Hence the ladder.** Thirteen encoder methods cost **3.8 MB** in total, because
90
  methods in one `.pte` share their constants: one window is 3753.4 MB in fp32 and
91
+ thirteen are 3757.2 MB. There is no reason to make a caller over-pad. (Those two fp32
92
+ figures are from the pre-rewrite graph; the rewrite moved the published 8da4w file by
93
+ 0.5 MB, and the +3.8 MB it is making a point about is unaffected.)
94
 
95
  ## Verification
96
 
 
101
  | bundle | worst CER | ignoring case and terminal punctuation |
102
  |---|---|---|
103
  | fp32 (3757.2 MB, not published) | **0.0000** | 0.0000 |
104
+ | 8da4w (697.0 MB, this file) | 0.1333 | **0.0667** |
105
 
106
  Both arms run at the same window on purpose, so this measures the conversion and not
107
  the window choice, which is measured separately above.
 
142
  Measured standalone at the 30-second window, swapping only the audio half and reading
143
  the transcript:
144
 
145
+ | build | size (MB) | Mac ms* | XNNPACK takes | corr vs eager | worst CER |
146
+ |---|---|---|---|---|---|
147
+ | XNNPACK fp32 | 746.0 | 143.6 | 71.9% | 0.999999 | 0.0000 |
148
+ | XNNPACK fp16 | 381.7 | 237.6 | 64.5% | 0.999995 | 0.0000 |
149
+ | XNNPACK int8 | 200.6 | 123.5 | 78.0% | 0.998266 | **0.0000** |
150
+ | Core ML | 374.1 | **40.6** | 100% | 0.972872 | **0.0690** |
151
+
152
+ \*Mac arm64, median of 10 — a reference point for relative cost, not a device number.
153
+ Torch eager fp32 on the same machine is 600.6 ms, so the XNNPACK build is 4.2x faster
154
+ than eager.
155
+
156
+ †The CER column was measured on the pre-rewrite graph (below). The rewrite agrees with
157
+ it to 1.3e-06 at the tensor level and leaves every transcript in this card unchanged,
158
+ but the transcripts behind that column were not re-read.
159
+
160
+ **Those speeds moved because the windowed attention stopped calling
161
+ `F.scaled_dot_product_attention`.** It decomposes to `_safe_softmax`, whose guard for
162
+ fully-masked rows is six operations XNNPACK cannot take — and the call has no mask and
163
+ `is_causal=False`, so the guard can never fire. There are 18 layers x 4 windows of it at
164
+ 30 seconds. Writing the softmax out took the fp32 tower from **55.8% delegated at
165
+ 170.0 ms to 71.9% at 143.6 ms**, and int8 from 149.4 ms to 123.5 ms, agreeing with the
166
+ old graph to 7.1e-07.
167
 
168
  Two things worth taking from that table. **int8 on the audio tower is free** in the
169
  only unit that matters, which is why the shipped bundle uses it there. And **the Core