mlboydaisuke commited on
Commit
0833e57
·
verified ·
1 Parent(s): bb6d7de

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +9 -0
README.md CHANGED
@@ -73,6 +73,15 @@ float32 and could no longer index — one line in ExecuTorch's
73
  `backends/xnnpack/quantizer/xnnpack_quantizer_utils.py`, still present on main, and nothing
74
  to do with the token ids.
75
 
 
 
 
 
 
 
 
 
 
76
  ## Conversion
77
 
78
  ```bash
 
73
  `backends/xnnpack/quantizer/xnnpack_quantizer_utils.py`, still present on main, and nothing
74
  to do with the token ids.
75
 
76
+ ### Checked in the task's own units
77
+
78
+ Correlation is a first filter. These are the numbers that decide:
79
+
80
+ - **encoder int8** — measured end to end — word error rate against the fp32 encoder: mean WER 0.0% (worst clip 0.0%) over 5 spoken sentences, int8 encoder against the fp32 encoder with the same fp32 decoder and the same waveform; the fp32 arm transcribes all five correctly, so the comparison is against a working control rather than against noise.
81
+ - **decoder int8** — measured end to end — word error rate against the fp32 decoder: mean WER 0.0% (worst clip 0.0%) over 5 spoken sentences, the int8 decoder against the fp32 one with the other half and the waveform held identical; the fp32 arm transcribes all five correctly, so the comparison is against a working control rather than against noise.
82
+
83
+ The sensitivity of that test, measured by injecting random noise into whisper-tiny's encoder output: rel_l2 0.03 (what int8 actually costs) and 0.10 both give WER 0.000; 0.20 and 0.40 give 0.025. Five clean sentences leave headroom, so a pass means *does not break the transcript*, not *indistinguishable at any error level*.
84
+
85
  ## Conversion
86
 
87
  ```bash