mlboydaisuke commited on
Commit
eb755f8
·
verified ·
1 Parent(s): 397f3f6

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +2 -5
README.md CHANGED
@@ -19,7 +19,7 @@ every step.
19
  |---|---|---|---|---|---|---|
20
  | encoder | XNNPACK fp32 | `whisper_base_encoder_xnnpack_fp32.pte` | 82.4 | 1.000000 | 120.1 | 43.8 |
21
  | encoder | XNNPACK fp16 | `whisper_base_encoder_xnnpack_fp16.pte` | 43.1 | 0.999997 | 217.6 | 43.0 |
22
- | encoder | XNNPACK int8 | `whisper_base_encoder_xnnpack_int8.pte` | 25.9 | 0.999340 | 111.9 | 43.7 |
23
  | encoder | Core ML | `whisper_base_encoder_coreml_all.pte` | 41.4 | 0.999983 | 28.1 | 44.2 |
24
  | decoder | XNNPACK fp32 | `whisper_base_decoder_xnnpack_fp32.pte` | 314.4 | 1.000000 | 36.1 | 20.0 |
25
  | decoder | XNNPACK fp16 | `whisper_base_decoder_xnnpack_fp16.pte` | 157.4 | 0.999981 | 77.3 | 19.4 |
@@ -60,10 +60,7 @@ delegate's blob for the output matmul. Tying them in PyTorch does not tie them h
60
  Referencing the weight through `F.linear` instead of the `proj_out` module does not either —
61
  exported both ways, whisper-tiny's decoder comes out at 198.0 MB exactly.
62
 
63
- **The decoder has no int8 build.** PT2E puts an observer on the int64 `decoder_input_ids`
64
- feeding the token embedding, and the lookup then refuses a float index (`tensors used as
65
- indices must be long, int, byte or bool`). The encoder takes float mel and quantizes without
66
- complaint, which is where the size is worth taking anyway.
67
 
68
  ## Conversion
69
 
 
19
  |---|---|---|---|---|---|---|
20
  | encoder | XNNPACK fp32 | `whisper_base_encoder_xnnpack_fp32.pte` | 82.4 | 1.000000 | 120.1 | 43.8 |
21
  | encoder | XNNPACK fp16 | `whisper_base_encoder_xnnpack_fp16.pte` | 43.1 | 0.999997 | 217.6 | 43.0 |
22
+ | encoder | XNNPACK int8 | `whisper_base_encoder_xnnpack_int8.pte` | 25.9 | 0.999374 | 114.3 | 45.7 |
23
  | encoder | Core ML | `whisper_base_encoder_coreml_all.pte` | 41.4 | 0.999983 | 28.1 | 44.2 |
24
  | decoder | XNNPACK fp32 | `whisper_base_decoder_xnnpack_fp32.pte` | 314.4 | 1.000000 | 36.1 | 20.0 |
25
  | decoder | XNNPACK fp16 | `whisper_base_decoder_xnnpack_fp16.pte` | 157.4 | 0.999981 | 77.3 | 19.4 |
 
60
  Referencing the weight through `F.linear` instead of the `proj_out` module does not either —
61
  exported both ways, whisper-tiny's decoder comes out at 198.0 MB exactly.
62
 
63
+
 
 
 
64
 
65
  ## Conversion
66