Add int4 (wi4b64) quantized variant
Browse filesThis PR adds an int4 quantized variant of the 30 s encode/decode-split Whisper tiny TFLite graph, drop-in compatible with the existing graph interface in this repo.
**Toolchain**: `ai-edge-quantizer` 0.8.0, post-training dynamic-range quantization.
**Recipe**: `dynamic_wi4b64_afp32` (int4 weights, blockwise-64, fp16 scales, fp32 activations) **with int8 overrides on sensitive scopes**: the encoder and the token-embedding/logits table are kept at int8 channelwise. Pure full-scope int4 was tested and rejected (Korean transcription errors); this mixed recipe is the validated configuration.
**Source checkpoint**: the f32 TFLite export in this repository (litert-community/whisper-tiny, derived from openai/whisper-tiny).
**Validation**: Korean/English gate clips transcribed exactly — Korean tactical-report sentence ("2025년 3월 5일 전술평가 결과 보고"), its English equivalent, and short Korean voice commands; CER scored against punctuation-normalized references. Size: 35 MB (vs 151 MB f32 / 41 MB i8).
**Runtime tested**: LiteRT v0.14.0-era runtime (LiteRT-LM v0.14.0 project pipeline) — Windows x86_64 CPU (XNNPACK) and Android arm64 (Snapdragon 865-class device).
**Known caveats**:
- Like the tiny/base int8 graphs, sub-1.2 s clips showed sensitivity on one Snapdragon 865 device (mel/STFT numerics, ~0.1 % energy delta vs desktop); larger Whisper variants are unaffected.
- tiny remains the weakest tier for short Korean voice commands; prefer base or larger for command-style audio.
Produced by the LiteRT-LM-Unity project (https://github.com/Leuconoe/LiteRT-LM-Unity, release v0.14.0-unity).
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fa63eead3061ee05df593b919b3886a300d934ad734e81c7d6270e275cab6a96
|
| 3 |
+
size 36477936
|