Add int4 (wi4b64) quantized variant
This PR adds an int4 quantized variant of the 30 s encode/decode-split Whisper tiny TFLite graph, drop-in compatible with the existing graph interface in this repo.
Toolchain: ai-edge-quantizer 0.8.0, post-training dynamic-range quantization.
Recipe: dynamic_wi4b64_afp32 (int4 weights, blockwise-64, fp16 scales, fp32 activations) with int8 overrides on sensitive scopes: the encoder and the token-embedding/logits table are kept at int8 channelwise. Pure full-scope int4 was tested and rejected (Korean transcription errors); this mixed recipe is the validated configuration.
Source checkpoint: the f32 TFLite export in this repository (litert-community/whisper-tiny, derived from openai/whisper-tiny).
Validation: Korean/English gate clips transcribed exactly โ Korean tactical-report sentence ("2025๋ 3์ 5์ผ ์ ์ ํ๊ฐ ๊ฒฐ๊ณผ ๋ณด๊ณ "), its English equivalent, and short Korean voice commands; CER scored against punctuation-normalized references. Size: 35 MB (vs 151 MB f32 / 41 MB i8).
Runtime tested: LiteRT v0.14.0-era runtime (LiteRT-LM v0.14.0 project pipeline) โ Windows x86_64 CPU (XNNPACK) and Android arm64 (Snapdragon 865-class device).
Known caveats:
- Like the tiny/base int8 graphs, sub-1.2 s clips showed sensitivity on one Snapdragon 865 device (mel/STFT numerics, ~0.1 % energy delta vs desktop); larger Whisper variants are unaffected.
- tiny remains the weakest tier for short Korean voice commands; prefer base or larger for command-style audio.
Produced by the LiteRT-LM-Unity project (https://github.com/Leuconoe/LiteRT-LM-Unity, release v0.14.0-unity).