litert-community

/

whisper-base

like 2

Follow

LiteRT Community (FKA TFLite) 8.42k

This PR adds int8 and int4 quantized variants of the 30 s encode/decode-split Whisper base TFLite graph, drop-in compatible with the existing graph interface in this repo.

Toolchain: ai-edge-quantizer 0.8.0, post-training dynamic-range quantization.

Recipes:

• whisper_base_30s_i8.tflite (74 MB): dynamic_wi8_afp32 — int8 weights channelwise, fp32 activations.
• whisper_base_30s_i4.tflite (44 MB): dynamic_wi4b64_afp32 — int4 weights, blockwise-64, fp16 scales, fp32 activations, applied full-scope (base tolerates full-scope int4; no int8 scope overrides needed, unlike tiny where the encoder and embedding/logits table must stay int8).

Source checkpoint: the f32 TFLite export in this repository (litert-community/whisper-base, derived from openai/whisper-base).

Validation: on the Korean/English gate set (Korean tactical-report sentence, English equivalent, short Korean voice commands; CER scored against punctuation-normalized references):

• i8: transcripts byte-identical to the f32 graph, ~1.6× faster decode on desktop CPU.
• i4: Korean/English gate clips exact.

Runtime tested: LiteRT v0.14.0-era runtime (LiteRT-LM v0.14.0 project pipeline) — Windows x86_64 CPU (XNNPACK) and Android arm64 (Snapdragon 865-class device).

Known caveats:

• Both i8 and i4 showed sub-1.2 s clip sensitivity on one Snapdragon 865 device (mel/STFT numerics, ~0.1 % energy delta vs desktop); larger Whisper variants are unaffected.
• i4 is slightly weaker than i8 on short Korean voice commands (e.g. 음량 → 음향 confusion on one command clip); i8 is the recommended tier when the 30 MB difference does not matter.

Produced by the LiteRT-LM-Unity project (https://github.com/Leuconoe/LiteRT-LM-Unity, release v0.14.0-unity).