README.md

---
license: other
license_name: mit-and-apache-2.0
license_link: https://huggingface.co/leuconoe/litert-lm-unity-quantized#known-caveats-disclose-in-model-cards
tags:
- litert
- tflite
- litertlm
- whisper
- qwen2.5
- asr
- text-generation
- quantized
- int8
- int4
base_model:
- openai/whisper-tiny
- openai/whisper-base
- openai/whisper-medium
- openai/whisper-large-v3
- openai/whisper-large-v3-turbo
- Qwen/Qwen2.5-0.5B-Instruct
- Qwen/Qwen2.5-1.5B-Instruct
---

# Community Release — Project-Quantized LiteRT Models (2026-07-23)

Quantized ASR/LLM artifacts produced by the LiteRT-LM-Unity project, prepared for
upstream contribution (e.g., litert-community on Hugging Face). All files were
validated before inclusion; validation data below.

## Toolchain / provenance

- Quantizer: `ai-edge-quantizer` 0.8.0 (post-training, dynamic-range).
- Recipes:
  - **i8** = `recipe.dynamic_wi8_afp32()` (int8 weights channelwise, fp32 activations).
  - **i4** = `recipe.dynamic_wi4b64_afp32()` (int4 weights, blockwise-64, fp16 scales)
    **with i8 overrides on sensitive scopes** (encoder / embedding-logits table /
    selected projections). Pure full-scope int4 was tested and REJECTED for several
    models (Korean transcription errors); the mixed recipes below are what passed.
  - int4 channelwise (`wi4c`) and blockwise-32 (`wi4b32`) are not used in this
    collection: in our tests `wi4c` degraded accuracy sharply and `wi4b32`
    produced an immediate EOS. Do not regenerate with them.
- Whisper medium / large-v3 / large-v3-turbo tflite graphs were exported by this
  project from the openai checkpoints (30 s fixed window, encode/decode split
  signatures matching the litert-community whisper-tiny/base graph interface;
  large-v3 family = 128 mel bins / 51866 vocab). f32 source exports (3–6 GB) are
  retained separately and available on request.
- Qwen2.5 `.litertlm` files were produced by unpack → quantize → repack:
  `litert-lm-peek` dump → `dynamic_wi4b64_afp32` on the f32 tflite section →
  rebuild with `litert-lm-builder` 0.14.0 (tokenizer + LlmMetadata mirrored from
  the official q8 release). Source: official litert-community f32 releases.
- Runtime compatibility: `.litertlm` format 1.5.0; validated on LiteRT-LM v0.14.0
  (Windows x86_64 CPU/GPU and Android arm64, Snapdragon 865-class device).

## Files and validation

> **Accuracy-claim scope**: "exact" below means the specific gate clips used during quantization validation (sentence-length Korean/English test audio), not general Korean capability. Per-tier CER across the full 10-clip matrix — including short voice commands, where small tiers degrade sharply — is published in the project repo's `docs/benchmarks/asr-model-matrix.md`.


Validation clips: Korean "2025년 3월 5일 전술평가 결과 보고" sentence + English
equivalent + short Korean voice commands ("볼륨 업" etc.). CER computed against
reference transcripts (punctuation-normalized). Full matrix:
`docs/benchmarks/asr-model-matrix.md` in the LiteRT-LM-Unity repo.

| File | Size | Base model | Recipe | Validation |
| --- | ---: | --- | --- | --- |
| `whisper-tiny/whisper_tiny_30s_i4.tflite` | 35 MB | openai/whisper-tiny (via litert-community f32) | wi4b64 + i8 encoder+embed table | en gate clips exact; **lower Korean accuracy — 3/9 exact, CER ko 0.281** (misses `볼륨 업`→`별념`, `음량 증가`→`능량 증가`). Not recommended for Korean; see the ASR matrix in the project repo |
| `whisper-base/whisper_base_30s_i8.tflite` | 74 MB | openai/whisper-base (via litert-community f32) | wi8 | transcripts byte-identical to f32; ~1.6× faster |
| `whisper-base/whisper_base_30s_i4.tflite` | 44 MB | same | wi4b64 (full scope OK for base) | ko/en gate clips exact (sentence-length audio); sub-1.2 s clips remain unreliable at this tier |
| `whisper-medium/whisper_medium_30s_i8.tflite` | 794 MB | openai/whisper-medium (project export) | wi8 (DRQ) | 7/9 exact, CER ko 0.042 / en 0.000 |
| `whisper-medium/whisper_medium_30s_i4.tflite` | 634 MB | same | wi4b64 mixed (L1) | same transcripts as i8 |
| `whisper-large-v3/whisper_large_v3_30s_i8.tflite` | 1.6 GB | openai/whisper-large-v3 (project export, 128-mel) | wi8 (DRQ) | CER 0.000/0.000, 7/9 exact (spacing-only misses) |
| `whisper-large-v3/whisper_large_v3_30s_i4.tflite` | 1.1 GB | same | wi4b64 mixed (mixD) | CER 0.000/0.000; ~20 % faster than i8 |
| `whisper-large-v3-turbo/whisper_large_v3_turbo_30s_i8.tflite` | 1.1 GB | openai/whisper-large-v3-turbo (project export, 128-mel) | wi8 | 5–6/9 exact, CER ko 0.000 |
| `whisper-large-v3-turbo/whisper_large_v3_turbo_30s_i4.tflite` | 721 MB | same | wi4b64 mixed (mixD) | **best tier overall: 8/9 exact, CER 0.000/0.000**; device-validated (Snapdragon 865, shape-based decode binding) |
| `qwen2.5-0.5b-instruct/Qwen2.5-0.5B-Instruct_wi4b64_ekv1280.litertlm` | 265 MB | litert-community/Qwen2.5-0.5B-Instruct f32 | wi4b64 full scope | inference-validated Windows + Android (35.5 tok/s device CPU, +38 % vs official q8) |
| `qwen2.5-1.5b-instruct/Qwen2.5-1.5B-Instruct_wi4b64_ekv4096.litertlm` | 790 MB | litert-community/Qwen2.5-1.5B-Instruct f32 | wi4b64 full scope | inference-validated Windows CPU (59.7 prefill / 11.8 decode tok/s, Korean QA correct) |

## Related: ACFT short-window Whisper models

Fixed 5 s/10 s/30 s short-window TFLite graphs (encode-signature-sized mel
window; method: [futo-org/whisper-acft](https://github.com/futo-org/whisper-acft))
live in two dedicated repos:

- **Pure ACFT** (TFLite conversions of the futo-org checkpoints, no retraining;
  tiny / base / small × multilingual / .en, dynamic-range int8):
  **[litert-community/whisper-acft](https://huggingface.co/litert-community/whisper-acft)**.
- **Korean ACFT** (ACFT-KO, Korean-trained for on-device Korean voice commands;
  tiny / base / medium / large-v3-turbo, dynamic-range int8):
  **[leuconoe/whisper-acft-ko](https://huggingface.co/leuconoe/whisper-acft-ko)**.
  Training data zeroth-korean (OpenSLR SLR40, CC-BY-4.0) + google/fleurs en_us
  (CC-BY-4.0).

## Known caveats (disclose in model cards)

- Whisper 30 s graphs; decoder is fixed-length re-run (no KV cache) matching the
  litert-community tiny/base interface. large-v3 family needs a 128-mel frontend
  and vocab 51866 (`<|transcribe|>`=50360, `<|notimestamps|>`=50364).
- tiny/base i8 showed sub-1.2 s clip sensitivity on one Snapdragon 865 device
  (mel/STFT numerics, ~0.1 % energy delta vs desktop); turbo/large-v3 unaffected.
- Qwen2.5 i4 litertlm metadata carries lowercase `model_type` (`tf_lite_prefill_decode`)
  vs uppercase in official files — loads fine on v0.14.0, noted for completeness.
- Licenses: whisper weights MIT (OpenAI); Qwen2.5 Apache-2.0. Quantized derivatives
  inherit the base licenses.