Add model card: usage example and measured performance (M4 Max)

README.md

@@ -3,4 +3,79 @@ license: apache-2.0
 base_model:
 - openai/whisper-tiny
 pipeline_tag: automatic-speech-recognition
----
+tags:
+- litert
+- tflite
+- whisper
+- asr
+- on-device
+---
+
+# whisper-tiny (LiteRT)
+
+[openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny) exported to
+LiteRT (`.tflite`) for on-device speech recognition. Each file bundles the
+encoder and decoder as two signatures over a fixed 30-second input window:
+
+- `encode`: log-mel spectrogram `[1, 80, 3000]` → encoder states `[1, 1500, 384]`
+- `decode`: (encoder states, token buffer `[1, 128]` int32, additive causal mask
+  `[1, 1, 128, 128]`) → logits `[1, 128, 51865]`
+
+## Files
+
+| File | Weights | Notes |
+|---|---|---|
+| `whisper_tiny_30s_f32.tflite` | float32 | reference |
+| `whisper_tiny_30s_i8.tflite` | int8 | ~4x smaller, same transcript and speed on the test clip |
+| `whisper_tiny_30s_f32_<SoC>.tflite` | float32 | 19 MediaTek / Qualcomm NPU-compiled variants |
+
+## Usage (Python, `ai-edge-litert`)
+
+```python
+import numpy as np
+import soundfile as sf
+from transformers import WhisperFeatureExtractor, WhisperTokenizer
+from ai_edge_litert.interpreter import Interpreter
+
+audio, sr = sf.read("speech16k.wav", dtype="float32")  # 16 kHz mono
+fe = WhisperFeatureExtractor.from_pretrained("openai/whisper-tiny")
+mel = fe(audio, sampling_rate=sr, return_tensors="np").input_features
+tok = WhisperTokenizer.from_pretrained("openai/whisper-tiny")
+
+it = Interpreter(model_path="whisper_tiny_30s_i8.tflite", num_threads=4)
+enc, dec = it.get_signature_runner("encode"), it.get_signature_runner("decode")
+
+enc_out = enc(args_0=mel)["output_0"]
+mask = np.triu(np.full((128, 128), -1e9, dtype=np.float32), k=1)[None, None]
+tokens = np.zeros((1, 128), dtype=np.int32)
+tokens[0, :4] = [50258, 50259, 50359, 50363]  # <|sot|><|en|><|transcribe|><|notimestamps|>
+seq, pos = [], 3
+while pos < 127:
+    logits = dec(args_0=enc_out, args_1=tokens, args_2=mask)["output_0"]
+    nxt = int(np.argmax(logits[0, pos]))
+    if nxt == 50257:  # <|endoftext|>
+        break
+    pos += 1
+    tokens[0, pos] = nxt
+    seq.append(nxt)
+print(tok.decode(seq, skip_special_tokens=True))
+```
+
+## Performance
+
+Measured on a 4.8 s window of real speech (19 output tokens), greedy decode,
+CPU inference via the Python Interpreter API (XNNPACK, 4 threads,
+`ai-edge-litert` 2.1.6), median of 10 runs. The graph always processes its
+fixed 30 s window, so RTF is reported against 30 s; decode scales with the
+number of emitted tokens.
+
+| Device | Variant | Encode | Decode (19 tok) | Per token | Window total | RTF |
+|---|---|---|---|---|---|---|
+| Apple M4 Max (macOS) | i8 | 30.3 ms | 203.4 ms | 10.7 ms | 233.7 ms | 0.008 |
+| Apple M4 Max (macOS) | f32 | 28.5 ms | 199.7 ms | 10.5 ms | 228.2 ms | 0.008 |
+
+RTF = processing time / 30 s window (lower is better; below 1.0 is faster than
+real time). Both variants produced an identical, fully correct transcript of the test
+clip, at effectively the same speed - on Apple silicon the win from i8 is the
+~4x smaller file, not latency. The SoC-specific variants target their named NPUs and were not
+measured here.