mlboydaisuke/whisper-large-v3-turbo-CoreAI-official

Whisper large-v3-turbo — Apple Core AI export (autoregressive)

A pre-converted .aimodel from Apple's official coreai-models Whisper recipe, packaged so it actually transcribes on the stock Core AI runtime (no engine patch).

Whisper large-v3-turbo (809 M) is OpenAI's multilingual ASR encoder-decoder: 100 languages, automatic language detection.

Whisper large-v3-turbo on iPhone 17 Pro — the zoo's coreai-audio app, real speed.

Use it

▶️ Run it (source) — the Transcribe runner (GUI + CLI, one app for every speech-to-text model in the catalog):

git clone https://github.com/john-rocky/coreai-kit
open coreai-kit/Examples/Transcribe/Transcribe.xcodeproj
# → Run, then pick "Whisper large-v3-turbo" in the model picker

# agents / headless (macOS):
cd coreai-kit/Examples/Transcribe
swift run transcribe-cli --model whisper-large-v3-turbo --audio sample.wav

💻 Build with it — complete; the glue is kit API, copy-paste runs:

import CoreAIKit

let transcriber = try await KitTranscriber(catalog: "whisper-large-v3-turbo")
let samples = try AudioFile.pcm16kMono(url)  // any wav/m4a/mp3 → 16 kHz mono Float
let result = try await transcriber.transcribe(samples: samples)
// result.text, result.language ("en", "ja", … auto-detected)

The take-home is Examples/Transcribe/Sources/QuickStart.swift — this exact code as one typed function, no UI; both the runner's GUI and its CLI call it. Recording? MicRecorder (kit API) captures mic audio as 16 kHz mono [Float] — the record button and permission prompt are your app's own chrome.

Integration checklist

• SPM: https://github.com/john-rocky/coreai-kit → product CoreAIKit
• Info.plist: NSMicrophoneUsageDescription — only if you record
• Entitlements (iOS): com.apple.developer.kernel.increased-memory-limit
• First run downloads the model — 1.6 GB (Mac) / 3.2 GB (iPhone) — then it loads from the local cache (Application Support; progress via the downloadProgress callback)
• Measure in Release — Debug is ~3× slower on per-token host work

What's different from the stock recipe (and why)

Apple's models/whisper/export.py traces the model with decoder_input_ids of shape [1, 1] — a single decode step, no KV cache. That graph can't be driven autoregressively: with one token and no cache, every step is "position 0" and loses all prior context (it emits nothing useful).

This bundle is the same recipe with one change: the decoder is traced at a fixed 128-token window (decoder_input_ids: [1, 128]). You pad the decoder buffer to 128 and read the logits at the real last position. Because the self-attention is causal, the real token at position k never attends to the padding, so the read is exact — and because the shape is constant, MPSGraph compiles once (a dynamic-length export instead recompiles every step → ~15 s/token; this is ~0.18 s/token). Everything else is the upstream recipe and it runs on the stock runtime.

# stock single-step recipe:           uv run models/whisper/export.py
# this bundle (fixed 128-token decode): see _export_whisper_fixed.py in the zoo conversion/ dir

Bundle

whisper-large-v3-turbo_float16_fixed128.aimodel/   main.mlirb + main.hash + metadata.json
tokenizer/                                         HF Whisper tokenizer (detokenize output ids)
mel_filters_128.npy                                [201, 128] mel filterbank for the audio frontend
preprocessor_config.json                           n_fft=400, hop=160, 128 mels, 16 kHz

File	SHA-256
…_fixed128.aimodel/main.mlirb (~1.5 GB)	f5824a2e01906ad72bb3241573e75a41ecbf89c2ebe5fb8b87716752cf144881
…_fixed128.aimodel/main.hash	2bd169f0ca2812f7a6321f973a7bf88ef0ff37bc823265efb13e1beb12a7bf2c
mel_filters_128.npy	4eb6b0fe7aa985fa2ce80d81260d8b5b30ff908d2808a5d637683228c648db6f

Measured (M4 Max, GPU)

Greedy decode, English clip, vs the HF PyTorch reference (generate, greedy):

Metric	Value
Transcript	token-for-token identical to PyTorch greedy
First step (compile + warmup)	0.68 s
Per token (steady state)	0.18 s

The fixed window caps a single 30 s decode at 128 tokens (enough for a 30 s window; chunk longer audio into 30 s segments).

How to run it (the decode loop)

The graph takes input_features [1, 128, 3000] (log-mel) + decoder_input_ids [1, 128] → logits [1, 128, 51866].

1. Audio → log-mel (mel_filters_128.npy, n_fft 400 / hop 160): STFT → power → mel filterbank → log10, clamp to max-8, (x+4)/4; pad/trim to 3000 frames.
2. Prompt: [<|startoftranscript|>, <|en|>, <|transcribe|>, <|notimestamps|>] (50258, 50259, 50360, 50364).
3. Loop: pad the prompt to 128, run, take argmax(logits[0, k]) at the real last index k, append, repeat until <|endoftext|> (50257).
4. Detokenize the generated ids with the bundled tokenizer.

Runs on macOS + iOS via coreai.runtime / the Swift CoreAI framework. A from-scratch reference implementation of this exact loop + the log-mel frontend (no kit dependency) is the CoreAITranscribe sample app (macOS + iOS, file or mic) — or skip the loop entirely and use the 3-line KitTranscriber snippet in Use it, which does it (plus the mel frontend, chunking, and auto language detect) for you.

Export environment

• macOS 27.0 beta · coreai-core 1.0.0b1 · coreai-torch 0.4.0 · transformers 4.57
• recipe: Apple models/whisper/export.py + a fixed 128-token decoder trace

License

Whisper is MIT (OpenAI). This bundle is a format conversion and inherits that license.

---

Maintained alongside coreai-model-zoo (card · official/).