Whisper Small: Core ML encoder, GGUF decoders and full models

This repository contains the optional Apple Silicon package for Whisper Small in Glimpse: a compiled Core ML encoder and decoder-only GGUFs. A decoder-only GGUF carries the decoder, tokenizer and metadata without the encoder weights, so the encoder is not downloaded twice. The encoder also works beside a full Whisper GGUF or whisper.cpp ggml model. Glimpse uses the encoder for every Whisper Small quantization.

File Download size Purpose
whisper-small-encoder.mlmodelc.zip 163.1 MB Compiled Core ML encoder
whisper-small-Q8_0-decoder.gguf 166.6 MB Decoder, tokenizer and metadata of whisper-small-Q8_0.gguf

A decoder-only GGUF needs the encoder. On Windows, Intel Macs, or anywhere without Core ML, use the full model file instead. The Core ML package requires Apple Silicon.

Full models

Complete model files for Windows, Intel Macs and any CPU/GPU path. They need no encoder.

File Download size Format
whisper-small-Q8_0.gguf 269.8 MB Full GGUF, Q8_0
whisper-small-Q5_K_M.gguf 193.7 MB Full GGUF, Q5_K_M
whisper-small-F16.gguf 492.9 MB Full GGUF, F16
ggml-small-q5_1.bin 190.1 MB whisper.cpp ggml, Q5_1
ggml-small-q8_0.bin 264.5 MB whisper.cpp ggml, Q8_0
ggml-small.bin 487.6 MB whisper.cpp ggml, F16

Provenance

The model originates from OpenAI Whisper Small, licensed Apache 2.0. The encoder was converted from whisper.cpp's F16 ggml-small.bin with scripts/convert-whisper-gguf-to-coreml.py from our transcribe.cpp fork, then zipped with ditto -c -k --keepParent. Each decoder-only GGUF comes from the full GGUF listed below via scripts/extract-coreml-decoder.py, which copies the decoder and frontend tensors and metadata byte for byte, omits the encoder tensors and sets stt.whisper.decoder_only=true.

The full GGUFs are unmodified copies of handy-computer/whisper-small-gguf at commit 4afdc794, converted by handy-computer for transcribe.cpp (see NOTICE). The .bin files are unmodified copies of ggerganov/whisper.cpp at commit 5359861c, under OpenAI's MIT license for Whisper (LICENSE-MIT).

Artifact SHA-256
Encoder ZIP 8a7eff95acc7a237731d778d2269ea7aa7338cd1190fd6ab4266be506886a625
whisper-small-Q8_0-decoder.gguf 5b8c6b995f6a6996dfe58defbfea799ac4efb0e764e1ca926007a3eeac15f359
whisper-small-Q8_0.gguf 9b9c8811bbcc82a7766f0fb0925614bdacb0923b2cc630daeac17108b655b860
whisper-small-Q5_K_M.gguf 326cd00c3e7217c751667c7c1600eaf7e0de174e186ca2c16b4bf590251c3c3b
whisper-small-F16.gguf bef65e1ac9d012269453243243aac0d0f67792693ba6c99b584a124e0ee326fc
ggml-small-q5_1.bin ae85e4a935d7a567bd102fe55afc16bb595bdb618e11b2fc7591bc08120411bb
ggml-small-q8_0.bin 49c8fb02b65e6049d5fa6c04f81f53b867b5ec9540406812c643f177317f779f
ggml-small.bin 1be3a9b2063867b937e64e2ec7483364a79917e157fa98c5d94b5c1fffea987b

Runtime requirements

Extract the ZIP next to the model file, keeping the whisper-small-encoder.mlmodelc directory name. Glimpse-Speech looks for the companion beside the model. A full model file falls back to its own encoder when the companion isn't there; a decoder-only GGUF has no encoder to fall back to, so a session fails without the companion. Decoder-only GGUFs need a transcribe.cpp build with decoder-only Whisper support.

The encoder follows Apple's Neural Engine transformer layout and computes in FP16. It takes one 30-second Whisper window. Core ML may run some operations on the CPU; this is not a guarantee of exclusive Neural Engine execution.

Glimpse

This encoder speeds up on-device dictation on Apple Silicon in Glimpse, a free, open-source dictation app for Mac and Windows. The source is on GitHub.

License

The model and its weights are OpenAI's, released under the Apache 2.0 license (LICENSE). This repository only converts and mirrors the original checkpoint. It is not an official OpenAI release. See the original model card for training data and intended use.

Downloads last month
92
GGUF
Model size
0.2B params
Architecture
whisper
Hardware compatibility
Log In to add your hardware

5-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Glimpse-Dictation/Whisper-Small-coreml

Quantized
(289)
this model

Collection including Glimpse-Dictation/Whisper-Small-coreml