| # Implementation Plan - Convert CrisperWhisper 2.0 Large to GGML/GGUF for SmartScribe |
|
|
| Convert the Hugging Face model [nyralabs/CrisperWhisper2.0_large](https://huggingface.co/nyralabs/CrisperWhisper2.0_large) to GGML format and quantize it to `q8_0` for use with SmartScribe's `whisper.cpp` engine. |
|
|
| ## User Review Required |
|
|
| > [!IMPORTANT] |
| > **Model Download Size & Processing**: The `nyralabs/CrisperWhisper2.0_large` weights are ~3 GB. Downloading the model, converting to GGML (f16 format ~3 GB), and quantizing to `Q8_0` will require approximately 6-8 GB of temporary local disk space. |
| > |
| > **Format Note**: SmartScribe uses `whisper.cpp` for Whisper architecture models, which expects GGML format (quantized as `q8_0` binary). We will produce `ggml-crisperwhisper2.0-large-q8_0.bin` and provide full conversion instructions and catalog integration. |
|
|
| ## Proposed Steps |
|
|
| ### 1. Model Download & Preparation |
| - Download model configuration, tokenizer, and safetensors weights from `nyralabs/CrisperWhisper2.0_large` using `huggingface-cli` or Python into a local workspace directory (`models/CrisperWhisper2.0_large`). |
|
|
| ### 2. GGML Conversion |
| - Run `native/whisper-vulkan-addon/third_party/whisper.cpp/models/convert-h5-to-ggml.py` using Python (referencing `whisper/` for mel-filter assets). |
| - Output intermediate float16 binary `ggml-crisperwhisper2.0-large-f16.bin`. |
|
|
| ### 3. Quantization |
| - Build the `quantize` utility in `native/whisper-vulkan-addon/third_party/whisper.cpp/build` if not already built. |
| - Quantize the f16 binary to `ggml-crisperwhisper2.0-large-q8_0.bin` (`q8_0`). |
|
|
| ### 4. Catalog Integration & Verification |
| - Verify shasum 256 and binary integrity. |
| - Document conversion log in `release/whisper-crisperwhisper2.0-large-ggml/CONVERSION.md`. |
| - Add catalog entry for `whisper-crisperwhisper2.0-large-q8` in `src/shared/models-catalog.ts` (if enabled in settings). |
|
|
| ## Proposed Changes |
|
|
| ### [NEW] `release/whisper-crisperwhisper2.0-large-ggml/CONVERSION.md` |
| - Conversion log documentation detailing environment, download steps, conversion script invocation, and shasum checksums. |
|
|
| ### [MODIFY] [models-catalog.ts](file:///Users/shrisha/Dev/Smartscribe_cleanup/src/shared/models-catalog.ts) |
| - Add entry for CrisperWhisper 2.0 Large (`whisper-crisperwhisper2.0-large-q8`) to `AVAILABLE_LOCAL_MODELS`. |
|
|
| ## Verification Plan |
|
|
| ### Manual Verification |
| 1. Verify `ggml-crisperwhisper2.0-large-q8_0.bin` creation and non-zero byte size (~1.5 GB). |
| 2. Test loading and transcribing a sample audio file (`jfk.wav`) using `test-whisper.js` / node script with the converted binary model. |
|
|
