Flatten structure: remove subdirectory
Browse files- README.md +0 -39
- whisper-tiny-mlx-4bit/Readme.md β Readme.md +0 -0
- whisper-tiny-mlx-4bit/added_tokens.json β added_tokens.json +0 -0
- whisper-tiny-mlx-4bit/generation_config.json β generation_config.json +0 -0
- whisper-tiny-mlx-4bit/merges.txt β merges.txt +0 -0
- whisper-tiny-mlx-4bit/model.safetensors β model.safetensors +0 -0
- whisper-tiny-mlx-4bit/normalizer.json β normalizer.json +0 -0
- whisper-tiny-mlx-4bit/preprocessor_config.json β preprocessor_config.json +0 -0
- whisper-tiny-mlx-4bit/special_tokens_map.json β special_tokens_map.json +0 -0
- whisper-tiny-mlx-4bit/tokenizer.json β tokenizer.json +0 -0
- whisper-tiny-mlx-4bit/tokenizer_config.json β tokenizer_config.json +0 -0
- whisper-tiny-mlx-4bit/vocab.json β vocab.json +0 -0
- whisper-tiny-mlx-4bit/weights.npz β weights.npz +0 -0
- whisper-tiny-mlx-4bit/config.json +0 -17
README.md
DELETED
|
@@ -1,39 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: mit
|
| 3 |
-
tags:
|
| 4 |
-
- mlx
|
| 5 |
-
- 4bit
|
| 6 |
-
- whisper
|
| 7 |
-
- stt
|
| 8 |
-
- metalrt
|
| 9 |
-
- apple-silicon
|
| 10 |
-
---
|
| 11 |
-
|
| 12 |
-
# Whisper Tiny β MLX 4-bit Quantized
|
| 13 |
-
|
| 14 |
-
Custom MLX 4-bit quantization of [OpenAI Whisper Tiny](https://huggingface.co/openai/whisper-tiny) optimized for [MetalRT](https://github.com/RunanywhereAI/metalrt-binaries) GPU inference on Apple Silicon.
|
| 15 |
-
|
| 16 |
-
## Usage
|
| 17 |
-
|
| 18 |
-
Used by [RCLI](https://github.com/RunanywhereAI/RCLI) with the MetalRT engine for speech-to-text:
|
| 19 |
-
|
| 20 |
-
```bash
|
| 21 |
-
rcli setup # select MetalRT or Both engines
|
| 22 |
-
```
|
| 23 |
-
|
| 24 |
-
## Performance (Apple M3 Max)
|
| 25 |
-
|
| 26 |
-
| Metric | Value |
|
| 27 |
-
|--------|-------|
|
| 28 |
-
| Latency (1.2s audio) | **46 ms** |
|
| 29 |
-
| RTF | 0.04x |
|
| 30 |
-
| Quantization | MLX 4-bit |
|
| 31 |
-
|
| 32 |
-
## License
|
| 33 |
-
|
| 34 |
-
Model weights: [MIT](https://github.com/openai/whisper/blob/main/LICENSE) (OpenAI)
|
| 35 |
-
MetalRT engine: [Proprietary](https://github.com/RunanywhereAI/metalrt-binaries/blob/main/LICENSE) (RunAnywhere, Inc.)
|
| 36 |
-
|
| 37 |
-
## Contact
|
| 38 |
-
|
| 39 |
-
founder@runanywhere.ai | https://runanywhere.ai
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
whisper-tiny-mlx-4bit/Readme.md β Readme.md
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/added_tokens.json β added_tokens.json
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/generation_config.json β generation_config.json
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/merges.txt β merges.txt
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/model.safetensors β model.safetensors
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/normalizer.json β normalizer.json
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/preprocessor_config.json β preprocessor_config.json
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/special_tokens_map.json β special_tokens_map.json
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/tokenizer.json β tokenizer.json
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/tokenizer_config.json β tokenizer_config.json
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/vocab.json β vocab.json
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/weights.npz β weights.npz
RENAMED
|
File without changes
|
whisper-tiny-mlx-4bit/config.json
DELETED
|
@@ -1,17 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"model_type": "whisper",
|
| 3 |
-
"n_audio_ctx": 1500,
|
| 4 |
-
"n_audio_head": 6,
|
| 5 |
-
"n_audio_layer": 4,
|
| 6 |
-
"n_audio_state": 384,
|
| 7 |
-
"n_mels": 80,
|
| 8 |
-
"n_text_ctx": 448,
|
| 9 |
-
"n_text_head": 6,
|
| 10 |
-
"n_text_layer": 4,
|
| 11 |
-
"n_text_state": 384,
|
| 12 |
-
"n_vocab": 51865,
|
| 13 |
-
"quantization": {
|
| 14 |
-
"bits": 4,
|
| 15 |
-
"group_size": 64
|
| 16 |
-
}
|
| 17 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|