| library_name: mlx | |
| pipeline_tag: image-text-to-text | |
| license: other | |
| license_name: bsl-1.1 | |
| license_link: LICENSE.md | |
| base_model: moondream/moondream3-preview | |
| tags: | |
| - mlx | |
| - moondream | |
| - moondream3 | |
| - vision-language | |
| - image-text-to-text | |
| # moondream3-preview-mlx-4bit | |
| A **4-bit** MLX quantization of [moondream/moondream3-preview](https://huggingface.co/moondream/moondream3-preview) | |
| for running on Apple Silicon with [mlx-vlm](https://github.com/Blaizzy/mlx-vlm). | |
| | | | | |
| |---|---| | |
| | Quantization | affine, **4 bits**, group size 64 (vision tower included) | | |
| | On-disk size | ~5 GB | | |
| | Peak memory | ~6.5 GB | | |
| | Tokenizer | loaded from [`moondream/starmie-v1`](https://huggingface.co/moondream/starmie-v1) at runtime (not bundled) | | |
| ## Usage | |
| ```bash | |
| pip install mlx-vlm | |
| python -m mlx_vlm.generate \ | |
| --model beshkenadze/moondream3-preview-mlx-4bit \ | |
| --image path/to/image.jpg \ | |
| --prompt "Describe this image." \ | |
| --max-tokens 128 --temperature 0.0 | |
| ``` | |
| ## License | |
| moondream3 is released under the **Business Source License 1.1 (BSL 1.1)** — see [LICENSE.md](LICENSE.md). This quantization is a derivative redistribution under the same terms. | |