| license: apache-2.0 | |
| base_model: mlx-community/FastVLM-0.5B-bf16 | |
| language: | |
| - en | |
| tags: | |
| - mlx | |
| - ffai | |
| - quantized | |
| - 4bit | |
| - affine | |
| # FastVLM-0.5B-4bit | |
| 4-bit affine quantization of [mlx-community/FastVLM-0.5B-bf16](https://huggingface.co/mlx-community/FastVLM-0.5B-bf16), produced with [FFAI](https://github.com/thewafflehaus/FFAI) 0.1.0's `ffai convert` (mlx-affine format, `group_size=64`). | |
| ## Conversion | |
| ```bash | |
| ffai convert mlx-community/FastVLM-0.5B-bf16 --bits 4 \ | |
| --upload-repo ekryski/FastVLM-0.5B-4bit | |
| ``` | |
| ## See also | |
| - [FFAI](https://github.com/thewafflehaus/FFAI) — fast Apple Silicon LLM inference. `Model.load("ekryski/FastVLM-0.5B-4bit")` runs this checkpoint end-to-end. | |
| - [FFAI quickstart](https://github.com/thewafflehaus/FFAI/blob/main/documentation/quickstart.md) | |
| - [FFAI quantization docs](https://github.com/thewafflehaus/FFAI/blob/main/documentation/quantization.md) |