| license: apache-2.0 | |
| base_model: Qwen/Qwen3-4B-Instruct-2507 | |
| library_name: baseRT | |
| tags: | |
| - baseRT | |
| - basecompute | |
| - apple-silicon | |
| # Qwen3-4B-Instruct-2507 (BaseRT) | |
| [BaseRT](https://github.com/basecompute/baseRT) `.base` builds of | |
| [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) for fast local inference on Apple Silicon. | |
| | File | Quant | | |
| | --- | --- | | |
| | `Qwen3-4B-Instruct-2507-Q4.base` | 4-bit (default) | | |
| | `Qwen3-4B-Instruct-2507-Q8.base` | 8-bit | | |
| ```sh | |
| basert serve basecompute/Qwen3-4B-Instruct-2507 # Q4 (default) | |
| basert serve basecompute/Qwen3-4B-Instruct-2507 --variant default-q8 # Q8 | |
| ``` | |
| Documentation: https://docs.basecompute.co | |