| license: apache-2.0 | |
| base_model: mistralai/Mistral-7B-Instruct-v0.3 | |
| library_name: basert | |
| pipeline_tag: text-generation | |
| tags: | |
| - basert | |
| - apple-silicon | |
| - quantized | |
| # Mistral-7B-Instruct-v0.3 | |
| [BaseRT](https://github.com/basecompute/baseRT) `.base` builds of | |
| [`mistralai/Mistral-7B-Instruct-v0.3`](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) for fast local inference on Apple | |
| Silicon (Metal). | |
| ## Files | |
| | File | Precision | Size | | |
| |------|-----------|------| | |
| | `Mistral-7B-Instruct-v0.3-Q4.base` | 4-bit | 3.8G | | |
| | `Mistral-7B-Instruct-v0.3-Q8.base` | 8-bit | 7.0G | | |
| ## Usage | |
| ```bash | |
| curl -LsSf https://basecompute.co/install.sh | sh | |
| basert pull basecompute/Mistral-7B-Instruct-v0.3 | |
| basert chat basecompute/Mistral-7B-Instruct-v0.3 | |
| ``` | |
| Released under the apache-2.0 license, inherited from the base model. | |