| license: llama3.1 | |
| base_model: meta-llama/Llama-3.1-8B-Instruct | |
| library_name: basert | |
| pipeline_tag: text-generation | |
| tags: | |
| - basert | |
| - apple-silicon | |
| - quantized | |
| # Llama-3.1-8B-Instruct | |
| [BaseRT](https://github.com/basecompute/baseRT) `.base` builds of | |
| [`meta-llama/Llama-3.1-8B-Instruct`](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) for fast local inference on Apple | |
| Silicon (Metal). | |
| ## Files | |
| | File | Precision | Size | | |
| |------|-----------|------| | |
| | `Llama-3.1-8B-Instruct-Q4.base` | 4-bit | 4.2G | | |
| | `Llama-3.1-8B-Instruct-Q8.base` | 8-bit | 7.7G | | |
| ## Usage | |
| ```bash | |
| curl -LsSf https://basecompute.co/install.sh | sh | |
| basert pull basecompute/Llama-3.1-8B-Instruct | |
| basert chat basecompute/Llama-3.1-8B-Instruct | |
| ``` | |
| Released under the llama3.1 license, inherited from the base model. | |