Update README.md
Browse files
README.md
CHANGED
|
@@ -16,7 +16,22 @@ tags:
|
|
| 16 |
- mlx
|
| 17 |
---
|
| 18 |
|
| 19 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
|
| 21 |
This model [NbAiLab/borealis-4b-instruct-preview-mlx-8bits](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-mlx-8bits) was
|
| 22 |
converted to MLX format from [NbAiLab/borealis-4b-instruct-preview](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview)
|
|
@@ -33,7 +48,7 @@ from mlx_lm import load, generate
|
|
| 33 |
|
| 34 |
model, tokenizer = load("NbAiLab/borealis-4b-instruct-preview-mlx-8bits")
|
| 35 |
|
| 36 |
-
prompt = "
|
| 37 |
|
| 38 |
if tokenizer.chat_template is not None:
|
| 39 |
messages = [{"role": "user", "content": prompt}]
|
|
|
|
| 16 |
- mlx
|
| 17 |
---
|
| 18 |
|
| 19 |
+
# Borealis 4B Instruct MLX (Preview)
|
| 20 |
+
|
| 21 |
+
## Model summary
|
| 22 |
+
**NbAiLab/borealis-4b-instruct-preview-mlx** is a MLX 8bit quantized version of a **4B-parameter** instruction-tuned **preview** model intended for early testing and feedback. It is an **experiment** and should be treated as pre-release quality.
|
| 23 |
+
|
| 24 |
+
The original model is [NbAiLab/borealis-4b-instruct-preview](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview).
|
| 25 |
+
|
| 26 |
+
| Model | Bits | Format |
|
| 27 |
+
|---|---:|---|
|
| 28 |
+
| [NbAiLab/borealis-4b-instruct-preview](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview) | BF16 | Transformers (safetensors) |
|
| 29 |
+
| [NbAiLab/borealis-4b-instruct-preview-gguf](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-gguf) | 8 | GGUF (`q8_0`) |
|
| 30 |
+
| [NbAiLab/borealis-4b-instruct-preview-gguf](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-gguf) | 16 | GGUF (`f16`) |
|
| 31 |
+
| [NbAiLab/borealis-4b-instruct-preview-gguf](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-gguf) | BF16 | GGUF (`bf16`) |
|
| 32 |
+
| [NbAiLab/borealis-4b-instruct-preview-mlx](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-mlx) | 32 | MLX |
|
| 33 |
+
| [NbAiLab/borealis-4b-instruct-preview-mlx-8bits](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-mlx-8bits) | 8 | MLX (quantized) |
|
| 34 |
+
|
| 35 |
|
| 36 |
This model [NbAiLab/borealis-4b-instruct-preview-mlx-8bits](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-mlx-8bits) was
|
| 37 |
converted to MLX format from [NbAiLab/borealis-4b-instruct-preview](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview)
|
|
|
|
| 48 |
|
| 49 |
model, tokenizer = load("NbAiLab/borealis-4b-instruct-preview-mlx-8bits")
|
| 50 |
|
| 51 |
+
prompt = "hei :)"
|
| 52 |
|
| 53 |
if tokenizer.chat_template is not None:
|
| 54 |
messages = [{"role": "user", "content": prompt}]
|