versae commited on
Commit
ec42c93
·
verified ·
1 Parent(s): 8d0aa0d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -2
README.md CHANGED
@@ -16,7 +16,22 @@ tags:
16
  - mlx
17
  ---
18
 
19
- # NbAiLab/borealis-4b-instruct-preview-mlx-8bits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
 
21
  This model [NbAiLab/borealis-4b-instruct-preview-mlx-8bits](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-mlx-8bits) was
22
  converted to MLX format from [NbAiLab/borealis-4b-instruct-preview](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview)
@@ -33,7 +48,7 @@ from mlx_lm import load, generate
33
 
34
  model, tokenizer = load("NbAiLab/borealis-4b-instruct-preview-mlx-8bits")
35
 
36
- prompt = "hello"
37
 
38
  if tokenizer.chat_template is not None:
39
  messages = [{"role": "user", "content": prompt}]
 
16
  - mlx
17
  ---
18
 
19
+ # Borealis 4B Instruct MLX (Preview)
20
+
21
+ ## Model summary
22
+ **NbAiLab/borealis-4b-instruct-preview-mlx** is a MLX 8bit quantized version of a **4B-parameter** instruction-tuned **preview** model intended for early testing and feedback. It is an **experiment** and should be treated as pre-release quality.
23
+
24
+ The original model is [NbAiLab/borealis-4b-instruct-preview](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview).
25
+
26
+ | Model | Bits | Format |
27
+ |---|---:|---|
28
+ | [NbAiLab/borealis-4b-instruct-preview](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview) | BF16 | Transformers (safetensors) |
29
+ | [NbAiLab/borealis-4b-instruct-preview-gguf](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-gguf) | 8 | GGUF (`q8_0`) |
30
+ | [NbAiLab/borealis-4b-instruct-preview-gguf](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-gguf) | 16 | GGUF (`f16`) |
31
+ | [NbAiLab/borealis-4b-instruct-preview-gguf](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-gguf) | BF16 | GGUF (`bf16`) |
32
+ | [NbAiLab/borealis-4b-instruct-preview-mlx](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-mlx) | 32 | MLX |
33
+ | [NbAiLab/borealis-4b-instruct-preview-mlx-8bits](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-mlx-8bits) | 8 | MLX (quantized) |
34
+
35
 
36
  This model [NbAiLab/borealis-4b-instruct-preview-mlx-8bits](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview-mlx-8bits) was
37
  converted to MLX format from [NbAiLab/borealis-4b-instruct-preview](https://huggingface.co/NbAiLab/borealis-4b-instruct-preview)
 
48
 
49
  model, tokenizer = load("NbAiLab/borealis-4b-instruct-preview-mlx-8bits")
50
 
51
+ prompt = "hei :)"
52
 
53
  if tokenizer.chat_template is not None:
54
  messages = [{"role": "user", "content": prompt}]