prabodbc commited on
Commit
db31662
verified
1 Parent(s): 8471e11

Professional model card

Browse files
Files changed (1) hide show
  1. README.md +11 -11
README.md CHANGED
@@ -1,24 +1,24 @@
1
  ---
2
  license: llama3.2
3
  base_model: meta-llama/Llama-3.2-1B-Instruct
 
 
4
  tags:
5
  - basert
6
- - base-format
7
  - apple-silicon
8
  - quantized
9
  ---
10
 
11
- # Llama-3.2-1B-Instruct (BaseRT `.base`)
12
 
13
- [BaseRT](https://github.com/basecompute/baseRT) `.base` conversions of
14
- [`meta-llama/Llama-3.2-1B-Instruct`](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) for the BaseRT Apple-Silicon (Metal)
15
- inference engine. Converted **directly from the bf16 source** (no
16
- quant-from-quant) to avoid compounding quantization error.
17
 
18
- | File | Quant | Size |
19
- |------|-------|------|
20
- | `Llama-3.2-1B-Instruct-Q4.base` | `base_q4` | 675M |
21
- | `Llama-3.2-1B-Instruct-Q8.base` | `base_q8` | 1.2G |
 
 
22
 
23
  ## Usage
24
 
@@ -28,4 +28,4 @@ basert pull basecompute/Llama-3.2-1B-Instruct
28
  basert chat basecompute/Llama-3.2-1B-Instruct
29
  ```
30
 
31
- Context: 128k License inherited from the base model.
 
1
  ---
2
  license: llama3.2
3
  base_model: meta-llama/Llama-3.2-1B-Instruct
4
+ library_name: basert
5
+ pipeline_tag: text-generation
6
  tags:
7
  - basert
 
8
  - apple-silicon
9
  - quantized
10
  ---
11
 
12
+ # Llama-3.2-1B-Instruct
13
 
14
+ [BaseRT](https://github.com/basecompute/baseRT) `.base` builds of [`meta-llama/Llama-3.2-1B-Instruct`](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) for fast local inference on Apple Silicon (Metal).
 
 
 
15
 
16
+ ## Files
17
+
18
+ | File | Precision | Size |
19
+ |------|-----------|------|
20
+ | `Llama-3.2-1B-Instruct-Q4.base` | 4-bit | 702 MB |
21
+ | `Llama-3.2-1B-Instruct-Q8.base` | 8-bit | 1.3 GB |
22
 
23
  ## Usage
24
 
 
28
  basert chat basecompute/Llama-3.2-1B-Instruct
29
  ```
30
 
31
+ Released under the llama3.2 license, inherited from the base model.