Improve language tag

#2
by lbourdois - opened
Files changed (1) hide show
  1. README.md +86 -74
README.md CHANGED
@@ -1,74 +1,86 @@
1
- ---
2
- license: apache-2.0
3
- license_link: https://huggingface.co/Qwen/Qwen2.5-14B-Instruct/blob/main/LICENSE
4
- language:
5
- - en
6
- pipeline_tag: text-generation
7
- base_model: Qwen/Qwen2.5-14B-Instruct
8
- base_model_relation: quantized
9
- tags:
10
- - chat
11
- quantized_by: bartowski
12
- ---
13
-
14
- ## Exllama v2 Quantizations of Qwen2.5-14B-Instruct
15
-
16
- Using <a href="https://github.com/turboderp/exllamav2/releases/tag/v0.2.2">turboderp's ExLlamaV2 v0.2.2</a> for quantization.
17
-
18
- <b>The "main" branch only contains the measurement.json, download one of the other branches for the model (see below)</b>
19
-
20
- Each branch contains an individual bits per weight, with the main one containing only the meaurement.json for further conversions.
21
-
22
- Original model: https://huggingface.co/Qwen/Qwen2.5-14B-Instruct
23
-
24
- ## Prompt format
25
-
26
- ```
27
- <|im_start|>system
28
- {system_prompt}<|im_end|>
29
- <|im_start|>user
30
- {prompt}<|im_end|>
31
- <|im_start|>assistant
32
-
33
- ```
34
-
35
- ## Available sizes
36
-
37
- | Branch | Bits | lm_head bits | VRAM (4k) | VRAM (16k) | Description |
38
- | ----- | ---- | ------- | ------ | ------ | ------------ |
39
- | [8_0](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/8_0) | 8.0 | 8.0 | 17.4 GB | 20.9 GB | Max quality that ExLlamaV2 can produce, **recommended**. |
40
- | [6_5](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/6_5) | 6.5 | 8.0 | 14.6 GB | 17.5 GB | Near unquantized performance at vastly reduced size, **recommended**. |
41
- | [5_0](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/5_0) | 5.0 | 6.0 | 11.6 GB | 14.4 GB | Slightly lower quality vs 6.5. |
42
- | [4_25](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/4_25) | 4.25 | 6.0 | 10.1 GB | 13.0 GB | GPTQ equivalent bits per weight. |
43
- | [3_5](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/3_5) | 3.5 | 6.0 | 8.7 GB | 11.5 GB | Lower quality, not recommended. |
44
- | [3_0](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/3_0) | 3.0 | 6.0 | 7.8 GB | 10.5 GB | Low quality, not recommended. |
45
-
46
- ## Download instructions
47
-
48
- With git:
49
-
50
- ```shell
51
- git clone --single-branch --branch 6_5 https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2 Qwen2.5-14B-Instruct-exl2-6_5
52
- ```
53
-
54
- With huggingface hub (credit to TheBloke for instructions):
55
-
56
- ```shell
57
- pip3 install huggingface-hub
58
- ```
59
-
60
- To download a specific branch, use the `--revision` parameter. For example, to download the 6.5 bpw branch:
61
-
62
- Linux:
63
-
64
- ```shell
65
- huggingface-cli download bartowski/Qwen2.5-14B-Instruct-exl2 --revision 6_5 --local-dir Qwen2.5-14B-Instruct-exl2-6_5
66
- ```
67
-
68
- Windows (which apparently doesn't like _ in folders sometimes?):
69
-
70
- ```shell
71
- huggingface-cli download bartowski/Qwen2.5-14B-Instruct-exl2 --revision 6_5 --local-dir Qwen2.5-14B-Instruct-exl2-6.5
72
- ```
73
-
74
- Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ license_link: https://huggingface.co/Qwen/Qwen2.5-14B-Instruct/blob/main/LICENSE
4
+ language:
5
+ - zho
6
+ - eng
7
+ - fra
8
+ - spa
9
+ - por
10
+ - deu
11
+ - ita
12
+ - rus
13
+ - jpn
14
+ - kor
15
+ - vie
16
+ - tha
17
+ - ara
18
+ pipeline_tag: text-generation
19
+ base_model: Qwen/Qwen2.5-14B-Instruct
20
+ base_model_relation: quantized
21
+ tags:
22
+ - chat
23
+ quantized_by: bartowski
24
+ ---
25
+
26
+ ## Exllama v2 Quantizations of Qwen2.5-14B-Instruct
27
+
28
+ Using <a href="https://github.com/turboderp/exllamav2/releases/tag/v0.2.2">turboderp's ExLlamaV2 v0.2.2</a> for quantization.
29
+
30
+ <b>The "main" branch only contains the measurement.json, download one of the other branches for the model (see below)</b>
31
+
32
+ Each branch contains an individual bits per weight, with the main one containing only the meaurement.json for further conversions.
33
+
34
+ Original model: https://huggingface.co/Qwen/Qwen2.5-14B-Instruct
35
+
36
+ ## Prompt format
37
+
38
+ ```
39
+ <|im_start|>system
40
+ {system_prompt}<|im_end|>
41
+ <|im_start|>user
42
+ {prompt}<|im_end|>
43
+ <|im_start|>assistant
44
+
45
+ ```
46
+
47
+ ## Available sizes
48
+
49
+ | Branch | Bits | lm_head bits | VRAM (4k) | VRAM (16k) | Description |
50
+ | ----- | ---- | ------- | ------ | ------ | ------------ |
51
+ | [8_0](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/8_0) | 8.0 | 8.0 | 17.4 GB | 20.9 GB | Max quality that ExLlamaV2 can produce, **recommended**. |
52
+ | [6_5](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/6_5) | 6.5 | 8.0 | 14.6 GB | 17.5 GB | Near unquantized performance at vastly reduced size, **recommended**. |
53
+ | [5_0](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/5_0) | 5.0 | 6.0 | 11.6 GB | 14.4 GB | Slightly lower quality vs 6.5. |
54
+ | [4_25](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/4_25) | 4.25 | 6.0 | 10.1 GB | 13.0 GB | GPTQ equivalent bits per weight. |
55
+ | [3_5](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/3_5) | 3.5 | 6.0 | 8.7 GB | 11.5 GB | Lower quality, not recommended. |
56
+ | [3_0](https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2/tree/3_0) | 3.0 | 6.0 | 7.8 GB | 10.5 GB | Low quality, not recommended. |
57
+
58
+ ## Download instructions
59
+
60
+ With git:
61
+
62
+ ```shell
63
+ git clone --single-branch --branch 6_5 https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-exl2 Qwen2.5-14B-Instruct-exl2-6_5
64
+ ```
65
+
66
+ With huggingface hub (credit to TheBloke for instructions):
67
+
68
+ ```shell
69
+ pip3 install huggingface-hub
70
+ ```
71
+
72
+ To download a specific branch, use the `--revision` parameter. For example, to download the 6.5 bpw branch:
73
+
74
+ Linux:
75
+
76
+ ```shell
77
+ huggingface-cli download bartowski/Qwen2.5-14B-Instruct-exl2 --revision 6_5 --local-dir Qwen2.5-14B-Instruct-exl2-6_5
78
+ ```
79
+
80
+ Windows (which apparently doesn't like _ in folders sometimes?):
81
+
82
+ ```shell
83
+ huggingface-cli download bartowski/Qwen2.5-14B-Instruct-exl2 --revision 6_5 --local-dir Qwen2.5-14B-Instruct-exl2-6.5
84
+ ```
85
+
86
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski