Hugging Face's logo

tensorblock
/
Qwen2.5-3B-Instruct-GGUF

Text Generation
Transformers
GGUF
English
chat
TensorBlock
GGUF
conversational
Model card Files
xet
 Community
1
morriszms commited on
Commit
3951aed
·
verified ·
1 Parent(s): ea59c45

Upload folder using huggingface_hub

Browse files
Files changed (13) hide show
  1. Qwen2.5-3B-Instruct-Q2_K.gguf +2 -2
  2. Qwen2.5-3B-Instruct-Q3_K_L.gguf +2 -2
  3. Qwen2.5-3B-Instruct-Q3_K_M.gguf +2 -2
  4. Qwen2.5-3B-Instruct-Q3_K_S.gguf +2 -2
  5. Qwen2.5-3B-Instruct-Q4_0.gguf +2 -2
  6. Qwen2.5-3B-Instruct-Q4_K_M.gguf +2 -2
  7. Qwen2.5-3B-Instruct-Q4_K_S.gguf +2 -2
  8. Qwen2.5-3B-Instruct-Q5_0.gguf +2 -2
  9. Qwen2.5-3B-Instruct-Q5_K_M.gguf +2 -2
  10. Qwen2.5-3B-Instruct-Q5_K_S.gguf +2 -2
  11. Qwen2.5-3B-Instruct-Q6_K.gguf +2 -2
  12. Qwen2.5-3B-Instruct-Q8_0.gguf +2 -2
  13. README.md +22 -22
Qwen2.5-3B-Instruct-Q2_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:03b1d04e01a318c128adbebf12bd26c340f945be4e6cea76b485e7837a3a286f
3
- size 1274755904
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c79f93fc98f6a030f1c2784ff37a20a2d22c3d84494bdfd98cd2286457fd3ea8
3
+ size 1274756000
Qwen2.5-3B-Instruct-Q3_K_L.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:eba9fdf5f1e38280c36d39c9d3d7aa60843660f026c5e72eb09fd00506f3e2fe
3
- size 1707391808
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dd5050e4b749f999c3156e2d75e7d0c55f305a79d7b541e53d9f29388d57114e
3
+ size 1707391904
Qwen2.5-3B-Instruct-Q3_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1c4e9fcdb3a7be8a7846888ca4b8f115d8edf178746155c5e10440ffb815b820
3
- size 1590475584
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9bbd3c5ecb9a56c582accd63f6677b3ecc19f92b7a8de542e041ceba9c192498
3
+ size 1590475680
Qwen2.5-3B-Instruct-Q3_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c87a36134e1e76f11b17dd2debe325117217dadaa3bfee87b1b24e940d8bf7fb
3
- size 1454357312
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7d2a10387d5d146dd7e913f9d8573c02cbe89c37d651798d420b2e3a874a1fc3
3
+ size 1454357408
Qwen2.5-3B-Instruct-Q4_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c76d654b2b96e472968227dc28809800207809613817936305babfc806f24971
3
- size 1822849856
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7fec8167834d6f9e264cbbb8eb9982d0462f652561f45415484328213544a6db
3
+ size 1822849952
Qwen2.5-3B-Instruct-Q4_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:662ad890ad2b6814f0cb22e4287bab8b31a0310416e5a3dcc20509b507a6839c
3
- size 1929902912
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:403a54ed5cd64fc91afeff2ef7c82da833da8c988b3f79ab04835016ed0c82f3
3
+ size 1929903008
Qwen2.5-3B-Instruct-Q4_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f3645d55b48f5c2c2a84e838ce145fbdcddb232250a7828071342acc9f30a5fe
3
- size 1834384192
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:83e1de83eba311b960a4fedceff42e6d3c6591cdf7c026965758958f686099af
3
+ size 1834384288
Qwen2.5-3B-Instruct-Q5_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:476ca726a82119d0032044e9109c087e2d0437285b9245aa0d93def102d713f1
3
- size 2169666368
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20156a1822f539d545ab97b42f3dd6592cf1aa0aebdac49811a78ae116ce0196
3
+ size 2169666464
Qwen2.5-3B-Instruct-Q5_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b952d138bc89a53d14b6ecd4fece300c4ec62c78e4b0d85daaa29628f41be230
3
- size 2224814912
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:57fcadd8f196d0bcfd4c078d48adc72b80f0ec27cacba05040743812c47d7242
3
+ size 2224815008
Qwen2.5-3B-Instruct-Q5_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7b8211c666e4e2f19670d60d530e110ba3f2fa2a2668fb29285661aac285698d
3
- size 2169666368
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:664365a22903013acbb757f987d80830a639525251ecb167f2a5ece466e75aa0
3
+ size 2169666464
Qwen2.5-3B-Instruct-Q6_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:17272db8b471409bc90be1ad435846f4c026295b826044058686fc97171ceca2
3
- size 2538158912
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9911a37e967e841a9ba67e871256b82fbc23d5b02544acfb1b988be415510cf
3
+ size 2538159008
Qwen2.5-3B-Instruct-Q8_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5c2fcd5ff136b8ed0670982ca73ec4ab9dc734654acf49272884456f587eaa38
3
- size 3285476160
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c5b531fae4490fc870e28d0076efbcf113707549d2a0ef1d95b06b7f917b6d05
3
+ size 3285476256
README.md CHANGED
@@ -1,14 +1,16 @@
1
  ---
2
- base_model: unsloth/Qwen2.5-3B-Instruct
 
 
3
  language:
4
  - en
5
- library_name: transformers
6
- license: other
7
  tags:
8
- - unsloth
9
- - transformers
10
  - TensorBlock
11
  - GGUF
 
12
  ---
13
 
14
  <div style="width: auto; margin-left: auto; margin-right: auto">
@@ -22,12 +24,11 @@ tags:
22
  </div>
23
  </div>
24
 
25
- ## unsloth/Qwen2.5-3B-Instruct - GGUF
26
-
27
- This repo contains GGUF format model files for [unsloth/Qwen2.5-3B-Instruct](https://huggingface.co/unsloth/Qwen2.5-3B-Instruct).
28
 
29
- The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
30
 
 
31
 
32
  <div style="text-align: left; margin: 20px 0;">
33
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
@@ -37,7 +38,6 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
37
 
38
  ## Prompt template
39
 
40
-
41
  ```
42
  <|im_start|>system
43
  {system_prompt}<|im_end|>
@@ -50,18 +50,18 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
50
 
51
  | Filename | Quant type | File Size | Description |
52
  | -------- | ---------- | --------- | ----------- |
53
- | [Qwen2.5-3B-Instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q2_K.gguf) | Q2_K | 1.187 GB | smallest, significant quality loss - not recommended for most purposes |
54
- | [Qwen2.5-3B-Instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q3_K_S.gguf) | Q3_K_S | 1.354 GB | very small, high quality loss |
55
- | [Qwen2.5-3B-Instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q3_K_M.gguf) | Q3_K_M | 1.481 GB | very small, high quality loss |
56
- | [Qwen2.5-3B-Instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q3_K_L.gguf) | Q3_K_L | 1.590 GB | small, substantial quality loss |
57
- | [Qwen2.5-3B-Instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q4_0.gguf) | Q4_0 | 1.698 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
58
- | [Qwen2.5-3B-Instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q4_K_S.gguf) | Q4_K_S | 1.708 GB | small, greater quality loss |
59
- | [Qwen2.5-3B-Instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q4_K_M.gguf) | Q4_K_M | 1.797 GB | medium, balanced quality - recommended |
60
- | [Qwen2.5-3B-Instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q5_0.gguf) | Q5_0 | 2.021 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
61
- | [Qwen2.5-3B-Instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q5_K_S.gguf) | Q5_K_S | 2.021 GB | large, low quality loss - recommended |
62
- | [Qwen2.5-3B-Instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q5_K_M.gguf) | Q5_K_M | 2.072 GB | large, very low quality loss - recommended |
63
- | [Qwen2.5-3B-Instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q6_K.gguf) | Q6_K | 2.364 GB | very large, extremely low quality loss |
64
- | [Qwen2.5-3B-Instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q8_0.gguf) | Q8_0 | 3.060 GB | very large, extremely low quality loss - not recommended |
65
 
66
 
67
  ## Downloading instruction
 
1
  ---
2
+ license: other
3
+ license_name: qwen-research
4
+ license_link: https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/main/LICENSE
5
  language:
6
  - en
7
+ pipeline_tag: text-generation
8
+ base_model: Qwen/Qwen2.5-3B-Instruct
9
  tags:
10
+ - chat
 
11
  - TensorBlock
12
  - GGUF
13
+ library_name: transformers
14
  ---
15
 
16
  <div style="width: auto; margin-left: auto; margin-right: auto">
 
24
  </div>
25
  </div>
26
 
27
+ ## Qwen/Qwen2.5-3B-Instruct - GGUF
 
 
28
 
29
+ This repo contains GGUF format model files for [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct).
30
 
31
+ The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit ec7f3ac](https://github.com/ggerganov/llama.cpp/commit/ec7f3ac9ab33e46b136eb5ab6a76c4d81f57c7f1).
32
 
33
  <div style="text-align: left; margin: 20px 0;">
34
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
 
38
 
39
  ## Prompt template
40
 
 
41
  ```
42
  <|im_start|>system
43
  {system_prompt}<|im_end|>
 
50
 
51
  | Filename | Quant type | File Size | Description |
52
  | -------- | ---------- | --------- | ----------- |
53
+ | [Qwen2.5-3B-Instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q2_K.gguf) | Q2_K | 1.275 GB | smallest, significant quality loss - not recommended for most purposes |
54
+ | [Qwen2.5-3B-Instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q3_K_S.gguf) | Q3_K_S | 1.454 GB | very small, high quality loss |
55
+ | [Qwen2.5-3B-Instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q3_K_M.gguf) | Q3_K_M | 1.590 GB | very small, high quality loss |
56
+ | [Qwen2.5-3B-Instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q3_K_L.gguf) | Q3_K_L | 1.707 GB | small, substantial quality loss |
57
+ | [Qwen2.5-3B-Instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q4_0.gguf) | Q4_0 | 1.823 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
58
+ | [Qwen2.5-3B-Instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q4_K_S.gguf) | Q4_K_S | 1.834 GB | small, greater quality loss |
59
+ | [Qwen2.5-3B-Instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q4_K_M.gguf) | Q4_K_M | 1.930 GB | medium, balanced quality - recommended |
60
+ | [Qwen2.5-3B-Instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q5_0.gguf) | Q5_0 | 2.170 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
61
+ | [Qwen2.5-3B-Instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q5_K_S.gguf) | Q5_K_S | 2.170 GB | large, low quality loss - recommended |
62
+ | [Qwen2.5-3B-Instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q5_K_M.gguf) | Q5_K_M | 2.225 GB | large, very low quality loss - recommended |
63
+ | [Qwen2.5-3B-Instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q6_K.gguf) | Q6_K | 2.538 GB | very large, extremely low quality loss |
64
+ | [Qwen2.5-3B-Instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-3B-Instruct-GGUF/blob/main/Qwen2.5-3B-Instruct-Q8_0.gguf) | Q8_0 | 3.285 GB | very large, extremely low quality loss - not recommended |
65
 
66
 
67
  ## Downloading instruction