Hugging Face's logo

tensorblock
/
Qwen2.5-32B-Instruct-GGUF

Text Generation
Transformers
GGUF
English
chat
TensorBlock
GGUF
conversational
Model card Files
xet
 Community
1
morriszms commited on
Commit
829f44a
·
verified ·
1 Parent(s): 6601be6

Upload folder using huggingface_hub

Browse files
Files changed (13) hide show
  1. Qwen2.5-32B-Instruct-Q2_K.gguf +2 -2
  2. Qwen2.5-32B-Instruct-Q3_K_L.gguf +2 -2
  3. Qwen2.5-32B-Instruct-Q3_K_M.gguf +2 -2
  4. Qwen2.5-32B-Instruct-Q3_K_S.gguf +2 -2
  5. Qwen2.5-32B-Instruct-Q4_0.gguf +2 -2
  6. Qwen2.5-32B-Instruct-Q4_K_M.gguf +2 -2
  7. Qwen2.5-32B-Instruct-Q4_K_S.gguf +2 -2
  8. Qwen2.5-32B-Instruct-Q5_0.gguf +2 -2
  9. Qwen2.5-32B-Instruct-Q5_K_M.gguf +2 -2
  10. Qwen2.5-32B-Instruct-Q5_K_S.gguf +2 -2
  11. Qwen2.5-32B-Instruct-Q6_K.gguf +2 -2
  12. Qwen2.5-32B-Instruct-Q8_0.gguf +2 -2
  13. README.md +21 -20
Qwen2.5-32B-Instruct-Q2_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b2e3bc5de3bdf7051fec54ac39b5a0bbfc945acf3d5c9d854df7d4a4b2360c8e
3
- size 12313098816
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33dd0781c3aaf72ab1a7fcf9199679b977272f4e150e6b6fa970520f7730c8c4
3
+ size 12313098848
Qwen2.5-32B-Instruct-Q3_K_L.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:84a87a71bd8912b73f70ce1be9e2f76bc34cbae5efaece806a6aa6275d2059ee
3
- size 17247078976
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b77748a87b2ac090327d9f33147210fb7bd8f57cbdd64c9e73f76ae9ee4d3b8e
3
+ size 17247079008
Qwen2.5-32B-Instruct-Q3_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:62f51d0f502aef7f43c43a29c3f61ec971a17fc292a71217c0d817a706c9364d
3
- size 15935048256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:420f0aa1aa34a417b8332e1bfcb5e29d5d76a7560b7ffef469890d27d22d1939
3
+ size 15935048288
Qwen2.5-32B-Instruct-Q3_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8f5d86ef78d07f20248576c7a2ee26b94d86a8fa1ae82c28cbb005f8c73a5c9d
3
- size 14392330816
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c4f902e56b7af8bbda45e69b7069b5af540a5a858cd952f1bef94f48e0121a1b
3
+ size 14392330848
Qwen2.5-32B-Instruct-Q4_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1d0be755ff00d865d72089e7429f4093b426af72bc32c48b22bf5bbdcb76e60b
3
- size 18640230976
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5678d2f9a37da898d38764a8a6c30367f1db1baccbd70fb82e7e3cf61331bbe8
3
+ size 18640231008
Qwen2.5-32B-Instruct-Q4_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4b26d2d713b4b6e0560b39dfe586b039664f6499c454c21be6b7ebc31271e8eb
3
- size 19851336256
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:82e471e517ea5b6f2ce32a8212c44c5acfd0392e51fc2ecf2edc3a1cf5a09125
3
+ size 19851336288
Qwen2.5-32B-Instruct-Q4_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:13928ec946bf6c1b29e9656f2964b98737d7997e16c6c9af9faaaf8f1cfa22f1
3
- size 18784410176
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5eeb67e98f208b69ad440275dd65c4ed088a4ce379fdd503246662fb8be1b73b
3
+ size 18784410208
Qwen2.5-32B-Instruct-Q5_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cf0919002dfe42562a6f03fd15b3cf44464ea4c1a0cfa0d83fc3d2a366e5b944
3
- size 22638254656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f8a76d615b11b8ea787c77d24c71e5c236262b404f0abd99807b620036e6932
3
+ size 22638254688
Qwen2.5-32B-Instruct-Q5_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:220e62ddde65b4035944429364d38c08f56eee1b130c3e0f90d1df2460f05c10
3
- size 23262157376
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f0bc736e92c10f621258975725a22f4a18926dc5af5044213fbe83ddc3de74b
3
+ size 23262157408
Qwen2.5-32B-Instruct-Q5_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5ac44fd4a65bad78e592366e6beaef4fcf9861c7ecb9d2ceed21a1bc365335dd
3
- size 22638254656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6571103650bbe1fcb51e253a4482a2aec98f076a971207f54322a815b8be46c
3
+ size 22638254688
Qwen2.5-32B-Instruct-Q6_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4d90a971e2ee53efcd11bb154f27ba5c0921e1d834a8da6105efeed0852980fb
3
- size 26886154816
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c8c52aeae4aac29a5aee723108dd8f62db966214431aa479ee249133d2d4b1c
3
+ size 26886154848
Qwen2.5-32B-Instruct-Q8_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:adce25ca19e438f4e03a1d58496193972d66863da975a6f9dfa664820bb7c668
3
- size 34820885056
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:21083b0d2e2b4c68d4e8396d0d9e9282c3a90eac6fc0d05ac062783d7fad4939
3
+ size 34820885088
README.md CHANGED
@@ -1,14 +1,15 @@
1
  ---
2
- base_model: unsloth/Qwen2.5-32B-Instruct
 
3
  language:
4
  - en
5
- library_name: transformers
6
- license: apache-2.0
7
  tags:
8
- - unsloth
9
- - transformers
10
  - TensorBlock
11
  - GGUF
 
12
  ---
13
 
14
  <div style="width: auto; margin-left: auto; margin-right: auto">
@@ -22,11 +23,11 @@ tags:
22
  </div>
23
  </div>
24
 
25
- ## unsloth/Qwen2.5-32B-Instruct - GGUF
26
 
27
- This repo contains GGUF format model files for [unsloth/Qwen2.5-32B-Instruct](https://huggingface.co/unsloth/Qwen2.5-32B-Instruct).
28
 
29
- The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
30
 
31
  <div style="text-align: left; margin: 20px 0;">
32
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
@@ -48,18 +49,18 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
48
 
49
  | Filename | Quant type | File Size | Description |
50
  | -------- | ---------- | --------- | ----------- |
51
- | [Qwen2.5-32B-Instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q2_K.gguf) | Q2_K | 11.467 GB | smallest, significant quality loss - not recommended for most purposes |
52
- | [Qwen2.5-32B-Instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q3_K_S.gguf) | Q3_K_S | 13.404 GB | very small, high quality loss |
53
- | [Qwen2.5-32B-Instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q3_K_M.gguf) | Q3_K_M | 14.841 GB | very small, high quality loss |
54
- | [Qwen2.5-32B-Instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q3_K_L.gguf) | Q3_K_L | 16.063 GB | small, substantial quality loss |
55
- | [Qwen2.5-32B-Instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q4_0.gguf) | Q4_0 | 17.360 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
56
- | [Qwen2.5-32B-Instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q4_K_S.gguf) | Q4_K_S | 17.494 GB | small, greater quality loss |
57
- | [Qwen2.5-32B-Instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q4_K_M.gguf) | Q4_K_M | 18.488 GB | medium, balanced quality - recommended |
58
- | [Qwen2.5-32B-Instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q5_0.gguf) | Q5_0 | 21.084 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
59
- | [Qwen2.5-32B-Instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q5_K_S.gguf) | Q5_K_S | 21.084 GB | large, low quality loss - recommended |
60
- | [Qwen2.5-32B-Instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q5_K_M.gguf) | Q5_K_M | 21.665 GB | large, very low quality loss - recommended |
61
- | [Qwen2.5-32B-Instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q6_K.gguf) | Q6_K | 25.040 GB | very large, extremely low quality loss |
62
- | [Qwen2.5-32B-Instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q8_0.gguf) | Q8_0 | 32.429 GB | very large, extremely low quality loss - not recommended |
63
 
64
 
65
  ## Downloading instruction
 
1
  ---
2
+ license: apache-2.0
3
+ license_link: https://huggingface.co/Qwen/Qwen2.5-32B-Instruct/blob/main/LICENSE
4
  language:
5
  - en
6
+ pipeline_tag: text-generation
7
+ base_model: Qwen/Qwen2.5-32B-Instruct
8
  tags:
9
+ - chat
 
10
  - TensorBlock
11
  - GGUF
12
+ library_name: transformers
13
  ---
14
 
15
  <div style="width: auto; margin-left: auto; margin-right: auto">
 
23
  </div>
24
  </div>
25
 
26
+ ## Qwen/Qwen2.5-32B-Instruct - GGUF
27
 
28
+ This repo contains GGUF format model files for [Qwen/Qwen2.5-32B-Instruct](https://huggingface.co/Qwen/Qwen2.5-32B-Instruct).
29
 
30
+ The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit ec7f3ac](https://github.com/ggerganov/llama.cpp/commit/ec7f3ac9ab33e46b136eb5ab6a76c4d81f57c7f1).
31
 
32
  <div style="text-align: left; margin: 20px 0;">
33
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
 
49
 
50
  | Filename | Quant type | File Size | Description |
51
  | -------- | ---------- | --------- | ----------- |
52
+ | [Qwen2.5-32B-Instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q2_K.gguf) | Q2_K | 12.313 GB | smallest, significant quality loss - not recommended for most purposes |
53
+ | [Qwen2.5-32B-Instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q3_K_S.gguf) | Q3_K_S | 14.392 GB | very small, high quality loss |
54
+ | [Qwen2.5-32B-Instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q3_K_M.gguf) | Q3_K_M | 15.935 GB | very small, high quality loss |
55
+ | [Qwen2.5-32B-Instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q3_K_L.gguf) | Q3_K_L | 17.247 GB | small, substantial quality loss |
56
+ | [Qwen2.5-32B-Instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q4_0.gguf) | Q4_0 | 18.640 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
57
+ | [Qwen2.5-32B-Instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q4_K_S.gguf) | Q4_K_S | 18.784 GB | small, greater quality loss |
58
+ | [Qwen2.5-32B-Instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q4_K_M.gguf) | Q4_K_M | 19.851 GB | medium, balanced quality - recommended |
59
+ | [Qwen2.5-32B-Instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q5_0.gguf) | Q5_0 | 22.638 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
60
+ | [Qwen2.5-32B-Instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q5_K_S.gguf) | Q5_K_S | 22.638 GB | large, low quality loss - recommended |
61
+ | [Qwen2.5-32B-Instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q5_K_M.gguf) | Q5_K_M | 23.262 GB | large, very low quality loss - recommended |
62
+ | [Qwen2.5-32B-Instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q6_K.gguf) | Q6_K | 26.886 GB | very large, extremely low quality loss |
63
+ | [Qwen2.5-32B-Instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q8_0.gguf) | Q8_0 | 34.821 GB | very large, extremely low quality loss - not recommended |
64
 
65
 
66
  ## Downloading instruction