Hugging Face's logo

bartowski
/
Mistral-Plus-7B-GGUF

Text Generation
GGUF
Model card Files
xet
 Community
bartowski commited on
Commit
3861c98
·
verified ·
1 Parent(s): 5b3cc07

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. Mistral-Plus-7B-IQ3_M.gguf +3 -0
  3. Mistral-Plus-7B-IQ3_S.gguf +3 -0
  4. Mistral-Plus-7B-IQ4_NL.gguf +3 -0
  5. Mistral-Plus-7B-IQ4_XS.gguf +3 -0
  6. Mistral-Plus-7B-Q2_K.gguf +3 -0
  7. Mistral-Plus-7B-Q3_K_L.gguf +3 -0
  8. Mistral-Plus-7B-Q3_K_M.gguf +3 -0
  9. Mistral-Plus-7B-Q3_K_S.gguf +3 -0
  10. Mistral-Plus-7B-Q4_0.gguf +3 -0
  11. Mistral-Plus-7B-Q4_K_M.gguf +3 -0
  12. Mistral-Plus-7B-Q4_K_S.gguf +3 -0
  13. Mistral-Plus-7B-Q5_0.gguf +3 -0
  14. Mistral-Plus-7B-Q5_K_M.gguf +3 -0
  15. Mistral-Plus-7B-Q5_K_S.gguf +3 -0
  16. Mistral-Plus-7B-Q6_K.gguf +3 -0
  17. Mistral-Plus-7B-Q8_0.gguf +3 -0
  18. README.md +30 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Mistral-Plus-7B-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Mistral-Plus-7B-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Mistral-Plus-7B-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Mistral-Plus-7B-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Mistral-Plus-7B-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Mistral-Plus-7B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Mistral-Plus-7B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Mistral-Plus-7B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Mistral-Plus-7B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Mistral-Plus-7B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Mistral-Plus-7B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Mistral-Plus-7B-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Mistral-Plus-7B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Mistral-Plus-7B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Mistral-Plus-7B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Mistral-Plus-7B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Mistral-Plus-7B-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:71fe2351128e97b94220d786cc2a14a12bbe0b03e416f5fc42e9356309706e62
3
+ size 3284891360
Mistral-Plus-7B-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6129139fd0823d71ce86da3ef35dcc52ef42d63c1b2ff794ecac2c4365cc965d
3
+ size 3182393056
Mistral-Plus-7B-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:699208d01bb61cff33f7ba3c8132ce0dad19f31ea148c42772fb5d348f83b3d5
3
+ size 4155053792
Mistral-Plus-7B-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0430fd1bcc2351de2744600c2749629c8348a13544d8b08b6481b45e2764aec0
3
+ size 3944388320
Mistral-Plus-7B-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8bffe37a6140acbc87b276c9043913740828fa264cabab68f1bb9c1501f6a39c
3
+ size 2719241952
Mistral-Plus-7B-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb37657610c3e2726ae66abe6b79c6d1cf0bbd2b6e35e042520de12e854e2dcb
3
+ size 3822024416
Mistral-Plus-7B-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ba958b110b3ad8a5ec42db4f95655421c32ef6c8e1d1bfc1d8930b58bfe01f2
3
+ size 3518985952
Mistral-Plus-7B-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d780b7536cbfdcb928b17664d08ef49461c8e8326460b2c128bdcc82b9f988e9
3
+ size 3164567264
Mistral-Plus-7B-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ac0c1897e0e37c6885c633b332e8e9f5f10623d9ace408068e5592a5a0ba8b1
3
+ size 4108916448
Mistral-Plus-7B-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84eac1003f367ed9bdd8e53ccd1b5b9ac56c4ccf0078690556d6c328a1c0f4d6
3
+ size 4368439008
Mistral-Plus-7B-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1c8b2b0663e2e1c5655abd6a1c10af2454501d06307adc53815db99304c81d4
3
+ size 4140373728
Mistral-Plus-7B-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c02f53711d24e8f5c66103e1d511152a8dc2dcac634c786518196b05ded620a3
3
+ size 4997715680
Mistral-Plus-7B-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:37400adcdb8dc19d4b2c6c3145a2c13711d3283cdd14a6891b383ed4aea34fde
3
+ size 5131409120
Mistral-Plus-7B-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:081f3dcafcb2e1fdd24d723f910e944148d112a78056c2a75f4ec388357db075
3
+ size 4997715680
Mistral-Plus-7B-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e0280a4ef8e5bfba8380c52e3644be20f8da8507ccb1d502184085bdd2af7fa7
3
+ size 5942064864
Mistral-Plus-7B-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac902650569b3b8de208bbdc84b9ce991c6f319be1bbf2cc512d4f501b0fc81a
3
+ size 7695857376
README.md ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ quantized_by: bartowski
4
+ pipeline_tag: text-generation
5
+ ---
6
+
7
+ ## Llamacpp Quantizations of Mistral-Plus-7B
8
+
9
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
10
+
11
+ Original model: https://huggingface.co/zhengchenphd/Mistral-Plus-7B
12
+
13
+ Download a file (not the whole branch) from below:
14
+
15
+ | Filename | Quant type | File Size | Description |
16
+ | -------- | ---------- | --------- | ----------- |
17
+ | [Mistral-Plus-7B-Q8_0.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
18
+ | [Mistral-Plus-7B-Q6_K.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
19
+ | [Mistral-Plus-7B-Q5_K_M.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, very usable. |
20
+ | [Mistral-Plus-7B-Q5_K_S.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, very usable. |
21
+ | [Mistral-Plus-7B-Q5_0.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
22
+ | [Mistral-Plus-7B-Q4_K_M.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, similar to 4.25 bpw. |
23
+ | [Mistral-Plus-7B-Q4_K_S.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
24
+ | [Mistral-Plus-7B-Q4_0.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
25
+ | [Mistral-Plus-7B-Q3_K_L.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
26
+ | [Mistral-Plus-7B-Q3_K_M.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
27
+ | [Mistral-Plus-7B-Q3_K_S.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
28
+ | [Mistral-Plus-7B-Q2_K.gguf](https://huggingface.co/bartowski/Mistral-Plus-7B-GGUF/blob/main/Mistral-Plus-7B-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended.
29
+
30
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski