Hugging Face's logo

tensorblock
/
Phi-3-mini-4k-instruct-GGUF

Text Generation
GGUF
English
French
nlp
code
TensorBlock
GGUF
conversational
Model card Files
xet
 Community
morriszms commited on
Commit
186b073
·
verified ·
1 Parent(s): c82600a

Upload folder using huggingface_hub

Browse files
Files changed (13) hide show
  1. Phi-3-mini-4k-instruct-Q2_K.gguf +2 -2
  2. Phi-3-mini-4k-instruct-Q3_K_L.gguf +2 -2
  3. Phi-3-mini-4k-instruct-Q3_K_M.gguf +2 -2
  4. Phi-3-mini-4k-instruct-Q3_K_S.gguf +2 -2
  5. Phi-3-mini-4k-instruct-Q4_0.gguf +2 -2
  6. Phi-3-mini-4k-instruct-Q4_K_M.gguf +2 -2
  7. Phi-3-mini-4k-instruct-Q4_K_S.gguf +2 -2
  8. Phi-3-mini-4k-instruct-Q5_0.gguf +2 -2
  9. Phi-3-mini-4k-instruct-Q5_K_M.gguf +2 -2
  10. Phi-3-mini-4k-instruct-Q5_K_S.gguf +2 -2
  11. Phi-3-mini-4k-instruct-Q6_K.gguf +2 -2
  12. Phi-3-mini-4k-instruct-Q8_0.gguf +2 -2
  13. README.md +29 -24
Phi-3-mini-4k-instruct-Q2_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9b82044047ddec6ab4137d06651361840a7f1008a0eae8eea597e27759fbadec
3
- size 1446880320
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5c0e83c5e290178f8b4c5269e9a23a57324732ac1b76153f97577fb1e5679aa4
3
+ size 1416203712
Phi-3-mini-4k-instruct-Q3_K_L.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:44808ba99c26ca5c89ee29d1ff1c294675d06c07b9cebda5d78841cd6830288c
3
- size 2045135424
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:331605d5190e26113ad753e1c5069f0b7df2dbe667c7e16de67f3cadde522e1b
3
+ size 2087596992
Phi-3-mini-4k-instruct-Q3_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:80f28d845dc4c6d0fef784362655f364c7a1ed196d9f858af06ca662e99065a4
3
- size 1877625408
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b3560662b4c07ce745c707e25c0e4abf6bafe089f5db34494d1a1bc41caa2bc3
3
+ size 1955476416
Phi-3-mini-4k-instruct-Q3_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:32dbfe5c6000c4c6bb4e3bc2f679f37329da4ccdb893948252ff225c28bff9cb
3
- size 1681803840
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:663014a5768d1019d81dafbe59cbf59635bd50726b9f55839fb91cd67d3dd1f4
3
+ size 1681798080
Phi-3-mini-4k-instruct-Q4_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a2cd87ccae8eb2b0836ffd7a7a3bc122ca6a62d0f5cd93dc983c0859f6e1e7b9
3
- size 2176182336
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:decac1a182b9f80385499afe2e36c2fee078a04ab99efa2e4e95eb86b1090785
3
+ size 2176176576
Phi-3-mini-4k-instruct-Q4_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f83f14c7bbfd894a9a7502cfbd9a6759ce8286aa9799924624f529c647a8efe5
3
- size 2318919744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe6cd2128eebaedf996a84706ba3218827a77680180dd36f34a5640d7a0f38c5
3
+ size 2393231808
Phi-3-mini-4k-instruct-Q4_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9e6ac67b3ed7929d3b63c1e00220340c19295c1d278ab11b7289c88fa7b187ec
3
- size 2193483840
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e1893c6b8cbd400b1eb4afa300fe9ce85874bb1e55136b72ac4ca6468438de6
3
+ size 2188759488
Phi-3-mini-4k-instruct-Q5_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ce547afba1d0927c083583b851ff27aaaaf9dbab2064ef92485fc6b9fd70fd35
3
- size 2641479744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d949ec396b38ea5087ccc2cdfa644c32f077d950cdf1e1cc2e51f4b63a75de9
3
+ size 2641473984
Phi-3-mini-4k-instruct-Q5_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ea219f963f6eee55169060fc1a54185dd308a7cac14061a4653d7ed9d06a3412
3
- size 2715011136
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6025d3f6250d1856442a8ee0e7c968315d8a131f8d4a9e7a53850e8062c99a01
3
+ size 2815275456
Phi-3-mini-4k-instruct-Q5_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a692c07003686dfa5bd7a39826217e45ca9db89021762c6ca4d0cdc769115b8d
3
- size 2641479744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:15db46bced3de796478eb4eb0b13c2382b25d1f4ccd09e98be42ae59f3537ae4
3
+ size 2641473984
Phi-3-mini-4k-instruct-Q6_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9c7e9e8bad768b2e4badcef1ec0d809fa6f81fb84a9353c22e31bfa0d5d4d1ab
3
- size 3135858240
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f09cd1c0e3aae6a64f6ed4d2b8bb059ca7880e35285b476b148d7b592c2bd4d
3
+ size 3135852480
Phi-3-mini-4k-instruct-Q8_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:863a8f851c2108ab1fed787f26deba4eeff4c5fa3e59ff7413363124e9493f35
3
- size 4061227584
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49a323d0b284896256bcb741c5ca60fee85ac8268f9d878cf97bc6cc150340f4
3
+ size 4061221824
README.md CHANGED
@@ -1,16 +1,23 @@
1
  ---
 
 
2
  language:
3
  - en
4
- library_name: transformers
5
- license: mit
6
  tags:
7
- - unsloth
8
- - transformers
9
- - phi3
10
- - phi
11
  - TensorBlock
12
  - GGUF
13
- base_model: unsloth/Phi-3-mini-4k-instruct
 
 
 
 
 
 
 
14
  ---
15
 
16
  <div style="width: auto; margin-left: auto; margin-right: auto">
@@ -24,12 +31,11 @@ base_model: unsloth/Phi-3-mini-4k-instruct
24
  </div>
25
  </div>
26
 
27
- ## unsloth/Phi-3-mini-4k-instruct - GGUF
28
 
29
- This repo contains GGUF format model files for [unsloth/Phi-3-mini-4k-instruct](https://huggingface.co/unsloth/Phi-3-mini-4k-instruct).
30
-
31
- The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
32
 
 
33
 
34
  <div style="text-align: left; margin: 20px 0;">
35
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
@@ -39,7 +45,6 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
39
 
40
  ## Prompt template
41
 
42
-
43
  ```
44
  <|system|>
45
  {system_prompt}<|end|>
@@ -52,18 +57,18 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
52
 
53
  | Filename | Quant type | File Size | Description |
54
  | -------- | ---------- | --------- | ----------- |
55
- | [Phi-3-mini-4k-instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q2_K.gguf) | Q2_K | 1.348 GB | smallest, significant quality loss - not recommended for most purposes |
56
- | [Phi-3-mini-4k-instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q3_K_S.gguf) | Q3_K_S | 1.566 GB | very small, high quality loss |
57
- | [Phi-3-mini-4k-instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q3_K_M.gguf) | Q3_K_M | 1.749 GB | very small, high quality loss |
58
- | [Phi-3-mini-4k-instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q3_K_L.gguf) | Q3_K_L | 1.905 GB | small, substantial quality loss |
59
- | [Phi-3-mini-4k-instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q4_0.gguf) | Q4_0 | 2.027 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
60
- | [Phi-3-mini-4k-instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q4_K_S.gguf) | Q4_K_S | 2.043 GB | small, greater quality loss |
61
- | [Phi-3-mini-4k-instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q4_K_M.gguf) | Q4_K_M | 2.160 GB | medium, balanced quality - recommended |
62
- | [Phi-3-mini-4k-instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q5_0.gguf) | Q5_0 | 2.460 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
63
- | [Phi-3-mini-4k-instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q5_K_S.gguf) | Q5_K_S | 2.460 GB | large, low quality loss - recommended |
64
- | [Phi-3-mini-4k-instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q5_K_M.gguf) | Q5_K_M | 2.529 GB | large, very low quality loss - recommended |
65
- | [Phi-3-mini-4k-instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q6_K.gguf) | Q6_K | 2.920 GB | very large, extremely low quality loss |
66
- | [Phi-3-mini-4k-instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q8_0.gguf) | Q8_0 | 3.782 GB | very large, extremely low quality loss - not recommended |
67
 
68
 
69
  ## Downloading instruction
 
1
  ---
2
+ license: mit
3
+ license_link: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/resolve/main/LICENSE
4
  language:
5
  - en
6
+ - fr
7
+ pipeline_tag: text-generation
8
  tags:
9
+ - nlp
10
+ - code
 
 
11
  - TensorBlock
12
  - GGUF
13
+ inference:
14
+ parameters:
15
+ temperature: 0
16
+ widget:
17
+ - messages:
18
+ - role: user
19
+ content: Can you provide ways to eat combinations of bananas and dragonfruits?
20
+ base_model: microsoft/Phi-3-mini-4k-instruct
21
  ---
22
 
23
  <div style="width: auto; margin-left: auto; margin-right: auto">
 
31
  </div>
32
  </div>
33
 
34
+ ## microsoft/Phi-3-mini-4k-instruct - GGUF
35
 
36
+ This repo contains GGUF format model files for [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct).
 
 
37
 
38
+ The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit ec7f3ac](https://github.com/ggerganov/llama.cpp/commit/ec7f3ac9ab33e46b136eb5ab6a76c4d81f57c7f1).
39
 
40
  <div style="text-align: left; margin: 20px 0;">
41
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
 
45
 
46
  ## Prompt template
47
 
 
48
  ```
49
  <|system|>
50
  {system_prompt}<|end|>
 
57
 
58
  | Filename | Quant type | File Size | Description |
59
  | -------- | ---------- | --------- | ----------- |
60
+ | [Phi-3-mini-4k-instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q2_K.gguf) | Q2_K | 1.416 GB | smallest, significant quality loss - not recommended for most purposes |
61
+ | [Phi-3-mini-4k-instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q3_K_S.gguf) | Q3_K_S | 1.682 GB | very small, high quality loss |
62
+ | [Phi-3-mini-4k-instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q3_K_M.gguf) | Q3_K_M | 1.955 GB | very small, high quality loss |
63
+ | [Phi-3-mini-4k-instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q3_K_L.gguf) | Q3_K_L | 2.088 GB | small, substantial quality loss |
64
+ | [Phi-3-mini-4k-instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q4_0.gguf) | Q4_0 | 2.176 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
65
+ | [Phi-3-mini-4k-instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q4_K_S.gguf) | Q4_K_S | 2.189 GB | small, greater quality loss |
66
+ | [Phi-3-mini-4k-instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q4_K_M.gguf) | Q4_K_M | 2.393 GB | medium, balanced quality - recommended |
67
+ | [Phi-3-mini-4k-instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q5_0.gguf) | Q5_0 | 2.641 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
68
+ | [Phi-3-mini-4k-instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q5_K_S.gguf) | Q5_K_S | 2.641 GB | large, low quality loss - recommended |
69
+ | [Phi-3-mini-4k-instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q5_K_M.gguf) | Q5_K_M | 2.815 GB | large, very low quality loss - recommended |
70
+ | [Phi-3-mini-4k-instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q6_K.gguf) | Q6_K | 3.136 GB | very large, extremely low quality loss |
71
+ | [Phi-3-mini-4k-instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Phi-3-mini-4k-instruct-GGUF/blob/main/Phi-3-mini-4k-instruct-Q8_0.gguf) | Q8_0 | 4.061 GB | very large, extremely low quality loss - not recommended |
72
 
73
 
74
  ## Downloading instruction