Snider Virgil commited on
Commit ·
0dedfe4
1
Parent(s): 802f6f1
feat: add Q4_K_M + Q8_0 + BF16 gguf for Ollama / llama.cpp consumers
Browse filesConverted from model.safetensors via llama.cpp convert_hf_to_gguf.py
(bf16 intermediate) and llama-quantize (Q4_K_M + Q8_0).
Ollama pull paths:
ollama pull hf.co/LetheanNetwork/lemmy:Q4_K_M
ollama pull hf.co/LetheanNetwork/lemmy:Q8_0
ollama pull hf.co/LetheanNetwork/lemmy:BF16
Co-Authored-By: Virgil <virgil@lethean.io>
- .gitattributes +1 -0
- lemmy-bf16.gguf +3 -0
- lemmy-q4_k_m.gguf +3 -0
- lemmy-q8_0.gguf +3 -0
.gitattributes
CHANGED
|
@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
model-00001-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 38 |
model-00002-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
model-00001-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 38 |
model-00002-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
*.gguf filter=lfs diff=lfs merge=lfs -text
|
lemmy-bf16.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8bf106f3601cfefb0f7ee17f55d56be2484e7feee81ff88a94144ddecb4ce369
|
| 3 |
+
size 50505130624
|
lemmy-q4_k_m.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:efe9c2eb37a167d8e4e500d13f9ad62d253c1f77f8aa49b3c36c79ea5a7a5293
|
| 3 |
+
size 16796011136
|
lemmy-q8_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:39c3e3939f89de76abc40482b6a97d35aa3fff970a85a9c7f15b6da3432b9c12
|
| 3 |
+
size 26859854464
|