Upload GGUF model

.gitattributes

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 keip-assistant.q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
+keip-assistant.q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -6,7 +6,7 @@ This is a GGUF version of the lora-merged model.
 
 - **Base Model:** /workspace/lora-merged
 - **Format:** GGUF 
-- **Quantization:** q4_k_m
+- **Quantization:** q8_0
 
 ## Usage
 
@@ -14,5 +14,5 @@ This model can be used with [llama.cpp](https://github.com/ggerganov/llama.cpp)
 
 ```bash
 # Example llama.cpp command
-./main -m keip-assistant.q4_k_m.gguf -n 1024 -p "Your prompt here"
+./main -m keip-assistant.q8_0.gguf -n 1024 -p "Your prompt here"
 ```

keip-assistant.q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:21ac1b4f17a7f386bdbb14680262ea79ba1fa30f40459af7c51b1035c42211e0
+size 8709518048