Upload GGUF model

README.md

@@ -6,7 +6,7 @@ This is a GGUF version of the lora-merged model.
 
 - **Base Model:** /workspace/lora-merged
 - **Format:** GGUF 
-- **Quantization:** q8_0
+- **Quantization:** q4_k_m
 
 ## Usage
 
@@ -14,5 +14,5 @@ This model can be used with [llama.cpp](https://github.com/ggerganov/llama.cpp)
 
 ```bash
 # Example llama.cpp command
-./main -m keip-assistant.q8_0.gguf -n 1024 -p "Your prompt here"
+./main -m keip-assistant.q4_k_m.gguf -n 1024 -p "Your prompt here"
 ```

keip-assistant.q4_k_m.gguf

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c61d9d1e050509384597e0d9102d829fd039f2a8dd8c640772e52dc8e2ecaccb
+oid sha256:17b97e70b84b1a6684f2de6078e280f4623409e4f07370b273d3df077ff39630
 size 5027783392