Update

Signed-off-by: Xin Liu <sam@secondstate.io>

README.md

@@ -1,15 +1,76 @@
-# Deepseek-Coder-6.7B-Instruct
-
-## Prompt Template
-
-```text
-You are an AI programming assistant, utilizing the DeepSeek Coder model, developed by DeepSeek Company, and you only answer questions related to computer science. For politically sensitive questions, security and privacy issues, and other non-computer science questions, you will refuse to answer.
-### Instruction:
-{question_1}
-### Response:
-{answer_1}
-<|EOT|>
-### Instruction:
-{question_2}
-### Response:
-```
+---
+base_model: deepseek-ai/deepseek-coder-6.7b-instruct
+inference: false
+license: other
+license_link: LICENSE
+license_name: deepseek
+model_creator: DeepSeek
+model_name: Deepseek Coder 6.7B Instruct
+model_type: deepseek
+quantized_by: Second State Inc.
+---
+
+<!-- header start -->
+<!-- 200823 -->
+<div style="width: auto; margin-left: auto; margin-right: auto">
+<img src="https://github.com/second-state/LlamaEdge/raw/dev/assets/logo.svg" style="width: 100%; min-width: 400px; display: block; margin: auto;">
+</div>
+<hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
+<!-- header end -->
+
+# Deepseek-Coder-6.7B-Instruct-GGUF
+
+## Original Model
+
+[deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct)
+
+## Run with LlamaEdge
+
+- LlamaEdge version: [v0.2.4](https://github.com/second-state/LlamaEdge/releases/tag/0.2.4)
+
+- Prompt template
+
+  - Prompt type: `deepseek-coder`
+
+  - Prompt string
+
+    ```text
+    {system}
+    ### Instruction:
+    {question_1}
+    ### Response:
+    {answer_1}
+    <|EOT|>
+    ### Instruction:
+    {question_2}
+    ### Response:
+    ```
+
+- Run as LlamaEdge service
+
+  ```bash
+  wasmedge --dir .:. --nn-preload default:GGML:AUTO:deepseek-coder-6.7b-instruct.Q5_K_M.gguf llama-api-server.wasm -p deepseek-coder
+  ```
+
+- Run as LlamaEdge command app
+
+  ```bash
+  wasmedge --dir .:. --nn-preload default:GGML:AUTO:deepseek-coder-6.7b-instruct.Q5_K_M.gguf llama-chat.wasm -p deepseek-coder
+  ```
+
+## Quantized GGUF Models
+
+| Name | Quant method | Bits | Size | Use case |
+| ---- | ---- | ---- | ---- | ----- |
+| [deepseek-coder-6.7b-instruct-Q2_K.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q2_K.gguf)     | Q2_K   | 2 | 2.53 GB| smallest, significant quality loss - not recommended for most purposes |
+| [deepseek-coder-6.7b-instruct-Q3_K_L.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q3_K_L.gguf) | Q3_K_L | 3 | 3.6 GB| small, substantial quality loss |
+| [deepseek-coder-6.7b-instruct-Q3_K_M.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q3_K_M.gguf) | Q3_K_M | 3 | 3.3 GB| very small, high quality loss |
+| [deepseek-coder-6.7b-instruct-Q3_K_S.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q3_K_S.gguf) | Q3_K_S | 3 | 2.95 GB| very small, high quality loss |
+| [deepseek-coder-6.7b-instruct-Q4_0.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q4_0.gguf)     | Q4_0   | 4 | 3.83 GB| legacy; small, very high quality loss - prefer using Q3_K_M |
+| [deepseek-coder-6.7b-instruct-Q4_K_M.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q4_K_M.gguf) | Q4_K_M | 4 | 4.08 GB| medium, balanced quality - recommended |
+| [deepseek-coder-6.7b-instruct-Q4_K_S.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q4_K_S.gguf) | Q4_K_S | 4 | 3.86 GB| small, greater quality loss |
+| [deepseek-coder-6.7b-instruct-Q5_0.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q5_0.gguf)     | Q5_0   | 5 | 4.65 GB| legacy; medium, balanced quality - prefer using Q4_K_M |
+| [deepseek-coder-6.7b-instruct-Q5_K_M.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q5_K_M.gguf) | Q5_K_M | 5 | 4.79 GB| large, very low quality loss - recommended |
+| [deepseek-coder-6.7b-instruct-Q5_K_S.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q5_K_S.gguf) | Q5_K_S | 5 | 4.65 GB| large, low quality loss - recommended |
+| [deepseek-coder-6.7b-instruct-Q6_K.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q6_K.gguf)     | Q6_K   | 6 | 5.53 GB| very large, extremely low quality loss |
+| [deepseek-coder-6.7b-instruct-Q8_0.gguf](https://huggingface.co/second-state/Deepseek-Coder-6.7B-Instruct-GGUF/blob/main/deepseek-coder-6.7b-instruct-Q8_0.gguf)     | Q8_0   | 8 | 7.16 GB| very large, extremely low quality loss - not recommended |