Add instructions for Ollama

README.md

@@ -97,6 +97,42 @@ We followed the instructions in the [dpo repo](https://github.com/eric-mitchell/
 | LLM360/AmberChat | 5.428125 |
 | **LLM360/AmberSafe** | **4.725000** |
 
+
+# Using Quantized Models with Ollama
+
+Please follow these steps to use a quantized version of AmberSafe on your personal computer or laptop:
+
+1. First, install Ollama by following the instructions provided [here](https://github.com/jmorganca/ollama/tree/main?tab=readme-ov-file#ollama). Next, create a quantized version of AmberSafe model (say ambersafe.Q8_0.gguf for 8 bit quantized version) following instructions [here](https://github.com/jmorganca/ollama/blob/main/docs/import.md#manually-converting--quantizing-models). 
+
+2. Create an Ollama Modelfile locally using the template provided below:
+```
+FROM ambersafe.Q8_0.gguf
+
+TEMPLATE """{{ .System }}
+USER: {{ .Prompt }}
+ASSISTANT:
+"""
+SYSTEM """A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
+"""
+PARAMETER stop "USER:"
+PARAMETER stop "ASSISTANT:"
+PARAMETER repeat_last_n   0
+PARAMETER num_ctx         2048
+PARAMETER seed            0
+PARAMETER num_predict    -1
+```
+Ensure that the FROM directive points to the created checkpoint file.
+
+3. Now, you can proceed to build the model by running:
+```bash
+ollama create ambersafe -f Modelfile
+```
+4. To run the model from the command line, execute the following:
+```bash
+ollama run ambersafe
+```
+You need to build the model once and can just run it afterwards.
+
 # Citation
 
 **BibTeX:**