Add llama.cpp to the examples

README.md

@@ -107,6 +107,7 @@ The model can also be deployed with the following libraries:
 - [`mistral-inference`](https://github.com/mistralai/mistral-inference): See [here](#mistral-inference)
 - [`transformers`](https://github.com/huggingface/transformers): See [here](#transformers)
 - [`LMStudio`](https://lmstudio.ai/): See [here](#lmstudio)
+- [`llama.cpp`](https://github.com/ggml-org/llama.cpp): See [here](#llama.cpp)
 - [`ollama`](https://github.com/ollama/ollama): See [here](#ollama)
 
 
@@ -394,6 +395,23 @@ docker run -it --rm --pull=always \
 Click “see advanced setting” on the second line. 
 In the new tab, toggle advanced to on. Set the custom model to be mistral/devstralq4_k_m and Base URL the api address we get from the last step in LM Studio. Set API Key to dummy. Click save changes.
 
+### llama.cpp
+
+Download the weights from huggingface:
+
+```
+pip install -U "huggingface_hub[cli]"
+huggingface-cli download \
+"mistralai/Devstral-Small-2505_gguf" \
+--include "devstralQ4_K_M.gguf" \
+--local-dir "mistralai/Devstral-Small-2505_gguf/"
+```
+
+Then run Devstral using the llama.cpp CLI.
+
+```bash
+./llama-cli -m Devstral-Small-2505_gguf/devstralQ4_K_M.gguf -cnv
+```
 
 ### Ollama