Create README.md

README.md

@@ -0,0 +1,99 @@
+---
+base_model:
+- microsoft/Phi-4-mini-instruct
+---
+# Phi-4-mini-instruct with llama-server
+
+This repository contains instructions for running the Phi-4-mini-instruct model using llama-server, which provides a ChatGPT-compatible API interface.
+
+## Prerequisites
+
+- [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) installed with server support
+- The Phi-4-mini-instruct model in GGUF format
+
+## Installation
+
+1. Install llama-cpp-python with server support:
+```bash
+pip install llama-cpp-python[server]
+```
+
+2. Ensure your model file is in the correct location:
+```bash
+models/Phi-4-mini-instruct-Q4_K_M-modified.gguf
+```
+
+## Running the Server
+
+Start the llama-server with the following command:
+
+```bash
+llama-server \
+    --model models/Phi-4-mini-instruct-Q4_K_M-modified.gguf \
+    --port 8082 \
+    --verbose \
+    --jinja
+```
+
+This will start the server with:
+- The model loaded in memory
+- Server running on port 8082
+- Verbose logging enabled
+- Jinja template support for chat formatting
+
+## Testing the API
+
+You can test the server using curl commands. Here are some examples:
+
+### Example 1: Generate HTML Hello World
+
+```bash
+curl http://localhost:8082/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "any-model",
+    "messages": [
+      {"role":"user","content":"give me an html hello world document"}
+    ]
+  }'
+```
+
+### Example 2: Tell a Joke
+
+```bash
+curl http://localhost:8082/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "any-model",
+    "messages": [
+      {"role":"user","content":"tell me a funny joke"}
+    ]
+  }'
+```
+
+## API Endpoints
+
+The server provides a ChatGPT-compatible API with the following main endpoints:
+
+- `/v1/chat/completions` - For chat completions
+- `/v1/completions` - For text completions
+- `/v1/models` - To list available models
+
+## Notes
+
+- The server uses the same API format as OpenAI's ChatGPT API, making it compatible with many existing tools and libraries
+- The `--jinja` flag enables proper chat template formatting for the model
+- The model name in the requests can be set to "any-model" as shown in the examples
+
+## Troubleshooting
+
+If you encounter issues:
+
+1. Ensure the model file exists in the specified path
+2. Check that port 8082 is not in use by another application
+3. Verify that llama-cpp-python is installed with server support
+4. Check the server logs with `--verbose` flag for detailed information
+
+## License
+
+Please ensure you comply with the model's license terms when using it.