ariel-pillar commited on
Commit
e29b963
·
verified ·
1 Parent(s): 8ae50d9

make clear that readme is about tool use

Browse files
Files changed (1) hide show
  1. README.md +39 -9
README.md CHANGED
@@ -1,10 +1,18 @@
1
- ---
2
- base_model:
3
- - microsoft/Phi-4-mini-instruct
4
- ---
5
- # Phi-4-mini-instruct with llama-server
6
 
7
- This repository contains instructions for running the Phi-4-mini-instruct model using llama-server, which provides a ChatGPT-compatible API interface.
 
 
 
 
 
 
 
 
8
 
9
  ## Prerequisites
10
 
@@ -31,7 +39,6 @@ Start the llama-server with the following command:
31
  llama-server \
32
  --model models/Phi-4-mini-instruct-Q4_K_M-modified.gguf \
33
  --port 8082 \
34
- --verbose \
35
  --jinja
36
  ```
37
 
@@ -71,6 +78,27 @@ curl http://localhost:8082/v1/chat/completions \
71
  }'
72
  ```
73
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
  ## API Endpoints
75
 
76
  The server provides a ChatGPT-compatible API with the following main endpoints:
@@ -82,8 +110,10 @@ The server provides a ChatGPT-compatible API with the following main endpoints:
82
  ## Notes
83
 
84
  - The server uses the same API format as OpenAI's ChatGPT API, making it compatible with many existing tools and libraries
85
- - The `--jinja` flag enables proper chat template formatting for the model
86
  - The model name in the requests can be set to "any-model" as shown in the examples
 
 
87
 
88
  ## Troubleshooting
89
 
@@ -96,4 +126,4 @@ If you encounter issues:
96
 
97
  ## License
98
 
99
- Please ensure you comply with the model's license terms when using it.
 
1
+ ---
2
+ base_model:
3
+ - microsoft/Phi-4-mini-instruct
4
+ ---
5
+ # Phi-4-mini-instruct with llama-server (Tool-Enhanced Version)
6
 
7
+ This repository contains instructions for running a modified version of the Phi-4-mini-instruct model using llama-server. This version has been enhanced to support tool usage, allowing the model to interact with external tools and APIs through a ChatGPT-compatible interface.
8
+
9
+ ## Model Capabilities
10
+
11
+ This modified version of Phi-4-mini-instruct includes:
12
+ - Full support for tool usage and function calling
13
+ - Custom chat template optimized for tool interactions
14
+ - Ability to process and respond to tool outputs
15
+ - ChatGPT-compatible API interface
16
 
17
  ## Prerequisites
18
 
 
39
  llama-server \
40
  --model models/Phi-4-mini-instruct-Q4_K_M-modified.gguf \
41
  --port 8082 \
 
42
  --jinja
43
  ```
44
 
 
78
  }'
79
  ```
80
 
81
+ ### Example 3: Using Tools
82
+
83
+ ```bash
84
+ curl http://localhost:8082/v1/chat/completions \
85
+ -H "Content-Type: application/json" \
86
+ -d '{
87
+ "model": "any-model",
88
+ "messages": [
89
+ {
90
+ "role": "system",
91
+ "content": "You are a helpful AI assistant that can use tools.",
92
+ "tools": "[{\"name\": \"calculator\", \"description\": \"Useful for performing mathematical calculations\", \"parameters\": {\"type\": \"object\", \"properties\": {\"expression\": {\"type\": \"string\", \"description\": \"The mathematical expression to evaluate\"}}}}]"
93
+ },
94
+ {
95
+ "role": "user",
96
+ "content": "What is 235 * 89?"
97
+ }
98
+ ]
99
+ }'
100
+ ```
101
+
102
  ## API Endpoints
103
 
104
  The server provides a ChatGPT-compatible API with the following main endpoints:
 
110
  ## Notes
111
 
112
  - The server uses the same API format as OpenAI's ChatGPT API, making it compatible with many existing tools and libraries
113
+ - The `--jinja` flag enables proper chat template formatting for the model, which is essential for tool usage
114
  - The model name in the requests can be set to "any-model" as shown in the examples
115
+ - This version supports system messages with tool definitions
116
+ - Tool responses are properly handled through the chat template
117
 
118
  ## Troubleshooting
119
 
 
126
 
127
  ## License
128
 
129
+ Please ensure you comply with the model's license terms when using it.