ariel-pillar commited on
Commit
44d566d
·
verified ·
1 Parent(s): 897eb59

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -25
README.md CHANGED
@@ -38,7 +38,7 @@ Start the llama-server with the following command:
38
  ```bash
39
  llama-server \
40
  --model models/Phi-4-mini-instruct-Q4_K_M-tool_use.gguf \
41
- --port 8082 \
42
  --jinja
43
  ```
44
 
@@ -52,53 +52,69 @@ This will start the server with:
52
 
53
  You can test the server using curl commands. Here are some examples:
54
 
55
- ### Example 1: Generate HTML Hello World
56
 
57
  ```bash
58
- curl http://localhost:8082/v1/chat/completions \
59
- -H "Content-Type: application/json" \
60
- -d '{
61
- "model": "any-model",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
62
  "messages": [
63
- {"role":"user","content":"give me an html hello world document"}
 
 
 
64
  ]
65
- }'
66
  ```
67
 
68
- ### Example 2: Tell a Joke
69
 
70
  ```bash
71
- curl http://localhost:8082/v1/chat/completions \
72
  -H "Content-Type: application/json" \
73
  -d '{
74
- "model": "any-model",
75
  "messages": [
76
- {"role":"user","content":"tell me a funny joke"}
 
77
  ]
78
  }'
79
  ```
80
 
81
- ### Example 3: Using Tools
82
 
83
  ```bash
84
- curl http://localhost:8082/v1/chat/completions \
85
  -H "Content-Type: application/json" \
86
  -d '{
87
- "model": "any-model",
88
  "messages": [
89
- {
90
- "role": "system",
91
- "content": "You are a helpful AI assistant that can use tools.",
92
- "tools": "[{\"name\": \"calculator\", \"description\": \"Useful for performing mathematical calculations\", \"parameters\": {\"type\": \"object\", \"properties\": {\"expression\": {\"type\": \"string\", \"description\": \"The mathematical expression to evaluate\"}}}}]"
93
- },
94
- {
95
- "role": "user",
96
- "content": "What is 235 * 89?"
97
- }
98
  ]
99
  }'
100
  ```
101
 
 
 
102
  ## API Endpoints
103
 
104
  The server provides a ChatGPT-compatible API with the following main endpoints:
@@ -122,7 +138,6 @@ If you encounter issues:
122
  1. Ensure the model file exists in the specified path
123
  2. Check that port 8082 is not in use by another application
124
  3. Verify that llama-cpp-python is installed with server support
125
- 4. Check the server logs with `--verbose` flag for detailed information
126
 
127
  ## License
128
 
 
38
  ```bash
39
  llama-server \
40
  --model models/Phi-4-mini-instruct-Q4_K_M-tool_use.gguf \
41
+ --port 8080 \
42
  --jinja
43
  ```
44
 
 
52
 
53
  You can test the server using curl commands. Here are some examples:
54
 
55
+ ### Example 1: Using Tools
56
 
57
  ```bash
58
+ curl http://localhost:8080/v1/chat/completions -d '{
59
+ "model": "phi-4-mini-instruct-with-tools",
60
+ "tools": [
61
+ {
62
+ "type":"function",
63
+ "function":{
64
+ "name":"python",
65
+ "description":"Runs code in an ipython interpreter and returns the result of the execution after 60 seconds.",
66
+ "parameters":{
67
+ "type":"object",
68
+ "properties":{
69
+ "code":{
70
+ "type":"string",
71
+ "description":"The code to run in the ipython interpreter."
72
+ }
73
+ },
74
+ "required":["code"]
75
+ }
76
+ }
77
+ }
78
+ ],
79
  "messages": [
80
+ {
81
+ "role": "user",
82
+ "content": "Print a hello world message with python."
83
+ }
84
  ]
85
+ }'
86
  ```
87
 
88
+ ### Example 2: Generate HTML Hello World
89
 
90
  ```bash
91
+ curl http://localhost:8080/v1/chat/completions \
92
  -H "Content-Type: application/json" \
93
  -d '{
94
+ "model": "phi-4-mini-instruct-with-tools",
95
  "messages": [
96
+ {"role":"system","content":"You are a helpful coding assistant"},
97
+ {"role":"user","content":"give me an html hello world document"}
98
  ]
99
  }'
100
  ```
101
 
102
+ ### Example 2: Tell a Joke
103
 
104
  ```bash
105
+ curl http://localhost:8080/v1/chat/completions \
106
  -H "Content-Type: application/json" \
107
  -d '{
108
+ "model": "phi-4-mini-instruct-with-tools",
109
  "messages": [
110
+ {"role":"system","content":"You are a helpful clown instruction assistant"},
111
+ {"role":"user","content":"tell me a funny joke"}
 
 
 
 
 
 
 
112
  ]
113
  }'
114
  ```
115
 
116
+
117
+
118
  ## API Endpoints
119
 
120
  The server provides a ChatGPT-compatible API with the following main endpoints:
 
138
  1. Ensure the model file exists in the specified path
139
  2. Check that port 8082 is not in use by another application
140
  3. Verify that llama-cpp-python is installed with server support
 
141
 
142
  ## License
143