feat: add tool_call example
#5
by airlsyn - opened
README.md
CHANGED
|
@@ -249,6 +249,57 @@ curl -s http://localhost:8000/v1/chat/completions \
|
|
| 249 |
}'
|
| 250 |
```
|
| 251 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 252 |
#### Handling Escaped Newlines in Model Outputs <!-- omit in toc -->
|
| 253 |
|
| 254 |
In some cases, the model might output escaped newline characters `\n` as string literals instead of actual newlines. To render the text correctly, especially in UI layers, you can use the following utility function. This function carefully replaces literal `\n` with real newlines while protecting scenarios where `\n` has specific semantic meaning.
|
|
|
|
| 249 |
}'
|
| 250 |
```
|
| 251 |
|
| 252 |
+
Tool calling example:
|
| 253 |
+
|
| 254 |
+
```bash
|
| 255 |
+
curl -s http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
|
| 256 |
+
"model": "openbmb/MiniCPM-V-4.6",
|
| 257 |
+
"messages": [{"role": "user", "content": [
|
| 258 |
+
{"type": "text", "text": "the weather of Beijing"}
|
| 259 |
+
]}],
|
| 260 |
+
"tools": [{
|
| 261 |
+
"type": "function",
|
| 262 |
+
"function": {
|
| 263 |
+
"name": "get_weather",
|
| 264 |
+
"description": "Get the current weather for a given location",
|
| 265 |
+
"parameters": {
|
| 266 |
+
"type": "object",
|
| 267 |
+
"properties": {
|
| 268 |
+
"location": {"type": "string", "description": "City name"}
|
| 269 |
+
},
|
| 270 |
+
"required": ["location"]
|
| 271 |
+
}
|
| 272 |
+
}
|
| 273 |
+
}]
|
| 274 |
+
}'
|
| 275 |
+
```
|
| 276 |
+
|
| 277 |
+
The model returns a natural-language explanation followed by a structured <tool_call> block embedded in the content field. Note that a dedicated tool call parser for this format has not yet been added to the transformers library, so the tool calls need to be extracted manually via regex for now.
|
| 278 |
+
|
| 279 |
+
```
|
| 280 |
+
{
|
| 281 |
+
"id": "f4f09c7d-8045-4cb1-ade9-07aa5dee637d",
|
| 282 |
+
"choices": [
|
| 283 |
+
{
|
| 284 |
+
"finish_reason": "stop",
|
| 285 |
+
"index": 0,
|
| 286 |
+
"message": {
|
| 287 |
+
"content": "I need to check the current weather for Beijing, so I will call the get_weather function.\n\n<tool_call>\n<function=get_weather>\n<parameter=location>\nBeijing\n</parameter>\n</function>\n</tool_call>",
|
| 288 |
+
"role": "assistant"
|
| 289 |
+
}
|
| 290 |
+
}
|
| 291 |
+
],
|
| 292 |
+
"created": 1778748859,
|
| 293 |
+
"model": "openbmb/MiniCPM-V-4.6@main",
|
| 294 |
+
"object": "chat.completion",
|
| 295 |
+
"usage": {
|
| 296 |
+
"completion_tokens": 47,
|
| 297 |
+
"prompt_tokens": 283,
|
| 298 |
+
"total_tokens": 330
|
| 299 |
+
}
|
| 300 |
+
}
|
| 301 |
+
```
|
| 302 |
+
|
| 303 |
#### Handling Escaped Newlines in Model Outputs <!-- omit in toc -->
|
| 304 |
|
| 305 |
In some cases, the model might output escaped newline characters `\n` as string literals instead of actual newlines. To render the text correctly, especially in UI layers, you can use the following utility function. This function carefully replaces literal `\n` with real newlines while protecting scenarios where `\n` has specific semantic meaning.
|