Qwen

/

Qwen3.5-9B

like 385

Follow

Qwen 72.7k

The sglang package has been installed using the following command:

uv pip install 'git+https://github.com/sgl-project/sglang.git#subdirectory=python&egg=sglang[all]'

Next, the server is started with the following command:

sglang serve \
    --model-path /local/path/to/Qwen3.5-9B \
    --served-model-name Qwen3.5-9B \
    --port 8964 \
    --tp-size 1 \
    --dp-size 8 \
    --mem-fraction-static 0.8 \
    --context-length 262144 \
    --reasoning-parser qwen3 \
    --tool-call-parser qwen3_coder \
    --api-key hahaha

Then, the following Python script is used for testing:

#!/usr/bin/env python
# encoding: utf-8
from openai import OpenAI
from pprint import pprint

def test_tool_call():
    client = OpenAI(base_url="http://127.0.0.1:8964/v1", api_key="hahaha")

    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get weather of a city",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "city": {
                            "type": "string",
                            "description": "city name",
                        },
                    },
                    "required": ["city"],  # Fixed: originally written as sign
                },
            },
        }
    ]

    messages = [
        {"role": "system", "content": "You are a helpful assistant. You can use the function get_weather to get the weather of a city."},
        {"role": "user", "content": "What is the weather like in Beijing?"},
    ]

    messages = [{'role': 'user', 'content': 'What is the weather like in Beijing?'}]
    tools = []

    response = client.chat.completions.create(
        model="Qwen3.5-4B",
        messages=messages,
        tools=tools,
        tool_choice="auto",
    )

    pprint(response.choices[0].message.dict())

if __name__ == "__main__":
    test_tool_call()

However, it does not return a tool call result.