# Standard vLLM Deployment

This directory contains launch files for the local model services used by `glm-5.2-visual-runtime`.

## Direct Launch

```bash
chmod +x vllm/serve_glm52_fp8.sh
PORT=8001 TENSOR_PARALLEL_SIZE=8 ./vllm/serve_glm52_fp8.sh
```

## Qwen Omni Vision / Audio / Video

```bash
chmod +x vllm/serve_qwen3_omni.sh
PORT=8002 TENSOR_PARALLEL_SIZE=2 ./vllm/serve_qwen3_omni.sh
```

## Optional Qwen3.6-27B Reasoning

Use this if you want Qwen3.6-27B as the reasoning engine instead of GLM-5.2:

```bash
chmod +x vllm/serve_qwen36_27b.sh
PORT=8001 TENSOR_PARALLEL_SIZE=2 ./vllm/serve_qwen36_27b.sh
```

## Docker Compose

```bash
docker compose -f vllm/docker-compose.vllm.yml up
```

## Smoke Test

```bash
pip install openai
VLLM_BASE_URL=http://localhost:8001/v1 VLLM_MODEL=glm-5.2 python vllm/openai_smoke_test.py
```

## Gateway Connection

Configure the visual runtime gateway:

```bash
GLM_BASE_URL=http://localhost:8001/v1
GLM_MODEL=glm-5.2
VISION_BASE_URL=http://localhost:8002/v1
VISION_MODEL=qwen3-omni
VISUAL_RUNTIME_MODE=cloud
```