Standard vLLM Deployment
This directory contains launch files for the local model services used by glm-5.2-visual-runtime.
Direct Launch
chmod +x vllm/serve_glm52_fp8.sh
PORT=8001 TENSOR_PARALLEL_SIZE=8 ./vllm/serve_glm52_fp8.sh
Qwen Omni Vision / Audio / Video
chmod +x vllm/serve_qwen3_omni.sh
PORT=8002 TENSOR_PARALLEL_SIZE=2 ./vllm/serve_qwen3_omni.sh
Optional Qwen3.6-27B Reasoning
Use this if you want Qwen3.6-27B as the reasoning engine instead of GLM-5.2:
chmod +x vllm/serve_qwen36_27b.sh
PORT=8001 TENSOR_PARALLEL_SIZE=2 ./vllm/serve_qwen36_27b.sh
Docker Compose
docker compose -f vllm/docker-compose.vllm.yml up
Smoke Test
pip install openai
VLLM_BASE_URL=http://localhost:8001/v1 VLLM_MODEL=glm-5.2 python vllm/openai_smoke_test.py
Gateway Connection
Configure the visual runtime gateway:
GLM_BASE_URL=http://localhost:8001/v1
GLM_MODEL=glm-5.2
VISION_BASE_URL=http://localhost:8002/v1
VISION_MODEL=qwen3-omni
VISUAL_RUNTIME_MODE=cloud