| # Standard vLLM Deployment | |
| This directory contains launch files for the local model services used by `glm-5.2-visual-runtime`. | |
| ## Direct Launch | |
| ```bash | |
| chmod +x vllm/serve_glm52_fp8.sh | |
| PORT=8001 TENSOR_PARALLEL_SIZE=8 ./vllm/serve_glm52_fp8.sh | |
| ``` | |
| ## Qwen Omni Vision / Audio / Video | |
| ```bash | |
| chmod +x vllm/serve_qwen3_omni.sh | |
| PORT=8002 TENSOR_PARALLEL_SIZE=2 ./vllm/serve_qwen3_omni.sh | |
| ``` | |
| ## Optional Qwen3.6-27B Reasoning | |
| Use this if you want Qwen3.6-27B as the reasoning engine instead of GLM-5.2: | |
| ```bash | |
| chmod +x vllm/serve_qwen36_27b.sh | |
| PORT=8001 TENSOR_PARALLEL_SIZE=2 ./vllm/serve_qwen36_27b.sh | |
| ``` | |
| ## Docker Compose | |
| ```bash | |
| docker compose -f vllm/docker-compose.vllm.yml up | |
| ``` | |
| ## Smoke Test | |
| ```bash | |
| pip install openai | |
| VLLM_BASE_URL=http://localhost:8001/v1 VLLM_MODEL=glm-5.2 python vllm/openai_smoke_test.py | |
| ``` | |
| ## Gateway Connection | |
| Configure the visual runtime gateway: | |
| ```bash | |
| GLM_BASE_URL=http://localhost:8001/v1 | |
| GLM_MODEL=glm-5.2 | |
| VISION_BASE_URL=http://localhost:8002/v1 | |
| VISION_MODEL=qwen3-omni | |
| VISUAL_RUNTIME_MODE=cloud | |
| ``` | |