GLM Visual Runtime Deployment Report

Summary

glm-5.2-visual-runtime is a training-free multimodal model gateway. It exposes an OpenAI-compatible model ID while routing text reasoning to GLM-5.2 and visual work to persistent visual variables and lazy lenses.

This repo is ready as a one-click all-local deployment manifest. It does not duplicate the upstream checkpoint weight shards because the official checkpoints are pulled from Hugging Face during deployment:

BF16: zai-org/GLM-5.2
FP8: zai-org/GLM-5.2-FP8
Vision / omni: Qwen/Qwen3-Omni-30B-A3B-Instruct
Optional alternate reasoning: Qwen/Qwen3.6-27B

For one-click deployment, run one_click/docker-compose.all-local.yml. It starts the gateway, local GLM-5.2 vLLM, local Qwen Omni vLLM-Omni, local OCR, PostgreSQL, and MinIO.

Deployment Topology

OpenAI client
  -> glm-5.2-visual-runtime gateway
  -> local reasoning model
       vLLM serving zai-org/GLM-5.2-FP8
  -> local vision model
       vLLM-Omni serving Qwen/Qwen3-Omni-30B-A3B-Instruct
  -> local OCR container
       repo-built OCR service
  -> local persistence
       PostgreSQL + MinIO

vLLM Requirements

vLLM >= 0.23.0 or vllm/vllm-openai:glm52.
Multi-GPU server suitable for a 753B-class MoE FP8 model.
Hugging Face access to zai-org/GLM-5.2-FP8.
Hugging Face access to Qwen/Qwen3-Omni-30B-A3B-Instruct.
Optional Hugging Face access to Qwen/Qwen3.6-27B.
OpenAI-compatible vLLM port, default 8001 in the included examples.

One-Click Command

cp one_click/.env.all-local.example one_click/.env.all-local
docker compose -f one_click/docker-compose.all-local.yml --env-file one_click/.env.all-local up --build

Reasoning vLLM Command

vllm serve zai-org/GLM-5.2-FP8 \
  --served-model-name glm-5.2 \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 8 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 5 \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice

Vision / Omni vLLM Command

vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct \
  --omni \
  --served-model-name qwen3-omni \
  --tensor-parallel-size 2 \
  --limit-mm-per-prompt '{"image": 8, "video": 1, "audio": 1}'

Optional Qwen3.6-27B Reasoning Command

vllm serve Qwen/Qwen3.6-27B \
  --served-model-name qwen3.6-27b \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

Gateway Environment For All-Local Mode

GLM_BASE_URL=http://glm52-vllm:8000/v1
GLM_MODEL=glm-5.2
VISION_BASE_URL=http://vision-vllm:8000/v1
VISION_MODEL=qwen3-omni
OCR_BASE_URL=http://ocr:8080
VISUAL_RUNTIME_MODE=local

Single-Repo Weights Bundle

The deployment package includes weights_manifest.json and scripts/materialize_weights.py.

To physically place all required model weights in this same repo:

pip install huggingface_hub
python scripts/materialize_weights.py --profile glm52_plus_qwen_omni
hf upload-large-folder wassemgtk/glm-5.2-visual-runtime models --type model

Alternate all-Qwen profile:

python scripts/materialize_weights.py --profile qwen36_plus_qwen_omni
hf upload-large-folder wassemgtk/glm-5.2-visual-runtime models --type model

These downloads are large. Confirm Hugging Face quota, local disk, and upload time before materializing full checkpoints.

Current Acceptance Status

Training-free runtime: complete.
HF model report: complete.
HF Docker Space: running in deterministic test mode.
OpenAI SDK compatibility: smoke-tested.
Standard vLLM reasoning deployment files: included.
Local Qwen Omni vLLM deployment files: included.
Single-repo checkpoint materialization manifest/script: included.
Local OCR container source: included.
All-local one-click Compose profile: included.

Important Constraint

Do not run:

vllm serve wassemgtk/glm-5.2-visual-runtime

This repo is a runtime deployment package. Run the included one-click compose profile; it pulls zai-org/GLM-5.2-FP8 and Qwen/Qwen3-Omni-30B-A3B-Instruct as part of deployment, or materialize models/ if you want the checkpoint files stored in the same repo.