deployment_report.md

# GLM Visual Runtime Deployment Report

## Summary

`glm-5.2-visual-runtime` is a training-free multimodal model gateway. It exposes an OpenAI-compatible model ID while routing text reasoning to GLM-5.2 and visual work to persistent visual variables and lazy lenses.

This repo is ready as a one-click all-local deployment manifest. It does not duplicate the upstream checkpoint weight shards because the official checkpoints are pulled from Hugging Face during deployment:

- BF16: `zai-org/GLM-5.2`
- FP8: `zai-org/GLM-5.2-FP8`
- Vision / omni: `Qwen/Qwen3-Omni-30B-A3B-Instruct`
- Optional alternate reasoning: `Qwen/Qwen3.6-27B`

For one-click deployment, run `one_click/docker-compose.all-local.yml`. It starts the gateway, local GLM-5.2 vLLM, local Qwen Omni vLLM-Omni, local OCR, PostgreSQL, and MinIO.

## Deployment Topology

```text
OpenAI client
  -> glm-5.2-visual-runtime gateway
  -> local reasoning model
       vLLM serving zai-org/GLM-5.2-FP8
  -> local vision model
       vLLM-Omni serving Qwen/Qwen3-Omni-30B-A3B-Instruct
  -> local OCR container
       repo-built OCR service
  -> local persistence
       PostgreSQL + MinIO
```

## vLLM Requirements

- vLLM `>= 0.23.0` or `vllm/vllm-openai:glm52`.
- Multi-GPU server suitable for a 753B-class MoE FP8 model.
- Hugging Face access to `zai-org/GLM-5.2-FP8`.
- Hugging Face access to `Qwen/Qwen3-Omni-30B-A3B-Instruct`.
- Optional Hugging Face access to `Qwen/Qwen3.6-27B`.
- OpenAI-compatible vLLM port, default `8001` in the included examples.

## One-Click Command

```bash
cp one_click/.env.all-local.example one_click/.env.all-local
docker compose -f one_click/docker-compose.all-local.yml --env-file one_click/.env.all-local up --build
```

## Reasoning vLLM Command

```bash
vllm serve zai-org/GLM-5.2-FP8 \
  --served-model-name glm-5.2 \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 8 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 5 \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice
```

## Vision / Omni vLLM Command

```bash
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct \
  --omni \
  --served-model-name qwen3-omni \
  --tensor-parallel-size 2 \
  --limit-mm-per-prompt '{"image": 8, "video": 1, "audio": 1}'
```

## Optional Qwen3.6-27B Reasoning Command

```bash
vllm serve Qwen/Qwen3.6-27B \
  --served-model-name qwen3.6-27b \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder
```

## Gateway Environment For All-Local Mode

```bash
GLM_BASE_URL=http://glm52-vllm:8000/v1
GLM_MODEL=glm-5.2
VISION_BASE_URL=http://vision-vllm:8000/v1
VISION_MODEL=qwen3-omni
OCR_BASE_URL=http://ocr:8080
VISUAL_RUNTIME_MODE=local
```

## Single-Repo Weights Bundle

The deployment package includes `weights_manifest.json` and `scripts/materialize_weights.py`.

To physically place all required model weights in this same repo:

```bash
pip install huggingface_hub
python scripts/materialize_weights.py --profile glm52_plus_qwen_omni
hf upload-large-folder wassemgtk/glm-5.2-visual-runtime models --type model
```

Alternate all-Qwen profile:

```bash
python scripts/materialize_weights.py --profile qwen36_plus_qwen_omni
hf upload-large-folder wassemgtk/glm-5.2-visual-runtime models --type model
```

These downloads are large. Confirm Hugging Face quota, local disk, and upload time before materializing full checkpoints.

## Current Acceptance Status

- Training-free runtime: complete.
- HF model report: complete.
- HF Docker Space: running in deterministic test mode.
- OpenAI SDK compatibility: smoke-tested.
- Standard vLLM reasoning deployment files: included.
- Local Qwen Omni vLLM deployment files: included.
- Single-repo checkpoint materialization manifest/script: included.
- Local OCR container source: included.
- All-local one-click Compose profile: included.

## Important Constraint

Do not run:

```bash
vllm serve wassemgtk/glm-5.2-visual-runtime
```

This repo is a runtime deployment package. Run the included one-click compose profile; it pulls `zai-org/GLM-5.2-FP8` and `Qwen/Qwen3-Omni-30B-A3B-Instruct` as part of deployment, or materialize `models/` if you want the checkpoint files stored in the same repo.