One-Click All-Local Deployment

This profile runs the full runtime without hosted model providers.

It starts:

GLM-5.2 reasoning through local vLLM using zai-org/GLM-5.2-FP8.
Qwen Omni vision/audio/video understanding through local vLLM-Omni using Qwen/Qwen3-Omni-30B-A3B-Instruct.
Local OCR service built from this repo.
Gateway API under glm-5.2-visual-runtime.
PostgreSQL and MinIO for persistence.

Start

cp one_click/.env.all-local.example one_click/.env.all-local
docker compose -f one_click/docker-compose.all-local.yml --env-file one_click/.env.all-local up --build

By default, vLLM downloads checkpoints from Hugging Face at startup. For a physically self-contained repo, first populate and upload models/, then set local paths in one_click/.env.all-local:

python scripts/materialize_weights.py --profile glm52_plus_qwen_omni
hf upload-large-folder wassemgtk/glm-5.2-visual-runtime models --type model

GLM_MODEL_PATH=/models/zai-org/GLM-5.2-FP8
QWEN_OMNI_MODEL_PATH=/models/Qwen/Qwen3-Omni-30B-A3B-Instruct

Gateway:

http://localhost:8000/v1

Reasoning vLLM:

http://localhost:8001/v1

Vision vLLM:

http://localhost:8002/v1

OCR:

http://localhost:8081/ocr

Hardware

This is a real local deployment profile. GLM-5.2-FP8 and Qwen3-Omni are large checkpoints and require a serious multi-GPU environment. If you do not have enough GPU memory, the gateway can still run in test mode, but the all-local model services will not start.

Smoke Test

python vllm/openai_smoke_test.py

For the gateway:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.responses.create(
    model="glm-5.2-visual-runtime",
    input=[{"role": "user", "content": [{"type": "input_text", "text": "Say ready"}]}],
)
print(response.output_text)