README.md

---
license: mit
library_name: sglang
pipeline_tag: image-text-to-text
tags:
  - multimodal
  - vision-language
  - glm
  - sglang
base_model:
  - zai-org/GLM-5.2
---

# GLM-5.2-Vision (NVFP4)

**GLM-5.2 with sight.** A vision-language model that bolts the MoonViT vision encoder from
[Kimi-K2.6](https://huggingface.co/moonshotai/Kimi-K2.6) onto
[GLM-5.2](https://huggingface.co/zai-org/GLM-5.2) through a trained PatchMerger projector.

GLM-5.2 is a strong open reasoning model with no vision input. This checkpoint adds it,
without touching a single GLM weight: the text backbone and the vision tower are both frozen
and byte-identical to their upstream releases. The only newly-trained parameters are the
**49.5M-parameter projector** that maps MoonViT's 1152-dim patch embeddings into GLM's 6144-dim
token space.

| Component | Detail |
|---|---|
| Text backbone | GLM-5.2 (744B total / A40B active, MoE + MLA + DSA sparse attention) — **frozen** |
| Vision tower | MoonViT-3d from Kimi-K2.6, 27 layers, 1152-dim — **frozen** |
| Projector | PatchMerger MLP (`pre_norm → linear_1 → GELU → linear_2`), 1152→4608→6144 — **trained** |
| Text weights | NVFP4, from [`nvidia/GLM-5.2-NVFP4`](https://huggingface.co/nvidia/GLM-5.2-NVFP4) |
| Size | ~466 GB |
| Hardware | 8×B200, or 4×B200 at 256k context — **Blackwell only** |
| Image tokens | up to 4096 per image (16384 MoonViT patches, 2×2 merge) |
| Max context | 1048576 (1M tokens) |

## Quickstart

SGLang needs a small out-of-tree plugin because `Glm5vForConditionalGeneration` is not yet an
upstream architecture. It ships inside this repo, so there is nothing else to clone:

```bash
uvx --from huggingface-hub hf download baseten/GLM-5.2-Vision-NVFP4 \
  --include 'plugins/*' --local-dir ./glm5v
uv pip install ./glm5v/plugins
```

### SGLang

```bash
export SGLANG_EXTERNAL_MODEL_PACKAGE=sglang_glm5v
export SGLANG_EXTERNAL_MM_PROCESSOR_PACKAGE=sglang_glm5v
export SGLANG_EXTERNAL_MM_MODEL_ARCH=Glm5vForConditionalGeneration
python -m sglang_glm5v.patch
```

#### 8×B200 — full 1M context

```bash
python -m sglang.launch_server \
  --model-path baseten/GLM-5.2-Vision-NVFP4 --trust-remote-code \
  --tp-size 8 \
  --quantization modelopt_fp4 \
  --disable-shared-experts-fusion --disable-flashinfer-autotune \
  --attention-backend dsa --mm-attention-backend sdpa \
  --kv-cache-dtype fp8_e4m3 --page-size 64 \
  --mem-fraction-static 0.85 \
  --context-length 1048576 \
  --reasoning-parser glm45 --tool-call-parser glm47 \
  --served-model-name glm-5.2-vision \
  --port 30000
```

#### 4×B200 — 256k context

Same command with `--tp-size 4`, a higher memory fraction, and a smaller context:

```bash
python -m sglang.launch_server \
  --model-path baseten/GLM-5.2-Vision-NVFP4 --trust-remote-code \
  --tp-size 4 \
  --quantization modelopt_fp4 \
  --disable-shared-experts-fusion --disable-flashinfer-autotune \
  --attention-backend dsa --mm-attention-backend sdpa \
  --kv-cache-dtype fp8_e4m3 --page-size 64 \
  --mem-fraction-static 0.90 \
  --context-length 262144 \
  --reasoning-parser glm45 --tool-call-parser glm47 \
  --served-model-name glm-5.2-vision \
  --port 30000
```

### Query it

Standard OpenAI multimodal messages deliver the image as `image_url`:

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="none")
r = client.chat.completions.create(
    model="glm-5.2-vision",
    messages=[{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "https://ultralytics.com/images/bus.jpg"}},
        {"type": "text", "text": "Describe this image in detail."},
    ]}],
    temperature=1.0, top_p=0.95, max_tokens=512,
)
print(r.choices[0].message.content)
```

GLM-5.2 is a reasoning model: with `--reasoning-parser glm45`, the chain of thought arrives in
`message.reasoning_content` and the answer in `message.content`.

## Deploy on Baseten

The repository includes ready-to-push [Truss](https://truss.baseten.co) configs. The only
credential you need is an API key for your own Baseten account; no Hugging Face token or
pre-created Baseten secret is required.

1. Install [`uv`](https://docs.astral.sh/uv/getting-started/installation/) and create a Baseten API key.
2. Export the key, download the small Truss directory, and deploy one of the two configurations:

```bash
export BASETEN_API_KEY="your-baseten-api-key"
uvx truss login --api-key "$BASETEN_API_KEY" --remote baseten --non-interactive

uvx --from huggingface-hub hf download baseten/GLM-5.2-Vision-NVFP4 \
  --include 'truss/*' --local-dir ./glm5v
cd glm5v/truss

# Recommended starting point: 4×B200 and 256k context.
uvx truss push --remote baseten --config config_nvfp4_4gpu.yaml --wait --output json

# Or use 8×B200 for the full 1M-token context.
# uvx truss push --remote baseten --config config_nvfp4.yaml --wait --output json
```

The command creates a new model and published deployment in your Baseten account and prints
JSON containing `model_id`, `model_version_id`, `predict_url`, and `logs_url`. It does not
promote the deployment to production.

Set `PREDICT_URL` to the returned `predict_url`, then query the model:

```bash
export PREDICT_URL="https://model-...api.baseten.co/deployment/.../predict"

curl -fsS "$PREDICT_URL" \
  -H "Authorization: Api-Key $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.2-vision",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "https://ultralytics.com/images/bus.jpg"}},
        {"type": "text", "text": "Describe this image in detail."}
      ]
    }],
    "max_tokens": 512,
    "temperature": 1.0,
    "top_p": 0.95
  }'
```

The first deployment downloads about 466 GB of weights and initializes SGLang, so startup can
take several minutes.

## License

MIT, following both parents: GLM-5.2 (MIT) and Kimi-K2.6 (Modified MIT). The projector weights
are released under MIT. Redistributed upstream weights remain under their original terms.

## Acknowledgements

Built on [Z.ai](https://huggingface.co/zai-org)'s GLM-5.2 and
[Moonshot AI](https://huggingface.co/moonshotai)'s Kimi-K2.6. Neither team was involved in this
work; please do not direct issues with this checkpoint to them.