README.md

---
license: mit
base_model: zai-org/GLM-5.2
tags:
- mlx
- vision
- glm
- moonvit
- exo
library_name: mlx
---

# GLM-5.2-Vision — MoonViT tower + trained projector (MLX)

Add **image input to any MLX quant of GLM-5.2** with a ~1 GB sidecar: the frozen
MoonViT-3d vision tower from Kimi K2.6 plus the trained 49.5M-parameter
PatchMerger projector from
[baseten/GLM-5.2-Vision-NVFP4](https://huggingface.co/baseten/GLM-5.2-Vision-NVFP4)
(Harry Partridge's vision retrofit), repackaged for Apple Silicon / MLX.
The GLM-5.2 text backbone is untouched — text-only behavior stays byte-identical.

## What's in this repo

| File | What it is |
|---|---|
| `glm52_vision.safetensors` (+ index) | MoonViT-3d tower (417M, 27 layers, 1152-dim, bf16, `vision_tower.*`) **and** the trained projector (`mm_projector.pre_norm/linear_1/linear_2`, 1152 → 2×2 merge → 4608 → 6144) in one file. Kimi's original 7168-dim projector is removed — the GLM-trained one replaces it. |
| `config.json` | `vision_config` (+ `text_config.hidden_size: 6144`, `media_placeholder_token_id: 154854`) |
| `preprocessor_config.json`, `kimi_k25_*.py`, `media_utils.py` | Baseten's reference image processor (NaViT resize, patch 14, 2×2 merge, ≤4096 tokens/image) — the exact preprocessing the projector was trained against |

GLM-5.2's stock tokenizer already contains the image tokens
(`<|begin_of_image|>` 154830, `<|image|>` 154854, `<|end_of_image|>` 154831) —
no tokenizer changes needed. Each image expands to its media-token count at the
`<|image|>` position and the projected features are substituted at those
embedding positions. You need a chat template that renders image content parts
into the marker triplet (GLM's stock template does not; Baseten ships one in
their repo).

## Using with [exo](https://github.com/exo-explore/exo)

exo's vision stack loads this repo directly as a `weights_repo`/`processor_repo`.
Model card stanza:

```toml
[vision]
image_token_id = 154854
model_type = "kimi_vl"
weights_repo = "<this repo id>"
processor_repo = "<this repo id>"
```

Point the card's model at a directory containing your GLM-5.2 MLX quant with
Baseten's `chat_template.jinja` and this repo's `config.json` additions
(`vision_config` / `text_config` / `media_placeholder_token_id`). Assembly
scripts (symlink the backbone — no weight duplication):
[`build_glm52_vision_dir.py`](https://github.com/aidiffuser/exo/blob/update-latest/scripts/build_glm52_vision_dir.py)
and
[`build_glm52_vision_tower.py`](https://github.com/aidiffuser/exo/blob/update-latest/scripts/build_glm52_vision_tower.py).

Verified live on a 2-Mac-Studio (M3 Ultra) tensor-parallel cluster over RDMA,
against both `mlx-community/GLM-5.2-DQ4plus-q8` and `mlx-community/GLM-5.2-mxfp4`:
temp-0 deterministic, no cross-image cache bleed, text-only outputs identical
to the plain model.

## What to expect (honest notes)

- Scene understanding, objects, colors, spatial layout and orientation: **good**.
  Reads real-world photos (e.g. product packaging labels) usefully.
- Dense/synthetic text OCR and fine-grained counting: **weak** — this is a 50M
  projector retrofit at the source's reported ~Haiku-4.5-level MMMU-Pro (55%),
  not a natively-trained VLM. A Kimi K2.6 A/B on the same pipeline is clearly
  stronger at fine detail.
- The projector was trained against the bf16/NVFP4 backbone; serving quantized
  backbones (mxfp4 / DQ4) costs some additional vision quality, never text
  quality.

## Provenance & license

MIT, following all parents. Full chain: **Z.ai** (GLM-5.2, MIT) →
**Moonshot AI** (Kimi K2.6 MoonViT tower, Modified MIT) → **Harry Partridge /
Baseten** (projector training + reference processor,
[baseten/GLM-5.2-Vision-NVFP4](https://huggingface.co/baseten/GLM-5.2-Vision-NVFP4),
MIT) → **exolabs** (original K2.6 tower extraction for MLX) → this repackaging
(tensor remap documented in the build script). None of the upstream teams were
involved in this packaging.