Instructions to use josefprusa/GLM-5.2-Vision-MLX-adapter with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use josefprusa/GLM-5.2-Vision-MLX-adapter with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("josefprusa/GLM-5.2-Vision-MLX-adapter")
config = load_config("josefprusa/GLM-5.2-Vision-MLX-adapter")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use josefprusa/GLM-5.2-Vision-MLX-adapter with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "josefprusa/GLM-5.2-Vision-MLX-adapter"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "josefprusa/GLM-5.2-Vision-MLX-adapter"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use josefprusa/GLM-5.2-Vision-MLX-adapter with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "josefprusa/GLM-5.2-Vision-MLX-adapter"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "josefprusa/GLM-5.2-Vision-MLX-adapter" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Hermes Agent

How to use josefprusa/GLM-5.2-Vision-MLX-adapter with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "josefprusa/GLM-5.2-Vision-MLX-adapter"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default josefprusa/GLM-5.2-Vision-MLX-adapter

Run Hermes

hermes

GLM-5.2 Vision adapter for MLX (MoonViT tower + PatchMerger projector)

Experimental, AI-built — not production software. This adapter and its serving stack were built largely by AI agents under human direction and validated by measurement, not review. Enjoy it, hack on it, review it before you depend on it — and if you'd like to adopt and polish it properly, please do; the author would be delighted and simply doesn't have the time.

The vision half of baseten/GLM-5.2-Vision-NVFP4, converted to MLX-native safetensors so a locally served GLM-5.2 on Apple Silicon can see images — no Blackwell GPUs, no SGLang.

GLM-5.2-Vision bolts the frozen MoonViT-3d encoder from Kimi-K2.6 onto a frozen GLM-5.2 through a trained 49.5M-parameter PatchMerger projector (1152→4608→6144). This repository packages exactly that vision side:

file	contents
`vision_tower.safetensors`	MoonViT-3d, 27 layers, 1152-dim (834 MB, frozen upstream weights)
`mm_projector.safetensors`	trained PatchMerger MLP (99 MB)
processor / config files	image preprocessing + chat template glue

Because the GLM text backbone is untouched by the vision training, this adapter composes with any GLM-5.2 text quantization — the tower runs on the image, the projector maps patch embeddings into GLM's token space, and the embeddings are spliced into the prompt.

Usage (omlx)

Serving is implemented in the omlx fork: point the model's vision_adapter_dir at this repository's contents in model_settings.json, restart, and send standard OpenAI-style image_url (base64) chat requests to /v1/chat/completions. Verified against 3.5–4.5 bpw GLM-5.2 quants on an M3 Ultra: image understanding works over the plain /v1 API with prefix caching intact.

Known limitation: multi-image prompts can bleed attention between images — single-image prompts are the reliable path today.

Credits

Baseten — trained the projector and released GLM-5.2-Vision-NVFP4 (MIT), from which these weights are extracted unchanged.
Moonshot AI — MoonViT encoder (Kimi-K2.6).
Z.ai — GLM-5.2.

License: MIT, following the source release.

Downloads last month: 346

MLX

Hardware compatibility

Quantized

Model tree for josefprusa/GLM-5.2-Vision-MLX-adapter

Base model

zai-org/GLM-5.2

Quantized

baseten/GLM-5.2-Vision-NVFP4

Quantized

(7)

this model