---
license: mit
base_model: zai-org/GLM-4.7-Flash
tags:
- mlx
- speculative-decoding
- multi-token-prediction
- drafter
- glm4_moe_lite_mtp
---

# GLM-4.7-Flash-MTP-bf16

The trained multi-token-prediction (MTP / nextn) layer of
[zai-org/GLM-4.7-Flash](https://huggingface.co/zai-org/GLM-4.7-Flash), split
into a standalone MLX drafter checkpoint. This is **not a standalone language
model** — it is a single-layer draft head that predicts one token ahead from a
target model's hidden states, for speculative decoding against GLM-4.7-Flash.

GLM-4.7-Flash ships this layer inside the full checkpoint at
`model.layers.47.*`; MLX conversions of the base model strip it (`sanitize`
drops layers past `num_hidden_layers`), so quantized community conversions do
not carry it. This repo preserves it in bf16, revision-pinned.

## Provenance

- **Source:** `zai-org/GLM-4.7-Flash`, revision
  `7dd20894a642a0aa287e9827cb1a1f7f91386b67` (MIT). All weights are Z.ai's
  trained parameters, unmodified except for the layout transforms below.
- **Tool:** the `glm4_moe_lite_mtp` drafter split from mlx-vlm's
  `speculative/drafters` convention
  ([Blaizzy/mlx-vlm#1570](https://github.com/Blaizzy/mlx-vlm/pull/1570)):

  ```bash
  python -m mlx_vlm.speculative.drafters.glm4_moe_lite_mtp.split \
    --model zai-org/GLM-4.7-Flash \
    --revision 7dd20894a642a0aa287e9827cb1a1f7f91386b67 \
    --output GLM-4.7-Flash-MTP-bf16
  ```

  Only the 3 (of 48) source shards holding the nextn tensors are read.

- **Checksum:**

  | file | sha256 |
  |---|---|
  | `model.safetensors` | `bdb948cbec1810d89910cc99dfbc85864acde45241337702b555e2a721254e7b` |

## Format

`model_type: glm4_moe_lite_mtp`, `block_size: 2`
(`num_nextn_predict_layers + 1`), untied embeddings; the source text config is
nested under `text_config`. 24 tensors, flat post-sanitize layout:

- dedicated nextn `embed_tokens` and untied `lm_head` (GLM's nextn head is not
  tied to the target, unlike DeepSeek/Qwen MTP)
- `enorm` / `hnorm` / `eh_proj` projections
- one MLA attention block in absorbed form (`kv_b_proj` split into
  `embed_q` / `unembed_out`)
- 64-expert MoE stacked into `switch_mlp` + shared expert; the `noaux_tc`
  router correction bias is kept fp32 and the router gate full precision
  (casting them breaks routing)

## Consumers

- [vllm-project/vllm-metal#484](https://github.com/vllm-project/vllm-metal/pull/484) /
  [#485](https://github.com/vllm-project/vllm-metal/pull/485) — native MTP
  speculative decoding for GLM-4.7-Flash on Apple Silicon (reference
  integration this format feeds; design thread in
  [#482](https://github.com/vllm-project/vllm-metal/issues/482))
- mlx-vlm's speculative-decoding runtime, once a `glm4_moe_lite` backbone
  lands there

Measured offline acceptance of this head replaying real target hidden states:
~0.806 mean over chat prompts (methodology and end-to-end results in the
vllm-metal links above).

A 4-bit variant is at
[samithaj/GLM-4.7-Flash-MTP-4bit](https://huggingface.co/samithaj/GLM-4.7-Flash-MTP-4bit).