---
license: mit
base_model: zai-org/GLM-4.7-Flash
tags:
- mlx
- speculative-decoding
- multi-token-prediction
- drafter
- glm4_moe_lite_mtp
---

# GLM-4.7-Flash-MTP-4bit

The trained multi-token-prediction (MTP / nextn) layer of
[zai-org/GLM-4.7-Flash](https://huggingface.co/zai-org/GLM-4.7-Flash), split
into a standalone MLX drafter checkpoint and quantized to 4-bit. This is **not
a standalone language model** — it is a single-layer draft head that predicts
one token ahead from a target model's hidden states, for speculative decoding
against GLM-4.7-Flash (pairs with
[mlx-community/GLM-4.7-Flash-4bit](https://huggingface.co/mlx-community/GLM-4.7-Flash-4bit)).

GLM-4.7-Flash ships this layer inside the full checkpoint at
`model.layers.47.*`; MLX conversions of the base model strip it (`sanitize`
drops layers past `num_hidden_layers`), so quantized community conversions do
not carry it. This repo preserves it, revision-pinned.

## Provenance

- **Source:** `zai-org/GLM-4.7-Flash`, revision
  `7dd20894a642a0aa287e9827cb1a1f7f91386b67` (MIT). All weights are Z.ai's
  trained parameters, unmodified except quantization and the layout transforms
  below.
- **Tool:** the `glm4_moe_lite_mtp` drafter split from mlx-vlm's
  `speculative/drafters` convention
  ([Blaizzy/mlx-vlm#1570](https://github.com/Blaizzy/mlx-vlm/pull/1570)):

  ```bash
  python -m mlx_vlm.speculative.drafters.glm4_moe_lite_mtp.split \
    --model zai-org/GLM-4.7-Flash \
    --revision 7dd20894a642a0aa287e9827cb1a1f7f91386b67 \
    --output GLM-4.7-Flash-MTP-4bit \
    --q-bits 4 --q-group-size 64
  ```

  Only the 3 (of 48) source shards holding the nextn tensors are read.

- **Checksum:**

  | file | sha256 |
  |---|---|
  | `model.safetensors` | `cc2a9750a6328a68b1758502a47f5d286cbc96c26859210fe7c751bbe6d328ba` |

## Format

`model_type: glm4_moe_lite_mtp`, `block_size: 2`
(`num_nextn_predict_layers + 1`), untied embeddings, affine quantization
(`bits: 4`, `group_size: 64`); the source text config is nested under
`text_config`. 54 tensors, flat post-sanitize layout:

- dedicated nextn `embed_tokens` and untied `lm_head` (GLM's nextn head is not
  tied to the target, unlike DeepSeek/Qwen MTP)
- `enorm` / `hnorm` / `eh_proj` projections
- one MLA attention block in absorbed form (`kv_b_proj` split into
  `embed_q` / `unembed_out`)
- 64-expert MoE stacked into `switch_mlp` + shared expert
- **not quantized:** norms, the router gate weight, and the `noaux_tc` router
  correction bias (kept fp32) — casting or quantizing them breaks routing

## Consumers

- [vllm-project/vllm-metal#484](https://github.com/vllm-project/vllm-metal/pull/484) /
  [#485](https://github.com/vllm-project/vllm-metal/pull/485) — native MTP
  speculative decoding for GLM-4.7-Flash on Apple Silicon (reference
  integration this format feeds; design thread in
  [#482](https://github.com/vllm-project/vllm-metal/issues/482))
- mlx-vlm's speculative-decoding runtime, once a `glm4_moe_lite` backbone
  lands there

Measured offline acceptance of this head replaying real target hidden states:
~0.806 mean over chat prompts; a 4-bit head forward measured ~6.7% of a target
decode step on M3 (methodology and end-to-end results in the vllm-metal links
above).

An unquantized variant is at
[samithaj/GLM-4.7-Flash-MTP-bf16](https://huggingface.co/samithaj/GLM-4.7-Flash-MTP-bf16).