Vontra/GLM-5.3-Flash-MLX-4bit-MTP

GLM-5.3-Flash — MLX 4-bit with native MTP

A native Apple-silicon conversion of zai-org/GLM-5.3-Flash, quantised from the official checkpoint with its matching next-token prediction block preserved.

Original model · Z.ai overview · Technical report · Apple MLX · MIT licence

At a glance

Item	Value
Base model	zai-org/GLM-5.3-Flash
Repository	Vontra/GLM-5.3-Flash-MLX-4bit-MTP
Format	MLX safetensors
Quantisation	Uniform affine 4-bit for eligible modules
Group size	64
Source revision	3f1971b7b5f7a528c9c4ef6212c8785298a8c24a
Source weight format	FP8 E4M3 with 128×128 block scaling
Native MTP	Included — one matching upstream prediction layer
MTP weight precision	4-bit affine for eligible matrices; source-compatible precision for norms and routing tensors
Weight tensors	114160 total, including 2641 MTP tensors
Weight shards	43
Download size	181.694 GB / 169.215 GiB
Configured context	1,048,576 tokens
Architecture	glm5_next, multimodal sparse MoE

Modules that are not eligible for MLX quantisation remain at their source-compatible precision. The checkpoint preserves the upstream tokenizer, chat template, multimodal processor, generation configuration, and native MTP metadata.

Quantisation recipe

Component	Treatment
Eligible language projections	4-bit affine, group size 64
Routed experts	4-bit affine for eligible expert projections; routing tensors preserved at source-compatible precision
Token embeddings and output head	BF16
Vision encoder and projector	BF16, preserved from the official source
Native MTP prediction layer	4-bit affine for eligible matrices; source-compatible precision for norms and routing tensors
Other non-quantisable tensors	Preserved at source-compatible precision
Converter	Vontra streamed MLX affine converter using MLX 0.32.0

This is a weight-only post-training quantisation. It does not retrain or fine-tune the upstream model.

Runtime compatibility

GLM-5.3-Flash uses the new glm5_next multimodal architecture, hybrid linear and sparse attention, manifold-constrained hyper-connections, and an embedded next-token prediction layer. All four features need explicit runtime support.

Runtime path	Status
oMLX 0.6.3rc3	Validated for baseline generation; native MTP tensors recognised
MLX 0.32.0	Conversion and tensor runtime
MLX-VLM 0.6.3 with GLM5 Next runtime integration	Required for glm5_next model and vision loading
Older or stock builds without GLM5 Next MTP integration	Unsupported for native MTP

A runtime that can load glm5_next is not automatically able to use its embedded MTP layer. Native MTP requires the GLM5 Next-specific draft, verification, and cache-reconciliation path. If that integration is absent, keep MTP disabled or use a compatible build.

Download and use

hf download Vontra/GLM-5.3-Flash-MLX-4bit-MTP \
  --local-dir GLM-5.3-Flash-MLX-4bit-MTP

In a compatible oMLX build:

1. Add the downloaded model directory.
2. Keep Native MTP disabled for the currently validated serving path.
3. Start a new conversation after changing model settings so the runtime rebuilds the model and caches.

For baseline generation without native MTP, a compatible MLX-VLM installation can use:

python -m mlx_vlm.generate \
  --model Vontra/GLM-5.3-Flash-MLX-4bit-MTP \
  --prompt "Explain how hybrid attention reduces long-context serving cost." \
  --max-tokens 512

Apple M3 Studio performance

Measured on an Apple M3 Studio with a warmed model state:

Mode	Prompt tokens	Output tokens	Decode	Model memory
Baseline, MTP off	63	512	6.2657 tokens/s	173.39 GB

Benchmark detail	Value
Chip	Apple M3 Ultra
Unified memory	256 GB
Runtime	oMLX 0.6.3rc3, MLX 0.32.0
Timed runs	three; median reported
Warm-up	one 64-token greedy request discarded before measurement
Sampling	greedy decoding, temperature 0, top-p 1, top-k 0, fixed seed 6330

These figures are a local reference, not a cross-platform guarantee. Prompt shape, image input, context growth, draft acceptance, sampling, memory pressure, thermal state, and runtime revisions can materially change performance.

Native MTP status

The checkpoint contains the official matching NextN layer. A short deterministic MTP smoke test produced the exact requested output and recorded an accepted draft. Sustained testing on the current oMLX build exposed repeated speculative state reinitialisation and lower throughput than baseline decode. Native MTP is therefore preserved for compatible future runtimes but is not recommended for sustained serving on the validated build. Keep MTP disabled unless your runtime has corrected that scheduler behaviour.

Architecture

GLM-5.3-Flash is Z.ai's first natively multimodal GLM-5 model. It combines KDA linear-attention layers with periodic sparse-attention layers, a sparse mixture-of-experts feed-forward stack, manifold-constrained hyper-connections, and a native next-token prediction layer.

Architecture detail	Upstream value
Parameters	320B total / 18B active
Language layers	45
Linear / sparse-attention layers	34 / 11
Dense / sparse-MoE feed-forward layers	3 / 42
Routed / active experts	288 / 8, plus 1 shared expert
Hidden size	4,096
Attention heads	64
Vocabulary size	154,880
Manifold-constrained hyper-connection width	4 streams
Native MTP layers	1
Vision encoder	24 layers, 1,024 hidden size, 448-pixel input
Configured maximum context	1,048,576 tokens

See the official model card, Z.ai overview, and GLM-5 technical report for upstream training, evaluations, intended uses, and safety guidance.

Validation

Check	Result
Config, processor, tokenizer, and chat-template files present	Passed — official tokenizer, updated chat template, processor, generation config, and MIT licence included
Safetensors index resolves every tensor to an existing shard	Passed — 114,160 indexed tensors resolve exactly
All shards open and tensor headers match the index	Passed — all 43 shards open; no missing, duplicate, or unexpected tensors
Native MTP configuration and 2641 MTP tensors present	Passed — the official layer 45 NextN block is preserved and strict MTP-enabled loading succeeds
Baseline text generation	Passed — three identical coherent 512-token runs and all instruction, factual, and arithmetic gates
Native-MTP text generation	Short deterministic smoke passed with an accepted draft; sustained MTP is slower than baseline on this runtime
Greedy baseline/MTP output equivalence	Not claimed; sustained MTP serving is not recommended on this runtime
Vision tensor structure	Passed structurally — all 347 official vision tensors are preserved; image generation was not benchmarked
Long-context generation	Not claimed — configured context metadata is preserved, but no long-context generation result is reported

Validation used oMLX 0.6.3rc3, MLX 0.32.0, and MLX-VLM 0.6.3 with GLM5 Next runtime integration.

Limitations

• Quantisation can reduce quality relative to the official checkpoint, especially on difficult reasoning, tool-use, multilingual, vision, and long-context workloads.
• Native MTP improves speed only when draft acceptance offsets drafting and verification cost. It may be neutral or slower for unpredictable output.
• The configured one-million-token context does not imply that every Apple-silicon system has enough memory for a full-context request.
• Image prefill has different memory and throughput characteristics from text-only generation and is not represented by the decode benchmark above.
• Runtime support for glm5_next and its native MTP path is evolving. Use the exact validated versions listed in this card.
• Validate numerical behaviour, output quality, safety, and tool use on representative workloads before production deployment.

This is a community quantisation and is not an official Z.ai release.

Licence and attribution

The upstream model is released under the MIT License. The required licence text is included in this repository.

Model design, training, upstream evaluations, and documentation belong to Z.ai and the GLM-5 contributors. The MLX conversion, Apple-silicon validation, native-MTP integration work, and packaging are provided by Vontra.

If you use this model in research, cite the upstream report:

@misc{glm5team2026glm5,
  title        = {GLM-5: from Vibe Coding to Agentic Engineering},
  author       = {GLM-5-Team and others},
  year         = {2026},
  eprint       = {2602.15763},
  archivePrefix= {arXiv},
  primaryClass = {cs.LG},
  url          = {https://arxiv.org/abs/2602.15763}
}