Vontra/GLM-5.3-Flash-MLX-oQ4-MTP

GLM-5.3-Flash, MLX oQ4 with native MTP

A sensitivity-guided mixed-precision MLX conversion of zai-org/GLM-5.3-Flash, built from the official checkpoint with its matching next-token prediction block preserved.

Original model · Z.ai overview · Technical report · Apple MLX · MIT licence

At a glance

Item	Value
Base model	zai-org/GLM-5.3-Flash
Repository	Vontra/GLM-5.3-Flash-MLX-oQ4-MTP
Format	MLX safetensors
Quantisation	oQ4 mixed precision: 4-bit affine base with 554 sensitivity-selected 5/6/8-bit overrides
Base group size	64
Source revision	3f1971b7b5f7a528c9c4ef6212c8785298a8c24a
Source weight format	FP8 E4M3 with 128x128 block scaling
Sensitivity method	relative activation MSE measured with a validated 4-bit proxy
Protected or overridden modules	554
Native MTP	Included, one matching upstream prediction layer
MTP weight precision	4-bit affine base with 12 native-MTP overrides at 5/6/8-bit
Weight tensors	3125 total, including 59 MTP tensors
Weight shards	35
Download size	185.749 GB / 172.992 GiB
Configured context	1,048,576 tokens
Architecture	glm5_next, multimodal sparse MoE

The upstream tokenizer, chat template, multimodal processor, generation configuration, and native MTP metadata are preserved.

oQ recipe

Component	Treatment
Eligible base modules	4-bit affine, group size 64
Sensitivity-protected modules	5-bit, 6-bit, and 8-bit affine, 554 modules
Routed experts	4-bit affine base with sensitivity-selected higher-precision expert projections
Token embeddings and output head	source-compatible precision
Vision encoder and projector	source-compatible precision
Native MTP prediction layer	4-bit affine base with 12 native-MTP overrides at 5/6/8-bit
Other non-quantisable tensors	Preserved at source-compatible precision
Converter	Vontra streamed oQ converter using MLX 0.32.0

Sensitivity was measured with the built-in multilingual code calibration set, 128 samples at 256 tokens. The allocation rule was byte-budgeted layer-sensitivity ranking under the oQ4 target and hard cap. These details are part of the release recipe and should be used when comparing oQ variants.

This is a weight-only post-training quantisation. It does not retrain or fine-tune the upstream model.

Runtime compatibility

GLM-5.3-Flash uses the new glm5_next multimodal architecture, hybrid linear and sparse attention, manifold-constrained hyper-connections, and an embedded next-token prediction layer. The oQ layout also requires a runtime that understands its per-module quantisation metadata.

Runtime path	Status
oMLX 0.6.3rc3 build 2475	Validated for oQ baseline and native-MTP generation
MLX 0.32.0	Mixed-precision tensor runtime
MLX-VLM 0.6.3 with GLM5 Next runtime integration	Required for glm5_next model and vision loading
Older builds without oQ or GLM5 Next MTP integration	Unsupported

A runtime that can load an ordinary MLX conversion is not necessarily compatible with this oQ checkpoint or its embedded MTP layer. Use a build that supports the recorded per-module recipe, GLM5 Next drafting, verification, and cache reconciliation.

Download and use

hf download Vontra/GLM-5.3-Flash-MLX-oQ4-MTP \
  --local-dir GLM-5.3-Flash-MLX-oQ4-MTP

In a compatible oMLX build:

1. Add the downloaded model directory.
2. Load it once with native MTP disabled to validate the oQ baseline path.
3. Enable Native MTP in the model settings and select the validated draft depth shown below.
4. Start a new conversation so the runtime rebuilds the model and caches with MTP active.

Apple M3 Studio performance

Measured on the same Apple M3 Studio, with identical prompts, sampling, context, output length, and warmed model state:

Mode	Output tokens	Median decode
oQ baseline, MTP off	128	6.2155 tokens/s
oQ with native MTP	128	4.9969 tokens/s

Benchmark detail	Value
Chip	Apple M3 Ultra
Unified memory	256 GB
Runtime	oMLX 0.6.3rc3 build 2475, MLX 0.32.0
Draft depth	1
Draft acceptance	100.0% on captured one-token drafts
Decode ratio, MTP/baseline	0.8039x
Timed runs	three per mode; median reported
Warm-up	one 32-token greedy request discarded before each measured mode
Sampling	greedy decoding with temperature 0

These figures are a local reference, not a cross-platform guarantee. Prompt shape, image input, context growth, draft acceptance, sampling, memory pressure, thermal state, and runtime revisions can materially change performance.

Architecture

GLM-5.3-Flash is Z.ai's first natively multimodal GLM-5 model. It combines KDA linear-attention layers with periodic sparse-attention layers, a sparse mixture-of-experts feed-forward stack, manifold-constrained hyper-connections, and a native next-token prediction layer.

Architecture detail	Upstream value
Parameters	320B total / 18B active
Language layers	45
Linear / sparse-attention layers	34 / 11
Dense / sparse-MoE feed-forward layers	3 / 42
Routed / active experts	288 / 8, plus 1 shared expert
Hidden size	4,096
Attention heads	64
Vocabulary size	154,880
Manifold-constrained hyper-connection width	4 streams
Native MTP layers	1
Vision encoder	24 layers, 1,024 hidden size, 448-pixel input
Configured maximum context	1,048,576 tokens

See the official model card, Z.ai overview, and GLM-5 technical report for upstream training, evaluations, intended uses, and safety guidance.

Validation

Check	Result
Config, processor, tokenizer, and chat-template files present	Passed; current upstream tokenizer, chat template, processor, generation config, and MIT licence are present
Safetensors index resolves every tensor to an existing shard	Passed; all 3,125 indexed tensors resolve to existing shards
All shards open and tensor headers match the index	Passed; all 35 shards open with no missing, duplicate, unexpected, or zero-length tensors
Per-module oQ metadata matches the saved tensor layouts	Passed; the 4-bit base and all 554 per-module overrides match the saved layouts
Native MTP configuration and 59 MTP tensors present	Passed; one configured NextN layer and 59 native-MTP tensors are present
Baseline text generation	Passed; exact instruction, factual, arithmetic, and sustained streaming tests completed
Native-MTP text generation	Passed for the same correctness gates; sustained decode is slower than baseline on this runtime
Greedy baseline/MTP output equivalence	Passed on instruction, factual, and arithmetic gates; sustained hash equivalence is not claimed
Image-and-text smoke test	Passed structurally; image generation was not benchmarked
Long-context smoke test	Not claimed; upstream context metadata is preserved

Validation used oMLX 0.6.3rc3 build 2475, MLX 0.32.0, and MLX-VLM 0.6.3 with GLM5 Next runtime integration.

Limitations

• Mixed-precision quantisation can reduce quality relative to the official checkpoint. The effect can differ by task and by oQ allocation.
• Sensitivity scores are calibration-dependent; protected modules are not a guarantee of BF16-equivalent quality.
• Native MTP improves speed only when draft acceptance offsets drafting and verification cost. It may be neutral or slower for unpredictable output.
• The configured one-million-token context does not imply that every Apple-silicon system has enough memory for a full-context request.
• Image prefill has different memory and throughput characteristics from text-only generation and is not represented by the decode benchmark above.
• Runtime support for glm5_next, oQ metadata, and native MTP is evolving. Use the exact validated versions listed in this card.
• Validate numerical behaviour, output quality, safety, and tool use on representative workloads before production deployment.

This is a community quantisation and is not an official Z.ai release.

Licence and attribution

The upstream model is released under the MIT License. The required licence text is included in this repository.

Model design, training, upstream evaluations, and documentation belong to Z.ai and the GLM-5 contributors. The oQ conversion, Apple-silicon validation, native-MTP integration work, and packaging are provided by Vontra.

If you use this model in research, cite the upstream report:

@misc{glm5team2026glm5,
  title        = {GLM-5: from Vibe Coding to Agentic Engineering},
  author       = {GLM-5-Team and others},
  year         = {2026},
  eprint       = {2602.15763},
  archivePrefix= {arXiv},
  primaryClass = {cs.LG},
  url          = {https://arxiv.org/abs/2602.15763}
}