GLM-5.3-Flash-MLX-8bit

MLX (Apple Silicon) build of GLM-5.3-Flash — 320B-A18B hybrid of 34 Kimi-Delta linear-attention layers and 11 DeepSeek-sparse-attention (NoPE MLA + lightning indexer) layers with manifold-constrained hyper-connections — quantized to 8-bit.

These files are modified: converted from the upstream bfloat16 release (GLM-5.3-Flash-BF16) to MLX and quantized; the architecture is unchanged. The multi-token-prediction layer (layer 45) is not included. The vision tower is carried in bfloat16.

Runtime

glm5_next landed in mlx-vlm main on 2026-08-26 (no release carries it yet). Validating that port against transformers 5.16 at tiny scale found two numerical bugs and two epsilon mismatches, which the runtime in https://github.com/PipeNetwork/glm53-flash-mlx fixes; parity is 1e-6 end to end, exact on cached decode.

what	reference	mlx-vlm main	effect
swiglu_limit	gate clamped at 10, up at ±10, in every text MLP	no clamp anywhere in the text stack	formula mismatch on all 45 FFN blocks
mHC base/scale dtype	float32	converter casts to bf16; the Metal kernel then reads base as float4	comb mixing matrix off by ~0.5 on every layer of a converted checkpoint
MLA low-rank norm eps	rms_norm_eps = 1e-5	1e-6	small
indexer LayerNorm eps	1e-6	1e-5	small

This checkpoint keeps the mHC arrays and KDA decay parameters in float32 as stored, so it is safe in either runtime; the clamp is a compute-path fix and needs the patched runtime:

git clone https://github.com/PipeNetwork/glm53-flash-mlx && cd glm53-flash-mlx && pip install -r requirements.txt
python scripts/smoke_generate.py /path/to/GLM-5.3-Flash-MLX-8bit

from glm53_flash_mlx.load import load
model, processor = load("/path/to/GLM-5.3-Flash-MLX-8bit")

Size and what is quantized

334.1 GB on disk (bfloat16 upstream: 642.7 GB).

group	share of parameters	this build
routed experts (switch_mlp, 42 layers × 288)	304B (97%)	8-bit, group 64
KDA and MLA projections, shared experts, dense MLPs, embeddings, lm_head	~9B	8-bit, group 64
lightning-indexer projections	0.06B	8-bit, group 64
MoE router + correction bias, mHC arrays (fp32), KDA A_log/dt_bias (fp32), convolutions, norms	—	as stored
vision tower	0.56B	bfloat16

Quality

Perplexity on wikitext-2 (test), 288,627 tokens in 141 windows of 2048, every build scored on identical windows through this runtime. The 643 GB bfloat16 model does not fit a 512 GB machine, so the 8-bit build is the anchor (on every model we have measured, 8-bit has been statistically indistinguishable from bfloat16). Per-window NLL differences against 8-bit, bootstrapped over one shared index set (20,000 resamples):

build	size	perplexity	ΔNLL/token vs 8-bit [95% CI]	windows worse
8bit	334.1 GB	3.4607	—	—
6bit	255.9 GB	3.4646	+0.0011 [−0.0017, +0.0038]	89/141
mixed-4_8bit	181.9 GB	3.5705	+0.0312 [+0.0271, +0.0355]	131/141
4bit	177.6 GB	3.7549	+0.0816 [+0.0755, +0.0879]	140/141

Read the interval, not the point estimate; "windows worse" counts how many of the 141 windows the build lost outright.

Against the 8-bit anchor: 6-bit +0.1%, mixed 4/8-bit +3.2%, uniform 4-bit +8.5%. Routed experts are 97% of the parameters; the mixed build keeps the other ~9B (KDA and MLA projections, shared experts, dense layers, embeddings) at 8-bit for 4.4 GB more than uniform 4-bit.

Greedy generation (a collapse detector, not a ranking) is coherent on every published build.

License

MIT, as the upstream model. Port code: https://github.com/PipeNetwork/glm53-flash-mlx.