infosave/GLM-5.3-Flash-cmf

GLM-5.3-Flash Q4TP — universal CPU/GPU CMF

This repository contains the text tower of zai-org/GLM-5.3-Flash, converted directly from the official FP8 checkpoint to one memory-mapped CMF file. The same artifact is mmap-backed for CPU and Cortiq's portable GPU backends; inference does not require Python, PyTorch, a CUDA toolkit, or full-model GPU residency. This release was validated on CPU and Vulkan; Metal validation is still pending.

The upstream model is a 320B/18B-active native multimodal MoE. This release is text-generation only: it intentionally omits the vision tower and the optional speculative MTP block. cortiq info reports 313.33B retained text parameters, 45 trunk layers (34 linear KDA and 11 full DSA), a 154,880-token vocabulary, and 37,534 payload tensors.

hf download infosave/GLM-5.3-Flash-cmf glm-5.3-flash-q4tp.cmf --local-dir .
cortiq verify glm-5.3-flash-q4tp.cmf
cortiq run glm-5.3-flash-q4tp.cmf \
  --prompt "What is 2+2? Answer briefly." --no-think

The file requires a GLM-5.3-compatible Cortiq build containing the dedicated glm5_next runtime. Older binaries reject this architecture instead of silently treating it as a generic Transformer. A packaged version will be listed here after the remaining GPU release gate.

File and integrity

file	profile	exact size	status
glm-5.3-flash-q4tp.cmf	mixed q4tp + q8_2f + f16	167,164,530,274 bytes (155.68 GiB)	37,534/37,534 tensor hashes pass

SHA-256:

6bd350591e1efd9dd098a189abf603467eed8387ee38d28f805e0891cedf36c6  glm-5.3-flash-q4tp.cmf

This is a quality-oriented mixed profile, not a uniform four-bit dump. The large routed/shared expert payloads use q4tp. Always-active KDA/DSA attention projections and both vocabulary edges use q8_2f to protect recurrent error and input-channel outliers. Small norms, routers, convolution taps, mHC controls, and other sensitive nonlinear control tensors remain f16.

The source checkpoint's fine-grained E4M3 weights are decoded with their F32 weight_scale_inv planes using the official 128×128 tile orientation, then quantized independently. The CMF embeds the tokenizer, exact chat template, tensor directory, architecture descriptor, and per-tensor hashes.

Exact GLM-5.3 runtime

Cortiq executes the release architecture explicitly rather than approximating it with a generic decoder:

• four-stream Manifold-Constrained Hyper-Connections and final stream mixing;
• 34 KDA linear-attention layers and 11 DSA/MLA full-attention layers;
• DSA indexer pooling, selected-token cache policy, and ordinary-context exact full-attention fallback;
• three dense SwiGLU layers followed by 288-expert routed MoE layers, top-8 sigmoid routing, correction bias, scaling, and the unconditional shared expert;
• untied input embedding and lm_head, partial RoPE, and the native one-million-token architecture limit;
• the upstream Jinja chat template, including reasoning_effort and the direct-answer <think></think> path used by --no-think.

The release oracle compares real source FP8 tensors with the CMF runtime. For the first routed layer, Q4TP output has source correlation 0.997998 and relative RMS error 6.36%. The deterministic CPU gate renders the official 24-token direct-answer prompt and emits token 19 (4), then the model stop marker. cortiq verify independently checks every payload hash.

Models larger than VRAM

The 155.68 GiB file does not need to fit in VRAM. CMF keeps weights mmap-backed in host memory and detects the available adapter budget. When dynamic pooling is enabled, routed experts use a bounded global GPU pool: resident experts run on the GPU, while cache misses are completed exactly on CPU and accumulated into the same MoE result. Q4TP automatic mode and systems without a supported adapter use the same file on CPU.

Useful controls:

cortiq gpu
CMF_GPU=0 cortiq run glm-5.3-flash-q4tp.cmf --prompt "CPU check" --no-think
CMF_GPU_VRAM_MB=16000 CMF_GLM_DYNAMIC_MOE=1 \
  cortiq run glm-5.3-flash-q4tp.cmf \
  --prompt "bounded forced GPU pool" --no-think
CMF_GPU_ADAPTER=0 cortiq run glm-5.3-flash-q4tp.cmf \
  --prompt "select adapter 0" --no-think
CMF_GLM_DYNAMIC_MOE=0 cortiq run glm-5.3-flash-q4tp.cmf \
  --prompt "force exact CPU MoE" --no-think

For this Q4TP artifact, automatic mode deliberately keeps the MoE on exact CPU. On the measured host, forcing Q4TP expert pooling was substantially slower than CPU execution. Automatic dynamic pooling is therefore reserved for Q2TP gate/up tensors on supported Vulkan devices with at least a 14 GB detected budget. Explicit CMF_GLM_DYNAMIC_MOE=1 remains available for bounded-memory compatibility testing; CMF_GLM_DYNAMIC_MOE=0 always disables it.

Measured on an RTX 3090 with dual AMD EPYC 7H12 CPUs and 32 CPU worker threads:

execution path	steady decode	notes
Q4TP CPU, final run A	1.912 tok/s	exact path, zero GPU submissions
Q4TP CPU, final run B	2.350 tok/s	exact path, zero GPU submissions
Q4TP CPU, final run C	2.207 tok/s	exact path, zero GPU submissions
Q4TP CPU, three-run mean	2.156 tok/s	1.26% below the earlier 2.184 tok/s baseline
Q4TP forced dynamic Vulkan	0.634 tok/s	40-token run, 42 submissions/token, 13.9 GB pool

The CPU result varies with NUMA placement; the three-run spread is reported instead of presenting one noisy maximum. Forced Vulkan was about 3.4× slower, which is why it is not the Q4TP default.

A bounded compatibility run with CMF_GPU_VRAM_MB=16000 CMF_GLM_DYNAMIC_MOE=1 completed successfully, emitted Hello, and used a 9.9 GB/792-slot expert pool. Process VRAM samples peaked at 10,260 MiB on the 24 GB test card. This proves that the dynamic pool respects a 16,000 MB budget; the one-token run is not a throughput claim. CPU-only systems and smaller adapters remain valid through the exact CPU fallback. Metal will be tested from this same immutable artifact.

Scope and license

This CMF is a redistribution/quantization of the official GLM-5.3-Flash checkpoint. The upstream model card is MIT-licensed. Refer to the upstream repository for model limitations, intended use, citation, and the authoritative training and benchmark description.