GLM-4.7-Flash-ONNX-WebGPU

zai-org/GLM-4.7-Flash converted to ONNX for running in a web browser with ONNX Runtime Web's WebGPU backend, as used by Quick Inference.

Changes from the original

• Converted to an ONNX decoder in ONNX Runtime GenAI's layout (input_ids, attention_mask, past_key_values.* → logits, present.*) by Quick Inference's own exporter.
• Attention is the absorbed form of multi-head latent attention: the KV cache holds the 512-wide normalised latent and the 64-wide rotary key (one shared head, 576 wide); the key and value projections of kv_b_proj are folded into the query projection and o_proj.
• Linear layers are quantised to 4 bits (round to nearest, block size 32); the routed experts to 4 bits for the QMoE operator. Embeddings, activations and the KV cache are float16.
• The multi-token-prediction layer is left out.

These are modified files: their outputs are close to, but not identical to, the original model's.

Licence

MIT, as the original (LICENSE).