GLM-4.7-Flash: model card

README.md

@@ -0,0 +1,80 @@
+---
+license: mit
+base_model: zai-org/GLM-4.7-Flash
+tags:
+  - coreai
+  - aimodel
+  - apple-silicon
+  - on-device
+  - glm
+  - mixture-of-experts
+  - mla
+pipeline_tag: text-generation
+---
+
+# GLM-4.7-Flash — Apple Core AI (`.aimodel`)
+
+**zai-org's GLM-4.7-Flash converted to Apple's Core AI** (the Core ML successor announced at
+WWDC26), ready to run on macOS 27. This is the **first model on the zoo with Multi-head Latent
+Attention (MLA)** — the DeepSeek-V3 attention family — riding Apple's **`coreai-pipelined` GPU
+engine** with zero custom kernels.
+
+> Requires the macOS 27 beta (Core AI ships with the OS). **Mac-only** (~30 GB int8 + a
+> materialized full-MHA KV cache). Conversion code, parity ladder, and the Swift runner:
+> **[coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo)** (see
+> [`zoo/glm-4.7-flash.md`](https://github.com/john-rocky/coreai-model-zoo/blob/main/zoo/glm-4.7-flash.md)).
+
+## Architecture
+
+**~30B parameters, ~3B active per token** — **MLA attention on all 47 layers + a sparse MoE FFN**:
+
+- **MLA** (heads 20, `q_lora_rank` 768, `kv_lora_rank` 512, `qk_nope` 192 + decoupled `qk_rope`
+  64 = head_dim 256, `v_head_dim` 256, RoPE θ=1e6 interleaved), authored in the **naive /
+  materialized** form: latent caches are projected up to per-head q/k/v, decoupled RoPE is
+  applied to the 64-dim rope slice, and the full per-head `[20, 256]` Q/K/V run through Apple's
+  standard SDPA composite. The KV cache stores the materialized per-head K/V.
+- **MoE FFN** (layers 1–46; layer 0 dense, `first_k_dense_replace=1`): 64 routed experts, top-4,
+  plus one **non-gated shared expert**. `noaux_tc` routing reduces (n_group=1) to sigmoid scoring
+  + selection-only bias correction; experts ride Apple's `SwitchGLU`/`GatherMM` composite.
+
+## Measured (macOS 27 beta, M4 Max 128 GB, release `llm-benchmark`, `COREAI_CHUNK_THRESHOLD=1`)
+
+| config | bundle | prefill tok/s | decode tok/s |
+|---|---:|---:|---:|
+| **int8 linear per-block-32 + untied absmax int8 head (`int8hu --head-sym`) = SHIP** | **30 GB** | **20.5** | **20.3** |
+
+Decode is context-stable (set by the per-token weight read, not KV growth). **~20 tok/s for a
+30B-class local coder on a Mac** — comfortably above reading speed.
+
+## Numerics (gated)
+
+- **Parity** (fp16 port vs fp32 HF oracle; 30B is too large for an fp32-resident oracle on
+  128 GB): RoPE / isolated MLA block / isolated MoE block all cosine **1.000000**; all 47 layers
+  ≥0.9995; logits **top-1 16/16**; stateful decode + teacher-forced sweep all match. The
+  interleaved decoupled-RoPE convention is verified **bit-exact vs HF**.
+- **int8 end-to-end**: the real `coreai-sequential` engine reproduces the eager-int8 CPU greedy
+  continuation **token-for-token** over 32 tokens — a coherent, on-topic continuation. Engine ≡
+  python at int8.
+
+## Run
+
+This is a **decode-only loop-free** bundle for the pipelined engine. Drive it token-by-token with
+the `coreai-sequential` engine variant (the default / `coreai-pipelined` variants feed a prefill
+chunk the static-`[1,1]` graph can't take):
+
+```bash
+# tok/s
+COREAI_CHUNK_THRESHOLD=1 llm-benchmark \
+    --model gpu-pipelined/glm_4_7_flash_decode_int8hu_block32_sym -p 64 -g 128 -n 3
+# greedy generation
+COREAI_CHUNK_THRESHOLD=1 llm-runner \
+    --model gpu-pipelined/glm_4_7_flash_decode_int8hu_block32_sym \
+    --inference-engine-variant coreai-sequential --sampling-strategy greedy \
+    --prompt "Write a Python function to merge two sorted lists." --max-tokens 128
+```
+
+## License
+
+GLM-4.7-Flash is released by zai-org under the **MIT License** (see `LICENSE`). The conversion
+code (in [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo)) derives from Apple's
+BSD-3-clause `coreai_models`.