Update model card: friendly factual lead, H100 KV note, tightened wording

README.md

@@ -22,9 +22,17 @@ tags:
 
 # GLM-5.2-W4AFP8
 
-GLM-5.2-W4AFP8 runs the full 1M-token context of [GLM-5.2](https://huggingface.co/zai-org/GLM-5.2-FP8) on a single Hopper node (8×H200) with no measurable quality loss. The model stores GLM-5.2's Mixture-of-Experts weights in 4-bit, which halves the weight memory footprint (755 GB to 368 GB) and frees enough GPU memory to hold the full 1M-token KV cache on one node. It is the same GLM-5.2 model with the same parameters, served at lower weight precision.
+GLM-5.2-W4AFP8 is a 4-bit quantization of [GLM-5.2](https://huggingface.co/zai-org/GLM-5.2-FP8) that runs the full 1M-token context on a single 8×H200 node.
 
-**Tested with SGLang only.** This checkpoint has been validated with [SGLang](https://github.com/sgl-project/sglang) v0.5.13.post1. It has not been tested with vLLM or any other engine. The 4-bit weight layout and the sparse-attention and speculative-decoding handling are SGLang-specific.
+Two things make it useful:
+
+- **The full 1M context fits on one node.** Quantizing the Mixture-of-Experts weights to 4-bit cuts their footprint from 755 GB to 368 GB, which frees enough memory for the complete 1M-token KV cache. The FP8 release reaches only about 500K tokens on the same hardware.
+- **No measurable quality loss.** Reasoning, instruction-following, long-context, and tool-calling benchmarks all land within measurement noise of the FP8 model.
+
+> [!NOTE]
+> This model keeps GLM-5.2's full 753B parameters. The ~387B in the sidebar is a counting quirk: two 4-bit weights share each int8 slot, so the metadata counts storage slots, not parameters.
+
+**Tested with SGLang only** ([v0.5.13.post1](https://github.com/sgl-project/sglang)). The 4-bit layout and speculative-decoding path are SGLang-specific and untried on vLLM or other engines.
 
 ## Benchmarks
 
@@ -35,10 +43,10 @@ Measured on 8×H200 with SGLang at GLM-5.2's recommended sampling (temperature 1
 | GPQA-Diamond (reasoning) | 90.9 | 91.2 |
 | IFBench (instruction following, strict) | 75.0 | 74.3 |
 | AA-LCR (long-context reasoning) | 76.0 | 69.7 |
-| Needle-in-a-haystack at ~983K tokens | 3 / 3 retrieved | n/a |
-| BFCL (tool calling) | matches reference | n/a |
+| Needle-in-a-haystack at ~983K tokens | 3 / 3 retrieved | not published |
+| BFCL (tool calling) | matches FP8 baseline | not published |
 
-Reasoning (GPQA) and instruction following (IFBench) match the published results within evaluation tolerance. Long-context retrieval and tool calling show no degradation. The model is aligned with the official GLM-5.2.
+Reasoning (GPQA) and instruction following (IFBench) match the published results within evaluation tolerance. GLM-5.2 publishes no BFCL or needle reference, so those compare against the FP8 release: tool calling matches (84% vs 87% overall, within sampling noise) and needle retrieval hit all three depths near the 1M-token limit.
 
 ## Performance (8×H200, single node)
 
@@ -49,8 +57,6 @@ The model includes a quantized Multi-Token Prediction (MTP) layer, so SGLang's E
 | Single-stream decode | 75 tok/s | 118 tok/s |
 | Throughput (8K in, 1K out, 16 concurrent) | 658 tok/s | 733 tok/s |
 
-Speculative decoding is lossless. The main model verifies every drafted token, so outputs are unchanged.
-
 ## Serving
 
 Requirements: Hopper GPUs (H100 or H200, SM90) and SGLang v0.5.13.post1 or later.
@@ -58,18 +64,26 @@ Requirements: Hopper GPUs (H100 or H200, SM90) and SGLang v0.5.13.post1 or later
 ```bash
 python -m sglang.launch_server \
   --model-path PhalaCloud/GLM-5.2-W4AFP8 \
-  --quantization w4afp8 --disable-shared-experts-fusion \
-  --tp 8 --kv-cache-dtype fp8_e4m3 \
-  --reasoning-parser glm45 --tool-call-parser glm47 \
-  --context-length 1048576 --mem-fraction-static 0.85 \
+  --quantization w4afp8 \
+  --disable-shared-experts-fusion \
+  --tp 8 \
+  --kv-cache-dtype fp8_e4m3 \
+  --reasoning-parser glm45 \
+  --tool-call-parser glm47 \
+  --context-length 1048576 \
+  --mem-fraction-static 0.85 \
   --trust-remote-code
 ```
 
+On 8×H200 (1128 GB) this serves the full 1M-token context. On 8×H100 (640 GB) the weights still fit and leave room for roughly 400K tokens; the FP8 release does not fit on 8×H100 at all.
+
 To enable speculative decoding (the 118 tok/s single-stream path), add:
 
 ```bash
-  --speculative-algorithm EAGLE --speculative-num-steps 1 \
-  --speculative-eagle-topk 1 --speculative-num-draft-tokens 2
+  --speculative-algorithm EAGLE \
+  --speculative-num-steps 1 \
+  --speculative-eagle-topk 1 \
+  --speculative-num-draft-tokens 2
 ```
 
 The server exposes the standard OpenAI-compatible API: chat completions, tool calls, and reasoning content.
@@ -78,8 +92,6 @@ The server exposes the standard OpenAI-compatible API: chat completions, tool ca
 
 The routed MoE experts use 4-bit integer weights (group size 128) with FP8 (E4M3) activations. Dense layers, shared experts, attention, the sparse-attention indexer, and the MTP layer's non-expert weights stay in FP8 or BF16. Activations are quantized at run time. This is the layout SGLang's `w4afp8` path loads. Weight quantization used activation-aware scaling (AWQ) with a short calibration set.
 
-The model keeps GLM-5.2's full parameter count (about 750B). Only the weight precision changes. The parameter count shown on this page (about 387B) is a storage artifact: two 4-bit weights pack into each byte, so the figure counts storage elements rather than logical parameters.
-
 ## Limitations
 
 - Tested with SGLang only. vLLM and other engines are unverified.