Upload README.md with huggingface_hub

README.md

@@ -16,11 +16,11 @@ pipeline_tag: text-generation
 
 # GLM-4.7-Flash NVFP4 (Mixed Precision)
 
-This is a **mixed precision NVFP4 quantization** of [zai-org/GLM-4.7-Flash](https://huggingface.co/zai-org/GLM-4.7-Flash), a 31B parameter Mixture-of-Experts model.
+This is a **mixed precision NVFP4 quantization** of [zai-org/GLM-4.7-Flash](https://huggingface.co/zai-org/GLM-4.7-Flash), a 30B-A3B (30B total, 3B active) Mixture-of-Experts model.
 
 ## Quantization Strategy
 
-Qunatizied with custom scripts / Calibration based on NVIDIA's approach for DeepSeek-V3, this model uses **mixed precision** to preserve accuracy:
+Based on NVIDIA's approach for DeepSeek-V3, this model uses **mixed precision** to preserve accuracy:
 
 | Component | Precision | Rationale |
 |-----------|-----------|-----------|
@@ -38,6 +38,8 @@ Qunatizied with custom scripts / Calibration based on NVIDIA's approach for Deep
 | Compression | 1x | 3.3x | **3.1x** |
 | Accuracy Loss | - | -8.0% | **-1.3%** |
 
+**Key Finding**: Uniform FP4 quantization causes 8% accuracy loss on this model due to the sensitive MLA attention layers. Mixed precision reduces this to only 1.3% while maintaining 3.1x compression.
+
 ## Usage
 
 ### Requirements
@@ -84,7 +86,7 @@ vllm serve GadflyII/GLM-4.7-Flash-NVFP4 \
 
 - **Base Model**: [zai-org/GLM-4.7-Flash](https://huggingface.co/zai-org/GLM-4.7-Flash)
 - **Architecture**: `Glm4MoeLiteForCausalLM`
-- **Parameters**: 31B total (5.5B active per token)
+- **Parameters**: 30B total, 3B active per token (30B-A3B)
 - **MoE Configuration**: 64 routed experts, 4 active, 1 shared expert
 - **Layers**: 47
 - **Context Length**: 202,752 tokens (max)
@@ -98,6 +100,15 @@ vllm serve GadflyII/GLM-4.7-Flash-NVFP4 \
 - **Calibration**: 128 samples from neuralmagic/calibration dataset
 - **Full Expert Calibration**: All 64 experts calibrated per sample
 
+## Why Mixed Precision?
+
+GLM-4.7-Flash uses **MLA (Multi-head Latent Attention)** from DeepSeek-V2, which compresses queries and key-values through low-rank projections:
+
+- `q_a_proj`, `q_b_proj` - Query compression
+- `kv_a_proj`, `kv_b_proj` - Key-Value compression
+
+These compressed representations are highly sensitive to quantization noise. NVIDIA's DeepSeek-V3 NVFP4 release explicitly keeps all attention layers in higher precision for this reason.
+
 ## Evaluation
 
 MMLU-Pro results by category: