Publish MXFP8 conversion of DFlash2 bf582e4

Offline ModelOpt MXFP8 conversion of incoai/GLM-5.3-Flash-DFlash2 revision bf582e4eacc1810f76656d1811693ff6c6737d2a. Exact re-quantization, preserved-tensor parity, TP4/DCP1 loading, decode CUDA-graph capture, and speculative inference were qualified with the Jovian Judgement r11 runtime.

README.md

@@ -18,7 +18,8 @@ tags:
 
 # GLM-5.3-Flash-DFlash2
 
-This repository contains an MXFP8-quantized DFlash 2 draft model for
+This repository contains an offline MXFP8 conversion of the DFlash 2 draft
+model for
 [`local-inference-lab/GLM-5.3-Flash-NVFP4`](https://huggingface.co/local-inference-lab/GLM-5.3-Flash-NVFP4).
 It is not a standalone language model. A compatible speculative-decoding
 server loads it beside the target model and verifies every drafted token
@@ -26,9 +27,9 @@ against the target.
 
 The source checkpoint is
 [`incoai/GLM-5.3-Flash-DFlash2`](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2)
-at immutable revision `dc77ff1c99eeb2df044ee3d4f0094eb033fee410`.
+at immutable revision `bf582e4eacc1810f76656d1811693ff6c6737d2a`.
 
-## Format
+## Serialized format
 
 - Linear weights: `float8_e4m3fn`
 - Scale values: biased E8M0 exponents stored as `uint8`
@@ -44,12 +45,10 @@ per-weight validation statistics.
 ## Validation status
 
 Status: **qualified** for checkpoint structure, exact format reproduction,
-loading, and smoke inference under the following conditions:
+loading, CUDA-graph capture, and speculative inference under these conditions:
 
-- Target: `local-inference-lab/GLM-5.3-Flash-NVFP4` revision
-  `520de24eabf507659eaef7c70f14fd584527facc`
-- Runtime:
-  `voipmonitor/vllm@sha256:ef53437759e3a41d5ee1c4e9045ffdd7df2972faad50d1dc687e3ab479c5867a`
+- Target: `local-inference-lab/GLM-5.3-Flash-NVFP4`
+- Runtime: `voipmonitor/vllm:jovian-judgement-community-20260901-r11`
 - Hardware: four NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPUs
 - Parallelism: tensor parallel size 4 and decode-context parallel size 1
 - Target attention, MoE, linear, and tensor-parallel all-reduce: B12X
@@ -57,18 +56,17 @@ loading, and smoke inference under the following conditions:
 - DFlash linear: B12X MXFP8
 - DFlash proposal length: seven tokens
 - DFlash KV cache: `auto` (BF16)
-- CUDA graph mode: `FULL` requested; target and DFlash2 decode are captured,
-  while target GDN prefill remains eager
+- CUDA graph mode: full decode graphs for the target and DFlash drafter
 
-The runtime detected ModelOpt MXFP8, selected `B12xMxfp8LinearKernel` for
-draft GEMMs and the fused DFlash context K/V projection, and loaded 1.20 GB of
-draft weights. With seven draft tokens, the qualified runtime measured a
-2.1157-second median time to first token for a 32,320-token prompt and
-185.5 output tokens per second at concurrency one. Speculative throughput
-depends on prompt content and acceptance length.
+The qualification re-quantized all 47 eligible weights and verified exact
+serialized weight and scale equality. All 34 preserved tensors remained
+bit-identical to the BF16 source. The runtime detected ModelOpt MXFP8, selected
+`B12xMxfp8LinearKernel` for draft GEMMs and the fused DFlash context K/V
+projection, captured target and draft decode graphs, and completed speculative
+inference.
 
-The checkpoint is unsupported in vLLM builds that do not contain the DFlash 2
-and ModelOpt MXFP8 integration used by the qualified runtime.
+The checkpoint is unsupported in vLLM builds without the DFlash 2 and
+ModelOpt MXFP8 integrations used by the qualified runtime.
 
 ## Serving
 
@@ -86,10 +84,9 @@ docker run --rm \
   -e MAX_NUM_SEQS=16 \
   -e MAX_MODEL_LEN=262144 \
   -e MAX_NUM_BATCHED_TOKENS=4096 \
-  -e SPECULATOR=dflash \
+  -e SPECULATOR=dflash2 \
   -e NUM_SPECULATIVE_TOKENS=7 \
   -e DFLASH_MODEL=local-inference-lab/GLM-5.3-Flash-DFlash2 \
-  -e DFLASH_MODEL_REVISION= \
   -e DFLASH_KV_CACHE_DTYPE=auto \
   -e DFLASH_ATTENTION_BACKEND=FLASH_ATTN \
   -e ATTENTION_BACKEND=B12X \
@@ -98,13 +95,13 @@ docker run --rm \
   -e B12X_PCIE_ALLREDUCE=1 \
   -e CUDAGRAPH_MODE=FULL \
   -e VLLM_B12X_MOE_FP4_FORCE_A16=0 \
-  voipmonitor/vllm:jovian-judgement-community-dflash2-20260830-r7
+  voipmonitor/vllm:jovian-judgement-community-20260901-r11
 ```
 
-An empty `DFLASH_MODEL_REVISION` makes the launcher resolve the repository's
-`main` branch. For reproducible deployments, replace the empty value with an
-immutable Hugging Face commit hash. The OpenAI-compatible endpoint is
-available at `http://127.0.0.1:8000/v1`.
+The repository name resolves the `main` branch, including subsequent model
+updates. Pin `MODEL_REVISION` and `DFLASH_MODEL_REVISION` to immutable Hugging
+Face commit hashes when deployment reproducibility is required. The
+OpenAI-compatible endpoint is available at `http://127.0.0.1:8000/v1`.
 
 ## License and attribution
 

conversion_manifest.json

@@ -8,8 +8,8 @@
   },
   "error": {
     "maximum_absolute_error": 0.125,
-    "relative_rmse": 0.026583480809130377,
-    "root_mean_squared_error": 0.0028997886924729786
+    "relative_rmse": 0.026583494878908744,
+    "root_mean_squared_error": 0.0028997909497189165
   },
   "format": {
     "block_shape": [
@@ -20,25 +20,41 @@
     "scale_layout": "row-major, unswizzled",
     "weight": "float8_e4m3fn"
   },
-  "output_weights_sha256": "c033e03d47c7d5608596c8fc4e9336a1fe086eb781c08fe031be2bdea1614e58",
+  "output_weights_sha256": "77cccd9287ba0e49d06c140f52de97de9195048229b93570a34406c45514974c",
   "output_files_sha256": {
     ".gitattributes": "70e0ed421d653865ad7ace84728e32ded01ec0a860e989a5ed736fe11b42214f",
-    "README.md": "5a74d5053c29116cb688f40e75aed4b9c21d28a2ad462e963667c3dae522b593",
+    "README.md": "3e89f7d7099635c5a7d38d36aa5d1147df4683525bbc1356711634bf89047dad",
     "config.json": "4ed95da7d0abecd3eb341e96464d381b474d24d8924bd5efba2dfa24b92690bf",
     "hf_quant_config.json": "b7f04ac7e85e73544f9411dbda70fd75ac3b116e6e239ebff5bc85b15caefd42",
-    "model.safetensors": "c033e03d47c7d5608596c8fc4e9336a1fe086eb781c08fe031be2bdea1614e58"
+    "model.safetensors": "77cccd9287ba0e49d06c140f52de97de9195048229b93570a34406c45514974c"
+  },
+  "qualification": {
+    "conditions": {
+      "cuda_graph_mode": "full decode graphs for target and draft",
+      "dflash_attention_backend": "FLASH_ATTN",
+      "dflash_linear_backend": "B12X MXFP8",
+      "dflash_proposal_tokens": 7,
+      "hardware": "4x NVIDIA RTX PRO 6000 Blackwell Workstation Edition",
+      "parallelism": {
+        "decode_context_parallel_size": 1,
+        "tensor_parallel_size": 4
+      },
+      "runtime_image": "voipmonitor/vllm:jovian-judgement-community-20260901-r11",
+      "target_model": "local-inference-lab/GLM-5.3-Flash-NVFP4"
+    },
+    "result": "Exact MXFP8 reproduction, preserved-tensor parity, checkpoint loading, CUDA-graph capture, and speculative inference passed."
   },
   "source": {
     "config_sha256": "c4aeac0101196a6e26705b34c45230bcd0c7c68ee2d2d1efdb242087f3712573",
     "model": "incoai/GLM-5.3-Flash-DFlash2",
-    "revision": "dc77ff1c99eeb2df044ee3d4f0094eb033fee410",
-    "weights_sha256": "b33c03475ba7322cf398828f2d8d1be376df30dc05c6b40c28c8ea8da23e410b"
+    "revision": "bf582e4eacc1810f76656d1811693ff6c6737d2a",
+    "weights_sha256": "b038e1d9d1e7833fa3880c2c0135ba9b673013f03da1b29fb831931584759dac"
   },
-  "status": "implemented",
+  "status": "qualified",
   "tensors": [
     {
-      "maximum_absolute_error": 0.015625,
-      "relative_rmse": 0.02662914012776337,
+      "maximum_absolute_error": 0.013671875,
+      "relative_rmse": 0.026627466688673662,
       "scale": "candidate_selector.hidden_projection.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -51,7 +67,7 @@
     },
     {
       "maximum_absolute_error": 0.0546875,
-      "relative_rmse": 0.026567000680209157,
+      "relative_rmse": 0.026566925402720494,
       "scale": "fc.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -64,7 +80,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.026598510847792713,
+      "relative_rmse": 0.026596101134996542,
       "scale": "layers.0.attention_conv.kernel_projection.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -77,7 +93,7 @@
     },
     {
       "maximum_absolute_error": 0.078125,
-      "relative_rmse": 0.02659119810765467,
+      "relative_rmse": 0.026591517153201866,
       "scale": "layers.0.mlp.down_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -90,7 +106,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.02661559519636154,
+      "relative_rmse": 0.026615793419055775,
       "scale": "layers.0.mlp.gate_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -103,7 +119,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.02661984011036928,
+      "relative_rmse": 0.026619734762732796,
       "scale": "layers.0.mlp.up_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -116,7 +132,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.02654063072293776,
+      "relative_rmse": 0.026541458828176743,
       "scale": "layers.0.mlp_conv.kernel_projection.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -129,7 +145,7 @@
     },
     {
       "maximum_absolute_error": 0.03125,
-      "relative_rmse": 0.026592116125447415,
+      "relative_rmse": 0.026592926809561147,
       "scale": "layers.0.self_attn.k_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -142,7 +158,7 @@
     },
     {
       "maximum_absolute_error": 0.0390625,
-      "relative_rmse": 0.02656823404746311,
+      "relative_rmse": 0.026569630160080508,
       "scale": "layers.0.self_attn.o_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -155,7 +171,7 @@
     },
     {
       "maximum_absolute_error": 0.03125,
-      "relative_rmse": 0.026567040624108283,
+      "relative_rmse": 0.026566880751187266,
       "scale": "layers.0.self_attn.q_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -168,7 +184,7 @@
     },
     {
       "maximum_absolute_error": 0.015625,
-      "relative_rmse": 0.026525638628768222,
+      "relative_rmse": 0.02652669368875796,
       "scale": "layers.0.self_attn.v_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -181,7 +197,7 @@
     },
     {
       "maximum_absolute_error": 0.125,
-      "relative_rmse": 0.026596001906054587,
+      "relative_rmse": 0.026595776616384383,
       "scale": "layers.1.attention_conv.kernel_projection.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -194,7 +210,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.0265787755625202,
+      "relative_rmse": 0.026578616467222597,
       "scale": "layers.1.mlp.down_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -207,7 +223,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.026587237640933212,
+      "relative_rmse": 0.026587099046581963,
       "scale": "layers.1.mlp.gate_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -220,7 +236,7 @@
     },
     {
       "maximum_absolute_error": 0.109375,
-      "relative_rmse": 0.02659690101329357,
+      "relative_rmse": 0.026596407623959584,
       "scale": "layers.1.mlp.up_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -233,7 +249,7 @@
     },
     {
       "maximum_absolute_error": 0.078125,
-      "relative_rmse": 0.02656689675876576,
+      "relative_rmse": 0.02656718684173611,
       "scale": "layers.1.mlp_conv.kernel_projection.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -246,7 +262,7 @@
     },
     {
       "maximum_absolute_error": 0.0546875,
-      "relative_rmse": 0.026569303009495438,
+      "relative_rmse": 0.026568511464967078,
       "scale": "layers.1.self_attn.k_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -259,7 +275,7 @@
     },
     {
       "maximum_absolute_error": 0.03125,
-      "relative_rmse": 0.026560798124397773,
+      "relative_rmse": 0.026560414275462026,
       "scale": "layers.1.self_attn.o_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -272,7 +288,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.026568386185886816,
+      "relative_rmse": 0.026569440403284544,
       "scale": "layers.1.self_attn.q_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -285,7 +301,7 @@
     },
     {
       "maximum_absolute_error": 0.03125,
-      "relative_rmse": 0.0265892482789613,
+      "relative_rmse": 0.026588824773246745,
       "scale": "layers.1.self_attn.v_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -298,7 +314,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.02657074981178284,
+      "relative_rmse": 0.0265710990326379,
       "scale": "layers.2.attention_conv.kernel_projection.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -311,7 +327,7 @@
     },
     {
       "maximum_absolute_error": 0.109375,
-      "relative_rmse": 0.026572407862604005,
+      "relative_rmse": 0.02657244217106894,
       "scale": "layers.2.mlp.down_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -324,7 +340,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.026568281345928493,
+      "relative_rmse": 0.026567826079008,
       "scale": "layers.2.mlp.gate_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -337,7 +353,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.026585907486245345,
+      "relative_rmse": 0.026585781483757672,
       "scale": "layers.2.mlp.up_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -350,7 +366,7 @@
     },
     {
       "maximum_absolute_error": 0.109375,
-      "relative_rmse": 0.026572698101393868,
+      "relative_rmse": 0.026572509631109388,
       "scale": "layers.2.mlp_conv.kernel_projection.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -363,7 +379,7 @@
     },
     {
       "maximum_absolute_error": 0.0546875,
-      "relative_rmse": 0.026577556384314666,
+      "relative_rmse": 0.026579215876272266,
       "scale": "layers.2.self_attn.k_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -376,7 +392,7 @@
     },
     {
       "maximum_absolute_error": 0.03125,
-      "relative_rmse": 0.026560576726262823,
+      "relative_rmse": 0.026561378029306894,
       "scale": "layers.2.self_attn.o_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -389,7 +405,7 @@
     },
     {
       "maximum_absolute_error": 0.0546875,
-      "relative_rmse": 0.026586736069846087,
+      "relative_rmse": 0.02658535159478217,
       "scale": "layers.2.self_attn.q_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -402,7 +418,7 @@
     },
     {
       "maximum_absolute_error": 0.03125,
-      "relative_rmse": 0.026586952065358875,
+      "relative_rmse": 0.026587201193069076,
       "scale": "layers.2.self_attn.v_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -415,7 +431,7 @@
     },
     {
       "maximum_absolute_error": 0.125,
-      "relative_rmse": 0.026572177207037927,
+      "relative_rmse": 0.026572178687421767,
       "scale": "layers.3.attention_conv.kernel_projection.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -428,7 +444,7 @@
     },
     {
       "maximum_absolute_error": 0.109375,
-      "relative_rmse": 0.02656499109738341,
+      "relative_rmse": 0.02656522979813248,
       "scale": "layers.3.mlp.down_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -441,7 +457,7 @@
     },
     {
       "maximum_absolute_error": 0.125,
-      "relative_rmse": 0.026570417196929284,
+      "relative_rmse": 0.026570669526774636,
       "scale": "layers.3.mlp.gate_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -454,7 +470,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.02658615684586314,
+      "relative_rmse": 0.026586021519426385,
       "scale": "layers.3.mlp.up_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -467,7 +483,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.026566849904042712,
+      "relative_rmse": 0.0265672785606844,
       "scale": "layers.3.mlp_conv.kernel_projection.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -480,7 +496,7 @@
     },
     {
       "maximum_absolute_error": 0.0546875,
-      "relative_rmse": 0.02657702447659922,
+      "relative_rmse": 0.026575981844464292,
       "scale": "layers.3.self_attn.k_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -493,7 +509,7 @@
     },
     {
       "maximum_absolute_error": 0.03125,
-      "relative_rmse": 0.026568267578320686,
+      "relative_rmse": 0.026568583531800763,
       "scale": "layers.3.self_attn.o_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -506,7 +522,7 @@
     },
     {
       "maximum_absolute_error": 0.0546875,
-      "relative_rmse": 0.026577504789784222,
+      "relative_rmse": 0.026579093946517987,
       "scale": "layers.3.self_attn.q_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -519,7 +535,7 @@
     },
     {
       "maximum_absolute_error": 0.03125,
-      "relative_rmse": 0.026590205296363092,
+      "relative_rmse": 0.02658932614592993,
       "scale": "layers.3.self_attn.v_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -532,7 +548,7 @@
     },
     {
       "maximum_absolute_error": 0.125,
-      "relative_rmse": 0.026593574382866452,
+      "relative_rmse": 0.02659544494101248,
       "scale": "layers.4.attention_conv.kernel_projection.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -545,7 +561,7 @@
     },
     {
       "maximum_absolute_error": 0.109375,
-      "relative_rmse": 0.02661912299049318,
+      "relative_rmse": 0.026619397530277512,
       "scale": "layers.4.mlp.down_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -558,7 +574,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.02657367966586845,
+      "relative_rmse": 0.026573592370896636,
       "scale": "layers.4.mlp.gate_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -571,7 +587,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.026602028239691457,
+      "relative_rmse": 0.02660203941644582,
       "scale": "layers.4.mlp.up_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -584,7 +600,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.02658053894333081,
+      "relative_rmse": 0.026581684820890152,
       "scale": "layers.4.mlp_conv.kernel_projection.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -597,7 +613,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.026593862409928445,
+      "relative_rmse": 0.026593581600441705,
       "scale": "layers.4.self_attn.k_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -610,7 +626,7 @@
     },
     {
       "maximum_absolute_error": 0.03125,
-      "relative_rmse": 0.02655268219685373,
+      "relative_rmse": 0.026552704535880215,
       "scale": "layers.4.self_attn.o_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -623,7 +639,7 @@
     },
     {
       "maximum_absolute_error": 0.0625,
-      "relative_rmse": 0.026596374233145523,
+      "relative_rmse": 0.02659594803833882,
       "scale": "layers.4.self_attn.q_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [
@@ -636,7 +652,7 @@
     },
     {
       "maximum_absolute_error": 0.03125,
-      "relative_rmse": 0.026648845046121847,
+      "relative_rmse": 0.02664900482452192,
       "scale": "layers.4.self_attn.v_proj.weight_scale",
       "scale_dtype": "uint8",
       "shape": [

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c033e03d47c7d5608596c8fc4e9336a1fe086eb781c08fe031be2bdea1614e58
+oid sha256:77cccd9287ba0e49d06c140f52de97de9195048229b93570a34406c45514974c
 size 1284719240