Void-Z/GLM-5.2-NVFP4-FP8-Hybrid

GLM-5.2 NVFP4–FP8 Hybrid

This is a mixed-precision GLM-5.2 checkpoint optimized for prefill performance.

In plain terms, the most compute-intensive MoE expert weights use NVFP4, eligible dense linear layers use FP8, and a small number of required partitions remain in BF16 or their original format. This is neither a pure NVFP4 model nor a pure FP8 model, and it is not the same checkpoint as nvidia/GLM-5.2-NVFP4 or zai-org/GLM-5.2-FP8.

This repository contains the checkpoint only. The matching SGLang/FlashInfer environment required to load this hybrid layout and reproduce the prefill result is published as a companion Docker image.

At a glance

Item	Value
Reference hardware	One host with 8× NVIDIA RTX 6000D (SM120)
Runner hardware policy	Exactly 8 visible GPUs for TP1/PP8; no GPU-model or SM-version restriction
Validated profile	Prefill 3s, TP1/PP8, ISL/OSL 8192/1, CC4
Checkpoint size	Approximately 447 GB in 47 safetensors shards
Companion image	voidobserver/glm52-nvfp4-fp8-hybrid-repro
Linux/amd64 runtime digest	sha256:1940fb97bf3f796e77d258ca1191ca32aa9fe0627dadbd9c84ee27a84c99da3f
OCI root index	sha256:7802cccee25f377f15018c7c9f733452e86fe8a58a689e45fb60cdeaf6f6fa0e
Reproducible checkpoint revision	0e98eb433502787e085fe6ac1a304730b58db8c6

Use the checkpoint revision and image digest shown above together. The model card on main may continue to evolve, while the complete checkpoint manifest embedded in the v2 image is intentionally pinned to this immutable checkpoint revision.

Quick reproduction of the 3s prefill result

Start with a Linux x86_64 server with exactly eight visible, idle NVIDIA GPUs, Docker Engine, and the NVIDIA Container Toolkit. The published performance reference was measured on 8× NVIDIA RTX 6000D (SM120). Other GPU models and compute capabilities are accepted by the runner on a best-effort basis. Reserve at least 500 GB on the model volume and 100 GB on the Docker data volume. If both share one filesystem, plan for at least 600 GB of free space in total.

Download the validated checkpoint revision:

python3 -m pip install -U huggingface_hub

export HOST_MODEL_DIR="$PWD/GLM-5.2-NVFP4-FP8-Hybrid"
export HOST_RESULTS_DIR="$PWD/glm52-prefill-results"
mkdir -p "${HOST_MODEL_DIR}" "${HOST_RESULTS_DIR}"

hf download Void-Z/GLM-5.2-NVFP4-FP8-Hybrid \
  --revision 0e98eb433502787e085fe6ac1a304730b58db8c6 \
  --local-dir "${HOST_MODEL_DIR}"

Run the immutable reproduction image:

export IMAGE="docker.io/voidobserver/glm52-nvfp4-fp8-hybrid-repro@sha256:1940fb97bf3f796e77d258ca1191ca32aa9fe0627dadbd9c84ee27a84c99da3f"
export RUN_ID="repro-$(date -u +%Y%m%dT%H%M%SZ)"

docker pull --platform linux/amd64 "${IMAGE}"

docker run --rm \
  --platform linux/amd64 \
  --gpus all \
  --ipc=host \
  --network=host \
  --ulimit memlock=-1 \
  -e RUN_ID="${RUN_ID}" \
  -e MODEL_DIR=/models/GLM-5.2-HYBRID \
  -e FULL_CHECKPOINT_HASH=1 \
  --mount "type=bind,src=${HOST_MODEL_DIR},dst=/models/GLM-5.2-HYBRID,readonly" \
  --mount "type=bind,src=${HOST_RESULTS_DIR},dst=/results" \
  "${IMAGE}" \
  reproduce 3s

Inspect the final result:

python3 -m json.tool "${HOST_RESULTS_DIR}/prefill-${RUN_ID}/summary.json"

A top-level "passed": true means that the checkpoint, workload contract, request correctness, and semantic checks passed. Read reference_performance_passed to see whether the run also reached both published SM120 performance thresholds. Add -e STRICT_REFERENCE_GATE=1 to docker run only when a miss against those reference thresholds should make the command fail.

Keep the in-container path /models/GLM-5.2-HYBRID when comparing against the validated SM120 result because it is part of the frozen autotune-cache identity. The host-side HOST_MODEL_DIR can be any directory. See REPRODUCTION.md for the full procedure, output layout, result interpretation, and troubleshooting guidance.

Validated reference result

The numbers below were produced with the original winner and the v1 public-image E2E run. The v2 image keeps the same checkpoint, SGLang/FlashInfer runtime, tuned kernel, launch arguments, and SM120 autotune payload; it changes hardware admission and performance-result reporting. The v2 image has passed build-time integrity checks and a container-entrypoint smoke test, but has not yet had another 447 GB full E2E run.

Test configuration: 8× NVIDIA RTX 6000D, TP1/PP8, ISL/OSL 8192/1, CC4, chunk size 2048, max prefill tokens 8192, 32 measured requests, 8 warmup requests, MTP disabled, FP8 E4M3 KV cache, and radix cache disabled.

Run	P50 TTFT	Average TTFT	Input TPS
Original winner	2170.861 ms	2171.674 ms	14,406.164 token/s
Public-image cluster E2E reproduction (Enroot/SquashFS)	2172.728 ms	2173.179 ms	14,394.746 token/s

The public-image cluster E2E run completed 32/32 requests with no failed requests and no cache hits. Input TPS differed from the original winner by only -0.079%. Each row above reports one fresh-server N32 lifecycle, not an aggregate across repeated runs. The public cluster rerun used NVIDIA driver 580.95.05.

The same run also passed:

• complete SHA256 validation of all 59 files listed in the image's embedded release checksum manifest;
• loading of all 47 safetensors shards;
• the 8-rank runtime-source gate;
• the FlashInfer JIT gate;
• server configuration validation;
• the deterministic France → Paris semantic smoke test;
• the P50 TTFT and Input TPS performance thresholds.

Clock settings, temperature, GPU topology, driver version, and system load can cause normal performance variation across machines.

Quantization layout

Model component	Format
Routed experts in layers 3–77	ModelOpt NVFP4
Shared-expert gate_proj, up_proj, and down_proj in layers 3–77	ModelOpt NVFP4
Other ordinary linear layers covered by the hybrid policy	Block FP8
kv_b, routers, and enabled indexer projections	Scalar FP8
Remaining retained and MTP partitions	BF16 or original format

The final assembly stage copied existing tensor payloads according to the policy and did not requantize them. Shared-expert fusion must be disabled at serving time; the companion image already provides the required arguments.

Checkpoint metadata	Value
Safetensors shards	47
Indexed tensors	233,555
Indexed tensor bytes	446,862,784,064
Model index SHA256	6d2b19476460be27433b1e6176a2834a6f9a21700212367a4f5118f3493cd5c4

See PROVENANCE.json and ATTRIBUTION.md for source and assembly details.

Validation scope and limitations

• The validated cluster E2E profile is reproduce 3s only; the public 1.5s path has not been validated end to end.
• TP1/PP8 requires exactly eight visible GPUs. The v2 runner does not reject a GPU model or compute capability. It warns outside SM120 and skips SM120-specific autotune/runtime/JIT validation there; actual runtime compatibility remains dependent on the included CUDA, SGLang, and FlashInfer stack.
• This is a fixed text-prefill reproduction environment, not a general-purpose SGLang image.
• Stock SGLang is not guaranteed to load this mixed-precision checkpoint correctly. Use the companion image above.
• The v1 public linux/amd64 manifest used for the reference E2E run was pulled from the registry and converted to Enroot/SquashFS. The v2 manifest above was independently built, integrity-checked, pushed with provenance, and published with a full SPDX SBOM OCI referrer, but a separate v2 full E2E run has not yet been performed.
• The public E2E run used the local checkpoint copy retained after the Hugging Face upload rather than downloading approximately 447 GB again. All 59 files listed in the image's embedded release checksum manifest matched the pinned Hugging Face revision by SHA256, which establishes byte identity but does not measure public download time.
• The GSM8K result of 122/128 is a fixed smoke gate, not a full quality evaluation. Perform application-specific accuracy, safety, and stability testing before deployment.

License

The checkpoint is released under the MIT License. See LICENSE and ATTRIBUTION.md.

The companion image also contains SGLang, FlashInfer, CUDA runtime components, and other third-party software. Each component remains subject to its own license and redistribution terms. Review the notices and licenses in the image and the attached SPDX SBOM before redistributing it.