Kimi-K3 NVFP4A16 β€” Lossless Transcode

Weight-only NVFP4A16 derivative of source revision moonshotai/Kimi-K3@9f62e4e9fffbd0a83ddd60e1c209d828994b3569.

Method

FP4 packed bytes are preserved exactly. Each active (nonzero) MXFP4 E8M0 group-32 scale is duplicated for two NVFP4 group-16 blocks and represented exactly with an E4M3 scale plus a power-of-two global scale. An out-of-window scale attached only to an all-zero group may be canonicalized because its dequantized values remain zero.

Verified on shard 49, expert 0, real K3 w1/w2/w3 weights: all packed bytes are identical and every reconstructed value has zero mismatch and zero maximum absolute error.

  • Format: compressed-tensors, nvfp4-pack-quantized
  • Weights: FP4 E2M1, tensor-group strategy, group size 16
  • Local scales: FP8 E4M3FN
  • Activations: unquantized (input_activations: null)
  • Quantized modules: routed MoE experts only; all ignored K3 modules remain bit-identical
  • Source revision: 9f62e4e9fffbd0a83ddd60e1c209d828994b3569
  • Source provenance: retroactive-hf-commit-window
  • Lossless over-span policy: fail; never silently requantize
  • Tensor data size: 1.497 TiB
  • Size change versus source tensor data: +5.45%
  • Companion artifact: GrEarl/Kimi-K3-NVFP4A16-Requantized

This is NVFP4A16, not calibrated W4A4 NVFP4. Kimi K3 was trained with MXFP8 activations; FP4 activation quantization would require a separate calibration and quality-validation run.

Runtime validation (2026-07-27 UTC)

A complete Stable LatentMoE block passed on vLLM for this published artifact using the official KimiMoE class. The test first loaded and ran the block through a purpose-built layout path, then rebuilt it and loaded the same 8,072 source tensors through the real K3 model-class chain: KimiK3ForConditionalGeneration.load_weights -> AutoWeightsLoader -> KimiLinearForCausalLM.load_weights -> KimiLinearModel.load_weights -> the generic process_weights_after_loading traversal.

  • Evidence run: 20260727T234215Z
  • Earlier routed-expert-core run: 20260727T220141Z
  • Tested artifact revision: 8c22f32993e94e063cb634a03b7fc2f9ff539621
  • Checkpoint shard: model-00049-of-000096.safetensors (17,916,197,160 bytes)
  • Runtime image: vllm/vllm-openai:kimi-k3
  • Pinned linux/amd64 image: vllm/vllm-openai@sha256:fb16b180bd9727600067e16fcd6a6de43fb4db1baf4298ef20b4dbdf6bfa5a0e
  • vLLM: 0.1.dev19262+gb6bbf29dd.d20260727
  • Hardware: NVIDIA B200, SM100; torch 2.13.0+cu130, CUDA 13.0
  • Complete path: FP32 router + correction bias, 7168 -> 3584 latent down, 896 routed experts with real top-k 16 and 3584 -> 3072 -> 3584, routed RMSNorm + 3584 -> 7168 up, and 2 shared experts (7168 -> 6144 -> 7168)
  • Loaded source tensors: 8,072 = 8 nonexpert block tensors + 8,064 routed-expert quant tensors; all 896 experts
  • SiTU: beta 4.0, linear beta 25.0
  • Backend: MARLIN / MarlinExperts
  • Observed operators: vllm::moe_forward_shared, _moe_C::moe_wna16_marlin_gemm, _moe_C::grouped_topk, _C::situ_and_mul, vllm_ir::rms_norm, routing alignment, and MoE sum
  • Router logits: [2, 896]; selected IDs [2, 16]; 16 unique in-range experts per token; finite normalized weights
  • Both direct and real model-class-loader outputs: [2, 7168], finite and nonzero; sampled parameter fingerprints matched exactly and outputs were bit-exact (max_abs_difference=0)
  • Complete-block validation run cost: $0.341178; all attempts including the externally canceled run: $3.892933 < $4.5

This verifies the real K3 outer/inner load_weights mapping and generic post-load traversal on a reduced one-block wrapper. DefaultModelLoader index/file traversal was not exercised, and the full model was not constructed.

SGLang runtime was not executed. A zero-cost gate against the official lmsysorg/sglang:kimi-k3 image (linux/amd64 digest sha256:2e8ef3746b2591287db0f37b4470910ab08974bdaf96fc5820bac35f6d3962bc) found that its compressed-tensors selector supports FLOAT tensor-group NVFP4 only as W4A4 with input activation scales; it has no matching NVFP4A16 FLOAT tensor-group MoE path for input_activations: null. The gate therefore returned BLOCKED / DO_NOT_RUN_GPU (20260727T221943Z) before weight download or GPU allocation.

Not verified: the full 93-layer / 2.8T model, all 96 shards resident together, DefaultModelLoader index/file traversal, KDA/MLA/vision integration, TP16, end-to-end token generation, serving quality, or SGLang load/forward/kernel dispatch.

License

Inherits the Kimi K3 License.

Downloads last month
222
Safetensors
Model size
2.8T params
Tensor type
F32
Β·
BF16
Β·
U8
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for GrEarl/Kimi-K3-NVFP4A16-Transcoded

Quantized
(57)
this model

Collection including GrEarl/Kimi-K3-NVFP4A16-Transcoded