---
license: apache-2.0
base_model: Kwaipilot/KAT-Coder-V2.5-Dev
tags:
- qwen3_5_moe
- code
- mxfp4
- compressed-tensors
- vllm
- xpu
library_name: vllm
---

# KAT-Coder-V2.5-Dev-MXFP4

MXFP4 (`compressed-tensors` `mxfp4-pack-quantized`: E2M1 group-32 symmetric,
E8M0 scales) quantization of [Kwaipilot/KAT-Coder-V2.5-Dev](https://huggingface.co/Kwaipilot/KAT-Coder-V2.5-Dev)
(35B-A3B MoE, 40 layers / 256 experts / 8 active), built for the
**vLLM XPU backend** (`CompressedTensorsW4A4Mxfp4` + `XPUExpertsMxFp4` on Intel Arc / Xe2).

## Contents

- **Trunk**: all 30,720 routed-expert projections (per-expert gate/up/down) +
  150 linear-attention projections, quantized with the inference kernel's own
  quantizer (`torch.ops.vllm.xpu_mxfp4_quantize` — format match by construction).
  Everything else (self-attn, norms, shared experts, embed, lm_head) is unchanged BF16.
- **MTP draft block** (`mtp.*`, from
  [SpectreSystems/KAT-Coder-V2.5-Dev-MTP](https://huggingface.co/SpectreSystems/KAT-Coder-V2.5-Dev-MTP)):
  768 experts unfused and MXFP4-quantized, 17 support tensors BF16 — ready for
  `--speculative-config '{"method":"mtp","num_speculative_tokens":2}'`.
- `text_config.mtp_num_hidden_layers` is set to 1 (the base checkpoint ships 0,
  which would build a zero-layer draft).

63,756 tensors / 29 shards / **19.60 GiB**. `SHA256SUMS` included.

## Measured quality of the quantization

- In-conversion spot checks (kernel dequantizer, 1-in-64): max mean rel-err 0.134.
- Independent post-hoc CPU audit of 2,075 pairs vs fresh BF16: mean 0.116,
  max 0.157 on a single pair (outlier-group concentrated, median-normal).

## Measured serving (Intel Arc 140V, vLLM 0.30.0+xpu, eager)

| Config | 1024/128/1 decode |
|---|---|
| Eager | ~13–14 tok/s (E-cores; ~19 expected on P-cores) |
| MTP K=1 | 23.22 tok/s |
| MTP K=2 | 24.82 tok/s |

10/10 golden-suite prompts clean, 0 errors. Example launch:

```bash
vllm serve KAT-Coder-V2.5-Dev-MXFP4 --served-model-name KAT-Coder-V2.5-Dev \
  --max-model-len 4096 --max-num-seqs 1 --gpu-memory-utilization 0.68 \
  --language-model-only --enforce-eager \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}' \
  --kv-cache-memory-bytes 500000000
```

## Provenance

Built with the `quantize_kat_mxfp4.py` / `quantize_kat_mtp_mxfp4.py` /
`finalize_kat_mxfp4.py` / `audit_kat_mxfp4.py` pipeline
(see [alpha2beta/vllm.xpu](https://github.com/alpha2beta/vllm.xpu)).
Base weights: `Kwaipilot/KAT-Coder-V2.5-Dev` (Apache-2.0).
MTP weights: `SpectreSystems/KAT-Coder-V2.5-Dev-MTP`.
Reference schema: `pahajokiconsulting/Qwen3.6-35B-A3B-MXFP4`.