# MiniCPM5-1B-NVFP4-W4A4

NVFP4 (W4A4) quantized version of [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B).

Quantized with [llm-compressor](https://github.com/vllm-project/llm-compressor)
using the `NVFP4` scheme, saved in compressed-tensors format. Both weights and
activations are FP4 (W4A4): per-group-16 static fp8 weight scales, and per-group
local dynamic fp8 activation scales (static_minmax observer).

Calibration: 128 samples drawn from the
[`canada-quant/hy3-w4a16-mtp-calibration`](https://huggingface.co/datasets/canada-quant/hy3-w4a16-mtp-calibration)
blend (code / math / chat, ~2048 tokens each), formatted through this model's
chat template. `lm_head` is left in bf16.

Loadable directly in vLLM. Needs a Blackwell GPU with CUDA 13 (e.g. RTX 5090)
for native FP4 kernels:

```bash
vllm serve MaxDevv/MiniCPM5-1B-NVFP4-W4A4 \
  --kv-cache-dtype fp8_e4m3 \
  --attention-backend flashinfer
```

---

Original model below:

- [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)