Qwen3-8B-NVFP4 / README.md

llmat

Add NVFP4 quantized model for Qwen/Qwen3-8B.

6464e57 verified 8 months ago

preview code

raw

history blame contribute delete

455 Bytes

metadata

language: en
license: apache-2.0
pipeline_tag: text-generation
tags:
  - quantization
  - nvfp4
  - qwen
base_model: Qwen/Qwen3-8B
model_name: Qwen3-8B-NVFP4

Qwen3-8B-NVFP4

NVFP4-quantized version of Qwen/Qwen3-8B produced with llmcompressor.

Notes

Quantization scheme: NVFP4 (linear layers, lm_head excluded)
Calibration samples: 512
Max sequence length during calibration: 2048