Hugging Face's logo

kaitchup
/
Nanbeige4.1-3B-FP8-Dynamic

Safetensors
llama
llm-compressor
compressed-tensors
Model card Files
xet
Community
Nanbeige4.1-3B-FP8-Dynamic / README.md
bnjmnmarie's picture
bnjmnmarie
Create README.md
9b420ae verified
preview code
|
raw
history blame contribute delete
938 Bytes
metadata 
license: apache-2.0
base_model:
  - Nanbeige/Nanbeige4.1-3B
tags:
  - llm-compressor

This is Nanbeige/Nanbeige4.1-3B quantized with llm-compressor to W8A8 (FP8) . The model is compatible with vLLM (tested: v0.15.1). Tested with an L4 (Google Colab).