AlphaOxO/Llama-3.3-8B-Instruct-NVFP4

Llama 3.3 8B Instruct NVFP4

Using Hardware

• CPU: AMD Ryzen Threadripper PRO 7995WX
• MB: GIGABYTE AI TOP TRX50
• GPU: RTX 5090*1
• RAM: RDIMM DDR5 5600 128GB*2

Using Software

• CUDA version: 13.0
• CUDA driver version: 580.95.05
• pyTorch: 2.10.0+cu130
• transformers: 5.3.0
• llmcompressor: 0.10.0.1
• vllm: 0.17.1

Backbone Model

shb777/Llama-3.3-8B-Instruct-128K

Using Dataset

tatsu-lab/alpaca convert as chat template.

Compare

--	origin model(BF16)	Quantization(This Model)
leaderboard_gpqa	0.303	0.298
VRAM Usage	14.99 GiB	5.69 GiB