NVFP4 Quantization
Collection
5 items • Updated
Using Hardware
- CPU: AMD Ryzen Threadripper PRO 7995WX
- MB: GIGABYTE AI TOP TRX50
- GPU: RTX 5090*1
- RAM: RDIMM DDR5 5600 128GB*2
Using Software
- CUDA version: 13.0
- CUDA driver version: 580.95.05
- pyTorch: 2.10.0+cu130
- transformers: 5.10.1
- llmcompressor: 0.12.0
- vllm: 0.24.0
Backbone Model
Using Dataset
tatsu-lab/alpaca convert as chat template.
Compare
| -- | origin model(BF16) | Quantization(This Model) |
|---|---|---|
| Leaderboard_GPQA | 0.2424 | 0.2525 |
| VRAM Usage | 2.21 GiB | 1.41 GiB |
- Downloads last month
- -
Model tree for AlphaOxO/GLM-OCR-NVFP4
Base model
zai-org/GLM-OCR