W4A16 GPTQ quantized version of TheDrummer/Behemoth-X-123B-v2.1
Using intel/auto-round version: git+1d91207
Generation command-line
auto-round --model behemoth-x-123b-v2.1 --scheme "W4A16" --format "auto_gptq" --enable_torch_compile
- Downloads last month
- 7
Model tree for hell0ks/Behemoth-X-123B-v2.1-AutoRound-GPTQ-4bit
Base model
TheDrummer/Behemoth-X-123B-v2.1