ALIA-40b-AutoRound-INT4_3b / quantization_config.json
agustim's picture
Upload folder using huggingface_hub
1209292 verified
{
"bits": 3,
"data_type": "int",
"group_size": 128,
"sym": false,
"iters": 300,
"seqlen": 256,
"autoround_version": "0.10.2",
"quant_method": "auto-round",
"packing_format": "auto_round"
}