INT4 quantization of HuggingFaceTB/SmolVLM-Instruct (LLM backbone quantized, vision encoder fp16)

113fe69 verified 2 months ago

404 Bytes

	{
	"base_model": "HuggingFaceTB/SmolVLM-Instruct",
	"quantization": "int4_per_group_symmetric",
	"group_size": 128,
	"bits": 4,
	"method": "static_int4_dequantized",
	"description": "INT4 per-group symmetric quantization of LLM backbone weights. Vision encoder kept in fp16. Weights stored as dequantized fp16 for maximum compatibility.",
	"quantized_layers": 170,
	"skipped_vision_layers": 162
	}