RedHatAI
/

Qwen2-7B-Instruct-quantized.w8a16

Text Generation

text-generation-inference

8-bit precision

Model card Files Files and versions

alexmarques commited on Jul 15, 2024

Commit

eeb5139

·

verified ·

1 Parent(s): 4fe9517

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -150,7 +150,7 @@ model = AutoGPTQForCausalLM.from_pretrained(
 )
 model.quantize(examples)
-model.save_pretrained("Meta-Llama-3-8B-Instruct-quantized.w8a16")
 ```

 )
 model.quantize(examples)
+model.save_pretrained("Qwen2-7B-Instruct-quantized.w8a16")
 ```