opt-350m - GPTQ
Source model: facebook/opt-350m
This model was quantized to 8-bit using GPTQModel. Quantization parameters:
- bits: 8
- group_size: 128
- damp_percent: 0.05
- desc_act: False
Usage
# pip install transformers gptqmodel --no-build-isolation
from gptqmodel import GPTQModel
model_id = "iproskurina/opt-350m-gptqmodel-8bit"
model = GPTQModel.load(model_id)
result = model.generate("Uncovering deep insights")[0]
print(model.tokenizer.decode(result))
- Downloads last month
- 5