add 8-bit quantization 05943ab
booksouls commited on
How to use booksouls/bart-large-cnn with Transformers:
# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("booksouls/bart-large-cnn")
model = AutoModelForSeq2SeqLM.from_pretrained("booksouls/bart-large-cnn", device_map="auto")