bigscience

/

bloomz

like 512

Follow

BigScience Workshop 1.32k

ur trying to load mt0 with the bloom model; u need to load it w/ the mt0 model (i.e. t5 i think) - the script is in its modelcard

@Muennighoff can you please share the code snippet how to do that?

From https://huggingface.co/bigscience/mt0-small

# pip install -q transformers accelerate
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

checkpoint = "bigscience/mt0-small"

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint, torch_dtype="auto", device_map="auto")

inputs = tokenizer.encode("Translate to English: Je t’aime.", return_tensors="pt").to("cuda")
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))

@Muennighoff Thanks!
It's working now.