Error on basic tutorial script provided
#22
by diegoAgher - opened
I am using the following code, provided in this space, to try to load falcon-40b-instruct on google colab:
import transformers
import torch
model = "tiiuae/falcon-40b-instruct"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
torch_dtype=torch.bfloat16,
trust_remote_code=True,
device_map="auto",
)
sequences = pipeline(
"Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\nDaniel: Hello, Girafatron!\nGirafatron:",
max_length=200,
do_sample=True,
top_k=10,
num_return_sequences=1,
eos_token_id=tokenizer.eos_token_id,
)
for seq in sequences:
print(f"Result: {seq['generated_text']}")```
but get the following error:
ValueError: Could not load model tiiuae/falcon-40b-instruct with any of the following classes: (<class
'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class
'transformers.models.auto.modeling_tf_auto.TFAutoModelForCausalLM'>).
Previously I was also getting this error:
NameError: name 'init_empty_weights' is not defined
(something must have changed when restarting the kernel)
On Colab, I have the following versions:
torch 2.0.1+cu118
transformers 4.29.2
Try to install accelerate, it does contain this function - from accelerate import init_empty_weights
pip install accelerate
I did install these :
!pip install transformers
!pip install einops
!pip install accelerate
