Anyone else having trouble loading the model?
#7
by darknoon - opened
Using more or less this code (transformers@caa0ff0bf104ee68ea46432b298a534e9a4a02a4):
from transformers import FuyuForCausalLM, AutoTokenizer
…
FuyuForCausalLM.from_pretrained("adept/fuyu-8b")
The checkpoint seems to download ok, but doesn't load:
Some weights of FuyuForCausalLM were not initialized from the model checkpoint at adept/fuyu-8b and are newly initialized: ['language_model.model.layers.27.self_attn.query_key_value.bias', 'language_model.model.layers.20.input_layernorm.weight', 'language_model.model.layers.10.self_attn.k_layernorm.weight', 'language_model.model.layers.21.mlp.dense_h_to_4h.weight', 'language_model.model.layers.3.input_layernorm.weight', 'language_model.model.layers.6.self_attn.dense.bias', 'language_model.model.layers.2.self_attn.query_key_value.weight', 'language_model.model.layers.11.self_attn.query_key_value.weight', 'language_model.model.layers.30.input_layernorm.weight', 'language_model.model.layers.22.mlp.dense_4h_to_h.bias', 'language_model.model.layers.16.post_attention_layernorm.bias'…
The PR has been merged and the weights updated in the repo, so you should be able to load the model using transformers @ main.
darknoon changed discussion status to closed