Error : Can't find 'blip2_t5_instruct' in lavis model registry.
AttributeError Traceback (most recent call last)
in <cell line: 1>()
----> 1 vlm = load_model(
2 name='blip2_t5_instruct',
3 model_type='flant5xxl',
4 checkpoint='/content/drive/MyDrive/temp/pgvlm_weights.bin', # replace with location of downloaded weights
5 is_eval=True,
/usr/local/lib/python3.10/dist-packages/lavis/models/init.py in load_model(name, model_type, is_eval, device, checkpoint)
105 """
106
--> 107 model = registry.get_model_class(name).from_pretrained(model_type=model_type)
108
109 if checkpoint is not None:
AttributeError: 'NoneType' object has no attribute 'from_pretrained'
Hello, I am also experiencing this problem, please do you have a solution
This is because blip2_t5_instruct is currently not included in LAVIS so there's no way to use it. You could try other models and try to fine-tune using the dataset provided:)
When using the git version of LAVIS at least, the model is still included. The issue with loading the model for me was caused by the base library_root path not being set, which is needed to load flant5xxxl. To fix this you need to import lavis before doing the other lavis imports in your script.