Trouble with running the model
I tried to run the qwen 2 vl notebook with the change of the model name to unsloth/Qwen2.5-VL-7B-Instruct, and it does not load the model.
%%capture
import os
if "COLAB_" not in "".join(os.environ.keys()):
!pip install unsloth
else:
# Do this only in Colab and Kaggle notebooks! Otherwise use pip install unsloth
!pip install git+https://github.com/huggingface/transformers accelerate
!pip install qwen-vl-utils[decord]==0.0.8
!pip install --no-deps bitsandbytes xformers peft trl triton
!pip install --no-deps cut_cross_entropy unsloth_zoo
!pip install sentencepiece protobuf datasets huggingface_hub hf_transfer
!pip install --no-deps unsloth
from unsloth import FastVisionModel # FastLanguageModel for LLMs
import torch
model, tokenizer = FastVisionModel.from_pretrained(
"unsloth/Qwen2.5-VL-7B-Instruct",
load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA.
use_gradient_checkpointing = "unsloth", # True or "unsloth" for long context
)
#ERROR MESSAGE
RuntimeError Traceback (most recent call last)
in <cell line: 0>()
20 ] # More models at https://huggingface.co/unsloth
21
---> 22 model, tokenizer = FastVisionModel.from_pretrained(
23 "unsloth/Qwen2.5-VL-7B-Instruct",
24 load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA.
8 frames
/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict, assign)
2582
2583 if len(error_msgs) > 0:
-> 2584 raise RuntimeError(
2585 "Error(s) in loading state_dict for {}:\n\t{}".format(
2586 self.class.name, "\n\t".join(error_msgs)
RuntimeError: Error(s) in loading state_dict for Linear4bit:
size mismatch for weight: copying a param with shape torch.Size([33947648, 1]) from checkpoint, the shape in current model is torch.Size([3584, 18944]).
Is this via colab? does Qwen2 VL work?
Yes im using collab. Qwen 2 VL works just fine.
Yes im using collab. Qwen 2 VL works just fine.
@ShaunShuster try this, works for me
# Cell 1
%%capture
import os
if "COLAB_" not in "".join(os.environ.keys()):
!pip install unsloth
else:
# Do this only in Colab and Kaggle notebooks! Otherwise use pip install unsloth
!pip install --no-deps bitsandbytes accelerate xformers==0.0.29 peft trl triton
!pip install --no-deps cut_cross_entropy unsloth_zoo
!pip install sentencepiece protobuf datasets huggingface_hub hf_transfer
!pip install --no-deps unsloth
!pip install --upgrade transformers
# Cell 2
from unsloth import FastVisionModel # FastLanguageModel for LLMs
import torch
model, tokenizer = FastVisionModel.from_pretrained(
"unsloth/Qwen2.5-VL-7B-Instruct",
load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA.
use_gradient_checkpointing = "unsloth", # True or "unsloth" for long context
)
if it does not work on first run try Restart session after cell 1 and then run cell 2
i am getting Image features and image tokens do not match