HuggingFaceTB

/

SmolVLM2-2.2B-Instruct

like 317

Follow

Hugging Face Smol Models Research 3.66k

(smolvlm) _  ocr_test python smolvlm.py
You have video processor config saved in `preprocessor.json` file which is deprecated. Video processor configs should be saved in their own `video_preprocessor.json` file. You can rename the file or load and save the processor back which renames it automatically. Loading from `preprocessor.json` will be removed in v5.0.
Loading checkpoint shards: 100%|____________________________________________________________________________________________________________________________________________________________________________________________________________________________| 2/2 [00:01<00:00,  1.75it/s]
cuda:0
Traceback (most recent call last):
  File "/mnt/internalSSD-3/2025/Work/new/ocr_test/smolvlm.py", line 31, in <module>
    generated_ids = model.generate(**inputs, do_sample=False, max_new_tokens=64)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/transformers/generation/utils.py", line 2634, in generate
    result = self._sample(
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/transformers/generation/utils.py", line 3615, in _sample
    outputs = self(**model_inputs, return_dict=True)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/transformers/utils/generic.py", line 959, in wrapper
    output = func(self, *args, **kwargs)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/transformers/models/smolvlm/modeling_smolvlm.py", line 950, in forward
    outputs = self.model(
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/transformers/utils/generic.py", line 959, in wrapper
    output = func(self, *args, **kwargs)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/transformers/models/smolvlm/modeling_smolvlm.py", line 763, in forward
    image_hidden_states = self.get_image_features(pixel_values, pixel_attention_mask).to(inputs_embeds.device)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/transformers/models/smolvlm/modeling_smolvlm.py", line 689, in get_image_features
    image_hidden_states = self.vision_model(pixel_values=pixel_values, patch_attention_mask=patch_attention_mask)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/transformers/models/smolvlm/modeling_smolvlm.py", line 445, in forward
    hidden_states = self.embeddings(pixel_values=pixel_values, patch_attention_mask=patch_attention_mask)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/new/anaconda3/envs/smolvlm/lib/python3.10/site-packages/transformers/models/smolvlm/modeling_smolvlm.py", line 151, in forward
    bucket_coords_h = torch.bucketize(fractional_coords_h, boundaries, right=True)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument boundaries in method wrapper_CUDA_Tensor_bucketize)

Getting this error when trying to tun this

from transformers import AutoProcessor, AutoModelForImageTextToText
import torch

model_path = "HuggingFaceTB/SmolVLM2-2.2B-Instruct"
processor = AutoProcessor.from_pretrained(model_path)
model = AutoModelForImageTextToText.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    _attn_implementation="flash_attention_2"
).to("cuda")

print(model.device)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg"},
            {"type": "text", "text": "Can you describe this image?"},
        ]
    },
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device, dtype=torch.bfloat16)

generated_ids = model.generate(**inputs, do_sample=False, max_new_tokens=64)
generated_texts = processor.batch_decode(
    generated_ids,
    skip_special_tokens=True,
)
print(generated_texts[0])

Any solution? Im loading both model and input on gpu only