StanfordAIMI

/

CheXagent-8b

like 46

Follow

Stanford AIMI 183

I am trying to run the model with multiple GPUs. But, I am facing some errors, warnings and unexpected model behaviour. How could I use the model with multiple GPUs correctly? Also the following code is given under the section "Get started", is not working as it should. As, I am running the code I am getting the following error.

CODE:

import io

import requests
import torch
from PIL import Image
from transformers import AutoModelForCausalLM, AutoProcessor, GenerationConfig

# step 1: Setup constant
device = "cuda"
dtype = torch.float16

# step 2: Load Processor and Model
processor = AutoProcessor.from_pretrained("StanfordAIMI/CheXagent-8b", trust_remote_code=True)
generation_config = GenerationConfig.from_pretrained("StanfordAIMI/CheXagent-8b")
model = AutoModelForCausalLM.from_pretrained("StanfordAIMI/CheXagent-8b", torch_dtype=dtype, trust_remote_code=True)

# step 3: Fetch the images
image_path = "https://upload.wikimedia.org/wikipedia/commons/3/3b/Pleural_effusion-Metastatic_breast_carcinoma_Case_166_%285477628658%29.jpg"
images = [Image.open(io.BytesIO(requests.get(image_path).content)).convert("RGB")]

# step 4: Generate the Findings section
prompt = f'Describe "Airway"'
inputs = processor(images=images, text=f" USER: <s>{prompt} ASSISTANT: <s>", return_tensors="pt").to(device=device, dtype=dtype)
output = model.generate(**inputs, generation_config=generation_config)[0]
response = processor.tokenizer.decode(output, skip_special_tokens=True)

ERROR:

UnidentifiedImageError                    Traceback (most recent call last)
Cell In[1], line 19
     17 # step 3: Fetch the images
     18 image_path = "https://upload.wikimedia.org/wikipedia/commons/3/3b/Pleural_effusion-Metastatic_breast_carcinoma_Case_166_%285477628658%29.jpg"
---> 19 images = [Image.open(io.BytesIO(requests.get(image_path).content)).convert("RGB")]
     21 # step 4: Generate the Findings section
     22 prompt = f'Describe "Airway"'

File ~/miniconda3/envs/test/lib/python3.11/site-packages/PIL/Image.py:3579, in open(fp, mode, formats)
   3577     warnings.warn(message)
   3578 msg = "cannot identify image file %r" % (filename if filename else fp)
-> 3579 raise UnidentifiedImageError(msg)

UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7e98527d1f80>

THE SOLUTION:
The following code was changed.
ORIGINAL :
images = [Image.open(io.BytesIO(requests.get(image_path).content)).convert("RGB")]
CHANGED :
images = [Image.open(io.BytesIO(requests.get(image_path, headers={"User-Agent": "Mozilla/5.0"}).content)).convert("RGB")]