Instructions to use HuggingFaceM4/idefics2-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HuggingFaceM4/idefics2-8b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="HuggingFaceM4/idefics2-8b")

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics2-8b")
model = AutoModelForMultimodalLM.from_pretrained("HuggingFaceM4/idefics2-8b")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use HuggingFaceM4/idefics2-8b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HuggingFaceM4/idefics2-8b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics2-8b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/HuggingFaceM4/idefics2-8b

SGLang

How to use HuggingFaceM4/idefics2-8b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HuggingFaceM4/idefics2-8b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics2-8b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HuggingFaceM4/idefics2-8b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics2-8b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use HuggingFaceM4/idefics2-8b with Docker Model Runner:
```
docker model run hf.co/HuggingFaceM4/idefics2-8b
```

load_image(URL) is invalid

#17

by micaelcarvalho - opened Apr 17, 2024

Discussion

micaelcarvalho

Apr 17, 2024

The "How to Get Started" instructions starts with an invalid call to load_image: https://huggingface.co/HuggingFaceM4/idefics2-8b
This results in PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO ...>
The instructions are likely incorrect/outdated and need to be fixed

VictorSanh

Apr 18, 2024

hi @micaelcarvalho ,
would you be able to share your full traceback? i am hitting no error with these image loads

tctrautman

Apr 20, 2024

I'm seeing the same thing, though I admittedly don't use Python all that much, so I might be doing something silly. Here's my traceback:

Traceback (most recent call last):
  File "[path-to-directory]/idefics2-test/test.py", line 12, in <module>
    image1 = load_image("https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg")
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/transformers/image_utils.py", line 314, in load_image
    image = PIL.Image.open(requests.get(image, stream=True, timeout=timeout).raw)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/PIL/Image.py", line 3280, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x12501b880>

tctrautman

Apr 20, 2024

My issue seems to be fixed by using different image URLs -- I'm guessing some sort of bot detection has been triggered on britannica.com

VictorSanh

Apr 20, 2024

yes, i suspect indeed that the download failed because of rate limiting (or something of that sort). would need to see the result of requests.get(image, stream=True, timeout=timeout). i am glad that different URLs work.

tctrautman

Apr 20, 2024

Yeah, it seems to be an intermittent issue on my end. I just tried to print the response with the images that were failing, but the requests are succeeding for me now. Thanks for following up here!

VictorSanh

Apr 22, 2024

i'll close this discussion, feel free to re-open if needed!
glad you found a way to move forward!

VictorSanh changed discussion status to closed Apr 22, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment