Instructions to use HuggingFaceM4/idefics-80b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HuggingFaceM4/idefics-80b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HuggingFaceM4/idefics-80b")

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics-80b")
model = AutoModelForMultimodalLM.from_pretrained("HuggingFaceM4/idefics-80b")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use HuggingFaceM4/idefics-80b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HuggingFaceM4/idefics-80b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/HuggingFaceM4/idefics-80b

SGLang

How to use HuggingFaceM4/idefics-80b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HuggingFaceM4/idefics-80b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HuggingFaceM4/idefics-80b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use HuggingFaceM4/idefics-80b with Docker Model Runner:
```
docker model run hf.co/HuggingFaceM4/idefics-80b
```

Leyo commited on Aug 4, 2023

Commit

455a3a6

1 Parent(s): 89c2eea

Add fairness evals for Idefics instruct

Browse files

Files changed (1) hide show

README.md +14 -0

README.md CHANGED Viewed

@@ -301,6 +301,20 @@ Idefics Instruct Evaluations:
 |                      |      16 |                 66.8 |                 51.7 |                   31.6 |                  44.8 |               70.2 |          128.8 |            101.5 |             75.8 |             - |                     51.7 |                   - |                      63.3 |                              - |
 |                      |      32 |                 66.9 |                 52.3 |                   32.0 |                  46.0 |               71.7 |          127.8 |            101.0 |             76.3 |             - |                     50.8 |                   - |                      60.9 |                              - |
 IDEFICS vs IDEFICS-instruct.
 | Model | Shots | <nobr>VQAv2<br>OE VQA acc.</nobr> | <nobr>OKVQA<br>OE VQA acc.</nobr> | <nobr>TextVQA<br>OE VQA acc.</nobr> | <nobr>VizWiz<br>OE VQA acc.</nobr> | <nobr>TextCaps<br>CIDEr</nobr> | <nobr>Coco<br>CIDEr</nobr> | <nobr>NoCaps<br>CIDEr</nobr> | <nobr>Flickr<br>CIDEr</nobr> | <nobr>VisDial<br>NDCG</nobr> | <nobr>HatefulMemes<br>ROC AUC</nobr> | <nobr>ScienceQA<br>acc.</nobr> | <nobr>RenderedSST2<br>acc.</nobr> | <nobr>Winoground<br>group (text/image)</nobr> |
 |:----------------------------------------|:--------|---------------------:|---------------------:|-----------------------:|----------------------:|-------------------:|---------------:|-----------------:|-----------------:|-----------------:|-------------------------:|-----------------------:|--------------------------:|----------------------------------:|

 |                      |      16 |                 66.8 |                 51.7 |                   31.6 |                  44.8 |               70.2 |          128.8 |            101.5 |             75.8 |             - |                     51.7 |                   - |                      63.3 |                              - |
 |                      |      32 |                 66.9 |                 52.3 |                   32.0 |                  46.0 |               71.7 |          127.8 |            101.0 |             76.3 |             - |                     50.8 |                   - |                      60.9 |                              - |
+Fairness Evaluations:
+| Model | Shots | <nobr>FairFaceGender<br>acc.</nobr> | <nobr>FairFaceRace<br>acc.</nobr> | <nobr>FairFaceAge<br>acc.</nobr> |
+|:---------------------|--------:|----------------------------:|--------------------------:|-------------------------:|
+| 80B IDEFICS Instruct |       0 |                        95.7 |                      63.4 |                     47.1 |
+|                      |       4 |                        95.6 |                      51.4 |                     48.3 |
+|                      |       8 |                        95.8 |                      51.0 |                     51.1 |
+|                      |      16 |                        96.1 |                      47.6 |                     51.8 |
+|                      |      32 |                        96.2 |                      36.8 |                     50.3 |
+| 9B IDEFICS Instruct  |       0 |                        92.7 |                      59.6 |                     43.9 |
+|                      |       4 |                        95.2 |                      43.3 |                     38.7 |
+|                      |       8 |                        95.8 |                      51.7 |                     40.1 |
+|                      |      16 |                        96.1 |                      58.9 |                     41.7 |
+|                      |      32 |                        96.1 |                      59.7 |                     44.8 |
 IDEFICS vs IDEFICS-instruct.
 | Model | Shots | <nobr>VQAv2<br>OE VQA acc.</nobr> | <nobr>OKVQA<br>OE VQA acc.</nobr> | <nobr>TextVQA<br>OE VQA acc.</nobr> | <nobr>VizWiz<br>OE VQA acc.</nobr> | <nobr>TextCaps<br>CIDEr</nobr> | <nobr>Coco<br>CIDEr</nobr> | <nobr>NoCaps<br>CIDEr</nobr> | <nobr>Flickr<br>CIDEr</nobr> | <nobr>VisDial<br>NDCG</nobr> | <nobr>HatefulMemes<br>ROC AUC</nobr> | <nobr>ScienceQA<br>acc.</nobr> | <nobr>RenderedSST2<br>acc.</nobr> | <nobr>Winoground<br>group (text/image)</nobr> |
 |:----------------------------------------|:--------|---------------------:|---------------------:|-----------------------:|----------------------:|-------------------:|---------------:|-----------------:|-----------------:|-----------------:|-------------------------:|-----------------------:|--------------------------:|----------------------------------:|