Instructions to use HuggingFaceM4/idefics-80b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HuggingFaceM4/idefics-80b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HuggingFaceM4/idefics-80b")

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics-80b")
model = AutoModelForMultimodalLM.from_pretrained("HuggingFaceM4/idefics-80b")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use HuggingFaceM4/idefics-80b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HuggingFaceM4/idefics-80b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/HuggingFaceM4/idefics-80b

SGLang

How to use HuggingFaceM4/idefics-80b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HuggingFaceM4/idefics-80b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HuggingFaceM4/idefics-80b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use HuggingFaceM4/idefics-80b with Docker Model Runner:
```
docker model run hf.co/HuggingFaceM4/idefics-80b
```

Leyo commited on Aug 4, 2023

Commit

8ded392

1 Parent(s): 7e556e9

switch Base vs Instruct

Browse files

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -319,14 +319,14 @@ Fairness Evaluations:
 IDEFICS vs IDEFICS-instruct.
 | Model | Shots | <nobr>VQAv2<br>OE VQA acc.</nobr> | <nobr>OKVQA<br>OE VQA acc.</nobr> | <nobr>TextVQA<br>OE VQA acc.</nobr> | <nobr>VizWiz<br>OE VQA acc.</nobr> | <nobr>TextCaps<br>CIDEr</nobr> | <nobr>Coco<br>CIDEr</nobr> | <nobr>NoCaps<br>CIDEr</nobr> | <nobr>Flickr<br>CIDEr</nobr> | <nobr>VisDial<br>NDCG</nobr> | <nobr>HatefulMemes<br>ROC AUC</nobr> | <nobr>ScienceQA<br>acc.</nobr> | <nobr>RenderedSST2<br>acc.</nobr> | <nobr>Winoground<br>group (text/image)</nobr> |
 |:----------------------------------------|:--------|---------------------:|---------------------:|-----------------------:|----------------------:|-------------------:|---------------:|-----------------:|-----------------:|-----------------:|-------------------------:|-----------------------:|--------------------------:|----------------------------------:|
-| Difference IDEFICS 80B Base vs Instruct | 0       |                -22.7 |                 -8.2 |                    1.9 |                  -9.8 |               19.7 |           25.4 |             39.5 |             11.7 |              0.4 |                     -1.7 |                    0.5 |                       6.8 |                               1.2 |
 |                                         | 4       |                  4.0 |                  1.7 |                    3.5 |                  -0.7 |               - |            6.6 |              4.4 |             -6.6 |              0.5 |                     -0.3 |                    1.6 |                      -1.1 |                              - |
 |                                         | 8       |                  3.4 |                  1.8 |                    2.5 |                  -1.3 |               -4.9 |            2.5 |             -0.9 |             -5.9 |              0.3 |                     -0.2 |                   - |                       0.8 |                              - |
 |                                         | 16      |                  3.2 |                  1.4 |                    2.8 |                   0.4 |               -4.5 |            4.0 |              0.4 |             -4.1 |             - |                      0.7 |                   - |                       2.4 |                              - |
 |                                         | 32      |                  2.9 |                  1.8 |                    2.6 |                   1.2 |               -3.0 |            6.5 |              1.0 |             -2.7 |             - |                      2.4 |                   - |                       3.2 |                              - |
 | Average Difference 80B                  |         |                 -1.8 |                 -0.3 |                    2.6 |                  -2.0 |                1.3 |            9.0 |              8.9 |             -1.5 |              0.4 |                      0.2 |                    1.1 |                       2.4 |                               1.2 |
 <br>
-| Difference IDEFICS 9B Base vs Instruct  | 0       |                 15.0 |                  7.6 |                    3.3 |                   5.6 |               41.7 |           83.0 |             64.3 |             44.6 |              0.5 |                      1.8 |                   16.4 |                       1.0 |                               0.8 |
 |                                         | 4       |                 10.8 |                  3.3 |                    3.4 |                   2.1 |                8.2 |           35.1 |             19.6 |             15.0 |              1.0 |                      1.1 |                   16.4 |                      -1.8 |                              - |
 |                                         | 8       |                 10.2 |                  3.1 |                    3.5 |                   1.6 |                6.7 |           31.8 |             14.8 |             13.6 |              0.6 |                      0.6 |                   - |                      -4.9 |                              - |
 |                                         | 16      |                  9.8 |                  3.3 |                    3.7 |                   2.3 |                2.7 |           29.1 |             12.2 |             11.4 |             - |                      0.7 |                   - |                      -4.6 |                              - |

 IDEFICS vs IDEFICS-instruct.
 | Model | Shots | <nobr>VQAv2<br>OE VQA acc.</nobr> | <nobr>OKVQA<br>OE VQA acc.</nobr> | <nobr>TextVQA<br>OE VQA acc.</nobr> | <nobr>VizWiz<br>OE VQA acc.</nobr> | <nobr>TextCaps<br>CIDEr</nobr> | <nobr>Coco<br>CIDEr</nobr> | <nobr>NoCaps<br>CIDEr</nobr> | <nobr>Flickr<br>CIDEr</nobr> | <nobr>VisDial<br>NDCG</nobr> | <nobr>HatefulMemes<br>ROC AUC</nobr> | <nobr>ScienceQA<br>acc.</nobr> | <nobr>RenderedSST2<br>acc.</nobr> | <nobr>Winoground<br>group (text/image)</nobr> |
 |:----------------------------------------|:--------|---------------------:|---------------------:|-----------------------:|----------------------:|-------------------:|---------------:|-----------------:|-----------------:|-----------------:|-------------------------:|-----------------------:|--------------------------:|----------------------------------:|
+| Difference IDEFICS 80B Instruct vs Base | 0       |                -22.7 |                 -8.2 |                    1.9 |                  -9.8 |               19.7 |           25.4 |             39.5 |             11.7 |              0.4 |                     -1.7 |                    0.5 |                       6.8 |                               1.2 |
 |                                         | 4       |                  4.0 |                  1.7 |                    3.5 |                  -0.7 |               - |            6.6 |              4.4 |             -6.6 |              0.5 |                     -0.3 |                    1.6 |                      -1.1 |                              - |
 |                                         | 8       |                  3.4 |                  1.8 |                    2.5 |                  -1.3 |               -4.9 |            2.5 |             -0.9 |             -5.9 |              0.3 |                     -0.2 |                   - |                       0.8 |                              - |
 |                                         | 16      |                  3.2 |                  1.4 |                    2.8 |                   0.4 |               -4.5 |            4.0 |              0.4 |             -4.1 |             - |                      0.7 |                   - |                       2.4 |                              - |
 |                                         | 32      |                  2.9 |                  1.8 |                    2.6 |                   1.2 |               -3.0 |            6.5 |              1.0 |             -2.7 |             - |                      2.4 |                   - |                       3.2 |                              - |
 | Average Difference 80B                  |         |                 -1.8 |                 -0.3 |                    2.6 |                  -2.0 |                1.3 |            9.0 |              8.9 |             -1.5 |              0.4 |                      0.2 |                    1.1 |                       2.4 |                               1.2 |
 <br>
+| Difference IDEFICS 9B Instruct vs Base  | 0       |                 15.0 |                  7.6 |                    3.3 |                   5.6 |               41.7 |           83.0 |             64.3 |             44.6 |              0.5 |                      1.8 |                   16.4 |                       1.0 |                               0.8 |
 |                                         | 4       |                 10.8 |                  3.3 |                    3.4 |                   2.1 |                8.2 |           35.1 |             19.6 |             15.0 |              1.0 |                      1.1 |                   16.4 |                      -1.8 |                              - |
 |                                         | 8       |                 10.2 |                  3.1 |                    3.5 |                   1.6 |                6.7 |           31.8 |             14.8 |             13.6 |              0.6 |                      0.6 |                   - |                      -4.9 |                              - |
 |                                         | 16      |                  9.8 |                  3.3 |                    3.7 |                   2.3 |                2.7 |           29.1 |             12.2 |             11.4 |             - |                      0.7 |                   - |                      -4.6 |                              - |