Instructions to use HuggingFaceM4/idefics-80b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HuggingFaceM4/idefics-80b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HuggingFaceM4/idefics-80b")

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics-80b")
model = AutoModelForImageTextToText.from_pretrained("HuggingFaceM4/idefics-80b")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use HuggingFaceM4/idefics-80b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HuggingFaceM4/idefics-80b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/HuggingFaceM4/idefics-80b

SGLang

How to use HuggingFaceM4/idefics-80b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HuggingFaceM4/idefics-80b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HuggingFaceM4/idefics-80b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use HuggingFaceM4/idefics-80b with Docker Model Runner:
```
docker model run hf.co/HuggingFaceM4/idefics-80b
```

VictorSanh commited on Aug 3, 2023

Commit

96b2b8c

1 Parent(s): 50966bf

image net a bit of formatting

Browse files

Files changed (1) hide show

README.md +7 -8

README.md CHANGED Viewed

@@ -176,16 +176,15 @@ We note that since IDEFICS was trained on PMD (which contains COCO), the evaluat
 |            |      16 |                 57.0 |                 48.4 |                   27.9 |                  42.6 |               67.4 |           99.7 |             89.4 |             64.5 |             - |                     50.9 |                   - |                      67.8 |                              - |
 |            |      32 |                 57.9 |                 49.6 |                   28.3 |                  43.7 |               68.1 |           98.0 |             90.5 |             64.4 |             - |                     49.8 |                   - |                      67.0 |                              - |
-For ImageNet-1k, we also report results where the priming samples are selected to be similar (i.e. close in a vector space) to the queried instance.
-ImageNet-1k Evaluation:
-| Model      |   Shots |   ImageNet-1k |
-|:-----------|--------:|-----------:|
-| IDEFIX 80B |      16, 1k support set |       65.4 |
-|            |      16, RICES 5k support set |       72.9 |
 <br>
-| IDEFIX 9B  |      16, 1k support set |       53.5 |
-|            |      16, RICES 5k support set |       64.5 |
 Fairness Evaluations:
 | Model      |   Shots |   FairFaceGender (accuracy) |   FairFaceRace (accuracy) |   FairFaceAge (accuracy) |

 |            |      16 |                 57.0 |                 48.4 |                   27.9 |                  42.6 |               67.4 |           99.7 |             89.4 |             64.5 |             - |                     50.9 |                   - |                      67.8 |                              - |
 |            |      32 |                 57.9 |                 49.6 |                   28.3 |                  43.7 |               68.1 |           98.0 |             90.5 |             64.4 |             - |                     49.8 |                   - |                      67.0 |                              - |
+For ImageNet-1k, we also report results where the priming samples are selected to be similar (i.e. close in a vector space) to the queried instance. This is the Retrieval-based In-Context Example Selection (RICES in short) approach introduced by [Yang et al. (2021)](https://arxiv.org/abs/2109.05014).
+| Model      |   Shots | Support set size | Shots selection | ImageNet-1k<br>Top-1 acc. |
+|:-----------|--------:|-----------------:|:----------------|--------------------------:|
+| IDEFIX 80B |      16 | 1K               | Random          |                      65.4 |
+|            |      16 | 5K               | RICES           |                      72.9 |
 <br>
+| IDEFIX 9B  |      16 | 1K               | Random          |                      53.5 |
+|            |      16 | 5K               | RICES           |                      64.5 |
 Fairness Evaluations:
 | Model      |   Shots |   FairFaceGender (accuracy) |   FairFaceRace (accuracy) |   FairFaceAge (accuracy) |