Instructions to use microsoft/Florence-2-large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use microsoft/Florence-2-large with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="microsoft/Florence-2-large", trust_remote_code=True)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("microsoft/Florence-2-large", trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained("microsoft/Florence-2-large", trust_remote_code=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use microsoft/Florence-2-large with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "microsoft/Florence-2-large"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/microsoft/Florence-2-large

SGLang

How to use microsoft/Florence-2-large with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "microsoft/Florence-2-large" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "microsoft/Florence-2-large" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use microsoft/Florence-2-large with Docker Model Runner:
```
docker model run hf.co/microsoft/Florence-2-large
```

Wrong bounding boxes

#13

by aleablu - opened Jun 21, 2024

Discussion

aleablu

Jun 21, 2024

Hi all, thank you team for the amazing work. Howewer I noticed that if I run the Object Detection task on the demo space of florence-2-large I'll get correct results, but if I download the sample_inference notebook and execute it on the same image as the online demo I'll get totally different results, with wrong bounding boxes that do not relate at all with the ones produced by the online demo, as an example:
TASK:

output from hf demo: {'bboxes': [[53.54999923706055, 0.675000011920929, 824.8499755859375, 1347.9749755859375], [61.64999771118164, 108.67500305175781, 815.8499755859375, 1110.375], [189.4499969482422, 1042.875, 709.6499633789062, 1347.9749755859375]], 'labels': ['man', 'shirt', 'trousers']}
output from sample_inference notebook run locally on my machine (it has an A40 gpu): {'bboxes': [[763.6499633789062, 922.7250366210938, 50.849998474121094, 1222.425048828125], [654.75, 1100.925048828125, 229.9499969482422, 276.07501220703125], [535.9500122070312, 945.6749877929688, 563.8499755859375, 1222.425048828125]], 'labels': ['man', 'shirt', 'trousers']}

The image is the same, it happens also with the sample image of the car provided in the notebook. How is it possible?!

I do not modify the notebook's code at all.

HMisHere

Jul 8, 2024

I'm getting the same problem bounding boxes are seriously wrong.. even with simple images and sample code

haipingwu

Jul 9, 2024

hi, can you check if your transformers and torch version is the consistent with demo?

abdkhan

Jul 29, 2024

The bounding boxes for VLLM seem to be quite cryptic. I got a hint: Try to use the supervision library, the bounding boxes are scaled concerning the LLM, and you need to rescale them back.

I solved the issue with PaliGema, and still trying to solve the problem with Florence-2.

aimagee

Aug 4, 2024

hi, can you check if your transformers and torch version is the consistent with demo?

@haipingwu so which version of torch and transformers are recommended?

felixoben

Dec 19, 2024

Are there a recommended lib versions?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment