Instructions to use microsoft/Florence-2-large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use microsoft/Florence-2-large with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="microsoft/Florence-2-large", trust_remote_code=True)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("microsoft/Florence-2-large", trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained("microsoft/Florence-2-large", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use microsoft/Florence-2-large with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "microsoft/Florence-2-large"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/microsoft/Florence-2-large

SGLang

How to use microsoft/Florence-2-large with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "microsoft/Florence-2-large" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "microsoft/Florence-2-large" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Florence-2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use microsoft/Florence-2-large with Docker Model Runner:
```
docker model run hf.co/microsoft/Florence-2-large
```

masterofaudio2077 commited on 14 days ago

Commit

5e92a22

verified ·

1 Parent(s): 67061fd

Fix silently-skipped resize/normalize from None-valued processor kwargs

Browse files

Florence2Processor.__call__ forwards do_resize/do_normalize/image_mean/image_std/resample/do_convert_rgb to the image processor unconditionally, defaulting to None when the caller doesn't override them. This transformers version's image processor treats an explicit None as "disable this feature", not "use the configured default" -- confirmed by direct testing: calling the bare CLIPImageProcessor with do_resize=None produces unresized 512x512 output (matching the raw input image) instead of the configured 768x768, and with do_normalize=None the output stays in raw [0,1] instead of the normalized [-2.12, 2.64] range, with no error in either case.

Only forward kwargs the caller actually set, so the image processor's own configured defaults apply otherwise. Verified live: resize now produces the correct 768x768 output for a 512x512 input image.

Files changed (1) hide show

processing_florence2.py +19 -7

processing_florence2.py CHANGED Viewed

@@ -251,17 +251,29 @@ class Florence2Processor(ProcessorMixin):
         elif isinstance(text, list) and _is_str_or_image(text[0]):
             pass
         pixel_values = self.image_processor(
             images,
-            do_resize=do_resize,
-            do_normalize=do_normalize,
             return_tensors=return_tensors,
-            image_mean=image_mean,
-            image_std=image_std,
-            input_data_format=input_data_format,
             data_format=data_format,
-            resample=resample,
-            do_convert_rgb=do_convert_rgb,
         )["pixel_values"]
         if max_length is not None:

         elif isinstance(text, list) and _is_str_or_image(text[0]):
             pass
+        # None-valued kwargs must not be forwarded: this image processor treats an
+        # explicit None as "disable this feature" rather than "use the configured
+        # default", so passing do_resize=None (the default here when the caller
+        # doesn't override it) silently skips resizing -- and do_normalize=None
+        # silently skips normalization, with no error either way.
+        image_processor_overrides = {
+            k: v
+            for k, v in {
+                "do_resize": do_resize,
+                "do_normalize": do_normalize,
+                "image_mean": image_mean,
+                "image_std": image_std,
+                "input_data_format": input_data_format,
+                "resample": resample,
+                "do_convert_rgb": do_convert_rgb,
+            }.items()
+            if v is not None
+        }
         pixel_values = self.image_processor(
             images,
             return_tensors=return_tensors,
             data_format=data_format,
+            **image_processor_overrides,
         )["pixel_values"]
         if max_length is not None: