Instructions to use OpenGVLab/InternVL3-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenGVLab/InternVL3-2B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="OpenGVLab/InternVL3-2B", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("OpenGVLab/InternVL3-2B", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use OpenGVLab/InternVL3-2B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenGVLab/InternVL3-2B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/InternVL3-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/OpenGVLab/InternVL3-2B

SGLang

How to use OpenGVLab/InternVL3-2B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenGVLab/InternVL3-2B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/InternVL3-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenGVLab/InternVL3-2B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/InternVL3-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use OpenGVLab/InternVL3-2B with Docker Model Runner:
```
docker model run hf.co/OpenGVLab/InternVL3-2B
```

Weiyun1025 commited on Apr 14, 2025

Commit

b6eca7a

verified ·

1 Parent(s): ea89a73

Upload README.md with huggingface_hub

Browse files

Files changed (1) hide show

README.md +15 -17

README.md CHANGED Viewed

@@ -33,7 +33,7 @@ We introduce InternVL3, an advanced multimodal large language model (MLLM) serie
 Compared to InternVL 2.5, InternVL3 exhibits superior multimodal perception and reasoning capabilities, while further extending its multimodal capabilities to encompass tool usage, GUI agents, industrial image analysis, 3D vision perception, and more.
 Additionally, we compare InternVL3 with  Qwen2.5 Chat models, whose corresponding pre-trained base models are employed as the initialization of the langauge component in InternVL3. Benefitting from Native Multimodal Pre-Training, the InternVL3 series achieves even better overall text performance than the Qwen2.5 series.
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/fMRWSzB8ysrafQ2XJW9WR.png)
 ## InternVL3 Family
@@ -49,7 +49,7 @@ In the following table, we provide an overview of the InternVL3 series.
 | InternVL3-38B |   [InternViT-6B-448px-V2_5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V2_5)   |             [Qwen2.5-32B](https://huggingface.co/Qwen/Qwen2.5-32B)             | [🤗 link](https://huggingface.co/OpenGVLab/InternVL3-38B) |
 | InternVL3-78B |   [InternViT-6B-448px-V2_5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V2_5)   |             [Qwen2.5-72B](https://huggingface.co/Qwen/Qwen2.5-72B)             | [🤗 link](https://huggingface.co/OpenGVLab/InternVL3-78B) |
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/9wf54ERGoiM3-QICkj3Oc.png)
 ## Model Architecture
@@ -106,39 +106,39 @@ In this work, we use the Best-of-N evaluation strategy and employ [VisualPRM-8B]
 ### Multimodal Reasoning and Mathematics
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/uVIhB9BKlirAc6zlQYI0a.png)
 ### OCR, Chart, and Document Understanding
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/ivWJi3Rx_snJTfq--_9kD.png)
 ### Multi-Image & Real-World Comprehension
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/y1OnFvJxei4dd9ZzCZ6yo.png)
 ### Comprehensive Multimodal & Hallucination Evaluation
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/XeBACd5_k_1lBbT70c6rp.png)
 ### Visual Grounding
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/LFBrfb6amBxL_WKOqs5gr.png)
 ### Multimodal Multilingual Understanding
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/1wc3dA3KwIeMWOVja2GF0.png)
 ### Video Understanding
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/qU5-6VDYtFcXxSq2bNH7T.png)
 ### GUI Grounding
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/9Zjve-VEmxFw5V-BT-wRA.png)
 ### Spatial Reasoning
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/J4AmWoDuJ6JwILdtsNJcN.png)
 ## Evaluation on Language Capability
@@ -146,8 +146,7 @@ We compare InternVL3 with  Qwen2.5 Chat models, whose corresponding pre-trained
 Benefitting from Native Multimodal Pre-Training, the InternVL3 series achieves even better overall text performance than the Qwen2.5 series.
 Please note that the evaluation scores of Qwen2.5 series  may differ from those officially reported, as we have adopted the prompt versions provided in the table across all datasets for OpenCompass evaluation.
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/XHOC-GZO7qAtG8seR05ig.png)
 ## Ablation Study
@@ -157,20 +156,19 @@ We conduct experiments on the InternVL2-8B model while keeping its architecture,
 The evaluation results in the Figure below shows that the model with native multimodal pre-training exhibits performance on most benchmarks that is comparable to the fully multi-stage-trained InternVL2-8B baseline. Furthermore, when followed by instruction tuning on higher-quality data, the model demonstrates further performance gains across evaluated multimodal tasks. These findings underscore the efficiency of native multimodal pre-training in imparting powerful multimodal capabilities to MLLMs.
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/X4PC_JDyPGWJF6gYMPSH6.png)
 ### Mixed Preference Optimization
 As shown in the table below, models fine-tuned with MPO demonstrate superior reasoning performance across seven multimodal reasoning benchmarks compared to their counterparts without MPO. Specifically, InternVL3-78B and InternVL3-38B outperform their counterparts by 4.1 and 4.5 points, respectively. Notably, the training data used for MPO is a subset of that used for SFT, indicating that the performance improvements primarily stem from the training algorithm rather than the training data.
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/bZuOd8yy_sFHcrkYoRERG.png)
 ### Variable Visual Position Encoding
 As reported in the table below, the introduction of V2PE leads to significant performance gains across most evaluation metrics. In addition, our ablation studies—by varying the positional increment \\( \delta \\)—reveal that even for tasks primarily involving conventional contexts, relatively small \\( \delta \\) values can achieve optimal performance. These findings provide important insights for future efforts aimed at refining position encoding strategies for visual tokens in MLLMs.
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/K6D0OP5ZiNLTJB51-HOWa.png)
 ## Quick Start

 Compared to InternVL 2.5, InternVL3 exhibits superior multimodal perception and reasoning capabilities, while further extending its multimodal capabilities to encompass tool usage, GUI agents, industrial image analysis, 3D vision perception, and more.
 Additionally, we compare InternVL3 with  Qwen2.5 Chat models, whose corresponding pre-trained base models are employed as the initialization of the langauge component in InternVL3. Benefitting from Native Multimodal Pre-Training, the InternVL3 series achieves even better overall text performance than the Qwen2.5 series.
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/overall.png)
 ## InternVL3 Family
 | InternVL3-38B |   [InternViT-6B-448px-V2_5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V2_5)   |             [Qwen2.5-32B](https://huggingface.co/Qwen/Qwen2.5-32B)             | [🤗 link](https://huggingface.co/OpenGVLab/InternVL3-38B) |
 | InternVL3-78B |   [InternViT-6B-448px-V2_5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V2_5)   |             [Qwen2.5-72B](https://huggingface.co/Qwen/Qwen2.5-72B)             | [🤗 link](https://huggingface.co/OpenGVLab/InternVL3-78B) |
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/overall-table.png)
 ## Model Architecture
 ### Multimodal Reasoning and Mathematics
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/reasoning.png)
 ### OCR, Chart, and Document Understanding
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/ocr.png)
 ### Multi-Image & Real-World Comprehension
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/multi-images.png)
 ### Comprehensive Multimodal & Hallucination Evaluation
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/comprehensive.png)
 ### Visual Grounding
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/grounding.png)
 ### Multimodal Multilingual Understanding
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/multilingual.png)
 ### Video Understanding
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/video.png)
 ### GUI Grounding
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/gui.png)
 ### Spatial Reasoning
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/vsi.png)
 ## Evaluation on Language Capability
 Benefitting from Native Multimodal Pre-Training, the InternVL3 series achieves even better overall text performance than the Qwen2.5 series.
 Please note that the evaluation scores of Qwen2.5 series  may differ from those officially reported, as we have adopted the prompt versions provided in the table across all datasets for OpenCompass evaluation.
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/text.png)
 ## Ablation Study
 The evaluation results in the Figure below shows that the model with native multimodal pre-training exhibits performance on most benchmarks that is comparable to the fully multi-stage-trained InternVL2-8B baseline. Furthermore, when followed by instruction tuning on higher-quality data, the model demonstrates further performance gains across evaluated multimodal tasks. These findings underscore the efficiency of native multimodal pre-training in imparting powerful multimodal capabilities to MLLMs.
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/ablation-native.png)
 ### Mixed Preference Optimization
 As shown in the table below, models fine-tuned with MPO demonstrate superior reasoning performance across seven multimodal reasoning benchmarks compared to their counterparts without MPO. Specifically, InternVL3-78B and InternVL3-38B outperform their counterparts by 4.1 and 4.5 points, respectively. Notably, the training data used for MPO is a subset of that used for SFT, indicating that the performance improvements primarily stem from the training algorithm rather than the training data.
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/ablation-mpo.png)
 ### Variable Visual Position Encoding
 As reported in the table below, the introduction of V2PE leads to significant performance gains across most evaluation metrics. In addition, our ablation studies—by varying the positional increment \\( \delta \\)—reveal that even for tasks primarily involving conventional contexts, relatively small \\( \delta \\) values can achieve optimal performance. These findings provide important insights for future efforts aimed at refining position encoding strategies for visual tokens in MLLMs.
+![image/png](https://huggingface.co/datasets/Weiyun1025/InternVL-Performance/resolve/main/internvl3/ablation-v2pe.png)
 ## Quick Start