Instructions to use OpenGVLab/InternVL2-Llama3-76B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenGVLab/InternVL2-Llama3-76B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="OpenGVLab/InternVL2-Llama3-76B", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("OpenGVLab/InternVL2-Llama3-76B", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use OpenGVLab/InternVL2-Llama3-76B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenGVLab/InternVL2-Llama3-76B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/InternVL2-Llama3-76B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/OpenGVLab/InternVL2-Llama3-76B

SGLang

How to use OpenGVLab/InternVL2-Llama3-76B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenGVLab/InternVL2-Llama3-76B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/InternVL2-Llama3-76B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenGVLab/InternVL2-Llama3-76B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/InternVL2-Llama3-76B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use OpenGVLab/InternVL2-Llama3-76B with Docker Model Runner:
```
docker model run hf.co/OpenGVLab/InternVL2-Llama3-76B
```

add AIBOM

#12

by sabato-nocera - opened Jun 14, 2025

base: refs/heads/main

←

from: refs/pr/12

Discussion Files changed

+75

-0

Files changed (1) hide show

OpenGVLab_InternVL2-Llama3-76B.json +75 -0

OpenGVLab_InternVL2-Llama3-76B.json ADDED Viewed

	@@ -0,0 +1,75 @@

+{
+    "bomFormat": "CycloneDX",
+    "specVersion": "1.6",
+    "serialNumber": "urn:uuid:45c0f496-c4cd-4ccf-997e-a8be48a1bace",
+    "version": 1,
+    "metadata": {
+        "timestamp": "2025-06-05T09:36:14.979943+00:00",
+        "component": {
+            "type": "machine-learning-model",
+            "bom-ref": "OpenGVLab/InternVL2-Llama3-76B-db72f665-71fd-5c8c-99f9-e23be02c9399",
+            "name": "OpenGVLab/InternVL2-Llama3-76B",
+            "externalReferences": [
+                {
+                    "url": "https://huggingface.co/OpenGVLab/InternVL2-Llama3-76B",
+                    "type": "documentation"
+                }
+            ],
+            "modelCard": {
+                "modelParameters": {
+                    "task": "image-text-to-text",
+                    "architectureFamily": "internvl_chat",
+                    "modelArchitecture": "InternVLChatModel"
+                },
+                "properties": [
+                    {
+                        "name": "library_name",
+                        "value": "transformers"
+                    },
+                    {
+                        "name": "base_model",
+                        "value": "OpenGVLab/InternViT-6B-448px-V1-5, NousResearch/Hermes-2-Theta-Llama-3-70B"
+                    },
+                    {
+                        "name": "base_model_relation",
+                        "value": "merge"
+                    }
+                ]
+            },
+            "authors": [
+                {
+                    "name": "OpenGVLab"
+                }
+            ],
+            "licenses": [
+                {
+                    "license": {
+                        "name": "llama3"
+                    }
+                }
+            ],
+            "description": "We are excited to announce the release of InternVL 2.0, the latest addition to the InternVL series of multimodal large language models. InternVL 2.0 features a variety of **instruction-tuned models**, ranging from 1 billion to 108 billion parameters. This repository contains the instruction-tuned InternVL2-Llama3-76B model.Compared to the state-of-the-art open-source multimodal large language models, InternVL 2.0 surpasses most open-source models. It demonstrates competitive performance on par with proprietary commercial models across various capabilities, including document and chart comprehension, infographics QA, scene text understanding and OCR tasks, scientific and mathematical problem solving, as well as cultural understanding and integrated multimodal capabilities.InternVL 2.0 is trained with an 8k context window and utilizes training data consisting of long texts, multiple images, and videos, significantly improving its ability to handle these types of inputs compared to InternVL 1.5. For more details, please refer to our [blog](https://internvl.github.io/blog/2024-07-02-InternVL-2.0/) and [GitHub](https://github.com/OpenGVLab/InternVL).|      Model Name      |                                     Vision Part                                     |                                        Language Part                                         |                             HF Link                              |                                MS Link                                 || :------------------: | :---------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------: | :--------------------------------------------------------------: | :--------------------------------------------------------------------: ||     InternVL2-1B     |    [InternViT-300M-448px](https://huggingface.co/OpenGVLab/InternViT-300M-448px)    |            [Qwen2-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2-0.5B-Instruct)            |     [\ud83e\udd17 link](https://huggingface.co/OpenGVLab/InternVL2-1B)     |     [\ud83e\udd16 link](https://modelscope.cn/models/OpenGVLab/InternVL2-1B)     ||     InternVL2-2B     |    [InternViT-300M-448px](https://huggingface.co/OpenGVLab/InternViT-300M-448px)    |          [internlm2-chat-1_8b](https://huggingface.co/internlm/internlm2-chat-1_8b)          |     [\ud83e\udd17 link](https://huggingface.co/OpenGVLab/InternVL2-2B)     |     [\ud83e\udd16 link](https://modelscope.cn/models/OpenGVLab/InternVL2-2B)     ||     InternVL2-4B     |    [InternViT-300M-448px](https://huggingface.co/OpenGVLab/InternViT-300M-448px)    |    [Phi-3-mini-128k-instruct](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct)     |     [\ud83e\udd17 link](https://huggingface.co/OpenGVLab/InternVL2-4B)     |     [\ud83e\udd16 link](https://modelscope.cn/models/OpenGVLab/InternVL2-4B)     ||     InternVL2-8B     |    [InternViT-300M-448px](https://huggingface.co/OpenGVLab/InternViT-300M-448px)    |          [internlm2_5-7b-chat](https://huggingface.co/internlm/internlm2_5-7b-chat)          |     [\ud83e\udd17 link](https://huggingface.co/OpenGVLab/InternVL2-8B)     |     [\ud83e\udd16 link](https://modelscope.cn/models/OpenGVLab/InternVL2-8B)     ||    InternVL2-26B     | [InternViT-6B-448px-V1-5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V1-5) |           [internlm2-chat-20b](https://huggingface.co/internlm/internlm2-chat-20b)           |    [\ud83e\udd17 link](https://huggingface.co/OpenGVLab/InternVL2-26B)     |    [\ud83e\udd16 link](https://modelscope.cn/models/OpenGVLab/InternVL2-26B)     ||    InternVL2-40B     | [InternViT-6B-448px-V1-5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V1-5) |       [Nous-Hermes-2-Yi-34B](https://huggingface.co/NousResearch/Nous-Hermes-2-Yi-34B)       |    [\ud83e\udd17 link](https://huggingface.co/OpenGVLab/InternVL2-40B)     |    [\ud83e\udd16 link](https://modelscope.cn/models/OpenGVLab/InternVL2-40B)     || InternVL2-Llama3-76B | [InternViT-6B-448px-V1-5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V1-5) | [Hermes-2-Theta-Llama-3-70B](https://huggingface.co/NousResearch/Hermes-2-Theta-Llama-3-70B) | [\ud83e\udd17 link](https://huggingface.co/OpenGVLab/InternVL2-Llama3-76B) | [\ud83e\udd16 link](https://modelscope.cn/models/OpenGVLab/InternVL2-Llama3-76B) |",
+            "tags": [
+                "transformers",
+                "safetensors",
+                "internvl_chat",
+                "feature-extraction",
+                "internvl",
+                "custom_code",
+                "image-text-to-text",
+                "conversational",
+                "multilingual",
+                "arxiv:2312.14238",
+                "arxiv:2404.16821",
+                "arxiv:2410.16261",
+                "arxiv:2412.05271",
+                "base_model:NousResearch/Hermes-2-Theta-Llama-3-70B",
+                "base_model:merge:NousResearch/Hermes-2-Theta-Llama-3-70B",
+                "base_model:OpenGVLab/InternViT-6B-448px-V1-5",
+                "base_model:merge:OpenGVLab/InternViT-6B-448px-V1-5",
+                "license:llama3",
+                "region:us"
+            ]
+        }
+    }
+}