Instructions to use OpenGVLab/InternVL2_5-26B-AWQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenGVLab/InternVL2_5-26B-AWQ with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="OpenGVLab/InternVL2_5-26B-AWQ", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("OpenGVLab/InternVL2_5-26B-AWQ", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OpenGVLab/InternVL2_5-26B-AWQ with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenGVLab/InternVL2_5-26B-AWQ"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/InternVL2_5-26B-AWQ",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/OpenGVLab/InternVL2_5-26B-AWQ

SGLang

How to use OpenGVLab/InternVL2_5-26B-AWQ with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenGVLab/InternVL2_5-26B-AWQ" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/InternVL2_5-26B-AWQ",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenGVLab/InternVL2_5-26B-AWQ" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenGVLab/InternVL2_5-26B-AWQ",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use OpenGVLab/InternVL2_5-26B-AWQ with Docker Model Runner:
```
docker model run hf.co/OpenGVLab/InternVL2_5-26B-AWQ
```

Adding `safetensors` variant of this model

by SFconvertbot - opened Mar 27, 2025

base: refs/heads/main

←

from: refs/pr/3

Discussion Files changed

+1460

-0

Files changed (14) hide show

model-00001-of-00013.safetensors +3 -0
model-00002-of-00013.safetensors +3 -0
model-00003-of-00013.safetensors +3 -0
model-00004-of-00013.safetensors +3 -0
model-00005-of-00013.safetensors +3 -0
model-00006-of-00013.safetensors +3 -0
model-00007-of-00013.safetensors +3 -0
model-00008-of-00013.safetensors +3 -0
model-00009-of-00013.safetensors +3 -0
model-00010-of-00013.safetensors +3 -0
model-00011-of-00013.safetensors +3 -0
model-00012-of-00013.safetensors +3 -0
model-00013-of-00013.safetensors +3 -0
model.safetensors.index.json +0 -0

+version https://git-lfs.github.com/spec/v1
+oid sha256:8491b50d3506e82138d66384ffab845da73b4563d410e8a1ff547462d86993bc
+size 1977055976

+version https://git-lfs.github.com/spec/v1
+oid sha256:5268e973700877dd94019650a477a07f82bae3ab17e29a734e20f2d50d424e3d
+size 1966706576

+version https://git-lfs.github.com/spec/v1
+oid sha256:1680bd57034b1a11e6d399dcac9863b91503a45bc667769b4a1068bac01c441d
+size 1966706608

+version https://git-lfs.github.com/spec/v1
+oid sha256:81f902b9f8de1af72bc117688a51e280fa7d89db8fdc7d147f3e3e6a2cf9e81e
+size 1229178584

+version https://git-lfs.github.com/spec/v1
+oid sha256:0715ad6ddc3e842d0c781431c51d390295b50cc8545fcb6b4b84af90a599d3ae
+size 1993774280

+version https://git-lfs.github.com/spec/v1
+oid sha256:f35d3575c8fa105d184e4e2423bd22bd19b2f0af3c2e29031b5990b071acb7e0
+size 1981043096

+version https://git-lfs.github.com/spec/v1
+oid sha256:c9bc9c80dbf46cf0c856b64c951fa8f8eb710f720562821f69bd3ed0ecfe6f44
+size 1974481536

+version https://git-lfs.github.com/spec/v1
+oid sha256:a12b6b9d290c6a1eae8fa4430f4d6c8b0470483cc23f1d746bbf1b97ae50b10f
+size 1974506352

+version https://git-lfs.github.com/spec/v1
+oid sha256:a785c56810917df2a744f4be1d807255956a095c604302e0c166d9bba66ae555
+size 1974506360

+version https://git-lfs.github.com/spec/v1
+oid sha256:57a19f5c077be7078fb266e9c8bd2efecdb3a533daab2c52705f94bdb4bf6079
+size 967653264

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3bd1457fccf7ae32122250cbb6aaf139b419d0e7150fc108513c266bb2b448ba
+size 1370151576