Instructions to use AI4Chem/ChemVLM-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AI4Chem/ChemVLM-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="AI4Chem/ChemVLM-8B", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("AI4Chem/ChemVLM-8B", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use AI4Chem/ChemVLM-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AI4Chem/ChemVLM-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AI4Chem/ChemVLM-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/AI4Chem/ChemVLM-8B

SGLang

How to use AI4Chem/ChemVLM-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AI4Chem/ChemVLM-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AI4Chem/ChemVLM-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AI4Chem/ChemVLM-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AI4Chem/ChemVLM-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use AI4Chem/ChemVLM-8B with Docker Model Runner:
```
docker model run hf.co/AI4Chem/ChemVLM-8B
```

ChemVLM-8B

Commit History

Update README.md

08ff593
verified

Duke-de-Artois commited on Nov 20, 2025

Add helper functions (#3)

3c555d5
verified

Duke-de-Artois

nielsr HF Staff commited on Apr 19, 2025

Improve model card: update pipeline tag, add library name and license, and improve content (#2)

beb1da5
verified

Duke-de-Artois

nielsr HF Staff commited on Apr 18, 2025

Update README.md

3e34f17
verified

Duke-de-Artois commited on Apr 14, 2025

Update README.md

a4dd284
verified

Duke-de-Artois commited on Apr 13, 2025

Upload 8 files

8a0f7aa
verified

Duke-de-Artois commited on Jan 5, 2025

Update README.md

8620d13
verified

Duke-de-Artois commited on Jan 5, 2025

Update README.md

a1c54ed
verified

Duke-de-Artois commited on Dec 30, 2024

Update README.md

7655e98
verified

Duke-de-Artois commited on Dec 30, 2024

Update config.json

dd1765b
verified

ajupyter commited on Oct 29, 2024

Update README.md

f328541
verified

Di Zhang commited on Oct 29, 2024

Update README.md

3bce133
verified

Duke-de-Artois commited on Oct 29, 2024

Update README.md

7be3e29
verified

Duke-de-Artois commited on Oct 29, 2024

Update README.md

00dcbe0
verified

Duke-de-Artois commited on Oct 29, 2024

Update README.md

3826b88
verified

Duke-de-Artois commited on Oct 29, 2024

Upload folder using huggingface_hub

dbabfb2
verified

ajupyter commited on Oct 28, 2024

Update README.md (#1)

b5ce69f
verified

ajupyter

Duke-de-Artois commited on Oct 28, 2024

initial commit

ef6ed32
verified

ajupyter commited on Oct 28, 2024

Commit History

Update README.md 08ff593 verified

Add helper functions (#3) 3c555d5 verified

Improve model card: update pipeline tag, add library name and license, and improve content (#2) beb1da5 verified

Update README.md 3e34f17 verified

Update README.md a4dd284 verified

Upload 8 files 8a0f7aa verified

Update README.md 8620d13 verified

Update README.md a1c54ed verified

Update README.md 7655e98 verified

Update config.json dd1765b verified

Update README.md f328541 verified

Update README.md 3bce133 verified

Update README.md 7be3e29 verified

Update README.md 00dcbe0 verified

Update README.md 3826b88 verified

Upload folder using huggingface_hub dbabfb2 verified

Update README.md (#1) b5ce69f verified

initial commit ef6ed32 verified

Update README.md

08ff593
verified

Add helper functions (#3)

3c555d5
verified

Improve model card: update pipeline tag, add library name and license, and improve content (#2)

beb1da5
verified

Update README.md

3e34f17
verified

Update README.md

a4dd284
verified

Upload 8 files

8a0f7aa
verified

Update README.md

8620d13
verified

Update README.md

a1c54ed
verified

Update README.md

7655e98
verified

Update config.json

dd1765b
verified

Update README.md

f328541
verified

Update README.md

3bce133
verified

Update README.md

7be3e29
verified

Update README.md

00dcbe0
verified

Update README.md

3826b88
verified

Upload folder using huggingface_hub

dbabfb2
verified

Update README.md (#1)

b5ce69f
verified

initial commit

ef6ed32
verified