Instructions to use lmms-lab/LLaVA-OneVision-1.5-4B-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lmms-lab/LLaVA-OneVision-1.5-4B-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="lmms-lab/LLaVA-OneVision-1.5-4B-Base", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("lmms-lab/LLaVA-OneVision-1.5-4B-Base", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use lmms-lab/LLaVA-OneVision-1.5-4B-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lmms-lab/LLaVA-OneVision-1.5-4B-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lmms-lab/LLaVA-OneVision-1.5-4B-Base",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/lmms-lab/LLaVA-OneVision-1.5-4B-Base

SGLang

How to use lmms-lab/LLaVA-OneVision-1.5-4B-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lmms-lab/LLaVA-OneVision-1.5-4B-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lmms-lab/LLaVA-OneVision-1.5-4B-Base",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lmms-lab/LLaVA-OneVision-1.5-4B-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lmms-lab/LLaVA-OneVision-1.5-4B-Base",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use lmms-lab/LLaVA-OneVision-1.5-4B-Base with Docker Model Runner:
```
docker model run hf.co/lmms-lab/LLaVA-OneVision-1.5-4B-Base
```

LLaVA-OneVision-1.5-4B-Base

Commit History

Improve model card: Add paper/code/demo links, sample usage, update title & citations (#1)

9724845
verified

xiangan

nielsr HF Staff commited on Oct 5, 2025

Update README.md

1b92d6b
verified

Yiye commited on Sep 30, 2025

Update README.md

fcdb69c
verified

Yiye commited on Sep 30, 2025

Upload configuration_llavaonevision1_5.py with huggingface_hub

361fdf4
verified

Yiye commited on Sep 30, 2025

Upload modeling_llavaonevision1_5.py with huggingface_hub

00050d9
verified

Yiye commited on Sep 30, 2025

Upload generation_config.json with huggingface_hub

f63d39b
verified

Yiye commited on Sep 30, 2025

Upload config.json with huggingface_hub

b5da650
verified

Yiye commited on Sep 30, 2025

Upload preprocessor_config.json with huggingface_hub

eed7fd8
verified

Yiye commited on Sep 30, 2025

Upload vocab.json with huggingface_hub

4fd0e91
verified

Yiye commited on Sep 30, 2025

Upload tokenizer_config.json with huggingface_hub

4f3fbdc
verified

Yiye commited on Sep 30, 2025

Upload tokenizer.json with huggingface_hub

86e446d
verified

Yiye commited on Sep 30, 2025

Upload model.safetensors.index.json with huggingface_hub

cf34426
verified

Yiye commited on Sep 30, 2025

Upload merges.txt with huggingface_hub

1050864
verified

Yiye commited on Sep 30, 2025

Upload model-00001-of-00002.safetensors with huggingface_hub

fb14904
verified

Yiye commited on Sep 30, 2025

Upload chat_template.jinja with huggingface_hub

ffe3a7d
verified

Yiye commited on Sep 30, 2025

Upload special_tokens_map.json with huggingface_hub

97071a7
verified

Yiye commited on Sep 30, 2025

Upload model-00002-of-00002.safetensors with huggingface_hub

0e8a1ec
verified

Yiye commited on Sep 30, 2025

Upload added_tokens.json with huggingface_hub

c349d16
verified

Yiye commited on Sep 30, 2025

initial commit

cf71e07
verified

Yiye commited on Sep 29, 2025

Commit History

Improve model card: Add paper/code/demo links, sample usage, update title & citations (#1) 9724845 verified

Update README.md 1b92d6b verified

Update README.md fcdb69c verified

Upload configuration_llavaonevision1_5.py with huggingface_hub 361fdf4 verified

Upload modeling_llavaonevision1_5.py with huggingface_hub 00050d9 verified

Upload generation_config.json with huggingface_hub f63d39b verified

Upload config.json with huggingface_hub b5da650 verified

Upload preprocessor_config.json with huggingface_hub eed7fd8 verified

Upload vocab.json with huggingface_hub 4fd0e91 verified

Upload tokenizer_config.json with huggingface_hub 4f3fbdc verified

Upload tokenizer.json with huggingface_hub 86e446d verified

Upload model.safetensors.index.json with huggingface_hub cf34426 verified

Upload merges.txt with huggingface_hub 1050864 verified

Upload model-00001-of-00002.safetensors with huggingface_hub fb14904 verified

Upload chat_template.jinja with huggingface_hub ffe3a7d verified

Upload special_tokens_map.json with huggingface_hub 97071a7 verified

Upload model-00002-of-00002.safetensors with huggingface_hub 0e8a1ec verified

Upload added_tokens.json with huggingface_hub c349d16 verified

initial commit cf71e07 verified

Improve model card: Add paper/code/demo links, sample usage, update title & citations (#1)

9724845
verified

Update README.md

1b92d6b
verified

Update README.md

fcdb69c
verified

Upload configuration_llavaonevision1_5.py with huggingface_hub

361fdf4
verified

Upload modeling_llavaonevision1_5.py with huggingface_hub

00050d9
verified

Upload generation_config.json with huggingface_hub

f63d39b
verified

Upload config.json with huggingface_hub

b5da650
verified

Upload preprocessor_config.json with huggingface_hub

eed7fd8
verified

Upload vocab.json with huggingface_hub

4fd0e91
verified

Upload tokenizer_config.json with huggingface_hub

4f3fbdc
verified

Upload tokenizer.json with huggingface_hub

86e446d
verified

Upload model.safetensors.index.json with huggingface_hub

cf34426
verified

Upload merges.txt with huggingface_hub

1050864
verified

Upload model-00001-of-00002.safetensors with huggingface_hub

fb14904
verified

Upload chat_template.jinja with huggingface_hub

ffe3a7d
verified

Upload special_tokens_map.json with huggingface_hub

97071a7
verified

Upload model-00002-of-00002.safetensors with huggingface_hub

0e8a1ec
verified

Upload added_tokens.json with huggingface_hub

c349d16
verified

initial commit

cf71e07
verified