Image-Text-to-Text
Transformers
Safetensors
llava_onevision2_5
text-generation
multimodal
video-understanding
streaming
llava-onevision
conversational
custom_code
Instructions to use lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct
- SGLang
How to use lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct with Docker Model Runner:
docker model run hf.co/lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct
Model card: TOMATO results under the official setting
Browse files- README.md +2 -2
- README_CN.md +2 -2
README.md
CHANGED
|
@@ -46,7 +46,7 @@ LLaVA-OneVision-2.5-4B-Instruct is a 4B vision-language model for single-image,
|
|
| 46 |
<tr><td align="left">VideoMME (no subtitles)</td><td align="center"><b>71.9</b></td><td align="center">69.7</td><td align="center">71.7</td></tr>
|
| 47 |
<tr><td align="left">VideoMME (subtitles)</td><td align="center">74.7</td><td align="center">75.2</td><td align="center"><b>75.9</b></td></tr>
|
| 48 |
<tr><td align="left">MotionBench</td><td align="center"><b>64.2</b> <sup>TC128×313,600</sup></td><td align="center">62.4 <sup>128f×153,664</sup></td><td align="center">60.1 <sup>128f×153,664</sup></td></tr>
|
| 49 |
-
<tr><td align="left">TOMATO
|
| 50 |
<tr><td align="left">MVBench</td><td align="center"><b>75.5</b> <sup>TC64×313,600</sup></td><td align="center">72.0 <sup>64f×153,664</sup></td><td align="center">64.0 <sup>64f×153,664</sup></td></tr>
|
| 51 |
<tr><td align="left">NextQA MC test</td><td align="center"><b>86.4</b> <sup>TC64×313,600</sup></td><td align="center">86.1 <sup>64f×153,664</sup></td><td align="center">80.7 <sup>64f×153,664</sup></td></tr>
|
| 52 |
<tr><td align="left">TempCompass MC</td><td align="center"><b>79.7</b> <sup>TC64×313,600</sup></td><td align="center">75.8 <sup>64f×153,664</sup></td><td align="center">73.4 <sup>64f×153,664</sup></td></tr>
|
|
@@ -88,7 +88,7 @@ LLaVA-OneVision-2.5-4B-Instruct is a 4B vision-language model for single-image,
|
|
| 88 |
</tbody>
|
| 89 |
</table>
|
| 90 |
|
| 91 |
-
<sub>Compared with Qwen3.5-4B, thinking disabled. **Bold** is the best score in each row; `--` means not available. The superscript on each video score is the setting that produced it: frames × pixels per frame, or for codec, canvases (TC) × pixels per canvas. VideoMME is the mean over the short, medium and long splits. Each side reports its best score over the settings tried, from single runs.
|
| 92 |
|
| 93 |
## Model Overview
|
| 94 |
|
|
|
|
| 46 |
<tr><td align="left">VideoMME (no subtitles)</td><td align="center"><b>71.9</b></td><td align="center">69.7</td><td align="center">71.7</td></tr>
|
| 47 |
<tr><td align="left">VideoMME (subtitles)</td><td align="center">74.7</td><td align="center">75.2</td><td align="center"><b>75.9</b></td></tr>
|
| 48 |
<tr><td align="left">MotionBench</td><td align="center"><b>64.2</b> <sup>TC128×313,600</sup></td><td align="center">62.4 <sup>128f×153,664</sup></td><td align="center">60.1 <sup>128f×153,664</sup></td></tr>
|
| 49 |
+
<tr><td align="left">TOMATO</td><td align="center">38.6 <sup>TC128×313,600</sup></td><td align="center"><b>39.2</b> <sup>official</sup></td><td align="center">32.4 <sup>official</sup></td></tr>
|
| 50 |
<tr><td align="left">MVBench</td><td align="center"><b>75.5</b> <sup>TC64×313,600</sup></td><td align="center">72.0 <sup>64f×153,664</sup></td><td align="center">64.0 <sup>64f×153,664</sup></td></tr>
|
| 51 |
<tr><td align="left">NextQA MC test</td><td align="center"><b>86.4</b> <sup>TC64×313,600</sup></td><td align="center">86.1 <sup>64f×153,664</sup></td><td align="center">80.7 <sup>64f×153,664</sup></td></tr>
|
| 52 |
<tr><td align="left">TempCompass MC</td><td align="center"><b>79.7</b> <sup>TC64×313,600</sup></td><td align="center">75.8 <sup>64f×153,664</sup></td><td align="center">73.4 <sup>64f×153,664</sup></td></tr>
|
|
|
|
| 88 |
</tbody>
|
| 89 |
</table>
|
| 90 |
|
| 91 |
+
<sub>Compared with Qwen3.5-4B, thinking disabled. **Bold** is the best score in each row; `--` means not available. The superscript on each video score is the setting that produced it: frames × pixels per frame, or for codec, canvases (TC) × pixels per canvas. VideoMME is the mean over the short, medium and long splits. Each side reports its best score over the settings tried, from single runs. The image table shows selected benchmarks; full results, settings and protocol will be given in the technical report.</sub>
|
| 92 |
|
| 93 |
## Model Overview
|
| 94 |
|
README_CN.md
CHANGED
|
@@ -46,7 +46,7 @@ LLaVA-OneVision-2.5-4B-Instruct 是一个 4B 视觉语言模型,支持单图
|
|
| 46 |
<tr><td align="left">VideoMME(无字幕)</td><td align="center"><b>71.9</b></td><td align="center">69.7</td><td align="center">71.7</td></tr>
|
| 47 |
<tr><td align="left">VideoMME(有字幕)</td><td align="center">74.7</td><td align="center">75.2</td><td align="center"><b>75.9</b></td></tr>
|
| 48 |
<tr><td align="left">MotionBench</td><td align="center"><b>64.2</b> <sup>TC128×313,600</sup></td><td align="center">62.4 <sup>128f×153,664</sup></td><td align="center">60.1 <sup>128f×153,664</sup></td></tr>
|
| 49 |
-
<tr><td align="left">TOMATO
|
| 50 |
<tr><td align="left">MVBench</td><td align="center"><b>75.5</b> <sup>TC64×313,600</sup></td><td align="center">72.0 <sup>64f×153,664</sup></td><td align="center">64.0 <sup>64f×153,664</sup></td></tr>
|
| 51 |
<tr><td align="left">NextQA MC test</td><td align="center"><b>86.4</b> <sup>TC64×313,600</sup></td><td align="center">86.1 <sup>64f×153,664</sup></td><td align="center">80.7 <sup>64f×153,664</sup></td></tr>
|
| 52 |
<tr><td align="left">TempCompass MC</td><td align="center"><b>79.7</b> <sup>TC64×313,600</sup></td><td align="center">75.8 <sup>64f×153,664</sup></td><td align="center">73.4 <sup>64f×153,664</sup></td></tr>
|
|
@@ -88,7 +88,7 @@ LLaVA-OneVision-2.5-4B-Instruct 是一个 4B 视觉语言模型,支持单图
|
|
| 88 |
</tbody>
|
| 89 |
</table>
|
| 90 |
|
| 91 |
-
<sub>对比对象为关闭 thinking 的 Qwen3.5-4B。**加粗**为该行最高分,`--` 表示暂无结果。视频分数的上标是取得该分数时的设置:帧数 × 每帧像素;codec 为画布数(TC)× 每画布像素。VideoMME 为 short、medium、long 三个子集的平均。两边各取已测设置中的最高分,均为单次运行。
|
| 92 |
|
| 93 |
## 模型概览
|
| 94 |
|
|
|
|
| 46 |
<tr><td align="left">VideoMME(无字幕)</td><td align="center"><b>71.9</b></td><td align="center">69.7</td><td align="center">71.7</td></tr>
|
| 47 |
<tr><td align="left">VideoMME(有字幕)</td><td align="center">74.7</td><td align="center">75.2</td><td align="center"><b>75.9</b></td></tr>
|
| 48 |
<tr><td align="left">MotionBench</td><td align="center"><b>64.2</b> <sup>TC128×313,600</sup></td><td align="center">62.4 <sup>128f×153,664</sup></td><td align="center">60.1 <sup>128f×153,664</sup></td></tr>
|
| 49 |
+
<tr><td align="left">TOMATO</td><td align="center">38.6 <sup>TC128×313,600</sup></td><td align="center"><b>39.2</b> <sup>official</sup></td><td align="center">32.4 <sup>official</sup></td></tr>
|
| 50 |
<tr><td align="left">MVBench</td><td align="center"><b>75.5</b> <sup>TC64×313,600</sup></td><td align="center">72.0 <sup>64f×153,664</sup></td><td align="center">64.0 <sup>64f×153,664</sup></td></tr>
|
| 51 |
<tr><td align="left">NextQA MC test</td><td align="center"><b>86.4</b> <sup>TC64×313,600</sup></td><td align="center">86.1 <sup>64f×153,664</sup></td><td align="center">80.7 <sup>64f×153,664</sup></td></tr>
|
| 52 |
<tr><td align="left">TempCompass MC</td><td align="center"><b>79.7</b> <sup>TC64×313,600</sup></td><td align="center">75.8 <sup>64f×153,664</sup></td><td align="center">73.4 <sup>64f×153,664</sup></td></tr>
|
|
|
|
| 88 |
</tbody>
|
| 89 |
</table>
|
| 90 |
|
| 91 |
+
<sub>对比对象为关闭 thinking 的 Qwen3.5-4B。**加粗**为该行最高分,`--` 表示暂无结果。视频分数的上标是取得该分数时的设置:帧数 × 每帧像素;codec 为画布数(TC)× 每画布像素。VideoMME 为 short、medium、long 三个子集的平均。两边各取已测设置中的最高分,均为单次运行。图像表为代表性 bench 的节选,完整结果、设置与评测口径见后续技术报告。</sub>
|
| 92 |
|
| 93 |
## 模型概览
|
| 94 |
|