--- license: apache-2.0 pipeline_tag: image-text-to-text library_name: transformers tags: - multimodal - video-understanding - streaming - llava-onevision ---
LLaVA-OneVision-2.5-4B-Instruct is a 4B vision-language model for single-image, multi-image and video input. Beyond the usual offline tasks, it is trained for **streaming** video: frames arrive one at a time, and at every step the model decides whether to stay silent or speak. The same weights and the same chat template serve both modes. ## Highlights - **Offline and streaming in one model.** `apply_chat_template(..., streaming=True)` switches mode; eight protocol tokens carry the per-frame decision to stay silent, respond or caption. Nothing else about the input changes. - **Codec-aligned video.** Besides uniform frame sampling, *codec mode* uses the encoder's bit-cost to pack the informative regions of a video into a handful of canvases. - **9× spatial compression.** A patch-16 vision tower with 3×3 merging gives one visual token per 48×48 pixels, so long videos fit comfortably in the 262k context. ## Performance ### Video| Benchmark | OV2.5-4B (codec) | OV2.5-4B (frames) | Qwen3.5-4B |
|---|---|---|---|
| General video understanding | |||
| VideoMME (no subtitles) | 71.9 | 69.7 | 71.7 |
| VideoMME (subtitles) | 74.7 | 75.2 | 75.9 |
| MotionBench | 64.2 TC128×313,600 | 62.4 128f×153,664 | 60.1 128f×153,664 |
| TOMATO | 36.0 TC16×153,664 | 39.2 16f×153,664 | 32.4 16f×153,664 |
| MVBench | 75.5 TC64×313,600 | 72.0 64f×153,664 | 64.0 64f×153,664 |
| NextQA MC test | 86.4 TC64×313,600 | 86.1 64f×153,664 | 80.7 64f×153,664 |
| TempCompass MC | 79.7 TC64×313,600 | 75.8 64f×153,664 | 73.4 64f×153,664 |
| Video-MME-v2 | 17.2 TC64×313,600 | 16.4 64f×330,000 | 15.5 64f×330,000 |
| EgoSchema subset | 79.8 TC64×313,600 | 80.0 64f×153,664 | 68.2 64f×153,664 |
| Video-Index | 23.1 TC512×55,296 | 20.0 1fps, ≤512f×55,296 | 19.6 1fps, ≤512f×55,296 |
| Long video | |||
| LVBench | 61.6 TC1024×156,800 | 62.0 950f×200,000 | 56.5 2048f×120,000 |
| MLVU dev | 77.1 TC512×313,600 | 75.3 512f×72,900 | 73.8 512f×72,900 |
| LongVideoBench val | 71.1 TC768×235,200 | 69.0 1280f×80,000 | 62.7 700f×300,000 |
| VideoEval-Pro | 64.6 TC768×235,200 | 64.4 950f×200,000 | 62.8 2048f×100,000 |
| Temporal grounding (mIoU) | |||
| TimeLens ActivityNet | 59.1 TC128×313,600 | 58.8 2048f×409,600, fps=4 | 53.8 2048f×409,600, fps=4 |
| TimeLens Charades | 61.3 TC128×313,600 | 60.6 2048f×409,600, fps=4 | 48.1 2048f×409,600, fps=4 |
| TimeLens QVHighlights | 70.3 TC128×313,600 | 68.4 2048f×409,600, fps=4 | 67.5 2048f×409,600, fps=4 |
| Benchmark | OV2.5-4B (codec, 24 fps) |
MOSS-VL-Realtime 11B (1 fps) |
JoyAI-VL-Interaction 8B (1 fps) |
|---|---|---|---|
| OVO-Bench | |||
| Real-Time Visual Perception (RTVP) | 81.0 | 75.9 | 76.4 |
| Backward Tracing (BT) | 68.8 | 72.6 | 53.3 |
| Forward Active Responding (FAR) | 60.7 | 62.1 | 54.6 |
| Overall | 70.2 | 70.2 | 61.4 |
| StreamingBench | |||
| Real-Time Visual Understanding (RTVU) | 85.3 | 82.9 | 83.2 |
| Misleading Context Understanding (MCU) | 36.8 | -- | 37.2 |
| Anomaly Context Understanding (ACU) | 67.6 | -- | 51.6 |
| Sequential Question Answering (SQA) | 54.8 | 50.4 | 53.6 |
| Proactive Output (PO) | 63.6 | 60.0 | 39.6 |
| Contextual Understanding (CU) | 55.7 | 56.4 | 45.5 |
| Proactive and interactive | |||
| OmniMMI: Proactive Alerting (PA) | 68.5 | 66.0 | 13.5 |
| ProactiveVideoQA: PAUC (ω=0.5) | 33.1 | 47.2 | 37.0 |
| InteractionBench: Total | 38.6 | -- | -- |
| StreamArena: Proactive Interaction (Pro) | 25.5 | 11.8 | 9.4 |
| General perception and reading | OV2.5-4B | Qwen3.5-4B | Grounding and spatial | OV2.5-4B | Qwen3.5-4B |
|---|---|---|---|---|---|
| MMStar | 63.9 | 66.9 | RefCOCO val (Acc@0.5) | 92.6 | 89.1 |
| MMBench-EN dev | 88.2 | 88.8 | RefCOCOg val (Acc@0.5) | 88.9 | 87.7 |
| RealWorldQA | 73.9 | 74.4 | RefSpatial-Bench | 62.1 | 29.1 |
| AI2D | 82.2 | 82.4 | CrossPoint | 80.5 | 33.7 |
| BLINK | 67.1 | 63.6 | CV-Bench 2D | 83.3 | 80.7 |
| V*-Bench | 89.5 | 81.7 | CV-Bench 3D | 94.5 | 92.0 |
| OCRBench | 79.8 | 85.2 | EmbSpatial | 83.0 | 74.7 |
| DocVQA val | 92.9 | 95.2 | MMSI-Bench | 33.3 | 31.4 |
| Type | instruction-tuned vision-language model, LLaVA-OneVision-2.5 architecture |
| Parameters | 5.25B total (4B language model plus vision tower and adapter) |
| Language side | 32 layers, hidden size 2560, FFN 9216, 16 attention heads / 4 KV heads, linear attention with one full-attention layer every 4 |
| Vision side | 24 layers, hidden size 1024, patch_size=16, spatial_merge_size=3, attention within a frame |
| Vocabulary | 248320, including the 8 streaming protocol tokens |
| Context length | 262144 |
| Weights | bfloat16, 3 safetensors shards, about 9.9 GB |