---
license: apache-2.0
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- internvl
- multimodal
- vision-language
- datacomp
- pretrained
base_model:
- Qwen/Qwen2.5-1.5B
- OpenGVLab/InternViT-300M-448px
datasets:
- mlfoundations/dcvlm-baseline-200b
---

# DCVLM-2B

**DCVLM-2B** is the `medium`-scale reference 2B-parameter VLM from our [DataComp-VLM](https://arxiv.org/abs/2606.28551)
paper, pretrained from scratch on **DCVLM-Baseline**.

> ⚠️ This is a **pretrained (base) model**, not an instruction-tuned assistant. It is released as a
> reproducible reference point for data-centric research: to compare pretraining datasets, and as an
> initialization for your own SFT. It has not been chat- or preference-tuned and should not be
> expected to follow instructions reliably.

## Model details
| | |
| :--- | :--- |
| Scale | `medium` |
| Parameters ($N$) | 2B |
| Training tokens ($D$) | 25B |
| Architecture | InternVL-2.5 (`InternVLChatModel`) |
| Vision encoder | InternViT-300M-448px |
| Language model | Qwen/Qwen2.5-1.5B |
| Precision | bfloat16 |
| Training compute | ~640 H100 hours |
| Training data | [DCVLM-Baseline](https://huggingface.co/datasets/mlfoundations/dcvlm-baseline-200b) |
| Data mixture | 10% captioning / 70% visual instruction / 15% text-only / 5% multimodal documents |

## Results

Core evaluation suite (33 tasks), compared against **FineVision**, the previous best open
pretraining dataset, trained at the same scale with the same architecture and token budget.

| Method | Gen | Know | OCR | Vision | MTL | Text | **Core Avg** |
| :----- | --: | ---: | --: | -----: | --: | ---: | -----------: |
| FineVision | 55.3 | 62.6 | 51.9 | 45.8 | 40.6 | 46.3 | 50.6 |
| **DCVLM-Baseline (this model)** | 62.3 | 60.5 | 45.8 | 47.3 | 44.2 | 47.8 | **51.7** |

Categories: **Gen** general understanding · **Know** knowledge · **OCR** OCR & charts ·
**Vision** vision-centric · **MTL** multilingual · **Text** text-only.

## Usage

The model uses custom modeling code, so `trust_remote_code=True` is required.

```python
import torch
from transformers import AutoModel, AutoTokenizer

path = "mlfoundations/dcvlm-2b-model"
model = AutoModel.from_pretrained(
    path, torch_dtype=torch.bfloat16, trust_remote_code=True, low_cpu_mem_usage=True
).eval().cuda()
tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True, use_fast=False)
```

Image preprocessing follows the standard InternVL-2.5 dynamic-tiling recipe
(448px tiles, `max_dynamic_patch=12`).

## Related releases

| | |
| :--- | :--- |
| Code | [github.com/mlfoundations/dcvlm](https://github.com/mlfoundations/dcvlm) |
| Datasets | [DCVLM-Baseline 6.25B](https://huggingface.co/datasets/mlfoundations/dcvlm-baseline-6_25b) · [DCVLM-Baseline 200B](https://huggingface.co/datasets/mlfoundations/dcvlm-baseline-200b) · [DCVLM-Balanced 200B](https://huggingface.co/datasets/mlfoundations/dcvlm-balanced-200b) |
| Models | [1B](https://huggingface.co/mlfoundations/dcvlm-1b-model) · [2B](https://huggingface.co/mlfoundations/dcvlm-2b-model) · [4B](https://huggingface.co/mlfoundations/dcvlm-4b-model) · [8B](https://huggingface.co/mlfoundations/dcvlm-8b-model) |

## Citation

```bibtex
@article{farina2026datacomp,
  title={DataComp-VLM: Improved Open Datasets for Vision-Language Models},
  author={Farina, Matteo and Udandarao, Vishaal and Nguyen, Thao and Kuzucu, Selim and B{\"o}ther, Maximilian and Hochlehnert, Andreas and Ghosh, Adhiraj and Nezhurina, Marianna and Roth, Karsten and Struber, Joschka and others},
  journal={arXiv preprint arXiv:2606.28551},
  year={2026}
}
```