README.md

---
license: apache-2.0
pipeline_tag: image-text-to-text
tags:
- vlzip
- qwen2_5_vl
- long-context
---

# Model Card for VLZip-3B

VLZip is a unified visual and textual token compression method for interleaved long-context
multimodal modeling, built on top of [Qwen2.5-VL](https://github.com/QwenLM/Qwen2.5-VL). It jointly
compresses image and text tokens with dedicated compressors, letting the model handle long
interleaved documents (text + many images) within a fixed context budget.

This checkpoint is the final **stage 4** model (long-context adaptation), produced after the full
4-stage training curriculum described in the paper.

Accepted at **ECCV 2026**.

## Model Sources

- **Repository:** [ShareLab-SII/VLZip](https://github.com/ShareLab-SII/VLZip)
- **Paper:** TODO

## How to Get Started with the Model

This model uses custom model/processor classes that aren't registered with `AutoModel`, so you
need the code from the VLZip repository to load it.

```bash
git clone https://github.com/ShareLab-SII/VLZip.git
cd VLZip/qwen-vl-finetune
```

```python
import torch
from model.vlzip import Qwen2_5_VL_VLZipForConditionalGeneration
from model.processor import Qwen2_5_VL_VLZipProcessor

model_path = "SII-BIU/VLZip-3B"  # or a local checkpoint path

processor = Qwen2_5_VL_VLZipProcessor.from_pretrained(model_path)
model = Qwen2_5_VL_VLZipForConditionalGeneration.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
    device_map="cuda",
).eval()

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": "path/to/image.jpg"},
        {"type": "text", "text": "Describe this image."},
    ],
}]
inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt"
).to(model.device)

with torch.no_grad():
    output_ids = model.generate(**inputs, max_new_tokens=256, do_sample=False)

response = processor.batch_decode(
    output_ids[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True
)[0]
print(response)
```

See the [repository README](https://github.com/ShareLab-SII/VLZip) for long-context (interleaved
text + image) inputs, which additionally require chunking long text through
`processor.encode_chunked_text`.

## Citation

```bibtex
@inproceedings{vlzip,
  title     = {VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling},
  author    = {Zhang, Yuqi and Chen, Cheng and Guo, Yuyu and Yang, Wenjie and Meng, Lingchen and Di, Peng and Yu, Hang and Wu, Zuxuan and Jiang, Yu-Gang},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}
```