File size: 2,692 Bytes
e464a61 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | ---
license: apache-2.0
pipeline_tag: image-text-to-text
tags:
- vlzip
- qwen2_5_vl
- long-context
---
# Model Card for VLZip-3B
VLZip is a unified visual and textual token compression method for interleaved long-context
multimodal modeling, built on top of [Qwen2.5-VL](https://github.com/QwenLM/Qwen2.5-VL). It jointly
compresses image and text tokens with dedicated compressors, letting the model handle long
interleaved documents (text + many images) within a fixed context budget.
This checkpoint is the final **stage 4** model (long-context adaptation), produced after the full
4-stage training curriculum described in the paper.
Accepted at **ECCV 2026**.
## Model Sources
- **Repository:** [ShareLab-SII/VLZip](https://github.com/ShareLab-SII/VLZip)
- **Paper:** TODO
## How to Get Started with the Model
This model uses custom model/processor classes that aren't registered with `AutoModel`, so you
need the code from the VLZip repository to load it.
```bash
git clone https://github.com/ShareLab-SII/VLZip.git
cd VLZip/qwen-vl-finetune
```
```python
import torch
from model.vlzip import Qwen2_5_VL_VLZipForConditionalGeneration
from model.processor import Qwen2_5_VL_VLZipProcessor
model_path = "SII-BIU/VLZip-3B" # or a local checkpoint path
processor = Qwen2_5_VL_VLZipProcessor.from_pretrained(model_path)
model = Qwen2_5_VL_VLZipForConditionalGeneration.from_pretrained(
model_path,
torch_dtype=torch.bfloat16,
attn_implementation="flash_attention_2",
device_map="cuda",
).eval()
messages = [{
"role": "user",
"content": [
{"type": "image", "image": "path/to/image.jpg"},
{"type": "text", "text": "Describe this image."},
],
}]
inputs = processor.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt"
).to(model.device)
with torch.no_grad():
output_ids = model.generate(**inputs, max_new_tokens=256, do_sample=False)
response = processor.batch_decode(
output_ids[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True
)[0]
print(response)
```
See the [repository README](https://github.com/ShareLab-SII/VLZip) for long-context (interleaved
text + image) inputs, which additionally require chunking long text through
`processor.encode_chunked_text`.
## Citation
```bibtex
@inproceedings{vlzip,
title = {VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling},
author = {Zhang, Yuqi and Chen, Cheng and Guo, Yuyu and Yang, Wenjie and Meng, Lingchen and Di, Peng and Yu, Hang and Wu, Zuxuan and Jiang, Yu-Gang},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2026}
}
```
|