README.md

metadata

license: apache-2.0
pipeline_tag: image-text-to-text
tags:
  - vlzip
  - qwen2_5_vl
  - long-context

Model Card for VLZip-3B

VLZip is a unified visual and textual token compression method for interleaved long-context multimodal modeling, built on top of Qwen2.5-VL. It jointly compresses image and text tokens with dedicated compressors, letting the model handle long interleaved documents (text + many images) within a fixed context budget.

This checkpoint is the final stage 4 model (long-context adaptation), produced after the full 4-stage training curriculum described in the paper.

Accepted at ECCV 2026.

Model Sources

• Repository: ShareLab-SII/VLZip
• Paper: TODO

How to Get Started with the Model

This model uses custom model/processor classes that aren't registered with AutoModel, so you need the code from the VLZip repository to load it.

git clone https://github.com/ShareLab-SII/VLZip.git
cd VLZip/qwen-vl-finetune

import torch
from model.vlzip import Qwen2_5_VL_VLZipForConditionalGeneration
from model.processor import Qwen2_5_VL_VLZipProcessor

model_path = "SII-BIU/VLZip-3B"  # or a local checkpoint path

processor = Qwen2_5_VL_VLZipProcessor.from_pretrained(model_path)
model = Qwen2_5_VL_VLZipForConditionalGeneration.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
    device_map="cuda",
).eval()

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": "path/to/image.jpg"},
        {"type": "text", "text": "Describe this image."},
    ],
}]
inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt"
).to(model.device)

with torch.no_grad():
    output_ids = model.generate(**inputs, max_new_tokens=256, do_sample=False)

response = processor.batch_decode(
    output_ids[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True
)[0]
print(response)

See the repository README for long-context (interleaved text + image) inputs, which additionally require chunking long text through processor.encode_chunked_text.

Citation

@inproceedings{vlzip,
  title     = {VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling},
  author    = {Zhang, Yuqi and Chen, Cheng and Guo, Yuyu and Yang, Wenjie and Meng, Lingchen and Di, Peng and Yu, Hang and Wu, Zuxuan and Jiang, Yu-Gang},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}