license: mit
base_model:
- Qwen/Qwen2.5-VL-3B-Instruct
- Qwen/Qwen2.5-VL-7B-Instruct
tags:
- vision-language-model
- token-compression
- multimodal
CRISPR — Checkpoints
Checkpoints for CRISPR: Context-Refined Information Spatial Pooling with Region-awareness for Efficient Visual Token Compression in VLMs, accepted at ACM MM 2026.
This repo hosts CRISPR checkpoints for the Qwen2.5-VL-3B-Instruct backbone (9x and 16x compression) and the Qwen2.5-VL-7B-Instruct backbone (16x compression). The 7B/9x checkpoint is not currently available (lost prior to this release) and is not planned unless retraining happens in the future.
Files
| Path | Compression ratio | Backbone | Notes |
|---|---|---|---|
3b_9x/checkpoint.pt |
9x (3x3 block) | Qwen2.5-VL-3B-Instruct | Stage-2, best checkpoint by val loss |
3b_9x/config.json |
training config used to produce this checkpoint | ||
3b_16x/checkpoint.pt |
16x (4x4 block) | Qwen2.5-VL-3B-Instruct | Stage-2, best checkpoint by val loss |
3b_16x/config.json |
training config used to produce this checkpoint | ||
7b_16x/checkpoint.pt |
16x (4x4 block) | Qwen2.5-VL-7B-Instruct | Stage-2, best checkpoint by val loss |
7b_16x/config.json |
training config used to produce this checkpoint |
Each checkpoint.pt is a plain torch.save dict with keys config,
token_mixer (TokenMixer state dict), and local_c3 (LocalC3 state dict,
which also contains the Global Token Fusion sub-module). Only the trainable
CRISPR modules are included — the frozen Qwen2.5-VL vision encoder and
decoder weights are not part of this checkpoint and must be obtained
separately from Qwen2.5-VL. Optimizer/
scheduler state is not included (only the model weights needed for inference
or further fine-tuning are provided).
Usage
from crispr import create_model_v7
model = create_model_v7(decoder_path="./Qwen/Qwen2.5-VL-3B-Instruct")
model.load_checkpoint("3b_9x/checkpoint.pt") # see crispr/model_v7.py for the loader
# for the 7B backbone: decoder_path="./Qwen/Qwen2.5-VL-7B-Instruct", checkpoint="7b_16x/checkpoint.pt"
See the main repository (https://github.com/ZuyiZhou/CRISPR) for the model code, training script, and evaluation scripts.
Citation
@inproceedings{zhou2026crispr,
author = {Zhou, Zuyi and Xue, Dizhan and Qian, Shengsheng and Xu, Changsheng},
title = {CRISPR: Context-Refined Information Spatial Pooling with
Region-awareness for Efficient Visual Token Compression in VLMs},
booktitle = {Proceedings of the 34th ACM International Conference on
Multimedia (MM '26)},
year = {2026},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
doi = {10.1145/3767308.3835007}
}