Add CRISPR 3B checkpoints (9x, 16x) and model card

3b_16x/checkpoint.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20caa8d00eacc5464c7b388707210da0e178c0013e22b35845ebceb2128b9899
+size 503644725

3b_16x/config.json

@@ -0,0 +1,40 @@
+{
+  "decoder_path": "./Qwen/Qwen2.5-VL-3B-Instruct",
+  "use_token_mixer": true,
+  "token_mixer_num_layers": 2,
+  "token_mixer_num_heads": 16,
+  "token_mixer_dropout": 0.1,
+  "freeze_token_mixer": false,
+  "use_local_c3": true,
+  "local_c3_block_size": 4,
+  "local_c3_num_heads": 8,
+  "num_post_encoder_layers": 2,
+  "use_rope_alignment": true,
+  "use_kl_distillation": true,
+  "kl_loss_weight": 0.5,
+  "teacher_temperature": 1.0,
+  "use_hidden_distillation": true,
+  "hidden_loss_weight": 0.1,
+  "use_decoder_lora": false,
+  "data_path": "./data/train/stage2_mixed/combined_v4.jsonl",
+  "max_length": 768,
+  "max_samples": null,
+  "max_image_size": 672,
+  "output_dir": "./outputs/image_c3_v7_3b_16x_stage2_20260318_160014",
+  "epochs": 3,
+  "batch_size": 2,
+  "gradient_accumulation_steps": 4,
+  "lr_local_c3": 1e-05,
+  "lr_token_mixer": 1e-05,
+  "lr_decoder_lora": 1e-05,
+  "weight_decay": 0.01,
+  "warmup_ratio": 0.1,
+  "max_grad_norm": 1.0,
+  "log_steps": 50,
+  "save_steps": 500,
+  "gpu": 0,
+  "num_workers": 4,
+  "resume_from": null,
+  "timestamp": "2026-03-18T19:00:35.309642",
+  "world_size": 8
+}

3b_9x/checkpoint.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b01b082b255f304cd176b1fd5373760505b09da8a28453addf238617069ca95
+size 503616053

3b_9x/config.json

@@ -0,0 +1,40 @@
+{
+  "decoder_path": "./Qwen/Qwen2.5-VL-3B-Instruct",
+  "use_token_mixer": true,
+  "token_mixer_num_layers": 2,
+  "token_mixer_num_heads": 16,
+  "token_mixer_dropout": 0.1,
+  "freeze_token_mixer": false,
+  "use_local_c3": true,
+  "local_c3_block_size": 3,
+  "local_c3_num_heads": 8,
+  "num_post_encoder_layers": 2,
+  "use_rope_alignment": true,
+  "use_kl_distillation": true,
+  "kl_loss_weight": 0.5,
+  "teacher_temperature": 1.0,
+  "use_hidden_distillation": true,
+  "hidden_loss_weight": 0.1,
+  "use_decoder_lora": false,
+  "data_path": "./data/train/stage2_mixed/combined_v4.jsonl",
+  "max_length": 768,
+  "max_samples": null,
+  "max_image_size": 672,
+  "output_dir": "./outputs/image_c3_v7_3b_9x_stage2_20260316_222808",
+  "epochs": 3,
+  "batch_size": 2,
+  "gradient_accumulation_steps": 4,
+  "lr_local_c3": 1e-05,
+  "lr_token_mixer": 1e-05,
+  "lr_decoder_lora": 1e-05,
+  "weight_decay": 0.01,
+  "warmup_ratio": 0.1,
+  "max_grad_norm": 1.0,
+  "log_steps": 50,
+  "save_steps": 500,
+  "gpu": 0,
+  "num_workers": 4,
+  "resume_from": null,
+  "timestamp": "2026-03-17T05:42:27.910473",
+  "world_size": 8
+}

README.md

@@ -1,3 +1,67 @@
 ---
 license: mit
+base_model: Qwen/Qwen2.5-VL-3B-Instruct
+tags:
+- vision-language-model
+- token-compression
+- multimodal
 ---
+
+# CRISPR — Checkpoints (Qwen2.5-VL-3B backbone)
+
+Checkpoints for **CRISPR: Context-Refined Information Spatial Pooling with
+Region-awareness for Efficient Visual Token Compression in VLMs**, accepted
+at ACM MM 2026.
+
+- Code: https://github.com/ZuyiZhou/CRISPR
+- Paper DOI: https://doi.org/10.1145/3767308.3835007
+
+This repo currently hosts the **Qwen2.5-VL-3B-Instruct** backbone checkpoints
+at 9x and 16x compression. Qwen2.5-VL-7B backbone checkpoints will be added
+in a subsequent update.
+
+## Files
+
+| Path | Compression ratio | Backbone | Notes |
+|---|---|---|---|
+| `3b_9x/checkpoint.pt` | 9x (3x3 block) | Qwen2.5-VL-3B-Instruct | Stage-2, best checkpoint by val loss |
+| `3b_9x/config.json` | | | training config used to produce this checkpoint |
+| `3b_16x/checkpoint.pt` | 16x (4x4 block) | Qwen2.5-VL-3B-Instruct | Stage-2, best checkpoint by val loss |
+| `3b_16x/config.json` | | | training config used to produce this checkpoint |
+
+Each `checkpoint.pt` is a plain `torch.save` dict with keys `config`,
+`token_mixer` (TokenMixer state dict), and `local_c3` (LocalC3 state dict,
+which also contains the Global Token Fusion sub-module). Only the trainable
+CRISPR modules are included — the frozen Qwen2.5-VL vision encoder and
+decoder weights are not part of this checkpoint and must be obtained
+separately from [Qwen2.5-VL](https://github.com/QwenLM/Qwen2.5-VL). Optimizer/
+scheduler state is not included (only the model weights needed for inference
+or further fine-tuning are provided).
+
+## Usage
+
+```python
+from crispr import create_model_v7
+
+model = create_model_v7(decoder_path="./Qwen/Qwen2.5-VL-3B-Instruct")
+model.load_checkpoint("3b_9x/checkpoint.pt")  # see crispr/model_v7.py for the loader
+```
+
+See the main repository (https://github.com/ZuyiZhou/CRISPR) for the model
+code, training script, and evaluation scripts.
+
+## Citation
+
+```bibtex
+@inproceedings{zhou2026crispr,
+  author    = {Zhou, Zuyi and Xue, Dizhan and Qian, Shengsheng and Xu, Changsheng},
+  title     = {CRISPR: Context-Refined Information Spatial Pooling with
+               Region-awareness for Efficient Visual Token Compression in VLMs},
+  booktitle = {Proceedings of the 34th ACM International Conference on
+               Multimedia (MM '26)},
+  year      = {2026},
+  publisher = {Association for Computing Machinery},
+  address   = {New York, NY, USA},
+  doi       = {10.1145/3767308.3835007}
+}
+```