metadata
license: apache-2.0
pipeline_tag: image-text-to-text
tags:
- multimodal
- knowledge-distillation
- compositional-reasoning
- compodistill
CompoDistill-2B
The final CompoDistill student (Qwen1.5-1.8B + SigLIP-so400m), trained with the three-stage DPT -> DFT -> SFT pipeline including visual attention distillation from the LLaVA-style Qwen1.5-4B teacher.
Released with the paper CompoDistill: Attention Distillation for Compositional Reasoning in Multimodal LLMs (arXiv:2510.12184). Training and evaluation code: https://github.com/ptkjw1997/CompoDistill
Usage
import torch
from PIL import Image
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoImageProcessor
repo = "JiwanKim/CompoDistill-2B"
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True,
torch_dtype=torch.float16).to("cuda")
tokenizer = AutoTokenizer.from_pretrained(repo, use_fast=False)
image_processor = AutoImageProcessor.from_pretrained(repo)
image = Image.open("example.jpg")
print(model.chat("What is happening in this image?", tokenizer,
image=image, image_processor=image_processor))
Citation
@article{kim2025compodistill,
title={CompoDistill: Attention Distillation for Compositional Reasoning in Multimodal LLMs},
author={Kim, Jiwan and Kim, Kibum and Seo, Sangwoo and Park, Chanyoung},
journal={arXiv preprint arXiv:2510.12184},
year={2025}
}