COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training
Paper • 2412.01814 • Published • 1
Image-Text-to-Text
Transformers
vision
vision-language-model
contrastive learning
self-supervised learning
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("sankim2/cosmos", dtype="auto")Quick Links
[CVPR 2025] COSMOS Model
Authors: Sanghwan Kim, Rui Xiao, Mariana-Iuliana Georgescu, Stephan Alaniz, Zeynep Akata
COSMOS is introduced in the paper COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training. COSMOS is trained in self-supervised learning framework with multi-modal augmentation and cross-attention module. It outperforms CLIP-based models trained on larger datasets in visual perception and contextual understanding tasks. COSMOS also achieves strong performance in downstream tasks including zero-shot image-text retrieval, classification, and semantic segmentation.
Usage
Please refer to our Github repo for detailed usage.
Citation
If you find our work useful, please consider citing:
@article{kim2024cosmos,
title={COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training},
author={Kim, Sanghwan and Xiao, Rui and Georgescu, Mariana-Iuliana and Alaniz, Stephan and Akata, Zeynep},
journal={arXiv preprint arXiv:2412.01814},
year={2024}
}
- Downloads last month
- 4
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="sankim2/cosmos")