Is this only the ContrastiveLoss finetuning? Did you use the Coarse-grained alignment loss proposed in LongClip?
#4
by cuifeng - opened
Is this only the ContrastiveLoss finetuning? Did you use the Coarse-grained alignment loss proposed in LongClip?
I have used the LongCLIP-L checkpoint kindly provided by the researchers of the Long-CLIP paper (starting from their model, not from OpenAI's pre-trained CLIP). However, I indeed then just used a "classic" contrastive loss to continue fine-tuning the model. You can find the code I used here: https://github.com/zer0int/Long-CLIP - feel free to modify & submit a pull request, if you'd like!