I think that example is very specific to multi-label classification. @praveenNathan if you want to continue the "pre-training" and use it for zero-shot classification (or create better vector representations for your specific domain) you can check open-clip implementation https://github.com/mlfoundations/open_clip ... they have the losses there for v1