Instructions to use zer0int/LongCLIP-GmP-ViT-L-14 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use zer0int/LongCLIP-GmP-ViT-L-14 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-image-classification", model="zer0int/LongCLIP-GmP-ViT-L-14") pipe( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png", candidate_labels=["animals", "humans", "landscape"], )# Load model directly from transformers import AutoProcessor, AutoModelForZeroShotImageClassification processor = AutoProcessor.from_pretrained("zer0int/LongCLIP-GmP-ViT-L-14") model = AutoModelForZeroShotImageClassification.from_pretrained("zer0int/LongCLIP-GmP-ViT-L-14") - Notebooks
- Google Colab
- Kaggle
Is the SeaArtLab node necessary?
I tested Long-ViT-L-14-GmP/Long-ViT-L-14-BEST-GmP-smooth in DualCLIPLoader + the T5 clip
and Long-ViT-L-14-GmP/Long-ViT-L-14-BEST-GmP-smooth in SeaArtLab node with Long-ViT-L-14-GmP/Long-ViT-L-14-BEST-GmP-smooth + t5 in DualCLIPLoader
There was not much of a difference between only DualCLIPLoader and SeaArtLab + DualCLIPLoader
I noticed only Long-ViT-L-14-GmP + t5 in DualCLIPLoader gives better results for human faces, while if adding the SeaArtLab node too it will focus more on the background, take details away from the face/body and putting them into background objects
Does the DualCLIPLoader node already support the Long-CLIP finetune or the difference between 77 tokens and 248 tokens is not that significant?
Oh, yeah, you're absolutely right - when I made the ComfyUI node initially, ComfyUI didn't natively support 248-tokens Long-CLIP. But now it does! So, you can remove the SeaArt node, you don't need it anymore. :)
There may be small difference due to internal handling of loading and weighting of prompts that ComfyUI uses, but you should be able to do any of that just using ComfyUI's internal mechanisms.