Instructions to use microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- OpenCLIP
How to use microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224 with OpenCLIP:
import open_clip model, preprocess_train, preprocess_val = open_clip.create_model_and_transforms('hf-hub:microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224') tokenizer = open_clip.get_tokenizer('hf-hub:microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224') - Notebooks
- Google Colab
- Kaggle
How to obtain the sequence output of Visual Encoder?
#8
by HeyQ8747 - opened
I wanted to get the sequence output from the Visual Encoder, but found that open_clip can only output cls token. I tried to rewrite the forward function myself but didn't get the same result.
Hmm, it should be doable, but I'm not sure of the best way off the top of my head. Can you open an issue on the main open_clip repo and link here?
HeyQ8747 changed discussion status to closed