Zero-Shot Image Classification
OpenCLIP
Safetensors
English
biology
CV
images
imageomics
clip
species-classification
biological visual task
multimodal
animals
plants
fungi
species
taxonomy
rare species
endangered species
evolutionary biology
knowledge-guided
Instructions to use imageomics/bioclip-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- OpenCLIP
How to use imageomics/bioclip-2 with OpenCLIP:
import open_clip model, preprocess_train, preprocess_val = open_clip.create_model_and_transforms('hf-hub:imageomics/bioclip-2') tokenizer = open_clip.get_tokenizer('hf-hub:imageomics/bioclip-2') - Notebooks
- Google Colab
- Kaggle
Update citation
#6
by egrace479 - opened
README.md
CHANGED
|
@@ -1,43 +1,43 @@
|
|
| 1 |
-
---
|
| 2 |
-
license:
|
| 3 |
-
- mit
|
| 4 |
-
language:
|
| 5 |
-
- en
|
| 6 |
-
library_name: open_clip
|
| 7 |
-
model_name: BioCLIP 2
|
| 8 |
-
model_description: >-
|
| 9 |
-
Foundation model for biology organismal images. It is trained on
|
| 10 |
-
TreeOfLife-200M on the basis of a CLIP model (ViT-14/L) pre-trained on
|
| 11 |
-
LAION-2B. BioCLIP 2 yields state-of-the-art performance in recognizing various
|
| 12 |
-
species. More importantly, it demonstrates emergent properties beyond species
|
| 13 |
-
classification after extensive hierarchical contrastive training.
|
| 14 |
-
tags:
|
| 15 |
-
- biology
|
| 16 |
-
- CV
|
| 17 |
-
- images
|
| 18 |
-
- imageomics
|
| 19 |
-
- clip
|
| 20 |
-
- species-classification
|
| 21 |
-
- biological visual task
|
| 22 |
-
- multimodal
|
| 23 |
-
- animals
|
| 24 |
-
- plants
|
| 25 |
-
- fungi
|
| 26 |
-
- species
|
| 27 |
-
- taxonomy
|
| 28 |
-
- rare species
|
| 29 |
-
- endangered species
|
| 30 |
-
- evolutionary biology
|
| 31 |
-
- knowledge-guided
|
| 32 |
-
- zero-shot-image-classification
|
| 33 |
-
datasets:
|
| 34 |
-
- imageomics/TreeOfLife-200M
|
| 35 |
-
- GBIF
|
| 36 |
-
- bioscan-ml/BIOSCAN-5M
|
| 37 |
-
- EOL
|
| 38 |
-
- FathomNet
|
| 39 |
-
new_version: imageomics/bioclip-2.5-vith14
|
| 40 |
-
---
|
| 41 |
|
| 42 |
<!--
|
| 43 |
Image with caption (jpg or png):
|
|
@@ -77,7 +77,7 @@ We evaluate BioCLIP 2 on a diverse set of biological tasks. Through training at
|
|
| 77 |
|
| 78 |
- **Homepage:** [BioCLIP 2 Project Page](https://imageomics.github.io/bioclip-2/)
|
| 79 |
- **Repository:** [BioCLIP 2](https://github.com/Imageomics/bioclip-2)
|
| 80 |
-
- **Paper:** [BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning](https://doi.org/10.48550/arXiv.2505.23883
|
| 81 |
- **Demo:** [BioCLIP 2 Demo](https://huggingface.co/spaces/imageomics/bioclip-2-demo)
|
| 82 |
|
| 83 |
## Uses
|
|
@@ -163,10 +163,10 @@ Then we tested the classifier with 18,901 images from the test set. Accuracy is
|
|
| 163 |
* [Herbarium19](https://www.kaggle.com/c/herbarium-2019-fgvc6): This is task to discover new species. We implement it as semi-supervised clustering. Clustering accuracy is calculated for the predictions on both seen and unseen classes.
|
| 164 |
* [PlantDoc](https://github.com/pratikkayal/PlantDoc-Dataset): 2,598 images of 13 plant species and up to 17 classes of diseases are included in this dataset. We conducted the experiment in a multi-fold 1-shot learning fashion. Average accuracy over the test samples is reported.
|
| 165 |
|
| 166 |
-
More details regarding the evaluation implementation can be referred to in the [paper](https://
|
| 167 |
|
| 168 |
### Results
|
| 169 |
-
We show the zero-shot classification and non-species classification task results here. For more detailed results, please check the [paper](https://
|
| 170 |
<table cellpadding="0" cellspacing="0">
|
| 171 |
<thead>
|
| 172 |
<tr>
|
|
@@ -353,7 +353,7 @@ Notably, BioCLIP 2 yields a 10.2% performance gap over DINOv2, which is broadly
|
|
| 353 |
|
| 354 |
## Model Examination
|
| 355 |
|
| 356 |
-
Please check Section 5.4 of our [paper](https://
|
| 357 |
|
| 358 |
## Technical Specifications
|
| 359 |
|
|
@@ -365,30 +365,33 @@ It took 10 days to complete the training of 30 epochs.
|
|
| 365 |
## Citation
|
| 366 |
|
| 367 |
**BibTeX:**
|
| 368 |
-
```
|
| 369 |
@software{Gu_BioCLIP_2_model,
|
| 370 |
author = {Jianyang Gu and Samuel Stevens and Elizabeth G Campolongo and Matthew J Thompson and Net Zhang and Jiaman Wu and Andrei Kopanev and Zheda Mai and Alexander E. White and James Balhoff and Wasila M Dahdul and Daniel Rubenstein and Hilmar Lapp and Tanya Berger-Wolf and Wei-Lun Chao and Yu Su},
|
| 371 |
license = {MIT},
|
| 372 |
title = {{BioCLIP 2}},
|
| 373 |
url = {https://huggingface.co/imageomics/bioclip-2},
|
| 374 |
version = {1.0.0},
|
| 375 |
-
doi = {},
|
| 376 |
publisher = {Hugging Face},
|
| 377 |
year = {2025}
|
| 378 |
}
|
| 379 |
```
|
| 380 |
Please also cite our paper:
|
| 381 |
```
|
| 382 |
-
@
|
| 383 |
-
|
| 384 |
-
|
| 385 |
-
|
| 386 |
-
|
| 387 |
-
|
| 388 |
-
|
| 389 |
-
|
|
|
|
|
|
|
| 390 |
}
|
| 391 |
```
|
|
|
|
| 392 |
|
| 393 |
Also consider citing OpenCLIP and BioCLIP:
|
| 394 |
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license:
|
| 3 |
+
- mit
|
| 4 |
+
language:
|
| 5 |
+
- en
|
| 6 |
+
library_name: open_clip
|
| 7 |
+
model_name: BioCLIP 2
|
| 8 |
+
model_description: >-
|
| 9 |
+
Foundation model for biology organismal images. It is trained on
|
| 10 |
+
TreeOfLife-200M on the basis of a CLIP model (ViT-14/L) pre-trained on
|
| 11 |
+
LAION-2B. BioCLIP 2 yields state-of-the-art performance in recognizing various
|
| 12 |
+
species. More importantly, it demonstrates emergent properties beyond species
|
| 13 |
+
classification after extensive hierarchical contrastive training.
|
| 14 |
+
tags:
|
| 15 |
+
- biology
|
| 16 |
+
- CV
|
| 17 |
+
- images
|
| 18 |
+
- imageomics
|
| 19 |
+
- clip
|
| 20 |
+
- species-classification
|
| 21 |
+
- biological visual task
|
| 22 |
+
- multimodal
|
| 23 |
+
- animals
|
| 24 |
+
- plants
|
| 25 |
+
- fungi
|
| 26 |
+
- species
|
| 27 |
+
- taxonomy
|
| 28 |
+
- rare species
|
| 29 |
+
- endangered species
|
| 30 |
+
- evolutionary biology
|
| 31 |
+
- knowledge-guided
|
| 32 |
+
- zero-shot-image-classification
|
| 33 |
+
datasets:
|
| 34 |
+
- imageomics/TreeOfLife-200M
|
| 35 |
+
- GBIF
|
| 36 |
+
- bioscan-ml/BIOSCAN-5M
|
| 37 |
+
- EOL
|
| 38 |
+
- FathomNet
|
| 39 |
+
new_version: imageomics/bioclip-2.5-vith14
|
| 40 |
+
---
|
| 41 |
|
| 42 |
<!--
|
| 43 |
Image with caption (jpg or png):
|
|
|
|
| 77 |
|
| 78 |
- **Homepage:** [BioCLIP 2 Project Page](https://imageomics.github.io/bioclip-2/)
|
| 79 |
- **Repository:** [BioCLIP 2](https://github.com/Imageomics/bioclip-2)
|
| 80 |
+
- **Paper:** [BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning](https://proceedings.neurips.cc/paper_files/paper/2025/file/94da80cbfe870c1db958c88a8a27018c-Paper-Conference.pdf) <!-- arXiv: https://doi.org/10.48550/arXiv.2505.23883 -->
|
| 81 |
- **Demo:** [BioCLIP 2 Demo](https://huggingface.co/spaces/imageomics/bioclip-2-demo)
|
| 82 |
|
| 83 |
## Uses
|
|
|
|
| 163 |
* [Herbarium19](https://www.kaggle.com/c/herbarium-2019-fgvc6): This is task to discover new species. We implement it as semi-supervised clustering. Clustering accuracy is calculated for the predictions on both seen and unseen classes.
|
| 164 |
* [PlantDoc](https://github.com/pratikkayal/PlantDoc-Dataset): 2,598 images of 13 plant species and up to 17 classes of diseases are included in this dataset. We conducted the experiment in a multi-fold 1-shot learning fashion. Average accuracy over the test samples is reported.
|
| 165 |
|
| 166 |
+
More details regarding the evaluation implementation can be referred to in the [paper](https://proceedings.neurips.cc/paper_files/paper/2025/file/94da80cbfe870c1db958c88a8a27018c-Paper-Conference.pdf).
|
| 167 |
|
| 168 |
### Results
|
| 169 |
+
We show the zero-shot classification and non-species classification task results here. For more detailed results, please check the [paper](https://proceedings.neurips.cc/paper_files/paper/2025/file/94da80cbfe870c1db958c88a8a27018c-Paper-Conference.pdf).
|
| 170 |
<table cellpadding="0" cellspacing="0">
|
| 171 |
<thead>
|
| 172 |
<tr>
|
|
|
|
| 353 |
|
| 354 |
## Model Examination
|
| 355 |
|
| 356 |
+
Please check Section 5.4 of our [paper](https://proceedings.neurips.cc/paper_files/paper/2025/file/94da80cbfe870c1db958c88a8a27018c-Paper-Conference.pdf), where we provide formal analysis for the emergent properties of BioCLIP 2.
|
| 357 |
|
| 358 |
## Technical Specifications
|
| 359 |
|
|
|
|
| 365 |
## Citation
|
| 366 |
|
| 367 |
**BibTeX:**
|
| 368 |
+
```
|
| 369 |
@software{Gu_BioCLIP_2_model,
|
| 370 |
author = {Jianyang Gu and Samuel Stevens and Elizabeth G Campolongo and Matthew J Thompson and Net Zhang and Jiaman Wu and Andrei Kopanev and Zheda Mai and Alexander E. White and James Balhoff and Wasila M Dahdul and Daniel Rubenstein and Hilmar Lapp and Tanya Berger-Wolf and Wei-Lun Chao and Yu Su},
|
| 371 |
license = {MIT},
|
| 372 |
title = {{BioCLIP 2}},
|
| 373 |
url = {https://huggingface.co/imageomics/bioclip-2},
|
| 374 |
version = {1.0.0},
|
| 375 |
+
doi = {10.57967/hf/5765},
|
| 376 |
publisher = {Hugging Face},
|
| 377 |
year = {2025}
|
| 378 |
}
|
| 379 |
```
|
| 380 |
Please also cite our paper:
|
| 381 |
```
|
| 382 |
+
@inproceedings{NEURIPS2025_94da80cb,
|
| 383 |
+
author = {Gu, Jianyang and Stevens, Sam and Campolongo, Elizabeth and Thompson, Matthew and Zhang, Net and Wu, Jiaman and Kopanev, Andrei and Mai, Zheda and White, Alexander and Balhoff, James and Dahdul, Wasila and Rubenstein, Daniel and Lapp, Hilmar and Berger-Wolf, Tanya and Chao, Wei-Lun (Harry) and Su, Yu},
|
| 384 |
+
booktitle = {Advances in Neural Information Processing Systems},
|
| 385 |
+
editor = {D. Belgrave and C. Zhang and H. Lin and R. Pascanu and P. Koniusz and M. Ghassemi and N. Chen},
|
| 386 |
+
pages = {102778--102811},
|
| 387 |
+
publisher = {Curran Associates, Inc.},
|
| 388 |
+
title = {BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning},
|
| 389 |
+
url = {https://proceedings.neurips.cc/paper_files/paper/2025/file/94da80cbfe870c1db958c88a8a27018c-Paper-Conference.pdf},
|
| 390 |
+
volume = {38},
|
| 391 |
+
year = {2025}
|
| 392 |
}
|
| 393 |
```
|
| 394 |
+
<!-- https://arxiv.org/abs/2505.23883-->
|
| 395 |
|
| 396 |
Also consider citing OpenCLIP and BioCLIP:
|
| 397 |
|