Files changed (1) hide show
  1. README.md +57 -54
README.md CHANGED
@@ -1,43 +1,43 @@
1
- ---
2
- license:
3
- - mit
4
- language:
5
- - en
6
- library_name: open_clip
7
- model_name: BioCLIP 2
8
- model_description: >-
9
- Foundation model for biology organismal images. It is trained on
10
- TreeOfLife-200M on the basis of a CLIP model (ViT-14/L) pre-trained on
11
- LAION-2B. BioCLIP 2 yields state-of-the-art performance in recognizing various
12
- species. More importantly, it demonstrates emergent properties beyond species
13
- classification after extensive hierarchical contrastive training.
14
- tags:
15
- - biology
16
- - CV
17
- - images
18
- - imageomics
19
- - clip
20
- - species-classification
21
- - biological visual task
22
- - multimodal
23
- - animals
24
- - plants
25
- - fungi
26
- - species
27
- - taxonomy
28
- - rare species
29
- - endangered species
30
- - evolutionary biology
31
- - knowledge-guided
32
- - zero-shot-image-classification
33
- datasets:
34
- - imageomics/TreeOfLife-200M
35
- - GBIF
36
- - bioscan-ml/BIOSCAN-5M
37
- - EOL
38
- - FathomNet
39
- new_version: imageomics/bioclip-2.5-vith14
40
- ---
41
 
42
  <!--
43
  Image with caption (jpg or png):
@@ -77,7 +77,7 @@ We evaluate BioCLIP 2 on a diverse set of biological tasks. Through training at
77
 
78
  - **Homepage:** [BioCLIP 2 Project Page](https://imageomics.github.io/bioclip-2/)
79
  - **Repository:** [BioCLIP 2](https://github.com/Imageomics/bioclip-2)
80
- - **Paper:** [BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning](https://doi.org/10.48550/arXiv.2505.23883)
81
  - **Demo:** [BioCLIP 2 Demo](https://huggingface.co/spaces/imageomics/bioclip-2-demo)
82
 
83
  ## Uses
@@ -163,10 +163,10 @@ Then we tested the classifier with 18,901 images from the test set. Accuracy is
163
  * [Herbarium19](https://www.kaggle.com/c/herbarium-2019-fgvc6): This is task to discover new species. We implement it as semi-supervised clustering. Clustering accuracy is calculated for the predictions on both seen and unseen classes.
164
  * [PlantDoc](https://github.com/pratikkayal/PlantDoc-Dataset): 2,598 images of 13 plant species and up to 17 classes of diseases are included in this dataset. We conducted the experiment in a multi-fold 1-shot learning fashion. Average accuracy over the test samples is reported.
165
 
166
- More details regarding the evaluation implementation can be referred to in the [paper](https://doi.org/10.48550/arXiv.2505.23883).
167
 
168
  ### Results
169
- We show the zero-shot classification and non-species classification task results here. For more detailed results, please check the [paper](https://doi.org/10.48550/arXiv.2505.23883).
170
  <table cellpadding="0" cellspacing="0">
171
  <thead>
172
  <tr>
@@ -353,7 +353,7 @@ Notably, BioCLIP 2 yields a 10.2% performance gap over DINOv2, which is broadly
353
 
354
  ## Model Examination
355
 
356
- Please check Section 5.4 of our [paper](https://doi.org/10.48550/arXiv.2505.23883), where we provide formal analysis for the emergent properties of BioCLIP 2.
357
 
358
  ## Technical Specifications
359
 
@@ -365,30 +365,33 @@ It took 10 days to complete the training of 30 epochs.
365
  ## Citation
366
 
367
  **BibTeX:**
368
- ```
369
  @software{Gu_BioCLIP_2_model,
370
  author = {Jianyang Gu and Samuel Stevens and Elizabeth G Campolongo and Matthew J Thompson and Net Zhang and Jiaman Wu and Andrei Kopanev and Zheda Mai and Alexander E. White and James Balhoff and Wasila M Dahdul and Daniel Rubenstein and Hilmar Lapp and Tanya Berger-Wolf and Wei-Lun Chao and Yu Su},
371
  license = {MIT},
372
  title = {{BioCLIP 2}},
373
  url = {https://huggingface.co/imageomics/bioclip-2},
374
  version = {1.0.0},
375
- doi = {},
376
  publisher = {Hugging Face},
377
  year = {2025}
378
  }
379
  ```
380
  Please also cite our paper:
381
  ```
382
- @article{gu2025bioclip,
383
- title = {{B}io{CLIP} 2: Emergent Properties from Scaling Hierarchical Contrastive Learning},
384
- author = {Jianyang Gu and Samuel Stevens and Elizabeth G Campolongo and Matthew J Thompson and Net Zhang and Jiaman Wu and Andrei Kopanev and Zheda Mai and Alexander E. White and James Balhoff and Wasila M Dahdul and Daniel Rubenstein and Hilmar Lapp and Tanya Berger-Wolf and Wei-Lun Chao and Yu Su},
385
- year = {2025},
386
- eprint={2505.23883},
387
- archivePrefix={arXiv},
388
- primaryClass={cs.CV},
389
- url={https://arxiv.org/abs/2505.23883},
 
 
390
  }
391
  ```
 
392
 
393
  Also consider citing OpenCLIP and BioCLIP:
394
 
 
1
+ ---
2
+ license:
3
+ - mit
4
+ language:
5
+ - en
6
+ library_name: open_clip
7
+ model_name: BioCLIP 2
8
+ model_description: >-
9
+ Foundation model for biology organismal images. It is trained on
10
+ TreeOfLife-200M on the basis of a CLIP model (ViT-14/L) pre-trained on
11
+ LAION-2B. BioCLIP 2 yields state-of-the-art performance in recognizing various
12
+ species. More importantly, it demonstrates emergent properties beyond species
13
+ classification after extensive hierarchical contrastive training.
14
+ tags:
15
+ - biology
16
+ - CV
17
+ - images
18
+ - imageomics
19
+ - clip
20
+ - species-classification
21
+ - biological visual task
22
+ - multimodal
23
+ - animals
24
+ - plants
25
+ - fungi
26
+ - species
27
+ - taxonomy
28
+ - rare species
29
+ - endangered species
30
+ - evolutionary biology
31
+ - knowledge-guided
32
+ - zero-shot-image-classification
33
+ datasets:
34
+ - imageomics/TreeOfLife-200M
35
+ - GBIF
36
+ - bioscan-ml/BIOSCAN-5M
37
+ - EOL
38
+ - FathomNet
39
+ new_version: imageomics/bioclip-2.5-vith14
40
+ ---
41
 
42
  <!--
43
  Image with caption (jpg or png):
 
77
 
78
  - **Homepage:** [BioCLIP 2 Project Page](https://imageomics.github.io/bioclip-2/)
79
  - **Repository:** [BioCLIP 2](https://github.com/Imageomics/bioclip-2)
80
+ - **Paper:** [BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning](https://proceedings.neurips.cc/paper_files/paper/2025/file/94da80cbfe870c1db958c88a8a27018c-Paper-Conference.pdf) <!-- arXiv: https://doi.org/10.48550/arXiv.2505.23883 -->
81
  - **Demo:** [BioCLIP 2 Demo](https://huggingface.co/spaces/imageomics/bioclip-2-demo)
82
 
83
  ## Uses
 
163
  * [Herbarium19](https://www.kaggle.com/c/herbarium-2019-fgvc6): This is task to discover new species. We implement it as semi-supervised clustering. Clustering accuracy is calculated for the predictions on both seen and unseen classes.
164
  * [PlantDoc](https://github.com/pratikkayal/PlantDoc-Dataset): 2,598 images of 13 plant species and up to 17 classes of diseases are included in this dataset. We conducted the experiment in a multi-fold 1-shot learning fashion. Average accuracy over the test samples is reported.
165
 
166
+ More details regarding the evaluation implementation can be referred to in the [paper](https://proceedings.neurips.cc/paper_files/paper/2025/file/94da80cbfe870c1db958c88a8a27018c-Paper-Conference.pdf).
167
 
168
  ### Results
169
+ We show the zero-shot classification and non-species classification task results here. For more detailed results, please check the [paper](https://proceedings.neurips.cc/paper_files/paper/2025/file/94da80cbfe870c1db958c88a8a27018c-Paper-Conference.pdf).
170
  <table cellpadding="0" cellspacing="0">
171
  <thead>
172
  <tr>
 
353
 
354
  ## Model Examination
355
 
356
+ Please check Section 5.4 of our [paper](https://proceedings.neurips.cc/paper_files/paper/2025/file/94da80cbfe870c1db958c88a8a27018c-Paper-Conference.pdf), where we provide formal analysis for the emergent properties of BioCLIP 2.
357
 
358
  ## Technical Specifications
359
 
 
365
  ## Citation
366
 
367
  **BibTeX:**
368
+ ```
369
  @software{Gu_BioCLIP_2_model,
370
  author = {Jianyang Gu and Samuel Stevens and Elizabeth G Campolongo and Matthew J Thompson and Net Zhang and Jiaman Wu and Andrei Kopanev and Zheda Mai and Alexander E. White and James Balhoff and Wasila M Dahdul and Daniel Rubenstein and Hilmar Lapp and Tanya Berger-Wolf and Wei-Lun Chao and Yu Su},
371
  license = {MIT},
372
  title = {{BioCLIP 2}},
373
  url = {https://huggingface.co/imageomics/bioclip-2},
374
  version = {1.0.0},
375
+ doi = {10.57967/hf/5765},
376
  publisher = {Hugging Face},
377
  year = {2025}
378
  }
379
  ```
380
  Please also cite our paper:
381
  ```
382
+ @inproceedings{NEURIPS2025_94da80cb,
383
+ author = {Gu, Jianyang and Stevens, Sam and Campolongo, Elizabeth and Thompson, Matthew and Zhang, Net and Wu, Jiaman and Kopanev, Andrei and Mai, Zheda and White, Alexander and Balhoff, James and Dahdul, Wasila and Rubenstein, Daniel and Lapp, Hilmar and Berger-Wolf, Tanya and Chao, Wei-Lun (Harry) and Su, Yu},
384
+ booktitle = {Advances in Neural Information Processing Systems},
385
+ editor = {D. Belgrave and C. Zhang and H. Lin and R. Pascanu and P. Koniusz and M. Ghassemi and N. Chen},
386
+ pages = {102778--102811},
387
+ publisher = {Curran Associates, Inc.},
388
+ title = {BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning},
389
+ url = {https://proceedings.neurips.cc/paper_files/paper/2025/file/94da80cbfe870c1db958c88a8a27018c-Paper-Conference.pdf},
390
+ volume = {38},
391
+ year = {2025}
392
  }
393
  ```
394
+ <!-- https://arxiv.org/abs/2505.23883-->
395
 
396
  Also consider citing OpenCLIP and BioCLIP:
397