Instructions to use wisdomik/QuiltNet-B-16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- OpenCLIP
How to use wisdomik/QuiltNet-B-16 with OpenCLIP:
import open_clip model, preprocess_train, preprocess_val = open_clip.create_model_and_transforms('hf-hub:wisdomik/QuiltNet-B-16') tokenizer = open_clip.get_tokenizer('hf-hub:wisdomik/QuiltNet-B-16') - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -9,7 +9,7 @@ library_tag: open_clip
|
|
| 9 |
license: mit
|
| 10 |
widget:
|
| 11 |
- src: >-
|
| 12 |
-
https://
|
| 13 |
candidate_labels: adipose tissue, debris tissue, lymphocytes tissue, mucus tissue, smooth muscle tissue, normal colon mucosa tissue, cancer-associated stroma tissue, colorectal adenocarcinoma epithelium tissue
|
| 14 |
example_title: Tissue phenotyping
|
| 15 |
- src: >-
|
|
@@ -24,11 +24,25 @@ pipeline_tag: zero-shot-image-classification
|
|
| 24 |
---
|
| 25 |
|
| 26 |
|
| 27 |
-
#
|
| 28 |
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
-
A CLIP ViT-B/16 model trained with the Quilt-1M dataset (https://quilt1m.github.io/) using OpenCLIP (https://github.com/mlfoundations/open_clip).
|
| 32 |
|
| 33 |
# Uses
|
| 34 |
|
|
@@ -81,19 +95,3 @@ It is important to note that the results obtained from this function are not int
|
|
| 81 |
In accordance with the privacy policy of Youtube, only Video IDs data is redistributed by us.
|
| 82 |
It is strictly prohibited to redistribute any content apart from the Video IDs.
|
| 83 |
Any distribution carried out must adhere to the laws and regulations applicable in your jurisdiction, including export control laws and embargoes.'
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
# Citation
|
| 88 |
-
|
| 89 |
-
**BibTeX:**
|
| 90 |
-
|
| 91 |
-
Quilt-1M
|
| 92 |
-
```bibtex
|
| 93 |
-
@inproceedings{ikezogwo2023quilt,
|
| 94 |
-
title={Quilt-1M: One Million Image-Text Pairs for Histopathology},
|
| 95 |
-
author={Wisdom O. Ikezogwo, Mehmet S. Seyfioglu, Fatemeh Ghezloo, Dylan Geva , Fatwir S. Mohammed, Pavan K. Anand, Ranjay Krishna, Linda G. Shapiro.},
|
| 96 |
-
year={2023},
|
| 97 |
-
journal={arXiv***},
|
| 98 |
-
}
|
| 99 |
-
```
|
|
|
|
| 9 |
license: mit
|
| 10 |
widget:
|
| 11 |
- src: >-
|
| 12 |
+
https://quilt1m.github.io/img/BREST092.jpg
|
| 13 |
candidate_labels: adipose tissue, debris tissue, lymphocytes tissue, mucus tissue, smooth muscle tissue, normal colon mucosa tissue, cancer-associated stroma tissue, colorectal adenocarcinoma epithelium tissue
|
| 14 |
example_title: Tissue phenotyping
|
| 15 |
- src: >-
|
|
|
|
| 24 |
---
|
| 25 |
|
| 26 |
|
| 27 |
+
## QuiltNet-B-16 Description
|
| 28 |
|
| 29 |
+
QuiltNet-B-16 is a CLIP ViT-B/16 vision-language foundation model trained on the [Quilt-1M](https://quilt1m.github.io/) dataset.
|
| 30 |
+
It can perform various vision-language processing (VLP) tasks such as cross-modal retrieval, image classification, and visual question answering.
|
| 31 |
+
QuiltNet establishes new state of the art in a wide range of standard datasets, and substantially outperforms prior VLP approaches:
|
| 32 |
+
|
| 33 |
+

|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
# Citation
|
| 37 |
+
```bibtex
|
| 38 |
+
@inproceedings{ikezogwo2023quilt,
|
| 39 |
+
title={Quilt-1M: One Million Image-Text Pairs for Histopathology},
|
| 40 |
+
author={Wisdom O. Ikezogwo, Mehmet S. Seyfioglu, Fatemeh Ghezloo, Dylan Geva , Fatwir S. Mohammed, Pavan K. Anand, Ranjay Krishna, Linda G. Shapiro.},
|
| 41 |
+
year={2023},
|
| 42 |
+
journal={arXiv***},
|
| 43 |
+
}
|
| 44 |
+
```
|
| 45 |
|
|
|
|
| 46 |
|
| 47 |
# Uses
|
| 48 |
|
|
|
|
| 95 |
In accordance with the privacy policy of Youtube, only Video IDs data is redistributed by us.
|
| 96 |
It is strictly prohibited to redistribute any content apart from the Video IDs.
|
| 97 |
Any distribution carried out must adhere to the laws and regulations applicable in your jurisdiction, including export control laws and embargoes.'
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|