Update model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: lucid
|
| 3 |
+
license: mit
|
| 4 |
+
tags:
|
| 5 |
+
- base
|
| 6 |
+
- CLIP
|
| 7 |
+
- lucid
|
| 8 |
+
datasets:
|
| 9 |
+
- WIT-400M
|
| 10 |
+
pipeline_tag: feature-extraction
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# Learning Transferable Visual Models From Natural Language Supervision
|
| 14 |
+
|
| 15 |
+
> https://arxiv.org/abs/2103.00020
|
| 16 |
+
|
| 17 |
+
[Lucid](https://github.com/ChanLumerico/lucid) port of `https://huggingface.co/openai/clip-vit-base-patch32`,
|
| 18 |
+
converted to Lucid-native safetensors.
|
| 19 |
+
|
| 20 |
+
## Available weights
|
| 21 |
+
|
| 22 |
+
| Tag | Params | GFLOPs | Size | Source |
|
| 23 |
+
|---|---|---|---|---|
|
| 24 |
+
| `OPENAI_WIT400M` *(default)* | — | — | 577.11 MB | https: |
|
| 25 |
+
|
| 26 |
+
## Usage
|
| 27 |
+
|
| 28 |
+
```python
|
| 29 |
+
import lucid
|
| 30 |
+
import lucid.models as models
|
| 31 |
+
from lucid.models.weights import ClipWeights
|
| 32 |
+
|
| 33 |
+
# default tag
|
| 34 |
+
model = models.clip_vit_base_32(pretrained=True)
|
| 35 |
+
|
| 36 |
+
# explicit tag (enum or string)
|
| 37 |
+
model = models.clip_vit_base_32(weights=ClipWeights.OPENAI_WIT400M)
|
| 38 |
+
model = models.clip_vit_base_32(pretrained="OPENAI_WIT400M")
|
| 39 |
+
|
| 40 |
+
# feed token ids (tokenize with the matching lucid.utils.tokenizer)
|
| 41 |
+
input_ids = lucid.tensor([[101, 7592, 2088, 102]], dtype=lucid.int64)
|
| 42 |
+
out = model(input_ids)
|
| 43 |
+
hidden = out.last_hidden_state # (B, T, hidden_size)
|
| 44 |
+
```
|
| 45 |
+
|
| 46 |
+
## Conversion
|
| 47 |
+
|
| 48 |
+
Converted from `https://huggingface.co/openai/clip-vit-base-patch32` via
|
| 49 |
+
`python -m tools.convert_weights CLIP --tag OPENAI_WIT400M`.
|
| 50 |
+
Key mapping + numerical parity verified against the source.
|
| 51 |
+
|
| 52 |
+
## License
|
| 53 |
+
|
| 54 |
+
`mit` — inherited from the original weights.
|
| 55 |
+
|
| 56 |
+
## Citation
|
| 57 |
+
|
| 58 |
+
```
|
| 59 |
+
@inproceedings{radford2021learning,
|
| 60 |
+
title={Learning Transferable Visual Models From Natural Language Supervision},
|
| 61 |
+
author={Radford, Alec and Kim, Jong Wook and Hallacy, Chris and Ramesh, Aditya and Goh, Gabriel and Agarwal, Sandhini and Sastry, Girish and Askell, Amanda and Mishkin, Pamela and Clark, Jack and Krueger, Gretchen and Sutskever, Ilya},
|
| 62 |
+
booktitle={ICML},
|
| 63 |
+
pages={8748--8763},
|
| 64 |
+
year={2021}
|
| 65 |
+
}
|
| 66 |
+
```
|