Initial upload: LibreCLIP zero-shot weights
Browse files
LICENSE
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2021 OpenAI
|
| 4 |
+
Copyright (c) 2012-2021 OpenCLIP authors
|
| 5 |
+
|
| 6 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 7 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 8 |
+
in the Software without restriction, including without limitation the rights
|
| 9 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 10 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 11 |
+
furnished to do so, subject to the following conditions:
|
| 12 |
+
|
| 13 |
+
The above copyright notice and this permission notice shall be included in all
|
| 14 |
+
copies or substantial portions of the Software.
|
| 15 |
+
|
| 16 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 17 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 18 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 19 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 20 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 21 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 22 |
+
SOFTWARE.
|
LibreCLIPb16-cls.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a57c2f5bcca165091689c6b4171d0be9ae11d00999b64416a18f568ded66e2d8
|
| 3 |
+
size 598618871
|
NOTICE
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
LibreCLIP weights
|
| 2 |
+
-----------------
|
| 3 |
+
|
| 4 |
+
This product contains weights derived from OpenCLIP
|
| 5 |
+
(https://github.com/mlfoundations/open_clip), built on OpenAI CLIP
|
| 6 |
+
(https://github.com/openai/CLIP).
|
| 7 |
+
Copyright (c) 2021 OpenAI; (c) 2012-2021 OpenCLIP authors.
|
| 8 |
+
Licensed under the MIT License.
|
| 9 |
+
|
| 10 |
+
Source weights: laion/CLIP-ViT-B-16-laion2B-s34B-b88K
|
| 11 |
+
(OpenCLIP arch ViT-B-16, pretrained tag laion2b_s34b_b88k).
|
| 12 |
+
|
| 13 |
+
Data provenance: trained on LAION-2B, which had a documented CSAM-content
|
| 14 |
+
history (Stanford, December 2023). LAION subsequently released Re-LAION, a
|
| 15 |
+
cleaned re-release. Prefer Re-LAION-derived weights where available. LibreCLIP
|
| 16 |
+
does not ship OpenAI-WIT (undisclosed-data) or any non-commercial CLIP weights.
|
README.md
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
library_name: libreyolo
|
| 4 |
+
pipeline_tag: zero-shot-image-classification
|
| 5 |
+
tags:
|
| 6 |
+
- clip
|
| 7 |
+
- zero-shot-image-classification
|
| 8 |
+
- open-vocabulary
|
| 9 |
+
- libreyolo
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# LibreCLIPb16-cls
|
| 13 |
+
|
| 14 |
+
OpenCLIP ViT-B/16 (LAION-2B), repackaged as a native LibreYOLO checkpoint for
|
| 15 |
+
**zero-shot, open-vocabulary** image classification with `LibreCLIP`. No
|
| 16 |
+
training and no fixed label set: call `set_classes([...])`, then predict.
|
| 17 |
+
|
| 18 |
+
## Source
|
| 19 |
+
|
| 20 |
+
Derived from [laion/CLIP-ViT-B-16-laion2B-s34B-b88K](https://huggingface.co/laion/CLIP-ViT-B-16-laion2B-s34B-b88K)
|
| 21 |
+
(OpenCLIP arch `ViT-B-16`, pretrained tag `laion2b_s34b_b88k`).
|
| 22 |
+
Copyright (c) 2021 OpenAI; (c) 2012-2021 OpenCLIP authors. Licensed under the
|
| 23 |
+
MIT License.
|
| 24 |
+
|
| 25 |
+
## Data provenance
|
| 26 |
+
|
| 27 |
+
These weights were trained on LAION-2B, which has a documented CSAM-content
|
| 28 |
+
history (Stanford, December 2023); LAION subsequently released the cleaned
|
| 29 |
+
Re-LAION. Prefer Re-LAION-derived weights where available. See `NOTICE`.
|
| 30 |
+
|
| 31 |
+
## Modifications
|
| 32 |
+
|
| 33 |
+
State-dict key remapping only — LibreCLIP's native towers mirror the OpenCLIP
|
| 34 |
+
module structure, so the load is 0-missing / 0-unexpected. Learned parameters
|
| 35 |
+
are unchanged. See `weights/convert_clip_weights.py` in the
|
| 36 |
+
[LibreYOLO source repository](https://github.com/LibreYOLO/libreyolo).
|
| 37 |
+
|
| 38 |
+
## Usage
|
| 39 |
+
|
| 40 |
+
```python
|
| 41 |
+
from libreyolo import LibreCLIP
|
| 42 |
+
|
| 43 |
+
model = LibreCLIP("LibreCLIPb16-cls.pt") # autodownloads from this repo
|
| 44 |
+
model.set_classes(["a forklift", "an empty aisle", "a spill"])
|
| 45 |
+
r = model.predict("warehouse.jpg")[0]
|
| 46 |
+
print(model.names[r.probs.top1], float(r.probs.top1conf))
|
| 47 |
+
```
|
| 48 |
+
|
| 49 |
+
## License
|
| 50 |
+
|
| 51 |
+
MIT License. See the [`LICENSE`](./LICENSE) and [`NOTICE`](./NOTICE) files.
|