CLIP_VITB32 / README.md
tillmannohm's picture
Upload 3 files
565db07 verified
|
Raw
History Blame Contribute Delete
3.76 kB
---
library_name: onnx
pipeline_tag: feature-extraction
license: mit
tags:
- eidora
- eidora-model-zoo
- onnx
- onnxruntime
- embeddings
- image
- clip
- compute:light
- modality:image
base_model: openai/clip-vit-base-patch32
datasets:
- openai/webimage-text
metrics:
- cosine-similarity
model-index:
- name: CLIP_VITB32_512
results: []
---
# CLIP_VITB32_512
CLIP_VITB32_512 is a light image embedding model for fast visual grouping in EIDORA. Choose it when you want a broadly useful visual model that runs on most laptops.
## Best For
- Fast first-pass visual grouping.
- Image collections where semantic similarity matters more than exact object classification.
- A broadly useful starter model for EIDORA image workflows.
## Not Ideal For
- Fine-grained visual similarity where a heavier model is acceptable.
- Text-only projects.
- Specialized domains that need a domain-trained visual encoder.
## Compute Tier
Light: small download, low memory, faster CPU runtime. Intended for laptop CPU use and large first-pass projects.
## Inputs
- `image`: required image input from `media_source`.
## Output
The primary output is `embedding`, a float32 tensor shaped `[batch, 512]`. Embeddings are already normalized and are intended for cosine similarity.
## Usage In EIDORA
EIDORA shows this package as a light image embedding model in the Model Zoo. Use it for discovery maps, grouping, retrieval, and related embedding workflows.
## Preprocessing
- `image`: resize mode defaults to `crop_center`; rescale uses `1/255`; normalize inside the ONNX graph with mean `[0.48145466, 0.4578275, 0.40821073]` and std `[0.26862954, 0.26130258, 0.27577711]`.
## Authorship And Citation
This ONNX package was produced by EIDORA from the original CLIP ViT-B/32 model. EIDORA converted the model to ONNX and is not the original model creator. Please cite Learning Transferable Visual Models From Natural Language Supervision and the original model repository when using this converted model.
Original model: https://github.com/openai/CLIP
Original paper: https://arxiv.org/abs/2103.00020
Authors: Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever
```bibtex
@inproceedings{radford2021learning,
title={Learning Transferable Visual Models From Natural Language Supervision},
author={Radford, Alec and Kim, Jong Wook and Hallacy, Chris and Ramesh, Aditya and Goh, Gabriel and Agarwal, Sandhini and Sastry, Girish and Askell, Amanda and Mishkin, Pamela and Clark, Jack and Krueger, Gretchen and Sutskever, Ilya},
booktitle={International Conference on Machine Learning},
year={2021}
}
```
## Training Data And Provenance
Base model: `openai/clip-vit-base-patch32`. Source repository: https://huggingface.co/openai/clip-vit-base-patch32. Known training data: WebImageText-style image/text pairs described by the upstream CLIP authors. Package payload size: 351584808 bytes.
## Evaluation And Validation
The package validation checks that the ONNX graph loads with ONNX Runtime CPU execution, runs the declared fixtures, returns finite float32 embeddings with the declared shape, and matches the artifact hash recorded in `config.yaml`.
## Limitations And Safety
CLIP embeddings can reflect web-scale training data biases and may miss fine visual details. Similarity scores should not be used as sole evidence for identity, authorship, intent, or sensitive attributes.
## License And Attribution
This package uses license `mit`. Upstream license: MIT. Converted to ONNX for EIDORA from the original OpenAI CLIP model.
## Version
Package version: 1.0.0. ONNX opset: 17. Exporter: eidora-onnx-exporter 0.1.0.