Commit ·
c60ac86
1
Parent(s): b0f1be3
Update README.md
Browse files
README.md
CHANGED
|
@@ -2,8 +2,11 @@
|
|
| 2 |
tags:
|
| 3 |
- vision
|
| 4 |
- coin
|
|
|
|
| 5 |
- coin-retrieval
|
| 6 |
- coin-recognition
|
|
|
|
|
|
|
| 7 |
widget:
|
| 8 |
- src: >-
|
| 9 |
https://huggingface.co/datasets/mishig/sample_images/resolve/main/cat-dog-music.png
|
|
@@ -13,13 +16,23 @@ license: apache-2.0
|
|
| 13 |
library_name: transformers
|
| 14 |
---
|
| 15 |
|
| 16 |
-
#
|
| 17 |
|
| 18 |
## Model Details / 模型细节
|
| 19 |
|
| 20 |
-
This model
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
-
这个模型是在 OpenAI 的 CLIP (ViT-B/32) 基础上,利用对比学习技术并使用硬币数据集进行微调得到的。它旨在提高硬币图像的特征提取能力,从而实现更准确的以图搜图功能。该模型结合了视觉变换器(ViT)的强大功能和 CLIP 的多模态学习能力,专门针对硬币图像进行了优化。
|
| 23 |
|
| 24 |
|
| 25 |
## Comparison: Coin-CLIP vs. CLIP / 效果对比
|
|
@@ -57,6 +70,7 @@ More examples can be found: [breezedeus/Coin-CLIP: Coin CLIP](https://github.com
|
|
| 57 |
|
| 58 |
## Model Use / 模型使用
|
| 59 |
|
|
|
|
| 60 |
```python3
|
| 61 |
from PIL import Image
|
| 62 |
import requests
|
|
@@ -74,6 +88,32 @@ img_features = model.get_image_features(**inputs)
|
|
| 74 |
img_features = F.normalize(img_features, dim=1)
|
| 75 |
```
|
| 76 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
|
| 78 |
|
| 79 |
## Training Data / 训练数据
|
|
|
|
| 2 |
tags:
|
| 3 |
- vision
|
| 4 |
- coin
|
| 5 |
+
- clip
|
| 6 |
- coin-retrieval
|
| 7 |
- coin-recognition
|
| 8 |
+
- coin-search-engine
|
| 9 |
+
- multi-modal learning
|
| 10 |
widget:
|
| 11 |
- src: >-
|
| 12 |
https://huggingface.co/datasets/mishig/sample_images/resolve/main/cat-dog-music.png
|
|
|
|
| 16 |
library_name: transformers
|
| 17 |
---
|
| 18 |
|
| 19 |
+
# Coin-CLIP 🪙 : Enhancing Coin Image Retrieval with CLIP
|
| 20 |
|
| 21 |
## Model Details / 模型细节
|
| 22 |
|
| 23 |
+
This model (**Coin-CLIP**) is built upon
|
| 24 |
+
OpenAI's **[CLIP](https://huggingface.co/openai/clip-vit-base-patch32) (ViT-B/32)** model and fine-tuned on
|
| 25 |
+
a dataset of more than `340,000` coin images using contrastive learning techniques. This specialized model is designed to significantly improve feature extraction for coin images, leading to more accurate image-based search capabilities. Coin-CLIP combines the power of Visual Transformer (ViT) with CLIP's multimodal learning capabilities, specifically tailored for the numismatic domain.
|
| 26 |
+
|
| 27 |
+
**Key Features:**
|
| 28 |
+
- State-of-the-art coin image retrieval;
|
| 29 |
+
- Enhanced feature extraction for numismatic images;
|
| 30 |
+
- Seamless integration with CLIP's multimodal learning.
|
| 31 |
+
|
| 32 |
+
本模型(**Coin-CLIP**)
|
| 33 |
+
在 OpenAI 的 **[CLIP](https://huggingface.co/openai/clip-vit-base-patch32) (ViT-B/32)** 模型基础上,利用对比学习技术在超过 `340,000` 张硬币图片数据上微调得到的。
|
| 34 |
+
**Coin-CLIP** 旨在提高模型针对硬币图片的特征提取能力,从而实现更准确的以图搜图功能。该模型结合了视觉变换器(ViT)的强大功能和 CLIP 的多模态学习能力,并专门针对硬币图片进行了优化。
|
| 35 |
|
|
|
|
| 36 |
|
| 37 |
|
| 38 |
## Comparison: Coin-CLIP vs. CLIP / 效果对比
|
|
|
|
| 70 |
|
| 71 |
## Model Use / 模型使用
|
| 72 |
|
| 73 |
+
### Transformers
|
| 74 |
```python3
|
| 75 |
from PIL import Image
|
| 76 |
import requests
|
|
|
|
| 88 |
img_features = F.normalize(img_features, dim=1)
|
| 89 |
```
|
| 90 |
|
| 91 |
+
### Tool / 工具
|
| 92 |
+
|
| 93 |
+
To further simplify the use of the **Coin-CLIP** model, we provide a simple Python library [breezedeus/Coin-CLIP: Coin CLIP](https://github.com/breezedeus/Coin-CLIP) for quickly building a coin image retrieval engine.
|
| 94 |
+
|
| 95 |
+
为了进一步简化 **Coin-CLIP** 模型的使用,我们提供了一个简单的 Python 库 [breezedeus/Coin-CLIP: Coin CLIP](https://github.com/breezedeus/Coin-CLIP),以便快速构建硬币图像检索引擎。
|
| 96 |
+
|
| 97 |
+
#### Install
|
| 98 |
+
|
| 99 |
+
```bash
|
| 100 |
+
pip install coin_clip
|
| 101 |
+
```
|
| 102 |
+
|
| 103 |
+
|
| 104 |
+
#### Extract Feature Vectors
|
| 105 |
+
|
| 106 |
+
```python
|
| 107 |
+
from coin_clip import CoinClip
|
| 108 |
+
|
| 109 |
+
# Automatically download the model from Huggingface
|
| 110 |
+
model = CoinClip(model_name='breezedeus/coin-clip-vit-base-patch32')
|
| 111 |
+
images = ['examples/10_back.jpg', 'examples/16_back.jpg']
|
| 112 |
+
img_feats, success_ids = model.get_image_features(images)
|
| 113 |
+
print(img_feats.shape) # --> (2, 512)
|
| 114 |
+
```
|
| 115 |
+
|
| 116 |
+
More Tools can be found: [breezedeus/Coin-CLIP: Coin CLIP](https://github.com/breezedeus/Coin-CLIP) .
|
| 117 |
|
| 118 |
|
| 119 |
## Training Data / 训练数据
|