File size: 2,018 Bytes
c597de1 99cde86 76302bb 99cde86 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
license: cc
datasets:
- Zilun/RS5M
language:
- en
metrics:
- accuracy
- recall
---
## GeoRSCLIP Model
* **GeoRSCLIP with ViT-B-32 and ViT-H-14 backbone**
* **GeoRSCLIP-FT for retrieval**
### Installation
* Install Pytorch following instructions from the official website (We tested in torch 2.0.1 with CUDA 11.8 and 2.1.0 with CUDA 12.1)
```bash
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
```
* Install other dependencies
```bash
pip install pillow pandas scikit-learn ftfy tqdm matplotlib transformers adapter-transformers open_clip_torch pycocotools timm clip-benchmark torch-rs
```
### Usage
* Clone the repo from: https://huggingface.co/Zilun/GeoRSCLIP
```bash
git clone https://huggingface.co/Zilun/GeoRSCLIP
cd GeoRSCLIP
```
* Unzip the test data
```bash
unzip data/rs5m_test_data.zip
```
* Run the inference script:
```bash
python codebase/inference.py --ckpt-path /your/local/path/to/RS5M_ViT-B-32.pt --test-dataset-dir /your/local/path/to/rs5m_test_data
```
* (Optional) If you just want to load the GeoRSCLIP model:
```python
import open_clip
import torch
from inference_tool import get_preprocess
ckpt_path = "/your/local/path/to/RS5M_ViT-B-32.pt"
model, _, _ = open_clip.create_model_and_transforms("ViT-B/32", pretrained="openai")
checkpoint = torch.load(ckpt_path, map_location="cpu")
msg = model.load_state_dict(checkpoint, strict=False)
model = model.to("cuda")
img_preprocess = get_preprocess(
image_resolution=224,
)
```
```python
import open_clip
import torch
from inference_tool import get_preprocess
ckpt_path = "/your/local/path/to/RS5M_ViT-H-14.pt"
model, _, _ = open_clip.create_model_and_transforms("ViT-H/14", pretrained="laion2b_s32b_b79k")
checkpoint = torch.load(ckpt_path, map_location="cpu")
msg = model.load_state_dict(checkpoint, strict=False)
model = model.to("cuda")
img_preprocess = get_preprocess(
image_resolution=224,
)
``` |