Update README.md
Browse files
README.md
CHANGED
|
@@ -98,6 +98,25 @@ AbstractPhil/gated-david/
|
|
| 98 |
|
| 99 |
### Loading the Model
|
| 100 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 101 |
```python
|
| 102 |
from geovocab2.train.model.core.david import David, DavidArchitectureConfig
|
| 103 |
from huggingface_hub import hf_hub_download
|
|
@@ -135,23 +154,10 @@ david.load_state_dict(load_file(weights_path))
|
|
| 135 |
david.eval()
|
| 136 |
```
|
| 137 |
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
```python
|
| 141 |
-
import torch
|
| 142 |
-
import torch.nn.functional as F
|
| 143 |
-
|
| 144 |
-
# Assuming you have CLIP features (512-dim for ViT-B/16)
|
| 145 |
-
features = get_clip_features(image) # [1, 512]
|
| 146 |
|
| 147 |
-
|
| 148 |
-
anchors_dict = torch.load("anchors.pth")
|
| 149 |
|
| 150 |
-
# Forward pass
|
| 151 |
-
with torch.no_grad():
|
| 152 |
-
logits, _ = david(features, anchors_dict)
|
| 153 |
-
predictions = logits.argmax(dim=-1)
|
| 154 |
-
```
|
| 155 |
|
| 156 |
## Architecture Overview
|
| 157 |
|
|
|
|
| 98 |
|
| 99 |
### Loading the Model
|
| 100 |
|
| 101 |
+
The description below is terrible.
|
| 102 |
+
|
| 103 |
+
Load a clip-vit and then load one of the pretrains. There is no anchors file nor any of that anymore.
|
| 104 |
+
|
| 105 |
+
I don't know why this piece of the readme got preserved, but I'll need to fix it.
|
| 106 |
+
|
| 107 |
+
1. Install the repo in your environment
|
| 108 |
+
|
| 109 |
+
```python
|
| 110 |
+
try:
|
| 111 |
+
!pip uninstall -qy geometricvocab
|
| 112 |
+
except:
|
| 113 |
+
pass
|
| 114 |
+
|
| 115 |
+
!pip install -q git+https://github.com/AbstractEyes/lattice_vocabulary.git
|
| 116 |
+
```
|
| 117 |
+
|
| 118 |
+
Find your target model, likely not all of this below will work currently.
|
| 119 |
+
|
| 120 |
```python
|
| 121 |
from geovocab2.train.model.core.david import David, DavidArchitectureConfig
|
| 122 |
from huggingface_hub import hf_hub_download
|
|
|
|
| 154 |
david.eval()
|
| 155 |
```
|
| 156 |
|
| 157 |
+
3. When in eval mode pass the output from clip-vit's feature into david. The feature does not require the token nor the label, simply pass the feature in and assess the outputs.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 158 |
|
| 159 |
+
I'll need to ensure this process is perfected later, so the trainings can actually be utilized.
|
|
|
|
| 160 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 161 |
|
| 162 |
## Architecture Overview
|
| 163 |
|