narugo1992 commited on
Update README.md
Browse files
README.md
CHANGED
|
@@ -14,6 +14,43 @@ tags:
|
|
| 14 |
|
| 15 |
Models of experiment: https://github.com/deepghs/tagger_embedding_aligner
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
| Name | Tagger | Embedding Width | Tags Count | FLOPS | Params | EMB Cosine | EMB Norm | Pred Loss | Pred MSE |
|
| 18 |
|:---------------------:|:-----------:|:-----------------:|:------------:|:---------:|:--------:|:------------:|:----------:|:-----------:|:----------:|
|
| 19 |
| ViT_v3_mnum2_all | ViT_v3 | 768 | 10861 | 0.000398G | 0.40M | 1 | 0.1712 | 0.004306 | 2.116e-08 |
|
|
|
|
| 14 |
|
| 15 |
Models of experiment: https://github.com/deepghs/tagger_embedding_aligner
|
| 16 |
|
| 17 |
+
```python
|
| 18 |
+
import numpy as np
|
| 19 |
+
|
| 20 |
+
from imgutils.tagging import get_wd14_tags, convert_wd14_emb_to_prediction, denormalize_wd14_emb
|
| 21 |
+
from test.testings import get_testfile
|
| 22 |
+
|
| 23 |
+
embedding, (r, g, c) = get_wd14_tags(
|
| 24 |
+
get_testfile('nian.png'),
|
| 25 |
+
fmt=('embedding', ('rating', 'general', 'character')),
|
| 26 |
+
)
|
| 27 |
+
# normal tag results
|
| 28 |
+
print('Expected result:')
|
| 29 |
+
print(r)
|
| 30 |
+
print(g)
|
| 31 |
+
print(c)
|
| 32 |
+
|
| 33 |
+
# normalize embedding
|
| 34 |
+
embedding = embedding / np.linalg.norm(embedding)
|
| 35 |
+
# bad tag results
|
| 36 |
+
br, bg, bc = convert_wd14_emb_to_prediction(embedding)
|
| 37 |
+
print('Bad results due to the embedding normalization:')
|
| 38 |
+
print(br)
|
| 39 |
+
print(bg)
|
| 40 |
+
print(bc)
|
| 41 |
+
|
| 42 |
+
# denormalize this embedding
|
| 43 |
+
output = denormalize_wd14_emb(embedding)
|
| 44 |
+
print(output.shape)
|
| 45 |
+
|
| 46 |
+
# should be similar to r, g, c, approx 1e-3 error
|
| 47 |
+
rating, general, character = convert_wd14_emb_to_prediction(output)
|
| 48 |
+
print('De-normalized result:')
|
| 49 |
+
print(rating)
|
| 50 |
+
print(general)
|
| 51 |
+
print(character)
|
| 52 |
+
```
|
| 53 |
+
|
| 54 |
| Name | Tagger | Embedding Width | Tags Count | FLOPS | Params | EMB Cosine | EMB Norm | Pred Loss | Pred MSE |
|
| 55 |
|:---------------------:|:-----------:|:-----------------:|:------------:|:---------:|:--------:|:------------:|:----------:|:-----------:|:----------:|
|
| 56 |
| ViT_v3_mnum2_all | ViT_v3 | 768 | 10861 | 0.000398G | 0.40M | 1 | 0.1712 | 0.004306 | 2.116e-08 |
|