goerkemsaylam commited on
Commit
727434b
·
verified ·
1 Parent(s): 7def9fa

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +83 -3
README.md CHANGED
@@ -1,3 +1,83 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - behAIvNET/TinyAestheticNet
5
+ ---
6
+
7
+ # TinyAestheticNet
8
+
9
+ ## DATA
10
+
11
+ To measure creativity in paintings, this study utilizes a dataset consisting of 878 open-access modern art paintings. Each artwork was rigorously evaluated by 3 fine arts experts specializing in painting. The experts reached a consensus to score the artworks on a scale of 1 to 10 across 5 distinct criteria:
12
+ * **K1:** Originality
13
+ * **K2:** Aesthetics
14
+ * **K3:** Design Principles and Elements
15
+ * **K4:** Technique Used
16
+ * **K5:** Unity and Wholeness
17
+ The dataset features a well-balanced distribution, with an almost equal representation of scores across the 1 to 10 scale for each criterion.
18
+
19
+ ## MODEL
20
+
21
+ ```bash
22
+ pip install torch torchvision git+https://github.com/openai/CLIP.git huggingface_hub
23
+ ```
24
+
25
+ ```python
26
+ import torch
27
+ import torch.nn as nn
28
+ from PIL import Image
29
+ import clip
30
+ from huggingface_hub import hf_hub_download
31
+
32
+ class CLIPMLPScorer(nn.Module):
33
+ def __init__(self, clip_model, num_factors=5, dropout=0.4):
34
+ super().__init__()
35
+ self.clip_model = clip_model
36
+
37
+ for p in self.clip_model.parameters():
38
+ p.requires_grad = False
39
+
40
+ self.head = nn.Sequential(
41
+ nn.Linear(512, 32),
42
+ nn.ReLU(),
43
+ nn.Dropout(dropout),
44
+ nn.Linear(32, num_factors),
45
+ nn.Sigmoid(),
46
+ )
47
+
48
+ def forward(self, x):
49
+ features = self.clip_model.encode_image(x).float()
50
+ x = self.head(features)
51
+ x = x * 9 + 1
52
+ return x
53
+
54
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
55
+ clip_model, preprocess = clip.load("ViT-B/32", device=device)
56
+
57
+ repo_id = "behAIvNET/TinyAestheticNet"
58
+ model_path = hf_hub_download(repo_id=repo_id, filename="TinyAestheticNet.pt")
59
+
60
+ model = CLIPMLPScorer(clip_model).to(device)
61
+ model.load_state_dict(torch.load(model_path, map_location=device))
62
+ model.eval()
63
+
64
+
65
+ image_path = "test_artwork.jpg"
66
+ img = preprocess(Image.open(image_path)).unsqueeze(0).to(device)
67
+
68
+ with torch.no_grad():
69
+ scores = model(img)[0].cpu().numpy()
70
+
71
+ criteria = ["K1", "K2", "K3", "K4", "K5"]
72
+ print("Artwork Scores:")
73
+ for c, s in zip(criteria, scores):
74
+ print(f"{c}: {s:.2f}/10")
75
+ ```
76
+
77
+ ## PERFORMANCE
78
+
79
+ ![Performance](TinyAestheticNet_performance.png)
80
+
81
+ ## EXPLAINABILITY
82
+
83
+ ![XAI](TinyAestheticNet_xai.png)