fault9 commited on
Commit
36cda9f
·
verified ·
1 Parent(s): ff8cfa4

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +77 -0
README.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ library_name: scikit-learn
4
+ pipeline_tag: image-classification
5
+ tags:
6
+ - art
7
+ - art-history
8
+ - clip
9
+ - embeddings
10
+ - scikit-learn
11
+ - image-classification
12
+ ---
13
+
14
+ # Art Movement Classifier with CLIP Embeddings
15
+
16
+ This model classifies paintings into eight art movements using frozen CLIP image embeddings and a trained scikit-learn classifier.
17
+
18
+ ## Lab Framing
19
+
20
+ **Issue:** Art movement classification is difficult because visual movements overlap in subject matter, period, materials, and artist-specific style.
21
+
22
+ **Challenge addressed with embeddings:** Instead of training a vision model from scratch on a small dataset, the project uses pretrained CLIP image embeddings as a compact visual representation, then trains a domain-specific classifier on top of those embeddings.
23
+
24
+ **Embedding type:** `openai/clip-vit-base-patch32` image embeddings.
25
+
26
+ **Classifier:** MLP classifier trained on CLIP embeddings.
27
+
28
+ ## Classes
29
+
30
+ - Renaissance
31
+ - Baroque
32
+ - Impressionism
33
+ - Expressionism
34
+ - Cubism
35
+ - Abstract
36
+ - Surrealism
37
+ - Pop Art
38
+
39
+ ## Results
40
+
41
+ Held-out test accuracy: **0.8093**
42
+
43
+ Cross-validation accuracy: **0.8138 ± 0.0173**
44
+
45
+ Per-class F1:
46
+
47
+ | Class | F1 |
48
+ |---|---:|
49
+ | Abstract | 0.727 |
50
+ | Baroque | 0.860 |
51
+ | Cubism | 0.774 |
52
+ | Expressionism | 0.758 |
53
+ | Impressionism | 0.906 |
54
+ | Pop Art | 0.761 |
55
+ | Renaissance | 0.904 |
56
+ | Surrealism | 0.684 |
57
+
58
+ ## Files
59
+
60
+ - `classifier.joblib`: trained scikit-learn classifier
61
+ - `label_encoder.joblib`: label encoder for class ids
62
+ - `config.json`: embedding/model configuration
63
+ - `evaluation_metrics.json`: held-out and cross-validation metrics
64
+ - `confusion_matrix.png`: normalized confusion matrix
65
+ - `wolfflin_pca_clip.png`: Wölfflin-inspired PCA analysis
66
+ - `arnheim_axes.npz`: Arnheim perceptual axis vectors used by the demo
67
+
68
+ ## Usage
69
+
70
+ The model is intended to be used through the accompanying Gradio Space. The app loads CLIP, embeds an uploaded image, and applies the trained classifier.
71
+
72
+ ## Limitations
73
+
74
+ - The model can confuse visually similar old-master religious paintings, especially Renaissance vs Baroque.
75
+ - Predictions reflect visual similarity in the training distribution, not definitive art-historical attribution.
76
+ - The Wölfflin tab in the demo is a theoretical profile of the predicted class, not a direct pixel-level measurement.
77
+ - Arnheim perceptual scores are anchor-based projections in CLIP space and should be interpreted as exploratory.