vasanthi8134 commited on
Commit
42c0e4d
·
verified ·
1 Parent(s): fdd2fc4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +71 -29
README.md CHANGED
@@ -7,29 +7,36 @@ sdk: gradio
7
  sdk_version: 6.12.0
8
  app_file: app.py
9
  pinned: false
10
- short_description: Compare ViT, CLIP, and OpenAI on 3 pet classes
11
  ---
12
 
13
  # Pet Classification Comparison
14
 
15
  This app compares 3 image classification approaches on pet images:
16
 
17
- - Fine-tuned ViT model (`vasanthi8134/oxford-pets-3class-vit`)
18
  - Zero-shot CLIP model (`openai/clip-vit-base-patch32`)
19
  - OpenAI vision model (LLM image classification)
20
 
21
  ## Dataset Used For Training
22
 
23
  - Hugging Face dataset loader: `load_dataset("pcuenq/oxford-pets")`
24
- - Dataset type used in this project: **custom 3-class subset** based on Oxford-IIIT Pet dataset
25
- - Selected classes: `Egyptian Mau`, `leonberger`, `samoyed`
 
 
 
 
26
  - Number of classes: **3**
27
  - Total images: **90**
28
 
29
  ### Custom Split
30
- - Train: **60** images total (**20 per class**)
31
- - Validation: **15** images total (**5 per class**)
32
- - Test: **15** images total (**5 per class**)
 
 
 
33
 
34
  ## Preprocessing Steps
35
 
@@ -39,7 +46,7 @@ This app compares 3 image classification approaches on pet images:
39
  - Conversion to tensor
40
  - Normalization with ViT image processor values
41
 
42
- ### Validation/Test transforms
43
  - Resize
44
  - Center crop
45
  - Conversion to tensor
@@ -48,55 +55,90 @@ This app compares 3 image classification approaches on pet images:
48
  ## Trained Model
49
 
50
  - Base model: `google/vit-base-patch16-224-in21k`
51
- - Fine-tuned model link: `https://huggingface.co/vasanthi8134/oxford-pets-3class-vit`
 
52
 
53
  ## Training Performance
54
 
55
- | Training Setup | Value |
 
 
56
  |---|---:|
57
  | Epochs | 5 |
58
  | Learning rate | 5e-5 |
59
  | Batch size | 8 |
60
 
 
 
61
  | Metric | Value |
62
  |---|---:|
63
  | Validation accuracy | 1.0 |
64
  | Test accuracy | 1.0 |
65
 
 
 
 
 
 
 
 
 
 
 
66
  ## Example Image Results
67
 
68
- The application compares:
69
- - my fine-tuned ViT model
70
- - CLIP zero-shot classification
71
- - OpenAI vision classification
 
 
72
 
73
- ### Example result: leonberger image
74
- | Model | Output |
75
- |---|---|
76
- | ViT | leonberger (0.4457) |
77
- | CLIP | leonberger (1.0) |
78
- | OpenAI | leonberger (0.95) |
 
79
 
80
  ### Short comparison
81
- - **My model** is specialized for the selected 3 classes because it was fine-tuned on the custom subset.
82
- - **CLIP** works zero-shot and still performs well on clear images.
83
- - **OpenAI vision** also performs well and returns a label, confidence, and short reasoning.
84
 
85
- ## Hugging Face App
 
 
 
 
86
 
87
- - App link: `https://huggingface.co/spaces/vasanthi8134/pet-classification-comparison`
 
 
 
 
88
 
89
  ## Application Features
90
 
91
  The Hugging Face Space includes:
92
  - image upload
93
- - predictions from all three models
94
- - comparison output in JSON format
 
 
 
 
 
 
 
 
 
 
 
95
 
96
  ## Notes
97
 
98
- This is a simplified educational computer vision project created for model comparison using:
 
99
  - transfer learning on custom data
100
  - Hugging Face model deployment
101
  - Hugging Face Space deployment
102
- - comparison between open-source and closed-source models
 
7
  sdk_version: 6.12.0
8
  app_file: app.py
9
  pinned: false
10
+ short_description: ViT vs CLIP vs OpenAI on 3 custom pet classes
11
  ---
12
 
13
  # Pet Classification Comparison
14
 
15
  This app compares 3 image classification approaches on pet images:
16
 
17
+ - Fine-tuned ViT model ([vasanthi8134/oxford-pets-3class-vit](https://huggingface.co/vasanthi8134/oxford-pets-3class-vit))
18
  - Zero-shot CLIP model (`openai/clip-vit-base-patch32`)
19
  - OpenAI vision model (LLM image classification)
20
 
21
  ## Dataset Used For Training
22
 
23
  - Hugging Face dataset loader: `load_dataset("pcuenq/oxford-pets")`
24
+ - Original dataset source: Oxford-IIIT Pet dataset
25
+ - Dataset used in this project: **custom 3-class subset** based on Oxford-IIIT Pet
26
+ - Selected classes:
27
+ - `Egyptian Mau`
28
+ - `leonberger`
29
+ - `samoyed`
30
  - Number of classes: **3**
31
  - Total images: **90**
32
 
33
  ### Custom Split
34
+
35
+ The custom subset was created by selecting **30 images per class** and splitting them into:
36
+
37
+ - **Train:** 60 images total (**20 per class**)
38
+ - **Validation:** 15 images total (**5 per class**)
39
+ - **Test:** 15 images total (**5 per class**)
40
 
41
  ## Preprocessing Steps
42
 
 
46
  - Conversion to tensor
47
  - Normalization with ViT image processor values
48
 
49
+ ### Validation / Test transforms
50
  - Resize
51
  - Center crop
52
  - Conversion to tensor
 
55
  ## Trained Model
56
 
57
  - Base model: `google/vit-base-patch16-224-in21k`
58
+ - Approach: **transfer learning / fine-tuning**
59
+ - Fine-tuned model link: [https://huggingface.co/vasanthi8134/oxford-pets-3class-vit](https://huggingface.co/vasanthi8134/oxford-pets-3class-vit)
60
 
61
  ## Training Performance
62
 
63
+ ### Training Setup
64
+
65
+ | Parameter | Value |
66
  |---|---:|
67
  | Epochs | 5 |
68
  | Learning rate | 5e-5 |
69
  | Batch size | 8 |
70
 
71
+ ### Final Evaluation
72
+
73
  | Metric | Value |
74
  |---|---:|
75
  | Validation accuracy | 1.0 |
76
  | Test accuracy | 1.0 |
77
 
78
+ Because this project uses a small and simplified custom subset with only 3 classes, the fine-tuned model performs very well on this task.
79
+
80
+ ## Evaluation Method
81
+
82
+ The final model was evaluated on:
83
+ - a **validation split** during training
84
+ - a separate **test split** after training
85
+
86
+ The model with the best validation performance was used as the final selected model.
87
+
88
  ## Example Image Results
89
 
90
+ The table below reports example predictions from all 3 approaches.
91
+
92
+ | Image | True Class | ViT Prediction | CLIP Prediction | OpenAI Prediction |
93
+ |---|---|---|---|---|
94
+ | `leonberger.jpg` | leonberger | leonberger (0.4457) | leonberger (1.0) | leonberger (0.95) |
95
+ | `Egyptian_Mau.jpg` | Egyptian Mau | Egyptian Mau (0.4171) | Egyptian Mau (1.0) | Egyptian Mau (0.95) |
96
 
97
+ ## Model Comparison
98
+
99
+ This application compares:
100
+
101
+ 1. **My fine-tuned ViT model**
102
+ 2. **CLIP zero-shot classification**
103
+ 3. **OpenAI vision classification**
104
 
105
  ### Short comparison
 
 
 
106
 
107
+ - **My fine-tuned ViT model** is specialized for the selected 3 classes because it was trained on the custom subset.
108
+ - **CLIP** works in a zero-shot setting and still performs well on clear images without task-specific fine-tuning.
109
+ - **OpenAI vision** also performs well and returns a label, confidence score, and short reasoning.
110
+
111
+ ## Hugging Face Links
112
 
113
+ ### Model
114
+ [https://huggingface.co/vasanthi8134/oxford-pets-3class-vit](https://huggingface.co/vasanthi8134/oxford-pets-3class-vit)
115
+
116
+ ### App
117
+ [https://huggingface.co/spaces/vasanthi8134/pet-classification-comparison](https://huggingface.co/spaces/vasanthi8134/pet-classification-comparison)
118
 
119
  ## Application Features
120
 
121
  The Hugging Face Space includes:
122
  - image upload
123
+ - prediction from the fine-tuned ViT model
124
+ - prediction from the zero-shot CLIP model
125
+ - prediction from the OpenAI vision model
126
+ - example images for quick testing
127
+ - JSON output for direct comparison
128
+
129
+ ## Final Selected Model
130
+
131
+ The final selected model for the custom classification task is:
132
+
133
+ - **ViT fine-tuned on the custom 3-class Oxford-IIIT Pet subset**
134
+
135
+ It was selected because it is the project-specific transfer learning model required by the assignment and achieved perfect accuracy on the simplified validation and test splits.
136
 
137
  ## Notes
138
 
139
+ This is a simplified educational computer vision project created to demonstrate:
140
+
141
  - transfer learning on custom data
142
  - Hugging Face model deployment
143
  - Hugging Face Space deployment
144
+ - comparison between open-source and closed-source image classification approaches