README.md

metadata

title: Pet Classification Comparison
emoji: 🐾
colorFrom: purple
colorTo: gray
sdk: gradio
sdk_version: 6.12.0
app_file: app.py
pinned: false
short_description: ViT vs CLIP vs OpenAI on 3 custom pet classes

Pet Classification Comparison

This app compares 3 image classification approaches on pet images:

• Fine-tuned ViT model (vasanthi8134/oxford-pets-3class-vit)
• Zero-shot CLIP model (openai/clip-vit-base-patch32)
• OpenAI vision model (LLM image classification)

Dataset Used For Training

• Hugging Face dataset loader: load_dataset("pcuenq/oxford-pets")
• Original dataset source: Oxford-IIIT Pet dataset
• Dataset used in this project: custom 3-class subset based on Oxford-IIIT Pet
• Selected classes:
  • Egyptian Mau
  • leonberger
  • samoyed
• Number of classes: 3
• Total images: 90

Custom Split

The custom subset was created by selecting 30 images per class and splitting them into:

• Train: 60 images total (20 per class)
• Validation: 15 images total (5 per class)
• Test: 15 images total (5 per class)

Preprocessing Steps

Training transforms

• Random resized crop
• Random horizontal flip
• Conversion to tensor
• Normalization with ViT image processor values

Validation / Test transforms

• Resize
• Center crop
• Conversion to tensor
• Normalization with ViT image processor values

Trained Model

• Base model: google/vit-base-patch16-224-in21k
• Approach: transfer learning / fine-tuning
• Fine-tuned model link: https://huggingface.co/vasanthi8134/oxford-pets-3class-vit

Training Performance

Training Setup

Parameter	Value
Epochs	5
Learning rate	5e-5
Batch size	8

Final Evaluation

Metric	Value
Validation accuracy	1.0
Test accuracy	1.0

Because this project uses a small and simplified custom subset with only 3 classes, the fine-tuned model performs very well on this task.

Evaluation Method

The final model was evaluated on:

• a validation split during training
• a separate test split after training

The model with the best validation performance was used as the final selected model.

Example Image Results

The table below reports example predictions from all 3 approaches.

Image	True Class	ViT Prediction	CLIP Prediction	OpenAI Prediction
leonberger.jpg	leonberger	leonberger (0.4457)	leonberger (1.0)	leonberger (0.95)
Egyptian_Mau.jpg	Egyptian Mau	Egyptian Mau (0.4171)	Egyptian Mau (1.0)	Egyptian Mau (0.95)

Model Comparison

This application compares:

1. My fine-tuned ViT model
2. CLIP zero-shot classification
3. OpenAI vision classification

Short comparison

• My fine-tuned ViT model is specialized for the selected 3 classes because it was trained on the custom subset.
• CLIP works in a zero-shot setting and still performs well on clear images without task-specific fine-tuning.
• OpenAI vision also performs well and returns a label, confidence score, and short reasoning.

Hugging Face Links

Model

https://huggingface.co/vasanthi8134/oxford-pets-3class-vit

App

https://huggingface.co/spaces/vasanthi8134/pet-classification-comparison

Application Features

The Hugging Face Space includes:

• image upload
• prediction from the fine-tuned ViT model
• prediction from the zero-shot CLIP model
• prediction from the OpenAI vision model
• example images for quick testing
• JSON output for direct comparison

Final Selected Model

The final selected model for the custom classification task is:

• ViT fine-tuned on the custom 3-class Oxford-IIIT Pet subset

It was selected because it is the project-specific transfer learning model required by the assignment and achieved perfect accuracy on the simplified validation and test splits.

Notes

This is a simplified educational computer vision project created to demonstrate:

• transfer learning on custom data
• Hugging Face model deployment
• Hugging Face Space deployment
• comparison between open-source and closed-source image classification approaches