| ---
|
| title: Computer Vision Classification Comparison
|
| emoji: 📷
|
| colorFrom: yellow
|
| colorTo: blue
|
| sdk: gradio
|
| sdk_version: "6.12.0"
|
| app_file: app.py
|
| pinned: false
|
| ---
|
| # Gradio Pokemon Classification App
|
|
|
| This app compares 3 image classification approaches on custom Pokemon images:
|
|
|
| - Fine-tuned transfer learning model (ResNet18) from this project
|
| - Zero-shot CLIP (`openai/clip-vit-base-patch32`)
|
| - OpenAI vision model (LLM image classification)
|
|
|
| ## Dataset Used For Training
|
|
|
| - Custom dataset from course materials (week 8 style): `data/pokemon`
|
| - Train split: `data/pokemon/train`
|
| - Test split: `data/pokemon/test`
|
| - Number of classes: `6` (`charizard`, `charmander`, `charmeleon`, `ditto`, `eevee`, `ekans`)
|
|
|
| ## Trained Model
|
|
|
| - Local model artifact: `models/custom_resnet18.pth`
|
| - Hugging Face model link: `https://huggingface.co/kukalend/pokemon-transfer-resnet18`
|
|
|
| ## Training Performance
|
|
|
| | Training Loss | Epoch | Step | Validation Loss | Accuracy |
|
| |---:|---:|---:|---:|---:|
|
| | 1.6525 | 1 | - | 0.4324 | 0.2848 |
|
| | 0.9985 | 2 | - | 0.6216 | 0.7285 |
|
| | 0.6668 | 3 | - | 0.7838 | 0.8344 |
|
| | 0.4450 | 4 | - | 0.8378 | 0.9007 |
|
|
|
| Notes:
|
| - Validation loss was not logged separately in the training script, so the table reports validation accuracy in the validation column context and train accuracy in the accuracy column.
|
| - Final test accuracy of the custom model: `0.80`.
|
|
|
| ## Example Image Results
|
|
|
| The table below reports the true class and Top-3 predictions for Custom ResNet and CLIP.
|
|
|
| | Image | True Class | Custom ResNet Top-3 (score) | CLIP Top-3 (score) | OpenAI LLM (label, confidence) |
|
| |---|---|---|---|---|
|
| | `charizard.png` | `charizard` | `eevee` (0.5464)<br>`charizard` (0.3056)<br>`charmander` (0.0711) | `charizard` (0.4536)<br>`charmander` (0.3524)<br>`charmeleon` (0.0896) | `charizard` (0.9000) |
|
| | `charmander.png` | `charmander` | `charmander` (0.5801)<br>`charmeleon` (0.3410)<br>`ekans` (0.0315) | `charmeleon` (0.5400)<br>`charmander` (0.4202)<br>`charizard` (0.0268) | `charmander` (0.9500) |
|
| | `charmeleon.png` | `charmeleon` | `charmeleon` (0.3503)<br>`eevee` (0.3164)<br>`ekans` (0.2516) | `charmeleon` (0.4247)<br>`eevee` (0.3977)<br>`charizard` (0.0802) | `charmeleon` (0.9500) |
|
| | `ditto.png` | `ditto` | `ditto` (0.8271)<br>`ekans` (0.0959)<br>`eevee` (0.0370) | `ditto` (0.5273)<br>`ekans` (0.2055)<br>`eevee` (0.1169) | `ekans` (0.9000) |
|
| | `eevee.png` | `eevee` | `eevee` (0.9962)<br>`ekans` (0.0022)<br>`charizard` (0.0006) | `eevee` (0.9984)<br>`ditto` (0.0008)<br>`charizard` (0.0004) | `eevee` (0.9500) |
|
| | `ekans.png` | `ekans` | `ekans` (0.5366)<br>`eevee` (0.3374)<br>`charmander` (0.0682) | `ekans` (0.5120)<br>`charmeleon` (0.2303)<br>`charmander` (0.2177) | `ekans` (0.9500) |
|
|
|
| ## Overall Comparison (Test Set)
|
|
|
| - Custom transfer learning model accuracy: `0.80`
|
| - CLIP zero-shot model accuracy: `0.72`
|
| - OpenAI vision model accuracy: `0.7083` on 24 evaluated samples
|
|
|
| ## Links
|
|
|
| - Hugging Face Space: `https://huggingface.co/spaces/kukalend/computer-Vision-classification`
|
| - Hugging Face model repo: `https://huggingface.co/kukalend/pokemon-transfer-resnet18` |
