---
title: Computer Vision Classification Comparison
emoji: 📷
colorFrom: yellow
colorTo: blue
sdk: gradio
sdk_version: "6.12.0"
app_file: app.py
pinned: false
---
# Gradio Pokemon Classification App

This app compares 3 image classification approaches on custom Pokemon images:

- Fine-tuned transfer learning model (ResNet18) from this project
- Zero-shot CLIP (`openai/clip-vit-base-patch32`)
- OpenAI vision model (LLM image classification)

## Dataset Used For Training

- Custom dataset from course materials (week 8 style): `data/pokemon`
- Train split: `data/pokemon/train`
- Test split: `data/pokemon/test`
- Number of classes: `6` (`charizard`, `charmander`, `charmeleon`, `ditto`, `eevee`, `ekans`)

## Trained Model

- Local model artifact: `models/custom_resnet18.pth`
- Hugging Face model link: `https://huggingface.co/kukalend/pokemon-transfer-resnet18`

## Training Performance

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---:|---:|---:|---:|---:|
| 1.6525 | 1 | - | 0.4324 | 0.2848 |
| 0.9985 | 2 | - | 0.6216 | 0.7285 |
| 0.6668 | 3 | - | 0.7838 | 0.8344 |
| 0.4450 | 4 | - | 0.8378 | 0.9007 |

Notes:
- Validation loss was not logged separately in the training script, so the table reports validation accuracy in the validation column context and train accuracy in the accuracy column.
- Final test accuracy of the custom model: `0.80`.

## Example Image Results

The table below reports the true class and Top-3 predictions for Custom ResNet and CLIP.

| Image | True Class | Custom ResNet Top-3 (score) | CLIP Top-3 (score) | OpenAI LLM (label, confidence) |
|---|---|---|---|---|
| `charizard.png` | `charizard` | `eevee` (0.5464)<br>`charizard` (0.3056)<br>`charmander` (0.0711) | `charizard` (0.4536)<br>`charmander` (0.3524)<br>`charmeleon` (0.0896) | `charizard` (0.9000) |
| `charmander.png` | `charmander` | `charmander` (0.5801)<br>`charmeleon` (0.3410)<br>`ekans` (0.0315) | `charmeleon` (0.5400)<br>`charmander` (0.4202)<br>`charizard` (0.0268) | `charmander` (0.9500) |
| `charmeleon.png` | `charmeleon` | `charmeleon` (0.3503)<br>`eevee` (0.3164)<br>`ekans` (0.2516) | `charmeleon` (0.4247)<br>`eevee` (0.3977)<br>`charizard` (0.0802) | `charmeleon` (0.9500) |
| `ditto.png` | `ditto` | `ditto` (0.8271)<br>`ekans` (0.0959)<br>`eevee` (0.0370) | `ditto` (0.5273)<br>`ekans` (0.2055)<br>`eevee` (0.1169) | `ekans` (0.9000) |
| `eevee.png` | `eevee` | `eevee` (0.9962)<br>`ekans` (0.0022)<br>`charizard` (0.0006) | `eevee` (0.9984)<br>`ditto` (0.0008)<br>`charizard` (0.0004) | `eevee` (0.9500) |
| `ekans.png` | `ekans` | `ekans` (0.5366)<br>`eevee` (0.3374)<br>`charmander` (0.0682) | `ekans` (0.5120)<br>`charmeleon` (0.2303)<br>`charmander` (0.2177) | `ekans` (0.9500) |

## Overall Comparison (Test Set)

- Custom transfer learning model accuracy: `0.80`
- CLIP zero-shot model accuracy: `0.72`
- OpenAI vision model accuracy: `0.7083` on 24 evaluated samples

## Links

- Hugging Face Space: `https://huggingface.co/spaces/kukalend/computer-Vision-classification`
- Hugging Face model repo: `https://huggingface.co/kukalend/pokemon-transfer-resnet18`