File size: 4,850 Bytes
1d4a8ad 277f1d7 1d4a8ad | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 | ---
license: apache-2.0
library_name: transformers
tags:
- vision
- image-classification
- vit
- fashion
- ecommerce
datasets:
- ashraq/fashion-product-images-small
base_model: google/vit-base-patch16-224-in21k
---
# ImagineClassification (fine-tuned ViT)
Fine-tuned **Vision Transformer (ViT-B/16, patch 16, 224×224)** for **coarse fashion product classification** into four `masterCategory` labels from the [Fashion Product Images (small)](https://huggingface.co/datasets/ashraq/fashion-product-images-small) dataset.
## Model summary
| Item | Detail |
|------|--------|
| **Base checkpoint** | [`google/vit-base-patch16-224-in21k`](https://huggingface.co/google/vit-base-patch16-224-in21k) |
| **Task** | Multi-class image classification (4 classes) |
| **Labels** | `Apparel`, `Accessories`, `Footwear`, `Personal Care` |
| **Input** | RGB images, 224×224 (use the bundled `ViTImageProcessor` / `AutoImageProcessor`) |
| **Framework** | PyTorch + [Transformers](https://github.com/huggingface/transformers) |
This repository was produced by comparing three candidate image classifiers (same data and training recipe), then packaging the **best** checkpoint by **test accuracy**.
## Training procedure (from experiment notebook)
Training and evaluation follow the pipeline described in `Pipeline_1_fine_tuning_models.ipynb`:
- **Data**: `ashraq/fashion-product-images-small` (`train` split), rows with `masterCategory` in the four classes above.
- **Balanced sampling**: For each class, **2,100** images sampled with `random_state=5` (**SEED = 5**):
- **100** images per class held out as a **stratified out-of-sample test set** (400 images total).
- Remaining **2,000** per class form the train/val pool; **stratified train/validation split** (same seed).
- **Optimization**: AdamW, learning rate **2e-5**, batch size **8**, **1** fine-tuning epoch, cross-entropy loss.
- **Candidates fine-tuned** (same recipe):
`google/vit-base-patch16-224-in21k`, `facebook/deit-tiny-patch16-224`, `google/mobilenet_v2_1.0_224`.
- **Selection rule**: Highest **test accuracy** on the held-out 400-sample test set; ties broken by the loop order in the notebook.
### Reported results (one Colab run, post fine-tuning)
On the **400-image** balanced test set; **runtime** = mean seconds per image for evaluation in that run (device-dependent).
| Model | Test accuracy | Runtime (s / image) |
|-------|----------------|----------------------|
| `google/vit-base-patch16-224-in21k` | **0.9975** | 0.000953 |
| `facebook/deit-tiny-patch16-224` | 0.9950 | 0.000886 |
| `google/mobilenet_v2_1.0_224` | 0.9375 | 0.001365 |
**Selected model:** `google/vit-base-patch16-224-in21k` (accuracy **1.0000** on this test split).
> **Note:** Perfect accuracy on 400 samples does not guarantee generalization to all real-world product photos. Performance depends on image quality, viewpoint, and domain shift relative to the dataset.
## Intended use
- **Primary:** Fast **coarse category** tagging for fashion e-commerce assets (four-way classification).
- **Out of scope:** Fine-grained SKU/subcategory prediction, non-fashion images, or classes outside the four labels above.
## Limitations and bias
- Trained only on **four** frequent `masterCategory` values for a class-balanced setup; other categories from the original catalog are **not** represented.
- The source dataset may reflect **commercial catalog** biases (presentation, demographics, geography).
- Do not use for **high-stakes** decisions (e.g., safety, compliance, or financial outcomes) without further validation.
## How to use
```python
from transformers import AutoImageProcessor, AutoModelForImageClassification
from PIL import Image
import requests
model_id = "Leoinhouse/ImagineClassification-finetuned-model" # or local path
processor = AutoImageProcessor.from_pretrained(model_id)
model = AutoModelForImageClassification.from_pretrained(model_id)
image = Image.open(requests.get("https://example.com/product.jpg", stream=True).raw).convert("RGB")
inputs = processor(images=image, return_tensors="pt")
outputs = model(**inputs)
predicted_id = outputs.logits.argmax(-1).item()
label = model.config.id2label[predicted_id]
print(label)
```
## Citation
If you use this model, cite the **Transformers** library and the **dataset** you rely on, for example:
```bibtex
@inproceedings{wolf-etal-2020-transformers,
title={Transformers: State-of-the-Art Natural Language Processing},
author={Wolf, Thomas and others},
booktitle={EMNLP 2020: System Demonstrations},
year={2020}
}
```
## Model card contact
Maintained for coursework / project use under Hugging Face user **Leoinhouse**. For the upstream ViT architecture and weights, see the **base model** card: [`google/vit-base-patch16-224-in21k`](https://huggingface.co/google/vit-base-patch16-224-in21k).
|