SamaGalagoda/threadcraft-garment-classifier

ThreadCraft Garment Classifier

Fine-tuned google/vit-base-patch16-224-in21k that predicts garment type from a clothing image.

Built for ThreadCraft, an AI-powered custom clothing design and ordering platform, as a final-year BSc Software Engineering project. In the product it runs on customer-uploaded reference photos at the design wizard's Step 2, to suggest which cloth type the customer is describing.

Intended use

Suggesting a garment category from a reference image, as an assistive default that the customer can override — not as an authoritative classification.

Results (held-out test split)

Metric	Score
Accuracy	0.6875
Macro F1	0.3482
Weighted F1	0.6517

Macro F1 is the headline number. Accuracy is inflated by the dominant classes; macro F1 weights every garment type equally and so reflects performance on the rarer ones.

Classes (25): Bra, Briefs, Capris, Dresses, Dupatta, Innerwear Vests, Jackets, Jeans, Kurtas, Kurtis, Leggings, Night suits, Nightdress, Sarees, Shirts, Shorts, Skirts, Sweaters, Sweatshirts, Tops, Track Pants, Trousers, Trunk, Tshirts, Tunics

Training

Base model	google/vit-base-patch16-224-in21k
Dataset	SamaGalagoda/threadcraft-garments-cleaned
Train / Val / Test	16,536 / 2,067 / 2,067
Epochs	4
Batch size	64 per device
Learning rate	0.0003
Precision	fp16
Hardware	Kaggle T4 x2
Model selection	best validation macro-F1

Limitations

• Source images are 60x80 px, upscaled to 224x224. Fine visual detail (fabric texture, stitching, small trims) is simply not present in the training data, so the model distinguishes garment silhouette far better than garment detail.
• Trained on catalogue product photography (clean background, flat-lay or on-model studio shots). Accuracy on casual user-taken photos, worn garments at odd angles, or cluttered backgrounds will be lower — this is a train/serve distribution shift and is the main reason the prediction is surfaced as an overridable suggestion.
• The class set is long-tailed; rarer garment types have materially lower per-class recall. See the per-class breakdown in classification_report.txt.
• The underlying dataset is Indian-retail-sourced (Myntra), so garment style distribution is regionally skewed. That happens to suit ThreadCraft's South Asian market, but it does limit generalisation elsewhere.
• Some confusions reflect genuinely ambiguous human labels (e.g. Tshirts vs Tops) rather than model error.

Usage

from transformers import pipeline

clf = pipeline("image-classification", model="SamaGalagoda/threadcraft-garment-classifier")
clf("reference_photo.jpg")

Citation of the source data

Aggarwal, P. (2019). Fashion Product Images Dataset. Kaggle. Accessed via the ashraq/fashion-product-images-small mirror on the Hugging Face Hub.