README.md

metadata

title: Wardrobe Us
emoji: 👕
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 6.17.3
python_version: '3.12'
app_file: app.py
pinned: false
license: mit
hardware: cpu-basic
short_description: AI wardrobe. catalog, combine and ask about your clothes

👕 Wardrobe AI

Turn a physical wardrobe into a searchable, AI-powered catalog — and get outfit ideas from clothes you already own.

Blog Post: Gradio x Hugging Face Hackaton Built for the Gradio × Hugging Face Build Small Hackathon (June 2026).

The original motivation: help someone with 200+ garments who forgets what they own, buys duplicates, and struggles to combine outfits every morning. Wardrobe AI is not a shopping app — it helps you use what you already have.

---

What it does

Step	Description
Capture	Upload photos of your clothes. A detector finds garments, you can adjust bounding boxes, and a VLM extracts structured attributes (type, color, material, pattern, season, formality, description).
Catalog	Browse your digital wardrobe with images and metadata. Click any garment for a detail panel with full attributes.
Combine	Generate top+bottom outfit combinations filtered by season and formality rules. Describe an occasion and the LLM re-ranks the best matches. Like/dislike outfits to build style preferences.
Ask	Chat with your wardrobe in natural language. Answers reference your actual garments with images and descriptions.

All inference runs locally — no external APIs.

---

Two frontends

The app ships with two UIs sharing the same backend:

	Custom UI (default)	Gradio Blocks (--default)
Launch	python app.py	python app.py --default
Stack	gradio.Server + Alpine.js + @gradio/client	Gradio 6.17 Blocks
Language	English	Spanish
Best for	End users — clean, minimal UX	Power users — full settings
Manual crop editor	Annotorious v3 bounding-box editor	gradio-image-annotation
Detection backend switch	—	Dropdown in settings
Dataset load logs	Real-time log dock (streaming)	Markdown + gallery preview
Ask tab	Garment chips with images in replies	Streaming chatbot

Both modes support sample dataset loading, outfit generation, and wardrobe chat.

---

Tech stack

Component	Choice
VLM + Chat LLM	Gemma 3 4B IT (Q4_K_M GGUF) via llama-cpp-python
Garment detection	YOLOS-tiny (default), YOLOv8n, or GroundingDINO — pluggable registry
Runtime	llama.cpp — CPU on HF Spaces, CUDA locally
UI	gradio.Server + Alpine.js (default) or Gradio Blocks
Storage	Local filesystem or S3 (configurable)
Catalog	data/catalog.json + data/garments/*.jpg
Preferences	data/outfits.json (liked combinations)

Total parameters: 4 billion — fits Tiny Titan (≤4B) and runs on CPU Basic (16 GB RAM) with Q4_K_M quantization (~3 GB model).

The same Gemma 3 4B model handles vision extraction, outfit ranking, and chat. A singleton _ModelManager hot-swaps between vision (MTMD) and text-only modes.

---

Bonus quests

Badge	Status
🔌 Off the Grid	All inference on Space hardware. No external APIs.
🦙 Llama Champion	Model runs through llama.cpp (llama-cpp-python).
🐜 Tiny Titan	Gemma 3 4B — under the 4B threshold.
🎨 Off-Brand	Custom frontend via gr.Server + Alpine.js.
📡 Sharing is Caring	Agent trace shared on the Hub.
📓 Field Notes	Build report in FIELD_NOTES.md.

---

How to use

On Hugging Face Spaces

Runs on CPU Basic (2 vCPU, 16 GB RAM). Set HF_TOKEN in Space Secrets before first use.

1. Load a sample wardrobe — Add Clothes → Load Dataset (50 garments from a public HF dataset; ~15–45 min on CPU with live progress logs).
2. Or upload your own — drag a flat-lay photo, review auto-detected boxes, click Analyse (~30–90 s per garment on CPU).
3. Get Dressed — type an occasion, hit Generate (~5–15 s for LLM ranking).
4. Ask — chat about outfits, care, or what you own (~5–15 s per response).

Sample datasets:

Key	Dataset	Notes
second-hand	fnauman/fashion-second-hand-front-only-rgb	Individual garments, no detection step
fashion-1k	Codatta/Fashion-1K	Multi-garment photos, slower (needs detection)

Local development (GPU accelerated)

cd packages/wardrobe-us
python -m venv .venv && source .venv/bin/activate

# Base deps (CPU wheels — same as HF Space):
pip install -r requirements.txt

# Override llama-cpp-python with CUDA 12.4 GPU wheel:
pip install llama-cpp-python==0.3.28 \
  --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124 \
  --force-reinstall --no-deps

# Custom minimal frontend (default):
python app.py

# Full Gradio Blocks UI:
python app.py --default

Requires a CUDA GPU with ≥8 GB VRAM for GPU mode. Without CUDA, inference falls back to CPU automatically. Copy .env.example to .env and set HF_TOKEN.

Pre-build a sample catalog offline (optional):

python scripts/build_sample_wardrobe.py --dataset second-hand --target 50

---

Architecture

app.py                         # Entry point (--ui default | --default)
src/
  ui/
    index.html                 # Custom frontend (Alpine.js + Annotorious)
    style.css
  model_loader.py              # GGUF singleton (Gemma 3 4B, n_ctx=4096)
  vision.py                    # VLM attribute extraction pipeline
  detector/                    # Pluggable garment detection
    _registry.py               # @register("yolos") pattern
    backends/                  # yolos | yolov8 | grounding_dino
  catalog.py                   # JSON catalog CRUD
  combinations.py              # Outfit generation + LLM ranking
  assistant.py                 # Chat with wardrobe context
  storage.py                   # Local / S3 image storage
  settings.py                  # Runtime config (data/settings.json)
data/
  catalog.json                 # Garment metadata
  garments/                    # Cropped garment images
  outfits.json                 # Liked outfit preferences
  _uploads/                    # Temp images during crop workflow

API endpoints (custom UI)

Exposed via gradio.Server and consumed by @gradio/client:

Endpoint	Purpose
prepare_image	Save upload, auto-detect boxes → token + image URL for editor
analyze_boxes	Crop user-confirmed boxes, VLM extract, add to catalog
add_photo	One-shot upload + auto-detect + extract (no manual crop)
get_wardrobe	Full catalog with cache-busted image URLs
get_combinations	Generate + LLM-rank outfits (top 20 returned)
rate_outfit	Save like/dislike preference
ask_question	Natural-language wardrobe chat
load_dataset	Stream dataset processing progress (generator)

Static mounts: /garments (catalog images), /uploads (temp crop images).

Outfit ranking

1. Rule-based generation: all compatible top+bottom pairs (season + formality filters).
2. LLM ranking: up to 20 diverse combinations sent to Gemma 3 4B with a compact prompt (fits n_ctx=4096). Remaining combos appended in original order.
3. User likes feed back into future ranking prompts as style signals.

---

Environment variables

Variable	Required	Description
HF_TOKEN	Yes	Hugging Face token for model/dataset downloads
STORAGE_BACKEND	No	local (default) or s3
S3_BUCKET_NAME	If S3	Bucket name
S3_ENDPOINT_URL	If S3	S3 endpoint
AWS_ACCESS_KEY_ID	If S3	AWS credentials
AWS_SECRET_ACCESS_KEY	If S3	AWS credentials
DETECTION_BACKEND	No	yolos (default), yolov8, or grounding_dino
CUDA_VISIBLE_DEVICES	No	GPU index (local only; forced to CPU on Spaces)

---

Performance notes

Task	CPU Basic (Space)	Local GPU
First model download	~2–3 min	~2–3 min
Garment extraction	~30–90 s each	~3–10 s each
Dataset load (50 items)	~15–45 min	~5–15 min
Outfit ranking	~5–15 s	~2–5 s
Ask response	~5–15 s	~2–5 s

Detection tips: YOLOS-tiny works best on flat-lay photos. Hanger or worn-garment photos are harder — use the manual bounding-box editor as fallback.

VLM accuracy: At 4B parameters, color and type labels are usually good but not perfect (e.g. navy vs black). Descriptions and structured JSON parsing with regex fallback help reliability.

---

Agent trace (Sharing is Caring)

The full development conversation is published as a dataset on the Hub:

huggingface-cli upload-large-folder build-small-hackathon/wardrobe-us-agent-trace ./agent-trace --repo-type dataset

See also FIELD_NOTES.md for architecture decisions, what worked, and lessons learned.

---

License

MIT