---
title: Wardrobe Us
emoji: 👕
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 6.17.3
python_version: "3.12"
app_file: app.py
pinned: false
license: mit
hardware: cpu-basic
short_description: AI wardrobe. catalog, combine and ask about your clothes
---

# 👕 Wardrobe AI

**Turn a physical wardrobe into a searchable, AI-powered catalog — and get outfit ideas from clothes you already own.**

Blog Post: [Gradio x Hugging Face Hackaton](https://ohgm.dev/en/journal/hugging-face-hackaton-2026/)
Built for the [Gradio × Hugging Face Build Small Hackathon](https://huggingface.co/build-small-hackathon) (June 2026).

The original motivation: help someone with 200+ garments who forgets what they own, buys duplicates, and struggles to combine outfits every morning. Wardrobe AI is not a shopping app — it helps you *use* what you already have.

---

## What it does

| Step | Description |
|------|-------------|
| **Capture** | Upload photos of your clothes. A detector finds garments, you can adjust bounding boxes, and a VLM extracts structured attributes (type, color, material, pattern, season, formality, description). |
| **Catalog** | Browse your digital wardrobe with images and metadata. Click any garment for a detail panel with full attributes. |
| **Combine** | Generate top+bottom outfit combinations filtered by season and formality rules. Describe an occasion and the LLM re-ranks the best matches. Like/dislike outfits to build style preferences. |
| **Ask** | Chat with your wardrobe in natural language. Answers reference your actual garments with images and descriptions. |

All inference runs locally — no external APIs.

---

## Two frontends


The app ships with two UIs sharing the same backend:

| | **Custom UI** (default) | **Gradio Blocks** (`--default`) |
|---|---|---|
| Launch | `python app.py` | `python app.py --default` |
| Stack | `gradio.Server` + Alpine.js + `@gradio/client` | Gradio 6.17 Blocks |
| Language | English | Spanish |
| Best for | End users — clean, minimal UX | Power users — full settings |
| Manual crop editor | Annotorious v3 bounding-box editor | `gradio-image-annotation` |
| Detection backend switch | — | Dropdown in settings |
| Dataset load logs | Real-time log dock (streaming) | Markdown + gallery preview |
| Ask tab | Garment chips with images in replies | Streaming chatbot |

Both modes support sample dataset loading, outfit generation, and wardrobe chat.

---

## Tech stack

| Component | Choice |
|-----------|--------|
| VLM + Chat LLM | **Gemma 3 4B IT** (Q4_K_M GGUF) via `llama-cpp-python` |
| Garment detection | **YOLOS-tiny** (default), YOLOv8n, or GroundingDINO — pluggable registry |
| Runtime | llama.cpp — CPU on HF Spaces, CUDA locally |
| UI | `gradio.Server` + Alpine.js (default) or Gradio Blocks |
| Storage | Local filesystem or S3 (configurable) |
| Catalog | `data/catalog.json` + `data/garments/*.jpg` |
| Preferences | `data/outfits.json` (liked combinations) |

**Total parameters: 4 billion** — fits Tiny Titan (≤4B) and runs on CPU Basic (16 GB RAM) with Q4_K_M quantization (~3 GB model).

The same Gemma 3 4B model handles vision extraction, outfit ranking, and chat. A singleton `_ModelManager` hot-swaps between vision (MTMD) and text-only modes.

---

## Bonus quests

| Badge | Status |
|-------|--------|
| 🔌 Off the Grid | All inference on Space hardware. No external APIs. |
| 🦙 Llama Champion | Model runs through llama.cpp (`llama-cpp-python`). |
| 🐜 Tiny Titan | Gemma 3 4B — under the 4B threshold. |
| 🎨 Off-Brand | Custom frontend via `gr.Server` + Alpine.js. |
| 📡 Sharing is Caring | Agent trace shared on the Hub. |
| 📓 Field Notes | Build report in `FIELD_NOTES.md`. |

---

## How to use

### On Hugging Face Spaces

Runs on **CPU Basic** (2 vCPU, 16 GB RAM). Set `HF_TOKEN` in Space Secrets before first use.

1. **Load a sample wardrobe** — *Add Clothes* → *Load Dataset* (50 garments from a public HF dataset; ~15–45 min on CPU with live progress logs).
2. **Or upload your own** — drag a flat-lay photo, review auto-detected boxes, click *Analyse* (~30–90 s per garment on CPU).
3. **Get Dressed** — type an occasion, hit *Generate* (~5–15 s for LLM ranking).
4. **Ask** — chat about outfits, care, or what you own (~5–15 s per response).

**Sample datasets:**

| Key | Dataset | Notes |
|-----|---------|-------|
| `second-hand` | `fnauman/fashion-second-hand-front-only-rgb` | Individual garments, no detection step |
| `fashion-1k` | `Codatta/Fashion-1K` | Multi-garment photos, slower (needs detection) |

### Local development (GPU accelerated)

```bash
cd packages/wardrobe-us
python -m venv .venv && source .venv/bin/activate

# Base deps (CPU wheels — same as HF Space):
pip install -r requirements.txt

# Override llama-cpp-python with CUDA 12.4 GPU wheel:
pip install llama-cpp-python==0.3.28 \
  --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124 \
  --force-reinstall --no-deps

# Custom minimal frontend (default):
python app.py

# Full Gradio Blocks UI:
python app.py --default
```

Requires a CUDA GPU with ≥8 GB VRAM for GPU mode. Without CUDA, inference falls back to CPU automatically. Copy `.env.example` to `.env` and set `HF_TOKEN`.

**Pre-build a sample catalog offline** (optional):

```bash
python scripts/build_sample_wardrobe.py --dataset second-hand --target 50
```

---

## Architecture

```
app.py                         # Entry point (--ui default | --default)
src/
  ui/
    index.html                 # Custom frontend (Alpine.js + Annotorious)
    style.css
  model_loader.py              # GGUF singleton (Gemma 3 4B, n_ctx=4096)
  vision.py                    # VLM attribute extraction pipeline
  detector/                    # Pluggable garment detection
    _registry.py               # @register("yolos") pattern
    backends/                  # yolos | yolov8 | grounding_dino
  catalog.py                   # JSON catalog CRUD
  combinations.py              # Outfit generation + LLM ranking
  assistant.py                 # Chat with wardrobe context
  storage.py                   # Local / S3 image storage
  settings.py                  # Runtime config (data/settings.json)
data/
  catalog.json                 # Garment metadata
  garments/                    # Cropped garment images
  outfits.json                 # Liked outfit preferences
  _uploads/                    # Temp images during crop workflow
```

### API endpoints (custom UI)

Exposed via `gradio.Server` and consumed by `@gradio/client`:

| Endpoint | Purpose |
|----------|---------|
| `prepare_image` | Save upload, auto-detect boxes → token + image URL for editor |
| `analyze_boxes` | Crop user-confirmed boxes, VLM extract, add to catalog |
| `add_photo` | One-shot upload + auto-detect + extract (no manual crop) |
| `get_wardrobe` | Full catalog with cache-busted image URLs |
| `get_combinations` | Generate + LLM-rank outfits (top 20 returned) |
| `rate_outfit` | Save like/dislike preference |
| `ask_question` | Natural-language wardrobe chat |
| `load_dataset` | Stream dataset processing progress (generator) |

Static mounts: `/garments` (catalog images), `/uploads` (temp crop images).

### Outfit ranking

1. Rule-based generation: all compatible top+bottom pairs (season + formality filters).
2. LLM ranking: up to 20 diverse combinations sent to Gemma 3 4B with a compact prompt (fits `n_ctx=4096`). Remaining combos appended in original order.
3. User likes feed back into future ranking prompts as style signals.

---

## Environment variables

| Variable | Required | Description |
|----------|----------|-------------|
| `HF_TOKEN` | Yes | Hugging Face token for model/dataset downloads |
| `STORAGE_BACKEND` | No | `local` (default) or `s3` |
| `S3_BUCKET_NAME` | If S3 | Bucket name |
| `S3_ENDPOINT_URL` | If S3 | S3 endpoint |
| `AWS_ACCESS_KEY_ID` | If S3 | AWS credentials |
| `AWS_SECRET_ACCESS_KEY` | If S3 | AWS credentials |
| `DETECTION_BACKEND` | No | `yolos` (default), `yolov8`, or `grounding_dino` |
| `CUDA_VISIBLE_DEVICES` | No | GPU index (local only; forced to CPU on Spaces) |

---

## Performance notes

| Task | CPU Basic (Space) | Local GPU |
|------|-------------------|-----------|
| First model download | ~2–3 min | ~2–3 min |
| Garment extraction | ~30–90 s each | ~3–10 s each |
| Dataset load (50 items) | ~15–45 min | ~5–15 min |
| Outfit ranking | ~5–15 s | ~2–5 s |
| Ask response | ~5–15 s | ~2–5 s |

**Detection tips:** YOLOS-tiny works best on flat-lay photos. Hanger or worn-garment photos are harder — use the manual bounding-box editor as fallback.

**VLM accuracy:** At 4B parameters, color and type labels are usually good but not perfect (e.g. navy vs black). Descriptions and structured JSON parsing with regex fallback help reliability.

---

## Agent trace (Sharing is Caring)

The full development conversation is published as a dataset on the Hub:

```bash
huggingface-cli upload-large-folder build-small-hackathon/wardrobe-us-agent-trace ./agent-trace --repo-type dataset
```

See also `FIELD_NOTES.md` for architecture decisions, what worked, and lessons learned.

---

## License

MIT