Spaces:
Runtime error
Runtime error
| title: Wardrobe Us | |
| emoji: 👕 | |
| colorFrom: green | |
| colorTo: blue | |
| sdk: gradio | |
| sdk_version: 6.17.3 | |
| python_version: "3.12" | |
| app_file: app.py | |
| pinned: false | |
| license: mit | |
| hardware: cpu-basic | |
| short_description: AI wardrobe. catalog, combine and ask about your clothes | |
| # 👕 Wardrobe AI | |
| **Turn a physical wardrobe into a searchable, AI-powered catalog — and get outfit ideas from clothes you already own.** | |
| Blog Post: [Gradio x Hugging Face Hackaton](https://ohgm.dev/en/journal/hugging-face-hackaton-2026/) | |
| Built for the [Gradio × Hugging Face Build Small Hackathon](https://huggingface.co/build-small-hackathon) (June 2026). | |
| The original motivation: help someone with 200+ garments who forgets what they own, buys duplicates, and struggles to combine outfits every morning. Wardrobe AI is not a shopping app — it helps you *use* what you already have. | |
| --- | |
| ## What it does | |
| | Step | Description | | |
| |------|-------------| | |
| | **Capture** | Upload photos of your clothes. A detector finds garments, you can adjust bounding boxes, and a VLM extracts structured attributes (type, color, material, pattern, season, formality, description). | | |
| | **Catalog** | Browse your digital wardrobe with images and metadata. Click any garment for a detail panel with full attributes. | | |
| | **Combine** | Generate top+bottom outfit combinations filtered by season and formality rules. Describe an occasion and the LLM re-ranks the best matches. Like/dislike outfits to build style preferences. | | |
| | **Ask** | Chat with your wardrobe in natural language. Answers reference your actual garments with images and descriptions. | | |
| All inference runs locally — no external APIs. | |
| --- | |
| ## Two frontends | |
| The app ships with two UIs sharing the same backend: | |
| | | **Custom UI** (default) | **Gradio Blocks** (`--default`) | | |
| |---|---|---| | |
| | Launch | `python app.py` | `python app.py --default` | | |
| | Stack | `gradio.Server` + Alpine.js + `@gradio/client` | Gradio 6.17 Blocks | | |
| | Language | English | Spanish | | |
| | Best for | End users — clean, minimal UX | Power users — full settings | | |
| | Manual crop editor | Annotorious v3 bounding-box editor | `gradio-image-annotation` | | |
| | Detection backend switch | — | Dropdown in settings | | |
| | Dataset load logs | Real-time log dock (streaming) | Markdown + gallery preview | | |
| | Ask tab | Garment chips with images in replies | Streaming chatbot | | |
| Both modes support sample dataset loading, outfit generation, and wardrobe chat. | |
| --- | |
| ## Tech stack | |
| | Component | Choice | | |
| |-----------|--------| | |
| | VLM + Chat LLM | **Gemma 3 4B IT** (Q4_K_M GGUF) via `llama-cpp-python` | | |
| | Garment detection | **YOLOS-tiny** (default), YOLOv8n, or GroundingDINO — pluggable registry | | |
| | Runtime | llama.cpp — CPU on HF Spaces, CUDA locally | | |
| | UI | `gradio.Server` + Alpine.js (default) or Gradio Blocks | | |
| | Storage | Local filesystem or S3 (configurable) | | |
| | Catalog | `data/catalog.json` + `data/garments/*.jpg` | | |
| | Preferences | `data/outfits.json` (liked combinations) | | |
| **Total parameters: 4 billion** — fits Tiny Titan (≤4B) and runs on CPU Basic (16 GB RAM) with Q4_K_M quantization (~3 GB model). | |
| The same Gemma 3 4B model handles vision extraction, outfit ranking, and chat. A singleton `_ModelManager` hot-swaps between vision (MTMD) and text-only modes. | |
| --- | |
| ## Bonus quests | |
| | Badge | Status | | |
| |-------|--------| | |
| | 🔌 Off the Grid | All inference on Space hardware. No external APIs. | | |
| | 🦙 Llama Champion | Model runs through llama.cpp (`llama-cpp-python`). | | |
| | 🐜 Tiny Titan | Gemma 3 4B — under the 4B threshold. | | |
| | 🎨 Off-Brand | Custom frontend via `gr.Server` + Alpine.js. | | |
| | 📡 Sharing is Caring | Agent trace shared on the Hub. | | |
| | 📓 Field Notes | Build report in `FIELD_NOTES.md`. | | |
| --- | |
| ## How to use | |
| ### On Hugging Face Spaces | |
| Runs on **CPU Basic** (2 vCPU, 16 GB RAM). Set `HF_TOKEN` in Space Secrets before first use. | |
| 1. **Load a sample wardrobe** — *Add Clothes* → *Load Dataset* (50 garments from a public HF dataset; ~15–45 min on CPU with live progress logs). | |
| 2. **Or upload your own** — drag a flat-lay photo, review auto-detected boxes, click *Analyse* (~30–90 s per garment on CPU). | |
| 3. **Get Dressed** — type an occasion, hit *Generate* (~5–15 s for LLM ranking). | |
| 4. **Ask** — chat about outfits, care, or what you own (~5–15 s per response). | |
| **Sample datasets:** | |
| | Key | Dataset | Notes | | |
| |-----|---------|-------| | |
| | `second-hand` | `fnauman/fashion-second-hand-front-only-rgb` | Individual garments, no detection step | | |
| | `fashion-1k` | `Codatta/Fashion-1K` | Multi-garment photos, slower (needs detection) | | |
| ### Local development (GPU accelerated) | |
| ```bash | |
| cd packages/wardrobe-us | |
| python -m venv .venv && source .venv/bin/activate | |
| # Base deps (CPU wheels — same as HF Space): | |
| pip install -r requirements.txt | |
| # Override llama-cpp-python with CUDA 12.4 GPU wheel: | |
| pip install llama-cpp-python==0.3.28 \ | |
| --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu124 \ | |
| --force-reinstall --no-deps | |
| # Custom minimal frontend (default): | |
| python app.py | |
| # Full Gradio Blocks UI: | |
| python app.py --default | |
| ``` | |
| Requires a CUDA GPU with ≥8 GB VRAM for GPU mode. Without CUDA, inference falls back to CPU automatically. Copy `.env.example` to `.env` and set `HF_TOKEN`. | |
| **Pre-build a sample catalog offline** (optional): | |
| ```bash | |
| python scripts/build_sample_wardrobe.py --dataset second-hand --target 50 | |
| ``` | |
| --- | |
| ## Architecture | |
| ``` | |
| app.py # Entry point (--ui default | --default) | |
| src/ | |
| ui/ | |
| index.html # Custom frontend (Alpine.js + Annotorious) | |
| style.css | |
| model_loader.py # GGUF singleton (Gemma 3 4B, n_ctx=4096) | |
| vision.py # VLM attribute extraction pipeline | |
| detector/ # Pluggable garment detection | |
| _registry.py # @register("yolos") pattern | |
| backends/ # yolos | yolov8 | grounding_dino | |
| catalog.py # JSON catalog CRUD | |
| combinations.py # Outfit generation + LLM ranking | |
| assistant.py # Chat with wardrobe context | |
| storage.py # Local / S3 image storage | |
| settings.py # Runtime config (data/settings.json) | |
| data/ | |
| catalog.json # Garment metadata | |
| garments/ # Cropped garment images | |
| outfits.json # Liked outfit preferences | |
| _uploads/ # Temp images during crop workflow | |
| ``` | |
| ### API endpoints (custom UI) | |
| Exposed via `gradio.Server` and consumed by `@gradio/client`: | |
| | Endpoint | Purpose | | |
| |----------|---------| | |
| | `prepare_image` | Save upload, auto-detect boxes → token + image URL for editor | | |
| | `analyze_boxes` | Crop user-confirmed boxes, VLM extract, add to catalog | | |
| | `add_photo` | One-shot upload + auto-detect + extract (no manual crop) | | |
| | `get_wardrobe` | Full catalog with cache-busted image URLs | | |
| | `get_combinations` | Generate + LLM-rank outfits (top 20 returned) | | |
| | `rate_outfit` | Save like/dislike preference | | |
| | `ask_question` | Natural-language wardrobe chat | | |
| | `load_dataset` | Stream dataset processing progress (generator) | | |
| Static mounts: `/garments` (catalog images), `/uploads` (temp crop images). | |
| ### Outfit ranking | |
| 1. Rule-based generation: all compatible top+bottom pairs (season + formality filters). | |
| 2. LLM ranking: up to 20 diverse combinations sent to Gemma 3 4B with a compact prompt (fits `n_ctx=4096`). Remaining combos appended in original order. | |
| 3. User likes feed back into future ranking prompts as style signals. | |
| --- | |
| ## Environment variables | |
| | Variable | Required | Description | | |
| |----------|----------|-------------| | |
| | `HF_TOKEN` | Yes | Hugging Face token for model/dataset downloads | | |
| | `STORAGE_BACKEND` | No | `local` (default) or `s3` | | |
| | `S3_BUCKET_NAME` | If S3 | Bucket name | | |
| | `S3_ENDPOINT_URL` | If S3 | S3 endpoint | | |
| | `AWS_ACCESS_KEY_ID` | If S3 | AWS credentials | | |
| | `AWS_SECRET_ACCESS_KEY` | If S3 | AWS credentials | | |
| | `DETECTION_BACKEND` | No | `yolos` (default), `yolov8`, or `grounding_dino` | | |
| | `CUDA_VISIBLE_DEVICES` | No | GPU index (local only; forced to CPU on Spaces) | | |
| --- | |
| ## Performance notes | |
| | Task | CPU Basic (Space) | Local GPU | | |
| |------|-------------------|-----------| | |
| | First model download | ~2–3 min | ~2–3 min | | |
| | Garment extraction | ~30–90 s each | ~3–10 s each | | |
| | Dataset load (50 items) | ~15–45 min | ~5–15 min | | |
| | Outfit ranking | ~5–15 s | ~2–5 s | | |
| | Ask response | ~5–15 s | ~2–5 s | | |
| **Detection tips:** YOLOS-tiny works best on flat-lay photos. Hanger or worn-garment photos are harder — use the manual bounding-box editor as fallback. | |
| **VLM accuracy:** At 4B parameters, color and type labels are usually good but not perfect (e.g. navy vs black). Descriptions and structured JSON parsing with regex fallback help reliability. | |
| --- | |
| ## Agent trace (Sharing is Caring) | |
| The full development conversation is published as a dataset on the Hub: | |
| ```bash | |
| huggingface-cli upload-large-folder build-small-hackathon/wardrobe-us-agent-trace ./agent-trace --repo-type dataset | |
| ``` | |
| See also `FIELD_NOTES.md` for architecture decisions, what worked, and lessons learned. | |
| --- | |
| ## License | |
| MIT | |