IMPLEMENTATION_README.md

StyleWell Backend Implementation README

This document explains how the StyleWell backend works from the frontend request all the way down to classification, outfit scoring, product scraping, caching, and fallbacks. It is intentionally implementation-focused: it names the actual files, functions, payloads, and decision paths used in this codebase.

1. System Shape

StyleWell is split into three related layers:

1. React frontend in src/.
2. FastAPI backend in StyleWellBackend/.
3. Supabase database, storage, auth, migrations, and older Edge Function logic in supabase/.

The most important thing to understand is that the frontend does not use only one backend surface. Wardrobe persistence is usually handled through Supabase from the frontend, while AI-heavy work is sent to the FastAPI backend.

Current practical flow:

• Authentication: frontend uses Supabase Auth through src/lib/supabase.ts.
• Wardrobe images and garment rows: frontend stores images in Supabase Storage and metadata in the garment_items table when Supabase env vars are configured.
• Classification: frontend calls FastAPI /classify before saving the classified garment to Supabase.
• Outfit matching: frontend reads wardrobe items from Supabase, normalizes them, then sends them to FastAPI /ai/recommend-outfits.
• Shopping suggestions: frontend sends the natural-language shopping request to FastAPI /scraper/recommend.
• Backend-local SQLite: FastAPI also has its own SQLite store for local uploads, backend-side item CRUD, feedback, and persistent matching cache.

This means StyleWell can run in a hybrid mode: Supabase owns user-facing data, and FastAPI owns AI inference, scoring, scraper planning, scraping, and fallback intelligence.

2. Backend Directory Map

StyleWellBackend/app.py

Main FastAPI application. It defines all production routes, CORS, startup lifecycle, NVIDIA model inference helpers, classification, outfit grid scoring, fallback scoring orchestration, scraper planning, scraper UI, image proxying, and error translation.

StyleWellBackend/db.py

SQLite persistence layer. It creates items, outfit_feedback, and search_cache, exposes CRUD helpers, and stores cache entries used by matching.

StyleWellBackend/scoring.py

Deterministic rule-based outfit scoring engine. Used whenever AI grid scoring or multimodal ranking is unavailable, and also used inside scraper product-to-wardrobe match explanations.

StyleWellBackend/scraper.py

Nike scraper and URL builder. It can be run as a separate FastAPI scraper API, but the main backend imports its Nike URL and product extraction functions.

StyleWellBackend/zalando_scraper.py

Zalando URL builder and scraper. It can use Apify first and HTML scraping as fallback. It also has query enrichment helpers for underspecified Zalando searches.

StyleWellBackend/fashion_ai/

Local multimodal recommendation subsystem:

• encoder.py: encodes garment text and images into vectors.
• retriever.py: splits wardrobe by slot and retrieves candidate garments.
• ranker.py: scores candidate outfits with a trained transformer if available, otherwise zero-shot vector scoring.
• classifier.py: model-backed or metadata-backed item classifier and item matcher.
• service.py: orchestrates encoder, retriever, ranker, and classifier.
• schemas.py: dataclasses for encoded items, context, weather, and candidates.

3. Frontend to Backend Connection

Base URL Resolution

Frontend API calls go through src/api/http.ts:

• requestJson() resolves a path into a full URL.
• resolveRequestUrl() calls getFlaskApiUrl() from src/lib/supabase.ts.
• getFlaskApiUrl() checks:
  1. VITE_FLASK_API_URL or VITE_API_BASE_URL.
  2. Supabase app_config row where key = 'flask_api_url'.
  3. Local fallback http://127.0.0.1:5000.

The base URL is normalized by src/config/api.ts:

• normalizeBaseUrl() trims trailing slashes.
• It fixes a common Hugging Face Space typo by converting <owner>-<space>.space into <owner>-<space>.hf.space.
• buildApiUrl(path, baseUrl) joins the backend base URL with endpoint paths.

API Mode

src/config/api.ts defines API_MODE:

• Explicit VITE_API_MODE wins.
• Otherwise frontend uses live if Supabase or API base env vars exist.
• Otherwise it uses mock.

Several frontend functions branch on isLiveApiMode():

• In live mode, matching and scraper calls go to FastAPI.
• In mock mode, frontend returns rule-based or provider-search fallback suggestions.

Route Fallbacks

The frontend tries multiple route shapes for compatibility with different deployments:

Matching in src/api/matching.ts tries:

• /ai/recommend-outfits
• /api/ai/recommend-outfits
• /recommend-outfits
• /api/recommend-outfits

Suggestions in src/api/client.ts tries:

• /suggestions
• /api/suggestions

Scraper recommendation in src/api/client.ts tries:

• /scraper/recommend
• /api/scraper/recommend

It only advances to the next route when the error looks like a 404. Network failures and server errors are surfaced instead of silently trying unrelated routes.

4. Wardrobe Upload and Classification Flow

The wardrobe upload UI lives in src/components/UploadModal.tsx.

Flow:

1. User picks an image.
2. classifyImage(file) in src/api/client.ts sends a multipart POST /classify request to FastAPI.
3. FastAPI classify() in app.py reads the file into a PIL image.
4. The image is converted to RGB and thumbnailed to 512x512.
5. run_nvidia_inference(pil_image, CLASSIFICATION_PROMPT) sends the image and classification prompt to NVIDIA chat completions.
6. parse_json_from_text() extracts JSON from the model output.
7. normalize_specs() guarantees the response has:
   • type
   • category
   • color
   • pattern
   • fabric
   • fit
   • occasion
   • season
8. The frontend lets the user edit the classification.
9. uploadClothingItem(file, classificationOverride) saves the final garment.

In the current frontend, when Supabase env vars exist, saving goes through saveGarmentWithClassification() in src/lib/supabase.ts:

• Requires an authenticated Supabase user.
• Builds a storage path like {user.id}/{timestamp}_{safe_filename}.{ext}.
• Uploads to the first working bucket among:
  • VITE_SUPABASE_STORAGE_BUCKET
  • clothing-images
  • garments
• Inserts a row into garment_items.
• Tries insert with user_id; if schema does not support user_id, it falls back to inserting without it.
• Resolves a signed URL for the stored image.

If Supabase is not available, uploadClothingItem() can fall back to FastAPI /upload in live mode or a local object URL in mock mode.

FastAPI /upload does its own classification and then stores a normalized item in SQLite. It uses memory://{item_id} as the image URL because the FastAPI backend does not persist the uploaded binary image itself.

5. FastAPI Classification Internals

Classification settings in app.py:

• NVIDIA_INVOKE_URL, default https://integrate.api.nvidia.com/v1/chat/completions
• NVIDIA_MODEL_ID, default qwen/qwen3.5-122b-a10b
• NVIDIA_FALLBACK_MODEL_IDS, comma-separated fallback model IDs.
• NVIDIA_MAX_TOKENS
• NVIDIA_REASONING_MAX_TOKENS
• NVIDIA_TEMPERATURE
• NVIDIA_TOP_P
• NVIDIA_TIMEOUT_SECONDS
• NVIDIA_MAX_RETRIES
• NVIDIA_RETRY_BACKOFF_SECONDS
• NVIDIA_ENABLE_THINKING
• NVIDIA_IMAGE_MAX_DIM

run_nvidia_inference() is used for image+text inference. It:

1. Checks NVIDIA_API_KEY.
2. Tries NVIDIA_MODEL_ID, then every configured fallback model from NVIDIA_FALLBACK_MODEL_IDS.
3. Sends an SSE streaming chat completion request.
4. Extracts streamed text with _extract_streamed_nvidia_text().
5. Retries transient status codes 429, 500, 502, 503, and 504.
6. Applies retry backoff.
7. If the response hits a token limit, it can retry with a larger token budget up to NVIDIA_REASONING_MAX_TOKENS.
8. If a provider says a model is degraded, it tries the next model.

The model response parser is defensive:

• _extract_text_from_nvidia_content() accepts string, list, or dict content shapes.
• _extract_nvidia_text() can read content, reasoning_content, or reasoning.
• parse_json_from_text() first tries full JSON, then extracts the first {...} block.
• normalize_specs() fills missing fields with "Unknown".

If every model/provider path fails, _raise_http_error() converts errors into HTTP responses:

• Missing NVIDIA key: 503.
• NVIDIA gateway error: configured status, usually 502 or 503.
• Bad model payload: 502.
• Unknown server problem: 500.

Frontend error messages are normalized by toUserFacingApiMessage() in src/api/http.ts.

6. Wardrobe Item Normalization

Backend scoring expects a compact normalized shape. _normalize_wardrobe_item() in app.py accepts raw records from SQLite, Supabase, or frontend payloads and returns:

• id
• image_url
• type
• category
• color
• pattern
• fabric
• fit
• season
• style
• occasion
• description

Important logic:

• If category is missing, it reads from description.category or description.type.
• If type is missing, it calls _infer_type(category).
• _infer_type() maps shirt, tee, top, kurta, blouse, hoodie, sweater, blazer, jacket, polo to topwear.
• It maps jean, pant, trouser, short, skirt, jogger, palazzo, chino to bottomwear.
• Everything else becomes others.
• style comes from item.style, description.occasion, description.style, or falls back to casual.

Frontend has a parallel slot-normalization helper in src/lib/wardrobeSlots.ts, and API normalizers call it so the UI can group items into topwear, bottomwear, and others.

7. SQLite Storage

db.py chooses the database path with _resolve_db_path():

1. DB_PATH if provided.
2. /data/wardrobe.db if running on Hugging Face Spaces with persistent storage.
3. ./wardrobe.db for local development.

_conn() opens SQLite with:

• check_same_thread=False
• row_factory=sqlite3.Row
• WAL journal mode
• foreign keys enabled
• automatic commit/rollback

init_db() creates:

items

Stores backend-local wardrobe items with scalar metadata and JSON description.

Columns:

• id
• image_url
• category
• color
• pattern
• fabric
• fit
• season
• style
• type
• description
• created_at

outfit_feedback

Stores feedback for a top/bottom pair.

Columns:

• id
• top_id
• bottom_id
• occasion
• action, constrained to wear, skip, or save
• score
• created_at

search_cache

Generic JSON cache table.

Columns:

• cache_key
• payload
• created_at
• expires_at

cache_get() deletes expired rows on read. cache_set() upserts payloads with TTL.

8. Outfit Matching: Frontend Flow

The matching page is src/pages/Matching.tsx.

User controls:

• Select occasion.
• Optionally lock a top.
• Optionally lock a bottom.
• Optionally lock an others item.
• Click Find Outfits.

State lives in src/store/outfitStore.ts.

When the page loads:

• fetchWardrobe() calls getWardrobeItems().
• Items are normalized into frontend ClothingItem objects.
• Slots are inferred with inferWardrobeSlot().

When the user finds outfits:

1. findOutfits() computes lock state.
2. It resolves a matching cache category:
   • locked other -> others
   • locked bottom only -> bottomwear
   • locked top -> topwear
   • default -> topwear
3. It builds a lock signature like top:{id|-}|bottom:{id|-}|other:{id|-}.
4. It gets the full wardrobe context.
5. It computes a wardrobe hash:
   • global hash when locked selections exist.
   • category hash for prefetch/default category caching.
6. It checks local cache in outfitMatchingCache.v1 in localStorage.
7. If no cache hit, it calls fetchWardrobeOutfits() in src/api/matching.ts.

fetchWardrobeOutfits() sends this payload to FastAPI:

{
  "occasion": "casual",
  "top_k": 5,
  "top_selected": null,
  "bottom_selected": null,
  "other_selected": null,
  "wardrobe_items": [],
  "user_id": "optional",
  "cache_category": "topwear",
  "wardrobe_hash": "optional",
  "lock_signature": "optional"
}

The backend response is normalized by normalizeMatchingResponse() before it is rendered in OutfitCard.

9. Outfit Matching: Backend Route

The main route is POST /ai/recommend-outfits in app.py.

It accepts:

• occasion
• wardrobe_items
• top_selected
• bottom_selected
• other_selected
• weather
• user_profile
• region
• top_k
• candidate_pool
• diversity_lambda
• cache_category
• wardrobe_hash
• lock_signature
• user_id

Validation and normalization:

• wardrobe_items must be a list.
• If wardrobe_items is empty, backend reads from SQLite item_get_all().
• Every item is passed through _normalize_wardrobe_item().
• top_k is clamped to 1..20.
• candidate_pool is clamped to 4..64.
• diversity_lambda is clamped to 0..0.95.

Case labels:

• Case A: no top or bottom locked.
• Case B: top locked.
• Case C: bottom locked.
• Case D: top and bottom locked.
• Case E: standalone others item.

Special handling:

• If a locked other_selected item is an others item, the backend returns standalone recommendations using fallback scoring, because others may be a dress, kurta, saree, jumpsuit, or another single-piece outfit.
• If there are no tops or no bottoms but there are others, it returns standalone others outfits.
• If there are no usable items, it returns an empty response with a notice.

10. Matching Cache

There are two caches:

Frontend cache

src/store/cacheUtils.ts stores matching responses in localStorage under outfitMatchingCache.v1.

Cache key:

{category}|{occasion}|{wardrobeHash}|{lockSignature}

buildWardrobeHash() hashes only items in a category. buildGlobalWardrobeHash() hashes all wardrobe items. The hash includes item type, id, and updated/created timestamp, so changes invalidate cached outfits.

Backend cache

app.py stores matching responses in:

• In-memory MATCHING_RESULT_CACHE.
• SQLite search_cache under keys prefixed by matching:.

Backend matching cache key:

{user_id}|{cache_category}|{occasion}|{wardrobe_hash}|{lock_signature}

The backend cache is used only when both cache_category and wardrobe_hash are supplied. TTL is controlled by MATCHING_RESULT_CACHE_TTL_SECONDS, default 86400.

11. AI Grid Outfit Scoring

The preferred backend scoring path is _recommend_outfits_with_ai_grid().

It works like this:

1. _resolve_outfit_grid_sources() splits wardrobe into top, bottom, and others pools.
2. If a top is locked, only that top is used and bottoms are ranked against it.
3. If a bottom is locked, only that bottom is used and tops are ranked against it.
4. If both are locked, exactly that pair is evaluated.
5. If other_selected is provided, it is included as a locked Row 3 candidate.
6. The candidate pools may be reduced before image grid scoring:
   • OUTFIT_GRID_MAX_TOP_ITEMS, default 4.
   • OUTFIT_GRID_MAX_BOTTOM_ITEMS, default 4.
7. If OUTFIT_TEXT_PRESELECT_ENABLED=true, _select_grid_candidates_with_text_ai() asks NVIDIA text inference to pick the strongest candidates before grid scoring.
8. _build_outfit_grid_session() creates a composite image:
   • Row 1: topwear.
   • Row 2: bottomwear.
   • Row 3: optional others.
   • Every cell gets a coordinate like 1:1, 2:3, 3:1.
9. The composite image is saved to OUTFIT_GRID_SESSION_DIR.
10. _grid_scoring_prompt() builds a prompt containing:
    • occasion
    • weather
    • user profile
    • region
    • anchor mode
    • locked cell indices
    • metadata map
    • combination count
    • requested top_k
11. run_nvidia_inference() sends the grid image plus prompt to the vision model.
12. parse_json_from_text() extracts JSON.
13. _normalize_ai_outfit_payload() maps returned grid coordinates back to actual wardrobe items.

The model must return recommendations with:

• top_index
• bottom_index
• other_index
• score
• breakdown
• reason
• tip

The backend then returns:

• recommendations for normal cases.
• selected_outfit_score plus improved_recommendations for Case D.
• engine_version: ai-grid-v1.

If the AI grid path fails, the backend does not fail the user. It falls back.

12. Multimodal Fallback Recommendation Service

If AI grid scoring fails, _current_fallback_recommendations() first tries fashion_ai:

get_recommendation_service().recommend(...)

That service is created once as a singleton in fashion_ai/service.py.

Encoder

FashionItemEncoder in encoder.py builds garment embeddings.

Item prompt example:

Fashion product photo of a navy solid cotton shirt, regular fit, casual style, suitable for all-season, worn as top.

It encodes both text and image when possible:

• Text is encoded with the configured model.
• Image URL is fetched and encoded if it is HTTP(S) or a local file.
• Text and image vectors are averaged.

Backends:

• open_clip when model id starts with marqo/ and OpenCLIP is installed.
• Hugging Face Transformers model, default patrickjohncyh/fashion-clip.
• Deterministic hash embedding fallback if model loading fails.

The fallback hash embedding means the system stays usable even without model weights, but the quality is lower.

Retriever

OutfitCandidateRetriever in retriever.py:

• Encodes every wardrobe item.
• Splits items into slots:
  • top
  • bottom
  • shoes
  • accessory
  • unknown
• Uses context vector similarity to rank each slot.
• Uses MMR diversification to avoid near-duplicate candidates.
• Honors locked top, bottom, and other items.

Ranker

NeuralOutfitScorer in ranker.py:

• Loads a trained transformer checkpoint from FASHION_RANKER_CHECKPOINT if available.
• If no checkpoint exists, uses zero-shot geometric scoring.

Transformer scoring input shape:

[CONTEXT, USER, TOP, BOTTOM, SHOES, ACCESSORY]

Zero-shot scoring blends:

• context alignment
• user alignment
• pairwise cohesion
• slot coverage

It returns:

• score
• breakdown
• reason
• tip

Service Cases

MultimodalOutfitRecommendationService.recommend() uses cases:

• A: no locked top/bottom.
• B: locked top.
• C: locked bottom.
• D: locked top and bottom.

For Case D, it scores the selected outfit separately and returns improvement suggestions in improved_recommendations.

13. Deterministic Rule-Based Scoring

If AI grid and fashion_ai both fail, the backend uses scoring.py.

The main entry points are:

• compute_score(top, bottom, occasion, other=None)
• score_pair_full(top, bottom, occasion, other=None)
• recommend_outfits(tops, bottoms, occasion, others, locked_top, locked_bottom, locked_other)

Runtime output details:

• score_pair_full() returns engine_version: scoring-v2.
• recommend_outfits() returns up to TOP_K = 6 combinations.

Weights

WEIGHTS:

• color: 0.30
• style: 0.25
• occasion: 0.20
• fit: 0.13
• pattern: 0.12

Color Score

_color_score() extracts base colors and scores:

• Known complementary pairs: high score, usually 90.
• Neutral with neutral: 82 unless same neutral, then 50.
• Neutral with non-neutral: 80.
• Analogous colors: 60.
• Same non-neutral color: 45.
• Unknown/missing: 60.

Complementary examples:

• blue + beige
• black + white
• navy + khaki
• olive + tan
• burgundy + grey
• mustard + navy

Style Score

_style_score() maps style or occasion to:

• casual
• formal
• streetwear
• party
• sports

Then _STYLE_MATRIX scores pair compatibility. Examples:

• formal + formal: 90
• casual + casual: 85
• streetwear + streetwear: 88
• sports + formal: 28
• formal + streetwear: 48

Occasion Score

_occasion_score() maps styles to valid occasions:

• casual: casual, everyday, weekend, college, brunch
• formal: formal, work, interview, business, office, wedding, meeting
• party: party, festive, ethnic, diwali, celebration, date
• sports: sports, gym, active, outdoor, trekking
• streetwear: casual, streetwear, everyday, college

Formal occasions are stricter:

• Both pieces fit: 90.
• One piece fits: 60 for formal, 70 otherwise.
• Neither fits: 25 for formal, 35 otherwise.

Fit Score

_fit_score() uses _FIT_MATRIX.

Examples:

• oversized top + slim bottom: 92.
• regular + regular: 80.
• oversized + oversized: 55.
• slim + regular: 82.

Pattern Score

_pattern_score():

• Both patterned: 55.
• One patterned, one solid: 88.
• Both solid/plain: 75.

Season and Fabric Penalties

_season_penalty() subtracts points:

• Heavy fabrics in summer: 18.
• Very light fabrics in winter: 12.

Heavy fabrics:

• wool
• leather
• velvet
• tweed
• corduroy
• fleece

Light fabrics:

• linen
• cotton
• silk
• chiffon
• georgette

Veto Caps

After weighted scoring, fatal flaws cap the final score:

• color score <= 50: cap final score at 68.
• style score <= 48: cap final score at 58.
• occasion score <= 40: cap final score at 52.
• both patterned plus weak color: cap final score at 72.

Other Items

When an other item is included, the backend blends top-bottom scores with top-other and bottom-other scores:

• primary top-bottom breakdown contributes 65%.
• extra pair average contributes 35%.

This lets accessories, footwear, or outerwear influence the outfit without overpowering the main top/bottom pair.

Explanations

build_reason() creates user-facing text from the strongest and weakest scoring dimensions. build_tip() gives the styling advice shown in the UI.

Diversity Penalty

recommend_outfits() applies _apply_diversity_penalty() before final ranking:

• Near-duplicate looks (same top color, bottom color, and other-item color) are penalized.
• Penalty is -10 per prior similar outfit.
• Outfits are re-sorted after penalty and then truncated to top TOP_K.

14. Standalone Others

The code treats others specially because many garments are complete outfits by themselves:

• dresses
• kurtas
• sarees
• lehengas
• jumpsuits
• rompers
• gowns
• co-ord sets

In fashion_ai/encoder.py, standalone outfit keywords become slot unknown.

In app.py:

• _fallback_rule_recommendations() can score others by pairing the item with itself.
• _occasion_prefers_standalone_others() boosts standalone others for wedding, festive, ethnic, ceremony, engagement, reception, sangeet, haldi, and mehndi occasions.
• _merge_standalone_others_for_priority_occasions() merges boosted standalone outfits into normal recommendations for those occasions.

This prevents ethnic or single-piece outfits from being ignored just because they are not topwear or bottomwear.

15. Score Outfit Endpoint

POST /ai/score-outfit scores one outfit.

Required:

• top
• bottom

Optional:

• other
• occasion
• weather
• user_profile
• region

Scoring path:

1. Try AI grid scoring with only the supplied items.
2. If that fails, try fashion_ai service score_outfit().
3. If that fails, use score_pair_full().

Response fields:

• score
• color_score
• style_score
• occasion_score
• fit_score
• pattern_score
• season_score
• reason
• tip
• engine_version

16. Gap Analysis

POST /ai/gap-analysis uses _gap_suggestions().

It is deterministic, not model-generated:

• If no topwear exists, suggest adding topwear.
• If no bottomwear exists, suggest adding bottomwear.
• If one category is much larger than the other by more than 2 items, suggest balancing the wardrobe.
• Otherwise suggest adding one occasion-specific versatile piece.

17. Feedback

POST /feedback records preference signals into SQLite.

Required:

• top_id
• bottom_id
• action

action must be:

• wear
• skip
• save

Optional:

• occasion
• score

The backend validates that both item IDs exist before writing feedback.

18. Shopping Suggestions: Frontend Flow

The shopping UI is src/pages/Suggestions.tsx.

User provides:

• natural-language request
• store selection: default, Nike, or Zalando
• gender: men, women, unisex

handleGenerate() calls getGemmaScraperRecommendations() in src/api/client.ts. The function name says Gemma, but the current backend planner uses the configured NVIDIA model through NVIDIA-compatible inference.

Before the scraper call, frontend:

• Reads wardrobe count with getWardrobeItems().
• Shows how many total, topwear, bottomwear, and others items are loaded.
• Checks backend /health and verifies nvidia_api_configured is truthy in live mode.

Then it posts:

{
  "user_prompt": "Need a formal office shirt...",
  "occasion": "auto",
  "gender": "men",
  "target_category": "both",
  "filters": {},
  "preferences": "",
  "store": "nike"
}

The response contains:

• query_plan
• search_urls
• product_urls
• products
• intermediate_steps
• plan_source
• plan_error
• scrape_error
• saved_json_path

The UI renders products directly from products.

19. Scraper Recommendation Route

The main route is POST /scraper/recommend.

It performs these steps:

1. Read user_prompt.
2. Infer structured intent from the prompt with _infer_structured_request_from_prompt().
3. Merge explicit payload fields with inferred fields.
4. Merge inferred colors, include keywords, and exclude keywords into filters.
5. Build a scraper query cache key.
6. Return cached result when available.
7. Otherwise call the scraper planner function in app.py.
8. Store the result in in-memory scraper cache.

The scraper cache key is:

md5(user_prompt.lower().strip()|store|gender|target_category)

TTL is SCRAPER_QUERY_CACHE_TTL_SECONDS, default 15 days.

20. Prompt Intent Inference

_infer_structured_request_from_prompt() reads the user prompt and extracts:

• target_category
• occasion
• gender
• preferred_colors
• include_keywords
• exclude_keywords

Target category hints:

• topwear tokens: top, shirt, blazer, jacket, polo, tee, t-shirt, kurta, upper
• bottomwear tokens: bottom, trouser, trousers, pants, jeans, shorts, joggers, lower

Occasion buckets:

• formal: formal, interview, office, work, business, meeting, wedding
• party: party, festive, diwali, celebration, date, ethnic
• sports: sports, gym, workout, training, running, active
• casual: casual, daily, everyday, weekend, outing

Color terms include:

• black
• white
• navy
• blue
• grey
• beige
• olive
• green
• brown
• khaki
• cream
• maroon
• charcoal
• tan

Include keywords currently recognized:

• formal
• structured
• minimal
• smart
• elegant
• tailored

Exclude keywords are recognized only when phrased like avoid hoodie, no hoodie, or without hoodie.

21. NVIDIA Model Shopping Planner

The scraper planner function in app.py is the heart of shopping suggestions. The implementation currently has a legacy function name, but the runtime behavior is NVIDIA-model-driven.

It builds:

• wardrobe snapshot from SQLite via _wardrobe_metadata_snapshot()
• requested target category
• safe filters
• planning context from _build_scraper_planning_context()
• prompt from _build_scraper_plan_prompt()

Wardrobe Snapshot

_wardrobe_metadata_snapshot() reads backend SQLite items and returns:

• total_items
• item metadata list
• counts by {type}|{occasion}

Important limitation: if your frontend wardrobe is stored only in Supabase and not mirrored into FastAPI SQLite, the shopping planner's backend SQLite wardrobe snapshot may be empty or incomplete. The frontend currently sends wardrobe data for /suggestions, but /scraper/recommend builds its snapshot from backend SQLite.

Planning Context

_build_scraper_planning_context() computes:

• requested target category
• resolved target category
• occasion bucket
• gender preference
• allowed categories
• color shortlist
• color resonance scores
• style direction
• reference slot
• reference item IDs
• dominant categories/colors by slot

If user asks for both, _resolve_target_category() chooses the category that is underrepresented:

• If top count is less than or equal to bottom count, recommend topwear.
• Otherwise recommend bottomwear.

Allowed categories come from SCRAPER_CATEGORY_POLICY.

For topwear:

• formal: shirt, polo, jacket
• party: shirt, jacket, polo
• sports: jersey, t-shirt, hoodie
• casual: shirt, t-shirt, polo, jacket, hoodie

For bottomwear:

• formal: trousers, pants
• party: trousers, pants, jeans
• sports: joggers, shorts, tights, leggings
• casual: jeans, pants, shorts, joggers, trousers

Formal disallowed terms are blocked:

• hoodie
• sweatshirt
• joggers
• shorts
• tank top
• tights
• leggings

Color Resonance

_rank_color_resonance() scores candidate colors using:

• colors from the reference slot
• colors across topwear and bottomwear
• user preferred colors
• occasion boost

The reference slot is the opposite of the target:

• If recommending topwear, reference bottomwear colors.
• If recommending bottomwear, reference topwear colors.

Score formula:

(reference_count * 3) + global_count + preferred_bonus + occasion_bonus

Preferred bonus is 2. Occasion bonus is 1 for formal-friendly colors or sports-friendly colors.

The planner uses the top ranked colors as color_shortlist.

Prompt Contract

The NVIDIA model planner prompt requires strict JSON:

{
  "target_category": "topwear|bottomwear",
  "color": "string from color_shortlist",
  "category": "string from allowed_categories post-vetting",
  "gender": "men|women|unisex",
  "style_direction": "formal-smart|business-casual|casual-polished|etc",
  "reference_item_ids": [],
  "query": "commerce-ready search string",
  "wardrobe_grounding": "specific evidence from wardrobe_snapshot",
  "reason": "concise strategic justification"
}

_recover_scraper_plan_from_text() makes the planner robust:

• It first tries normal JSON parsing.
• If parsing fails, it scans model text for allowed categories, colors, gender, and a quoted query.
• If enough fields can be recovered, it builds a valid plan.

If the NVIDIA model planner fails and strict planner mode is disabled, _fallback_scraper_plan() builds a deterministic plan using:

• first allowed category
• first color shortlist entry, or black
• normalized gender
• style direction from planning context

The plan source becomes fallback.

22. Search URL Formation

After a query plan is normalized, the scraper planner creates a ScraperRecommendation:

ScraperRecommendation(
    color=color,
    category=category,
    gender=plan_gender,
)

Then it calls _build_store_search_urls_from_query().

Nike URL Formation

Nike functions come from scraper.py.

build_search_urls_from_query(query, store='nike', gender=None):

• If gender is provided, prefixes the query with gender if it is not already present.
• Returns one Nike URL:

https://www.nike.com/w?q={encoded_query}&vst={encoded_query}

• If gender is not provided, returns three URLs:
  • men + query
  • women + query
  • unmodified query

build_nike_search_url(color, category, gender) uses:

• CATEGORY_ALIASES to normalize category words.
• query parts: gender plural, color, category.
• urlencode({"q": query, "vst": query}).

Example:

https://www.nike.com/w?q=mens+navy+shirt&vst=mens+navy+shirt

Zalando URL Formation

Zalando functions come from zalando_scraper.py.

build_zalando_search_url(query, gender):

1. Normalizes gender to men, women, or unisex.
2. Picks a path with _pick_category_path().
3. URL-encodes q.
4. Returns:

https://www.zalando.co.uk/{path}?q={encoded_query}

Path examples:

• men clothing: mens-clothing
• women clothing: womens-clothing
• unisex clothing: clothing
• women dresses: womens-clothing-dresses
• men shoes: mens-shoes

build_zalando_search_urls_from_request() can enrich underspecified queries, compose a final search query, and return URLs plus enrichment metadata.

In the main backend app, the completion function is passed as None for this enrichment step because the NVIDIA model planner already produced the query.

23. Product Scraping

After URLs are generated, the scraper planner loops through each URL and calls _extract_store_product_summaries().

Nike Scraping

extract_product_summaries() in scraper.py:

1. Downloads the page using requests with a desktop browser user agent.
2. Parses with BeautifulSoup and lxml.
3. Looks for div.product-card__body.
4. Inside each card, finds a.product-card__link-overlay.
5. Extracts:
   • product link
   • title
   • price
   • image URL
6. Deduplicates by link.

If Nike markup changes and no summaries are found:

• It falls back to extract_product_urls().
• That scans anchors with product URL patterns like /t/.
• Products get N/A for title/price and empty image.

Zalando Scraping

extract_product_summaries() in zalando_scraper.py:

1. Validates search URL.
2. Uses Apify if use_apify=True and APIFY_API_TOKEN is configured.
3. If Apify returns no products or errors, falls back to HTML scraping.
4. Optionally runs postprocess if products are missing important fields.
5. If both Apify and HTML fail, raises requests.RequestException.

Apify path:

• _scrape_with_apify() calls the actor endpoint.
• It tries two payload variants:
  • startUrls as string array
  • startUrls as object array
• It caps result count to APIFY_MAX_RESULTS, default 20.
• If sync dataset call fails or returns empty, it tries _scrape_with_apify_run_dataset_fallback().

Apify run-dataset fallback:

• Starts actor run through /acts/{actor_id}/runs.
• Waits for finish using waitForFinish.
• Reads the actor default dataset.
• Normalizes the dataset items.

HTML fallback:

• Requests the Zalando search page.
• Selects product-like cards using article, div[data-testid*="product"], and li[data-testid*="product"].
• Finds product links with /p/.
• Extracts name, price, image, and link.

Zalando normalization handles different Apify payload shapes:

• name, title, productName, product_name
• price, currentPrice, displayPrice, priceLabel
• promotionalPrice, originalPrice, discountPercent
• brand, brandName
• image, imageUrl, image_url, thumbnail
• url, productUrl, item_link, link

24. Product Relevance Filtering

After scraping, the scraper planner filters products with _is_relevant_scraped_product().

It rejects products when:

• Product text is empty.
• Product text contains excluded categories such as socks, underwear, swimwear, belts, hats, wallets, bags, watches, shoes, sneakers, boots, sandals, or slippers.
• Product text does not contain planned category keywords.
• Target is topwear but no topwear terms appear.
• Target is bottomwear but no bottomwear terms appear.
• Occasion is formal and product text contains blocked casual/sport terms.

If strict filtering removes everything but there are unfiltered products available, backend uses scrape_fallback:

• It returns the first unfiltered products up to the scrape limit.
• It records an intermediate step explaining that strict relevance filtering returned no products.

25. Product to Wardrobe Matching

Each accepted product is enriched by _enrich_scraper_products_with_matches().

For each product, _build_product_match_context():

1. Determines product slot:
   • target topwear -> product is topwear.
   • target bottomwear -> product is bottomwear.
2. Determines the complementary wardrobe slot:
   • topwear product matches existing bottomwear.
   • bottomwear product matches existing topwear.
3. Builds a product stub using:
   • planned category
   • planned color
   • planned style direction
   • occasion bucket
4. Scores that stub against each complementary wardrobe item using score_pair_full().
5. Sorts matches by score.
6. Adds top 3 matched garments.
7. Creates a reason that explicitly says it only evaluates against the complementary slot.

Returned product enrichment:

• reason
• match_score
• matched_with_slot
• matched_garments

This prevents bad explanations like matching a recommended shirt against existing shirts.

26. Suggestions Endpoint

POST /suggestions and POST /api/suggestions wrap the scraper recommendation flow for the older suggestions UI/API shape.

Input:

• occasion
• target_category
• gender_preference
• filters
• max_results
• store

It calls _build_shopping_suggestions_from_scraper():

1. Converts filters into a preferences string.
2. Calls the scraper planner function.
3. Maps products into suggestions.
4. Assigns a basic match score: max(65, 95 - index * 4).
5. Includes product URL, image, reason, category, color, query, scrape status, gender, and matched garments.

The frontend getSuggestions() has additional fallback behavior:

• It times out after VITE_SCRAPER_TIMEOUT_MS, default 20000.
• It throttles requests by VITE_SCRAPER_MIN_INTERVAL_MS, default 1200.
• It logs scrape events.
• It builds fallback suggestions if live scraping times out, fails, returns malformed data, or returns too few results.

27. Frontend Shopping Fallbacks

Fallback suggestion logic lives in src/api/client.ts.

If live scraper data fails, frontend creates suggestions from templates and provider search URLs.

Provider fallback behavior:

• buildFallbackSuggestions() produces ranked fallback cards using wardrobe profile and request filters.
• createProviderFallbackSuggestions() appends external provider search links when live results are below SCRAPER_MIN_RESULTS, default 4.
• Fallback URLs often point to Google searches, for example:

https://www.google.com/search?q={encoded_search_query}

The frontend labels these with:

• scrape_status: "fallback"
• scrape_error explaining why fallback was used.

Live suggestions get a small ranking boost over fallback suggestions.

Filtering and ranking:

• computeFilterBoost() gives boosts for preferred colors, patterns, fabrics, fits, styles, seasons, and include keywords.
• Exclude keywords reject a candidate.
• Gender filtering removes mismatched products unless gender is any or unisex.
• Final ranking uses match score + filter boost + live boost.

28. Image Proxy

GET /image-proxy?url=... fetches a remote image through the backend.

It:

• Accepts only HTTP(S).
• Uses browser-like headers.
• Returns the original content type or image/jpeg.
• Adds Cache-Control: public, max-age=3600.

This is useful when browser CORS or hotlinking blocks direct product image rendering.

29. Health Endpoint

GET /health returns:

• status
• classification_provider
• model
• nvidia_api_configured
• nvidia_invoke_url
• engine_version
• outfit_matching_provider

The frontend uses nvidia_api_configured before scraper recommendation requests. If it is false, it shows a configuration error rather than waiting for a planner call that cannot work.

30. Supabase Edge Functions

The repo still contains Supabase Edge Functions:

• supabase/functions/wardrobe/index.ts
• supabase/functions/matching/index.ts
• supabase/functions/recommendations/index.ts
• shared scoring in supabase/functions/_shared/scoring.ts

These functions implement an earlier/alternate backend path:

• Authenticated wardrobe CRUD.
• Mock image classification and embedding generation.
• Rule-based matching with an AI placeholder.
• Occasion recommendation with outfit history writes.

The current React API clients mostly call Supabase directly for storage/data and FastAPI for AI work. The Edge Functions are useful historical context or a possible serverless deployment path, but they are not the main implementation path for the current matching and scraper UI.

31. Environment Variables

Important FastAPI variables:

• DB_PATH: override SQLite file path.
• NVIDIA_API_KEY: required for image classification, NVIDIA model planning, and AI grid scoring.
• NVIDIA_INVOKE_URL: NVIDIA-compatible chat completions endpoint.
• NVIDIA_MODEL_ID: primary vision/text model.
• NVIDIA_FALLBACK_MODEL_IDS: comma-separated fallback model IDs.
• NVIDIA_MAX_TOKENS
• NVIDIA_REASONING_MAX_TOKENS
• NVIDIA_TEMPERATURE
• NVIDIA_TOP_P
• NVIDIA_TIMEOUT_SECONDS
• NVIDIA_MAX_RETRIES
• NVIDIA_RETRY_BACKOFF_SECONDS
• NVIDIA_ENABLE_THINKING
• NVIDIA_IMAGE_MAX_DIM
• NVIDIA planner model ID: shopping planner model setting used by the backend.
• NVIDIA planner max tokens: planner token budget used by the backend.
• SCRAPER_DEFAULT_STORE: default nike.
• SCRAPER_QUERY_CACHE_TTL_SECONDS: default 15 days.
• MATCHING_RESULT_CACHE_MAX: default 500.
• MATCHING_RESULT_CACHE_TTL_SECONDS: default 1 day.
• APIFY_API_TOKEN: enables Zalando Apify scraping.
• APIFY_ACTOR_ENDPOINT
• APIFY_MIN_TIMEOUT_SECONDS
• APIFY_WAIT_FOR_FINISH_SECONDS
• ZALANDO_HTML_TIMEOUT_SECONDS
• FASHION_ENCODER_MODEL_ID
• FASHION_EMBEDDING_DIM
• FASHION_IMAGE_TIMEOUT_SECONDS
• FASHION_EMBEDDING_CACHE_SIZE
• FASHION_RANKER_CHECKPOINT
• FASHION_CLASSIFIER_NVIDIA_MODEL
• FASHION_CLASSIFIER_HF_MODEL
• FASHION_CLASSIFIER_CACHE_SIZE
• OUTFIT_GRID_CELL_SIZE
• OUTFIT_GRID_LABEL_HEIGHT
• OUTFIT_GRID_PADDING
• OUTFIT_GRID_FETCH_TIMEOUT_SECONDS
• OUTFIT_GRID_SESSION_TTL_SECONDS
• OUTFIT_GRID_SESSION_DIR
• OUTFIT_GRID_MAX_TOP_ITEMS
• OUTFIT_GRID_MAX_BOTTOM_ITEMS
• OUTFIT_ANCHOR_MIN_SCORE
• OUTFIT_TEXT_PRESELECT_ENABLED
• OUTFIT_TEXT_SELECTOR_MAX_TOKENS
• OUTFIT_AI_MAX_TOKENS

Important frontend variables:

• VITE_API_MODE
• VITE_API_BASE_URL
• VITE_FLASK_API_URL
• VITE_SUPABASE_URL
• VITE_SUPABASE_ANON_KEY
• VITE_SUPABASE_STORAGE_BUCKET
• VITE_SCRAPER_TIMEOUT_MS
• VITE_SCRAPER_MIN_INTERVAL_MS
• VITE_SCRAPER_MIN_RESULTS

32. End-to-End Feature Flow Summary

Add Wardrobe Item

UploadModal
-> classifyImage()
-> FastAPI POST /classify
-> NVIDIA vision classification
-> frontend editable classification form
-> uploadClothingItem()
-> Supabase Storage + garment_items
-> wardrobe store updates
-> matching cache prefetch may run

Find Outfits

Matching page
-> outfitStore.findOutfits()
-> frontend cache check
-> fetchWardrobeOutfits()
-> FastAPI POST /ai/recommend-outfits
-> backend cache check
-> AI grid scoring
-> fashion_ai fallback
-> scoring.py fallback
-> frontend response normalizer
-> OutfitCard rendering

Score One Outfit

POST /ai/score-outfit
-> normalize top/bottom/other
-> AI grid scoring
-> fashion_ai score_outfit fallback
-> scoring.py fallback
-> score/breakdown/reason/tip response

Generate Shopping Suggestions

Suggestions page
-> getGemmaScraperRecommendations()
-> backend health check
-> FastAPI POST /scraper/recommend
-> prompt intent inference
-> wardrobe snapshot and planning context
-> NVIDIA model query planner
-> deterministic planner fallback if needed
-> Nike/Zalando URL generation
-> product scraping
-> strict relevance filtering
-> scrape fallback if strict filter removed everything
-> product-to-wardrobe match enrichment
-> saved JSON payload
-> product cards in frontend

33. Known Implementation Nuances

• getGemmaScraperRecommendations() is named after Gemma, but the current backend route uses the configured NVIDIA model through the NVIDIA-compatible inference path.
• The existing StyleWellBackend/README.md mentions Hugging Face primary and NVIDIA fallback, but the current app.py path relies heavily on NVIDIA-compatible chat completions for classification, NVIDIA model planning, and AI grid scoring.
• /scraper/recommend builds wardrobe context from backend SQLite. If the active wardrobe is only in Supabase, planner grounding may not reflect the user's actual Supabase wardrobe unless items are mirrored or this route is updated to accept wardrobe payloads.
• /upload in FastAPI stores memory:// image URLs, so it is useful for backend-local tests but not equivalent to Supabase image persistence.
• The Supabase Edge Functions contain useful earlier scoring and CRUD logic, but the main frontend path now uses Supabase client APIs plus FastAPI.
• There are many fallback layers by design. A failed AI model should degrade to multimodal scoring, deterministic scoring, provider search links, or readable user errors rather than leaving the UI blank.

34. Local Run

From StyleWellBackend/:

pip install -r requirements.txt
python app.py

The backend starts on:

http://0.0.0.0:7860

Health check:

curl http://127.0.0.1:7860/health

Classification:

curl -X POST http://127.0.0.1:7860/classify -F "image=@/path/to/garment.jpg"

Outfit recommendation:

curl -X POST http://127.0.0.1:7860/ai/recommend-outfits \
  -H "Content-Type: application/json" \
  -d '{"occasion":"casual","wardrobe_items":[]}'

Scraper recommendation:

curl -X POST http://127.0.0.1:7860/scraper/recommend \
  -H "Content-Type: application/json" \
  -d '{"user_prompt":"Need a formal navy shirt","gender":"men","store":"nike","max_products":6}'