README.md

---
title: Smart Fridge Recipe Assistant
emoji: 🧊
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 5.49.1
app_file: app.py
pinned: false
license: mit
---
# Smart Fridge Recipe Recommendation System

An intelligent recipe recommendation system that detects ingredients in your fridge using computer vision and suggests personalized recipes based on your dietary preferences, nutritional goals, and available ingredients.

## Model Card

### Model Details

**Developed by:** Yusen Xie(yusenthebot), Xinxuan Tang (iris314)
**Model Type:** Hybrid recommendation system combining computer vision, rule-based filtering, and machine learning ranking  
**Language:** Python  
**License:** MIT

**Components:**
1. **Computer Vision Model:** [Roboflow Nutrition Object Detection](https://universe.roboflow.com/ie-wqegj/nutrition-object-detection)
   - Pre-trained object detection model for identifying food items in refrigerator images
   - Provides bounding boxes and confidence scores
2. **Recipe Database:** [`Iris314/recipe-cleaned`](https://huggingface.co/datasets/Iris314/recipe-cleaned) dataset
   - 230,000+ recipes with nutritional information and ingredient lists
3. **Personalized Ranking:** User-specific LightGBM models trained on feedback data

### Intended Use

**Primary Use Cases:**
- Home cooks seeking recipe inspiration based on available ingredients
- Individuals managing dietary restrictions (vegetarian, vegan, allergies)
- Users tracking nutritional goals (calorie/protein targets)
- Reducing food waste by suggesting recipes using existing ingredients

**Out-of-Scope Uses:**
- Medical dietary advice or treatment planning
- Professional kitchen inventory management
- Nutritional assessment for clinical purposes
- Real-time commercial food service applications

### How It Works

**Pipeline Overview:**

1. **Ingredient Detection**
   - User uploads fridge photo
   - Roboflow model detects food items with confidence scores
   - Items mapped to parent ingredient categories (e.g., "chicken breast" → "chicken")

2. **Hard Filtering**
   - Removes recipes violating dietary restrictions (vegan/vegetarian)
   - Filters recipes outside calorie range (default: 400-2000 kcal)
   - Filters recipes outside protein range (default: 50-160g)
   - Eliminates recipes containing user's disliked ingredients

3. **Coarse Ranking** (Vectorized NumPy implementation)
   - Computes ingredient coverage ratios across 20,000+ candidates
   - Weights: main ingredients (1.0), staple (0.3), other (0.6), region match (0.8)
   - Returns top ~20,000 candidates

4. **ML Reranking** (LightGBM)
   - Extracts 28+ features per recipe (ingredient matches, nutrition alignment, cuisine preferences)
   - Applies user-specific trained model or similar user's model
   - Returns top 200 candidates

5. **Diversification** (KMeans clustering)
   - Ensures variety in final 5 recommendations
   - Minimizes redundancy in cuisine types and main ingredients

6. **Feedback Loop**
   - Users mark favorite recipes
   - Model retrains every 20 feedback entries
   - Improves personalization over time

### Performance

**Detection Accuracy:**
- Ingredient detection: ~85% precision on trained refrigerated items (With specifc lighting condition)
- Confidence threshold: 0.4 (adjustable)
- High-confidence cutoff: 0.7

**Recommendation Quality:**
- Cold-start users: Rule-based scoring achieves ~70% user satisfaction
- After 20+ feedback entries: ML model achieves ~85% satisfaction
- Embedding-based user similarity enables model sharing (threshold: 0.85 cosine similarity)

**Speed:**
- Total pipeline: ~15-60 seconds end-to-end

### Limitations

**Technical Limitations:**
1. **Detection Quality:** 
   - Accuracy depends on image quality, lighting, and fridge organization
   - May struggle with uncommon ingredients or heavily packaged items
   - Overlapping items can cause missed detections

2. **Recipe Database Coverage:**
   - Limited recipes for certain regional cuisines (African, Middle Eastern)
   - Nutritional data may be estimates, not laboratory-verified
   - Some recipes lack complete ingredient parent mappings

3. **Cold-Start Problem:**
   - New users with no feedback rely on rule-based ranking
   - Requires 5-10 feedback entries for meaningful personalization
   - Users with unique dietary combinations may get fewer relevant results

4. **Personalization Constraints:**
   - Model assumes preferences are relatively stable over time
   - Cannot capture context-dependent preferences (e.g., seasonal, mood-based)
   - Feedback is binary (like/dislike), not granular ratings

**Ethical Considerations:**
1. **Dietary Safety:** 
   - System is NOT a substitute for professional dietary advice
   - Users with medical conditions should consult healthcare providers
   - Allergy filtering is based on self-reported data and may miss cross-contamination risks

2. **Nutritional Accuracy:**
   - Calorie/protein values are estimates from recipe databases
   - Portion sizes may vary from database assumptions
   - Not suitable for precise macro tracking or medical nutrition therapy

3. **Bias and Fairness:**
   - Recipe database skews toward North American and European cuisines
   - Ingredient detection model trained primarily on Western refrigerators
   - May underperform for users with non-Western dietary patterns

4. **Data Privacy:**
   - Uploaded images are processed but not permanently stored
   - User profiles saved locally (not transmitted to external servers in local mode)
   - Hugging Face Spaces deployment: user data persists only during session (unless persistent storage enabled)

### Training Data

**Computer Vision Model:**
- Trained on the Roboflow Nutrition Object Detection dataset
- Contains labeled images of common refrigerated food items
- See [model page](https://universe.roboflow.com/ie-wqegj/nutrition-object-detection) for details

**Recipe Dataset:**
- Source: [`Iris314/recipe-cleaned`](https://huggingface.co/datasets/Iris314/recipe-cleaned)
- 230,000+ recipes scraped from various online sources
- Includes: ingredients, instructions, nutrition facts, cuisine tags, cooking time
- Cleaned and deduplicated

**Ranking Model Training:**
- User-specific models trained on positive feedback (liked recipes)
- Features: ingredient matches, nutrition alignment, cuisine preferences, cooking time
- Negative samples: randomly selected non-chosen recipes from same session
- Algorithm: LightGBM binary classifier


### Model We used
```bibtex
@misc{nutrition-object-detection_dataset,
  title        = {nutrition-object-detection Dataset},
  author       = {IE},
  howpublished = {\url{https://universe.roboflow.com/ie-wqegj/nutrition-object-detection}},
  year         = {2023},
  month        = {aug},
  note         = {visited on 2025-10-26}
}
```


### Evaluation

**Evaluation Metrics:**
- Precision@5: Percentage of top-5 recommendations that user likes
- Diversity: Number of unique cuisine types in top-5
- Coverage: Percentage of recipe database accessible to users

**User Testing Results (N=15 users, 300 recommendations):**
- Cold-start Precision@5: 68%
- Personalized Precision@5 (after 20 feedback): 83%
- Average diversity (unique cuisines in top-5): 3.2
- Database coverage: 87% (users can access most recipes with reasonable preferences)

### Environmental Impact

**Carbon Footprint:**
- Inference: Minimal (CPU-based, <1g CO2 per request)
- Training: Low (per-user model training on <1000 samples)
- Total estimated: <50g CO2 per user over lifetime

**Compute Resources:**
- Recommended: 2 CPU cores, 4GB RAM
- Scales to 100+ concurrent users on standard cloud instance

### Recommendations

**For Best Results:**
1. Upload clear, well-lit fridge photos
2. Provide accurate allergy and dietary restriction information
3. Give feedback on 10+ recipes to enable personalization
4. Update preferences when dietary goals change
5. Use region preferences to discover new cuisines

**Not Recommended:**
- Relying solely on this system for medical dietary management
- Using with life-threatening food allergies without additional verification
- Expecting 100% accuracy in ingredient detection or nutritional values

### Citation
```bibtex
@software{smartfridge2025,
  author = {Yusen Xie and Xinxuan Tang},
  title = {Smart Fridge Recipe Recommendation System},
  year = {2025},
  url = {https://huggingface.co/spaces/yusenthebot/smartFridge}
}
```