Spaces:
Sleeping
Sleeping
| title: Smart Fridge Recipe Assistant | |
| emoji: 🧊 | |
| colorFrom: blue | |
| colorTo: green | |
| sdk: gradio | |
| sdk_version: 5.49.1 | |
| app_file: app.py | |
| pinned: false | |
| license: mit | |
| # Smart Fridge Recipe Recommendation System | |
| An intelligent recipe recommendation system that detects ingredients in your fridge using computer vision and suggests personalized recipes based on your dietary preferences, nutritional goals, and available ingredients. | |
| ## Model Card | |
| ### Model Details | |
| **Developed by:** Yusen Xie(yusenthebot), Xinxuan Tang (iris314) | |
| **Model Type:** Hybrid recommendation system combining computer vision, rule-based filtering, and machine learning ranking | |
| **Language:** Python | |
| **License:** MIT | |
| **Components:** | |
| 1. **Computer Vision Model:** [Roboflow Nutrition Object Detection](https://universe.roboflow.com/ie-wqegj/nutrition-object-detection) | |
| - Pre-trained object detection model for identifying food items in refrigerator images | |
| - Provides bounding boxes and confidence scores | |
| 2. **Recipe Database:** [`Iris314/recipe-cleaned`](https://huggingface.co/datasets/Iris314/recipe-cleaned) dataset | |
| - 230,000+ recipes with nutritional information and ingredient lists | |
| 3. **Personalized Ranking:** User-specific LightGBM models trained on feedback data | |
| ### Intended Use | |
| **Primary Use Cases:** | |
| - Home cooks seeking recipe inspiration based on available ingredients | |
| - Individuals managing dietary restrictions (vegetarian, vegan, allergies) | |
| - Users tracking nutritional goals (calorie/protein targets) | |
| - Reducing food waste by suggesting recipes using existing ingredients | |
| **Out-of-Scope Uses:** | |
| - Medical dietary advice or treatment planning | |
| - Professional kitchen inventory management | |
| - Nutritional assessment for clinical purposes | |
| - Real-time commercial food service applications | |
| ### How It Works | |
| **Pipeline Overview:** | |
| 1. **Ingredient Detection** | |
| - User uploads fridge photo | |
| - Roboflow model detects food items with confidence scores | |
| - Items mapped to parent ingredient categories (e.g., "chicken breast" → "chicken") | |
| 2. **Hard Filtering** | |
| - Removes recipes violating dietary restrictions (vegan/vegetarian) | |
| - Filters recipes outside calorie range (default: 400-2000 kcal) | |
| - Filters recipes outside protein range (default: 50-160g) | |
| - Eliminates recipes containing user's disliked ingredients | |
| 3. **Coarse Ranking** (Vectorized NumPy implementation) | |
| - Computes ingredient coverage ratios across 20,000+ candidates | |
| - Weights: main ingredients (1.0), staple (0.3), other (0.6), region match (0.8) | |
| - Returns top ~20,000 candidates | |
| 4. **ML Reranking** (LightGBM) | |
| - Extracts 28+ features per recipe (ingredient matches, nutrition alignment, cuisine preferences) | |
| - Applies user-specific trained model or similar user's model | |
| - Returns top 200 candidates | |
| 5. **Diversification** (KMeans clustering) | |
| - Ensures variety in final 5 recommendations | |
| - Minimizes redundancy in cuisine types and main ingredients | |
| 6. **Feedback Loop** | |
| - Users mark favorite recipes | |
| - Model retrains every 20 feedback entries | |
| - Improves personalization over time | |
| ### Performance | |
| **Detection Accuracy:** | |
| - Ingredient detection: ~85% precision on trained refrigerated items (With specifc lighting condition) | |
| - Confidence threshold: 0.4 (adjustable) | |
| - High-confidence cutoff: 0.7 | |
| **Recommendation Quality:** | |
| - Cold-start users: Rule-based scoring achieves ~70% user satisfaction | |
| - After 20+ feedback entries: ML model achieves ~85% satisfaction | |
| - Embedding-based user similarity enables model sharing (threshold: 0.85 cosine similarity) | |
| **Speed:** | |
| - Total pipeline: ~15-60 seconds end-to-end | |
| ### Limitations | |
| **Technical Limitations:** | |
| 1. **Detection Quality:** | |
| - Accuracy depends on image quality, lighting, and fridge organization | |
| - May struggle with uncommon ingredients or heavily packaged items | |
| - Overlapping items can cause missed detections | |
| 2. **Recipe Database Coverage:** | |
| - Limited recipes for certain regional cuisines (African, Middle Eastern) | |
| - Nutritional data may be estimates, not laboratory-verified | |
| - Some recipes lack complete ingredient parent mappings | |
| 3. **Cold-Start Problem:** | |
| - New users with no feedback rely on rule-based ranking | |
| - Requires 5-10 feedback entries for meaningful personalization | |
| - Users with unique dietary combinations may get fewer relevant results | |
| 4. **Personalization Constraints:** | |
| - Model assumes preferences are relatively stable over time | |
| - Cannot capture context-dependent preferences (e.g., seasonal, mood-based) | |
| - Feedback is binary (like/dislike), not granular ratings | |
| **Ethical Considerations:** | |
| 1. **Dietary Safety:** | |
| - System is NOT a substitute for professional dietary advice | |
| - Users with medical conditions should consult healthcare providers | |
| - Allergy filtering is based on self-reported data and may miss cross-contamination risks | |
| 2. **Nutritional Accuracy:** | |
| - Calorie/protein values are estimates from recipe databases | |
| - Portion sizes may vary from database assumptions | |
| - Not suitable for precise macro tracking or medical nutrition therapy | |
| 3. **Bias and Fairness:** | |
| - Recipe database skews toward North American and European cuisines | |
| - Ingredient detection model trained primarily on Western refrigerators | |
| - May underperform for users with non-Western dietary patterns | |
| 4. **Data Privacy:** | |
| - Uploaded images are processed but not permanently stored | |
| - User profiles saved locally (not transmitted to external servers in local mode) | |
| - Hugging Face Spaces deployment: user data persists only during session (unless persistent storage enabled) | |
| ### Training Data | |
| **Computer Vision Model:** | |
| - Trained on the Roboflow Nutrition Object Detection dataset | |
| - Contains labeled images of common refrigerated food items | |
| - See [model page](https://universe.roboflow.com/ie-wqegj/nutrition-object-detection) for details | |
| **Recipe Dataset:** | |
| - Source: [`Iris314/recipe-cleaned`](https://huggingface.co/datasets/Iris314/recipe-cleaned) | |
| - 230,000+ recipes scraped from various online sources | |
| - Includes: ingredients, instructions, nutrition facts, cuisine tags, cooking time | |
| - Cleaned and deduplicated | |
| **Ranking Model Training:** | |
| - User-specific models trained on positive feedback (liked recipes) | |
| - Features: ingredient matches, nutrition alignment, cuisine preferences, cooking time | |
| - Negative samples: randomly selected non-chosen recipes from same session | |
| - Algorithm: LightGBM binary classifier | |
| ### Model We used | |
| ```bibtex | |
| @misc{nutrition-object-detection_dataset, | |
| title = {nutrition-object-detection Dataset}, | |
| author = {IE}, | |
| howpublished = {\url{https://universe.roboflow.com/ie-wqegj/nutrition-object-detection}}, | |
| year = {2023}, | |
| month = {aug}, | |
| note = {visited on 2025-10-26} | |
| } | |
| ``` | |
| ### Evaluation | |
| **Evaluation Metrics:** | |
| - Precision@5: Percentage of top-5 recommendations that user likes | |
| - Diversity: Number of unique cuisine types in top-5 | |
| - Coverage: Percentage of recipe database accessible to users | |
| **User Testing Results (N=15 users, 300 recommendations):** | |
| - Cold-start Precision@5: 68% | |
| - Personalized Precision@5 (after 20 feedback): 83% | |
| - Average diversity (unique cuisines in top-5): 3.2 | |
| - Database coverage: 87% (users can access most recipes with reasonable preferences) | |
| ### Environmental Impact | |
| **Carbon Footprint:** | |
| - Inference: Minimal (CPU-based, <1g CO2 per request) | |
| - Training: Low (per-user model training on <1000 samples) | |
| - Total estimated: <50g CO2 per user over lifetime | |
| **Compute Resources:** | |
| - Recommended: 2 CPU cores, 4GB RAM | |
| - Scales to 100+ concurrent users on standard cloud instance | |
| ### Recommendations | |
| **For Best Results:** | |
| 1. Upload clear, well-lit fridge photos | |
| 2. Provide accurate allergy and dietary restriction information | |
| 3. Give feedback on 10+ recipes to enable personalization | |
| 4. Update preferences when dietary goals change | |
| 5. Use region preferences to discover new cuisines | |
| **Not Recommended:** | |
| - Relying solely on this system for medical dietary management | |
| - Using with life-threatening food allergies without additional verification | |
| - Expecting 100% accuracy in ingredient detection or nutritional values | |
| ### Citation | |
| ```bibtex | |
| @software{smartfridge2025, | |
| author = {Yusen Xie and Xinxuan Tang}, | |
| title = {Smart Fridge Recipe Recommendation System}, | |
| year = {2025}, | |
| url = {https://huggingface.co/spaces/yusenthebot/smartFridge} | |
| } | |
| ``` |