Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,211 @@
+# Phase 3 — Product Domain + Taxonomy + Product Grouping
+
+## Compliments Reference DB Pipeline
+
+**Version:** 1.0.0
+**Date:** 2026-07-30
+**Status:** PASS
+
+---
+
+## Input
+
+| Property | Value |
+|----------|-------|
+| Source | `phase2/outputs/phase2_output.parquet` |
+| HF Repo | `saraNour/compliments-reference-phase2` |
+| Row Count | 4,440 |
+| Column Count | 32 |
+
+---
+
+## What Phase 3 Does
+
+### A. Food / Non-Food Classification
+
+Deterministic keyword-based rules. Creates `product_domain` column with values: `food`, `non_food`, `unknown`.
+
+- **Food:** Dairy, meat, seafood, bakery, beverages, frozen, produce, pantry, snacks, confectionery
+- **Non-food:** Pet food, medications, personal care, household cleaning, kitchenware, health remedies
+- **Unknown:** Products that don't match any rule
+
+### B. Taxonomy Classification
+
+Deterministic keyword-based taxonomy. Classification metadata only — NOT used as grouping key.
+
+19 categories: DAIRY, MEAT_SEAFOOD, BEVERAGES, BAKERY, FROZEN, BREAKFAST, PASTA_RICE, CANNED_GOODS, CONDIMENTS_SAUCES, SNACKS, CONFECTIONERY, PRODUCE, PERSONAL_CARE, HOUSEHOLD_CLEANING, HEALTH_REMEDIES, PET_FOOD, HOUSEHOLD_SUPPLIES, BABY_CARE, GENERAL_GROCERY
+
+### C. Product Grouping
+
+Identity-based deterministic grouping:
+
+```
+group_key = identity_hash + product_domain + normalized_core_title
+```
+
+- `identity_hash`: 14-field hash from Phase 2 (9 flags + fat + product_line + flavour + formulation)
+- `product_domain`: food/non_food/unknown — **structurally prevents food/non-food mixing**
+- `normalized_core_title`: Core title with generic packaging words removed, tokens sorted
+
+### D. Ambiguous Case Detection
+
+Groups with >1 product are inspected for:
+- Multiple core_titles
+- Multiple flavours
+- Multiple formulations
+- Multiple fat levels
+- Multiple product lines
+
+### E. Rule-Based Resolution
+
+All grouping is deterministic. No LLM was required.
+
+---
+
+## Output Files
+
+### A. reference_product_catalog.csv (Group-Level)
+
+| Column | Type | Description |
+|--------|------|-------------|
+| group_id | str | Deterministic UUID |
+| group_name | str | Most frequent core_title |
+| brand | str | Normalized brand |
+| product_domain | str | food/non_food/unknown |
+| taxonomy | str | Taxonomy category |
+| identity_hash | str | 14-field identity hash |
+| product_count | int | Products in group |
+| unique_upcs | int | Unique UPCs in group |
+
+### B. product_group_mapping.csv (Product-Level)
+
+| Column | Type | Description |
+|--------|------|-------------|
+| upc | str | Barcode |
+| external_id | str | Voila ID |
+| group_id | str | Group UUID |
+| group_name | str | Core title |
+| core_title | str | Core title |
+| original_title | str | Raw title |
+| brand | str | Normalized brand |
+| size | str | Raw size |
+| variant_attributes | str | Parsed size JSON |
+| identity_hash | str | Identity hash |
+| product_domain | str | Food/non-food |
+| taxonomy | str | Taxonomy |
+| product_line | str | Product line |
+| source | str | Data source |
+| source_url | str | Source URL |
+
+### C. ambiguous_cases.csv
+
+Groups with potential ambiguity and the reason.
+
+---
+
+## Deterministic Grouping Rules
+
+| # | Rule | Implementation |
+|---|------|----------------|
+| 1 | Identity hash captures 14 fields | `build_identity_hash()` in Phase 2 |
+| 2 | Product domain is part of grouping key | `build_group_key()` in Phase 3 |
+| 3 | Core title normalized (generic words removed, sorted) | `normalize_title_for_grouping()` |
+| 4 | Same normalized title + same identity = same group | Group key construction |
+| 5 | Different identity = different group | Identity hash is first component |
+| 6 | Food vs non_food = always different group | Product domain is second component |
+
+---
+
+## Food/Non-Food Rules
+
+100+ deterministic regex rules based on product title keywords.
+
+- Pet food → non_food
+- Medications (ibuprofen, acetaminophen, etc.) → non_food
+- Personal care (tampons, shampoo, etc.) → non_food
+- Household (detergent, garbage bags, etc.) → non_food
+- Kitchenware (light bulbs, pans, etc.) → non_food
+- Everything else matching food keywords → food
+- No match → unknown
+
+---
+
+## Regression Tests — ALL PASS
+
+| Test | Result |
+|------|--------|
+| PB smooth sizes same group | PASS |
+| PB crunchy sizes same group | PASS |
+| PB organic smooth different | PASS |
+| PB naturally simple different | PASS |
+| PB light different | PASS |
+| PB honey different | PASS |
+| PB cookies different | PASS |
+| PB ice cream different | PASS |
+| PB dog treats different | PASS |
+| CC 1% vs 2% different | PASS |
+| CC 1% vs fat-free different | PASS |
+| Food vs non-food split | PASS |
+
+---
+
+## Validation Results
+
+| Check | Result |
+|-------|--------|
+| Row count preserved | PASS |
+| No duplicate mappings | PASS |
+| No food/non-food mixing | PASS |
+| No brand conflicts | PASS |
+| Identity consistent | PASS |
+| Taxonomy populated | PASS |
+| No empty group names | PASS |
+| **Overall** | **PASS** |
+
+---
+
+## Files
+
+```
+phase3/
+    README.md
+    src/phase3.py
+    notebooks/phase3.ipynb
+    outputs/
+        reference_product_catalog.csv
+        product_group_mapping.csv
+        ambiguous_cases.csv
+        phase3_executed.ipynb
+    validation/
+        phase3_validation.json
+    statistics/
+        phase3_statistics.json
+```
+
+---
+
+## How to Run
+
+```bash
+cd phase3/
+python3 src/phase3.py
+```
+
+---
+
+## LLM Usage
+
+None. All grouping is deterministic.
+
+---
+
+## Handoff
+
+Phase 3 outputs are the frozen product-grouping layer.
+
+Future phases will add:
+- Phase 4: Variant assignment
+- Phase 5: Nutrition merge
+- Phase 6: Nutrition scoring (NutriScore, NOVA, EcoScore)
+- GPC integration
+- Agribalyse integration