samples/maria_crm_export.NOTES.md

# maria_crm_export.csv — demo cheat-sheet & planted-issue inventory

**Persona.** Maria, an ops coordinator, exported the company CRM on Monday morning.
383 rows × 12 columns. Engineered against the *shipped* pipeline
(`scrubdata.planner.mock_plan` + `reconcile.grounded_mapping` + `pii.detect_column_pii`),
regenerate with `uv run python samples/make_maria_crm.py` (deterministic, seed 20260612).

Columns: `company, contact_name, email, phone, country, state, signup_date, plan_tier,
mrr, status, cc_on_file, notes`.

## What ScrubData should do, by column

| Column | Planted mess | Expected action |
|---|---|---|
| **company** | mojibake (`Müller GmbH`); case/whitespace variants of Acme; `Globex` vs `Globex Corp` | `fix_encoding` → `strip_whitespace` → `canonicalize_categories` folds `acme corp`/`ACME Corp`→`Acme Corp`. **Does NOT merge `ACME Corporation`** (see YOUR CALL #1). |
| **contact_name** | ALLCAPS / lowercase / leading-trailing spaces | `fix_encoding`, `strip_whitespace`, case-fold of exact case-variants. Rare names → **`suspect_values` review flag** (won't merge distinct people — abstention-first). |
| **email** | UPPERCASE + whitespace | `flag_pii [email]` (flag only, not masked — users need it) → `strip_whitespace` → `normalize_email`. |
| **phone** | 5 formats: `(555) x-y`, `555.x.y`, `+1 555 x y`, `555xy`, `1-555-x-y` | `flag_pii [phone]` → `standardize_phone`. |
| **country** | 4+ USA spellings + rare typos | `canonicalize_categories`: `U.S.A`/`us`/`USA.`→United States, `germny`→Germany, `Nigeia`→Nigeria, `Polnd`/`Swedn`→Poland/Sweden. **Abstains on the YOUR CALL items.** |
| **state** | abbrev chaos `CA`/`California`, typos | `canonicalize_categories`: `Calfornia`→California, `Wahsington`→Washington, `Virgina`→Virginia, `Mississipi`→Mississippi (cast to column's dominant case = ALLCAPS abbrevs). |
| **signup_date** | 3 formats: ISO, `M/D/YY`, `Month D YYYY`, `DD-MM-YYYY` | `parse_date` → ISO `YYYY-MM-DD`. |
| **plan_tier** | `Premium/premium/PREMIUM/Prem`, `Basic/…`, `Ent`, `Free` | `canonicalize_categories` case-folds the variants. |
| **mrr** | `$1,200.00` / `1200.00` / `1,200 USD` / `1200` | `parse_currency` → numeric. |
| **status** | `Active/active/ACTIVE`, `Churned`, `Trial`, `Lapsed` | case-fold canonicalization. |
| **cc_on_file** | 8 **Luhn-valid** fake cards, 4 surface formats | `flag_pii [credit_card]` (checksum-confirmed) → **`mask_pii`** → `****-****-****-5559` (last 4 kept). |
| **notes** | clearly-fake SSNs (`123-45-6789`, `078-05-1120`) in free text | Sparse free-text → column PII typer does NOT fire (needs ≥60% of distinct cells to match). SSNs survive as text — honest limitation; tier-2 NER would be needed to mask in-prose PII. |

## THE YOUR CALL moments (the abstention wow — these SHOULD surface, not auto-merge)

1. **Entity near-tie — `Acme Corp` (7+3 case-variants) vs `ACME Corporation` (4).**
   Genuinely unclear if the same company. The tool folds only the case/whitespace
   variants into `Acme Corp` and **leaves `ACME Corporation` untouched** — it does not
   silently merge two plausibly-distinct entities. Human judgment call. ✅ behaves correctly.

2. **Ambiguous country `Slovia`.** Reconciles to `Slovakia` (0.857) and `Slovenia`
   essentially tied → **margin 0.0 < 0.03 ⇒ ABSTAIN**. Surfaces in the
   `uncertain_canonicalization` review flag. The hero wow-moment.

3. **Ambiguous country `Austrai`.** `Australia` (0.875) vs `Austria` → **margin 0.018 < 0.03
   ⇒ ABSTAIN**. Second genuine geographic ambiguity.

Bonus near-miss abstentions also fire (`Indai`→India 0.80 < 0.84 threshold; `Frnace`,
`Brasil`) — they look like typos but don't clear the confidence bar, so they're flagged,
not guessed.

## Contrast: clean auto-fixes vs abstentions (the trust story)
- **Confident typos auto-fix:** `Nigeia`→Nigeria, `germny`→Germany, `Calfornia`→California
  (high score, wide margin to the next candidate).
- **Ambiguous/low-confidence ABSTAIN:** `Slovia`, `Austrai`, `Indai` — left unchanged,
  raised as `country | uncertain_canonicalization` for Maria to decide.
That split (fix the obvious, flag the genuine judgment calls) is the demo's whole pitch.