README.md

---
title: Subdivision Cluster FIPS Validation
emoji: 🗺️
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 6.19.0
python_version: "3.12"
app_file: app_simple_ui.py
pinned: false
---

# Subdivision Cluster / FIPS Validation

Ported from the Colab notebook
`LatestUndeterminedZipCityCensusTract70_Threshold_95_Clusters.ipynb`.

Upload one or more Excel files. The app:

1. Merges them into one table.
2. Validates every `(city, FIPS)` pair against the **Subdivision Program
   Safety List** (the reference file).
3. Fuzzy-clusters `LegalSubDivision` names (RapidFuzz).
4. Fills city / ZIP / census-tract by majority vote per cluster.
5. Returns the cities, zips, census-tracts, cluster, and full-output files,
   plus the FIPS validation audit.

## How the reference (safety) file is handled

The app does **not** accept the safety list as an upload — it's your trusted
whitelist, so it stays under your control. `get_safety_list_path()` resolves it
two ways:

### Option A — bundle it in this repo (Space must be PRIVATE)

Commit the xlsx into the Space repo and name it
`Subdivision_Program_Safety_List.xlsx` (or set the `SAFETY_LIST_FILE` variable
to your filename). Then set the Space visibility to **Private** in Settings —
otherwise the file is publicly downloadable.

### Option B — private dataset + token (recommended)

1. Create a **private dataset** repo and upload the safety list there.
2. In this Space, add these under Settings → **Variables and secrets**:
   - Secret `HF_TOKEN` — a read access token.
   - Variable `SAFETY_LIST_REPO` — e.g. `your-username/subdivision-safety-list`.
   - Variable `SAFETY_LIST_REPO_FILE` — the filename inside that dataset
     (optional; defaults to `Subdivision_Program_Safety_List.xlsx`).

The list is downloaded once at startup and cached. To update it, just replace
the file in the dataset — no app redeploy needed.

### Optional per-run override (sidebar)

The sidebar's **"2. Upload Reference File (Optional)"** slot lets you swap in a
different safety list (xlsx or csv, same `City Name` + `Acceptable FIPS *`
columns) for a **single run** — handy for testing a candidate list before
committing it. It is not persisted and does not touch the server-side list.

Security note: because this lets anyone with access to the Space substitute the
whitelist at request time, it's only appropriate for a **private** Space whose
members you trust (which is the current setup). To lock the whitelist to the
server copy only, delete the `reference_in` component and the `reference_file`
branch in `run_pipeline` (it falls back to `load_whitelist()`).

## Deploy

```bash
# one-time
huggingface-cli login

# create the Space (or do it in the web UI: New Space -> Gradio)
huggingface-cli repo create subdivision-space --type space --space_sdk gradio

git clone https://huggingface.co/spaces/<your-username>/subdivision-space
cd subdivision-space
cp /path/to/app.py /path/to/requirements.txt /path/to/README.md .
git add .
git commit -m "Initial app"
git push
```

The Space builds automatically on push.

## Notes

- The "filter to single most-dominant county" toggle reproduces the notebook's
  behavior. Turn it off to process every county in the upload at once
  (clustering already runs per-county, so this is safe).
- Large outputs: the full CSV is written whole. On the free CPU tier (limited
  RAM), very large merges may need the upgraded hardware tier.