new features

.env.example

@@ -1,3 +1,2 @@
 HF_TOKEN=hf_your_token_here
 HF_SCRAPER_REPO=Apf-AI4Good/kys-school-data
-HF_RESOLVER_REPO=Apf-AI4Good/school-name-resolver-data

README.md

@@ -1,140 +1,80 @@
----
-title: School Name Resolver
-emoji: 🏫
-colorFrom: indigo
-colorTo: purple
-sdk: docker
-pinned: false
----
-
 # 🏫 School Name Resolver
 
-A comprehensive, Gradio-based web application built for the **Scholarship Operations Team** to efficiently resolve discrepancies in school names when validating applicant marksheets. 
-
----
-
-## 📖 The Problem Context
-
-In the Indian education system, a school's **11-digit UDISE Code** acts as a permanent, immutable identifier. However, the **name of the school** can (and often does) change over time due to upgrades (e.g., from a Junior College to a Senior Secondary School), government renamings, or simply varying data entry practices.
-
-When an Operations Team member reviews a student's printed marksheet, they might see a name like `"N. C. HIGH SCHOOL"`. When they look up that student's UDISE code in the internal Scholarship System (the "Old Master"), it might show the older or newer name `"NANDESWAR CHAKRAVARTY HIGH SCHOOL"`. This discrepancy causes confusion and delays in verification.
-
-## 💡 The Solution
-
-The **School Name Resolver** bridges this gap by acting as a unified search engine and mapping utility. It allows the Ops Team to:
-1. **Search** a UDISE code and instantly pull every known name for that school across historical datasets.
-2. **Compare** the name written on the physical marksheet against historical names.
-3. **Map** all these names together into a persistent, cloud-hosted **Alias Dictionary**.
+The **School Name Resolver** is an internal tool built for the Scholarship Operations Team. It is designed to quickly cross-reference, verify, and resolve discrepancies between the school names written on student marksheets and the official government UDISE records.
 
----
-
-## 🏗️ Architecture & Data Flow
-
-This application is built with a serverless data architecture using **HuggingFace Datasets** as the primary storage layer. It does not require a traditional SQL database.
-
-### 1. Data Sources Searched
-When a UDISE code is searched, the application concurrently queries:
-- **📄 The Marksheet Name**: Entered manually by the Ops user.
-- **📚 Old Master (`master_all_states.csv`)**: The current database used by the Scholarship Application system. This file contains over 200,000 baseline school records.
-- **✅ Latest Scraped Master**: The most recent Parquet file generated by the upstream KYS (Know Your School) Scraper app.
-- **🕐 Older Scraped Masters**: Historical Parquet snapshots to catch historical name changes.
-
-### 2. The Alias Dictionary (`school_aliases.json`)
-When the user clicks "Map All Names Together", the application collects all unique names found during the search and upserts them into a persistent dictionary.
-- **Storage Strategy**: The dictionary is stored as a standard JSON file (`school_aliases.json`) directly on a private HuggingFace dataset.
-- **Schema**: It uses a clean JSON dictionary format where keys are UDISE codes.
-  ```json
-  {
-    "18050406004": {
-      "names": ["SARUPETA GIRLS HE SCHOOL", "VIDYAMANDIR HIGH SCHOOL,SARUPETA"],
-      "last_updated": "2026-07-07"
-    }
-  }
-  ```
-- **Deduplication**: Mapping is strictly case-insensitive. If an Ops user attempts to map a name that already exists in the `names` array, the application will silently skip the duplicate to maintain a clean dataset.
-
-### 3. In-Memory Caching (`hf_reader.py`)
-To ensure high performance without straining HuggingFace bandwidth, the application uses module-level caching:
-- The 45MB `master_all_states.csv` and all mapped `.parquet` files are downloaded and loaded into Pandas DataFrames on the *first* search request.
-- Subsequent searches pull directly from RAM, making them virtually instantaneous.
-- A "Refresh Data Cache" button in the UI allows users to force-reload the cache if the upstream KYS scraper pushes new master files.
+By allowing the operations team to save custom "Marksheet Aliases," the system learns different variations of the same school name, completely eliminating the need to manually verify the same mismatched school names year after year!
 
 ---
 
-## 🚀 Setup & Deployment Guide
-
-This app is designed to be hosted on **HuggingFace Spaces** using Docker.
+## 🏗️ Architecture & Data Sources
 
-### Step 1: Create the Resolver Dataset
-The app requires its own dedicated dataset repository to store the baseline and alias dictionary.
-1. Go to HuggingFace and create a new **Private Dataset** (e.g., `Apf-AI4Good/school-name-resolver-data`).
-2. Upload your existing `master_all_states.csv` (the Old Master) to the root of this dataset.
-*(Note: You do not need to create `school_aliases.json`. The app will automatically create it the first time an ops user maps a school.)*
+The application is entirely stateless. It reads from and writes directly to a centralized, cloud-hosted dataset repository on HuggingFace: **`Apf-AI4Good/kys-school-data`**.
 
-### Step 2: Configure Space Secrets & Variables
-When creating the HuggingFace Space for this application, configure the following in the **Settings → Variables and Secrets** tab:
+This central repository houses two critical types of data:
 
-**🔒 Secrets:**
-| Name | Description |
-|---|---|
-| `HF_TOKEN` | A HuggingFace Access Token with **Write** permissions. Used to read the baselines and write the alias JSON. |
+1. **Master Records (`.parquet` files)**:
+   - **✅ Latest Master**: The most recently scraped government UDISE data, representing the absolute source of truth for the current year. (e.g., `scraped_data/mapped/mapped_master_2026_jul_01_10_19_pm.parquet`)
+   - **📚 Old Master**: The historical baseline dataset used in previous scholarship cycles. It serves as a fallback to ensure continuity for students who applied in previous years. (`mapping_rules/baseline_master.parquet`)
 
-**⚙️ Variables:**
-| Name | Description |
-|---|---|
-| `HF_SCRAPER_REPO` | The Repository ID of your upstream KYS Scraper dataset (e.g., `Apf-AI4Good/kys-school-data`). |
-| `HF_RESOLVER_REPO` | The Repository ID of the new dataset created in Step 1 (e.g., `Apf-AI4Good/school-name-resolver-data`). |
-
-### Step 3: Deploy
-Push all files in this directory (including the provided `Dockerfile` and `requirements.txt`) to the HuggingFace Space. The Docker environment will automatically build and launch the Gradio server on port `7860`.
+2. **Alias Dictionary (`school_aliases.json`)**:
+   - A dynamic JSON file that acts as the memory for the application. Whenever an operations team member encounters a marksheet name that doesn't perfectly match the official masters, they map it in the UI. The app immediately updates this JSON file and pushes it back to HuggingFace.
 
 ---
 
-## 💻 Local Development
-
-If you need to test the application locally on a Windows machine:
+## 🖥️ Using the App
 
-```powershell
-# 1. Create and activate a virtual environment
-python -m venv venv
-.\venv\Scripts\activate
+The app is divided into two distinct steps for the user:
 
-# 2. Install lightweight dependencies
-pip install -r requirements.txt
+### Step 1 — Enter Location & Name
+You can locate a school in two ways:
+- **Location Search**: Use the cascading dropdowns to select the `State` ➔ `District` ➔ `Block` ➔ `Village`. The app will fuzzy-match the school name you enter against all schools in that specific village. (Note: The dropdowns strictly enforce hierarchy to prevent invalid selections).
+- **UDISE Search**: If you already have the 11-digit UDISE code, you can select it directly from the dropdown to instantly pull up the school's records.
 
-# 3. Configure environment variables
-# Copy the example env file and fill in your HuggingFace details
-copy .env.example .env
+### Step 2 — Search Results & Mapping
+The app will display all known names for the selected school(s) as distinct color-coded cards:
+- <strong style="color: #16a34a;">Green Cards (✅ Latest Master)</strong>: The official, current name from the government registry.
+- <strong style="color: #94a3b8;">Grey Cards (🕐 Older Master)</strong>: The official name from the historical baseline dataset.
+- <strong style="color: #d97706;">Yellow Cards (📄 Marksheet)</strong>: Custom aliases that were previously saved by the operations team.
 
-# 4. Run the application
-python app.py
-```
-*Note: The local app is configured with `inbrowser=True`, meaning it will automatically open a new tab in your default web browser (usually at `http://127.0.0.1:7862`) once the server starts.*
+**Saving an Alias**:
+If the student's marksheet says something different (e.g., "NC High School" instead of "Nandeswar Chakravarty High School"), simply type "NC High School" into the input box on the right and click **Save Alias**. The app will push this mapping to the cloud, and future searches for this school will instantly recognize "NC High School" as a valid alias!
 
----
-
-## 🎨 UI Layout & User Flow
-
-The User Interface was completely custom-designed using Gradio Blocks with injected CSS for a modern, premium experience.
-
-1. **Step 1 — Enter Details**: A clean input form requiring only the UDISE code and an optional physical marksheet name.
-2. **Step 2 — Review Names**: Results are rendered as visual, color-coded HTML cards rather than a raw data table.
-   - 🟡 **Amber**: Marksheet Name
-   - 🔵 **Blue**: Old Master Name
-   - 🟢 **Green**: Latest Scraped Master
-   - ⚫ **Grey**: Older Scraped Masters
-3. **Step 3 — Save Aliases**: A one-click mapping button that executes the upsert logic and commits to HuggingFace.
-4. **Alias Dictionary View**: A live, searchable HTML render of the `school_aliases.json` file, visually grouping all known aliases into interactive accordion dropdowns under distinct UDISE code blocks for easy reading.
+**Deleting an Alias**:
+If an incorrect alias was saved by mistake, every Yellow Marksheet card features a red **🗑️ Delete** button on the right. Clicking it will instantly purge that specific alias from the cloud dictionary.
 
 ---
 
-## 🛠️ File Structure
-
-| File | Purpose |
-|---|---|
-| `app.py` | The main Gradio interface, routing, custom CSS, and HTML component builders. |
-| `hf_reader.py` | The data layer responsible for fetching, caching, and searching CSV/Parquet files from HuggingFace. |
-| `alias_store.py` | The CRUD operations and deduplication logic for managing the `school_aliases.json` dictionary. |
-| `requirements.txt` | Python dependencies (`gradio`, `huggingface-hub`, `pandas`, `pyarrow`, `python-dotenv`). |
-| `Dockerfile` | Python 3.11-slim container configuration for HF Spaces deployment. |
-| `.env.example` | Template for local environment variable setup. |
+## ⚙️ Setup & Deployment
+
+### Local Development
+To run this application on your local machine for development:
+
+1. **Clone the repository**:
+   ```bash
+   git clone <your-repo-url>
+   cd school_name_resolver
+   ```
+
+2. **Install dependencies**:
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+3. **Configure Environment Variables**:
+   Create a `.env` file in the root directory (you can copy `.env.example`). You only need two variables:
+   ```env
+   HF_TOKEN=your_huggingface_write_token
+   HF_SCRAPER_REPO=Apf-AI4Good/kys-school-data
+   ```
+
+4. **Run the App**:
+   ```bash
+   python app.py
+   ```
+   The app will start a local server, usually at `http://localhost:7862`.
+
+### Production Deployment
+This application is designed to be hosted directly as a **HuggingFace Space**.
+- **Docker**: The repository includes a `Dockerfile` that HuggingFace automatically uses to build and launch the environment.
+- **Secrets**: In your HuggingFace Space Settings, simply add your `HF_TOKEN` as a Secret, and `HF_SCRAPER_REPO` as a public Variable.
+- **Updates**: Any code pushed to the `main` branch of this repository will trigger HuggingFace to automatically rebuild and redeploy the live application!

admin_patterns.py

@@ -0,0 +1,205 @@
+# admin_patterns.py
+import os
+import json
+import re
+import shutil
+import pandas as pd
+from datetime import datetime
+
+BASE_DIR = os.path.dirname(os.path.abspath(__file__))
+PATTERN_FILE = os.path.join(BASE_DIR, "patterns.json")
+pattern_config = None
+DEFAULT_PATTERN_CONFIG = {"global": [], "states": {}}
+
+
+def _make_backup_of_patterns():
+    """
+    If PATTERN_FILE exists, copy it to patterns_backup_{timestamp}.json
+    Returns backup path or None on failure/if not exists.
+    """
+    try:
+        if os.path.exists(PATTERN_FILE):
+            ts = datetime.utcnow().strftime("%Y%m%dT%H%M%SZ")
+            backup_name = f"patterns_backup_{ts}.json"
+            backup_path = os.path.join(BASE_DIR, backup_name)
+            shutil.copyfile(PATTERN_FILE, backup_path)
+            return backup_path
+    except Exception as e:
+        print("ERROR creating patterns.json backup:", e)
+    return None
+
+
+def load_pattern_config(debug: bool = False):
+    """
+    Simplified config loader for Hugging Face Spaces:
+     - Checks common locations in order
+     - Returns first valid config found
+     - Falls back to default if none found
+    """
+    global pattern_config
+    
+    # Search paths in priority order
+    search_paths = [
+        PATTERN_FILE,                              # Same dir as module
+        "/app/patterns.json",                      # HF Spaces mount point
+        os.path.join(os.getcwd(), "patterns.json") # Current directory
+    ]
+    
+    # Try each path
+    for path in search_paths:
+        if not os.path.exists(path):
+            continue
+            
+        try:
+            with open(path, "r", encoding="utf-8") as f:
+                cfg = json.load(f)
+            
+            # Validate it has expected structure
+            if isinstance(cfg, dict) and ("global" in cfg or "states" in cfg):
+                pattern_config = cfg
+                if debug:
+                    print(f"Loaded config from: {path}")
+                return cfg
+                
+        except (json.JSONDecodeError, IOError) as e:
+            if debug:
+                print(f"Failed to load {path}: {e}")
+            continue
+    
+    # No valid config found - use default
+    pattern_config = DEFAULT_PATTERN_CONFIG.copy()
+    if debug:
+        print("No valid config found, using defaults")
+    
+    return pattern_config
+
+
+def save_pattern_config(cfg: dict):
+    """
+    Save the given config to PATTERN_FILE with a timestamped backup of the previous file.
+    """
+    global pattern_config
+    # create backup first
+    backup_path = _make_backup_of_patterns()
+    if backup_path:
+        print(f"INFO: patterns.json backed up to {backup_path}")
+    # write new file
+    with open(PATTERN_FILE, "w", encoding="utf-8") as f:
+        json.dump(cfg, f, indent=2, ensure_ascii=False)
+    pattern_config = cfg.copy()
+    return True
+
+
+def build_patterns_from_config(cfg: dict, state_key: str | None):
+    global_list = [(p["pattern"], p["replacement"]) for p in cfg.get("global", [])]
+    state_list = []
+    if state_key:
+        state_key_up = state_key.upper().strip()
+        state_patterns = cfg.get("states", {}).get(state_key_up, [])
+        state_list = [(p["pattern"], p["replacement"]) for p in state_patterns]
+    return global_list, state_list
+
+
+def normalize_with_patterns_dynamic(s: str, state_key: str | None):
+    global pattern_config
+    if not isinstance(s, str):
+        return ""
+    s = s.upper()
+    if pattern_config is None:
+        load_pattern_config(debug=False)
+    cfg = pattern_config or DEFAULT_PATTERN_CONFIG
+    global_patterns, state_patterns = build_patterns_from_config(cfg, state_key)
+    for pat, repl in global_patterns:
+        try:
+            s = re.sub(pat, repl, s)
+        except re.error:
+            continue
+    for pat, repl in state_patterns:
+        try:
+            s = re.sub(pat, repl, s)
+        except re.error:
+            continue
+    s = re.sub(r"[^A-Z0-9]+", " ", s)
+    s = re.sub(r"\s+", " ", s).strip()
+    return s
+
+
+# Admin helpers for the UI
+def load_global_patterns_for_editor():
+    cfg = load_pattern_config(debug=False)
+    return pd.DataFrame(cfg.get("global", []))
+
+
+def load_state_patterns_for_editor(selected_state: str | None, new_state_name: str | None):
+    cfg = load_pattern_config(debug=False)
+    key = None
+    if new_state_name and new_state_name.strip():
+        key = new_state_name.strip().upper()
+    elif selected_state:
+        key = selected_state.strip().upper()
+    if not key:
+        return pd.DataFrame(columns=["pattern", "replacement"])
+    state_patterns = cfg.get("states", {}).get(key, [])
+    return pd.DataFrame(state_patterns)
+
+
+def save_global_patterns_from_editor(df: pd.DataFrame, admin_password: str, expected_password: str):
+    if expected_password is None:
+        return "❌ ADMIN password not configured in environment."
+    if admin_password != expected_password:
+        return "❌ Invalid admin password. Global patterns NOT saved."
+    cfg = load_pattern_config(debug=False)
+    cfg["global"] = df.fillna("").to_dict(orient="records")
+    save_pattern_config(cfg)
+    gcount = len(cfg.get("global", []))
+    skeys = sorted(list(cfg.get("states", {}).keys()))
+    return f"✅ Global patterns saved — global={gcount}, state_keys={skeys}"
+
+
+def save_state_patterns_from_editor(selected_state: str | None, new_state_name: str | None, df: pd.DataFrame, admin_password: str, expected_password: str):
+    if expected_password is None:
+        return "❌ ADMIN password not configured in environment."
+    if admin_password != expected_password:
+        return "❌ Invalid admin password. State patterns NOT saved."
+    key = None
+    if new_state_name and new_state_name.strip():
+        key = new_state_name.strip().upper()
+    elif selected_state:
+        key = selected_state.strip().upper()
+    if not key:
+        return "⚠ Please select a state or type a new state key."
+    cfg = load_pattern_config(debug=False)
+    cfg.setdefault("states", {})[key] = df.fillna("").to_dict(orient="records")
+    save_pattern_config(cfg)
+    gcount = len(cfg.get("global", []))
+    skeys = sorted(list(cfg.get("states", {}).keys()))
+    return f"✅ Patterns for {key} saved — global={gcount}, state_keys={skeys}"
+
+
+def refresh_pattern_config():
+    cfg = load_pattern_config(debug=True)
+    gcount = len(cfg.get("global", []))
+    skeys = sorted(list(cfg.get("states", {}).keys()))
+    return f"Refreshed patterns.json — global={gcount}, state_keys={skeys}"
+
+
+def show_patterns_file_info():
+    info_lines = []
+    info_lines.append(f"PATTERN_FILE: {PATTERN_FILE}")
+    info_lines.append(f"Exists: {os.path.exists(PATTERN_FILE)}")
+    if os.path.exists(PATTERN_FILE):
+        try:
+            size = os.path.getsize(PATTERN_FILE)
+            info_lines.append(f"Size (bytes): {size}")
+            with open(PATTERN_FILE, "r", encoding="utf-8") as f:
+                txt = f.read(1000)
+            info_lines.append("Preview (first 1000 chars):")
+            info_lines.append("```json\n" + txt + ("\n... (truncated)" if len(txt) >= 1000 else "") + "\n```")
+        except Exception as e:
+            info_lines.append("ERROR reading file: " + str(e))
+    try:
+        listing = os.listdir(BASE_DIR)
+        info_lines.append("Files in BASE_DIR: " + ", ".join(listing))
+    except Exception as e:
+        info_lines.append("ERROR listing BASE_DIR: " + str(e))
+    return "\n\n".join(info_lines)

alias_store.py

@@ -4,11 +4,9 @@ alias_store.py — Manages the school_aliases.json file stored on HuggingFace.
 Schema:
 {
   "18050406004": {
-    "names": ["SARUPETA GIRLS HE SCHOOL", "VIDYAMANDIR HIGH SCHOOL,SARUPETA"],
-    "last_updated": "2026-07-07"
-  },
-  "18150113702": {
-    "names": ["N. C. HIGH SCHOOL", "NANDESWAR CHAKRAVARTY HIGH SCHOOL"],
+    "names": [
+      {"name": "SARUPETA GIRLS HE SCHOOL", "year_month": "2025-07", "source": "Old Master"}
+    ],
     "last_updated": "2026-07-07"
   }
 }
@@ -21,27 +19,42 @@ from datetime import datetime, timezone
 from huggingface_hub import HfApi, hf_hub_download
 
 HF_TOKEN = os.getenv("HF_TOKEN", "")
-HF_RESOLVER_REPO = os.getenv("HF_RESOLVER_REPO", "")
+HF_SCRAPER_REPO = os.getenv("HF_SCRAPER_REPO", "")
 ALIAS_FILE = "school_aliases.json"
 
 
+def _migrate_names(names: list) -> list[dict]:
+    migrated = []
+    for item in names:
+        if isinstance(item, str):
+            migrated.append({"name": item, "year_month": "", "source": "Unknown (Migrated)"})
+        elif isinstance(item, dict):
+            migrated.append(item)
+    return migrated
+
+
 def load_aliases() -> dict:
     """
     Download and return the alias dictionary from HF as a Python dict.
     Returns an empty dict {} if the file doesn't exist yet.
     """
-    if not HF_RESOLVER_REPO:
+    if not HF_SCRAPER_REPO:
         return {}
     try:
         path = hf_hub_download(
-            repo_id=HF_RESOLVER_REPO,
+            repo_id=HF_SCRAPER_REPO,
             filename=ALIAS_FILE,
             repo_type="dataset",
             token=HF_TOKEN or None,
             force_download=True,
         )
         with open(path, "r", encoding="utf-8") as f:
-            return json.load(f)
+            data = json.load(f)
+            # Migrate old string arrays to object arrays on load
+            for udise, info in data.items():
+                if "names" in info:
+                    info["names"] = _migrate_names(info["names"])
+            return data
     except Exception as e:
         if "404" in str(e) or "not found" in str(e).lower() or "Entry Not Found" in str(e):
             return {}
@@ -55,8 +68,8 @@ def save_aliases(new_entries: list[dict]) -> str:
 
     Each entry should have: udise_code, alias_name, source_label
     """
-    if not HF_RESOLVER_REPO:
-        return "⚠️ HF_RESOLVER_REPO is not configured — cannot save aliases."
+    if not HF_SCRAPER_REPO:
+        return "⚠️ HF_SCRAPER_REPO is not configured — cannot save aliases."
     if not new_entries:
         return "⚠️ No entries provided."
 
@@ -69,19 +82,26 @@ def save_aliases(new_entries: list[dict]) -> str:
     for entry in new_entries:
         code = str(entry.get("udise_code", "")).strip()
         name = str(entry.get("alias_name", "")).strip()
+        source = str(entry.get("source_label", "")).strip()
+        year_month = str(entry.get("year_month", "")).strip()
+        
         if not code or not name:
             continue
 
         if code not in data:
             data[code] = {"names": [], "last_updated": today}
 
-        existing_upper = {n.upper() for n in data[code]["names"]}
+        existing_upper = {n["name"].upper() for n in data[code]["names"]}
         
         if name.upper() in existing_upper:
             skipped_count += 1
             continue
 
-        data[code]["names"].append(name)
+        data[code]["names"].append({
+            "name": name,
+            "source": source,
+            "year_month": year_month
+        })
         data[code]["last_updated"] = today
         added_count += 1
 
@@ -99,7 +119,7 @@ def save_aliases(new_entries: list[dict]) -> str:
         api.upload_file(
             path_or_fileobj=tmp_path,
             path_in_repo=ALIAS_FILE,
-            repo_id=HF_RESOLVER_REPO,
+            repo_id=HF_SCRAPER_REPO,
             repo_type="dataset",
             token=HF_TOKEN or None,
             commit_message=f"Add aliases for UDISE {udise_sample} ({added_count} new name(s))",
@@ -114,7 +134,53 @@ def save_aliases(new_entries: list[dict]) -> str:
         return f"❌ Failed to save aliases: {e}"
 
 
-def get_names_for_udise(udise_code: str) -> list[str]:
-    """Return all known names for a specific UDISE code as a list."""
+def get_names_for_udise(udise_code: str) -> list[dict]:
+    """Return all known names for a specific UDISE code as a list of dicts."""
     data = load_aliases()
     return data.get(str(udise_code).strip(), {}).get("names", [])
+
+def delete_alias(udise_code: str, alias_name: str) -> str:
+    """Delete a specific alias name for a UDISE code from the cloud JSON."""
+    if not HF_SCRAPER_REPO:
+        return "⚠️ HF_SCRAPER_REPO is not configured — cannot delete aliases."
+        
+    code = str(udise_code).strip()
+    name_to_delete = str(alias_name).strip().upper()
+    
+    if not code or not name_to_delete:
+        return "⚠️ Invalid UDISE code or name."
+
+    data = load_aliases()
+    if code not in data:
+        return "⚠️ UDISE code not found in saved aliases."
+
+    original_count = len(data[code].get("names", []))
+    data[code]["names"] = [
+        n for n in data[code].get("names", [])
+        if n.get("name", "").strip().upper() != name_to_delete
+    ]
+    
+    if len(data[code]["names"]) == original_count:
+        return f"⚠️ Alias '{alias_name}' not found for UDISE {code}."
+
+    data[code]["last_updated"] = datetime.now(timezone.utc).strftime("%Y-%m-%d")
+
+    # Upload to HF
+    try:
+        api = HfApi()
+        with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False, encoding="utf-8") as f:
+            json.dump(data, f, indent=2, ensure_ascii=False)
+            tmp_path = f.name
+
+        api.upload_file(
+            path_or_fileobj=tmp_path,
+            path_in_repo=ALIAS_FILE,
+            repo_id=HF_SCRAPER_REPO,
+            repo_type="dataset",
+            token=HF_TOKEN or None,
+            commit_message=f"Delete alias '{alias_name}' for UDISE {code}",
+        )
+
+        return f"🗑️ Deleted '{alias_name}' successfully!"
+    except Exception as e:
+        return f"❌ Failed to delete alias: {e}"

app.py

@@ -7,13 +7,13 @@ HF Spaces:    launched automatically by Dockerfile
 
 import os
 import gradio as gr
-import pandas as pd
 from dotenv import load_dotenv
+from datetime import datetime
 
 load_dotenv()
 
-from hf_reader import search_udise, refresh_all_caches, get_config_status
-from alias_store import load_aliases, save_aliases
+from hf_reader import search_udise, refresh_all_caches, get_config_status, get_state_hierarchy, search_by_name_fuzzy, get_udise_choices
+from alias_store import load_aliases, save_aliases, get_names_for_udise, delete_alias
 
 # ─── CSS ──────────────────────────────────────────────────────────────────────
 CSS = """
@@ -93,6 +93,21 @@ input[type=text]:focus, textarea:focus {
 }
 
 /* ── Buttons ── */
+.btn-delete button {
+    background: #fee2e2 !important;
+    color: #ef4444 !important;
+    border: 1.5px solid #fca5a5 !important;
+    height: 100% !important;
+    min-height: 80px !important;
+    border-radius: 11px !important;
+    font-weight: 700 !important;
+    font-size: 0.95rem !important;
+    transition: all 0.2s !important;
+}
+.btn-delete button:hover {
+    background: #fecaca !important;
+    color: #b91c1c !important;
+}
 .btn-search button {
     background: linear-gradient(135deg, #4f46e5, #7c3aed) !important;
     color: white !important;
@@ -155,11 +170,12 @@ input[type=text]:focus, textarea:focus {
     border-bottom: 1.5px solid #f1f5f9;
     margin-bottom: 14px;
 }
+.marksheet-row { align-items: stretch !important; margin-bottom: 10px !important; gap: 12px !important; }
 .name-card {
     padding: 14px 16px;
     border-radius: 11px;
-    margin-bottom: 10px;
     border-left: 4px solid;
+    height: 100%;
 }
 .card-marksheet   { background: #fffbeb; border-color: #f59e0b; }
 .card-legacy      { background: #eff6ff; border-color: #3b82f6; }
@@ -211,26 +227,6 @@ input[type=text]:focus, textarea:focus {
     border-radius: 10px !important;
 }
 
-/* ── Alias table ── */
-.alias-section {
-    background: #ffffff;
-    border-radius: 14px;
-    padding: 22px 24px;
-    box-shadow: 0 2px 12px rgba(0,0,0,0.07);
-    margin-top: 6px;
-}
-.section-heading {
-    font-size: 1rem;
-    font-weight: 700;
-    color: #1e293b;
-    margin-bottom: 14px;
-}
-.section-sub {
-    font-size: 0.8rem;
-    color: #94a3b8;
-    margin-bottom: 14px;
-    font-weight: 400;
-}
 
 /* ── Empty / no result ── */
 .empty-msg {
@@ -252,184 +248,110 @@ details summary {
 }
 """
 
-# ─── HTML builders ────────────────────────────────────────────────────────────
-
-def build_results_html(results: list[dict], udise_code: str) -> str:
-    if not results:
-        return ""
-
-    ms_name = next((r["name"] for r in results if r["source_type"] == "marksheet"), "")
-
-    cards = []
-    for r in results:
-        st   = r.get("source_type", "")
-        name = r.get("name", "").strip()
-        src  = r.get("source", "")
-        loc  = " › ".join(p for p in [r.get("state",""), r.get("district",""), r.get("block","")] if p and p not in ("nan","none",""))
-
-        css  = {"marksheet": "card-marksheet", "legacy_db": "card-legacy",
-                "latest_master": "card-latest", "old_master": "card-old"}.get(st, "card-old")
-        icon = {"marksheet": "📄 Marksheet", "legacy_db": "📚 Old Master",
-                "latest_master": "✅ Latest Master", "old_master": "🕐 Older Master"}.get(st, src)
-
-        badge = ""
-        if ms_name and st != "marksheet" and name.upper() == ms_name.upper():
-            badge = '<span class="match-badge">✓ Same Name</span>'
-
-        display = name if name and name.lower() not in ("nan","none","") else "<em style='color:#cbd5e1'>Not found</em>"
-        loc_html = f'<div class="card-location">📍 {loc}</div>' if loc else ""
-
-        cards.append(f"""
-        <div class="name-card {css}">
-            <div class="card-tag">{icon}</div>
-            <div class="card-name">{display}{badge}</div>
-            {loc_html}
-        </div>""")
+# ─── Data Initialization ──────────────────────────────────────────────────────
+try:
+    STATE_HIER = get_state_hierarchy()
+    STATE_CHOICES = sorted(STATE_HIER.keys())
+except Exception as e:
+    print(f"Warning: Could not load state hierarchy on startup: {e}")
+    STATE_HIER = {}
+    STATE_CHOICES = []
 
+# ─── HTML builders ────────────────────────────────────────────────────────────
+def build_results_heading_html(results: list[dict], udise_code: str) -> str:
+    if not results: return ""
     unique = len(set(r["name"].strip().upper() for r in results if r.get("name") and r["name"].lower() not in ("nan","none","")))
     diff_note = f"<strong style='color:#e11d48'>{unique} different names found</strong>" if unique > 1 else "<strong style='color:#16a34a'>All sources agree on the name ✓</strong>"
-
+    return f'<div class="result-heading" style="margin-bottom: 10px;">UDISE: {udise_code} &nbsp;·&nbsp; {len(results)} source(s) checked &nbsp;·&nbsp; {diff_note}</div>'
+
+def build_single_card_html(r: dict, ms_name: str, margin_bottom="10px") -> str:
+    st   = r.get("source_type", "")
+    name = r.get("name", "").strip()
+    src  = r.get("source", "")
+    ym   = r.get("year_month", "")
+    loc  = " › ".join(p for p in [r.get("state",""), r.get("district",""), r.get("block","")] if p and p not in ("nan","none",""))
+    css  = {"marksheet": "card-marksheet", "legacy_db": "card-legacy", "latest_master": "card-latest", "old_master": "card-old"}.get(st, "card-old")
+    if ym and "(" not in src: src = f"{src} ({ym})"
+    icon = {"marksheet": "📄 Marksheet", "legacy_db": "📚 Old Master", "latest_master": "✅ Latest Master", "old_master": "🕐 Older Master"}.get(st, src)
+    if ym: icon = f"{icon} ({ym})"
+    badge = ""
+    if ms_name and st != "marksheet" and name.upper() == ms_name.upper(): badge = '<span class="match-badge">✓ Same Name</span>'
+    display = name if name and name.lower() not in ("nan","none","") else "<em style='color:#cbd5e1'>Not found</em>"
+    loc_html = f'<div class="card-location">📍 {loc}</div>' if loc else ""
     return f"""
-    <div class="result-panel">
-        <div class="result-heading">
-            UDISE: {udise_code} &nbsp;·&nbsp; {len(results)} source(s) checked &nbsp;·&nbsp; {diff_note}
-        </div>
-        {"".join(cards)}
+    <div class="name-card {css}" style="margin-bottom:{margin_bottom}">
+        <div class="card-tag">{icon}</div>
+        <div class="card-name">{display}{badge}</div>
+        {loc_html}
     </div>"""
 
-
 def build_empty_html():
     return """<div class="result-panel">
         <div class="empty-msg">
-            🔍 Enter a UDISE code above and click <strong>Search</strong><br>
-            <span style="font-size:0.82rem;color:#e2e8f0">All known names for that school will appear here</span>
+            🔍 Select a location, enter a school name or UDISE code, and click <strong>Find Schools</strong><br>
+            <span style="font-size:0.82rem;color:#e2e8f0">Matching schools and their historical names will appear here</span>
         </div>
     </div>"""
 
 
 # ─── Event handlers ───────────────────────────────────────────────────────────
-
-def on_search(udise_code: str, marksheet_name: str):
-    udise_code = str(udise_code).strip()
-    if not udise_code:
-        return gr.update(value=build_empty_html()), gr.update(visible=False), gr.update(visible=False, value=""), []
-
-    results = search_udise(udise_code, marksheet_name.strip() if marksheet_name else "")
-
-    if not results:
-        html = f"""<div class="result-panel">
-            <div class="empty-msg">❌ No schools found for UDISE <strong>{udise_code}</strong> in any data source.</div>
-        </div>"""
-        return gr.update(value=html), gr.update(visible=False), gr.update(visible=False, value=""), []
-
-    return (
-        gr.update(value=build_results_html(results, udise_code)),
-        gr.update(visible=True),
-        gr.update(visible=False, value=""),
-        results,
-    )
-
-
-def on_map(udise_code: str, marksheet_name: str, search_results: list):
-    udise_code = str(udise_code).strip()
-    if not udise_code or not search_results:
-        return gr.update(visible=True, value="⚠️ Search for a UDISE code first."), build_alias_html()
-
-    entries, seen = [], set()
-    if marksheet_name and marksheet_name.strip():
-        ms = marksheet_name.strip()
-        entries.append({"udise_code": udise_code, "alias_name": ms, "source_label": "Marksheet"})
-        seen.add(ms.upper())
-
-    for r in search_results:
-        name = str(r.get("name","")).strip()
-        if name and name.lower() not in ("nan","none","") and name.upper() not in seen:
-            entries.append({"udise_code": udise_code, "alias_name": name, "source_label": r.get("source","")})
-            seen.add(name.upper())
-
-    msg = save_aliases(entries)
-    return gr.update(visible=True, value=msg), build_alias_html()
-
-
-def build_alias_html(query: str = "") -> str:
-    """Render the alias dictionary as accordion cards (one per UDISE code)."""
-    data = load_aliases()
-    if not data:
-        return '<div style="text-align:center;padding:40px;color:#94a3b8;font-size:0.95rem">📭 No aliases saved yet. Search for a school and click <strong>Map All Names Together</strong>.</div>'
-
-    filtered = {}
-    if query and query.strip():
-        q = query.strip().upper()
-        for udise, info in data.items():
-            if q in udise.upper() or any(q in n.upper() for n in info.get("names", [])):
-                filtered[udise] = info
-    else:
-        filtered = data
-
-    if not filtered:
-        return f'<div style="text-align:center;padding:30px;color:#94a3b8">No results for <strong>"{query}"</strong></div>'
-
-    cards = []
-    for udise in sorted(filtered.keys()):
-        info = filtered[udise]
-        names = info.get("names", [])
-        updated = info.get("last_updated", "")[:10]
-
-        pills = ""
-        for i, name in enumerate(names):
-            if i == 0:
-                bg, dot = "#fffbeb", "🟡"
-            elif i == len(names) - 1:
-                bg, dot = "#f0fdf4", "🟢"
-            else:
-                bg, dot = "#eff6ff", "🔵"
-
-            pills += f"""
-            <div style="display:flex;align-items:center;gap:10px;padding:9px 12px;
-                        background:{bg};border-radius:9px;margin-bottom:6px;">
-                <span>{dot}</span>
-                <span style="font-size:0.97rem;font-weight:700;color:#1e293b">{name}</span>
-            </div>"""
-
-        open_attr = "open" if query and query.strip() else ""
-
-        cards.append(f"""
-        <details {open_attr} style="background:#fff;border-radius:14px;padding:12px 20px;
-                        box-shadow:0 2px 10px rgba(0,0,0,0.07);margin-bottom:14px;cursor:pointer;">
-            <summary style="display:flex;align-items:center;justify-content:space-between;list-style:none;">
-                <div style="display:flex;align-items:center;">
-                    <span style="font-size:0.65rem;font-weight:700;text-transform:uppercase;
-                                 letter-spacing:1px;color:#6d28d9;background:#ede9fe;
-                                 padding:3px 9px;border-radius:20px">UDISE</span>
-                    <span style="font-size:1.05rem;font-weight:800;color:#1e293b;margin-left:12px">{udise}</span>
-                </div>
-                <span style="font-size:0.75rem;color:#94a3b8">{len(names)} name(s) · ▼</span>
-            </summary>
-            <div style="margin-top:16px;">
-                {pills}
-            </div>
-        </details>""")
-
-    return f"""
-    <div style="font-size:0.78rem;color:#94a3b8;font-weight:600;text-transform:uppercase;
-                letter-spacing:0.6px;margin-bottom:12px">{len(filtered)} school(s) in dictionary</div>
-    {"".join(cards)}
-    <style>details > summary::-webkit-details-marker {{ display: none; }}</style>
-    """
-
-
-
-def on_alias_search(query: str) -> str:
-    return build_alias_html(query)
-
+def update_udise_dropdown(state, district, block, village):
+    choices = get_udise_choices(state, district, block, village)
+    return gr.update(choices=choices, value=None)
+
+def on_state_change(state):
+    if not state:
+        return gr.update(info="Select State"), gr.update(choices=[], value=None, interactive=False, info="Select State first")
+    districts = sorted(STATE_HIER.get(state, {}).keys())
+    return gr.update(info=""), gr.update(choices=districts, value=None, interactive=True, info="Select District")
+
+def on_district_change(state, district):
+    if not state:
+        return gr.update(info="Select State first"), gr.update(choices=[], value=None, interactive=False, info="Select State first")
+    if not district:
+        return gr.update(info="Select District"), gr.update(choices=[], value=None, interactive=False, info="Select District first")
+    blocks = sorted(STATE_HIER.get(state, {}).get(district, {}).keys())
+    return gr.update(info=""), gr.update(choices=blocks, value=None, interactive=True, info="Select Block")
+
+def on_block_change(state, district, block):
+    if not state:
+        return gr.update(info="Select State first"), gr.update(choices=[], value=None, interactive=False, info="Select State first")
+    if not district:
+        return gr.update(info="Select District first"), gr.update(choices=[], value=None, interactive=False, info="Select District first")
+    if not block:
+        return gr.update(info="Select Block"), gr.update(choices=[], value=None, interactive=False, info="Select Block first")
+    villages = sorted(STATE_HIER.get(state, {}).get(district, {}).get(block, []))
+    return gr.update(info=""), gr.update(choices=villages, value=None, interactive=True, info="Select Village")
+
+def on_village_change(state, district, block, village):
+    choices = get_udise_choices(state, district, block, village)
+    if not state:
+        return gr.update(info="Select State first"), gr.update(choices=choices, value=None)
+    if not district:
+        return gr.update(info="Select District first"), gr.update(choices=choices, value=None)
+    if not block:
+        return gr.update(info="Select Block first"), gr.update(choices=choices, value=None)
+    if not village:
+        return gr.update(info="Select Village"), gr.update(choices=choices, value=None)
+    return gr.update(info=""), gr.update(choices=choices, value=None)
+
+def on_search(query_name, state, district, block, village, udise_val):
+    if udise_val and str(udise_val).strip():
+        # User selected or typed a UDISE. Extract the code if they selected from dropdown (which is "UDISE - Name")
+        u_code = str(udise_val).split(" - ")[0].strip()
+        return [u_code]
+        
+    if not query_name:
+        return []
+    matched_udises = search_by_name_fuzzy(query_name, state, district, block, village, max_results=5)
+    return matched_udises
 
 def on_refresh_caches():
-    return refresh_all_caches()
-
-
-def _alias_df() -> str:
-    return build_alias_html()
-
+    res = refresh_all_caches()
+    global STATE_HIER, STATE_CHOICES
+    STATE_HIER = get_state_hierarchy()
+    STATE_CHOICES = sorted(STATE_HIER.keys())
+    return res, gr.update(choices=STATE_CHOICES)
 
 # ─── Theme ────────────────────────────────────────────────────────────────────
 THEME = gr.themes.Base(
@@ -443,74 +365,117 @@ THEME = gr.themes.Base(
 with gr.Blocks(title="School Name Resolver") as app:
 
     search_state = gr.State([])
+    render_trigger = gr.State(0)
 
-    # ── Header ──────────────────────────────────────────────────────────────
     gr.HTML("""
     <div class="app-header">
         <h1>🏫 School Name Resolver</h1>
-        <p>Look up all known names for a school using its UDISE code</p>
+        <p>Cross-reference marksheet names against historical UDISE records to resolve discrepancies</p>
     </div>
     """)
 
-    # ── Step 1 + 2: Inputs ──────────────────────────────────────────────────
-    gr.HTML('<div class="step-box"><span class="step-label">Step 1 — Enter Details</span>')
-
+    gr.HTML('<div class="step-box"><div style="display:flex;justify-content:space-between;align-items:center;margin-bottom:10px;"><span class="step-label" style="margin-bottom:0">Step 1 — Enter Location & Name</span><button id="clear-form-btn" style="display:none"></button></div>')
+    
     with gr.Row():
-        udise_input = gr.Textbox(
-            label="UDISE Code",
-            placeholder="e.g.  18150113702",
-            max_lines=1,
-            scale=1,
-        )
-        marksheet_input = gr.Textbox(
-            label="School Name on Marksheet  (optional)",
-            placeholder="Paste the name exactly as written on the student's marksheet…",
-            max_lines=1,
-            scale=2,
-        )
-
-    search_btn = gr.Button("🔍  Search All Sources", elem_classes="btn-search", size="lg")
+        state_dd = gr.Dropdown(label="State", choices=STATE_CHOICES, value=None, info="Select State")
+        district_dd = gr.Dropdown(label="District", choices=[], value=None, interactive=False, info="Select State first")
+    with gr.Row():
+        block_dd = gr.Dropdown(label="Block", choices=[], value=None, interactive=False, info="Select District first")
+        village_dd = gr.Dropdown(label="Village", choices=[], value=None, interactive=False, info="Select Block first")
+        
+    with gr.Row():
+        school_input = gr.Textbox(label="School Name", placeholder="Enter school name from marksheet...", max_lines=1, scale=1)
+        udise_input = gr.Dropdown(label="Or Search by UDISE Code", choices=[], allow_custom_value=True, scale=1)
+        
+    with gr.Row():
+        clear_btn = gr.Button("🗑️ Clear Form", elem_classes="btn-secondary", size="lg", scale=1)
+        search_btn = gr.Button("🔍  Find Schools", elem_classes="btn-search", size="lg", scale=3)
     gr.HTML('</div>')
 
-    # ── Step 2: Results ──────────────────────────────────────────────────────
-    gr.HTML('<div style="margin-bottom:4px"><span class="step-label" style="background:#dbeafe;color:#1d4ed8">Step 2 — Review Names</span></div>')
-    results_html = gr.HTML(value=build_empty_html())
-
-    # ── Step 3: Map ──────────────────────────────────────────────────────────
-    gr.HTML('<div style="margin:12px 0 4px"><span class="step-label" style="background:#dcfce7;color:#15803d">Step 3 — Save Aliases</span></div>')
-    map_btn = gr.Button(
-        "🗺️  Map All Names Together",
-        elem_classes="btn-map",
-        size="lg",
-        visible=False,
-    )
-    map_status = gr.Textbox(
-        show_label=False,
-        interactive=False,
-        visible=False,
-        placeholder="",
-        elem_classes="map-status",
-        max_lines=1,
-    )
-
-    gr.HTML("<hr style='border:none;border-top:1.5px solid #e2e8f0;margin:24px 0'>")
-
-    # ── Alias Dictionary ─────────────────────────────────────────────────────
-    with gr.Accordion("���� Alias Dictionary", open=False, elem_classes="alias-section"):
-        gr.HTML('<div class="section-sub" style="margin-bottom:15px">All school names that have been mapped together so far</div>')
-
-        with gr.Row():
-            alias_search_box = gr.Textbox(
-                show_label=False,
-                placeholder="🔎  Search by UDISE code or school name…",
-                max_lines=1,
-                scale=4,
-            )
-            refresh_alias_btn = gr.Button("🔄 Refresh", scale=1, elem_classes="btn-secondary")
-
-        alias_html = gr.HTML(value="")
-
-    # ── Settings accordion ───────────────────────────────────────────────────
+    gr.HTML('<div style="margin-bottom:4px"><span class="step-label" style="background:#dbeafe;color:#1d4ed8">Step 2 — Search Results & Mapping</span></div>')
+    
+    # ── Dynamic Results Rendering ──
+    @gr.render(inputs=[search_state, render_trigger])
+    def render_results(udises, trigger):
+        if not udises:
+            gr.HTML(build_empty_html())
+            return
+            
+        for udise in udises:
+            details = search_udise(udise, "")
+            
+            # Cross-reference with alias dictionary to pull in custom names
+            mapped_names = get_names_for_udise(udise)
+            detail_names_upper = {r.get("name","").upper() for r in details}
+            
+            for m_info in mapped_names:
+                m_name = m_info.get("name", "").strip()
+                if m_name and m_name.upper() not in detail_names_upper:
+                    details.append({
+                        "source": m_info.get("source", "Saved Marksheet"),
+                        "source_type": "marksheet",
+                        "name": m_name,
+                        "year_month": m_info.get("year_month", "")
+                    })
+            
+            with gr.Row(elem_classes="step-box"):
+                with gr.Column(scale=3):
+                    with gr.Column(elem_classes="result-panel"):
+                        gr.HTML(build_results_heading_html(details, udise))
+                        ms_name = next((r["name"] for r in details if r["source_type"] == "marksheet"), "")
+                        
+                        for r in details:
+                            if r.get("source_type") == "marksheet":
+                                with gr.Row(elem_classes="marksheet-row"):
+                                    with gr.Column(scale=5, min_width=100):
+                                        gr.HTML(build_single_card_html(r, ms_name, margin_bottom="0"))
+                                    with gr.Column(scale=1, min_width=60):
+                                        del_btn = gr.Button("🗑️", elem_classes="btn-delete")
+                                        
+                                        def make_on_delete(u_code=udise, n=r.get("name")):
+                                            def on_delete(trigger_val):
+                                                delete_alias(u_code, n)
+                                                return trigger_val + 1
+                                            return on_delete
+                                            
+                                        del_btn.click(fn=make_on_delete(), inputs=[render_trigger], outputs=[render_trigger])
+                            else:
+                                gr.HTML(build_single_card_html(r, ms_name))
+
+                with gr.Column(scale=1):
+                    marksheet_input = gr.Textbox(label="Marksheet Name", placeholder="Enter exact name to map...", max_lines=1)
+                    map_btn = gr.Button("Save Alias", elem_classes="btn-map")
+                    status_text = gr.Textbox(show_label=False, interactive=False, visible=False, elem_classes="map-status", max_lines=1)
+                    
+                    def on_map_inline(u, m, trigger_val, d=details):
+                        m = m.strip()
+                        if not m: return gr.update(visible=True, value="⚠️ Enter a name."), trigger_val
+                        
+                        current_ym = datetime.now().strftime("%Y-%b").capitalize()
+                        entries = [{"udise_code": u, "alias_name": m, "source_label": "Marksheet", "year_month": current_ym}]
+                        
+                        seen = {m.upper()}
+                        for r in d:
+                            name = str(r.get("name","")).strip()
+                            if name and name.lower() not in ("nan","none","") and name.upper() not in seen:
+                                entries.append({
+                                    "udise_code": u, 
+                                    "alias_name": name, 
+                                    "source_label": r.get("source",""),
+                                    "year_month": r.get("year_month", "")
+                                })
+                                seen.add(name.upper())
+                                
+                        msg = save_aliases(entries)
+                        return gr.update(visible=True, value=msg), trigger_val + 1
+
+                    map_btn.click(
+                        fn=on_map_inline,
+                        inputs=[gr.State(udise), marksheet_input, render_trigger],
+                        outputs=[status_text, render_trigger]
+                    )
+
+    # ── Settings ──
     with gr.Accordion("⚙️  Settings & Cache", open=False):
         gr.Markdown("""
 **Environment Variables required:**
@@ -518,35 +483,45 @@ with gr.Blocks(title="School Name Resolver") as app:
 | Variable | Purpose |
 |---|---|
 | `HF_TOKEN` | HuggingFace token (read + write) |
-| `HF_SCRAPER_REPO` | Existing KYS scraper dataset (has scraped masters) |
-| `HF_RESOLVER_REPO` | New resolver dataset (has master_all_states.csv) |
-
-Use **Refresh Cache** after building a new master sheet in the scraper app.
+| `HF_SCRAPER_REPO` | Existing KYS scraper dataset (has scraped masters and will store aliases) |
         """)
         refresh_cache_btn = gr.Button("🔄 Refresh Data Cache", elem_classes="btn-secondary")
         cache_status = gr.Textbox(show_label=False, interactive=False, max_lines=2)
 
-    # ── Events ───────────────────────────────────────────────────────────────
+    def on_clear_form():
+        return (
+            gr.update(value=None), # state
+            gr.update(value=None), # school_input
+            [] # search_state
+        )
+
+    clear_btn.click(
+        fn=on_clear_form,
+        outputs=[state_dd, school_input, search_state]
+    )
+
+    state_dd.change(fn=on_state_change, inputs=[state_dd], outputs=[state_dd, district_dd])
+    district_dd.change(fn=on_district_change, inputs=[state_dd, district_dd], outputs=[district_dd, block_dd])
+    block_dd.change(fn=on_block_change, inputs=[state_dd, district_dd, block_dd], outputs=[block_dd, village_dd])
+    village_dd.change(fn=on_village_change, inputs=[state_dd, district_dd, block_dd, village_dd], outputs=[village_dd, udise_input])
+    
     search_btn.click(
         fn=on_search,
-        inputs=[udise_input, marksheet_input],
-        outputs=[results_html, map_btn, map_status, search_state],
+        inputs=[school_input, state_dd, district_dd, block_dd, village_dd, udise_input],
+        outputs=[search_state],
     )
-    udise_input.submit(
+    school_input.submit(
         fn=on_search,
-        inputs=[udise_input, marksheet_input],
-        outputs=[results_html, map_btn, map_status, search_state],
+        inputs=[school_input, state_dd, district_dd, block_dd, village_dd, udise_input],
+        outputs=[search_state],
     )
-    map_btn.click(
-        fn=on_map,
-        inputs=[udise_input, marksheet_input, search_state],
-        outputs=[map_status, alias_html],
+    udise_input.change(
+        fn=on_search,
+        inputs=[school_input, state_dd, district_dd, block_dd, village_dd, udise_input],
+        outputs=[search_state],
     )
-    alias_search_box.change(fn=on_alias_search, inputs=[alias_search_box], outputs=[alias_html])
-    refresh_alias_btn.click(fn=lambda: build_alias_html(), outputs=[alias_html])
-    refresh_cache_btn.click(fn=on_refresh_caches, outputs=[cache_status])
-    app.load(fn=build_alias_html, outputs=[alias_html])
 
+    refresh_cache_btn.click(fn=on_refresh_caches, outputs=[cache_status, state_dd])
 
 # ─── Launch ───────────────────────────────────────────────────────────────────
 if __name__ == "__main__":

hf_reader.py

@@ -13,10 +13,14 @@ import re
 import pandas as pd
 from huggingface_hub import HfApi, hf_hub_download
 from datetime import datetime
+from rapidfuzz import process, fuzz
+try:
+    from admin_patterns import normalize_with_patterns_dynamic
+except ImportError:
+    normalize_with_patterns_dynamic = lambda s, st: s
 
 HF_TOKEN = os.getenv("HF_TOKEN", "")
 HF_SCRAPER_REPO = os.getenv("HF_SCRAPER_REPO", "")
-HF_RESOLVER_REPO = os.getenv("HF_RESOLVER_REPO", "")
 
 # ─── Module-level caches ───────────────────────────────────────────────────────
 _old_master_df: pd.DataFrame | None = None
@@ -27,24 +31,25 @@ _cache_loaded: bool = False
 # ─── Internal helpers ──────────────────────────────────────────────────────────
 
 def _load_old_master() -> pd.DataFrame:
-    """Download and return master_all_states.csv from HF_RESOLVER_REPO."""
-    if not HF_RESOLVER_REPO:
-        print("[hf_reader] HF_RESOLVER_REPO not set — skipping Old Master load.")
+    """Download and return baseline_master.parquet from HF_SCRAPER_REPO."""
+    if not HF_SCRAPER_REPO:
+        print("[hf_reader] HF_SCRAPER_REPO not set — skipping Old Master load.")
         return pd.DataFrame()
     try:
         path = hf_hub_download(
-            repo_id=HF_RESOLVER_REPO,
-            filename="master_all_states.csv",
+            repo_id=HF_SCRAPER_REPO,
+            filename="mapping_rules/baseline_master.parquet",
             repo_type="dataset",
             token=HF_TOKEN or None,
             force_download=True,
         )
-        df = pd.read_csv(path, dtype=str, low_memory=False)
-        df["School_Udise_Code__c"] = df["School_Udise_Code__c"].astype(str).str.strip()
+        df = pd.read_parquet(path)
+        if "School_Udise_Code__c" in df.columns:
+            df["School_Udise_Code__c"] = df["School_Udise_Code__c"].astype(str).str.strip()
         print(f"[hf_reader] Old Master loaded: {len(df):,} rows.")
         return df
     except Exception as e:
-        print(f"[hf_reader] Could not load master_all_states.csv: {e}")
+        print(f"[hf_reader] Could not load baseline_master.parquet: {e}")
         return pd.DataFrame()
 
 
@@ -102,6 +107,147 @@ def _ensure_loaded():
         _cache_loaded = True
 
 
+def get_state_hierarchy() -> dict:
+    """
+    Returns a nested dictionary of the geographic hierarchy:
+    { state: { district: { block: [villages] } } }
+    Built by combining all loaded data sources.
+    """
+    _ensure_loaded()
+    hier = {}
+    
+    dfs_to_process = []
+    if _old_master_df is not None and not _old_master_df.empty:
+        dfs_to_process.append(_old_master_df)
+    dfs_to_process.extend(_mapped_masters.values())
+    
+    for df in dfs_to_process:
+        if df.empty: continue
+        
+        s_col = "School_State__c"
+        d_col = "School_District__c"
+        b_col = "School_Block__c"
+        v_col = "School_Village__c"
+        
+        if s_col not in df.columns: continue
+        
+        df_tmp = df.copy()
+        for col in [s_col, d_col, b_col, v_col]:
+            if col not in df_tmp.columns:
+                df_tmp[col] = ""
+            else:
+                df_tmp[col] = df_tmp[col].fillna("").astype(str).str.strip().str.upper()
+                
+        for _, r in df_tmp.iterrows():
+            st = r[s_col]
+            di = r[d_col]
+            bl = r[b_col]
+            vi = r[v_col]
+            
+            if not st: continue
+            if st not in hier: hier[st] = {}
+            if not di: continue
+            if di not in hier[st]: hier[st][di] = {}
+            if not bl: continue
+            if bl not in hier[st][di]: hier[st][di][bl] = set()
+            if vi:
+                hier[st][di][bl].add(vi)
+                
+    for s in hier:
+        for d in hier[s]:
+            for b in hier[s][d]:
+                hier[s][d][b] = sorted(list(hier[s][d][b]))
+                
+    return hier
+
+
+def search_by_name_fuzzy(query_name: str, state: str, district: str, block: str, village: str = None, max_results=10) -> list[str]:
+    """
+    Fuzzy searches the combined data sources for a school name.
+    Returns a list of unique matched UDISE codes.
+    """
+    _ensure_loaded()
+    if not query_name:
+        return []
+        
+    dfs_to_process = []
+    if _old_master_df is not None and not _old_master_df.empty:
+        dfs_to_process.append(_old_master_df)
+    dfs_to_process.extend(_mapped_masters.values())
+    
+    if not dfs_to_process:
+        return []
+        
+    combined_rows = []
+    
+    for df in dfs_to_process:
+        if df.empty: continue
+        if "School_Name__c" not in df.columns or "School_Udise_Code__c" not in df.columns: continue
+        
+        mask = pd.Series(True, index=df.index)
+        
+        if state:
+            if "School_State__c" in df.columns:
+                mask = mask & (df["School_State__c"].astype(str).str.strip().str.upper() == state.upper())
+            else:
+                continue
+        
+        if district:
+            if "School_District__c" in df.columns:
+                mask = mask & (df["School_District__c"].astype(str).str.strip().str.upper() == district.upper())
+            else:
+                continue
+                
+        if block:
+            if "School_Block__c" in df.columns:
+                mask = mask & (df["School_Block__c"].astype(str).str.strip().str.upper() == block.upper())
+            else:
+                continue
+                
+        if village:
+            if "School_Village__c" in df.columns:
+                mask = mask & (df["School_Village__c"].astype(str).str.strip().str.upper() == village.upper())
+            else:
+                continue
+                
+        filtered = df[mask]
+        if not filtered.empty:
+            filtered_sub = filtered[["School_Udise_Code__c", "School_Name__c"]].copy()
+            combined_rows.append(filtered_sub)
+            
+    if not combined_rows:
+        return []
+        
+    combined_df = pd.concat(combined_rows, ignore_index=True).drop_duplicates()
+    combined_df["School_Name__c"] = combined_df["School_Name__c"].astype(str).str.strip()
+    combined_df = combined_df[combined_df["School_Name__c"] != ""]
+    
+    if combined_df.empty:
+        return []
+        
+    choices = combined_df["School_Name__c"].tolist()
+    state_for_patterns = (state or "ARUNACHAL PRADESH").upper()
+    
+    candidates_raw = process.extract(
+        query_name,
+        choices,
+        scorer=fuzz.token_set_ratio,
+        processor=lambda s: normalize_with_patterns_dynamic(s, state_for_patterns),
+        limit=max_results,
+    )
+    
+    matched_udises = []
+    seen_udises = set()
+    for choice, score, idx in candidates_raw:
+        udise = combined_df.iloc[idx]["School_Udise_Code__c"]
+        udise = str(udise).strip()
+        if udise and udise not in seen_udises:
+            seen_udises.add(udise)
+            matched_udises.append(udise)
+            
+    return matched_udises
+
+
 # ─── Public API ───────────────────────────────────────────────────────────────
 
 def refresh_all_caches() -> str:
@@ -119,32 +265,34 @@ def refresh_all_caches() -> str:
     )
 
 
-def _pretty_label(stem: str, idx: int) -> str:
-    """Turn a filename stem like 'mapped_master_20250707_123456' into a readable label."""
-    match = re.search(r"(\d{4})(\d{2})(\d{2})", stem)
-    if match:
-        y, m, d = match.group(1), match.group(2), match.group(3)
-        date_str = f"{d}/{m}/{y}"
+def _pretty_label(stem: str, idx: int) -> tuple[str, str]:
+    """Turn a filename stem like 'mapped_master_2026_jul_01_10_19_pm' into a readable label and year_month."""
+    match_str = re.search(r"(\d{4})_([a-zA-Z]{3})_(\d{2})", stem)
+    if match_str:
+        y, m_str, d = match_str.groups()
+        date_str = f"{d}-{m_str.capitalize()}-{y}"
+        ym = f"{y}-{m_str.capitalize()}"
     else:
-        date_str = stem
+        match_num = re.search(r"(\d{4})(\d{2})(\d{2})", stem)
+        if match_num:
+            y, m, d = match_num.groups()
+            date_str = f"{d}/{m}/{y}"
+            month_str = datetime.strptime(m, "%m").strftime("%b").capitalize()
+            ym = f"{y}-{month_str}"
+        else:
+            date_str = stem
+            ym = ""
 
     if idx == 0:
-        return f"Latest Scraped Master ({date_str})"
+        return f"Latest Scraped Master ({date_str})", ym
     else:
-        return f"Older Scraped Master ({date_str})"
+        return f"Older Scraped Master ({date_str})", ym
 
 
 def search_udise(udise_code: str, marksheet_name: str = "") -> list[dict]:
     """
     Search for a UDISE code across all data sources.
-
-    Returns a list of dicts, each with:
-        source       str  — human-readable source label
-        source_type  str  — 'marksheet' | 'legacy_db' | 'latest_master' | 'old_master'
-        name         str  — school name found
-        state        str  — state (if available)
-        district     str  — district (if available)
-        block        str  — block (if available)
+    Returns a list of dicts.
     """
     _ensure_loaded()
     results: list[dict] = []
@@ -157,6 +305,7 @@ def search_udise(udise_code: str, marksheet_name: str = "") -> list[dict]:
             "state": "",
             "district": "",
             "block": "",
+            "year_month": datetime.now().strftime("%Y-%b").capitalize(),
         })
 
     if _old_master_df is not None and not _old_master_df.empty:
@@ -164,12 +313,13 @@ def search_udise(udise_code: str, marksheet_name: str = "") -> list[dict]:
         matches = _old_master_df[mask]
         for _, row in matches.iterrows():
             results.append({
-                "source": "Old Master (master_all_states.csv)",
+                "source": "Old Master (baseline_master.parquet)",
                 "source_type": "legacy_db",
                 "name": str(row.get("School_Name__c", "")).strip(),
                 "state": str(row.get("School_State__c", "")).strip(),
                 "district": str(row.get("School_District__c", "")).strip(),
                 "block": str(row.get("School_Block__c", "")).strip(),
+                "year_month": "2025",
             })
 
     for idx, (stem, df) in enumerate(_mapped_masters.items()):
@@ -177,7 +327,7 @@ def search_udise(udise_code: str, marksheet_name: str = "") -> list[dict]:
             continue
         mask = df["School_Udise_Code__c"] == str(udise_code).strip()
         matches = df[mask]
-        label = _pretty_label(stem, idx)
+        label, ym = _pretty_label(stem, idx)
         source_type = "latest_master" if idx == 0 else "old_master"
         for _, row in matches.iterrows():
             results.append({
@@ -187,17 +337,72 @@ def search_udise(udise_code: str, marksheet_name: str = "") -> list[dict]:
                 "state": str(row.get("School_State__c", "")).strip(),
                 "district": str(row.get("School_District__c", "")).strip(),
                 "block": str(row.get("School_Block__c", "")).strip(),
+                "year_month": ym,
             })
 
     return results
 
 
+def get_udise_choices(state: str, district: str, block: str, village: str = None) -> list[str]:
+    """Returns a list of 'UDISE - School Name' for the given location filter."""
+    _ensure_loaded()
+    
+    dfs_to_process = []
+    if _old_master_df is not None and not _old_master_df.empty:
+        dfs_to_process.append(_old_master_df)
+    dfs_to_process.extend(_mapped_masters.values())
+    
+    if not dfs_to_process:
+        return []
+        
+    combined_rows = []
+    for df in dfs_to_process:
+        if df.empty: continue
+        if "School_Name__c" not in df.columns or "School_Udise_Code__c" not in df.columns: continue
+        
+        mask = pd.Series(True, index=df.index)
+        if state:
+            if "School_State__c" in df.columns:
+                mask &= (df["School_State__c"].astype(str).str.strip().str.upper() == state.upper())
+            else: continue
+        if district:
+            if "School_District__c" in df.columns:
+                mask &= (df["School_District__c"].astype(str).str.strip().str.upper() == district.upper())
+            else: continue
+        if block:
+            if "School_Block__c" in df.columns:
+                mask &= (df["School_Block__c"].astype(str).str.strip().str.upper() == block.upper())
+            else: continue
+        if village:
+            if "School_Village__c" in df.columns:
+                mask &= (df["School_Village__c"].astype(str).str.strip().str.upper() == village.upper())
+            else: continue
+                
+        filtered = df[mask]
+        if not filtered.empty:
+            filtered_sub = filtered[["School_Udise_Code__c", "School_Name__c"]].copy()
+            combined_rows.append(filtered_sub)
+            
+    if not combined_rows:
+        return []
+        
+    combined_df = pd.concat(combined_rows, ignore_index=True).drop_duplicates()
+    combined_df["School_Udise_Code__c"] = combined_df["School_Udise_Code__c"].astype(str).str.strip()
+    combined_df["School_Name__c"] = combined_df["School_Name__c"].astype(str).str.strip()
+    combined_df = combined_df[(combined_df["School_Udise_Code__c"] != "") & (combined_df["School_Name__c"] != "")]
+    
+    if combined_df.empty:
+        return []
+        
+    choices = (combined_df["School_Udise_Code__c"] + " - " + combined_df["School_Name__c"]).unique().tolist()
+    return sorted(choices)
+
+
 def get_config_status() -> dict:
     """Return a dict describing the current configuration state."""
     _ensure_loaded()
     return {
         "scraper_repo": HF_SCRAPER_REPO or "⚠️ Not set (HF_SCRAPER_REPO)",
-        "resolver_repo": HF_RESOLVER_REPO or "⚠️ Not set (HF_RESOLVER_REPO)",
         "sf_baseline_rows": len(_old_master_df) if _old_master_df is not None else 0,
         "mapped_masters_count": len(_mapped_masters),
         "token_set": bool(HF_TOKEN),

patterns.json

@@ -0,0 +1,83 @@
+{
+  "global": [
+    {"pattern": "\\bH\\.?\\s*/?\\s*S\\.?\\b", "replacement": " HIGH SCHOOL "},
+    {"pattern": "\\bH\\.?\\s*SC(H)?\\.?\\b", "replacement": " HIGH SCHOOL "},
+    {"pattern": "\\bHS\\b", "replacement": " HIGH SCHOOL "},
+    {"pattern": "\\bH\\.?\\s*SEC\\.?\\b", "replacement": " HIGHER SECONDARY "},
+    {"pattern": "\\bHR\\.?\\s*SEC\\.?\\b", "replacement": " HIGHER SECONDARY "},
+    {"pattern": "\\bHSS\\b", "replacement": " HIGHER SECONDARY SCHOOL "},
+    {"pattern": "\\bH\\.?S\\.?S\\.?\\b", "replacement": " HIGHER SECONDARY SCHOOL "},
+    {"pattern": "\\bGHS\\b", "replacement": " GOVERNMENT HIGH SCHOOL "},
+    {"pattern": "\\bGHSS\\b", "replacement": " GOVERNMENT HIGHER SECONDARY SCHOOL "},
+    {"pattern": "\\bIC\\b", "replacement": " INTER COLLEGE "},
+    {"pattern": "\\bINTER\\s+COLLEGE\\b", "replacement": " INTER COLLEGE "},
+    {"pattern": "\\b\\+2\\b", "replacement": " PLUS TWO "},
+
+    {"pattern": "\\bVIDYALAYA\\b", "replacement": " SCHOOL "},
+    {"pattern": "\\bVIDYA(LAY|LAYA)\\b", "replacement": " SCHOOL "},
+    {"pattern": "\\bUCHCH\\b", "replacement": " HIGHER "},
+    {"pattern": "\\bUCHCHA\\b", "replacement": " HIGHER "},
+    {"pattern": "\\bUCHCHATAR(A)?\\b", "replacement": " HIGHER "},
+    {"pattern": "\\bMADHYAMIK\\b", "replacement": " SECONDARY "},
+    {"pattern": "\\bUCHHATARA\\s+MADHYAMIK\\b", "replacement": " HIGHER SECONDARY "},
+    {"pattern": "\\bHIGHER\\s+SCHOOL\\b", "replacement": " HIGH SCHOOL "},
+    {"pattern": "\\bUCHCH\\s+VIDYALAYA\\b", "replacement": " HIGH SCHOOL "},
+    {"pattern": "\\bUCHCHA\\s+VIDYALAYA\\b", "replacement": " HIGH SCHOOL "},
+
+    {"pattern": "\\bBALIKA\\b", "replacement": " GIRLS "},
+    {"pattern": "\\bBALIKA\\s+VIDYALAYA\\b", "replacement": " GIRLS SCHOOL "},
+    {"pattern": "\\bBAL\\b", "replacement": " GIRLS "},
+    {"pattern": "\\bGIRL['’]S\\b", "replacement": " GIRLS "},
+    {"pattern": "\\bBOY['’]S\\b", "replacement": " BOYS "},
+
+    {"pattern": "\\bGOVT\\.?\\b", "replacement": " GOVT "},
+    {"pattern": "\\bGOVERNMENT\\b", "replacement": " GOVT "},
+    {"pattern": "\\bRAJKIYA\\b", "replacement": " GOVT "},
+    {"pattern": "\\bRAJKEEYA\\b", "replacement": " GOVT "},
+
+    {"pattern": "\\bST\\.?\\b", "replacement": " SAINT "},
+    {"pattern": "\\bSRI\\b", "replacement": " SHRI "},
+    {"pattern": "\\bSHREE\\b", "replacement": " SHRI "},
+
+    {"pattern": "\\bKV\\b", "replacement": " KENDRIYA VIDYALAYA "},
+    {"pattern": "\\bK\\.?\\s*V\\.?\\b", "replacement": " KENDRIYA VIDYALAYA "},
+    {"pattern": "\\bKENDRIYA\\s+VIDYALAYA\\b", "replacement": " KENDRIYA VIDYALAYA "},
+    {"pattern": "\\bNVS\\b", "replacement": " JAWAHAR NAVODAYA VIDYALAYA "},
+    {"pattern": "\\bJNV\\b", "replacement": " JAWAHAR NAVODAYA VIDYALAYA "},
+    {"pattern": "\\bNAVODAYA\\s+VIDYALAYA\\b", "replacement": " JAWAHAR NAVODAYA VIDYALAYA "},
+
+    {"pattern": "\\bPROJ\\.?\\b", "replacement": " PROJECT "},
+    {"pattern": "\\bUPG\\.?\\b", "replacement": " UPGRADED "},
+    {"pattern": "\\bUPGRADED\\b", "replacement": " UPGRADED "},
+    {"pattern": "\\bTECH\\.?\\b", "replacement": " TECHNICAL "},
+    {"pattern": "\\bJR\\.?\\b", "replacement": " JUNIOR "},
+    {"pattern": "\\bSR\\.?\\b", "replacement": " SENIOR "},
+    {"pattern": "\\bPRASTABIT\\b", "replacement": " PRASTAWIT "},
+    {"pattern": "\\bPRASTAWIT\\b", "replacement": " PRASTAWIT "},
+    {"pattern": "\\bPUB\\.?\\s*SCH\\.?\\b", "replacement": " PUBLIC SCHOOL "},
+    {"pattern": "\\bCONVENT\\b", "replacement": " CONVENT "},
+    {"pattern": "\\bMISSION\\b", "replacement": " MISSION "}
+  ],
+
+  "states": {
+    "KARNATAKA": [
+      {"pattern": "\\bHR\\.?\\s*SEC\\.?\\b", "replacement": " HIGHER SECONDARY "},
+      {"pattern": "\\bHR\\.?\\s*SEC\\.?\\s*SCHOOL\\b", "replacement": " HIGHER SECONDARY SCHOOL "},
+      {"pattern": "\\bMAT(H)?\\b", "replacement": " MATRICULATION "},
+
+      {"pattern": "\\bP\\.?\\s*U\\.?\\b", "replacement": " PRE UNIVERSITY "},
+      {"pattern": "\\bPU\\s+COLLEGE\\b", "replacement": " PRE UNIVERSITY COLLEGE "},
+      {"pattern": "\\bPRE\\s*UNIVERSITY\\s*COLLEGE\\b", "replacement": " PRE UNIVERSITY COLLEGE "},
+      {"pattern": "\\bPRE\\s*UNIVERSITY\\b", "replacement": " PRE UNIVERSITY "},
+      {"pattern": "\\bGOVT[-\\s]*PU\\s+COLLEGE\\b", "replacement": " GOVT PRE UNIVERSITY COLLEGE "},
+      {"pattern": "\\bGOVT[-\\s]*P\\s*U\\s+COLLEGE\\b", "replacement": " GOVT PRE UNIVERSITY COLLEGE "},
+      {"pattern": "\\bGOVT[-\\s]*JUNIOR\\s+COLLEGE\\b", "replacement": " GOVT JUNIOR COLLEGE "},
+      {"pattern": "\\bGJC\\b", "replacement": " GOVT JUNIOR COLLEGE "},
+      {"pattern": "\\bGPUC\\b", "replacement": " GOVT PRE UNIVERSITY COLLEGE "},
+      {"pattern": "\\bKARNATAKA\\s+PUBLIC\\s+SCHOOLS\\b", "replacement": " KARNATAKA PUBLIC SCHOOL "},
+      {"pattern": "\\bHIGH\\s+SCHOOL\\s+SECTION\\b", "replacement": " HIGH SCHOOL SECTION "},
+
+      {"pattern": "\\bJUNIOR\\s+COLLEGE\\b", "replacement": " PRE UNIVERSITY COLLEGE "}
+    ]
+  }
+}

requirements.txt

@@ -3,3 +3,5 @@ huggingface-hub>=0.24.0
 pandas>=2.0.0
 pyarrow>=14.0.0
 python-dotenv>=1.0.0
+rapidfuzz>=3.0.0  
+