docs/PROJECT_STATUS.md

# THAMAN — Project Status
> Dual-City AI-Powered AVM (NYC + Riyadh)
> Last updated: 2026-07-10

---

## ✅ Project Complete — Deployed & Verified

- **HuggingFace Space:** https://huggingface.co/spaces/Turki-Almurahhem/thaman
- **GitHub:** https://github.com/turkialm/thaman-v2
- **Model hub (runtime artifacts):** https://huggingface.co/Turki-Almurahhem/thaman-models

### Launch Locally

```bash
cd /Users/totam/Desktop/THAMAN/new_try
uvicorn api.main:app --port 8000
# Open: http://localhost:8000/ui
```

Large files (models, feature CSVs) are gitignored and fetched at startup from
the `thaman-models` hub repo via `download_models.py`. **Any new data file the
API needs must be added to `download_models.py` FILES and uploaded to that
repo** — otherwise the live Space silently degrades. A startup integrity check
(`/health` → `integrity` block) now catches this at boot.

---

## Current Models (canonical — source of truth: `models/meta.json` / `models/riyadh_meta.json`)

### NYC — Stack v22

| Metric | Value |
|---|---|
| R² (holdout) | 0.6495 |
| MedAPE (holdout) | 20.32% |
| MAE (holdout) | $1,047,004 |
| Holdout rows | 27,763 |
| Features | 134 |
| CV | 10-fold Spatial GroupKFold (by NTA) |
| Stack | XGB-A + XGB-B + LightGBM + CatBoost + Ridge meta |

### Riyadh — Stack v12

| Metric | Value |
|---|---|
| R² (holdout) | 0.8014 |
| MedAPE (holdout) | 15.59% |
| MAE (holdout) | 986 SAR/m² |
| OOF R² / MedAPE | 0.9348 / 8.25% |
| Train / holdout rows | 5,531 / 1,730 |
| Features | 149 |
| CV | 5-fold Spatial GroupKFold (by district_ar) |
| Stack | XGB-A + XGB-B + LightGBM + CatBoost + Ridge meta |

Per-type holdout MedAPE: apartment 12.83%, villa 12.39%, plot 20.82%, building 18.61%.

---

## API (FastAPI, `api/main.py`)

| Endpoint | Description |
|---|---|
| `GET /health` | Model + spatial status + **data-integrity check** |
| `GET /metrics` | Live model metrics (feeds analytics dashboard) |
| `GET /bldgclasses` | NYC building class codes |
| `POST /predict` | NYC price (USD) + SHAP drivers + QC flags |
| `POST /predict/riyadh` | Riyadh price (SAR/m² + total) + spatial features |
| `POST /batch`, `/batch/riyadh` | Batch predictions (up to 50) |
| `GET /layers/nta`, `/layers/district` | Choropleth GeoJSON layers |
| `GET /nearby`, `/sales/tile` | NYC comparable sales |
| `GET /riyadh/stats` | Riyadh market analytics |
| `GET /ui` | Bilingual (EN/AR) map interface |

---

## Tests — 109 total (`pytest tests/ -v`)

api (22) · scorer (17) · feature parity (15) · SHAP (14) · regression pins (6) ·
golden ranges (9) · distribution (12) · load (5) + others.
Regression pins re-pinned 2026-07-10 (`python tests/update_regression_pins.py`
prints old→new values; apply manually to `tests/test_regression.py`).

---

## Incident Log (production bug classes — all fixed, all guarded)

1. **2026-06→07** — `features_riyadh.csv` missing from hub download list →
   all Riyadh districts collapsed to ~2,500 SAR/m² fallback.
2. **2026-07-07** — `overture_places.geojson` (619MB) never shipped → all 13
   NYC POI features zero live. Fixed with compact `overture_poi_buckets.npz`
   (1.2MB). Also fixed NYC `/predict` crash (UnboundLocalError in citibike
   fallback).
3. **2026-07-10** — Riyadh v10/v11 feature injection read `district_ar` from a
   dict that never contains it → metro + type-lag features dead for every
   request (5 km metro fallback). Fixed; regression pins re-pinned.

Guard: `_startup_integrity_check()` in `api/main.py` verifies all hub files +
non-empty POI/district lookups at boot; `/health` reports `status: degraded`
with issue list on failure.

---

## Docs

- `docs/{thaman_paper, technical_report, defense_qa, demo_script}` in
  .tex/.txt/.md + .docx. **The .docx files are generated by
  `docs/generate_docx.py` whose content is hardcoded — editing .tex does NOT
  update .docx; edit the script and re-run it.**
- Official report: root `THAMAN_Graduation_Project_2_Final_Report.tex`
  (compile on Overleaf).
- Historic metrics that must stay in version-history tables only:
  Riyadh v1 23.43%/0.675 (buggy), v2 18.16%/0.798, v11 0.8003/15.56%;
  NYC v11 0.6450/20.24%.