generation/README.md

# PRWP Dataset Extraction Dashboard Generator

This folder contains the scripts and canonical metadata mapping artifacts required to regenerate the database payload for the **PRWP Dataset Extraction Dashboard**.

## Directory Layout

```text
ai4data/docs/dashboards/prwp_extraction/
├── index.html                  # Static dashboard frontend
├── data/
│   ├── dashboard_data.json     # Generated JSON payload
│   └── dashboard_data.js       # Serialized JS variables payload (loaded by index.html)
└── generation/                 # Generator tools & scripts (this folder)
    ├── generate_dashboard_data.py  # Main database payload generator
    ├── run_harmonization.py        # Entity clustering/harmonization script
    ├── canonical_map.json          # Pre-compiled canonical mapping file
    ├── dataset_families.json       # Dataset families lookup file
    └── README.md                   # This instruction file
```

## Running the Data Compilation

To recompile the dashboard data payload after updating any raw extraction batches:

1. Navigate to the `ai4data/` repository root:
   ```bash
   cd /Users/rafaelmacalaba/WBG/ai4data
   ```

2. Run the generator script using `uv`:
   ```bash
   uv run python docs/dashboards/prwp_extraction/generation/generate_dashboard_data.py
   ```

This will read the raw standardized JSON output files from `/Users/rafaelmacalaba/WBG/fetch_prwp/data/standardized_outputs`, apply the canonical name mappings, compute metrics, and write the updated `dashboard_data.json` and `dashboard_data.js` directly to `docs/dashboards/prwp_extraction/data/`.