generation/README.md

PRWP Dataset Extraction Dashboard Generator

This folder contains the scripts and canonical metadata mapping artifacts required to regenerate the database payload for the PRWP Dataset Extraction Dashboard.

Directory Layout

ai4data/docs/dashboards/prwp_extraction/
├── index.html                  # Static dashboard frontend
├── data/
│   ├── dashboard_data.json     # Generated JSON payload
│   └── dashboard_data.js       # Serialized JS variables payload (loaded by index.html)
└── generation/                 # Generator tools & scripts (this folder)
    ├── generate_dashboard_data.py  # Main database payload generator
    ├── run_harmonization.py        # Entity clustering/harmonization script
    ├── canonical_map.json          # Pre-compiled canonical mapping file
    ├── dataset_families.json       # Dataset families lookup file
    └── README.md                   # This instruction file

Running the Data Compilation

To recompile the dashboard data payload after updating any raw extraction batches:

1. Navigate to the ai4data/ repository root:

   cd /Users/rafaelmacalaba/WBG/ai4data
   

2. Run the generator script using uv:

   uv run python docs/dashboards/prwp_extraction/generation/generate_dashboard_data.py
   

This will read the raw standardized JSON output files from /Users/rafaelmacalaba/WBG/fetch_prwp/data/standardized_outputs, apply the canonical name mappings, compute metrics, and write the updated dashboard_data.json and dashboard_data.js directly to docs/dashboards/prwp_extraction/data/.