fugee / data /README.md
helmo's picture
[T001-T016] Phase 0 β€” project setup: agent loop, state machine, Gradio skeleton
944f1ac
|
Raw
History Blame Contribute Delete
1.13 kB
# data/
Runtime data directory for UNHCR Refugee Statistics.
## Structure
```
data/
β”œβ”€β”€ scripts/
β”‚ β”œβ”€β”€ unhcr_downloader.py # Full UNHCR API downloader (all endpoints)
β”‚ β”œβ”€β”€ enrich_downloader.py # Targeted per-country enrichment downloader
β”‚ β”œβ”€β”€ query_data.py # Runtime query layer (used by agent tools)
β”‚ └── refresh_data.py # Staleness check + re-download
β”œβ”€β”€ raw/ # Raw API responses (gitignored, regenerate)
β”‚ └── .gitkeep
└── processed/ # Merged flat JSON per endpoint (gitignored)
└── .gitkeep
```
## Setup
```bash
cd data/scripts
# First run β€” downloads all UNHCR data (~5-7 min):
python3 unhcr_downloader.py --all
# Then enrich countries.json with live stats (~4 min):
python3 enrich_downloader.py
# Output: specs/data/countries_enriched.json
# Annual refresh:
python3 refresh_data.py
```
## Data license
All UNHCR data is published under **CC BY 4.0**.
Source: [UNHCR Refugee Statistics API](https://api.unhcr.org/population/v1/)
Attribution required in any public-facing use.