expected-greenness / README.md
Konuktor92's picture
expected-greenness v1
cb29b3c verified
|
Raw History Blame Contribute Delete
4 kB
---
license: other
library_name: xgboost
tags: [nasa, modis, ndvi, imerg, gldas, pastures, kyrgyzstan, tabular-regression]
datasets: [TrendeX90/naryn-cube]
---
# TrendeX expected-greenness model (v1)
Predicts the **summer greenness (NDVI) anomaly** of every 1 km pasture cell in Naryn oblast, Kyrgyzstan
(May-September, 2001-2025) **from weather and terrain alone**. The gap between observed and expected NDVI is
change **not explained by precipitation, temperature, snow and soil moisture**. It is not a cause and says
nothing about forage quality. Built for NASA Space Apps 2026 ("Be An Earth System Trend Detective!").
## Inputs
- Weather anomalies vs each cell's 2001-2025 mean for the same month: GPM IMERG precipitation (month, 3-month,
Oct-Apr), MODIS LST day/night (month, previous month), MODIS snow (Mar-May, month), GLDAS soil moisture
(month) and April snow water.
- Terrain from NASADEM (elevation, spread, slope, steep share, aspect, TPI) and land cover (MCD12Q1 2020).
- The cell's own NDVI climatology for that month. **Not used on purpose:** year, coordinates, past NDVI.
## How it was tested (protocol locked in git before the first fit: `model_v1.yaml`, commit 92ed159)
- Tuning only on 2001-2015 -> 2016-2019. The 2020-2025 years were used once, for the final test.
- Spatial test: 5-fold GroupKFold over ~30 x 30 km super-blocks (all years; easier than the temporal test
because the same years' weather appears in training).
- Baselines: climatology (anomaly = 0) and ridge regression on the same features.
- Pre-registered gate: ship XGBoost only if the 95% CI lower bound of its skill over ridge is > 0 on both tests.
**Gate passed: True. Shipped: xgb.**
| Test | RMSE xgb | RMSE ridge | RMSE climatology | anomaly corr xgb | anomaly corr ridge | tercile bal. acc. xgb |
|---|---|---|---|---|---|---|
| Temporal (2020-2025) | 0.0554 | 0.0588 | 0.0616 | 0.424 | 0.288 | 0.455 |
| Spatial (unseen areas) | 0.0523 | 0.0578 | 0.0619 | 0.540 | 0.358 | 0.524 |
- Skill vs ridge, temporal: 0.113 (95% CI 0.089 to 0.133, 6 units). Five seeds: 0.110 +/- 0.003.
- Skill vs ridge, spatial: 0.180 (95% CI 0.169 to 0.192, 76 units).
- Skill vs climatology, temporal: 0.191 (95% CI 0.160 to 0.215, 6 units).
- What each NASA dataset adds (skill vs climatology, temporal test): precip: 0.055 -> +lst: 0.167 -> +snow: 0.181 -> +gldas (full): 0.188.
- Most used inputs: lst_day_m, lst_day_prev, precip_3m, lst_night_m, soil_m, month.
## Limits (read before using)
- The model explains about a fifth of the year-to-year NDVI anomaly on unseen years; most variation is not weather-explained.
- P10-P90 band: on 2020-2025 it covered **74% instead of 80%**, so treat it as too narrow.
- Weather comes from coarse grids (IMERG 0.1 deg, GLDAS 0.25 deg) interpolated to 1 km.
- NDVI is greenness, not forage: shrub encroachment can look like improvement.
- Land cover is a fixed 2020 map; MODIS Terra drifted in recent years (see Aqua-Terra check in the dataset products).
- What-if runs: drier and hotter-than-normal conditions lower expected July NDVI, but wetter-than-normal
precipitation does not raise it (slightly lowers it on average). Treat what-if outputs as model
sensitivities, not causal effects.
- No field validation. Relative to Naryn pastures only.
## Use
```python
from predict import load
m = load(".") # needs only numpy, pandas, xgboost (CPU)
out = m.predict(features_df) # columns as in feature_schema.json
scen = m.what_if(cells, cell_ids, month=7, precip_pct=-30) # cells = naryn-cube cells.parquet
```
`pytest test_golden.py` reproduces the 100 golden rows.
## Data and tools
MOD13A3.061, MYD13A3.061, MOD11A2.061, MOD10A2.061, MCD12Q1.061, NASADEM_HGT.001 (LP DAAC / NSIDC via earthaccess),
GPM_3IMERGM.07 and GLDAS_NOAH025_M.2.1 (GES DISC). XGBoost, Optuna, scikit-learn, pymannkendall.
AI use: implementation was assisted by Claude (Anthropic); design decisions and the product idea are the team's.