{ "cells": [ { "cell_type": "markdown", "id": "8c71f5a1", "metadata": {}, "source": [ "# Phase 3 — Random Forest vs Presto Embeddings (Comparison)\n", "\n", "**Original notebook left unchanged:** `Phase3 (2).ipynb`\n", "\n", "This copy keeps your same Random Forest idea (predict healthy NDVI from location/season, then detect 2025 health drops), and adds:\n", "\n", "1. **Baseline score** — same features as your RF: `day_of_year`, `latitude`, `longitude`\n", "2. **Presto score** — Random Forest trained on **128-D Presto embeddings**\n", "3. **Side-by-side metrics** — R², RMSE, MAE on the same validation split\n", "\n", "### Important (data reality check)\n", "Your Drive CSVs contain **NDVI + lat/lon** (NDVI from GEE = NIR & Red). They do **not** contain full Sentinel-1/2 + climate stacks.\n", "\n", "So Presto is run with:\n", "- **NDVI** in band 16\n", "- other EO bands **masked** (ignored by Presto)\n", "- lat/lon + month\n", "\n", "That is the fair best use of Presto with your current files. Full 17-band Presto would need a new GEE export.\n", "\n", "Use **Runtime → Run all** (GPU optional).\n" ] }, { "cell_type": "markdown", "id": "0e558b99", "metadata": {}, "source": [ "## 0. Install dependencies" ] }, { "cell_type": "code", "execution_count": null, "id": "5cc8b10f", "metadata": {}, "outputs": [], "source": [ "%pip install -q --upgrade pip\n", "%pip install -q scikit-learn pandas numpy matplotlib seaborn torch einops folium\n", "\n", "import warnings\n", "warnings.filterwarnings(\"ignore\")\n", "print(\"OK: packages ready\")" ] }, { "cell_type": "markdown", "id": "92ceafeb", "metadata": {}, "source": [ "## 1. Mount Drive & load Michigan NDVI CSVs (same logic as your original)" ] }, { "cell_type": "code", "execution_count": null, "id": "2ad7a0cd", "metadata": {}, "outputs": [], "source": [ "import os, glob\n", "import numpy as np\n", "import pandas as pd\n", "from google.colab import drive\n", "\n", "if not os.path.exists(\"/content/drive\"):\n", " drive.mount(\"/content/drive\")\n", "\n", "path = \"/content/drive/MyDrive/*.csv\"\n", "all_files = [f for f in glob.glob(path) if any(x in f for x in [\"Michigan\", \"Summer\", \"NDVI\"])]\n", "print(f\"Found {len(all_files)} candidate CSV files\")\n", "\n", "df_list = []\n", "print(\"Loading data and separating years...\")\n", "\n", "for filename in all_files:\n", " try:\n", " peek = pd.read_csv(filename, nrows=1)\n", " date_col = [c for c in peek.columns if c.lower() in [\"date\", \"system:time_start\", \"time\"]][0]\n", " temp = pd.read_csv(filename, usecols=[date_col, \"NDVI\", \"latitude\", \"longitude\", \"park\"])\n", " temp = temp[temp[\"NDVI\"] > 0.1]\n", " temp = temp.rename(columns={date_col: \"raw_date\"})\n", " temp[\"year\"] = pd.to_datetime(temp[\"raw_date\"], errors=\"coerce\").dt.year\n", "\n", " if temp[\"year\"].isnull().any():\n", " temp[\"year\"] = temp[\"raw_date\"].astype(str).str.extract(r\"(\\d{4})\").astype(float)\n", "\n", " if temp[\"year\"].isnull().any():\n", " fname = filename.split(\"/\")[-1]\n", " if \"2025\" in fname and \"2022\" not in fname:\n", " temp[\"year\"] = temp[\"year\"].fillna(2025)\n", " elif \"2024\" in fname:\n", " temp[\"year\"] = temp[\"year\"].fillna(2024)\n", " elif \"2023\" in fname:\n", " temp[\"year\"] = temp[\"year\"].fillna(2023)\n", " else:\n", " temp[\"year\"] = temp[\"year\"].fillna(2022)\n", "\n", " if (temp[\"year\"] == 2025).any():\n", " df_list.append(temp)\n", " else:\n", " df_list.append(temp.sample(frac=0.3, random_state=42))\n", "\n", " print(f\"-> Processed: {filename.split('/')[-1]}\")\n", " except Exception as e:\n", " print(f\"skip {filename}: {e}\")\n", " continue\n", "\n", "df = pd.concat(df_list, ignore_index=True)\n", "del df_list\n", "\n", "df[\"day_of_year\"] = 180\n", "df = df.dropna(subset=[\"year\", \"NDVI\", \"latitude\", \"longitude\"])\n", "df[\"year\"] = df[\"year\"].astype(int)\n", "df[\"month\"] = pd.to_datetime(df[\"raw_date\"], errors=\"coerce\").dt.month.fillna(6).astype(int)\n", "\n", "train_df = df[df[\"year\"] < 2025].copy()\n", "test_df = df[df[\"year\"] == 2025].copy()\n", "del df\n", "\n", "print(f\"Baseline Training Data: {len(train_df):,} rows\")\n", "print(f\"2025 Target Data: {len(test_df):,} rows\")\n", "assert len(train_df) > 0, \"No training rows found\"\n", "assert len(test_df) > 0, \"No 2025 rows found — check Drive CSV names\"\n", "print(\"OK: data loaded\")" ] }, { "cell_type": "markdown", "id": "1d7cfa78", "metadata": {}, "source": [ "## 2. BASELINE Random Forest (same model as your notebook) + scores\n", "\n", "Your original notebook did not print R² / RMSE. Here we evaluate on a **held-out 20% of pre-2025 data** (same target: NDVI).\n" ] }, { "cell_type": "code", "execution_count": null, "id": "0333a535", "metadata": {}, "outputs": [], "source": [ "from sklearn.ensemble import RandomForestRegressor\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.metrics import r2_score, mean_squared_error, mean_absolute_error\n", "\n", "FEATURES_BASE = [\"day_of_year\", \"latitude\", \"longitude\"]\n", "TARGET = \"NDVI\"\n", "\n", "# Subsample for speed/RAM while keeping comparison fair and repeatable\n", "MAX_BASE_ROWS = 250_000\n", "train_base = train_df.sample(n=min(MAX_BASE_ROWS, len(train_df)), random_state=42)\n", "\n", "X_train, X_val, y_train, y_val = train_test_split(\n", " train_base[FEATURES_BASE],\n", " train_base[TARGET],\n", " test_size=0.2,\n", " random_state=42,\n", ")\n", "\n", "rf_base = RandomForestRegressor(\n", " n_estimators=25,\n", " max_depth=12,\n", " n_jobs=-1,\n", " random_state=42,\n", ")\n", "rf_base.fit(X_train, y_train)\n", "\n", "pred_val_base = rf_base.predict(X_val)\n", "pred_2025_base = rf_base.predict(test_df[FEATURES_BASE])\n", "\n", "metrics_base = {\n", " \"model\": \"Baseline RF (doy, lat, lon)\",\n", " \"features\": \"day_of_year + latitude + longitude\",\n", " \"R2_val\": float(r2_score(y_val, pred_val_base)),\n", " \"RMSE_val\": float(np.sqrt(mean_squared_error(y_val, pred_val_base))),\n", " \"MAE_val\": float(mean_absolute_error(y_val, pred_val_base)),\n", " \"R2_2025\": float(r2_score(test_df[TARGET], pred_2025_base)),\n", " \"RMSE_2025\": float(np.sqrt(mean_squared_error(test_df[TARGET], pred_2025_base))),\n", " \"MAE_2025\": float(mean_absolute_error(test_df[TARGET], pred_2025_base)),\n", "}\n", "\n", "print(\"BASELINE SCORES\")\n", "for k, v in metrics_base.items():\n", " print(f\" {k}: {v}\")\n", "\n", "# Same anomaly rule as your original notebook\n", "test_df = test_df.copy()\n", "test_df[\"predicted_NDVI_base\"] = pred_2025_base\n", "test_df[\"health_drop_base\"] = test_df[\"predicted_NDVI_base\"] - test_df[\"NDVI\"]\n", "sick_base = test_df[test_df[\"health_drop_base\"] > 0.2]\n", "print(f\"2025 sick locations (baseline, health_drop>0.2): {len(sick_base):,}\")\n", "print(\"OK: baseline complete\")" ] }, { "cell_type": "markdown", "id": "9b932a3f", "metadata": {}, "source": [ "## 3. Download & load Presto\n", "\n", "Uses NASA Harvest `single_file_presto` + default weights (same as your working Colab).\n" ] }, { "cell_type": "code", "execution_count": null, "id": "14744d20", "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "import sys\n", "import urllib.request\n", "import torch\n", "\n", "ROOT = Path(\"/content/presto_run\") if Path(\"/content\").exists() else Path.cwd() / \"presto_run\"\n", "ROOT.mkdir(parents=True, exist_ok=True)\n", "\n", "files = {\n", " \"single_file_presto.py\": \"https://raw.githubusercontent.com/nasaharvest/presto/main/single_file_presto.py\",\n", " \"default_model.pt\": \"https://github.com/nasaharvest/presto/raw/main/data/default_model.pt\",\n", "}\n", "for name, url in files.items():\n", " dest = ROOT / name\n", " if dest.exists() and dest.stat().st_size > 0:\n", " print(f\"already present: {name}\")\n", " continue\n", " print(f\"downloading {name} ...\")\n", " urllib.request.urlretrieve(url, dest)\n", " print(f\"saved {dest} ({dest.stat().st_size} bytes)\")\n", "\n", "sys.path.insert(0, str(ROOT))\n", "from single_file_presto import NUM_DYNAMIC_WORLD_CLASSES, Presto\n", "\n", "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", "presto = Presto.construct()\n", "try:\n", " state = torch.load(ROOT / \"default_model.pt\", map_location=device, weights_only=True)\n", "except TypeError:\n", " state = torch.load(ROOT / \"default_model.pt\", map_location=device)\n", "presto.load_state_dict(state)\n", "presto.eval()\n", "presto.to(device)\n", "print(\"OK: Presto loaded on\", device)" ] }, { "cell_type": "markdown", "id": "b4c8db31", "metadata": {}, "source": [ "## 4. Build Presto embeddings for a comparable sample\n", "\n", "Because full 17-band stacks are not in your CSVs, each pixel uses:\n", "\n", "| Band | Value | Mask |\n", "|------|-------|------|\n", "| 0–15 (S1/S2/climate/terrain) | 0 | 1 (ignored) |\n", "| 16 (NDVI) | your NDVI | 0 (used) |\n", "\n", "Embeddings encode **NDVI + location + month** through Presto.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "705cb162", "metadata": {}, "outputs": [], "source": [ "NUM_BANDS = 17\n", "NDVI_BAND = 16\n", "# Keep Presto feasible on Colab free GPU/CPU\n", "PRESTO_SAMPLE_N = 12_000\n", "BATCH = 256\n", "\n", "# Use same sampled pre-2025 universe as baseline where possible\n", "presto_pool = train_base.sample(n=min(PRESTO_SAMPLE_N, len(train_base)), random_state=42).reset_index(drop=True)\n", "print(f\"Computing Presto embeddings for {len(presto_pool):,} points on {device}...\")\n", "\n", "emb_list = []\n", "with torch.no_grad():\n", " for i in range(0, len(presto_pool), BATCH):\n", " chunk = presto_pool.iloc[i : i + BATCH]\n", " b = len(chunk)\n", " x = torch.zeros(b, 1, NUM_BANDS, dtype=torch.float32, device=device)\n", " mask = torch.ones(b, 1, NUM_BANDS, dtype=torch.float32, device=device)\n", " # unmask NDVI band and write values\n", " mask[:, 0, NDVI_BAND] = 0.0\n", " x[:, 0, NDVI_BAND] = torch.tensor(chunk[\"NDVI\"].to_numpy(), dtype=torch.float32, device=device)\n", "\n", " dw = torch.full((b, 1), NUM_DYNAMIC_WORLD_CLASSES, dtype=torch.long, device=device)\n", " latlons = torch.tensor(chunk[[\"latitude\", \"longitude\"]].to_numpy(), dtype=torch.float32, device=device)\n", " month = torch.tensor(chunk[\"month\"].to_numpy(), dtype=torch.long, device=device)\n", "\n", " emb = presto.encoder(\n", " x=x,\n", " dynamic_world=dw,\n", " latlons=latlons,\n", " mask=mask,\n", " month=month,\n", " eval_task=True,\n", " )\n", " emb_list.append(emb.detach().cpu().numpy())\n", " if (i // BATCH) % 10 == 0:\n", " print(f\" batch {i // BATCH + 1}/{(len(presto_pool) + BATCH - 1) // BATCH}\")\n", "\n", "E = np.vstack(emb_list)\n", "print(\"OK: embeddings shape\", E.shape)" ] }, { "cell_type": "markdown", "id": "f792141c", "metadata": {}, "source": [ "## 5. PRESTO Random Forest score (same target: NDVI, same split style)" ] }, { "cell_type": "code", "execution_count": null, "id": "7d31484c", "metadata": {}, "outputs": [], "source": [ "# Split embeddings with same random_state for fairness\n", "idx = np.arange(len(presto_pool))\n", "idx_tr, idx_va = train_test_split(idx, test_size=0.2, random_state=42)\n", "\n", "Xtr, Xva = E[idx_tr], E[idx_va]\n", "ytr = presto_pool.loc[idx_tr, TARGET].to_numpy()\n", "yva = presto_pool.loc[idx_va, TARGET].to_numpy()\n", "\n", "rf_presto = RandomForestRegressor(\n", " n_estimators=25,\n", " max_depth=12,\n", " n_jobs=-1,\n", " random_state=42,\n", ")\n", "rf_presto.fit(Xtr, ytr)\n", "pred_va_p = rf_presto.predict(Xva)\n", "\n", "metrics_presto = {\n", " \"model\": \"Presto RF (128-D embeddings)\",\n", " \"features\": \"Presto encoder [NDVI band + lat/lon + month]\",\n", " \"R2_val\": float(r2_score(yva, pred_va_p)),\n", " \"RMSE_val\": float(np.sqrt(mean_squared_error(yva, pred_va_p))),\n", " \"MAE_val\": float(mean_absolute_error(yva, pred_va_p)),\n", "}\n", "\n", "print(\"PRESTO SCORES (validation)\")\n", "for k, v in metrics_presto.items():\n", " print(f\" {k}: {v}\")\n", "\n", "# Optional: Presto embeddings on a 2025 sample for disease counts\n", "sample_2025 = test_df.sample(n=min(PRESTO_SAMPLE_N, len(test_df)), random_state=42).reset_index(drop=True)\n", "emb_2025 = []\n", "with torch.no_grad():\n", " for i in range(0, len(sample_2025), BATCH):\n", " chunk = sample_2025.iloc[i : i + BATCH]\n", " b = len(chunk)\n", " x = torch.zeros(b, 1, NUM_BANDS, dtype=torch.float32, device=device)\n", " mask = torch.ones(b, 1, NUM_BANDS, dtype=torch.float32, device=device)\n", " mask[:, 0, NDVI_BAND] = 0.0\n", " x[:, 0, NDVI_BAND] = torch.tensor(chunk[\"NDVI\"].to_numpy(), dtype=torch.float32, device=device)\n", " dw = torch.full((b, 1), NUM_DYNAMIC_WORLD_CLASSES, dtype=torch.long, device=device)\n", " latlons = torch.tensor(chunk[[\"latitude\", \"longitude\"]].to_numpy(), dtype=torch.float32, device=device)\n", " month = torch.tensor(chunk[\"month\"].to_numpy(), dtype=torch.long, device=device)\n", " emb = presto.encoder(x=x, dynamic_world=dw, latlons=latlons, mask=mask, month=month, eval_task=True)\n", " emb_2025.append(emb.detach().cpu().numpy())\n", "\n", "E2025 = np.vstack(emb_2025)\n", "pred_2025_p = rf_presto.predict(E2025)\n", "sample_2025 = sample_2025.copy()\n", "sample_2025[\"predicted_NDVI_presto\"] = pred_2025_p\n", "sample_2025[\"health_drop_presto\"] = sample_2025[\"predicted_NDVI_presto\"] - sample_2025[\"NDVI\"]\n", "sick_presto = sample_2025[sample_2025[\"health_drop_presto\"] > 0.2]\n", "\n", "metrics_presto[\"R2_2025_sample\"] = float(r2_score(sample_2025[TARGET], pred_2025_p))\n", "metrics_presto[\"RMSE_2025_sample\"] = float(np.sqrt(mean_squared_error(sample_2025[TARGET], pred_2025_p)))\n", "metrics_presto[\"MAE_2025_sample\"] = float(mean_absolute_error(sample_2025[TARGET], pred_2025_p))\n", "metrics_presto[\"sick_2025_sample\"] = int(len(sick_presto))\n", "metrics_presto[\"sick_2025_sample_rate\"] = float(len(sick_presto) / max(len(sample_2025), 1))\n", "\n", "print(f\"2025 sample sick (Presto, health_drop>0.2): {len(sick_presto):,} / {len(sample_2025):,}\")\n", "print(\"OK: Presto RF complete\")" ] }, { "cell_type": "markdown", "id": "6d827607", "metadata": {}, "source": [ "## 6. Side-by-side comparison table + winner" ] }, { "cell_type": "code", "execution_count": null, "id": "cb839566", "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "\n", "# Align baseline metrics onto the SAME Presto validation rows for a fair head-to-head\n", "val_rows = presto_pool.loc[idx_va]\n", "pred_val_base_aligned = rf_base.predict(val_rows[FEATURES_BASE])\n", "aligned_base = {\n", " \"R2_val\": float(r2_score(val_rows[TARGET], pred_val_base_aligned)),\n", " \"RMSE_val\": float(np.sqrt(mean_squared_error(val_rows[TARGET], pred_val_base_aligned))),\n", " \"MAE_val\": float(mean_absolute_error(val_rows[TARGET], pred_val_base_aligned)),\n", "}\n", "\n", "comparison = pd.DataFrame([\n", " {\n", " \"Model\": \"A — Baseline RF\",\n", " \"Features\": \"doy, lat, lon (your original)\",\n", " \"R2 (same val set)\": aligned_base[\"R2_val\"],\n", " \"RMSE (same val set)\": aligned_base[\"RMSE_val\"],\n", " \"MAE (same val set)\": aligned_base[\"MAE_val\"],\n", " },\n", " {\n", " \"Model\": \"B — Presto RF\",\n", " \"Features\": \"128-D Presto embeddings\",\n", " \"R2 (same val set)\": metrics_presto[\"R2_val\"],\n", " \"RMSE (same val set)\": metrics_presto[\"RMSE_val\"],\n", " \"MAE (same val set)\": metrics_presto[\"MAE_val\"],\n", " },\n", "])\n", "\n", "print(\"\\n\" + \"=\" * 72)\n", "print(\"FAIR COMPARISON (identical validation pixels)\")\n", "print(\"=\" * 72)\n", "print(comparison.to_string(index=False))\n", "print(\"=\" * 72)\n", "\n", "# Higher R2 wins; lower RMSE/MAE wins\n", "r2_winner = \"Presto RF\" if metrics_presto[\"R2_val\"] > aligned_base[\"R2_val\"] else \"Baseline RF\"\n", "rmse_winner = \"Presto RF\" if metrics_presto[\"RMSE_val\"] < aligned_base[\"RMSE_val\"] else \"Baseline RF\"\n", "print(f\"Best R2: {r2_winner}\")\n", "print(f\"Best RMSE: {rmse_winner}\")\n", "\n", "# Also show full-baseline metrics (larger sample) for reference\n", "print(\"\\nReference — Baseline on large holdout (section 2):\")\n", "print(f\" R2_val={metrics_base['R2_val']:.4f} RMSE_val={metrics_base['RMSE_val']:.4f} MAE_val={metrics_base['MAE_val']:.4f}\")\n", "print(f\" R2_2025={metrics_base['R2_2025']:.4f} sick_count={len(sick_base):,}\")\n", "\n", "fig, ax = plt.subplots(1, 3, figsize=(12, 3.5))\n", "labels = [\"Baseline RF\", \"Presto RF\"]\n", "ax[0].bar(labels, [aligned_base[\"R2_val\"], metrics_presto[\"R2_val\"]], color=[\"#4C78A8\", \"#F58518\"])\n", "ax[0].set_title(\"R2 (higher better)\")\n", "ax[1].bar(labels, [aligned_base[\"RMSE_val\"], metrics_presto[\"RMSE_val\"]], color=[\"#4C78A8\", \"#F58518\"])\n", "ax[1].set_title(\"RMSE (lower better)\")\n", "ax[2].bar(labels, [aligned_base[\"MAE_val\"], metrics_presto[\"MAE_val\"]], color=[\"#4C78A8\", \"#F58518\"])\n", "ax[2].set_title(\"MAE (lower better)\")\n", "plt.tight_layout()\n", "plt.show()\n", "\n", "comparison.to_csv(\"/content/RF_vs_Presto_comparison_scores.csv\", index=False)\n", "print(\"Saved: /content/RF_vs_Presto_comparison_scores.csv\")\n", "print(\"OK: comparison complete\")" ] }, { "cell_type": "markdown", "id": "788e9061", "metadata": {}, "source": [ "## 7. How to read the result\n", "\n", "- **Baseline RF** = your current approach (geo + day → expected NDVI).\n", "- **Presto RF** = Presto turns NDVI+location+month into a 128-D embedding, then RF predicts NDVI.\n", "- If Presto has **higher R²** and **lower RMSE/MAE**, embeddings help on this dataset.\n", "- Because other EO bands are masked, this is **not yet** a full multi-band Presto test. Next upgrade: export S1/S2/ERA5/SRTM from GEE and unmask those bands.\n", "\n", "Original notebook untouched: `Phase3 (2).ipynb` \n", "Hosted Presto UI: https://huggingface.co/spaces/morta00/presto-free \n", "Presto Colab: https://huggingface.co/morta00/presto-colab/colab\n" ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 5 }