KSvend Claude Happy commited on
Commit
3a71197
·
1 Parent(s): 7403c8c

docs: add Phase B implementation plan — migrate 4 indicators

Browse files

Water (MNDWI via openEO), LST (Sentinel-3 SLSTR via openEO),
Rainfall (CHIRPS direct download), Nightlights (VIIRS EOG direct
download). 7 tasks with full TDD code.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

docs/superpowers/plans/2026-03-31-phase-b-migrate-indicators.md ADDED
@@ -0,0 +1,2007 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Phase B: Migrate Remaining Indicators to openEO / Direct Download — Implementation Plan
2
+
3
+ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
+
5
+ **Goal:** Migrate water, LST, rainfall, and nightlights indicators from scene-level metadata / centroid queries to pixel-level raster products — water and LST via CDSE openEO, rainfall via CHIRPS direct download, nightlights via VIIRS EOG direct download.
6
+
7
+ **Architecture:** openEO indicators (water, LST) follow the NDVI pattern from Phase A: build processing graph → download GeoTIFF → compute stats → classify. Direct download indicators (rainfall, nightlights) use `httpx` to fetch GeoTIFFs from public archives, then follow the same post-processing pipeline. All indicators set `SpatialData(map_type="raster")` with indicator-specific `vmin`/`vmax` stored on the SpatialData dataclass.
8
+
9
+ **Tech Stack:** Python 3.11, openeo, rasterio, httpx, numpy, matplotlib
10
+
11
+ **Spec:** `docs/superpowers/specs/2026-03-31-openeo-eo-upgrade-design.md`
12
+
13
+ ---
14
+
15
+ ## File Map
16
+
17
+ | File | Action | Responsibility |
18
+ |------|--------|----------------|
19
+ | `app/indicators/base.py` | Modify | Add `vmin`/`vmax` fields to `SpatialData` |
20
+ | `app/worker.py` | Modify | Read `vmin`/`vmax` from SpatialData instead of hardcoding |
21
+ | `app/openeo_client.py` | Modify | Add `build_mndwi_graph()` and `build_lst_graph()` |
22
+ | `app/indicators/water.py` | Rewrite | MNDWI pixel-level water classification via openEO |
23
+ | `app/indicators/lst.py` | Rewrite | Sentinel-3 SLSTR LST via openEO |
24
+ | `app/indicators/rainfall.py` | Rewrite | CHIRPS precipitation via direct download |
25
+ | `app/indicators/nightlights.py` | Rewrite | VIIRS DNB monthly via EOG direct download |
26
+ | `tests/test_openeo_client.py` | Modify | Add tests for new graph builders |
27
+ | `tests/test_indicator_water.py` | Rewrite | Tests for openEO-based water indicator |
28
+ | `tests/test_indicator_lst.py` | Rewrite | Tests for openEO-based LST indicator |
29
+ | `tests/test_indicator_rainfall.py` | Rewrite | Tests for CHIRPS-based rainfall |
30
+ | `tests/test_indicator_nightlights.py` | Rewrite | Tests for VIIRS-based nightlights |
31
+
32
+ ---
33
+
34
+ ### Task 1: Generalize Raster Map vmin/vmax in SpatialData and Worker
35
+
36
+ **Files:**
37
+ - Modify: `app/indicators/base.py:13-22`
38
+ - Modify: `app/worker.py:95-113`
39
+ - Modify: `app/indicators/ndvi.py` (set vmin/vmax on SpatialData)
40
+
41
+ - [ ] **Step 1: Add vmin/vmax fields to SpatialData**
42
+
43
+ In `app/indicators/base.py`, replace the `SpatialData` dataclass (lines 13-22):
44
+
45
+ ```python
46
+ @dataclass
47
+ class SpatialData:
48
+ """Spatial data produced by an indicator for map rendering."""
49
+ data: np.ndarray | None = None
50
+ lons: np.ndarray | None = None
51
+ lats: np.ndarray | None = None
52
+ label: str = ""
53
+ colormap: str = "RdYlGn"
54
+ geojson: dict | None = None
55
+ map_type: str = "grid"
56
+ vmin: float | None = None
57
+ vmax: float | None = None
58
+ ```
59
+
60
+ - [ ] **Step 2: Update NDVI indicator to set vmin/vmax on SpatialData**
61
+
62
+ In `app/indicators/ndvi.py`, find the line where `self._spatial_data` is assigned (in `_process_openeo`). Replace:
63
+
64
+ ```python
65
+ self._spatial_data = SpatialData(
66
+ map_type="raster",
67
+ label="NDVI",
68
+ colormap="RdYlGn",
69
+ )
70
+ ```
71
+
72
+ With:
73
+
74
+ ```python
75
+ self._spatial_data = SpatialData(
76
+ map_type="raster",
77
+ label="NDVI",
78
+ colormap="RdYlGn",
79
+ vmin=-0.2,
80
+ vmax=0.9,
81
+ )
82
+ ```
83
+
84
+ - [ ] **Step 3: Update worker to read vmin/vmax from SpatialData**
85
+
86
+ In `app/worker.py`, replace lines 95-113 (the raster map rendering block):
87
+
88
+ ```python
89
+ if spatial is not None and spatial.map_type == "raster":
90
+ # Raster-on-true-color rendering for openEO/download indicators
91
+ indicator_obj = registry.get(result.indicator_id)
92
+ raster_path = getattr(indicator_obj, '_indicator_raster_path', None)
93
+ true_color_path = getattr(indicator_obj, '_true_color_path', None)
94
+ render_band = getattr(indicator_obj, '_render_band', 1)
95
+ from app.outputs.maps import render_raster_map
96
+ render_raster_map(
97
+ true_color_path=true_color_path,
98
+ indicator_path=raster_path,
99
+ indicator_band=render_band,
100
+ aoi=job.request.aoi,
101
+ status=result.status,
102
+ output_path=map_path,
103
+ cmap=spatial.colormap,
104
+ vmin=spatial.vmin,
105
+ vmax=spatial.vmax,
106
+ label=spatial.label,
107
+ )
108
+ ```
109
+
110
+ Note: renamed `_ndvi_peak_band` to generic `_render_band` in the worker. Update NDVI indicator to match:
111
+
112
+ In `app/indicators/ndvi.py`, find `self._ndvi_peak_band = current_stats["peak_month_band"]` and add after it:
113
+
114
+ ```python
115
+ self._render_band = current_stats["peak_month_band"]
116
+ ```
117
+
118
+ - [ ] **Step 4: Run all tests**
119
+
120
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/ -v --timeout=120 2>&1 | tail -10`
121
+
122
+ Expected: All 136 tests PASS (the worker test and NDVI tests should still work since `_ndvi_peak_band` still exists alongside `_render_band`).
123
+
124
+ - [ ] **Step 5: Commit**
125
+
126
+ ```bash
127
+ git add app/indicators/base.py app/worker.py app/indicators/ndvi.py
128
+ git commit -m "refactor: generalize raster map vmin/vmax via SpatialData fields
129
+
130
+ Generated with [Claude Code](https://claude.ai/code)
131
+ via [Happy](https://happy.engineering)
132
+
133
+ Co-Authored-By: Claude <noreply@anthropic.com>
134
+ Co-Authored-By: Happy <yesreply@happy.engineering>"
135
+ ```
136
+
137
+ ---
138
+
139
+ ### Task 2: Add MNDWI and LST Graph Builders to openEO Client
140
+
141
+ **Files:**
142
+ - Modify: `app/openeo_client.py`
143
+ - Modify: `tests/test_openeo_client.py`
144
+
145
+ - [ ] **Step 1: Write tests for new graph builders**
146
+
147
+ Append to `tests/test_openeo_client.py`:
148
+
149
+ ```python
150
+ def test_build_mndwi_graph():
151
+ """build_mndwi_graph() loads Sentinel-2 with water index bands."""
152
+ mock_conn = MagicMock()
153
+ mock_cube = MagicMock()
154
+ mock_conn.load_collection.return_value = mock_cube
155
+
156
+ from app.openeo_client import build_mndwi_graph
157
+
158
+ bbox = {"west": 32.45, "south": 15.65, "east": 32.65, "north": 15.8}
159
+ result = build_mndwi_graph(
160
+ conn=mock_conn,
161
+ bbox=bbox,
162
+ temporal_extent=["2025-03-01", "2026-03-01"],
163
+ resolution_m=100,
164
+ )
165
+
166
+ mock_conn.load_collection.assert_called_once()
167
+ call_kwargs = mock_conn.load_collection.call_args
168
+ assert call_kwargs[1]["collection_id"] == "SENTINEL2_L2A"
169
+ assert "B03" in call_kwargs[1]["bands"]
170
+ assert "B11" in call_kwargs[1]["bands"]
171
+ assert "SCL" in call_kwargs[1]["bands"]
172
+
173
+
174
+ def test_build_lst_graph():
175
+ """build_lst_graph() loads Sentinel-3 SLSTR LST data."""
176
+ mock_conn = MagicMock()
177
+ mock_cube = MagicMock()
178
+ mock_conn.load_collection.return_value = mock_cube
179
+
180
+ from app.openeo_client import build_lst_graph
181
+
182
+ bbox = {"west": 32.45, "south": 15.65, "east": 32.65, "north": 15.8}
183
+ result = build_lst_graph(
184
+ conn=mock_conn,
185
+ bbox=bbox,
186
+ temporal_extent=["2025-03-01", "2026-03-01"],
187
+ resolution_m=1000,
188
+ )
189
+
190
+ mock_conn.load_collection.assert_called_once()
191
+ call_kwargs = mock_conn.load_collection.call_args
192
+ # Should load Sentinel-3 SLSTR
193
+ assert "SENTINEL3" in call_kwargs[1]["collection_id"].upper() or "SLSTR" in call_kwargs[1]["collection_id"].upper()
194
+ ```
195
+
196
+ - [ ] **Step 2: Run tests to verify they fail**
197
+
198
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/test_openeo_client.py::test_build_mndwi_graph tests/test_openeo_client.py::test_build_lst_graph -v`
199
+
200
+ Expected: FAIL — `ImportError`
201
+
202
+ - [ ] **Step 3: Implement graph builders**
203
+
204
+ Append to `app/openeo_client.py` after the `build_true_color_graph` function:
205
+
206
+ ```python
207
+
208
+
209
+ def build_mndwi_graph(
210
+ *,
211
+ conn: openeo.Connection,
212
+ bbox: dict[str, float],
213
+ temporal_extent: list[str],
214
+ resolution_m: int = 100,
215
+ ) -> openeo.DataCube:
216
+ """Build an openEO process graph for monthly MNDWI water index composites.
217
+
218
+ MNDWI = (B03 - B11) / (B03 + B11). Positive values indicate water.
219
+ Cloud-masked via SCL band, aggregated to monthly medians.
220
+ """
221
+ cube = conn.load_collection(
222
+ collection_id="SENTINEL2_L2A",
223
+ spatial_extent=bbox,
224
+ temporal_extent=temporal_extent,
225
+ bands=["B03", "B11", "SCL"],
226
+ )
227
+
228
+ scl = cube.band("SCL")
229
+ cloud_mask = (scl == 4) | (scl == 5) | (scl == 6)
230
+ cube = cube.mask(~cloud_mask)
231
+
232
+ b03 = cube.band("B03")
233
+ b11 = cube.band("B11")
234
+ mndwi = (b03 - b11) / (b03 + b11)
235
+
236
+ monthly = mndwi.aggregate_temporal_period("month", reducer="median")
237
+
238
+ if resolution_m > 10:
239
+ monthly = monthly.resample_spatial(resolution=resolution_m / 111320)
240
+
241
+ return monthly
242
+
243
+
244
+ def build_lst_graph(
245
+ *,
246
+ conn: openeo.Connection,
247
+ bbox: dict[str, float],
248
+ temporal_extent: list[str],
249
+ resolution_m: int = 1000,
250
+ ) -> openeo.DataCube:
251
+ """Build an openEO process graph for Sentinel-3 SLSTR land surface temperature.
252
+
253
+ Loads LST from Sentinel-3 SLSTR Level-2 product, aggregated to monthly means.
254
+ Resolution is typically 1km (SLSTR native).
255
+ """
256
+ cube = conn.load_collection(
257
+ collection_id="SENTINEL3_SLSTR_L2_LST",
258
+ spatial_extent=bbox,
259
+ temporal_extent=temporal_extent,
260
+ bands=["LST"],
261
+ )
262
+
263
+ monthly = cube.aggregate_temporal_period("month", reducer="mean")
264
+
265
+ if resolution_m > 1000:
266
+ monthly = monthly.resample_spatial(resolution=resolution_m / 111320)
267
+
268
+ return monthly
269
+ ```
270
+
271
+ - [ ] **Step 4: Run tests**
272
+
273
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/test_openeo_client.py -v`
274
+
275
+ Expected: All PASS (6 tests now).
276
+
277
+ - [ ] **Step 5: Commit**
278
+
279
+ ```bash
280
+ git add app/openeo_client.py tests/test_openeo_client.py
281
+ git commit -m "feat: add MNDWI and LST graph builders to openEO client
282
+
283
+ Generated with [Claude Code](https://claude.ai/code)
284
+ via [Happy](https://happy.engineering)
285
+
286
+ Co-Authored-By: Claude <noreply@anthropic.com>
287
+ Co-Authored-By: Happy <yesreply@happy.engineering>"
288
+ ```
289
+
290
+ ---
291
+
292
+ ### Task 3: Rewrite Water Indicator (MNDWI via openEO)
293
+
294
+ **Files:**
295
+ - Rewrite: `app/indicators/water.py`
296
+ - Rewrite: `tests/test_indicator_water.py`
297
+
298
+ - [ ] **Step 1: Write tests for openEO-based water indicator**
299
+
300
+ Replace `tests/test_indicator_water.py` entirely:
301
+
302
+ ```python
303
+ """Tests for app.indicators.water — pixel-level MNDWI via openEO."""
304
+ from __future__ import annotations
305
+
306
+ import os
307
+ import tempfile
308
+ from unittest.mock import MagicMock, patch
309
+ from datetime import date
310
+
311
+ import numpy as np
312
+ import rasterio
313
+ from rasterio.transform import from_bounds
314
+ import pytest
315
+
316
+ from app.models import AOI, TimeRange, StatusLevel, TrendDirection, ConfidenceLevel
317
+
318
+ BBOX = [32.45, 15.65, 32.65, 15.8]
319
+
320
+
321
+ @pytest.fixture
322
+ def test_aoi():
323
+ return AOI(name="Test", bbox=BBOX)
324
+
325
+
326
+ @pytest.fixture
327
+ def test_time_range():
328
+ return TimeRange(start=date(2025, 3, 1), end=date(2026, 3, 1))
329
+
330
+
331
+ def _mock_mndwi_tif(path: str, n_months: int = 12, water_fraction: float = 0.15):
332
+ """Create synthetic MNDWI GeoTIFF. Values > 0 are water."""
333
+ rng = np.random.default_rng(44)
334
+ data = np.zeros((n_months, 10, 10), dtype=np.float32)
335
+ for m in range(n_months):
336
+ vals = rng.normal(-0.2, 0.3, (10, 10))
337
+ # Set some pixels as water (positive MNDWI)
338
+ water_mask = rng.random((10, 10)) < water_fraction
339
+ vals[water_mask] = rng.uniform(0.1, 0.6, water_mask.sum())
340
+ data[m] = vals
341
+ with rasterio.open(
342
+ path, "w", driver="GTiff", height=10, width=10, count=n_months,
343
+ dtype="float32", crs="EPSG:4326",
344
+ transform=from_bounds(*BBOX, 10, 10), nodata=-9999.0,
345
+ ) as dst:
346
+ for i in range(n_months):
347
+ dst.write(data[i], i + 1)
348
+
349
+
350
+ def _mock_rgb_tif(path: str):
351
+ rng = np.random.default_rng(43)
352
+ data = rng.integers(500, 1500, (3, 10, 10), dtype=np.uint16)
353
+ with rasterio.open(
354
+ path, "w", driver="GTiff", height=10, width=10, count=3,
355
+ dtype="uint16", crs="EPSG:4326",
356
+ transform=from_bounds(*BBOX, 10, 10), nodata=0,
357
+ ) as dst:
358
+ for i in range(3):
359
+ dst.write(data[i], i + 1)
360
+
361
+
362
+ @pytest.mark.asyncio
363
+ async def test_water_process_returns_result(test_aoi, test_time_range):
364
+ """WaterIndicator.process() returns a valid IndicatorResult."""
365
+ from app.indicators.water import WaterIndicator
366
+
367
+ indicator = WaterIndicator()
368
+
369
+ with tempfile.TemporaryDirectory() as tmpdir:
370
+ mndwi_path = os.path.join(tmpdir, "mndwi.tif")
371
+ rgb_path = os.path.join(tmpdir, "rgb.tif")
372
+ _mock_mndwi_tif(mndwi_path)
373
+ _mock_rgb_tif(rgb_path)
374
+
375
+ mock_cube = MagicMock()
376
+
377
+ def fake_download(path, **kwargs):
378
+ import shutil
379
+ if "mndwi" in path or "water" in path:
380
+ shutil.copy(mndwi_path, path)
381
+ else:
382
+ shutil.copy(rgb_path, path)
383
+
384
+ mock_cube.download = MagicMock(side_effect=fake_download)
385
+
386
+ with patch("app.indicators.water.get_connection"), \
387
+ patch("app.indicators.water.build_mndwi_graph", return_value=mock_cube), \
388
+ patch("app.indicators.water.build_true_color_graph", return_value=mock_cube):
389
+ result = await indicator.process(test_aoi, test_time_range)
390
+
391
+ assert result.indicator_id == "water"
392
+ assert result.status in (StatusLevel.GREEN, StatusLevel.AMBER, StatusLevel.RED)
393
+ assert "MNDWI" in result.methodology or "water" in result.methodology.lower()
394
+ assert result.data_source == "satellite"
395
+ assert len(result.chart_data.get("dates", [])) > 0
396
+
397
+
398
+ @pytest.mark.asyncio
399
+ async def test_water_falls_back_on_failure(test_aoi, test_time_range):
400
+ """WaterIndicator falls back gracefully when openEO fails."""
401
+ from app.indicators.water import WaterIndicator
402
+
403
+ indicator = WaterIndicator()
404
+
405
+ with patch("app.indicators.water.get_connection", side_effect=Exception("CDSE down")):
406
+ result = await indicator.process(test_aoi, test_time_range)
407
+
408
+ assert result.indicator_id == "water"
409
+ assert result.data_source == "placeholder"
410
+
411
+
412
+ def test_water_compute_stats():
413
+ """_compute_stats() extracts water fraction from MNDWI raster."""
414
+ from app.indicators.water import WaterIndicator
415
+
416
+ with tempfile.TemporaryDirectory() as tmpdir:
417
+ path = os.path.join(tmpdir, "mndwi.tif")
418
+ _mock_mndwi_tif(path, n_months=12, water_fraction=0.2)
419
+ stats = WaterIndicator._compute_stats(path)
420
+
421
+ assert "monthly_water_fractions" in stats
422
+ assert len(stats["monthly_water_fractions"]) == 12
423
+ assert "overall_water_fraction" in stats
424
+ assert 0 < stats["overall_water_fraction"] < 1
425
+ assert "valid_months" in stats
426
+ ```
427
+
428
+ - [ ] **Step 2: Run tests to verify they fail**
429
+
430
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/test_indicator_water.py -v`
431
+
432
+ Expected: FAIL — the old WaterIndicator doesn't have `_compute_stats` or the new import pattern.
433
+
434
+ - [ ] **Step 3: Rewrite water indicator**
435
+
436
+ Replace `app/indicators/water.py` entirely:
437
+
438
+ ```python
439
+ """Water Bodies Indicator — pixel-level MNDWI via CDSE openEO.
440
+
441
+ Computes monthly MNDWI composites from Sentinel-2 L2A, classifies water
442
+ pixels (MNDWI > 0), and tracks water extent change against a 3-year baseline.
443
+ """
444
+ from __future__ import annotations
445
+
446
+ import logging
447
+ import os
448
+ import tempfile
449
+ from datetime import date
450
+ from typing import Any
451
+
452
+ import numpy as np
453
+ import rasterio
454
+
455
+ from app.config import RESOLUTION_M
456
+ from app.indicators.base import BaseIndicator, SpatialData
457
+ from app.models import (
458
+ AOI,
459
+ TimeRange,
460
+ IndicatorResult,
461
+ StatusLevel,
462
+ TrendDirection,
463
+ ConfidenceLevel,
464
+ )
465
+ from app.openeo_client import get_connection, build_mndwi_graph, build_true_color_graph, _bbox_dict
466
+
467
+ logger = logging.getLogger(__name__)
468
+
469
+ BASELINE_YEARS = 3
470
+ WATER_THRESHOLD = 0.0 # MNDWI > 0 = water
471
+
472
+
473
+ class WaterIndicator(BaseIndicator):
474
+ id = "water"
475
+ name = "Water Bodies"
476
+ category = "D9"
477
+ question = "Are rivers and lakes stable?"
478
+ estimated_minutes = 8
479
+
480
+ _true_color_path: str | None = None
481
+
482
+ async def process(
483
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None = None
484
+ ) -> IndicatorResult:
485
+ try:
486
+ return await self._process_openeo(aoi, time_range, season_months)
487
+ except Exception as exc:
488
+ logger.warning("Water openEO processing failed, using placeholder: %s", exc)
489
+ return self._fallback(aoi, time_range)
490
+
491
+ async def _process_openeo(
492
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None
493
+ ) -> IndicatorResult:
494
+ import asyncio
495
+
496
+ conn = get_connection()
497
+ bbox = _bbox_dict(aoi.bbox)
498
+
499
+ current_start = time_range.start.isoformat()
500
+ current_end = time_range.end.isoformat()
501
+ baseline_start = date(
502
+ time_range.start.year - BASELINE_YEARS,
503
+ time_range.start.month,
504
+ time_range.start.day,
505
+ ).isoformat()
506
+ baseline_end = time_range.start.isoformat()
507
+
508
+ results_dir = tempfile.mkdtemp(prefix="aperture_water_")
509
+
510
+ current_cube = build_mndwi_graph(
511
+ conn=conn, bbox=bbox,
512
+ temporal_extent=[current_start, current_end],
513
+ resolution_m=RESOLUTION_M,
514
+ )
515
+ baseline_cube = build_mndwi_graph(
516
+ conn=conn, bbox=bbox,
517
+ temporal_extent=[baseline_start, baseline_end],
518
+ resolution_m=RESOLUTION_M,
519
+ )
520
+ true_color_cube = build_true_color_graph(
521
+ conn=conn, bbox=bbox,
522
+ temporal_extent=[current_start, current_end],
523
+ resolution_m=RESOLUTION_M,
524
+ )
525
+
526
+ loop = asyncio.get_event_loop()
527
+ current_path = os.path.join(results_dir, "mndwi_current.tif")
528
+ baseline_path = os.path.join(results_dir, "mndwi_baseline.tif")
529
+ true_color_path = os.path.join(results_dir, "true_color.tif")
530
+
531
+ await loop.run_in_executor(None, current_cube.download, current_path)
532
+ await loop.run_in_executor(None, baseline_cube.download, baseline_path)
533
+ await loop.run_in_executor(None, true_color_cube.download, true_color_path)
534
+
535
+ self._true_color_path = true_color_path
536
+
537
+ current_stats = self._compute_stats(current_path)
538
+ baseline_stats = self._compute_stats(baseline_path)
539
+
540
+ current_frac = current_stats["overall_water_fraction"]
541
+ baseline_frac = baseline_stats["overall_water_fraction"]
542
+ change_pp = (current_frac - baseline_frac) * 100 # percentage points
543
+
544
+ status = self._classify(abs(change_pp))
545
+ trend = self._compute_trend(change_pp)
546
+ confidence = (
547
+ ConfidenceLevel.HIGH if current_stats["valid_months"] >= 6
548
+ else ConfidenceLevel.MODERATE if current_stats["valid_months"] >= 3
549
+ else ConfidenceLevel.LOW
550
+ )
551
+
552
+ chart_data = self._build_chart_data(
553
+ current_stats["monthly_water_fractions"],
554
+ baseline_stats["monthly_water_fractions"],
555
+ time_range,
556
+ )
557
+
558
+ direction = "increase" if change_pp > 0 else "decrease"
559
+ if abs(change_pp) <= 5:
560
+ headline = f"Water extent stable ({current_frac*100:.1f}%, \u0394{change_pp:+.1f}pp)"
561
+ else:
562
+ headline = f"Water extent {direction} ({change_pp:+.1f}pp vs baseline)"
563
+
564
+ self._spatial_data = SpatialData(
565
+ map_type="raster",
566
+ label="MNDWI",
567
+ colormap="Blues",
568
+ vmin=-0.5,
569
+ vmax=0.5,
570
+ )
571
+ self._indicator_raster_path = current_path
572
+ self._true_color_path = true_color_path
573
+ self._render_band = current_stats["peak_water_band"]
574
+
575
+ return IndicatorResult(
576
+ indicator_id=self.id,
577
+ headline=headline,
578
+ status=status,
579
+ trend=trend,
580
+ confidence=confidence,
581
+ map_layer_path=current_path,
582
+ chart_data=chart_data,
583
+ data_source="satellite",
584
+ summary=(
585
+ f"Water covers {current_frac*100:.1f}% of the AOI compared to "
586
+ f"{baseline_frac*100:.1f}% baseline ({change_pp:+.1f}pp). "
587
+ f"Pixel-level MNDWI analysis at {RESOLUTION_M}m resolution."
588
+ ),
589
+ methodology=(
590
+ f"Sentinel-2 L2A pixel-level MNDWI = (B03 \u2212 B11) / (B03 + B11). "
591
+ f"Cloud-masked using SCL band. Water classified as MNDWI > {WATER_THRESHOLD}. "
592
+ f"Monthly median composites at {RESOLUTION_M}m. "
593
+ f"Baseline: {BASELINE_YEARS}-year water extent frequency. "
594
+ f"Processed via CDSE openEO."
595
+ ),
596
+ limitations=[
597
+ f"Resampled to {RESOLUTION_M}m \u2014 small water bodies may be missed.",
598
+ "Cloud/shadow contamination can cause false water detections.",
599
+ "Seasonal flooding may appear as change if analysis windows differ.",
600
+ "MNDWI threshold is fixed; turbid water may be misclassified.",
601
+ ],
602
+ )
603
+
604
+ @staticmethod
605
+ def _compute_stats(tif_path: str) -> dict[str, Any]:
606
+ """Extract monthly water fraction statistics from MNDWI GeoTIFF."""
607
+ with rasterio.open(tif_path) as src:
608
+ n_bands = src.count
609
+ monthly_fractions = []
610
+ peak_frac = -1.0
611
+ peak_band = 1
612
+ for band in range(1, n_bands + 1):
613
+ data = src.read(band).astype(np.float32)
614
+ nodata = src.nodata
615
+ if nodata is not None:
616
+ valid = data[data != nodata]
617
+ else:
618
+ valid = data.ravel()
619
+ if len(valid) > 0:
620
+ water_pixels = np.sum(valid > WATER_THRESHOLD)
621
+ frac = float(water_pixels / len(valid))
622
+ monthly_fractions.append(frac)
623
+ if frac > peak_frac:
624
+ peak_frac = frac
625
+ peak_band = band
626
+ else:
627
+ monthly_fractions.append(0.0)
628
+
629
+ valid_months = sum(1 for f in monthly_fractions if f > 0)
630
+ overall = float(np.mean(monthly_fractions)) if monthly_fractions else 0.0
631
+
632
+ return {
633
+ "monthly_water_fractions": monthly_fractions,
634
+ "overall_water_fraction": overall,
635
+ "valid_months": max(valid_months, len(monthly_fractions)),
636
+ "peak_water_band": peak_band,
637
+ }
638
+
639
+ @staticmethod
640
+ def _classify(change_pp: float) -> StatusLevel:
641
+ if change_pp <= 10:
642
+ return StatusLevel.GREEN
643
+ if change_pp <= 25:
644
+ return StatusLevel.AMBER
645
+ return StatusLevel.RED
646
+
647
+ @staticmethod
648
+ def _compute_trend(change_pp: float) -> TrendDirection:
649
+ if abs(change_pp) <= 5:
650
+ return TrendDirection.STABLE
651
+ if change_pp > 0:
652
+ return TrendDirection.DETERIORATING # flooding
653
+ return TrendDirection.DETERIORATING # drought
654
+
655
+ @staticmethod
656
+ def _build_chart_data(
657
+ current_monthly: list[float],
658
+ baseline_monthly: list[float],
659
+ time_range: TimeRange,
660
+ ) -> dict[str, Any]:
661
+ year = time_range.end.year
662
+ n = min(len(current_monthly), len(baseline_monthly))
663
+ dates = [f"{year}-{m + 1:02d}" for m in range(n)]
664
+ values = [round(v * 100, 1) for v in current_monthly[:n]]
665
+ b_mean = [round(v * 100, 1) for v in baseline_monthly[:n]]
666
+ b_min = [round(max(v * 100 - 5, 0), 1) for v in baseline_monthly[:n]]
667
+ b_max = [round(min(v * 100 + 5, 100), 1) for v in baseline_monthly[:n]]
668
+
669
+ return {
670
+ "dates": dates,
671
+ "values": values,
672
+ "baseline_mean": b_mean,
673
+ "baseline_min": b_min,
674
+ "baseline_max": b_max,
675
+ "label": "Water extent (%)",
676
+ }
677
+
678
+ def _fallback(self, aoi: AOI, time_range: TimeRange) -> IndicatorResult:
679
+ rng = np.random.default_rng(9)
680
+ baseline = float(rng.uniform(5, 20))
681
+ current = baseline * float(rng.uniform(0.85, 1.15))
682
+ change = current - baseline
683
+
684
+ return IndicatorResult(
685
+ indicator_id=self.id,
686
+ headline=f"Water data degraded ({current:.1f}% extent)",
687
+ status=StatusLevel.GREEN if abs(change) < 5 else StatusLevel.AMBER,
688
+ trend=TrendDirection.STABLE,
689
+ confidence=ConfidenceLevel.LOW,
690
+ map_layer_path="",
691
+ chart_data={
692
+ "dates": [str(time_range.start.year), str(time_range.end.year)],
693
+ "values": [round(baseline, 1), round(current, 1)],
694
+ "label": "Water extent (%)",
695
+ },
696
+ data_source="placeholder",
697
+ summary="openEO processing unavailable. Showing placeholder values.",
698
+ methodology="Placeholder \u2014 no satellite data processed.",
699
+ limitations=["Data is synthetic. openEO backend was unreachable."],
700
+ )
701
+ ```
702
+
703
+ - [ ] **Step 4: Run tests**
704
+
705
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/test_indicator_water.py -v`
706
+
707
+ Expected: All PASS.
708
+
709
+ - [ ] **Step 5: Run full suite**
710
+
711
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/ --timeout=120 2>&1 | tail -5`
712
+
713
+ Expected: All tests PASS.
714
+
715
+ - [ ] **Step 6: Commit**
716
+
717
+ ```bash
718
+ git add app/indicators/water.py tests/test_indicator_water.py
719
+ git commit -m "feat: rewrite water indicator with pixel-level MNDWI via openEO
720
+
721
+ Generated with [Claude Code](https://claude.ai/code)
722
+ via [Happy](https://happy.engineering)
723
+
724
+ Co-Authored-By: Claude <noreply@anthropic.com>
725
+ Co-Authored-By: Happy <yesreply@happy.engineering>"
726
+ ```
727
+
728
+ ---
729
+
730
+ ### Task 4: Rewrite LST Indicator (Sentinel-3 SLSTR via openEO)
731
+
732
+ **Files:**
733
+ - Rewrite: `app/indicators/lst.py`
734
+ - Rewrite: `tests/test_indicator_lst.py`
735
+
736
+ - [ ] **Step 1: Write tests for openEO-based LST indicator**
737
+
738
+ Replace `tests/test_indicator_lst.py` entirely:
739
+
740
+ ```python
741
+ """Tests for app.indicators.lst — Sentinel-3 SLSTR LST via openEO."""
742
+ from __future__ import annotations
743
+
744
+ import os
745
+ import tempfile
746
+ from unittest.mock import MagicMock, patch
747
+ from datetime import date
748
+
749
+ import numpy as np
750
+ import rasterio
751
+ from rasterio.transform import from_bounds
752
+ import pytest
753
+
754
+ from app.models import AOI, TimeRange, StatusLevel, ConfidenceLevel
755
+
756
+ BBOX = [32.45, 15.65, 32.65, 15.8]
757
+
758
+
759
+ @pytest.fixture
760
+ def test_aoi():
761
+ return AOI(name="Test", bbox=BBOX)
762
+
763
+
764
+ @pytest.fixture
765
+ def test_time_range():
766
+ return TimeRange(start=date(2025, 3, 1), end=date(2026, 3, 1))
767
+
768
+
769
+ def _mock_lst_tif(path: str, n_months: int = 12, mean_k: float = 310.0):
770
+ """Create synthetic LST GeoTIFF in Kelvin."""
771
+ rng = np.random.default_rng(45)
772
+ data = np.zeros((n_months, 10, 10), dtype=np.float32)
773
+ for m in range(n_months):
774
+ seasonal = 5.0 * np.sin(np.pi * (m - 1) / 6)
775
+ data[m] = mean_k + seasonal + rng.normal(0, 2, (10, 10))
776
+ with rasterio.open(
777
+ path, "w", driver="GTiff", height=10, width=10, count=n_months,
778
+ dtype="float32", crs="EPSG:4326",
779
+ transform=from_bounds(*BBOX, 10, 10), nodata=-9999.0,
780
+ ) as dst:
781
+ for i in range(n_months):
782
+ dst.write(data[i], i + 1)
783
+
784
+
785
+ def _mock_rgb_tif(path: str):
786
+ rng = np.random.default_rng(43)
787
+ data = rng.integers(500, 1500, (3, 10, 10), dtype=np.uint16)
788
+ with rasterio.open(
789
+ path, "w", driver="GTiff", height=10, width=10, count=3,
790
+ dtype="uint16", crs="EPSG:4326",
791
+ transform=from_bounds(*BBOX, 10, 10), nodata=0,
792
+ ) as dst:
793
+ for i in range(3):
794
+ dst.write(data[i], i + 1)
795
+
796
+
797
+ @pytest.mark.asyncio
798
+ async def test_lst_process_returns_result(test_aoi, test_time_range):
799
+ from app.indicators.lst import LSTIndicator
800
+
801
+ indicator = LSTIndicator()
802
+
803
+ with tempfile.TemporaryDirectory() as tmpdir:
804
+ lst_path = os.path.join(tmpdir, "lst.tif")
805
+ rgb_path = os.path.join(tmpdir, "rgb.tif")
806
+ _mock_lst_tif(lst_path)
807
+ _mock_lst_tif(os.path.join(tmpdir, "lst_baseline.tif"), mean_k=308.0)
808
+ _mock_rgb_tif(rgb_path)
809
+
810
+ mock_cube = MagicMock()
811
+
812
+ def fake_download(path, **kwargs):
813
+ import shutil
814
+ if "lst" in path and "baseline" not in path:
815
+ shutil.copy(lst_path, path)
816
+ elif "lst" in path:
817
+ shutil.copy(os.path.join(tmpdir, "lst_baseline.tif"), path)
818
+ else:
819
+ shutil.copy(rgb_path, path)
820
+
821
+ mock_cube.download = MagicMock(side_effect=fake_download)
822
+
823
+ with patch("app.indicators.lst.get_connection"), \
824
+ patch("app.indicators.lst.build_lst_graph", return_value=mock_cube), \
825
+ patch("app.indicators.lst.build_true_color_graph", return_value=mock_cube):
826
+ result = await indicator.process(test_aoi, test_time_range)
827
+
828
+ assert result.indicator_id == "lst"
829
+ assert result.data_source == "satellite"
830
+ assert len(result.chart_data.get("dates", [])) > 0
831
+
832
+
833
+ @pytest.mark.asyncio
834
+ async def test_lst_falls_back_on_failure(test_aoi, test_time_range):
835
+ from app.indicators.lst import LSTIndicator
836
+ indicator = LSTIndicator()
837
+
838
+ with patch("app.indicators.lst.get_connection", side_effect=Exception("CDSE down")):
839
+ result = await indicator.process(test_aoi, test_time_range)
840
+
841
+ assert result.indicator_id == "lst"
842
+ assert result.data_source == "placeholder"
843
+
844
+
845
+ def test_lst_compute_stats():
846
+ from app.indicators.lst import LSTIndicator
847
+
848
+ with tempfile.TemporaryDirectory() as tmpdir:
849
+ path = os.path.join(tmpdir, "lst.tif")
850
+ _mock_lst_tif(path, mean_k=310.0)
851
+ stats = LSTIndicator._compute_stats(path)
852
+
853
+ assert "monthly_means_celsius" in stats
854
+ assert len(stats["monthly_means_celsius"]) == 12
855
+ assert "overall_mean_celsius" in stats
856
+ assert 30 < stats["overall_mean_celsius"] < 45 # ~310K = ~37°C
857
+ ```
858
+
859
+ - [ ] **Step 2: Run tests to verify they fail**
860
+
861
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/test_indicator_lst.py -v`
862
+
863
+ Expected: FAIL — old LSTIndicator doesn't match new interface.
864
+
865
+ - [ ] **Step 3: Rewrite LST indicator**
866
+
867
+ Replace `app/indicators/lst.py` entirely:
868
+
869
+ ```python
870
+ """Land Surface Temperature Indicator — Sentinel-3 SLSTR via CDSE openEO.
871
+
872
+ Retrieves monthly mean LST from Sentinel-3 SLSTR, compares to 5-year
873
+ baseline, and classifies using Z-score anomaly thresholds.
874
+ """
875
+ from __future__ import annotations
876
+
877
+ import logging
878
+ import os
879
+ import tempfile
880
+ from datetime import date
881
+ from typing import Any
882
+
883
+ import numpy as np
884
+ import rasterio
885
+
886
+ from app.config import RESOLUTION_M
887
+ from app.indicators.base import BaseIndicator, SpatialData
888
+ from app.models import (
889
+ AOI,
890
+ TimeRange,
891
+ IndicatorResult,
892
+ StatusLevel,
893
+ TrendDirection,
894
+ ConfidenceLevel,
895
+ )
896
+ from app.openeo_client import get_connection, build_lst_graph, build_true_color_graph, _bbox_dict
897
+
898
+ logger = logging.getLogger(__name__)
899
+
900
+ BASELINE_YEARS = 5
901
+
902
+
903
+ class LSTIndicator(BaseIndicator):
904
+ id = "lst"
905
+ name = "Land Surface Temperature"
906
+ category = "D6"
907
+ question = "Unusual heat patterns?"
908
+ estimated_minutes = 8
909
+
910
+ _true_color_path: str | None = None
911
+
912
+ async def process(
913
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None = None
914
+ ) -> IndicatorResult:
915
+ try:
916
+ return await self._process_openeo(aoi, time_range, season_months)
917
+ except Exception as exc:
918
+ logger.warning("LST openEO processing failed, using placeholder: %s", exc)
919
+ return self._fallback(aoi, time_range)
920
+
921
+ async def _process_openeo(
922
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None
923
+ ) -> IndicatorResult:
924
+ import asyncio
925
+
926
+ conn = get_connection()
927
+ bbox = _bbox_dict(aoi.bbox)
928
+
929
+ current_start = time_range.start.isoformat()
930
+ current_end = time_range.end.isoformat()
931
+ baseline_start = date(
932
+ time_range.start.year - BASELINE_YEARS,
933
+ time_range.start.month,
934
+ time_range.start.day,
935
+ ).isoformat()
936
+ baseline_end = time_range.start.isoformat()
937
+
938
+ results_dir = tempfile.mkdtemp(prefix="aperture_lst_")
939
+
940
+ # LST at 1km (SLSTR native)
941
+ lst_resolution = max(RESOLUTION_M, 1000)
942
+
943
+ current_cube = build_lst_graph(
944
+ conn=conn, bbox=bbox,
945
+ temporal_extent=[current_start, current_end],
946
+ resolution_m=lst_resolution,
947
+ )
948
+ baseline_cube = build_lst_graph(
949
+ conn=conn, bbox=bbox,
950
+ temporal_extent=[baseline_start, baseline_end],
951
+ resolution_m=lst_resolution,
952
+ )
953
+ true_color_cube = build_true_color_graph(
954
+ conn=conn, bbox=bbox,
955
+ temporal_extent=[current_start, current_end],
956
+ resolution_m=RESOLUTION_M,
957
+ )
958
+
959
+ loop = asyncio.get_event_loop()
960
+ current_path = os.path.join(results_dir, "lst_current.tif")
961
+ baseline_path = os.path.join(results_dir, "lst_baseline.tif")
962
+ true_color_path = os.path.join(results_dir, "true_color.tif")
963
+
964
+ await loop.run_in_executor(None, current_cube.download, current_path)
965
+ await loop.run_in_executor(None, baseline_cube.download, baseline_path)
966
+ await loop.run_in_executor(None, true_color_cube.download, true_color_path)
967
+
968
+ self._true_color_path = true_color_path
969
+
970
+ current_stats = self._compute_stats(current_path)
971
+ baseline_stats = self._compute_stats(baseline_path)
972
+
973
+ current_temp = current_stats["overall_mean_celsius"]
974
+ baseline_temp = baseline_stats["overall_mean_celsius"]
975
+ baseline_std = float(np.std(baseline_stats["monthly_means_celsius"])) if baseline_stats["monthly_means_celsius"] else 1.0
976
+ z_score = (current_temp - baseline_temp) / max(baseline_std, 0.1)
977
+
978
+ status = self._classify(abs(z_score))
979
+ trend = self._compute_trend(z_score)
980
+ confidence = (
981
+ ConfidenceLevel.HIGH if current_stats["valid_months"] >= 6
982
+ else ConfidenceLevel.MODERATE if current_stats["valid_months"] >= 3
983
+ else ConfidenceLevel.LOW
984
+ )
985
+
986
+ chart_data = self._build_chart_data(
987
+ current_stats["monthly_means_celsius"],
988
+ baseline_stats["monthly_means_celsius"],
989
+ time_range,
990
+ )
991
+
992
+ anomaly = current_temp - baseline_temp
993
+ if abs(z_score) < 1.0:
994
+ headline = f"Temperature normal ({current_temp:.1f}\u00b0C, z={z_score:+.1f})"
995
+ elif z_score > 0:
996
+ headline = f"Above-normal temperature ({current_temp:.1f}\u00b0C, +{anomaly:.1f}\u00b0C)"
997
+ else:
998
+ headline = f"Below-normal temperature ({current_temp:.1f}\u00b0C, {anomaly:.1f}\u00b0C)"
999
+
1000
+ self._spatial_data = SpatialData(
1001
+ map_type="raster",
1002
+ label="LST (\u00b0C)",
1003
+ colormap="coolwarm",
1004
+ vmin=current_temp - 10,
1005
+ vmax=current_temp + 10,
1006
+ )
1007
+ self._indicator_raster_path = current_path
1008
+ self._true_color_path = true_color_path
1009
+ self._render_band = current_stats.get("hottest_band", 1)
1010
+
1011
+ return IndicatorResult(
1012
+ indicator_id=self.id,
1013
+ headline=headline,
1014
+ status=status,
1015
+ trend=trend,
1016
+ confidence=confidence,
1017
+ map_layer_path=current_path,
1018
+ chart_data=chart_data,
1019
+ data_source="satellite",
1020
+ summary=(
1021
+ f"Mean LST is {current_temp:.1f}\u00b0C compared to "
1022
+ f"{baseline_temp:.1f}\u00b0C baseline (z-score: {z_score:+.2f}). "
1023
+ f"Sentinel-3 SLSTR at 1km resolution."
1024
+ ),
1025
+ methodology=(
1026
+ f"Sentinel-3 SLSTR Level-2 LST product. "
1027
+ f"Monthly mean composites. "
1028
+ f"Baseline: {BASELINE_YEARS}-year monthly means. "
1029
+ f"Z-score anomaly classification. "
1030
+ f"Processed via CDSE openEO."
1031
+ ),
1032
+ limitations=[
1033
+ "Sentinel-3 SLSTR resolution is ~1km \u2014 urban heat islands may be smoothed.",
1034
+ "Cloud cover creates data gaps in monthly composites.",
1035
+ "LST varies with land cover; change may reflect land use, not climate.",
1036
+ "Daytime overpass only \u2014 nighttime temperatures not captured.",
1037
+ ],
1038
+ )
1039
+
1040
+ @staticmethod
1041
+ def _compute_stats(tif_path: str) -> dict[str, Any]:
1042
+ """Extract monthly LST statistics, converting Kelvin to Celsius."""
1043
+ with rasterio.open(tif_path) as src:
1044
+ n_bands = src.count
1045
+ monthly_means_c = []
1046
+ hottest = -999.0
1047
+ hottest_band = 1
1048
+ for band in range(1, n_bands + 1):
1049
+ data = src.read(band).astype(np.float32)
1050
+ nodata = src.nodata
1051
+ if nodata is not None:
1052
+ valid = data[data != nodata]
1053
+ else:
1054
+ valid = data.ravel()
1055
+ if len(valid) > 0:
1056
+ mean_k = float(np.nanmean(valid))
1057
+ # Convert Kelvin to Celsius if needed (values > 100 assumed Kelvin)
1058
+ mean_c = mean_k - 273.15 if mean_k > 100 else mean_k
1059
+ monthly_means_c.append(mean_c)
1060
+ if mean_c > hottest:
1061
+ hottest = mean_c
1062
+ hottest_band = band
1063
+ else:
1064
+ monthly_means_c.append(0.0)
1065
+
1066
+ valid_months = sum(1 for m in monthly_means_c if m != 0.0)
1067
+ overall = float(np.mean([m for m in monthly_means_c if m != 0.0])) if valid_months > 0 else 0.0
1068
+
1069
+ return {
1070
+ "monthly_means_celsius": monthly_means_c,
1071
+ "overall_mean_celsius": overall,
1072
+ "valid_months": valid_months,
1073
+ "hottest_band": hottest_band,
1074
+ }
1075
+
1076
+ @staticmethod
1077
+ def _classify(abs_z: float) -> StatusLevel:
1078
+ if abs_z < 1.0:
1079
+ return StatusLevel.GREEN
1080
+ if abs_z < 2.0:
1081
+ return StatusLevel.AMBER
1082
+ return StatusLevel.RED
1083
+
1084
+ @staticmethod
1085
+ def _compute_trend(z_score: float) -> TrendDirection:
1086
+ if abs(z_score) < 1.0:
1087
+ return TrendDirection.STABLE
1088
+ if z_score > 0:
1089
+ return TrendDirection.DETERIORATING
1090
+ return TrendDirection.IMPROVING
1091
+
1092
+ @staticmethod
1093
+ def _build_chart_data(
1094
+ current_monthly: list[float],
1095
+ baseline_monthly: list[float],
1096
+ time_range: TimeRange,
1097
+ ) -> dict[str, Any]:
1098
+ year = time_range.end.year
1099
+ n = min(len(current_monthly), len(baseline_monthly))
1100
+ dates = [f"{year}-{m + 1:02d}" for m in range(n)]
1101
+ values = [round(v, 1) for v in current_monthly[:n]]
1102
+ b_mean = [round(v, 1) for v in baseline_monthly[:n]]
1103
+ b_min = [round(v - 3.0, 1) for v in baseline_monthly[:n]]
1104
+ b_max = [round(v + 3.0, 1) for v in baseline_monthly[:n]]
1105
+
1106
+ return {
1107
+ "dates": dates,
1108
+ "values": values,
1109
+ "baseline_mean": b_mean,
1110
+ "baseline_min": b_min,
1111
+ "baseline_max": b_max,
1112
+ "label": "Temperature (\u00b0C)",
1113
+ }
1114
+
1115
+ def _fallback(self, aoi: AOI, time_range: TimeRange) -> IndicatorResult:
1116
+ rng = np.random.default_rng(6)
1117
+ baseline = float(rng.uniform(30, 38))
1118
+ current = baseline + float(rng.uniform(-2, 3))
1119
+ z = (current - baseline) / 2.0
1120
+
1121
+ return IndicatorResult(
1122
+ indicator_id=self.id,
1123
+ headline=f"Temperature data degraded ({current:.1f}\u00b0C)",
1124
+ status=self._classify(abs(z)),
1125
+ trend=self._compute_trend(z),
1126
+ confidence=ConfidenceLevel.LOW,
1127
+ map_layer_path="",
1128
+ chart_data={
1129
+ "dates": [str(time_range.start.year), str(time_range.end.year)],
1130
+ "values": [round(baseline, 1), round(current, 1)],
1131
+ "label": "Temperature (\u00b0C)",
1132
+ },
1133
+ data_source="placeholder",
1134
+ summary="openEO processing unavailable. Showing placeholder values.",
1135
+ methodology="Placeholder \u2014 no satellite data processed.",
1136
+ limitations=["Data is synthetic. openEO backend was unreachable."],
1137
+ )
1138
+ ```
1139
+
1140
+ - [ ] **Step 4: Run tests**
1141
+
1142
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/test_indicator_lst.py -v`
1143
+
1144
+ Expected: All PASS.
1145
+
1146
+ - [ ] **Step 5: Commit**
1147
+
1148
+ ```bash
1149
+ git add app/indicators/lst.py tests/test_indicator_lst.py
1150
+ git commit -m "feat: rewrite LST indicator with Sentinel-3 SLSTR via openEO
1151
+
1152
+ Generated with [Claude Code](https://claude.ai/code)
1153
+ via [Happy](https://happy.engineering)
1154
+
1155
+ Co-Authored-By: Claude <noreply@anthropic.com>
1156
+ Co-Authored-By: Happy <yesreply@happy.engineering>"
1157
+ ```
1158
+
1159
+ ---
1160
+
1161
+ ### Task 5: Rewrite Rainfall Indicator (CHIRPS Direct Download)
1162
+
1163
+ **Files:**
1164
+ - Rewrite: `app/indicators/rainfall.py`
1165
+ - Rewrite: `tests/test_indicator_rainfall.py`
1166
+
1167
+ - [ ] **Step 1: Write tests for CHIRPS-based rainfall indicator**
1168
+
1169
+ Replace `tests/test_indicator_rainfall.py` entirely:
1170
+
1171
+ ```python
1172
+ """Tests for app.indicators.rainfall — CHIRPS precipitation via direct download."""
1173
+ from __future__ import annotations
1174
+
1175
+ import os
1176
+ import tempfile
1177
+ from unittest.mock import MagicMock, patch, AsyncMock
1178
+ from datetime import date
1179
+
1180
+ import numpy as np
1181
+ import rasterio
1182
+ from rasterio.transform import from_bounds
1183
+ import pytest
1184
+
1185
+ from app.models import AOI, TimeRange, StatusLevel, ConfidenceLevel
1186
+
1187
+ BBOX = [32.45, 15.65, 32.65, 15.8]
1188
+
1189
+
1190
+ @pytest.fixture
1191
+ def test_aoi():
1192
+ return AOI(name="Test", bbox=BBOX)
1193
+
1194
+
1195
+ @pytest.fixture
1196
+ def test_time_range():
1197
+ return TimeRange(start=date(2025, 3, 1), end=date(2026, 3, 1))
1198
+
1199
+
1200
+ def _mock_precip_tif(path: str, n_months: int = 12, mean_mm: float = 50.0):
1201
+ """Create synthetic monthly precipitation GeoTIFF in mm."""
1202
+ rng = np.random.default_rng(46)
1203
+ data = np.zeros((n_months, 10, 10), dtype=np.float32)
1204
+ for m in range(n_months):
1205
+ seasonal = mean_mm * (0.5 + 0.8 * np.sin(np.pi * (m - 2) / 6))
1206
+ data[m] = np.maximum(0, seasonal + rng.normal(0, 10, (10, 10)))
1207
+ with rasterio.open(
1208
+ path, "w", driver="GTiff", height=10, width=10, count=n_months,
1209
+ dtype="float32", crs="EPSG:4326",
1210
+ transform=from_bounds(*BBOX, 10, 10), nodata=-9999.0,
1211
+ ) as dst:
1212
+ for i in range(n_months):
1213
+ dst.write(data[i], i + 1)
1214
+
1215
+
1216
+ @pytest.mark.asyncio
1217
+ async def test_rainfall_process_returns_result(test_aoi, test_time_range):
1218
+ from app.indicators.rainfall import RainfallIndicator
1219
+
1220
+ indicator = RainfallIndicator()
1221
+
1222
+ with tempfile.TemporaryDirectory() as tmpdir:
1223
+ current_path = os.path.join(tmpdir, "precip_current.tif")
1224
+ baseline_path = os.path.join(tmpdir, "precip_baseline.tif")
1225
+ _mock_precip_tif(current_path, mean_mm=45.0)
1226
+ _mock_precip_tif(baseline_path, mean_mm=50.0)
1227
+
1228
+ with patch.object(indicator, '_download_chirps', new_callable=AsyncMock) as mock_dl:
1229
+ async def fake_dl(bbox, start, end, output_path):
1230
+ import shutil
1231
+ if "current" in output_path:
1232
+ shutil.copy(current_path, output_path)
1233
+ else:
1234
+ shutil.copy(baseline_path, output_path)
1235
+
1236
+ mock_dl.side_effect = fake_dl
1237
+ result = await indicator.process(test_aoi, test_time_range)
1238
+
1239
+ assert result.indicator_id == "rainfall"
1240
+ assert result.data_source == "satellite"
1241
+ assert "CHIRPS" in result.methodology
1242
+ assert len(result.chart_data.get("dates", [])) > 0
1243
+
1244
+
1245
+ @pytest.mark.asyncio
1246
+ async def test_rainfall_falls_back_on_failure(test_aoi, test_time_range):
1247
+ from app.indicators.rainfall import RainfallIndicator
1248
+ indicator = RainfallIndicator()
1249
+
1250
+ with patch.object(indicator, '_download_chirps', new_callable=AsyncMock, side_effect=Exception("Download failed")):
1251
+ result = await indicator.process(test_aoi, test_time_range)
1252
+
1253
+ assert result.indicator_id == "rainfall"
1254
+ assert result.data_source == "placeholder"
1255
+
1256
+
1257
+ def test_rainfall_compute_stats():
1258
+ from app.indicators.rainfall import RainfallIndicator
1259
+
1260
+ with tempfile.TemporaryDirectory() as tmpdir:
1261
+ path = os.path.join(tmpdir, "precip.tif")
1262
+ _mock_precip_tif(path, mean_mm=50.0)
1263
+ stats = RainfallIndicator._compute_stats(path)
1264
+
1265
+ assert "monthly_means_mm" in stats
1266
+ assert len(stats["monthly_means_mm"]) == 12
1267
+ assert "total_mm" in stats
1268
+ assert stats["total_mm"] > 0
1269
+ ```
1270
+
1271
+ - [ ] **Step 2: Run tests to verify they fail**
1272
+
1273
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/test_indicator_rainfall.py -v`
1274
+
1275
+ Expected: FAIL — old RainfallIndicator interface doesn't match.
1276
+
1277
+ - [ ] **Step 3: Rewrite rainfall indicator**
1278
+
1279
+ Replace `app/indicators/rainfall.py` entirely:
1280
+
1281
+ ```python
1282
+ """Rainfall Indicator — CHIRPS v2.0 precipitation via direct download.
1283
+
1284
+ Downloads monthly CHIRPS GeoTIFFs, computes SPI-like deviation from
1285
+ a 5-year climatological baseline, and classifies drought severity.
1286
+ """
1287
+ from __future__ import annotations
1288
+
1289
+ import logging
1290
+ import os
1291
+ import tempfile
1292
+ from datetime import date
1293
+ from typing import Any
1294
+
1295
+ import numpy as np
1296
+ import rasterio
1297
+ import httpx
1298
+
1299
+ from app.indicators.base import BaseIndicator, SpatialData
1300
+ from app.models import (
1301
+ AOI,
1302
+ TimeRange,
1303
+ IndicatorResult,
1304
+ StatusLevel,
1305
+ TrendDirection,
1306
+ ConfidenceLevel,
1307
+ )
1308
+
1309
+ logger = logging.getLogger(__name__)
1310
+
1311
+ BASELINE_YEARS = 5
1312
+
1313
+ # CHIRPS monthly data URL pattern (public, no auth needed)
1314
+ # Format: https://data.chc.ucsb.edu/products/CHIRPS-2.0/global_monthly/tifs/chirps-v2.0.YYYY.MM.tif.gz
1315
+ CHIRPS_BASE = "https://data.chc.ucsb.edu/products/CHIRPS-2.0/global_monthly/tifs"
1316
+
1317
+
1318
+ class RainfallIndicator(BaseIndicator):
1319
+ id = "rainfall"
1320
+ name = "Rainfall Adequacy"
1321
+ category = "D5"
1322
+ question = "Is this area getting enough rain?"
1323
+ estimated_minutes = 10
1324
+
1325
+ _true_color_path: str | None = None
1326
+
1327
+ async def process(
1328
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None = None
1329
+ ) -> IndicatorResult:
1330
+ try:
1331
+ return await self._process_chirps(aoi, time_range, season_months)
1332
+ except Exception as exc:
1333
+ logger.warning("Rainfall CHIRPS processing failed, using placeholder: %s", exc)
1334
+ return self._fallback(aoi, time_range)
1335
+
1336
+ async def _process_chirps(
1337
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None
1338
+ ) -> IndicatorResult:
1339
+ results_dir = tempfile.mkdtemp(prefix="aperture_rainfall_")
1340
+
1341
+ current_path = os.path.join(results_dir, "precip_current.tif")
1342
+ baseline_path = os.path.join(results_dir, "precip_baseline.tif")
1343
+
1344
+ await self._download_chirps(
1345
+ aoi.bbox, time_range.start, time_range.end, current_path,
1346
+ )
1347
+ baseline_start = date(
1348
+ time_range.start.year - BASELINE_YEARS,
1349
+ time_range.start.month,
1350
+ time_range.start.day,
1351
+ )
1352
+ await self._download_chirps(
1353
+ aoi.bbox, baseline_start, time_range.start, baseline_path,
1354
+ )
1355
+
1356
+ current_stats = self._compute_stats(current_path)
1357
+ baseline_stats = self._compute_stats(baseline_path)
1358
+
1359
+ current_total = current_stats["total_mm"]
1360
+ baseline_total = baseline_stats["total_mm"]
1361
+ deviation_pct = (
1362
+ ((current_total - baseline_total) / baseline_total * 100.0)
1363
+ if baseline_total > 0 else 0.0
1364
+ )
1365
+
1366
+ status = self._classify(deviation_pct)
1367
+ trend = self._compute_trend(deviation_pct)
1368
+ confidence = (
1369
+ ConfidenceLevel.HIGH if current_stats["valid_months"] >= 6
1370
+ else ConfidenceLevel.MODERATE if current_stats["valid_months"] >= 3
1371
+ else ConfidenceLevel.LOW
1372
+ )
1373
+
1374
+ chart_data = self._build_chart_data(
1375
+ current_stats["monthly_means_mm"],
1376
+ baseline_stats["monthly_means_mm"],
1377
+ time_range,
1378
+ )
1379
+
1380
+ if abs(deviation_pct) <= 15:
1381
+ headline = f"Rainfall near normal ({current_total:.0f}mm, {deviation_pct:+.0f}%)"
1382
+ elif deviation_pct < 0:
1383
+ headline = f"Rainfall deficit ({deviation_pct:.0f}% below baseline)"
1384
+ else:
1385
+ headline = f"Above-normal rainfall ({deviation_pct:+.0f}% above baseline)"
1386
+
1387
+ self._spatial_data = SpatialData(
1388
+ map_type="raster",
1389
+ label="Precipitation (mm)",
1390
+ colormap="YlGnBu",
1391
+ vmin=0,
1392
+ vmax=max(current_stats["monthly_means_mm"]) * 1.5 if current_stats["monthly_means_mm"] else 100,
1393
+ )
1394
+ self._indicator_raster_path = current_path
1395
+ self._true_color_path = None # No true-color for rainfall (different resolution)
1396
+ self._render_band = current_stats.get("wettest_band", 1)
1397
+
1398
+ return IndicatorResult(
1399
+ indicator_id=self.id,
1400
+ headline=headline,
1401
+ status=status,
1402
+ trend=trend,
1403
+ confidence=confidence,
1404
+ map_layer_path=current_path,
1405
+ chart_data=chart_data,
1406
+ data_source="satellite",
1407
+ summary=(
1408
+ f"Total precipitation is {current_total:.0f}mm compared to "
1409
+ f"{baseline_total:.0f}mm baseline ({deviation_pct:+.1f}%). "
1410
+ f"CHIRPS v2.0 at ~5km resolution."
1411
+ ),
1412
+ methodology=(
1413
+ f"CHIRPS v2.0 monthly precipitation estimates (0.05\u00b0 resolution). "
1414
+ f"Clipped to AOI bounding box. "
1415
+ f"Baseline: {BASELINE_YEARS}-year monthly climatology. "
1416
+ f"Deviation from baseline classified as drought severity."
1417
+ ),
1418
+ limitations=[
1419
+ "CHIRPS resolution is ~5km \u2014 local rainfall variability not captured.",
1420
+ "Satellite-gauge blend may underestimate in data-sparse regions.",
1421
+ "Orographic effects poorly represented at this resolution.",
1422
+ "No distinction between effective and non-effective rainfall.",
1423
+ ],
1424
+ )
1425
+
1426
+ async def _download_chirps(
1427
+ self,
1428
+ bbox: list[float],
1429
+ start: date,
1430
+ end: date,
1431
+ output_path: str,
1432
+ ) -> None:
1433
+ """Download CHIRPS monthly data and create a multi-band GeoTIFF clipped to AOI.
1434
+
1435
+ Each band is one month of precipitation data (mm).
1436
+ """
1437
+ import asyncio
1438
+ from rasterio.windows import from_bounds as window_from_bounds
1439
+
1440
+ monthly_data = []
1441
+ transform = None
1442
+ height = width = None
1443
+
1444
+ async def _fetch_month(year: int, month: int) -> np.ndarray | None:
1445
+ url = f"{CHIRPS_BASE}/chirps-v2.0.{year}.{month:02d}.tif.gz"
1446
+ try:
1447
+ async with httpx.AsyncClient(timeout=60) as client:
1448
+ resp = await client.get(url)
1449
+ if resp.status_code != 200:
1450
+ logger.warning("CHIRPS download failed for %d-%02d: %d", year, month, resp.status_code)
1451
+ return None
1452
+
1453
+ # Decompress and read the subset
1454
+ import gzip
1455
+ import io
1456
+ decompressed = gzip.decompress(resp.content)
1457
+ with rasterio.open(io.BytesIO(decompressed)) as src:
1458
+ window = window_from_bounds(*bbox, transform=src.transform)
1459
+ data = src.read(1, window=window).astype(np.float32)
1460
+ return data
1461
+ except Exception as exc:
1462
+ logger.warning("CHIRPS fetch error for %d-%02d: %s", year, month, exc)
1463
+ return None
1464
+
1465
+ # Collect monthly data
1466
+ current = date(start.year, start.month, 1)
1467
+ months_collected = []
1468
+ while current < end:
1469
+ data = await _fetch_month(current.year, current.month)
1470
+ if data is not None:
1471
+ monthly_data.append(data)
1472
+ months_collected.append(current)
1473
+ if current.month == 12:
1474
+ current = date(current.year + 1, 1, 1)
1475
+ else:
1476
+ current = date(current.year, current.month + 1, 1)
1477
+
1478
+ if not monthly_data:
1479
+ raise ValueError("No CHIRPS data available for the requested period")
1480
+
1481
+ # Write as multi-band GeoTIFF
1482
+ h, w = monthly_data[0].shape
1483
+ from rasterio.transform import from_bounds as transform_from_bounds
1484
+ t = transform_from_bounds(*bbox, w, h)
1485
+
1486
+ with rasterio.open(
1487
+ output_path, "w", driver="GTiff",
1488
+ height=h, width=w, count=len(monthly_data),
1489
+ dtype="float32", crs="EPSG:4326",
1490
+ transform=t, nodata=-9999.0,
1491
+ ) as dst:
1492
+ for i, data in enumerate(monthly_data):
1493
+ dst.write(data, i + 1)
1494
+
1495
+ @staticmethod
1496
+ def _compute_stats(tif_path: str) -> dict[str, Any]:
1497
+ """Extract monthly precipitation statistics from a multi-band GeoTIFF."""
1498
+ with rasterio.open(tif_path) as src:
1499
+ n_bands = src.count
1500
+ monthly_means = []
1501
+ peak_val = -1.0
1502
+ peak_band = 1
1503
+ for band in range(1, n_bands + 1):
1504
+ data = src.read(band).astype(np.float32)
1505
+ nodata = src.nodata
1506
+ if nodata is not None:
1507
+ valid = data[data != nodata]
1508
+ else:
1509
+ valid = data.ravel()
1510
+ if len(valid) > 0:
1511
+ mean = float(np.nanmean(valid))
1512
+ monthly_means.append(mean)
1513
+ if mean > peak_val:
1514
+ peak_val = mean
1515
+ peak_band = band
1516
+ else:
1517
+ monthly_means.append(0.0)
1518
+
1519
+ valid_months = sum(1 for m in monthly_means if m > 0)
1520
+ total = float(np.sum(monthly_means))
1521
+
1522
+ return {
1523
+ "monthly_means_mm": monthly_means,
1524
+ "total_mm": total,
1525
+ "valid_months": valid_months,
1526
+ "wettest_band": peak_band,
1527
+ }
1528
+
1529
+ @staticmethod
1530
+ def _classify(deviation_pct: float) -> StatusLevel:
1531
+ if deviation_pct >= -15:
1532
+ return StatusLevel.GREEN
1533
+ if deviation_pct >= -30:
1534
+ return StatusLevel.AMBER
1535
+ return StatusLevel.RED
1536
+
1537
+ @staticmethod
1538
+ def _compute_trend(deviation_pct: float) -> TrendDirection:
1539
+ if abs(deviation_pct) <= 15:
1540
+ return TrendDirection.STABLE
1541
+ if deviation_pct < 0:
1542
+ return TrendDirection.DETERIORATING
1543
+ return TrendDirection.IMPROVING
1544
+
1545
+ @staticmethod
1546
+ def _build_chart_data(
1547
+ current_monthly: list[float],
1548
+ baseline_monthly: list[float],
1549
+ time_range: TimeRange,
1550
+ ) -> dict[str, Any]:
1551
+ year = time_range.end.year
1552
+ n = min(len(current_monthly), len(baseline_monthly))
1553
+ dates = [f"{year}-{m + 1:02d}" for m in range(n)]
1554
+ values = [round(v, 1) for v in current_monthly[:n]]
1555
+ b_mean = [round(v, 1) for v in baseline_monthly[:n]]
1556
+ b_min = [round(max(v - 15, 0), 1) for v in baseline_monthly[:n]]
1557
+ b_max = [round(v + 15, 1) for v in baseline_monthly[:n]]
1558
+
1559
+ return {
1560
+ "dates": dates,
1561
+ "values": values,
1562
+ "baseline_mean": b_mean,
1563
+ "baseline_min": b_min,
1564
+ "baseline_max": b_max,
1565
+ "label": "Precipitation (mm)",
1566
+ }
1567
+
1568
+ def _fallback(self, aoi: AOI, time_range: TimeRange) -> IndicatorResult:
1569
+ rng = np.random.default_rng(5)
1570
+ baseline = float(rng.uniform(300, 600))
1571
+ current = baseline * float(rng.uniform(0.7, 1.1))
1572
+ deviation = ((current - baseline) / baseline) * 100
1573
+
1574
+ return IndicatorResult(
1575
+ indicator_id=self.id,
1576
+ headline=f"Rainfall data degraded ({current:.0f}mm)",
1577
+ status=self._classify(deviation),
1578
+ trend=self._compute_trend(deviation),
1579
+ confidence=ConfidenceLevel.LOW,
1580
+ map_layer_path="",
1581
+ chart_data={
1582
+ "dates": [str(time_range.start.year), str(time_range.end.year)],
1583
+ "values": [round(baseline, 0), round(current, 0)],
1584
+ "label": "Precipitation (mm)",
1585
+ },
1586
+ data_source="placeholder",
1587
+ summary="CHIRPS download unavailable. Showing placeholder values.",
1588
+ methodology="Placeholder \u2014 no satellite data processed.",
1589
+ limitations=["Data is synthetic. CHIRPS data was unreachable."],
1590
+ )
1591
+ ```
1592
+
1593
+ - [ ] **Step 4: Run tests**
1594
+
1595
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/test_indicator_rainfall.py -v`
1596
+
1597
+ Expected: All PASS.
1598
+
1599
+ - [ ] **Step 5: Commit**
1600
+
1601
+ ```bash
1602
+ git add app/indicators/rainfall.py tests/test_indicator_rainfall.py
1603
+ git commit -m "feat: rewrite rainfall indicator with CHIRPS direct download
1604
+
1605
+ Generated with [Claude Code](https://claude.ai/code)
1606
+ via [Happy](https://happy.engineering)
1607
+
1608
+ Co-Authored-By: Claude <noreply@anthropic.com>
1609
+ Co-Authored-By: Happy <yesreply@happy.engineering>"
1610
+ ```
1611
+
1612
+ ---
1613
+
1614
+ ### Task 6: Rewrite Nightlights Indicator (VIIRS EOG Direct Download)
1615
+
1616
+ **Files:**
1617
+ - Rewrite: `app/indicators/nightlights.py`
1618
+ - Rewrite: `tests/test_indicator_nightlights.py`
1619
+
1620
+ - [ ] **Step 1: Write tests**
1621
+
1622
+ Replace `tests/test_indicator_nightlights.py` entirely:
1623
+
1624
+ ```python
1625
+ """Tests for app.indicators.nightlights — VIIRS DNB via EOG direct download."""
1626
+ from __future__ import annotations
1627
+
1628
+ import os
1629
+ import tempfile
1630
+ from unittest.mock import MagicMock, patch, AsyncMock
1631
+ from datetime import date
1632
+
1633
+ import numpy as np
1634
+ import rasterio
1635
+ from rasterio.transform import from_bounds
1636
+ import pytest
1637
+
1638
+ from app.models import AOI, TimeRange, StatusLevel, ConfidenceLevel
1639
+
1640
+ BBOX = [32.45, 15.65, 32.65, 15.8]
1641
+
1642
+
1643
+ @pytest.fixture
1644
+ def test_aoi():
1645
+ return AOI(name="Test", bbox=BBOX)
1646
+
1647
+
1648
+ @pytest.fixture
1649
+ def test_time_range():
1650
+ return TimeRange(start=date(2025, 3, 1), end=date(2026, 3, 1))
1651
+
1652
+
1653
+ def _mock_radiance_tif(path: str, mean_nw: float = 5.0):
1654
+ """Create synthetic VIIRS DNB radiance GeoTIFF (nW/cm²/sr)."""
1655
+ rng = np.random.default_rng(47)
1656
+ data = np.maximum(0, mean_nw + rng.normal(0, 2, (10, 10))).astype(np.float32)
1657
+ with rasterio.open(
1658
+ path, "w", driver="GTiff", height=10, width=10, count=1,
1659
+ dtype="float32", crs="EPSG:4326",
1660
+ transform=from_bounds(*BBOX, 10, 10), nodata=-9999.0,
1661
+ ) as dst:
1662
+ dst.write(data, 1)
1663
+
1664
+
1665
+ @pytest.mark.asyncio
1666
+ async def test_nightlights_process_returns_result(test_aoi, test_time_range):
1667
+ from app.indicators.nightlights import NightlightsIndicator
1668
+
1669
+ indicator = NightlightsIndicator()
1670
+
1671
+ with tempfile.TemporaryDirectory() as tmpdir:
1672
+ current_path = os.path.join(tmpdir, "viirs_current.tif")
1673
+ baseline_path = os.path.join(tmpdir, "viirs_baseline.tif")
1674
+ _mock_radiance_tif(current_path, mean_nw=5.0)
1675
+ _mock_radiance_tif(baseline_path, mean_nw=6.0)
1676
+
1677
+ with patch.object(indicator, '_download_viirs', new_callable=AsyncMock) as mock_dl:
1678
+ async def fake_dl(bbox, year, output_path):
1679
+ import shutil
1680
+ if "current" in output_path:
1681
+ shutil.copy(current_path, output_path)
1682
+ else:
1683
+ shutil.copy(baseline_path, output_path)
1684
+
1685
+ mock_dl.side_effect = fake_dl
1686
+ result = await indicator.process(test_aoi, test_time_range)
1687
+
1688
+ assert result.indicator_id == "nightlights"
1689
+ assert result.data_source == "satellite"
1690
+ assert len(result.chart_data.get("dates", [])) > 0
1691
+
1692
+
1693
+ @pytest.mark.asyncio
1694
+ async def test_nightlights_falls_back_on_failure(test_aoi, test_time_range):
1695
+ from app.indicators.nightlights import NightlightsIndicator
1696
+ indicator = NightlightsIndicator()
1697
+
1698
+ with patch.object(indicator, '_download_viirs', new_callable=AsyncMock, side_effect=Exception("Download failed")):
1699
+ result = await indicator.process(test_aoi, test_time_range)
1700
+
1701
+ assert result.indicator_id == "nightlights"
1702
+ assert result.data_source == "placeholder"
1703
+
1704
+
1705
+ def test_nightlights_compute_stats():
1706
+ from app.indicators.nightlights import NightlightsIndicator
1707
+
1708
+ with tempfile.TemporaryDirectory() as tmpdir:
1709
+ path = os.path.join(tmpdir, "viirs.tif")
1710
+ _mock_radiance_tif(path, mean_nw=5.0)
1711
+ stats = NightlightsIndicator._compute_stats(path)
1712
+
1713
+ assert "mean_radiance" in stats
1714
+ assert stats["mean_radiance"] > 0
1715
+ assert "valid_pixel_fraction" in stats
1716
+ ```
1717
+
1718
+ - [ ] **Step 2: Run tests to verify they fail**
1719
+
1720
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/test_indicator_nightlights.py -v`
1721
+
1722
+ Expected: FAIL.
1723
+
1724
+ - [ ] **Step 3: Rewrite nightlights indicator**
1725
+
1726
+ Replace `app/indicators/nightlights.py` entirely:
1727
+
1728
+ ```python
1729
+ """Nighttime Lights Indicator — VIIRS DNB via EOG direct download.
1730
+
1731
+ Downloads annual VIIRS DNB composites from Colorado School of Mines EOG,
1732
+ compares current-year radiance to a 3-year baseline.
1733
+ """
1734
+ from __future__ import annotations
1735
+
1736
+ import logging
1737
+ import os
1738
+ import tempfile
1739
+ from datetime import date
1740
+ from typing import Any
1741
+
1742
+ import numpy as np
1743
+ import rasterio
1744
+ import httpx
1745
+
1746
+ from app.indicators.base import BaseIndicator, SpatialData
1747
+ from app.models import (
1748
+ AOI,
1749
+ TimeRange,
1750
+ IndicatorResult,
1751
+ StatusLevel,
1752
+ TrendDirection,
1753
+ ConfidenceLevel,
1754
+ )
1755
+
1756
+ logger = logging.getLogger(__name__)
1757
+
1758
+ BASELINE_YEARS = 3
1759
+
1760
+ # EOG VIIRS DNB annual composites (public, COG format)
1761
+ EOG_BASE = "https://eogdata.mines.edu/nighttime_light/annual/v22"
1762
+
1763
+
1764
+ class NightlightsIndicator(BaseIndicator):
1765
+ id = "nightlights"
1766
+ name = "Nighttime Lights"
1767
+ category = "D3"
1768
+ question = "Is the local economy active?"
1769
+ estimated_minutes = 10
1770
+
1771
+ _true_color_path: str | None = None
1772
+
1773
+ async def process(
1774
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None = None
1775
+ ) -> IndicatorResult:
1776
+ try:
1777
+ return await self._process_viirs(aoi, time_range)
1778
+ except Exception as exc:
1779
+ logger.warning("Nightlights download failed, using placeholder: %s", exc)
1780
+ return self._fallback(aoi, time_range)
1781
+
1782
+ async def _process_viirs(
1783
+ self, aoi: AOI, time_range: TimeRange
1784
+ ) -> IndicatorResult:
1785
+ results_dir = tempfile.mkdtemp(prefix="aperture_nightlights_")
1786
+
1787
+ current_year = time_range.end.year
1788
+ current_path = os.path.join(results_dir, "viirs_current.tif")
1789
+ baseline_path = os.path.join(results_dir, "viirs_baseline.tif")
1790
+
1791
+ await self._download_viirs(aoi.bbox, current_year, current_path)
1792
+ await self._download_viirs(aoi.bbox, current_year - BASELINE_YEARS, baseline_path)
1793
+
1794
+ current_stats = self._compute_stats(current_path)
1795
+ baseline_stats = self._compute_stats(baseline_path)
1796
+
1797
+ current_rad = current_stats["mean_radiance"]
1798
+ baseline_rad = baseline_stats["mean_radiance"]
1799
+ pct_change = ((current_rad - baseline_rad) / baseline_rad * 100) if baseline_rad > 0 else 0.0
1800
+
1801
+ status = self._classify(pct_change)
1802
+ trend = self._compute_trend(pct_change)
1803
+ confidence = (
1804
+ ConfidenceLevel.HIGH if current_stats["valid_pixel_fraction"] >= 0.7
1805
+ else ConfidenceLevel.MODERATE if current_stats["valid_pixel_fraction"] >= 0.4
1806
+ else ConfidenceLevel.LOW
1807
+ )
1808
+
1809
+ chart_data = {
1810
+ "dates": [str(current_year - BASELINE_YEARS), str(current_year)],
1811
+ "values": [round(baseline_rad, 2), round(current_rad, 2)],
1812
+ "baseline_range_mean": round(baseline_rad, 2),
1813
+ "baseline_range_min": round(baseline_rad * 0.85, 2),
1814
+ "baseline_range_max": round(baseline_rad * 1.15, 2),
1815
+ "label": "Radiance (nW/cm\u00b2/sr)",
1816
+ }
1817
+
1818
+ if abs(pct_change) <= 15:
1819
+ headline = f"Nighttime lights stable ({current_rad:.1f} nW, {pct_change:+.0f}%)"
1820
+ elif pct_change < 0:
1821
+ headline = f"Nighttime lights declining ({pct_change:.0f}%)"
1822
+ else:
1823
+ headline = f"Nighttime lights increasing (+{pct_change:.0f}%)"
1824
+
1825
+ self._spatial_data = SpatialData(
1826
+ map_type="raster",
1827
+ label="Radiance (nW/cm\u00b2/sr)",
1828
+ colormap="inferno",
1829
+ vmin=0,
1830
+ vmax=max(current_rad * 2, 10),
1831
+ )
1832
+ self._indicator_raster_path = current_path
1833
+ self._true_color_path = None
1834
+ self._render_band = 1
1835
+
1836
+ return IndicatorResult(
1837
+ indicator_id=self.id,
1838
+ headline=headline,
1839
+ status=status,
1840
+ trend=trend,
1841
+ confidence=confidence,
1842
+ map_layer_path=current_path,
1843
+ chart_data=chart_data,
1844
+ data_source="satellite",
1845
+ summary=(
1846
+ f"Mean radiance is {current_rad:.2f} nW/cm\u00b2/sr compared to "
1847
+ f"{baseline_rad:.2f} baseline ({pct_change:+.1f}%). "
1848
+ f"VIIRS DNB annual composite at ~500m resolution."
1849
+ ),
1850
+ methodology=(
1851
+ f"VIIRS Day-Night Band annual composites from Colorado School of Mines EOG. "
1852
+ f"Stray-light corrected, cloud-free composite. "
1853
+ f"Clipped to AOI bounding box. "
1854
+ f"Baseline: {BASELINE_YEARS}-year prior annual composite."
1855
+ ),
1856
+ limitations=[
1857
+ "Annual composites \u2014 cannot detect sub-annual changes.",
1858
+ "Moonlight, fires, and gas flaring inflate radiance values.",
1859
+ "~500m resolution smooths urban-rural boundaries.",
1860
+ "Most recent annual composite may lag by several months.",
1861
+ ],
1862
+ )
1863
+
1864
+ async def _download_viirs(
1865
+ self, bbox: list[float], year: int, output_path: str
1866
+ ) -> None:
1867
+ """Download VIIRS DNB annual composite and clip to AOI bbox.
1868
+
1869
+ Uses COG (Cloud-Optimized GeoTIFF) with HTTP range requests
1870
+ to read only the AOI window from the full global file.
1871
+ """
1872
+ # Try reading via rasterio with HTTP range requests (COG)
1873
+ import asyncio
1874
+ from rasterio.windows import from_bounds as window_from_bounds
1875
+
1876
+ loop = asyncio.get_event_loop()
1877
+
1878
+ def _read_cog():
1879
+ # EOG provides global annual composites as COGs
1880
+ url = f"{EOG_BASE}/{year}/VNP46A4_t{year}.average_masked.tif"
1881
+
1882
+ with rasterio.open(url) as src:
1883
+ window = window_from_bounds(*bbox, transform=src.transform)
1884
+ data = src.read(1, window=window).astype(np.float32)
1885
+ win_transform = src.window_transform(window)
1886
+
1887
+ from rasterio.transform import from_bounds
1888
+ h, w = data.shape
1889
+ t = from_bounds(*bbox, w, h)
1890
+
1891
+ with rasterio.open(
1892
+ output_path, "w", driver="GTiff",
1893
+ height=h, width=w, count=1,
1894
+ dtype="float32", crs="EPSG:4326",
1895
+ transform=t, nodata=-9999.0,
1896
+ ) as dst:
1897
+ dst.write(data, 1)
1898
+
1899
+ await loop.run_in_executor(None, _read_cog)
1900
+
1901
+ @staticmethod
1902
+ def _compute_stats(tif_path: str) -> dict[str, Any]:
1903
+ """Extract radiance statistics from VIIRS GeoTIFF."""
1904
+ with rasterio.open(tif_path) as src:
1905
+ data = src.read(1).astype(np.float32)
1906
+ nodata = src.nodata
1907
+ if nodata is not None:
1908
+ valid = data[data != nodata]
1909
+ else:
1910
+ valid = data.ravel()
1911
+
1912
+ valid = valid[valid >= 0] # Remove negative radiance
1913
+ total_pixels = data.size
1914
+ valid_fraction = len(valid) / total_pixels if total_pixels > 0 else 0.0
1915
+
1916
+ return {
1917
+ "mean_radiance": float(np.mean(valid)) if len(valid) > 0 else 0.0,
1918
+ "valid_pixel_fraction": valid_fraction,
1919
+ }
1920
+
1921
+ @staticmethod
1922
+ def _classify(pct_change: float) -> StatusLevel:
1923
+ if pct_change > -15:
1924
+ return StatusLevel.GREEN
1925
+ if pct_change > -40:
1926
+ return StatusLevel.AMBER
1927
+ return StatusLevel.RED
1928
+
1929
+ @staticmethod
1930
+ def _compute_trend(pct_change: float) -> TrendDirection:
1931
+ if abs(pct_change) <= 15:
1932
+ return TrendDirection.STABLE
1933
+ if pct_change < 0:
1934
+ return TrendDirection.DETERIORATING
1935
+ return TrendDirection.IMPROVING
1936
+
1937
+ def _fallback(self, aoi: AOI, time_range: TimeRange) -> IndicatorResult:
1938
+ rng = np.random.default_rng(3)
1939
+ baseline = float(rng.uniform(2, 10))
1940
+ current = baseline * float(rng.uniform(0.7, 1.1))
1941
+ pct = ((current - baseline) / baseline) * 100
1942
+
1943
+ return IndicatorResult(
1944
+ indicator_id=self.id,
1945
+ headline=f"Nightlights data degraded ({current:.1f} nW)",
1946
+ status=self._classify(pct),
1947
+ trend=self._compute_trend(pct),
1948
+ confidence=ConfidenceLevel.LOW,
1949
+ map_layer_path="",
1950
+ chart_data={
1951
+ "dates": [str(time_range.start.year), str(time_range.end.year)],
1952
+ "values": [round(baseline, 2), round(current, 2)],
1953
+ "label": "Radiance (nW/cm\u00b2/sr)",
1954
+ },
1955
+ data_source="placeholder",
1956
+ summary="VIIRS download unavailable. Showing placeholder values.",
1957
+ methodology="Placeholder \u2014 no satellite data processed.",
1958
+ limitations=["Data is synthetic. VIIRS data was unreachable."],
1959
+ )
1960
+ ```
1961
+
1962
+ - [ ] **Step 4: Run tests**
1963
+
1964
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/test_indicator_nightlights.py -v`
1965
+
1966
+ Expected: All PASS.
1967
+
1968
+ - [ ] **Step 5: Commit**
1969
+
1970
+ ```bash
1971
+ git add app/indicators/nightlights.py tests/test_indicator_nightlights.py
1972
+ git commit -m "feat: rewrite nightlights indicator with VIIRS EOG direct download
1973
+
1974
+ Generated with [Claude Code](https://claude.ai/code)
1975
+ via [Happy](https://happy.engineering)
1976
+
1977
+ Co-Authored-By: Claude <noreply@anthropic.com>
1978
+ Co-Authored-By: Happy <yesreply@happy.engineering>"
1979
+ ```
1980
+
1981
+ ---
1982
+
1983
+ ### Task 7: Full Test Suite Verification
1984
+
1985
+ - [ ] **Step 1: Run complete test suite**
1986
+
1987
+ Run: `cd /Users/kmini/Github/Aperture && python -m pytest tests/ -v --timeout=120 2>&1 | tail -30`
1988
+
1989
+ Expected: All tests PASS. Count should be ~136+ (existing tests, some rewritten).
1990
+
1991
+ - [ ] **Step 2: Verify all indicators register**
1992
+
1993
+ Run: `cd /Users/kmini/Github/Aperture && python -c "from app.indicators import registry; ids = registry.list_ids(); print(f'{len(ids)} indicators:', ids)"`
1994
+
1995
+ Expected: `10 indicators: ['ndvi', 'fires', 'cropland', 'vegetation', 'rainfall', 'water', 'no2', 'lst', 'nightlights', 'food_security']`
1996
+
1997
+ - [ ] **Step 3: Verify imports don't cycle**
1998
+
1999
+ Run: `cd /Users/kmini/Github/Aperture && python -c "from app.indicators.water import WaterIndicator; from app.indicators.lst import LSTIndicator; from app.indicators.rainfall import RainfallIndicator; from app.indicators.nightlights import NightlightsIndicator; print('All imports OK')"`
2000
+
2001
+ Expected: `All imports OK`
2002
+
2003
+ - [ ] **Step 4: Check git log**
2004
+
2005
+ Run: `cd /Users/kmini/Github/Aperture && git log --oneline -8`
2006
+
2007
+ Expected: 6-7 new commits from this plan.