KSvend Claude Happy commited on
Commit
c6c1ec2
·
1 Parent(s): 152957b

docs: implementation plan for batch SAR/buildup/water + NDVI fix

Browse files

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

docs/superpowers/plans/2026-04-01-batch-sar-buildup-water.md ADDED
@@ -0,0 +1,1416 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Batch Support for SAR, Buildup, Water Implementation Plan
2
+
3
+ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
+
5
+ **Goal:** Add batch processing (submit_batch + harvest) to SAR, buildup, and water indicators, matching the existing NDVI batch pattern.
6
+
7
+ **Architecture:** Each indicator gets `uses_batch = True`, a `submit_batch()` that submits 3 openEO batch jobs (current, baseline, true-color), and a `harvest()` that downloads results and runs full post-processing. The worker already handles batch indicators — no orchestration changes needed.
8
+
9
+ **Tech Stack:** Python, openEO, rasterio, pytest, CDSE backend
10
+
11
+ **Spec:** `docs/superpowers/specs/2026-04-01-batch-sar-buildup-water-design.md`
12
+
13
+ ---
14
+
15
+ ### Task 0: Fix NDVI Batch Baseline Bug + Add Debug Logging
16
+
17
+ There is a bug in `app/indicators/ndvi.py` `harvest()`: after downloading the baseline job results (line 111), `baseline_path` is never assigned the actual tif path — it stays `None`. This means harvest always runs in degraded mode (no baseline comparison). This task fixes that bug and adds worker-level debug logging.
18
+
19
+ **Files:**
20
+ - Modify: `app/indicators/ndvi.py:106-113`
21
+ - Test: `tests/test_indicator_ndvi.py` (existing tests cover this)
22
+
23
+ - [ ] **Step 1: Write a failing test that exposes the baseline bug**
24
+
25
+ Add to `tests/test_indicator_ndvi.py`:
26
+
27
+ ```python
28
+ @pytest.mark.asyncio
29
+ async def test_ndvi_harvest_uses_baseline_when_available(test_aoi, test_time_range):
30
+ """harvest() computes change vs baseline when baseline download succeeds."""
31
+ from app.indicators.ndvi import NdviIndicator
32
+
33
+ indicator = NdviIndicator()
34
+
35
+ with tempfile.TemporaryDirectory() as tmpdir:
36
+ # Current: higher NDVI
37
+ current_path = os.path.join(tmpdir, "current.tif")
38
+ _mock_ndvi_tif(current_path, n_months=12)
39
+
40
+ # Baseline: lower NDVI (shift down by 0.1)
41
+ baseline_path = os.path.join(tmpdir, "baseline.tif")
42
+ import rasterio
43
+ from rasterio.transform import from_bounds
44
+ rng = np.random.default_rng(99)
45
+ data = np.zeros((12, 10, 10), dtype=np.float32)
46
+ for m in range(12):
47
+ data[m] = 0.2 + 0.1 * np.sin(np.pi * (m - 3) / 6) + rng.normal(0, 0.02, (10, 10))
48
+ with rasterio.open(
49
+ baseline_path, "w", driver="GTiff", height=10, width=10, count=12,
50
+ dtype="float32", crs="EPSG:4326",
51
+ transform=from_bounds(32.45, 15.65, 32.65, 15.8, 10, 10),
52
+ nodata=-9999.0,
53
+ ) as dst:
54
+ for i in range(12):
55
+ dst.write(data[i], i + 1)
56
+
57
+ rgb_path = os.path.join(tmpdir, "rgb.tif")
58
+ _mock_true_color_tif(rgb_path)
59
+
60
+ def make_mock_job(src_path):
61
+ job = MagicMock()
62
+ job.job_id = "j-test"
63
+ def fake_download_results(target):
64
+ import shutil
65
+ os.makedirs(target, exist_ok=True)
66
+ dest = os.path.join(target, "result.tif")
67
+ shutil.copy(src_path, dest)
68
+ from pathlib import Path
69
+ return {Path(dest): {"type": "image/tiff"}}
70
+ job.download_results.side_effect = fake_download_results
71
+ job.status.return_value = "finished"
72
+ return job
73
+
74
+ current_job = make_mock_job(current_path)
75
+ baseline_job = make_mock_job(baseline_path)
76
+ true_color_job = make_mock_job(rgb_path)
77
+
78
+ result = await indicator.harvest(
79
+ test_aoi, test_time_range,
80
+ batch_jobs=[current_job, baseline_job, true_color_job],
81
+ )
82
+
83
+ assert result.data_source == "satellite"
84
+ # With distinct current vs baseline, confidence should NOT be LOW
85
+ # (it should be HIGH with 12 valid months)
86
+ assert result.confidence == ConfidenceLevel.HIGH
87
+ # Baseline band data should be present in chart
88
+ assert "baseline_mean" in result.chart_data
89
+ assert len(result.chart_data["baseline_mean"]) > 0
90
+ ```
91
+
92
+ - [ ] **Step 2: Run the test to verify it fails**
93
+
94
+ Run: `pytest tests/test_indicator_ndvi.py::test_ndvi_harvest_uses_baseline_when_available -v`
95
+
96
+ Expected: FAIL — `result.confidence` is `LOW` because `baseline_path` is always `None`.
97
+
98
+ - [ ] **Step 3: Fix the baseline_path assignment in harvest()**
99
+
100
+ In `app/indicators/ndvi.py`, replace lines 106-113:
101
+
102
+ ```python
103
+ # Download baseline — optional (degrades gracefully)
104
+ baseline_path = None
105
+ try:
106
+ baseline_dir = os.path.join(results_dir, "baseline")
107
+ os.makedirs(baseline_dir, exist_ok=True)
108
+ paths = baseline_job.download_results(baseline_dir)
109
+ except Exception as exc:
110
+ logger.warning("NDVI baseline batch download failed, degrading: %s", exc)
111
+ ```
112
+
113
+ With:
114
+
115
+ ```python
116
+ # Download baseline — optional (degrades gracefully)
117
+ baseline_path = None
118
+ try:
119
+ baseline_dir = os.path.join(results_dir, "baseline")
120
+ os.makedirs(baseline_dir, exist_ok=True)
121
+ paths = baseline_job.download_results(baseline_dir)
122
+ baseline_path = self._find_tif(paths, baseline_dir)
123
+ except Exception as exc:
124
+ logger.warning("NDVI baseline batch download failed, degrading: %s", exc)
125
+ ```
126
+
127
+ - [ ] **Step 4: Run all NDVI tests to verify the fix**
128
+
129
+ Run: `pytest tests/test_indicator_ndvi.py tests/test_ndvi_e2e.py -v`
130
+
131
+ Expected: ALL PASS
132
+
133
+ - [ ] **Step 5: Commit**
134
+
135
+ ```bash
136
+ git add app/indicators/ndvi.py tests/test_indicator_ndvi.py
137
+ git commit -m "fix: NDVI harvest baseline_path never assigned after download"
138
+ ```
139
+
140
+ ---
141
+
142
+ ### Task 1: SAR Batch — Tests
143
+
144
+ **Files:**
145
+ - Modify: `tests/test_indicator_sar.py`
146
+
147
+ - [ ] **Step 1: Add submit_batch test**
148
+
149
+ Append to `tests/test_indicator_sar.py`:
150
+
151
+ ```python
152
+ @pytest.mark.asyncio
153
+ async def test_sar_submit_batch_creates_three_jobs(test_aoi, test_time_range):
154
+ """submit_batch() creates current, baseline, and true-color batch jobs."""
155
+ from app.indicators.sar import SarIndicator
156
+
157
+ indicator = SarIndicator()
158
+
159
+ mock_conn = MagicMock()
160
+ mock_job = MagicMock()
161
+ mock_job.job_id = "j-test"
162
+ mock_conn.create_job.return_value = mock_job
163
+
164
+ with patch("app.indicators.sar.get_connection", return_value=mock_conn), \
165
+ patch("app.indicators.sar.build_sar_graph") as mock_sar_graph, \
166
+ patch("app.indicators.sar.build_true_color_graph") as mock_tc_graph:
167
+
168
+ mock_sar_graph.return_value = MagicMock()
169
+ mock_tc_graph.return_value = MagicMock()
170
+
171
+ jobs = await indicator.submit_batch(test_aoi, test_time_range)
172
+
173
+ assert len(jobs) == 3
174
+ assert mock_sar_graph.call_count == 2 # current + baseline
175
+ assert mock_tc_graph.call_count == 1 # true-color
176
+ ```
177
+
178
+ - [ ] **Step 2: Add harvest success test**
179
+
180
+ Append to `tests/test_indicator_sar.py`:
181
+
182
+ ```python
183
+ @pytest.mark.asyncio
184
+ async def test_sar_harvest_computes_result_from_batch_jobs(test_aoi, test_time_range):
185
+ """harvest() downloads batch results and returns IndicatorResult."""
186
+ from app.indicators.sar import SarIndicator
187
+
188
+ indicator = SarIndicator()
189
+
190
+ with tempfile.TemporaryDirectory() as tmpdir:
191
+ sar_path = os.path.join(tmpdir, "sar.tif")
192
+ rgb_path = os.path.join(tmpdir, "rgb.tif")
193
+ _mock_sar_tif(sar_path)
194
+ _mock_rgb_tif(rgb_path)
195
+
196
+ def make_mock_job(src_path):
197
+ job = MagicMock()
198
+ job.job_id = "j-test"
199
+ def fake_download_results(target):
200
+ import shutil
201
+ os.makedirs(target, exist_ok=True)
202
+ dest = os.path.join(target, "result.tif")
203
+ shutil.copy(src_path, dest)
204
+ from pathlib import Path
205
+ return {Path(dest): {"type": "image/tiff"}}
206
+ job.download_results.side_effect = fake_download_results
207
+ job.status.return_value = "finished"
208
+ return job
209
+
210
+ current_job = make_mock_job(sar_path)
211
+ baseline_job = make_mock_job(sar_path)
212
+ true_color_job = make_mock_job(rgb_path)
213
+
214
+ result = await indicator.harvest(
215
+ test_aoi, test_time_range,
216
+ batch_jobs=[current_job, baseline_job, true_color_job],
217
+ )
218
+
219
+ assert result.indicator_id == "sar"
220
+ assert result.data_source == "satellite"
221
+ assert result.status in (StatusLevel.GREEN, StatusLevel.AMBER, StatusLevel.RED)
222
+ assert result.confidence in (ConfidenceLevel.HIGH, ConfidenceLevel.MODERATE, ConfidenceLevel.LOW)
223
+ assert len(result.chart_data.get("dates", [])) > 0
224
+ ```
225
+
226
+ - [ ] **Step 3: Add harvest current-fails test**
227
+
228
+ Append to `tests/test_indicator_sar.py`:
229
+
230
+ ```python
231
+ @pytest.mark.asyncio
232
+ async def test_sar_harvest_falls_back_when_current_fails(test_aoi, test_time_range):
233
+ """harvest() returns placeholder when current SAR job failed."""
234
+ from app.indicators.sar import SarIndicator
235
+
236
+ indicator = SarIndicator()
237
+
238
+ current_job = MagicMock()
239
+ current_job.download_results.side_effect = Exception("failed")
240
+ baseline_job = MagicMock()
241
+ true_color_job = MagicMock()
242
+
243
+ result = await indicator.harvest(
244
+ test_aoi, test_time_range,
245
+ batch_jobs=[current_job, baseline_job, true_color_job],
246
+ )
247
+
248
+ assert result.data_source == "placeholder"
249
+ ```
250
+
251
+ - [ ] **Step 4: Add harvest baseline-fails test**
252
+
253
+ Append to `tests/test_indicator_sar.py`:
254
+
255
+ ```python
256
+ @pytest.mark.asyncio
257
+ async def test_sar_harvest_degrades_when_baseline_fails(test_aoi, test_time_range):
258
+ """harvest() returns degraded result when baseline SAR job failed."""
259
+ from app.indicators.sar import SarIndicator
260
+
261
+ indicator = SarIndicator()
262
+
263
+ with tempfile.TemporaryDirectory() as tmpdir:
264
+ sar_path = os.path.join(tmpdir, "sar.tif")
265
+ rgb_path = os.path.join(tmpdir, "rgb.tif")
266
+ _mock_sar_tif(sar_path)
267
+ _mock_rgb_tif(rgb_path)
268
+
269
+ def make_mock_job(src_path, fail=False):
270
+ job = MagicMock()
271
+ job.job_id = "j-test"
272
+ def fake_download_results(target):
273
+ if fail:
274
+ raise Exception("Batch job failed on CDSE")
275
+ import shutil
276
+ os.makedirs(target, exist_ok=True)
277
+ dest = os.path.join(target, "result.tif")
278
+ shutil.copy(src_path, dest)
279
+ from pathlib import Path
280
+ return {Path(dest): {"type": "image/tiff"}}
281
+ job.download_results.side_effect = fake_download_results
282
+ job.status.return_value = "finished"
283
+ return job
284
+
285
+ current_job = make_mock_job(sar_path)
286
+ baseline_job = make_mock_job(sar_path, fail=True)
287
+ true_color_job = make_mock_job(rgb_path)
288
+
289
+ result = await indicator.harvest(
290
+ test_aoi, test_time_range,
291
+ batch_jobs=[current_job, baseline_job, true_color_job],
292
+ )
293
+
294
+ assert result.indicator_id == "sar"
295
+ assert result.data_source == "satellite"
296
+ assert result.confidence == ConfidenceLevel.LOW
297
+ ```
298
+
299
+ - [ ] **Step 5: Run SAR tests to verify they fail**
300
+
301
+ Run: `pytest tests/test_indicator_sar.py -v -k "batch or harvest"`
302
+
303
+ Expected: FAIL — `SarIndicator` doesn't have `submit_batch` or `harvest` yet.
304
+
305
+ - [ ] **Step 6: Commit test file**
306
+
307
+ ```bash
308
+ git add tests/test_indicator_sar.py
309
+ git commit -m "test: add SAR batch submit and harvest tests (red)"
310
+ ```
311
+
312
+ ---
313
+
314
+ ### Task 2: SAR Batch — Implementation
315
+
316
+ **Files:**
317
+ - Modify: `app/indicators/sar.py`
318
+
319
+ - [ ] **Step 1: Update imports, baseline years, and batch flag**
320
+
321
+ In `app/indicators/sar.py`, change the import line:
322
+
323
+ ```python
324
+ from app.openeo_client import get_connection, build_sar_graph, build_true_color_graph, _bbox_dict
325
+ ```
326
+
327
+ To:
328
+
329
+ ```python
330
+ from app.openeo_client import get_connection, build_sar_graph, build_true_color_graph, _bbox_dict, submit_as_batch
331
+ ```
332
+
333
+ Change `BASELINE_YEARS = 3` to `BASELINE_YEARS = 5`.
334
+
335
+ In the `SarIndicator` class, add after `_true_color_path: str | None = None`:
336
+
337
+ ```python
338
+ uses_batch = True
339
+ ```
340
+
341
+ - [ ] **Step 2: Add _find_tif helper**
342
+
343
+ Add as a static method in `SarIndicator`, after the `_fallback` method:
344
+
345
+ ```python
346
+ @staticmethod
347
+ def _find_tif(download_paths: dict, fallback_dir: str) -> str:
348
+ """Find the GeoTIFF file from batch job download results."""
349
+ if download_paths:
350
+ for p in download_paths:
351
+ if str(p).endswith(".tif") or str(p).endswith(".tiff"):
352
+ return str(p)
353
+ for f in os.listdir(fallback_dir):
354
+ if f.endswith(".tif") or f.endswith(".tiff"):
355
+ return os.path.join(fallback_dir, f)
356
+ raise FileNotFoundError(f"No GeoTIFF found in {fallback_dir}")
357
+ ```
358
+
359
+ - [ ] **Step 3: Add submit_batch method**
360
+
361
+ Add after the `_true_color_path` / `uses_batch` declarations, before `process()`:
362
+
363
+ ```python
364
+ async def submit_batch(
365
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None = None
366
+ ) -> list:
367
+ conn = get_connection()
368
+ bbox = _bbox_dict(aoi.bbox)
369
+
370
+ current_start = time_range.start.isoformat()
371
+ current_end = time_range.end.isoformat()
372
+ baseline_start = date(
373
+ time_range.start.year - BASELINE_YEARS,
374
+ time_range.start.month,
375
+ time_range.start.day,
376
+ ).isoformat()
377
+ baseline_end = time_range.start.isoformat()
378
+
379
+ current_cube = build_sar_graph(
380
+ conn=conn, bbox=bbox,
381
+ temporal_extent=[current_start, current_end],
382
+ resolution_m=RESOLUTION_M,
383
+ )
384
+ baseline_cube = build_sar_graph(
385
+ conn=conn, bbox=bbox,
386
+ temporal_extent=[baseline_start, baseline_end],
387
+ resolution_m=RESOLUTION_M,
388
+ )
389
+ true_color_cube = build_true_color_graph(
390
+ conn=conn, bbox=bbox,
391
+ temporal_extent=[current_start, current_end],
392
+ resolution_m=RESOLUTION_M,
393
+ )
394
+
395
+ return [
396
+ submit_as_batch(conn, current_cube, f"sar-current-{aoi.name}"),
397
+ submit_as_batch(conn, baseline_cube, f"sar-baseline-{aoi.name}"),
398
+ submit_as_batch(conn, true_color_cube, f"sar-truecolor-{aoi.name}"),
399
+ ]
400
+ ```
401
+
402
+ - [ ] **Step 4: Add harvest method**
403
+
404
+ Add after `submit_batch()`, before `process()`:
405
+
406
+ ```python
407
+ async def harvest(
408
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None = None,
409
+ batch_jobs: list | None = None,
410
+ ) -> IndicatorResult:
411
+ """Download completed batch job results and compute SAR statistics."""
412
+ current_job, baseline_job, true_color_job = batch_jobs
413
+
414
+ results_dir = tempfile.mkdtemp(prefix="aperture_sar_batch_")
415
+
416
+ # Download current SAR — required
417
+ try:
418
+ current_dir = os.path.join(results_dir, "current")
419
+ os.makedirs(current_dir, exist_ok=True)
420
+ paths = current_job.download_results(current_dir)
421
+ current_path = self._find_tif(paths, current_dir)
422
+ except Exception as exc:
423
+ logger.warning("SAR current batch download failed: %s", exc)
424
+ return self._fallback(aoi, time_range)
425
+
426
+ # Download baseline — optional
427
+ baseline_path = None
428
+ try:
429
+ baseline_dir = os.path.join(results_dir, "baseline")
430
+ os.makedirs(baseline_dir, exist_ok=True)
431
+ paths = baseline_job.download_results(baseline_dir)
432
+ baseline_path = self._find_tif(paths, baseline_dir)
433
+ except Exception as exc:
434
+ logger.warning("SAR baseline batch download failed, degrading: %s", exc)
435
+
436
+ # Download true-color — optional
437
+ true_color_path = None
438
+ try:
439
+ tc_dir = os.path.join(results_dir, "truecolor")
440
+ os.makedirs(tc_dir, exist_ok=True)
441
+ paths = true_color_job.download_results(tc_dir)
442
+ true_color_path = self._find_tif(paths, tc_dir)
443
+ except Exception as exc:
444
+ logger.warning("SAR true-color batch download failed: %s", exc)
445
+
446
+ self._true_color_path = true_color_path
447
+
448
+ current_stats = self._compute_stats(current_path)
449
+
450
+ if current_stats["valid_months"] == 0:
451
+ return self._insufficient_data(aoi, time_range)
452
+
453
+ if baseline_path:
454
+ baseline_stats = self._compute_stats(baseline_path)
455
+ change_db = current_stats["overall_vv_mean"] - baseline_stats["overall_vv_mean"]
456
+ change_pct = self._compute_change_area_pct(
457
+ current_path, baseline_path, current_stats, baseline_stats
458
+ )
459
+ flood_months = self._count_flood_months(
460
+ current_stats["monthly_vv_means"],
461
+ baseline_stats["overall_vv_mean"],
462
+ baseline_stats["vv_std"],
463
+ )
464
+ confidence = (
465
+ ConfidenceLevel.HIGH if current_stats["valid_months"] >= 6
466
+ else ConfidenceLevel.MODERATE if current_stats["valid_months"] >= 3
467
+ else ConfidenceLevel.LOW
468
+ )
469
+ chart_data = self._build_chart_data(
470
+ current_stats["monthly_vv_means"],
471
+ baseline_stats["monthly_vv_means"],
472
+ time_range,
473
+ )
474
+
475
+ parts = []
476
+ if change_pct >= 5:
477
+ parts.append(f"{change_pct:.0f}% ground surface change")
478
+ if flood_months > 0:
479
+ parts.append(f"{flood_months} potential flood event{'s' if flood_months > 1 else ''}")
480
+ if parts:
481
+ headline = f"SAR detects {', '.join(parts)}"
482
+ else:
483
+ headline = "Stable backscatter conditions — no significant ground change detected"
484
+
485
+ change_map_path = os.path.join(results_dir, "sar_change.tif")
486
+ self._write_change_raster(current_path, baseline_path, change_map_path)
487
+
488
+ self._spatial_data = SpatialData(
489
+ map_type="raster",
490
+ label="SAR VV Change (dB)",
491
+ colormap="RdBu_r",
492
+ vmin=-6, vmax=6,
493
+ )
494
+ self._indicator_raster_path = change_map_path
495
+ self._render_band = 1
496
+ else:
497
+ change_db = 0.0
498
+ change_pct = 0.0
499
+ flood_months = 0
500
+ confidence = ConfidenceLevel.LOW
501
+ chart_data = {
502
+ "dates": [f"{time_range.end.year}-{m+1:02d}" for m in range(len(current_stats["monthly_vv_means"]))],
503
+ "values": [round(v, 2) for v in current_stats["monthly_vv_means"]],
504
+ "label": "VV Backscatter (dB)",
505
+ }
506
+ headline = "SAR data available \u2014 baseline unavailable for change detection"
507
+
508
+ self._spatial_data = SpatialData(
509
+ map_type="raster",
510
+ label="SAR VV (dB)",
511
+ colormap="gray",
512
+ vmin=-25, vmax=0,
513
+ )
514
+ self._indicator_raster_path = current_path
515
+ self._render_band = 1
516
+
517
+ status = self._classify(change_pct, flood_months)
518
+ trend = self._compute_trend(current_stats["monthly_vv_means"]) if baseline_path else TrendDirection.STABLE
519
+
520
+ return IndicatorResult(
521
+ indicator_id=self.id,
522
+ headline=headline,
523
+ status=status,
524
+ trend=trend,
525
+ confidence=confidence,
526
+ map_layer_path=self._indicator_raster_path,
527
+ chart_data=chart_data,
528
+ data_source="satellite",
529
+ summary=(
530
+ f"Mean VV backscatter change: {change_db:+.1f} dB. "
531
+ f"{change_pct:.1f}% of AOI shows significant change (>{CHANGE_THRESHOLD_DB} dB). "
532
+ f"{flood_months} month(s) with potential flood signals. "
533
+ f"Pixel-level analysis at {RESOLUTION_M}m resolution from "
534
+ f"{current_stats['valid_months']} monthly composites."
535
+ ),
536
+ methodology=(
537
+ f"Sentinel-1 GRD IW VV/VH polarizations, ascending orbit. "
538
+ f"Linear backscatter converted to dB (10\u00b7log\u2081\u2080). "
539
+ f"Monthly median composites at {RESOLUTION_M}m resolution. "
540
+ f"Change detection: >{CHANGE_THRESHOLD_DB} dB difference vs "
541
+ f"{BASELINE_YEARS}-year baseline. "
542
+ f"Flood mapping: VV < baseline_mean \u2212 {FLOOD_SIGMA}\u03c3. "
543
+ f"Processed via CDSE openEO batch jobs."
544
+ ),
545
+ limitations=[
546
+ f"Resampled to {RESOLUTION_M}m \u2014 fine-scale changes not captured.",
547
+ "Ascending orbit filter may reduce temporal coverage in some areas.",
548
+ "Sentinel-1 coverage over East Africa can be inconsistent.",
549
+ "VV decrease may indicate flooding, moisture, or vegetation change \u2014 not uniquely flood.",
550
+ ] + (["Baseline unavailable \u2014 change detection and flood analysis not computed."] if not baseline_path else []),
551
+ )
552
+ ```
553
+
554
+ - [ ] **Step 5: Run SAR tests**
555
+
556
+ Run: `pytest tests/test_indicator_sar.py -v`
557
+
558
+ Expected: ALL PASS
559
+
560
+ - [ ] **Step 6: Commit**
561
+
562
+ ```bash
563
+ git add app/indicators/sar.py tests/test_indicator_sar.py
564
+ git commit -m "feat: add batch support to SAR indicator"
565
+ ```
566
+
567
+ ---
568
+
569
+ ### Task 3: Buildup Batch — Tests
570
+
571
+ **Files:**
572
+ - Modify: `tests/test_indicator_buildup.py`
573
+
574
+ - [ ] **Step 1: Add submit_batch test**
575
+
576
+ Append to `tests/test_indicator_buildup.py`:
577
+
578
+ ```python
579
+ @pytest.mark.asyncio
580
+ async def test_buildup_submit_batch_creates_three_jobs(test_aoi, test_time_range):
581
+ """submit_batch() creates current, baseline, and true-color batch jobs."""
582
+ from app.indicators.buildup import BuiltupIndicator
583
+
584
+ indicator = BuiltupIndicator()
585
+
586
+ mock_conn = MagicMock()
587
+ mock_job = MagicMock()
588
+ mock_job.job_id = "j-test"
589
+ mock_conn.create_job.return_value = mock_job
590
+
591
+ with patch("app.indicators.buildup.get_connection", return_value=mock_conn), \
592
+ patch("app.indicators.buildup.build_buildup_graph") as mock_bu_graph, \
593
+ patch("app.indicators.buildup.build_true_color_graph") as mock_tc_graph:
594
+
595
+ mock_bu_graph.return_value = MagicMock()
596
+ mock_tc_graph.return_value = MagicMock()
597
+
598
+ jobs = await indicator.submit_batch(test_aoi, test_time_range)
599
+
600
+ assert len(jobs) == 3
601
+ assert mock_bu_graph.call_count == 2 # current + baseline
602
+ assert mock_tc_graph.call_count == 1
603
+ ```
604
+
605
+ - [ ] **Step 2: Add harvest success test**
606
+
607
+ Append to `tests/test_indicator_buildup.py`:
608
+
609
+ ```python
610
+ @pytest.mark.asyncio
611
+ async def test_buildup_harvest_computes_result_from_batch_jobs(test_aoi, test_time_range):
612
+ """harvest() downloads batch results and returns IndicatorResult."""
613
+ from app.indicators.buildup import BuiltupIndicator
614
+
615
+ indicator = BuiltupIndicator()
616
+
617
+ with tempfile.TemporaryDirectory() as tmpdir:
618
+ ndbi_path = os.path.join(tmpdir, "ndbi.tif")
619
+ rgb_path = os.path.join(tmpdir, "rgb.tif")
620
+ _mock_ndbi_tif(ndbi_path)
621
+ _mock_rgb_tif(rgb_path)
622
+
623
+ def make_mock_job(src_path):
624
+ job = MagicMock()
625
+ job.job_id = "j-test"
626
+ def fake_download_results(target):
627
+ import shutil
628
+ os.makedirs(target, exist_ok=True)
629
+ dest = os.path.join(target, "result.tif")
630
+ shutil.copy(src_path, dest)
631
+ from pathlib import Path
632
+ return {Path(dest): {"type": "image/tiff"}}
633
+ job.download_results.side_effect = fake_download_results
634
+ job.status.return_value = "finished"
635
+ return job
636
+
637
+ current_job = make_mock_job(ndbi_path)
638
+ baseline_job = make_mock_job(ndbi_path)
639
+ true_color_job = make_mock_job(rgb_path)
640
+
641
+ result = await indicator.harvest(
642
+ test_aoi, test_time_range,
643
+ batch_jobs=[current_job, baseline_job, true_color_job],
644
+ )
645
+
646
+ assert result.indicator_id == "buildup"
647
+ assert result.data_source == "satellite"
648
+ assert result.status in (StatusLevel.GREEN, StatusLevel.AMBER, StatusLevel.RED)
649
+ assert result.confidence in (ConfidenceLevel.HIGH, ConfidenceLevel.MODERATE, ConfidenceLevel.LOW)
650
+ assert len(result.chart_data.get("dates", [])) > 0
651
+ ```
652
+
653
+ - [ ] **Step 3: Add harvest current-fails test**
654
+
655
+ Append to `tests/test_indicator_buildup.py`:
656
+
657
+ ```python
658
+ @pytest.mark.asyncio
659
+ async def test_buildup_harvest_falls_back_when_current_fails(test_aoi, test_time_range):
660
+ """harvest() returns placeholder when current NDBI job failed."""
661
+ from app.indicators.buildup import BuiltupIndicator
662
+
663
+ indicator = BuiltupIndicator()
664
+
665
+ current_job = MagicMock()
666
+ current_job.download_results.side_effect = Exception("failed")
667
+ baseline_job = MagicMock()
668
+ true_color_job = MagicMock()
669
+
670
+ result = await indicator.harvest(
671
+ test_aoi, test_time_range,
672
+ batch_jobs=[current_job, baseline_job, true_color_job],
673
+ )
674
+
675
+ assert result.data_source == "placeholder"
676
+ ```
677
+
678
+ - [ ] **Step 4: Add harvest baseline-fails test**
679
+
680
+ Append to `tests/test_indicator_buildup.py`:
681
+
682
+ ```python
683
+ @pytest.mark.asyncio
684
+ async def test_buildup_harvest_degrades_when_baseline_fails(test_aoi, test_time_range):
685
+ """harvest() returns degraded result when baseline NDBI job failed."""
686
+ from app.indicators.buildup import BuiltupIndicator
687
+
688
+ indicator = BuiltupIndicator()
689
+
690
+ with tempfile.TemporaryDirectory() as tmpdir:
691
+ ndbi_path = os.path.join(tmpdir, "ndbi.tif")
692
+ rgb_path = os.path.join(tmpdir, "rgb.tif")
693
+ _mock_ndbi_tif(ndbi_path)
694
+ _mock_rgb_tif(rgb_path)
695
+
696
+ def make_mock_job(src_path, fail=False):
697
+ job = MagicMock()
698
+ job.job_id = "j-test"
699
+ def fake_download_results(target):
700
+ if fail:
701
+ raise Exception("Batch job failed on CDSE")
702
+ import shutil
703
+ os.makedirs(target, exist_ok=True)
704
+ dest = os.path.join(target, "result.tif")
705
+ shutil.copy(src_path, dest)
706
+ from pathlib import Path
707
+ return {Path(dest): {"type": "image/tiff"}}
708
+ job.download_results.side_effect = fake_download_results
709
+ job.status.return_value = "finished"
710
+ return job
711
+
712
+ current_job = make_mock_job(ndbi_path)
713
+ baseline_job = make_mock_job(ndbi_path, fail=True)
714
+ true_color_job = make_mock_job(rgb_path)
715
+
716
+ result = await indicator.harvest(
717
+ test_aoi, test_time_range,
718
+ batch_jobs=[current_job, baseline_job, true_color_job],
719
+ )
720
+
721
+ assert result.indicator_id == "buildup"
722
+ assert result.data_source == "satellite"
723
+ assert result.confidence == ConfidenceLevel.LOW
724
+ ```
725
+
726
+ - [ ] **Step 5: Run buildup batch tests to verify they fail**
727
+
728
+ Run: `pytest tests/test_indicator_buildup.py -v -k "batch or harvest"`
729
+
730
+ Expected: FAIL — `BuiltupIndicator` doesn't have `submit_batch` or `harvest` yet.
731
+
732
+ - [ ] **Step 6: Commit test file**
733
+
734
+ ```bash
735
+ git add tests/test_indicator_buildup.py
736
+ git commit -m "test: add buildup batch submit and harvest tests (red)"
737
+ ```
738
+
739
+ ---
740
+
741
+ ### Task 4: Buildup Batch — Implementation
742
+
743
+ **Files:**
744
+ - Modify: `app/indicators/buildup.py`
745
+
746
+ - [ ] **Step 1: Update imports, baseline years, and batch flag**
747
+
748
+ In `app/indicators/buildup.py`, change the import line:
749
+
750
+ ```python
751
+ from app.openeo_client import get_connection, build_buildup_graph, build_true_color_graph, _bbox_dict
752
+ ```
753
+
754
+ To:
755
+
756
+ ```python
757
+ from app.openeo_client import get_connection, build_buildup_graph, build_true_color_graph, _bbox_dict, submit_as_batch
758
+ ```
759
+
760
+ Change `BASELINE_YEARS = 3` to `BASELINE_YEARS = 5`.
761
+
762
+ In the `BuiltupIndicator` class, add after `_true_color_path: str | None = None`:
763
+
764
+ ```python
765
+ uses_batch = True
766
+ ```
767
+
768
+ - [ ] **Step 2: Add _find_tif helper**
769
+
770
+ Add as a static method in `BuiltupIndicator`, after the `_fallback` method:
771
+
772
+ ```python
773
+ @staticmethod
774
+ def _find_tif(download_paths: dict, fallback_dir: str) -> str:
775
+ """Find the GeoTIFF file from batch job download results."""
776
+ if download_paths:
777
+ for p in download_paths:
778
+ if str(p).endswith(".tif") or str(p).endswith(".tiff"):
779
+ return str(p)
780
+ for f in os.listdir(fallback_dir):
781
+ if f.endswith(".tif") or f.endswith(".tiff"):
782
+ return os.path.join(fallback_dir, f)
783
+ raise FileNotFoundError(f"No GeoTIFF found in {fallback_dir}")
784
+ ```
785
+
786
+ - [ ] **Step 3: Add submit_batch method**
787
+
788
+ Add after the `uses_batch` declaration, before `process()`:
789
+
790
+ ```python
791
+ async def submit_batch(
792
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None = None
793
+ ) -> list:
794
+ conn = get_connection()
795
+ bbox = _bbox_dict(aoi.bbox)
796
+
797
+ current_start = time_range.start.isoformat()
798
+ current_end = time_range.end.isoformat()
799
+ baseline_start = date(
800
+ time_range.start.year - BASELINE_YEARS,
801
+ time_range.start.month,
802
+ time_range.start.day,
803
+ ).isoformat()
804
+ baseline_end = time_range.start.isoformat()
805
+
806
+ current_cube = build_buildup_graph(
807
+ conn=conn, bbox=bbox,
808
+ temporal_extent=[current_start, current_end],
809
+ resolution_m=RESOLUTION_M,
810
+ )
811
+ baseline_cube = build_buildup_graph(
812
+ conn=conn, bbox=bbox,
813
+ temporal_extent=[baseline_start, baseline_end],
814
+ resolution_m=RESOLUTION_M,
815
+ )
816
+ true_color_cube = build_true_color_graph(
817
+ conn=conn, bbox=bbox,
818
+ temporal_extent=[current_start, current_end],
819
+ resolution_m=RESOLUTION_M,
820
+ )
821
+
822
+ return [
823
+ submit_as_batch(conn, current_cube, f"buildup-current-{aoi.name}"),
824
+ submit_as_batch(conn, baseline_cube, f"buildup-baseline-{aoi.name}"),
825
+ submit_as_batch(conn, true_color_cube, f"buildup-truecolor-{aoi.name}"),
826
+ ]
827
+ ```
828
+
829
+ - [ ] **Step 4: Add harvest method**
830
+
831
+ Add after `submit_batch()`, before `process()`:
832
+
833
+ ```python
834
+ async def harvest(
835
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None = None,
836
+ batch_jobs: list | None = None,
837
+ ) -> IndicatorResult:
838
+ """Download completed batch job results and compute buildup statistics."""
839
+ current_job, baseline_job, true_color_job = batch_jobs
840
+
841
+ results_dir = tempfile.mkdtemp(prefix="aperture_buildup_batch_")
842
+
843
+ # Download current NDBI — required
844
+ try:
845
+ current_dir = os.path.join(results_dir, "current")
846
+ os.makedirs(current_dir, exist_ok=True)
847
+ paths = current_job.download_results(current_dir)
848
+ current_path = self._find_tif(paths, current_dir)
849
+ except Exception as exc:
850
+ logger.warning("Buildup current batch download failed: %s", exc)
851
+ return self._fallback(aoi, time_range)
852
+
853
+ # Download baseline — optional
854
+ baseline_path = None
855
+ try:
856
+ baseline_dir = os.path.join(results_dir, "baseline")
857
+ os.makedirs(baseline_dir, exist_ok=True)
858
+ paths = baseline_job.download_results(baseline_dir)
859
+ baseline_path = self._find_tif(paths, baseline_dir)
860
+ except Exception as exc:
861
+ logger.warning("Buildup baseline batch download failed, degrading: %s", exc)
862
+
863
+ # Download true-color — optional
864
+ true_color_path = None
865
+ try:
866
+ tc_dir = os.path.join(results_dir, "truecolor")
867
+ os.makedirs(tc_dir, exist_ok=True)
868
+ paths = true_color_job.download_results(tc_dir)
869
+ true_color_path = self._find_tif(paths, tc_dir)
870
+ except Exception as exc:
871
+ logger.warning("Buildup true-color batch download failed: %s", exc)
872
+
873
+ self._true_color_path = true_color_path
874
+
875
+ current_stats = self._compute_stats(current_path)
876
+ current_frac = current_stats["overall_buildup_fraction"]
877
+ aoi_ha = aoi.area_km2 * 100
878
+
879
+ if baseline_path:
880
+ baseline_stats = self._compute_stats(baseline_path)
881
+ baseline_frac = baseline_stats["overall_buildup_fraction"]
882
+ current_ha = current_frac * aoi_ha
883
+ baseline_ha = baseline_frac * aoi_ha
884
+
885
+ if baseline_frac > 0:
886
+ change_pct = ((current_frac - baseline_frac) / baseline_frac) * 100
887
+ else:
888
+ change_pct = 100.0 if current_frac > 0 else 0.0
889
+
890
+ confidence = (
891
+ ConfidenceLevel.HIGH if current_stats["valid_months"] >= 6
892
+ else ConfidenceLevel.MODERATE if current_stats["valid_months"] >= 3
893
+ else ConfidenceLevel.LOW
894
+ )
895
+ chart_data = self._build_chart_data(
896
+ current_stats["monthly_buildup_fractions"],
897
+ baseline_stats["monthly_buildup_fractions"],
898
+ time_range, aoi_ha,
899
+ )
900
+
901
+ if abs(change_pct) < 10:
902
+ headline = f"Built-up extent stable at approximately {current_ha:.0f} ha"
903
+ elif change_pct > 0:
904
+ headline = f"Settlement area expanded {change_pct:.0f}% ({baseline_ha:.0f} \u2192 {current_ha:.0f} ha) compared to baseline"
905
+ else:
906
+ headline = f"Potential settlement contraction: {abs(change_pct):.0f}% decrease in built-up area"
907
+
908
+ change_map_path = os.path.join(results_dir, "buildup_change.tif")
909
+ self._write_change_raster(current_path, baseline_path, change_map_path)
910
+
911
+ self._spatial_data = SpatialData(
912
+ map_type="raster",
913
+ label="Built-up Change",
914
+ colormap="PiYG",
915
+ vmin=-1, vmax=1,
916
+ )
917
+ self._indicator_raster_path = change_map_path
918
+ self._render_band = 1
919
+ else:
920
+ baseline_frac = current_frac
921
+ change_pct = 0.0
922
+ current_ha = current_frac * aoi_ha
923
+ baseline_ha = current_ha
924
+ confidence = ConfidenceLevel.LOW
925
+ chart_data = {
926
+ "dates": [f"{time_range.end.year}-{m+1:02d}" for m in range(len(current_stats["monthly_buildup_fractions"]))],
927
+ "values": [round(v * aoi_ha, 1) for v in current_stats["monthly_buildup_fractions"]],
928
+ "label": "Built-up area (ha)",
929
+ }
930
+ headline = f"Built-up extent: {current_frac*100:.1f}% \u2014 baseline unavailable"
931
+
932
+ self._spatial_data = SpatialData(
933
+ map_type="raster",
934
+ label="NDBI",
935
+ colormap="PiYG",
936
+ vmin=-0.5, vmax=0.5,
937
+ )
938
+ self._indicator_raster_path = current_path
939
+ self._render_band = current_stats["peak_buildup_band"]
940
+
941
+ status = self._classify(change_pct)
942
+ trend = self._compute_trend(change_pct) if baseline_path else TrendDirection.STABLE
943
+
944
+ return IndicatorResult(
945
+ indicator_id=self.id,
946
+ headline=headline,
947
+ status=status,
948
+ trend=trend,
949
+ confidence=confidence,
950
+ map_layer_path=self._indicator_raster_path,
951
+ chart_data=chart_data,
952
+ data_source="satellite",
953
+ summary=(
954
+ f"Built-up area covers {current_frac*100:.1f}% of the AOI "
955
+ f"({current_ha:.0f} ha) compared to {baseline_frac*100:.1f}% baseline "
956
+ f"({baseline_ha:.0f} ha), a {change_pct:+.1f}% change. "
957
+ f"Pixel-level NDBI analysis at {RESOLUTION_M}m resolution."
958
+ ),
959
+ methodology=(
960
+ f"Sentinel-2 L2A pixel-level NDBI = (B11 \u2212 B08) / (B11 + B08). "
961
+ f"Built-up classified as NDBI > {NDBI_THRESHOLD}. "
962
+ f"Cloud-masked using SCL band. "
963
+ f"Monthly median composites at {RESOLUTION_M}m. "
964
+ f"Baseline: {BASELINE_YEARS}-year built-up extent. "
965
+ f"Processed via CDSE openEO batch jobs."
966
+ ),
967
+ limitations=[
968
+ f"Resampled to {RESOLUTION_M}m \u2014 detects settlement extent, not individual buildings.",
969
+ "NDBI may confuse bare rock/sand with built-up in arid landscapes.",
970
+ "Seasonal vegetation cycles can cause false positives at settlement fringes.",
971
+ "For building-level analysis, the SR4S pipeline (GPU-dependent) would be needed.",
972
+ ] + (["Baseline unavailable \u2014 change detection not computed."] if not baseline_path else []),
973
+ )
974
+ ```
975
+
976
+ - [ ] **Step 5: Run buildup tests**
977
+
978
+ Run: `pytest tests/test_indicator_buildup.py -v`
979
+
980
+ Expected: ALL PASS
981
+
982
+ - [ ] **Step 6: Commit**
983
+
984
+ ```bash
985
+ git add app/indicators/buildup.py tests/test_indicator_buildup.py
986
+ git commit -m "feat: add batch support to buildup indicator"
987
+ ```
988
+
989
+ ---
990
+
991
+ ### Task 5: Water Batch — Tests
992
+
993
+ **Files:**
994
+ - Modify: `tests/test_indicator_water.py`
995
+
996
+ - [ ] **Step 1: Add submit_batch test**
997
+
998
+ Append to `tests/test_indicator_water.py`:
999
+
1000
+ ```python
1001
+ @pytest.mark.asyncio
1002
+ async def test_water_submit_batch_creates_three_jobs(test_aoi, test_time_range):
1003
+ """submit_batch() creates current, baseline, and true-color batch jobs."""
1004
+ from app.indicators.water import WaterIndicator
1005
+
1006
+ indicator = WaterIndicator()
1007
+
1008
+ mock_conn = MagicMock()
1009
+ mock_job = MagicMock()
1010
+ mock_job.job_id = "j-test"
1011
+ mock_conn.create_job.return_value = mock_job
1012
+
1013
+ with patch("app.indicators.water.get_connection", return_value=mock_conn), \
1014
+ patch("app.indicators.water.build_mndwi_graph") as mock_water_graph, \
1015
+ patch("app.indicators.water.build_true_color_graph") as mock_tc_graph:
1016
+
1017
+ mock_water_graph.return_value = MagicMock()
1018
+ mock_tc_graph.return_value = MagicMock()
1019
+
1020
+ jobs = await indicator.submit_batch(test_aoi, test_time_range)
1021
+
1022
+ assert len(jobs) == 3
1023
+ assert mock_water_graph.call_count == 2 # current + baseline
1024
+ assert mock_tc_graph.call_count == 1
1025
+ ```
1026
+
1027
+ - [ ] **Step 2: Add harvest success test**
1028
+
1029
+ Append to `tests/test_indicator_water.py`:
1030
+
1031
+ ```python
1032
+ @pytest.mark.asyncio
1033
+ async def test_water_harvest_computes_result_from_batch_jobs(test_aoi, test_time_range):
1034
+ """harvest() downloads batch results and returns IndicatorResult."""
1035
+ from app.indicators.water import WaterIndicator
1036
+
1037
+ indicator = WaterIndicator()
1038
+
1039
+ with tempfile.TemporaryDirectory() as tmpdir:
1040
+ mndwi_path = os.path.join(tmpdir, "mndwi.tif")
1041
+ rgb_path = os.path.join(tmpdir, "rgb.tif")
1042
+ _mock_mndwi_tif(mndwi_path)
1043
+ _mock_rgb_tif(rgb_path)
1044
+
1045
+ def make_mock_job(src_path):
1046
+ job = MagicMock()
1047
+ job.job_id = "j-test"
1048
+ def fake_download_results(target):
1049
+ import shutil
1050
+ os.makedirs(target, exist_ok=True)
1051
+ dest = os.path.join(target, "result.tif")
1052
+ shutil.copy(src_path, dest)
1053
+ from pathlib import Path
1054
+ return {Path(dest): {"type": "image/tiff"}}
1055
+ job.download_results.side_effect = fake_download_results
1056
+ job.status.return_value = "finished"
1057
+ return job
1058
+
1059
+ current_job = make_mock_job(mndwi_path)
1060
+ baseline_job = make_mock_job(mndwi_path)
1061
+ true_color_job = make_mock_job(rgb_path)
1062
+
1063
+ result = await indicator.harvest(
1064
+ test_aoi, test_time_range,
1065
+ batch_jobs=[current_job, baseline_job, true_color_job],
1066
+ )
1067
+
1068
+ assert result.indicator_id == "water"
1069
+ assert result.data_source == "satellite"
1070
+ assert result.status in (StatusLevel.GREEN, StatusLevel.AMBER, StatusLevel.RED)
1071
+ assert result.confidence in (ConfidenceLevel.HIGH, ConfidenceLevel.MODERATE, ConfidenceLevel.LOW)
1072
+ assert len(result.chart_data.get("dates", [])) > 0
1073
+ ```
1074
+
1075
+ - [ ] **Step 3: Add harvest current-fails test**
1076
+
1077
+ Append to `tests/test_indicator_water.py`:
1078
+
1079
+ ```python
1080
+ @pytest.mark.asyncio
1081
+ async def test_water_harvest_falls_back_when_current_fails(test_aoi, test_time_range):
1082
+ """harvest() returns placeholder when current MNDWI job failed."""
1083
+ from app.indicators.water import WaterIndicator
1084
+
1085
+ indicator = WaterIndicator()
1086
+
1087
+ current_job = MagicMock()
1088
+ current_job.download_results.side_effect = Exception("failed")
1089
+ baseline_job = MagicMock()
1090
+ true_color_job = MagicMock()
1091
+
1092
+ result = await indicator.harvest(
1093
+ test_aoi, test_time_range,
1094
+ batch_jobs=[current_job, baseline_job, true_color_job],
1095
+ )
1096
+
1097
+ assert result.data_source == "placeholder"
1098
+ ```
1099
+
1100
+ - [ ] **Step 4: Add harvest baseline-fails test**
1101
+
1102
+ Append to `tests/test_indicator_water.py`:
1103
+
1104
+ ```python
1105
+ @pytest.mark.asyncio
1106
+ async def test_water_harvest_degrades_when_baseline_fails(test_aoi, test_time_range):
1107
+ """harvest() returns degraded result when baseline MNDWI job failed."""
1108
+ from app.indicators.water import WaterIndicator
1109
+
1110
+ indicator = WaterIndicator()
1111
+
1112
+ with tempfile.TemporaryDirectory() as tmpdir:
1113
+ mndwi_path = os.path.join(tmpdir, "mndwi.tif")
1114
+ rgb_path = os.path.join(tmpdir, "rgb.tif")
1115
+ _mock_mndwi_tif(mndwi_path)
1116
+ _mock_rgb_tif(rgb_path)
1117
+
1118
+ def make_mock_job(src_path, fail=False):
1119
+ job = MagicMock()
1120
+ job.job_id = "j-test"
1121
+ def fake_download_results(target):
1122
+ if fail:
1123
+ raise Exception("Batch job failed on CDSE")
1124
+ import shutil
1125
+ os.makedirs(target, exist_ok=True)
1126
+ dest = os.path.join(target, "result.tif")
1127
+ shutil.copy(src_path, dest)
1128
+ from pathlib import Path
1129
+ return {Path(dest): {"type": "image/tiff"}}
1130
+ job.download_results.side_effect = fake_download_results
1131
+ job.status.return_value = "finished"
1132
+ return job
1133
+
1134
+ current_job = make_mock_job(mndwi_path)
1135
+ baseline_job = make_mock_job(mndwi_path, fail=True)
1136
+ true_color_job = make_mock_job(rgb_path)
1137
+
1138
+ result = await indicator.harvest(
1139
+ test_aoi, test_time_range,
1140
+ batch_jobs=[current_job, baseline_job, true_color_job],
1141
+ )
1142
+
1143
+ assert result.indicator_id == "water"
1144
+ assert result.data_source == "satellite"
1145
+ assert result.confidence == ConfidenceLevel.LOW
1146
+ ```
1147
+
1148
+ - [ ] **Step 5: Run water batch tests to verify they fail**
1149
+
1150
+ Run: `pytest tests/test_indicator_water.py -v -k "batch or harvest"`
1151
+
1152
+ Expected: FAIL — `WaterIndicator` doesn't have `submit_batch` or `harvest` yet.
1153
+
1154
+ - [ ] **Step 6: Commit test file**
1155
+
1156
+ ```bash
1157
+ git add tests/test_indicator_water.py
1158
+ git commit -m "test: add water batch submit and harvest tests (red)"
1159
+ ```
1160
+
1161
+ ---
1162
+
1163
+ ### Task 6: Water Batch — Implementation
1164
+
1165
+ **Files:**
1166
+ - Modify: `app/indicators/water.py`
1167
+
1168
+ - [ ] **Step 1: Update imports, baseline years, and batch flag**
1169
+
1170
+ In `app/indicators/water.py`, change the import line:
1171
+
1172
+ ```python
1173
+ from app.openeo_client import get_connection, build_mndwi_graph, build_true_color_graph, _bbox_dict
1174
+ ```
1175
+
1176
+ To:
1177
+
1178
+ ```python
1179
+ from app.openeo_client import get_connection, build_mndwi_graph, build_true_color_graph, _bbox_dict, submit_as_batch
1180
+ ```
1181
+
1182
+ Change `BASELINE_YEARS = 3` to `BASELINE_YEARS = 5`.
1183
+
1184
+ In the `WaterIndicator` class, add after `_true_color_path: str | None = None`:
1185
+
1186
+ ```python
1187
+ uses_batch = True
1188
+ ```
1189
+
1190
+ - [ ] **Step 2: Add _find_tif helper**
1191
+
1192
+ Add as a static method in `WaterIndicator`, after the `_fallback` method:
1193
+
1194
+ ```python
1195
+ @staticmethod
1196
+ def _find_tif(download_paths: dict, fallback_dir: str) -> str:
1197
+ """Find the GeoTIFF file from batch job download results."""
1198
+ if download_paths:
1199
+ for p in download_paths:
1200
+ if str(p).endswith(".tif") or str(p).endswith(".tiff"):
1201
+ return str(p)
1202
+ for f in os.listdir(fallback_dir):
1203
+ if f.endswith(".tif") or f.endswith(".tiff"):
1204
+ return os.path.join(fallback_dir, f)
1205
+ raise FileNotFoundError(f"No GeoTIFF found in {fallback_dir}")
1206
+ ```
1207
+
1208
+ - [ ] **Step 3: Add submit_batch method**
1209
+
1210
+ Add after the `uses_batch` declaration, before `process()`:
1211
+
1212
+ ```python
1213
+ async def submit_batch(
1214
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None = None
1215
+ ) -> list:
1216
+ conn = get_connection()
1217
+ bbox = _bbox_dict(aoi.bbox)
1218
+
1219
+ current_start = time_range.start.isoformat()
1220
+ current_end = time_range.end.isoformat()
1221
+ baseline_start = date(
1222
+ time_range.start.year - BASELINE_YEARS,
1223
+ time_range.start.month,
1224
+ time_range.start.day,
1225
+ ).isoformat()
1226
+ baseline_end = time_range.start.isoformat()
1227
+
1228
+ current_cube = build_mndwi_graph(
1229
+ conn=conn, bbox=bbox,
1230
+ temporal_extent=[current_start, current_end],
1231
+ resolution_m=RESOLUTION_M,
1232
+ )
1233
+ baseline_cube = build_mndwi_graph(
1234
+ conn=conn, bbox=bbox,
1235
+ temporal_extent=[baseline_start, baseline_end],
1236
+ resolution_m=RESOLUTION_M,
1237
+ )
1238
+ true_color_cube = build_true_color_graph(
1239
+ conn=conn, bbox=bbox,
1240
+ temporal_extent=[current_start, current_end],
1241
+ resolution_m=RESOLUTION_M,
1242
+ )
1243
+
1244
+ return [
1245
+ submit_as_batch(conn, current_cube, f"water-current-{aoi.name}"),
1246
+ submit_as_batch(conn, baseline_cube, f"water-baseline-{aoi.name}"),
1247
+ submit_as_batch(conn, true_color_cube, f"water-truecolor-{aoi.name}"),
1248
+ ]
1249
+ ```
1250
+
1251
+ - [ ] **Step 4: Add harvest method**
1252
+
1253
+ Add after `submit_batch()`, before `process()`:
1254
+
1255
+ ```python
1256
+ async def harvest(
1257
+ self, aoi: AOI, time_range: TimeRange, season_months: list[int] | None = None,
1258
+ batch_jobs: list | None = None,
1259
+ ) -> IndicatorResult:
1260
+ """Download completed batch job results and compute water statistics."""
1261
+ current_job, baseline_job, true_color_job = batch_jobs
1262
+
1263
+ results_dir = tempfile.mkdtemp(prefix="aperture_water_batch_")
1264
+
1265
+ # Download current MNDWI — required
1266
+ try:
1267
+ current_dir = os.path.join(results_dir, "current")
1268
+ os.makedirs(current_dir, exist_ok=True)
1269
+ paths = current_job.download_results(current_dir)
1270
+ current_path = self._find_tif(paths, current_dir)
1271
+ except Exception as exc:
1272
+ logger.warning("Water current batch download failed: %s", exc)
1273
+ return self._fallback(aoi, time_range)
1274
+
1275
+ # Download baseline — optional
1276
+ baseline_path = None
1277
+ try:
1278
+ baseline_dir = os.path.join(results_dir, "baseline")
1279
+ os.makedirs(baseline_dir, exist_ok=True)
1280
+ paths = baseline_job.download_results(baseline_dir)
1281
+ baseline_path = self._find_tif(paths, baseline_dir)
1282
+ except Exception as exc:
1283
+ logger.warning("Water baseline batch download failed, degrading: %s", exc)
1284
+
1285
+ # Download true-color — optional
1286
+ true_color_path = None
1287
+ try:
1288
+ tc_dir = os.path.join(results_dir, "truecolor")
1289
+ os.makedirs(tc_dir, exist_ok=True)
1290
+ paths = true_color_job.download_results(tc_dir)
1291
+ true_color_path = self._find_tif(paths, tc_dir)
1292
+ except Exception as exc:
1293
+ logger.warning("Water true-color batch download failed: %s", exc)
1294
+
1295
+ self._true_color_path = true_color_path
1296
+
1297
+ current_stats = self._compute_stats(current_path)
1298
+ current_frac = current_stats["overall_water_fraction"]
1299
+
1300
+ if baseline_path:
1301
+ baseline_stats = self._compute_stats(baseline_path)
1302
+ baseline_frac = baseline_stats["overall_water_fraction"]
1303
+ change_pp = (current_frac - baseline_frac) * 100
1304
+
1305
+ confidence = (
1306
+ ConfidenceLevel.HIGH if current_stats["valid_months"] >= 6
1307
+ else ConfidenceLevel.MODERATE if current_stats["valid_months"] >= 3
1308
+ else ConfidenceLevel.LOW
1309
+ )
1310
+ chart_data = self._build_chart_data(
1311
+ current_stats["monthly_water_fractions"],
1312
+ baseline_stats["monthly_water_fractions"],
1313
+ time_range,
1314
+ )
1315
+
1316
+ direction = "increase" if change_pp > 0 else "decrease"
1317
+ if abs(change_pp) <= 5:
1318
+ headline = f"Water extent stable ({current_frac*100:.1f}%, \u0394{change_pp:+.1f}pp)"
1319
+ else:
1320
+ headline = f"Water extent {direction} ({change_pp:+.1f}pp vs baseline)"
1321
+ else:
1322
+ baseline_frac = current_frac
1323
+ change_pp = 0.0
1324
+ confidence = ConfidenceLevel.LOW
1325
+ chart_data = {
1326
+ "dates": [f"{time_range.end.year}-{m+1:02d}" for m in range(len(current_stats["monthly_water_fractions"]))],
1327
+ "values": [round(v * 100, 1) for v in current_stats["monthly_water_fractions"]],
1328
+ "label": "Water extent (%)",
1329
+ }
1330
+ headline = f"Water extent: {current_frac*100:.1f}% \u2014 baseline unavailable"
1331
+
1332
+ status = self._classify(abs(change_pp))
1333
+ trend = self._compute_trend(change_pp) if baseline_path else TrendDirection.STABLE
1334
+
1335
+ self._spatial_data = SpatialData(
1336
+ map_type="raster",
1337
+ label="MNDWI",
1338
+ colormap="Blues",
1339
+ vmin=-0.5, vmax=0.5,
1340
+ )
1341
+ self._indicator_raster_path = current_path
1342
+ self._render_band = current_stats["peak_water_band"]
1343
+
1344
+ return IndicatorResult(
1345
+ indicator_id=self.id,
1346
+ headline=headline,
1347
+ status=status,
1348
+ trend=trend,
1349
+ confidence=confidence,
1350
+ map_layer_path=current_path,
1351
+ chart_data=chart_data,
1352
+ data_source="satellite",
1353
+ summary=(
1354
+ f"Water covers {current_frac*100:.1f}% of the AOI compared to "
1355
+ f"{baseline_frac*100:.1f}% baseline ({change_pp:+.1f}pp). "
1356
+ f"Pixel-level MNDWI analysis at {RESOLUTION_M}m resolution."
1357
+ ),
1358
+ methodology=(
1359
+ f"Sentinel-2 L2A pixel-level MNDWI = (B03 \u2212 B11) / (B03 + B11). "
1360
+ f"Cloud-masked using SCL band. Water classified as MNDWI > {WATER_THRESHOLD}. "
1361
+ f"Monthly median composites at {RESOLUTION_M}m. "
1362
+ f"Baseline: {BASELINE_YEARS}-year water extent frequency. "
1363
+ f"Processed via CDSE openEO batch jobs."
1364
+ ),
1365
+ limitations=[
1366
+ f"Resampled to {RESOLUTION_M}m \u2014 small water bodies may be missed.",
1367
+ "Cloud/shadow contamination can cause false water detections.",
1368
+ "Seasonal flooding may appear as change if analysis windows differ.",
1369
+ "MNDWI threshold is fixed; turbid water may be misclassified.",
1370
+ ] + (["Baseline unavailable \u2014 change detection not computed."] if not baseline_path else []),
1371
+ )
1372
+ ```
1373
+
1374
+ - [ ] **Step 5: Run water tests**
1375
+
1376
+ Run: `pytest tests/test_indicator_water.py -v`
1377
+
1378
+ Expected: ALL PASS
1379
+
1380
+ - [ ] **Step 6: Commit**
1381
+
1382
+ ```bash
1383
+ git add app/indicators/water.py tests/test_indicator_water.py
1384
+ git commit -m "feat: add batch support to water indicator"
1385
+ ```
1386
+
1387
+ ---
1388
+
1389
+ ### Task 7: Full Test Suite + Final Verification
1390
+
1391
+ - [ ] **Step 1: Run the complete test suite**
1392
+
1393
+ Run: `pytest tests/ -v`
1394
+
1395
+ Expected: ALL PASS — no regressions in existing tests, all new batch tests pass.
1396
+
1397
+ - [ ] **Step 2: Verify batch flags are set correctly**
1398
+
1399
+ Run: `python -c "from app.indicators.sar import SarIndicator; from app.indicators.buildup import BuiltupIndicator; from app.indicators.water import WaterIndicator; print('SAR:', SarIndicator().uses_batch); print('Buildup:', BuiltupIndicator().uses_batch); print('Water:', WaterIndicator().uses_batch)"`
1400
+
1401
+ Expected:
1402
+ ```
1403
+ SAR: True
1404
+ Buildup: True
1405
+ Water: True
1406
+ ```
1407
+
1408
+ - [ ] **Step 3: Verify baseline years are standardized**
1409
+
1410
+ Run: `python -c "from app.indicators.sar import BASELINE_YEARS as s; from app.indicators.buildup import BASELINE_YEARS as b; from app.indicators.water import BASELINE_YEARS as w; from app.indicators.ndvi import BASELINE_YEARS as n; print(f'NDVI={n}, SAR={s}, Buildup={b}, Water={w}')"`
1411
+
1412
+ Expected: `NDVI=5, SAR=5, Buildup=5, Water=5`
1413
+
1414
+ - [ ] **Step 4: Commit any remaining changes**
1415
+
1416
+ If all tests pass and no changes remain, this step is a no-op.