Update README.md
Browse files
README.md
CHANGED
|
@@ -1,53 +1,66 @@
|
|
| 1 |
---
|
| 2 |
base_model:
|
| 3 |
-
- ibm-esa-geospatial/TerraMind-1.0-base
|
| 4 |
-
pipeline_tag:
|
| 5 |
tags:
|
| 6 |
-
- geospatial
|
| 7 |
-
- regression
|
| 8 |
-
- soil
|
| 9 |
-
-
|
|
|
|
| 10 |
---
|
| 11 |
|
|
|
|
| 12 |
|
| 13 |
-
|
| 14 |
|
| 15 |
-
|
| 16 |
|
| 17 |
-
|
| 18 |
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
- `
|
| 22 |
-
-
|
| 23 |
-
- `
|
| 24 |
-
-
|
| 25 |
-
|
| 26 |
-
- `
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
- checkpoint inference and export of `submission.csv` and metrics.
|
| 28 |
-
- `hyperview_subimssion.py`
|
| 29 |
- baseline, class mapping, and evaluation helpers.
|
| 30 |
-
- `hybrid_models.ipynb`
|
| 31 |
- notebook for hybrid evaluation and ranking of full/small/big submission combinations.
|
| 32 |
|
| 33 |
-
## Configuration
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
## Training
|
| 53 |
|
|
@@ -81,67 +94,47 @@ terratorch fit -c configs/terramind_v1_base_hyperview_upernet_none_i1.yaml
|
|
| 81 |
terratorch fit -c configs/terramind_v1_base_hyperview_unet_none.yaml
|
| 82 |
```
|
| 83 |
|
| 84 |
-
## End-to-
|
| 85 |
|
| 86 |
Run the full train+test pipeline with:
|
| 87 |
|
| 88 |
```bash
|
| 89 |
-
./
|
| 90 |
```
|
| 91 |
|
| 92 |
The script executes, in order:
|
| 93 |
-
- train `upernet_none` and generate `submissions/upernet_none`
|
| 94 |
-
- train `upernet_none_small` and generate `submissions/upernet_none_small`
|
| 95 |
-
- train `upernet_none_big` and generate `submissions/upernet_none_big`
|
| 96 |
-
- train `unet_none` and generate `submissions/unet_none`
|
| 97 |
-
- evaluate `upernet_none_i1` using `configs/terramind_v1_base_hyperview_upernet_none_i1.yaml`
|
| 98 |
- run external-model evaluations for:
|
| 99 |
-
- `upernet_none_external`
|
| 100 |
-
- `upernet_none_i1_external`
|
| 101 |
-
- `upernet_none_enmap_20231109T101043Z_external`
|
| 102 |
-
- `upernet_none_enmap_20231109T101043Z`
|
| 103 |
|
| 104 |
-
## Submission
|
| 105 |
|
| 106 |
```bash
|
| 107 |
-
python3 prepare_submission.py
|
| 108 |
-
--model_dir runs/terratorch_hyperview_upernet_none \
|
| 109 |
-
--config configs/terramind_v1_base_hyperview_upernet_none.yaml \
|
| 110 |
-
--output_dir submissions/upernet_none
|
| 111 |
```
|
| 112 |
|
| 113 |
Example for `small`:
|
| 114 |
|
| 115 |
```bash
|
| 116 |
-
python3 prepare_submission.py
|
| 117 |
-
--model_dir runs/terratorch_hyperview_upernet_none_small \
|
| 118 |
-
--config configs/terramind_v1_base_hyperview_upernet_none_small.yaml \
|
| 119 |
-
--output_dir submissions/upernet_none_small
|
| 120 |
```
|
| 121 |
|
| 122 |
Example for `big`:
|
| 123 |
|
| 124 |
```bash
|
| 125 |
-
python3 prepare_submission.py
|
| 126 |
-
--model_dir runs/terratorch_hyperview_upernet_none_big \
|
| 127 |
-
--config configs/terramind_v1_base_hyperview_upernet_none_big.yaml \
|
| 128 |
-
--output_dir submissions/upernet_none_big
|
| 129 |
```
|
| 130 |
|
| 131 |
Example for `test_intuition`:
|
| 132 |
|
| 133 |
```bash
|
| 134 |
-
python3 prepare_submission.py
|
| 135 |
-
--model_dir runs/terratorch_hyperview_upernet_none \
|
| 136 |
-
--config configs/terramind_v1_base_hyperview_upernet_none_i1.yaml \
|
| 137 |
-
--output_dir submissions/upernet_none_i1
|
| 138 |
```
|
| 139 |
|
| 140 |
-
## Notes
|
| 141 |
-
|
| 142 |
-
- `prepare_submission.py` reads `split`, `aoi`, `label_test_path`, `only_11x11`, `exclude_11x11`, and `drop_last` from the config.
|
| 143 |
-
- Metrics and predictions are written to `--output_dir`:
|
| 144 |
-
- `submission.csv`
|
| 145 |
-
- `gt.csv`
|
| 146 |
-
- `all_metrics.json`
|
| 147 |
-
- `all_metrics.csv`
|
|
|
|
| 1 |
---
|
| 2 |
base_model:
|
| 3 |
+
- ibm-esa-geospatial/TerraMind-1.0-base
|
| 4 |
+
pipeline_tag: tabular-regression
|
| 5 |
tags:
|
| 6 |
+
- geospatial
|
| 7 |
+
- regression
|
| 8 |
+
- soil
|
| 9 |
+
- foundation-model
|
| 10 |
+
- terratorch
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# FAST-EO Use Case 4 - Estimation of Soil Properties
|
| 14 |
|
| 15 |
+
This repository provides a training and evaluation pipeline for **soil-property regression** (`P`, `K`, `Mg`, `pH`) on **Hyperview** data using **Terratorch** and the `terramind_v1_base` backbone.
|
| 16 |
|
| 17 |
+
## Overview
|
| 18 |
|
| 19 |
+
The goal of this use case is to fine-tune TerraMind-Base for predicting soil properties from remote-sensing inputs. The codebase supports multiple dataset splits (including external test splits) and multiple configuration variants (full vs. small vs. big vs. intuition/enmap evaluation).
|
| 20 |
|
| 21 |
+
## Repository structure
|
| 22 |
+
|
| 23 |
+
- `configs/`
|
| 24 |
+
- Terratorch training/inference configs.
|
| 25 |
+
- Decoder variants: `UNetDecoder`, `UperNetDecoder`.
|
| 26 |
+
- Split/size variants: `none` (full), `small`, `big`, `i1`.
|
| 27 |
+
- `datasets/`
|
| 28 |
+
- `hyperview_dataset.py`: dataset with split support:
|
| 29 |
+
- `test`, `test_dat`, `test_intuition`, `test_enmap`
|
| 30 |
+
- size filters: `only_11x11`, `exclude_11x11`
|
| 31 |
+
- `hyperview_datamodule.py`: Lightning DataModule and transforms.
|
| 32 |
+
- `callback_hooks/`
|
| 33 |
+
- `loss_logging_callback.py`: epoch-level loss logging callback.
|
| 34 |
+
- `prepare_submission.py`
|
| 35 |
- checkpoint inference and export of `submission.csv` and metrics.
|
| 36 |
+
- `hyperview_subimssion.py`
|
| 37 |
- baseline, class mapping, and evaluation helpers.
|
| 38 |
+
- `hybrid_models.ipynb`
|
| 39 |
- notebook for hybrid evaluation and ranking of full/small/big submission combinations.
|
| 40 |
|
| 41 |
+
## Configuration rules
|
| 42 |
+
|
| 43 |
+
In each config, set dataset and label paths correctly:
|
| 44 |
+
- `data.init_args.data_root`
|
| 45 |
+
- `data.init_args.label_train_path`
|
| 46 |
+
- `data.init_args.label_test_path`
|
| 47 |
+
|
| 48 |
+
### Supported splits
|
| 49 |
+
|
| 50 |
+
Training is supported on standard Hyperview:
|
| 51 |
+
- full (`test_data`)
|
| 52 |
+
- `small` (11x11 only)
|
| 53 |
+
- `big` (excluding 11x11)
|
| 54 |
+
|
| 55 |
+
Testing/evaluation is supported on:
|
| 56 |
+
- Hyperview full
|
| 57 |
+
- Hyperview `small`
|
| 58 |
+
- Hyperview `big`
|
| 59 |
+
- `test_dat`
|
| 60 |
+
- `test_intuition` (`intuition1`)
|
| 61 |
+
- `test_enmap`
|
| 62 |
+
|
| 63 |
+
For `test_enmap`, `aoi` is mandatory.
|
| 64 |
|
| 65 |
## Training
|
| 66 |
|
|
|
|
| 94 |
terratorch fit -c configs/terramind_v1_base_hyperview_unet_none.yaml
|
| 95 |
```
|
| 96 |
|
| 97 |
+
## End-to-end script
|
| 98 |
|
| 99 |
Run the full train+test pipeline with:
|
| 100 |
|
| 101 |
```bash
|
| 102 |
+
./run_train_test.sh
|
| 103 |
```
|
| 104 |
|
| 105 |
The script executes, in order:
|
| 106 |
+
- train `upernet_none` and generate `submissions/upernet_none`
|
| 107 |
+
- train `upernet_none_small` and generate `submissions/upernet_none_small`
|
| 108 |
+
- train `upernet_none_big` and generate `submissions/upernet_none_big`
|
| 109 |
+
- train `unet_none` and generate `submissions/unet_none`
|
| 110 |
+
- evaluate `upernet_none_i1` using `configs/terramind_v1_base_hyperview_upernet_none_i1.yaml`
|
| 111 |
- run external-model evaluations for:
|
| 112 |
+
- `upernet_none_external`
|
| 113 |
+
- `upernet_none_i1_external`
|
| 114 |
+
- `upernet_none_enmap_20231109T101043Z_external`
|
| 115 |
+
- `upernet_none_enmap_20231109T101043Z`
|
| 116 |
|
| 117 |
+
## Submission generation
|
| 118 |
|
| 119 |
```bash
|
| 120 |
+
python3 prepare_submission.py --model_dir runs/terratorch_hyperview_upernet_none --config configs/terramind_v1_base_hyperview_upernet_none.yaml --output_dir submissions/upernet_none
|
|
|
|
|
|
|
|
|
|
| 121 |
```
|
| 122 |
|
| 123 |
Example for `small`:
|
| 124 |
|
| 125 |
```bash
|
| 126 |
+
python3 prepare_submission.py --model_dir runs/terratorch_hyperview_upernet_none_small --config configs/terramind_v1_base_hyperview_upernet_none_small.yaml --output_dir submissions/upernet_none_small
|
|
|
|
|
|
|
|
|
|
| 127 |
```
|
| 128 |
|
| 129 |
Example for `big`:
|
| 130 |
|
| 131 |
```bash
|
| 132 |
+
python3 prepare_submission.py --model_dir runs/terratorch_hyperview_upernet_none_big --config configs/terramind_v1_base_hyperview_upernet_none_big.yaml --output_dir submissions/upernet_none_big
|
|
|
|
|
|
|
|
|
|
| 133 |
```
|
| 134 |
|
| 135 |
Example for `test_intuition`:
|
| 136 |
|
| 137 |
```bash
|
| 138 |
+
python3 prepare_submission.py --model_dir runs/terratorch_hyperview_upernet_none --config configs/terramind_v1_base_hyperview_upernet_none_i1.yaml --output_dir submissions/upernet_none_i1
|
|
|
|
|
|
|
|
|
|
| 139 |
```
|
| 140 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|