File size: 7,418 Bytes
32dd8b9 ddadf71 32dd8b9 2227bb5 32dd8b9 2227bb5 32dd8b9 ddadf71 32dd8b9 2227bb5 32dd8b9 2227bb5 32dd8b9 2227bb5 32dd8b9 ddadf71 e6d3182 ddadf71 32dd8b9 2227bb5 32dd8b9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 | # Typical Marine Ecological Environment Feature Dataset Standard
This project uses a manifest-driven dataset layout for marine ecological environment
feature recognition. It is designed to keep different patch sizes, satellites,
sensors, spatial resolutions, and fusion states instead of forcing everything into
one fixed tile format.
## Scope
Target feature families:
- `green_tide`: Enteromorpha / green tide / seaweed.
- `red_tide`: red tide / harmful algal bloom.
- `golden_tide`: Sargassum / golden tide.
- `aquaculture`: aquaculture area, rafts, cages, ponds, or related facilities.
- `water`: valid water background.
- `land`: land or non-water hard negative.
- `invalid`: black border, no-data, saturated, missing, or otherwise unusable pixels.
- `cloud_shadow`: cloud, haze, cloud shadow, or bright atmospheric interference.
- `other`: valid but not assigned to the categories above.
No external coastline, land-mask vector, or GIS mask is assumed to exist. Land,
invalid area, and water/background handling must be represented by dataset labels,
hard-negative samples, or image-derived validity rules.
## Canonical Directory Layout
```text
data_marine_features/
manifests/
assets_raw.jsonl
samples.jsonl
splits/
train.txt
val.txt
test.txt
images/
<asset_id>.<ext> # optional symlink or copied tile
masks/
<sample_id>.<ext> # optional symlink or copied mask
previews/
<sample_id>.png
reports/
asset_inventory.csv
```
The canonical source of truth is `manifests/samples.jsonl`. Files may remain in
their original locations; `images/` and `masks/` are optional conveniences.
## `samples.jsonl` Schema
Each line is one image sample or tile.
```json
{
"sample_id": "gf6_20250604_green_tide_000001",
"element": "green_tide",
"task_type": "semantic_segmentation",
"image_path": "D:/.../image.tif",
"mask_path": "D:/.../mask.tif",
"annotation_path": null,
"annotation_format": "mask",
"label_encoding": {"0": "background", "1": "green_tide"},
"satellite": "GF6",
"sensor": "PMS",
"resolution_m": 2.0,
"patch_size": 256,
"bands": ["blue", "green", "red", "nir"],
"band_count": 4,
"dtype": "uint16",
"fusion": {
"state": "fused_product",
"method": "unknown_vendor_product",
"sources": [
{"role": "MSS", "path": null, "resolution_m": 8.0},
{"role": "PAN", "path": null, "resolution_m": 2.0}
],
"target_resolution_m": 2.0,
"native_multispectral_resolution_m": 8.0,
"persisted": true,
"reproducible": false,
"spectral_preservation": "unknown",
"notes": "Fused image supplied as a finished raster; source PAN/MSS not guaranteed."
},
"acquired_at": "2025-06-04",
"source_project": "EntGreenTide",
"source_dataset": "GF6_PMS_E121.1_N33.6_20250604_L1A1420584616",
"split": "train",
"quality_flags": ["valid_image"],
"notes": ""
}
```
Required fields:
- `sample_id`
- `element`
- `task_type`
- `image_path`
- `satellite`
- `sensor`
- `patch_size`
- `band_count`
- `fusion`
- `source_project`
Recommended fields:
- `mask_path`
- `annotation_path`
- `annotation_format`
- `resolution_m`
- `bands`
- `dtype`
- `acquired_at`
- `split`
- `quality_flags`
## Multi-Scale Policy
Patch sizes such as 128, 256, 512, and full-scene windows are all valid. They are
not merged destructively. Training code should sample them with metadata-aware
transforms:
- Keep `patch_size` in the manifest.
- Resize only inside the training transform when the model requires it.
- Preserve the original spatial resolution in `resolution_m`.
- For full-scene inference, use sliding windows whose size is a runtime parameter.
## Multi-Sensor And Fusion Policy
Different satellites and sensors are expected:
- GF1, GF2, GF6, and other optical satellites can coexist.
- Fused PMS/MUX products and raw PAN+MSS products can coexist.
- If PAN+MSS are available, keep both paths in `fusion.sources`.
- Do not assume band order from the filename alone; record `bands` when known.
Fusion must be represented explicitly. A boolean such as `is_fused=true` is not
enough because it loses the source products, method, target resolution, and
reproducibility.
Recommended `fusion` object for an already fused raster:
```json
{
"state": "fused_product",
"method": "unknown_vendor_product",
"sources": [
{"role": "MSS", "path": "D:/.../MSS2.tiff", "resolution_m": 8.0},
{"role": "PAN", "path": null, "resolution_m": 2.0}
],
"target_resolution_m": 2.0,
"native_multispectral_resolution_m": 8.0,
"persisted": true,
"reproducible": false,
"spectral_preservation": "unknown",
"notes": "Fused image supplied as a finished raster; original PAN is unavailable."
}
```
Recommended `fusion` object for streaming PAN+MSS fusion:
```json
{
"state": "runtime_fusion",
"method": "additive_component_substitution",
"sources": [
{"role": "MSS", "path": "D:/.../MSS2.tiff", "resolution_m": 8.0},
{"role": "PAN", "path": "D:/.../PAN2.tiff", "resolution_m": 2.0}
],
"target_resolution_m": 2.0,
"native_multispectral_resolution_m": 8.0,
"persisted": false,
"reproducible": true,
"tile_aligned": true,
"notes": "Fusion is performed per inference tile and is not written as a full-scene raster."
}
```
Allowed `fusion.state` values:
- `none`: native product, no fusion.
- `fused_product`: fused raster exists on disk.
- `runtime_fusion`: fusion happens during training/inference.
- `unknown`: insufficient information; do not pretend it is native.
## Label Policy
The final model should not be binary seaweed/background. It should learn ecological
elements and scene context. A recommended unified semantic target is:
| Class ID | Name |
| --- | --- |
| 0 | other/background |
| 1 | invalid |
| 2 | water |
| 3 | land |
| 4 | cloud_shadow |
| 5 | green_tide |
| 6 | red_tide |
| 7 | golden_tide |
| 8 | aquaculture |
When legacy binary masks are imported, they should be represented as task-specific
labels plus metadata. Do not silently treat unlabeled pixels as true negatives for
all other ecological elements.
## Quality Gate Policy
Discovered samples are not automatically training samples. A sample may enter
`accepted_samples.jsonl` only after an explicit adapter verifies:
- the data source is relevant to marine or coastal remote sensing;
- image and label assets are reliably paired;
- label semantics map to a registry element card;
- label format is parsed successfully, such as mask, polygon, GeoJSON, SHP, or RLE;
- box-only labels are rejected from SAMPoly-style polygon training because they
cannot supervise true vertices or boundaries;
- unusable images, empty labels, unknown elements, and ambiguous licenses are
rejected or kept for manual review;
- quality flags record why the sample was accepted.
Unknown or unpaired samples must not be treated as negative samples.
## Inventory Rules
The inventory scanner should:
- Preserve original file paths.
- Infer element type from directory names and known project folders.
- Infer satellite/sensor/acquisition date from filenames when possible.
- Infer fusion state from filenames such as `fuse`, `PAN`, and `MSS`, but mark
uncertain products as `unknown` instead of guessing method details.
- Pair image and mask files by stem when masks exist.
- Record unpaired full-scene images as inference assets.
- Flag ambiguous data instead of guessing labels.
|