RhodWeo commited on
Commit
452b4a9
·
verified ·
1 Parent(s): 3b2bdb2

docs: clean benchmark section + proper model-index (mean metrics, no per-dataset wall)

Browse files
Files changed (1) hide show
  1. README.md +84 -112
README.md CHANGED
@@ -8,114 +8,87 @@ tags:
8
  - pytorch
9
  pipeline_tag: image-to-image
10
  model-index:
11
- - name: sen2sr (main SEN2SRLite RGBN 4x)
12
  results:
13
  - task:
14
  type: image-to-image
15
  name: Super Resolution 4x
16
  dataset:
17
- type: opensr-test-naip
18
- name: opensr-test NAIP
19
  metrics:
20
- - {type: reflectance, name: "Reflectance L1 ↓", value: 0.001016}
21
- - {type: spectral, name: "Spectral Angle", value: 0.471423}
22
- - {type: synthesis, name: "Synthesis ↑", value: 0.072529}
23
- - {type: hallucination, name: "Hallucination ↓", value: 0.056076}
24
- - {type: omission, name: "Omission ↓", value: 0.069583}
25
- - {type: improvement, name: "Improvement ↑", value: 0.874341}
26
- - task:
27
- type: image-to-image
28
- name: Super Resolution 4x
29
- dataset:
30
- type: opensr-test-spot
31
- name: opensr-test SPOT
32
- metrics:
33
- - {type: reflectance, name: "Reflectance L1 ", value: 0.001976}
34
- - {type: spectral, name: "Spectral Angle ↓", value: 0.628618}
35
- - {type: synthesis, name: "Synthesis ↑", value: 0.076452}
36
- - {type: hallucination, name: "Hallucination ↓", value: 0.073418}
37
- - {type: omission, name: "Omission ↓", value: 0.127390}
38
- - {type: improvement, name: "Improvement ↑", value: 0.799192}
39
- - task:
40
- type: image-to-image
41
- name: Super Resolution 4x
42
- dataset:
43
- type: opensr-test-spain-crops
44
- name: opensr-test Spain Crops
45
- metrics:
46
- - {type: reflectance, name: "Reflectance L1 ↓", value: 0.001188}
47
- - {type: spectral, name: "Spectral Angle ↓", value: 0.546995}
48
- - {type: synthesis, name: "Synthesis ↑", value: 0.077599}
49
- - {type: hallucination, name: "Hallucination ↓", value: 0.073548}
50
- - {type: omission, name: "Omission ↓", value: 0.085900}
51
- - {type: improvement, name: "Improvement ↑", value: 0.840552}
52
- - task:
53
- type: image-to-image
54
- name: Super Resolution 4x
55
- dataset:
56
- type: opensr-test-spain-urban
57
- name: opensr-test Spain Urban
58
- metrics:
59
- - {type: reflectance, name: "Reflectance L1 ↓", value: 0.002167}
60
- - {type: spectral, name: "Spectral Angle ↓", value: 0.899134}
61
- - {type: synthesis, name: "Synthesis ↑", value: 0.065831}
62
- - {type: hallucination, name: "Hallucination ↓", value: 0.115622}
63
- - {type: omission, name: "Omission ↓", value: 0.189015}
64
- - {type: improvement, name: "Improvement ↑", value: 0.695363}
65
- - name: sen2sr (mamba-rgbn-x4 — SEN2SR Mamba RGBN 4x)
66
  results:
67
  - task:
68
  type: image-to-image
69
  name: Super Resolution 4x
70
  dataset:
71
- type: opensr-test-naip
72
- name: opensr-test NAIP
73
  metrics:
74
- - {type: reflectance, name: "Reflectance L1 ↓", value: 0.001255}
75
- - {type: spectral, name: "Spectral Angle", value: 0.674253}
76
- - {type: synthesis, name: "Synthesis ↑", value: 0.072526}
77
- - {type: hallucination, name: "Hallucination ↓", value: 0.056509}
78
- - {type: omission, name: "Omission ↓", value: 0.070804}
79
- - {type: improvement, name: "Improvement ↑", value: 0.872687}
80
- - task:
81
- type: image-to-image
82
- name: Super Resolution 4x
83
- dataset:
84
- type: opensr-test-spot
85
- name: opensr-test SPOT
86
- metrics:
87
- - {type: reflectance, name: "Reflectance L1 ", value: 0.002166}
88
- - {type: spectral, name: "Spectral Angle ↓", value: 0.728039}
89
- - {type: synthesis, name: "Synthesis ↑", value: 0.076409}
90
- - {type: hallucination, name: "Hallucination ↓", value: 0.074770}
91
- - {type: omission, name: "Omission ↓", value: 0.131327}
92
- - {type: improvement, name: "Improvement ↑", value: 0.793903}
93
- - task:
94
- type: image-to-image
95
- name: Super Resolution 4x
96
- dataset:
97
- type: opensr-test-spain-crops
98
- name: opensr-test Spain Crops
99
- metrics:
100
- - {type: reflectance, name: "Reflectance L1 ↓", value: 0.001476}
101
- - {type: spectral, name: "Spectral Angle ↓", value: 0.803401}
102
- - {type: synthesis, name: "Synthesis ↑", value: 0.077594}
103
- - {type: hallucination, name: "Hallucination ↓", value: 0.074450}
104
- - {type: omission, name: "Omission ↓", value: 0.088468}
105
- - {type: improvement, name: "Improvement ↑", value: 0.837082}
106
  - task:
107
  type: image-to-image
108
  name: Super Resolution 4x
109
  dataset:
110
- type: opensr-test-spain-urban
111
- name: opensr-test Spain Urban
112
  metrics:
113
- - {type: reflectance, name: "Reflectance L1 ↓", value: 0.002514}
114
- - {type: spectral, name: "Spectral Angle", value: 1.184267}
115
- - {type: synthesis, name: "Synthesis ↑", value: 0.065839}
116
- - {type: hallucination, name: "Hallucination ↓", value: 0.118179}
117
- - {type: omission, name: "Omission ↓", value: 0.195655}
118
- - {type: improvement, name: "Improvement ↑", value: 0.686166}
 
 
 
 
 
 
 
 
 
 
 
 
119
  ---
120
 
121
 
@@ -288,35 +261,34 @@ f2_model.safetensor / f2_hard_constraint.safetensor (RSWIR 2x stage)
288
 
289
  ---
290
 
 
291
  ## Benchmark
292
 
293
- Evaluated on [opensr-test](https://github.com/ESAOpenSR/opensr-test) across 4 datasets (NAIP, SPOT, Spain Crops, Spain Urban).
294
- Venus is a 2x dataset and is excluded from 4x model evaluation.
295
- Full results: [eval/sen2sr_eval_results.csv](eval/sen2sr_eval_results.csv)
 
 
 
 
 
 
296
 
297
  <p align="center">
298
- <img src="eval/sen2sr_benchmark.png" width="100%">
299
  </p>
300
 
301
- | Variant | Reflectance↓ | Spectral↓ | Synthesis↑ | Hallucination↓ | Omission↓ | Improvement↑ |
302
- |---|---|---|---|---|---|---|
303
- | `main` (CNN RGBN 4x) | 0.0016 | 0.637 | 0.073 | 0.080 | 0.118 | 0.802 |
304
- | `mamba-rgbn-x4` (Mamba RGBN 4x) | 0.0019 | 0.848 | 0.073 | 0.081 | 0.122 | 0.797 |
305
- | `lite-main` (CNN 10-band 4x) | 0.0033 | 2.006 | 0.073 | 0.080 | 0.120 | 0.801 |
306
 
307
- *Mean across NAIP, SPOT, Spain Crops, Spain Urban. `mamba-main` results pending (see open issues).*
308
 
309
- ---
 
 
310
 
311
- ## Citation
312
 
313
- If you use these models please cite the original work:
 
 
314
 
315
- ```bibtex
316
- @software{sen2sr2024,
317
- author = {Aybar, Cesar and others},
318
- title = {SEN2SR: Sentinel-2 Super-Resolution},
319
- url = {https://github.com/ESAOpenSR/sen2sr},
320
- year = {2024}
321
- }
322
- ```
 
8
  - pytorch
9
  pipeline_tag: image-to-image
10
  model-index:
11
+ - name: sen2sr main (CNN RGBN 4x)
12
  results:
13
  - task:
14
  type: image-to-image
15
  name: Super Resolution 4x
16
  dataset:
17
+ name: opensr-test
18
+ type: opensr-test
19
  metrics:
20
+ - type: reflectance
21
+ name: Reflectance L1 ()
22
+ value: 0.001587
23
+ - type: spectral
24
+ name: Spectral Angle (↓)
25
+ value: 0.636542
26
+ - type: synthesis
27
+ name: Synthesis (↑)
28
+ value: 0.073103
29
+ - type: hallucination
30
+ name: Hallucination (↓)
31
+ value: 0.079666
32
+ - type: omission
33
+ name: Omission ()
34
+ value: 0.117972
35
+ - type: improvement
36
+ name: Improvement (↑)
37
+ value: 0.802362
38
+ - name: sen2sr mamba-rgbn-x4 (Mamba RGBN 4x)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  results:
40
  - task:
41
  type: image-to-image
42
  name: Super Resolution 4x
43
  dataset:
44
+ name: opensr-test
45
+ type: opensr-test
46
  metrics:
47
+ - type: reflectance
48
+ name: Reflectance L1 ()
49
+ value: 0.001853
50
+ - type: spectral
51
+ name: Spectral Angle (↓)
52
+ value: 0.847490
53
+ - type: synthesis
54
+ name: Synthesis (↑)
55
+ value: 0.073092
56
+ - type: hallucination
57
+ name: Hallucination (↓)
58
+ value: 0.080977
59
+ - type: omission
60
+ name: Omission ()
61
+ value: 0.121563
62
+ - type: improvement
63
+ name: Improvement (↑)
64
+ value: 0.797460
65
+ - name: sen2sr lite-main (CNN 10-band 4x)
66
+ results:
 
 
 
 
 
 
 
 
 
 
 
 
67
  - task:
68
  type: image-to-image
69
  name: Super Resolution 4x
70
  dataset:
71
+ name: opensr-test
72
+ type: opensr-test
73
  metrics:
74
+ - type: reflectance
75
+ name: Reflectance L1 ()
76
+ value: 0.003309
77
+ - type: spectral
78
+ name: Spectral Angle (↓)
79
+ value: 2.006296
80
+ - type: synthesis
81
+ name: Synthesis (↑)
82
+ value: 0.073082
83
+ - type: hallucination
84
+ name: Hallucination (↓)
85
+ value: 0.079739
86
+ - type: omission
87
+ name: Omission (↓)
88
+ value: 0.119464
89
+ - type: improvement
90
+ name: Improvement (↑)
91
+ value: 0.800797
92
  ---
93
 
94
 
 
261
 
262
  ---
263
 
264
+
265
  ## Benchmark
266
 
267
+ Evaluated on [opensr-test](https://github.com/ESAOpenSR/opensr-test) NAIP, SPOT, Spain Crops, Spain Urban (4× datasets). Means across all 4 datasets. Venus excluded (2× dataset, incompatible with 4× models).
268
+
269
+ | Variant | Architecture | Reflectance↓ | Spectral↓ | Hallucination↓ | Omission↓ | Improvement↑ |
270
+ |---|---|---|---|---|---|---|
271
+ | `main` | CNN | 0.0016 | 0.64 | 0.080 | 0.118 | **0.802** |
272
+ | `mamba-rgbn-x4` | Mamba | 0.0019 | 0.85 | 0.081 | 0.122 | 0.797 |
273
+ | `lite-main` | CNN (10-band) | 0.0033 | 2.01 | 0.080 | 0.119 | 0.801 |
274
+
275
+ > `mamba-main` results pending. Full per-dataset breakdown: [`eval/sen2sr_eval_results.csv`](eval/sen2sr_eval_results.csv)
276
 
277
  <p align="center">
278
+ <img src="eval/sen2sr_benchmark.png" width="95%">
279
  </p>
280
 
281
+ ### Visual comparison Spain Urban (opensr-test)
 
 
 
 
282
 
283
+ LR (Sentinel-2 bicubic) · SR (sen2sr `main`) · HR (SPOT reference)
284
 
285
+ <p align="center">
286
+ <img src="eval_images/spain_urban.png" width="95%">
287
+ </p>
288
 
289
+ ### Visual comparison — Spain Crops (opensr-test)
290
 
291
+ <p align="center">
292
+ <img src="eval_images/spain_crops.png" width="95%">
293
+ </p>
294