Dhruv1000/TSRDA

T-SRDA — satellite image time-series crop mapping (PASTIS + MTLCC)

Custom deep-learning architecture (Temporal Spatial-Reduction Dual-Attention), motivated by the STCLN paper and benchmarked on the same two datasets STCLN used, each with that dataset's own protocol. Architecture and optimisation hyper-parameters are identical across the two datasets; only dataset-dependent values follow the respective STCLN scripts.

Repository layout

Folder	Dataset	Contents
PASTIS/	PASTIS (10 Sentinel-2 bands, 128×128 patches)	All PASTIS work: main method, ablations, checkpoints, notebooks, reports. The full PASTIS description is reproduced below, unchanged.
MTLCC/	MTLCC (13 Sentinel-2 bands, 48×48 tiles)	MTLCC port: code, tools, results, checkpoints. Full write-up: MTLCC/REPORT.md.

MTLCC benchmark (added 7 November 2026)

Protocol of the STCLN MTLCC scripts: 2016 data, 30 randomly drawn dates, pre-training on train_fold0, fine-tuning on 78 hand-picked tiles (13 classes × 6), evaluation on eval.tileids (1,495 usable tiles) with test_STCLN.py's metrics. The paper's values below are those released by the STCLN authors; _wp = with pre-training.

Model	OA	mIoU	mF1
STCLN (paper, no pre-training)	0.8942	0.6448	0.7574
STCLN_wp (paper, pre-trained)	0.9086	0.6953	0.7974
T-SRDA, core network (ours)	0.8928	0.6895	0.7936
T-SRDA, full ensemble (ours; validation-selected variant)	0.8943	0.6869	0.7884

• Versus the paper's pre-trained STCLN_wp, ours is −0.006 mIoU, −0.004 mF1, −0.016 OA: close in mIoU/mF1, behind in OA; we do not beat it.
• Versus the paper's non-pre-trained STCLN, ours is +0.045 mIoU, +0.036 mF1 at equal OA (−0.001).
• The full ensemble (SAM + readouts + temporal CNNs + frozen TESSERA, PASTIS-tuned blend weights, not re-tuned) gives no gain over the core network on MTLCC.
• Single seed; the 0.006 mIoU gap to STCLN_wp is within the run-to-run noise we cannot resolve. Our fine-tuning recipe (the PASTIS one) is richer than STCLN's plain recipe, so the comparison mixes architecture and recipe. Details and caveats: MTLCC/REPORT.md.

Per-class IoU on the MTLCC test set:

Class	STCLN	STCLN_wp	T-SRDA (ours)
Sugar beet	0.8828	0.8904	0.8556
Summer oat	0.4205	0.5838	0.5765
Meadow	0.7463	0.7763	0.7712
Rape	0.9204	0.9135	0.9310
Hop	0.7152	0.8023	0.7079
Winter spelt	0.3434	0.3516	0.3531
Winter triticale	0.1863	0.2504	0.2464
Beans	0.7838	0.7890	0.7851
Peas	0.4358	0.3938	0.6312
Potatoes	0.7506	0.8372	0.8135
Soybeans	0.5505	0.6681	0.7558
Asparagus	0.6001	0.7842	0.7394
Winter wheat	0.8697	0.8695	0.8520
Winter barley	0.8611	0.8888	0.8154
Winter rye	0.2773	0.3416	0.2244
Summer barley	0.6885	0.7334	0.7293
Maize	0.9300	0.9454	0.9342

Ours is higher than STCLN on 12 of 17 classes and higher than STCLN_wp on 4 of 17 (largest gains vs STCLN_wp: peas +0.237, soybeans +0.088; largest losses: winter rye −0.117, hop −0.094, winter barley −0.073).

Pre-training took 10 h 46 min (100 epochs, 4,873 unlabelled tiles); fine-tuning 19 min; test evaluation under 1 min.

Reproduce: MTLCC/notebooks/TSRDA_MTLCC_Pretraining_to_Ensemble_Recipe.ipynb — one Run-All notebook (data download with disk guard → pre-training → fine-tuning → full ensemble → figures → comparison with the paper); SMOKE = True gives a minutes-long integration check.

---

PASTIS (original README, unchanged except that links now point into PASTIS/)

T-SRDA — current results and reproducible research

17 October 2026 — single supervisor notebook. The 58.63 and 60.44 notebooks are combined. Fill in only the PASTIS path; Run All performs scratch pretraining, downstream training, frozen TESSERA feature fitting and fresh evaluation. Instructions and verification. A fresh run may differ from the historical score.

Publication update — 12 October 2026. Highest directly evaluated full-method Test mIoU: 60.4429% (validation 65.8657%). Primary highest-validation-selected method: 66.3467% validation / 59.9973% test.

The new improvement comes from spatial statistics of frozen optical temporal features, not further encoder fine-tuning. The 60.4429% method adds a 5×5 local feature mean and standard deviation to every 256D pixel vector, fits a training-only 768D shrinkage-covariance Gaussian classifier, and blends it with the preceding 59.0010% ensemble. The validation-selected method instead performs three rounds of feature-affinity probability diffusion. The difference in selection status is retained throughout this release.

Reviewer-readiness audit: publication risks, completed audits, remaining controls and the recommended defensible claim. The standalone local-moments score was recovered as a post-hoc diagnostic (60.4573% test mIoU); the highest directly evaluated full method remains the 60.4429% blend. Neither is an independent test confirmation.

Current detailed release report · All 63 context experiments · Per-class scores and uncertainty · Current inference guide

Method	Validation mIoU %	Test mIoU %	Interpretation
Historical foundation ensemble	64.1505	58.6346	Historical validation-selected control
Quadratic readout, eta 0.05	64.4319	58.4853	Rejected: lower test score
Covariance shrinkage 0.03	64.3801	58.5651	Rejected: lower test score
5×5 neighborhood mean	65.3036	59.0010	Parent spatial baseline
Entropy-weighted diverse ensemble	64.6969	58.4377	Rejected: lower test score
Three-pass feature-affinity diffusion	66.3467	59.9973	Primary highest-validation selection
Mean of affinity and moments branches	66.2347	60.4334	Tested validation-promoted alternative
5×5 feature mean + standard deviation, blended	65.8657	60.4429	Highest directly evaluated full method
Standalone 5×5 local moments, recovered from saved blend	65.6812	60.4573	Post-hoc diagnostic; not independent confirmation
Summer-crop conditional specialist	65.6879	59.2033	Tested validation-promoted alternative

The supervised STCLN configuration remains 76 ordered training entries (72 unique, fold 1), 76 ordered validation entries (71 unique, fold 2), seed 3407, and two fixed 32×32 crops per entry: [0:32,0:32] and [32:64,32:64]. Duplicates remain included. There are 99,345 scored training pixel entries and 93,361 scored validation pixel entries, ten Sentinel-2 bands and twenty output columns. Metrics score ground-truth classes 1–18, excluding ground-truth 0/19. Full test evaluation uses 482 fold-4 native 128×128 patches and 3,961,114 scored pixels.

This matches the downloaded reference's supervised dataset configuration, not every experimental resource or evaluation choice. TESSERA is externally pretrained/distilled and uses acquisition-calendar inputs; its geographic pretraining overlap is uncertified. Seed 3407 does not imply an identical optimization trajectory or upstream pretraining seed. Context is restricted to each original 32×32 crop during fitting/validation and spans full 128×128 patches at test time. Repeated inspection of the same test fold makes these exploratory results. Best observed test performance must not be presented as a pristine validation-selected benchmark.

Preserved history through 11 October

The following sections describe the historical 58.6346% system, retained as a component and included in the unified supervisor notebook. Current scores and recipes are above.

T-SRDA — satellite time-series crop mapping

Latest audited result: 58.6346% Test mIoU · 64.1505% validation mIoU
Updated 11 October 2026. +8.3883 test percentage points over the preceding 50.2463% incumbent.

Main method, architecture and code · FINAL REPORT · All results · Ablations · Publication report

Complete executable notebook

Open the unified from-scratch pretraining → fine-tuning → 60.44% recipe notebook · Download .ipynb

The notebook includes 98 cells and 31 complete source files. Fill in the one blank PASTIS path and Run All: it pretrains the atrous/SRDA encoder from scratch, saves and uses that new checkpoint, trains every downstream branch, fits all heads and evaluates raw data. It automatically downloads only the frozen external TESSERA checkpoint. Historical 60.4429% is a recipe reference; a new training run can differ. The earlier DaViT/Swin encoder was replaced by Temporal Spatial-Reduction Dual-Attention in the supplied checkpoint; TESSERA's optical encoder remains frozen. Notebook guide and validation.

What actually improved the result?

We added frozen pretrained TESSERA optical temporal-Transformer features, fitted a Gaussian classifier with shared shrinkage covariance on the original training crops, and averaged its probabilities with our existing ensemble. The new encoder was pretrained/distilled by TESSERA's authors and was not fine-tuned here. Alone, this new branch achieves 57.1841% test mIoU; the validation-selected 50/50 blend reaches 58.6346%.

The earlier “above + new CNN control” remains inside the incumbent: it is a separately trained small temporal CNN with dilated convolutions and masked mean/max pooling, added to the spatial–temporal network, a cosine class-cluster classifier and an older temporal CNN. SAM means Sharpness-Aware Minimization, not Segment Anything. The main-method walkthrough explains the full chain, including that CNN.

Comparison scope: supervised sample lists, duplicates, folds, crop coordinates and local numerical seed match the downloaded STCLN code. This is an external-pretraining + acquisition-calendar track, not an identical-pretraining-budget STCLN comparison. External pretraining geographic disjointness is not certified. Checkpoint-specific normalization, temporal padding, selection criteria, ensemble size and confusion-matrix conventions also differ. Repeated test inspection makes the project exploratory; validation-frozen promotion in this round does not erase that history.

Measured results

Configuration	Validation mIoU	Test mIoU	Status
Original SAM optimizer network	50.0067%	47.5001%	Historical single-network baseline
Spatial network + cluster readout + old/new temporal CNNs	52.5640%	49.4555%	Historical validation-selected reference
Reference + Gaussian readout + structured self-supervised branch	53.3274%	50.2463%	Incumbent before foundation transfer
Frozen pretrained optical encoder + Gaussian readout alone	63.1343%	57.1841%	New standalone branch
50% incumbent + 50% pretrained optical branch	64.1505%	58.6346%	New validation-selected recommendation
50% incumbent + missing-radar fused pretrained branch	61.1794%	57.1967%	Tested alternative
Matched random optical encoder alone	35.0747%	—	Rejected on validation
Matched random fused encoder alone	26.7142%	—	Rejected by promotion gate

Final test metrics: pixel accuracy 84.8792%, macro F1 70.9305%, kappa 0.814272. Full 482-patch fold-4 evaluation; 3,961,114 scored pixels. Independent audit.

Dataset configuration preserved

Item	Verified configuration
Local fine-tuning / fitting seed	3407; not a claim of identical RNG trajectories or external pretraining seed
Training	76 ordered entries, 72 unique, fold 1; upstream duplicates retained
Validation	76 ordered entries, 71 unique, fold 2; upstream duplicates retained
Training / validation crops	Two fixed 32×32 windows: [0:32,0:32] and [32:64,32:64]; 152 crop entries per full pass
Classifier fit	99,345 foreground training pixel entries, including duplicated entries
Test	482 unique fold-4 patches, native 128×128, batch 1
Bands / labels	10 Sentinel-2 bands; 20 output columns; score classes 1–18, ignore ground truth 0 and 19
Test-time augmentation	None in this scored recipe
Original self-supervised pool	496 fold-5 patches; external TESSERA has its own different pretraining data

Exact lists and crop manifest · Fresh STCLN recheck. This is a same-supervised-sampling comparison, not a controlled claim of beating STCLN under equal external pretraining.

Architecture and training history

The supplied checkpoint supplies the spatial–temporal branch. Fine-tuning and subsequent cluster/CNN/statistical/structured-self-supervised experiments build the 50.2463% incumbent. The new frozen optical encoder adds a stronger per-pixel representation of seasonal multispectral trajectories. A fitted Gaussian classifier maps these features to eighteen crop classes. The old ensemble retains complementary context.

P_final = 0.50 P_incumbent + 0.50 P_TESSERA_optical_Gaussian
P_incumbent = 0.65 P_reference + 0.25 P_original_Gaussian + 0.10 P_structured_SSL
P_reference = 0.28125 P_SAM_optimizer + 0.28125 P_cosine_class_clusters
            + 0.1875 P_old_temporal_CNN + 0.25 P_new_temporal_CNN

These are nested probability averages, not logit averages. Preserve the nesting to reproduce floating-point arithmetic. SAM means Sharpness-Aware Minimization, not Segment Anything. Class clusters are several training-derived representative feature vectors per crop class, not extra semantic classes.

The GIF is explanatory, not a recording of training. See the detailed architecture, input adaptation and classifier mathematics.

Repository guide

main_method/
  README.md                           # Full method, mathematics, code map, reproduction
  code/                               # Portable current optical transfer + original model code
  configurations/32X32_tessera_transfer_seed3407/
  checkpoints/32X32_tessera_transfer_seed3407/
  checkpoints/shared/                 # Earlier ensemble components, preserved
ablations/
  01_... through 09_...                # Historical experiments, preserved
  10_plateau_breakthrough/workspace/   # ConvNeXt controls, all later rounds, logs/weights/caches
reports/oct11/                        # Fresh metrics, audits, CSV/JSON experiment ledger
docs/                                 # Research rationale, release report, glossary, histories
visuals/                              # New architecture, scores, per-class plot and schematic GIF
FINAL_REPORT.md                        # Full consolidated current and historical report
RESULTS.md                            # Results navigation

The original pretrained folders were not modified. All prior HF files remain. New negative results and checkpoints are retained rather than silently dropped. Read-only historical snapshot READMEs keep their original wording; active indexes link to the current result.

Reproduce and inspect

export PASTIS_ROOT=/path/to/PASTIS
python main_method/code/foundation_transfer.py verify
python main_method/code/foundation_transfer.py evaluate-cached --check-raw-labels --output /tmp/tsrda-replay
python main_method/code/foundation_transfer.py extract --split val --output /tmp/tessera-val

The first checks checkpoint hashes. The second independently replays the complete result from saved probabilities and checks raw targets. The third performs fresh optical-branch inference, not the full ensemble. See reproduction boundaries and full commands. The original portable evaluator remains for the historical 49.4555% reference.

Research evidence and remaining weaknesses

Seventeen classes improve. Largest gains include Sunflower +25.38, Winter barley +24.64, Winter triticale +17.32 and Mixed cereal +15.95 IoU points. Sorghum falls −5.62. Orchard, Sorghum, Winter triticale and Mixed cereal remain difficult.

Scratch class-query models overfit, short reconstruction SSL did not help them, calendar-only CNN changes did not beat the incumbent, and spatial refinements produced tiny validation gains that did not transfer to test. The matched random optical encoder obtains only 35.0747% validation versus 63.1343% pretrained. The decisive evidence supports representation transfer, not a generic optimizer trick or a proof of one particular pretraining objective. Detailed research interpretation.

Runtime, attribution and history

The final transfer pilot queue took 5.01 minutes, with cached incumbent predictions; this excludes download, implementation, independent audit and historical model training. See timings and limitations.

• TESSERA original research.
• TESSERA v2 research.
• Official TESSERA project.
• Pinned small-student release.

The bundled upstream model card states that the paper's reported student benchmarks use a different teacher release; they are not expected scores for this checkpoint. Our PASTIS scores are measured locally. The external model card declares CC0; original authors and provenance are retained.

Earlier detailed explanations and reports remain under history and the ablation index. The original artifact inventory describes the previous release; the new release inventory adds this update.

The self-contained supervisor notebook now displays live epoch/batch progress bars, elapsed time and measured stage ETA throughout pretraining, fine-tuning and evaluation. Progress and usage guide.

Supervisor notebook setup now provisions a compatible training Python independently of the Jupyter kernel. Its first code block reads your PASTIS metadata and selected array headers, verifies every selected ID/file/fold, and calculates actual fold/sample/crop counts and supervised proportions before training. Setup uses standalone uv without system pip/ensurepip.

The supervisor notebook preserves hosted NVIDIA driver paths, supports a CPU-only dataset audit, and provides check_gpu() diagnostics. Neural training still requires CUDA.