Improve model card and publish dated expanded benchmark progress

README.md

@@ -9,6 +9,10 @@ tags:
 
 # Trackformer 1.2
 
+**[Download 1.2: weights + code](https://github.com/yu314-coder/typhoon-predict/releases/download/trackformer-1.2/trackformer_1_2_field_20260929.tar.gz)** · **[Read the illustrated paper](paper/trackformer.pdf)** · **[Watch Fung-wong](https://yu314-coder.github.io/typhoon-predict/trackformer_1_2_fung_wong.mp4)** · **[Hugging Face model](https://huggingface.co/euler314/typhoon-predict)**
+
+Release status: **research prerelease**, not operational certification. The package contains the released 1.2 weights, inference code, input contract, saved evaluation data and illustrated paper. The original 1.1 release remains separate.
+
 Trackformer 1.2 is a **research-only Western Pacific tropical-cyclone forecast model**. It evolves a sea-level-pressure (MSLP) field and a moving storm-centred pressure core every six hours through +120 h. A track and central-pressure estimate are extracted from the evolving core, not independently drawn on top of a pressure image. The prior [Trackformer 1.1 release](https://github.com/yu314-coder/typhoon-predict/releases/tag/trackformer-1.1) remains available and unchanged.
 
 **Not an operational warning system.** Do not use these forecasts for evacuation, aviation, maritime, or other safety-critical decisions. The public 1.2 name identifies one selected development checkpoint; a genuinely untouched storm-level holdout has not yet established generalization.
@@ -35,6 +39,21 @@ The two models use different input pipelines. A direct chart is therefore a *for
 
 The implementation is in [`models/trackformer_1_2_field/`](models/trackformer_1_2_field/); its [`manifest.json`](models/trackformer_1_2_field/manifest.json) records exact grids, normalization, architecture and SHA-256 provenance. Internally the selected checkpoint came from experiment version `1.2.73`, epoch 4. The public model is **Trackformer 1.2**; that internal identifier is retained only for reproducibility.
 
+### Architecture at a glance
+
+| Component | Released configuration |
+| --- | --- |
+| Trainable parameters | 21,452,595 |
+| Input history | Nine analyses, −48 to 0 h; basin tensor 9 × 8 × 25 × 33 |
+| Basin evolution network | Widths 72 / 144 / 288 / 432; residual depths 2 / 2 / 4 / 4 |
+| Core evolution network | Widths 64 / 128 / 256 / 384; moving 65 × 65 pressure grid, 20-km spacing |
+| Environmental attention | 134 pooled tokens; width 64; four heads; two Transformer blocks; 825 basin queries |
+| Recurrent memory | 32 channels for each of basin and core |
+| Geographic readout | Pressure-associated centre within 300 km; central pressure sampled at that centre |
+| Five-day forecast | 20 autoregressive six-hour transitions; future observations excluded |
+
+Grid spacing describes the representation, not independently demonstrated effective resolution. A mean of member centres is not necessarily the minimum of the displayed mean pressure field.
+
 ## 270-case comparison: direction, route shape and position
 
 This comparison uses **270 forecast cases from 90 storms**, with the same issue rows, observed routes and 20 six-hour leads through +120 h. The number 270 counts forecast cases. The 1.2 prediction for each case is a **mean of 50 causal input-perturbation members**; 1.1 uses its archived causal route pipeline.
@@ -65,7 +84,21 @@ The 1.2 mean has better aggregate direction, shape and position scores on this p
 
 ### Expanded daily-issue benchmark: 270 distinct typhoons
 
-A new **1,473-case / 270-storm** benchmark is in progress; it is not the completed 270-case result above. Each storm contributes at most one forecast per UTC day, through +120 h. Daily errors are averaged within each storm, then the 270 storm scores receive equal weight. Selection was frozen before new inference, without filtering on forecast quality. See the [protocol](docs/daily_storm_benchmark.md) and [frozen cohort](evaluation/release_data/daily_storm_cohort.json). Results are pending; no improvement is claimed from the partial run.
+A **1,473-case / 270-storm** benchmark is in progress; it is separate from the completed 270-case comparison above. Each storm contributes at most one forecast per UTC day, through +120 h. Daily errors are averaged within each storm, then storm scores receive equal weight. Selection was frozen before inference without filtering on forecast quality. See the [protocol](docs/daily_storm_benchmark.md) and [frozen cohort](evaluation/release_data/daily_storm_cohort.json).
+
+**Dated snapshot — 2026-09-29T06:09:06.508182+00:00 (UTC), not a live counter:** 808/1,473 daily forecasts saved; **159/270 storms fully completed**. The 1.2 forecasts use 50 causal input-perturbation members. [Machine-readable snapshot](evaluation/daily_storm_progress_20260929.json).
+
+| Preliminary equal-storm metric | 1.1 | 1.2 · mean of 50 |
+| --- | ---: | ---: |
+| Mean track error, +6 to +120 h ↓ | 843.9 km | 487.1 km |
+| Track error at +120 h ↓ | 1733.3 km | 1068.8 km |
+| Six-hour direction error ↓ | 51.81° | 35.33° |
+| Centred route-shape similarity ↑ | 0.7497 | 0.8842 |
+| Geographic path similarity ↑ | 0.5310 | 0.6611 |
+
+**Incomplete development evidence:** only the 159 fully completed storms enter this table. Execution order is not random, and the remaining storms can change the result. These are not the final 270-storm scores or an untouched-test claim. Do not compare this subset directly with the differently sampled 270-case table above.
+
+1.2-only pressure diagnostics: central-pressure MAE **12.62 hPa** over 40 storms with valid pressure labels; basin MSLP MAE **2.70 hPa** over 159 storms. There is no matched 1.1 pressure result in this run, and basin-wide error does not establish core-pressure-map accuracy.
 
 ## Fung-wong pressure forecast — MP4
 
@@ -101,6 +134,18 @@ This recent example is **one deterministic forecast**, separate from the benchma
 
 The inference-only `weights.pt` is hosted in the release/Hugging Face model folder, **not committed to GitHub source**. The manifest records its SHA-256 and the original training-checkpoint SHA-256. The three source modules are copied unchanged from the verified training implementation. Use **`models/trackformer_1_2_field/predict.py`** as the inference entry point. The withdrawn route/scalar candidate and its incompatible example have been removed from the current source tree.
 
+### Run one forecast
+
+Download and extract the complete package above, or obtain the source and place the [Hugging Face weights](https://huggingface.co/euler314/typhoon-predict/resolve/main/models/trackformer_1_2_field/weights.pt) at `models/trackformer_1_2_field/weights.pt`. In an environment with PyTorch and NumPy, prepare a normalized **causal issue packet** using the [input schema](models/trackformer_1_2_field/README.md), then run from the package root:
+
+```bash
+python models/trackformer_1_2_field/predict.py causal_issue_packet.npz forecast.npz --device mps
+```
+
+Use `--device cuda` for a compatible NVIDIA setup or `--device cpu` for CPU inference. `causal_issue_packet.npz` is a user-prepared input, not a bundled example or automatic live-data download. Output includes latitude/longitude track, central pressure and basin/regional MSLP arrays in hPa through +120 h. **This command produces one clean forecast, not the benchmark's 50-member mean.**
+
+Released weight SHA-256: `db49f36e85a3766defc4c172746897a1f783705d1ce8e6f9dfb8e87ae1d902cb`.
+
 Only issue-time and earlier analyses are permitted. The prediction wrapper rejects future-dated history and unexpected input keys, but cannot certify an externally built packet's data provenance. The operator must verify source timestamps and training-only normalization. The included wrapper runs one unperturbed forecast; it does not reproduce the saved 50-member mean automatically.
 
 ## Repository guide

RELEASE_NOTES_TRACKFORMER_1_2.md

@@ -1,5 +1,7 @@
 # Trackformer 1.2 — research candidate
 
+**[Download weights + code](https://github.com/yu314-coder/typhoon-predict/releases/download/trackformer-1.2/trackformer_1_2_field_20260929.tar.gz)** · **[Download the illustrated PDF](https://github.com/yu314-coder/typhoon-predict/releases/download/trackformer-1.2/trackformer.pdf)** · **[Hugging Face model card](https://huggingface.co/euler314/typhoon-predict)**
+
 This release packages a verified moving-pressure-core model as **Trackformer 1.2**. It supersedes the withdrawn, unrelated route/scalar 1.2 candidate. Use `models/trackformer_1_2_field/predict.py`; the incompatible root-level candidate and example are no longer in the current source tree. Trackformer 1.1 remains available as a separate release.
 
 The attached archive includes inference-only weights, exact source modules, an input wrapper, a data-contract/provenance manifest, detailed documentation, **270-case direction/route-shape benchmark bars**, paired route examples and recent **Surigae pressure maps with labelled isobars**. The training checkpoint was internally labelled version `1.2.73`, epoch 4; its SHA-256 is `f194a23d3f91ea76ad776dfad942fabd669eeae8b3fd665815463095367e9ee0`. The exported inference weights SHA-256 is `db49f36e85a3766defc4c172746897a1f783705d1ce8e6f9dfb8e87ae1d902cb`.
@@ -15,3 +17,7 @@ The upgraded **illustrated technical paper** includes the detailed model archite
 The Surigae illustration is a single forecast issued 2026-09-27 12 UTC, with 4 hPa isobars at +6/+24/+36 h. It is separate from the 50-member historical benchmark. Reproduction data and metric definitions are included. The model weights are unchanged in this documentation revision.
 
 This is not an operational warning service. Do not use for safety-critical decisions. A genuinely untouched storm-level holdout is still required before generalization claims.
+
+## Expanded benchmark progress (dated snapshot)
+
+At 2026-09-29T06:09:06.508182+00:00, 808/1,473 daily forecasts and 159/270 storms were complete. This is an incomplete development snapshot, not final validation. The README includes separate preliminary direction, shape and position metrics and pressure-label coverage. The machine-readable snapshot is `evaluation/daily_storm_progress_20260929.json`. Model weights and published paper are unchanged in this documentation update.

evaluation/daily_storm_progress_20260929.json

@@ -0,0 +1,81 @@
+{
+  "status": "running_partial",
+  "updated_utc": "2026-09-29T06:09:06.508182+00:00",
+  "cohort_sha256": "965184f1e5ab3fcb52e304f39b38583a14f753870aa2295f2d01942a441dcfd6",
+  "target_storm_count": 270,
+  "target_daily_cases": 1473,
+  "completed_daily_cases": 808,
+  "completed_storms": 159,
+  "aggregation": "Mean across 20 leads within each daily issue; mean of daily scores within each storm; equal-weight mean of storm scores",
+  "evaluation_label": "Broader development benchmark. Excludes 1.2 fitting/validation years; not certified untouched for all historical model versions.",
+  "aggregate_equal_storm": {
+    "1.1": {
+      "mean_track_error_km": 843.8621114511382,
+      "track_error_120h_km": 1733.2726378192087,
+      "direction_error_deg": 51.80990739742571,
+      "shape_similarity": 0.7497360046168909,
+      "frechet_distance_km": 1747.8702008094688,
+      "path_similarity": 0.5310399910204798,
+      "track_error_by_lead_km": [
+        70.07905921221005,
+        141.47157741053823,
+        213.71663132065547,
+        288.0827303125307,
+        366.6206773075269,
+        443.71663052326375,
+        522.0229150940693,
+        600.2095479405816,
+        680.9489001185718,
+        765.3511050266936,
+        851.273415712845,
+        937.8791152207183,
+        1029.786198881821,
+        1123.316585567243,
+        1219.8504751458606,
+        1317.1924939609435,
+        1420.7865879143453,
+        1523.7928891051765,
+        1627.8720554279569,
+        1733.2726378192087
+      ]
+    },
+    "1.2": {
+      "mean_track_error_km": 487.07502229168585,
+      "track_error_120h_km": 1068.7845200065838,
+      "direction_error_deg": 35.32516513287079,
+      "shape_similarity": 0.8841944328847995,
+      "frechet_distance_km": 1081.361206304057,
+      "path_similarity": 0.6610640343692109,
+      "track_error_by_lead_km": [
+        49.320702362800006,
+        91.93377381158523,
+        128.56644864243074,
+        163.48005333561366,
+        198.97070599127827,
+        237.04318919785257,
+        276.97202860768,
+        318.34651333401285,
+        363.26530711018916,
+        411.3252930097807,
+        464.2790038446656,
+        519.9047678712388,
+        578.6886955998453,
+        640.53742606808,
+        706.620202015708,
+        772.7616953132481,
+        843.2899944159193,
+        915.1638893217782,
+        992.2462359734283,
+        1068.7845200065842
+      ]
+    }
+  },
+  "pressure_equal_storm": {
+    "v12_pressure_mae_hpa": 12.620734815892849,
+    "v12_pressure_mae_hpa_valid_storms": 40,
+    "v12_basin_area_weighted_mslp_mae_hpa": 2.696000601689426,
+    "v12_basin_area_weighted_mslp_mae_hpa_valid_storms": 159
+  },
+  "pressure_note": "1.2-only diagnostics; no matched 1.1 central-pressure comparison is claimed. Basin fields are on a fixed common grid.",
+  "snapshot_note": "Dated partial development snapshot, not live status or final results. Only fully completed storms enter aggregates; execution order is not random."
+}