File size: 3,963 Bytes
3787357 77bf483 3787357 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 | ---
license: mit
tags:
- energy-demand-forecasting
- cnn-transformer
- iso-new-england
- time-series-forecasting
- multi-modal
language: en
library_name: pytorch
pipeline_tag: tabular-regression
---
# `baseline/best.pt` β Multi-Modal CNN-Transformer (Part 1 baseline)
**Headline:** 5.24 % test MAPE on the last 2 days of 2022 (8 ISO New England load zones, 24-hour day-ahead horizon).
## Model summary
A hybrid CNN-Transformer that fuses HRRR-style weather rasters with per-zone demand history and 44-d calendar features into a unified token sequence, then decodes 24 hourly per-zone demand values for all 8 ISO-NE zones.
| Field | Value |
|---|---|
| Architecture | CNN-Transformer (joint encoder over unified sequence) |
| Parameters | **1,753,200** (1.75 M) |
| Spatial token grid | 8 Γ 8 (P = 64 spatial tokens per timestep) |
| Sequence length | (S+24) Β· (P+1) = 48 Β· 65 = **3,120** tokens |
| Embedding dim D | 128 |
| Transformer layers | 4 encoder, 4 heads, MLP ratio 4, pre-norm |
| Total `epoch` at best | 13 (continuous training, no chained resume) |
| Best val MAPE | 6.92 % on val 2021 |
| **Test MAPE (2022-12-30/31)** | **5.24 %** |
| File size | 21 MB |
| SHA256 | `91069db5bc8f93f832aa0a4e4fb600f075ef382617049225d828003c99ae05c0` |
## Per-zone test MAPE (last 2 days of 2022)
| Zone | MAPE |
|---|---|
| ME | **2.31 %** β |
| NH | 3.69 % |
| VT | 5.95 % |
| CT | 7.28 % |
| RI | 5.27 % |
| SEMA | 5.44 % |
| WCMA | 5.87 % |
| NEMA_BOST | 6.09 % |
| **Overall** | **5.24 %** |
## Inputs
- **Weather rasters** `X β β^{(S+24) Γ 7 Γ 450 Γ 449}` β HRRR-style 7-channel hourly snapshots (S = 24 history hours, 24 future hours)
- **Per-zone demand** `Y β β^{S Γ 8}` β historical MWh demand for the 8 ISO-NE zones
- **Calendar features** `C β β^{(S+24) Γ 44}` β one-hot hour (24) + day-of-week (7) + month (12) + US-holiday flag (1)
## Outputs
- 24-hour day-ahead per-zone demand forecast `ΕΆ β β^{24 Γ 8}` in MWh
## Loading
```python
import torch
from models.cnn_transformer_baseline import CNNTransformerBaselineForecaster
ckpt = torch.load("pretrained_models/baseline/best.pt",
map_location="cpu", weights_only=False)
args = ckpt["args"]
model = CNNTransformerBaselineForecaster(
n_weather_channels=7, n_zones=8, cal_dim=44,
history_len=args["history_len"], # 24
embed_dim=args["embed_dim"], # 128
grid_size=args["grid_size"], # 8
n_layers=args["n_layers"], # 4
n_heads=args["n_heads"], # 4
dropout=args["dropout"], # 0.1
)
model.load_state_dict(ckpt["model"])
model.eval()
norm_stats = ckpt["norm_stats"] # {weather_mean, weather_std, energy_mean, energy_std}
```
## Training
- Optimizer: AdamW, base LR 1e-3, weight decay 1e-4
- LR schedule: CosineAnnealingLR (T_max = 14 epochs, no chained resume)
- Loss: MSE in z-score space (per the four-step normalization chain)
- Validation: MAPE in physical MWh space, per-zone + overall
- Batch size: 4 per A100 GPU
- Hardware: A100 40 GB
- Wall time: ~22 hours
- Train years: 2019β2020
- Validation year: 2021
- Self-eval test slice: 2022-12-30 to 2022-12-31
## Limitations
1. Test numbers are on a 2-window slice; small-sample variance non-negligible.
2. The CNN trunk is a fixed 5-stage residual stack; spatial-encoder design space not explored.
3. Random seeds (`torch.manual_seed` / `np.random.seed`) are NOT set in the training pipeline β headline MAPE is not bit-reproducible across re-training runs. Empirical claims are pinned to **this specific** checkpoint.
## Citation
```
Liu, Pang. "Multi-Modal Deep Learning for Energy Demand Forecasting"
(real-time-power-predict v1.5, 2026).
GitHub: https://github.com/jeffliulab/real-time-power-predict
```
## License
MIT (see top-level LICENSE file in the repo).
|