File size: 3,731 Bytes
eb41275 8569405 eb41275 7c089a9 8244d5f 7c089a9 2a2043f 9ff1f4e 2176e4b 7ef8760 2176e4b 2a2043f 800d3a7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | ---
license: mit
tags:
- weather
- tornado
- tornado-detection
- tornadoes
metrics:
- recall
- precision
---
# NOTE: **ALL** ten folds are in the ensemblev8+v9.zip, the five are in the zip for v8 file. A like would be appreciated!
### 📈 Model Comparison Table
| Model Name | Threshold | Model parameters | AUC | AUCPR | BinaryAccuracy | TruePositives | FalsePositives | TrueNegatives | FalseNegatives | Precision | Recall | FalseAlarmRate | F1 | ThreatScore |
|-------------------|-----------|------------------|-------|--------|----------------|---------------|----------------|---------------|----------------|-----------|--------|----------------|-------|-------------|
| baseline | 0.0101 | 4665409 | 0.8742 | 0.5349 | 0.9456 | 915 | 635 | 28841 | 1076 | 0.5903 | 0.4596 | 0.4097 | 0.5168 | 0.3484 |
| WRN | 0.4444 | 229759 | 0.882 | 0.5498 | 0.9456 | 973 | 693 | 28783 | 1018 | 0.5840 | 0.4887 | 0.416 | 0.5321 | 0.3625 |
| gated | 0.5353 | 514905 | 0.8862 | 0.5574 | 0.9436 | 976 | 761 | 28715 | 1015 | 0.5619 | 0.4902 | 0.4381 | 0.5236 | 0.3547 |
| WRN+Gated | 0.4898 | 744664 | 0.8928 | 0.5705 | 0.9474 | 975 | 640 | 28836 | 1016 | 0.6037 | 0.4897 | 0.3963 | 0.5408 | 0.3706 |
| PReLU_textbook | 0.4242 | 102524 | 0.8976 | 0.5853 | 0.9481 | 1032 | 673 | 28803 | 959 | 0.6053 | 0.5183 | 0.3947 | 0.5584 | 0.3874 |
| old_ensemble | 0.3899 | 512620 | 0.9047 | 0.5947 | 0.9479 | 1020 | 669 | 28807 | 971 | 0.6039 | 0.5123 | 0.3961 | 0.5543 | 0.3835 |
| ensemble_V8TH | 0.3999 | 1050714 | 0.911 | 0.6179 | 0.9497 | **1068** | 659 | 28817 | **923** | 0.6184 | **0.5364** | 0.3816 | **0.5745** | **0.403** |
| ensemble_v9 | 0.46 | 1379898 | 0.9115 | 0.6143 | 0.9492 | 1038 | 644 | 28832 | 953 | 0.6171 | 0.5213 | 0.3829 | 0.5652 | 0.3939 |
| ensemble_v8+v9 | 0.4353 | **2430612** | **0.9128** | **0.6195** | **0.9504** | 1050 | **621** | **28855** | 941 | **0.6284** | 0.5274 | **0.3716** | 0.5735 | 0.402 |
## Cross Val ensemble Table
| Name | seed | Params | fold 1 | fold 2 | fold 3 | fold 4 | fold 5 | Average |
|-------------------------|--------|--------|--------|--------|--------|--------|--------|---------|
| baseline | | 4.5M | 0.5121 | 0.4405 | 0.4647 | 0.4544 | 0.4451 | 0.4634 |
| WRN | | 230k | 0.6184 | 0.5546 | 0.5986 | 0.6152 | 0.6072 | 0.5988 |
| Gated | | 600k | 0.6401 | 0.5516 | 0.5247 | 0.5878 | 0.5716 | 0.5752 |
| PReLU | | 102k | 0.6314 | 0.6210 | 0.6006 | 0.6266 | 0.6132 | 0.6186 |
| ensemble_v8 | | | 0.6338 | **0.6359** | 0.6141 | 0.6342 | **0.6211** | 0.62782 |
| fold 1 script cross val | | ~300k | **0.6627** | 0.6312 | **0.6171** | **0.6488** | 0.6187 | **0.6357** |
## Tornado Detection Model
Instructions on evaluation ensemble usage can be found at: https://github.com/Higgs32584/tornet-Higgins
Instructions for downloading the data can be found at: https://github.com/mit-ll/tornet |