Publish MargaDrishti models, cards and reports

model_cards/A1.md

@@ -8,34 +8,36 @@ How many parking violations will this H3 cell see in this hour?
 
 ## Caveats
 
-- Counts are enforcement observations, not violation occurrences: 94% of per-cell variance is explained by patrol history (F6).
+- Counts are enforcement observations, not violation occurrences: 93% of per-cell variance is explained by patrol history (F6, r=0.96 across all 2,534 res-9 cells).
 - Poisson deviance is the ranking metric, not MAE. On a 97%-zero target MAE is minimised by the conditional median (zero), so ranking by it rewards under-prediction.
 
 ## Results
 
-Ranked by **poisson_deviance** (lower is better). Every family that ran is listed, including those that lost — the comparison is the deliverable.
-
-| Model | Family | poisson_deviance | Baseline? | Beats best baseline |
-|---|---|---:|---|---|
-| `xgboost` | gbdt | 0.4999 |  | yes |
-| `hist_gbm_poisson` | gbdt | 0.5042 |  | yes |
-| `lightgbm` | gbdt | 0.5055 |  | yes |
-| `stacked` | ensemble | 0.5071 |  | yes |
-| `extra_trees` | trees | 0.5086 |  | yes |
-| `catboost` | gbdt | 0.5162 |  | yes |
-| `hist_gbm` | gbdt | 0.5189 |  | yes |
-| `poisson_glm` | linear | 0.5668 |  | yes |
-| `torch_mlp` | deep | 0.5787 |  | yes |
-| `random_forest` | trees | 0.5793 |  | yes |
-| `ridge` | linear | 0.6026 |  | yes |
-| `bayesian_hierarchical` | bayesian | 0.6207 |  | yes |
-| `torch_lstm` | deep | 0.6279 |  | yes |
-| `graphsage` | gnn | 0.7767 |  | yes |
-| `mean` | baseline | 0.8775 | baseline | — |
-| `cell_hour_mean` | baseline | 1.3910 | baseline | — |
-| `seasonal_naive` | baseline | 3.5434 | baseline | — |
-
-**Best learned family: `xgboost`** at 0.4999, +43.0% against the strongest baseline (0.8775).
+Ranked by **Deviance ↓** (lower is better). Every family that ran is listed, including those that lost — the comparison is the deliverable.
+
+| Model | Family | Deviance ↓ | P@10 ↑ | NDCG@10 ↑ | τ ↑ | MAE ↓ | Fit | Beats baseline |
+|---|---|---:|---:|---:|---:|---:|---:|---|
+| `xgboost` | gbdt | 0.4999 | 0.9000 | 0.9902 | 0.7439 | 0.1656 | 21s | yes |
+| `hist_gbm_poisson` | gbdt | 0.5042 | 0.9000 | 0.9902 | 0.7540 | 0.1641 | 18s | yes |
+| `lightgbm` | gbdt | 0.5055 | 0.9000 | 0.9817 | 0.6880 | 0.1672 | 15s | yes |
+| `stacked` | ensemble | 0.5071 | 0.9000 | 0.9912 | 0.7502 | 0.1713 | 288s | yes |
+| `extra_trees` | trees | 0.5086 | 1.0000 | 0.9965 | 0.7275 | 0.1784 | 191s | yes |
+| `catboost` | gbdt | 0.5162 | 0.9000 | 0.9893 | 0.7339 | 0.1614 | 33s | yes |
+| `hist_gbm` | gbdt | 0.5189 | 0.9000 | 0.9916 | 0.7805 | 0.1713 | 6s | yes |
+| `poisson_glm` | linear | 0.5668 | 0.9000 | 0.9910 | 0.6457 | 0.1724 | 7s | yes |
+| `torch_mlp` | deep | 0.5787 | 1.0000 | 0.9704 | 0.7273 | 0.1287 | 26s | yes |
+| `random_forest` | trees | 0.5793 | 1.0000 | 0.9977 | 0.4064 | 0.2305 | 696s | yes |
+| `ridge` | linear | 0.6026 | 1.0000 | 0.9996 | 0.7722 | 0.1777 | 5s | yes |
+| `bayesian_hierarchical` | bayesian | 0.6207 | 0.8000 | 0.9522 | 0.5264 | 0.1815 | 27s | yes |
+| `torch_lstm` | deep | 0.6279 | 0.8000 | 0.9704 | 0.8810 | 0.1883 | 154s | yes |
+| `graphsage` | gnn | 0.7767 | 0.7000 | 0.9468 | 0.6260 | 0.1228 | 38s | yes |
+| `mean` | baseline | 0.8775 | 0.0000 | 0.0118 | — | 0.2191 | 0s | *baseline* |
+| `cell_hour_mean` | baseline | 1.3910 | 0.9000 | 0.9641 | 0.4254 | 0.1813 | 0s | *baseline* |
+| `seasonal_naive` | baseline | 3.5434 | 0.9000 | 0.9893 | 0.7874 | 0.1809 | 0s | *baseline* |
+
+**Best learned family: `xgboost`** at 0.4999, +43.0% against the strongest baseline (`mean` at 0.8775).
+
+**The ranking metrics do not order these families the way Deviance ↓ does.** `xgboost` wins on Deviance ↓ with precision@10 of 0.90, while `extra_trees` reaches 1.00. A baseline reaches 0.90. Placing the worst cells at the top of a list is a much easier task than predicting their counts, so a card that reported only Deviance ↓ would overstate how much the winner buys you for the way this system is actually used.
 
 ## Reproduction
 

model_cards/A2.md

@@ -8,26 +8,26 @@ Is this cell-hour in the top decile of violation intensity?
 
 ## Caveats
 
-- The original goal of PR-AUC >= 0.45 was unreachable by construction (F10): it assumed ~10% prevalence, but only 2.94% of cell-hours are non-zero, so the label lands at 0.33%.
-- Read scores against the 0.29% base rate, not against 0.45.
+- The original goal of PR-AUC >= 0.45 was unreachable by construction (F10): it assumed ~10% prevalence, but only 2.94% of cell-hours are non-zero, so the label lands at 0.291%.
+- Read scores against the 0.291% base rate, not against 0.45. The best learned family is a 46.9x lift over it.
 
 ## Results
 
-Ranked by **pr_auc** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable.
-
-| Model | Family | pr_auc | Baseline? | Beats best baseline |
-|---|---|---:|---|---|
-| `random_forest` | trees | 0.1365 |  | yes |
-| `catboost` | gbdt | 0.1305 |  | yes |
-| `xgboost` | gbdt | 0.1281 |  | yes |
-| `extra_trees` | trees | 0.1218 |  | yes |
-| `stacked` | ensemble | 0.1200 |  | yes |
-| `logistic` | linear | 0.1127 |  | yes |
-| `hist_gbm` | gbdt | 0.0881 |  | yes |
-| `lightgbm` | gbdt | 0.0357 |  | yes |
-| `prior` | baseline | 0.0029 | baseline | — |
-
-**Best learned family: `random_forest`** at 0.1365, +4594.1% against the strongest baseline (0.0029).
+Ranked by **PR-AUC ↑** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable.
+
+| Model | Family | PR-AUC ↑ | ROC-AUC ↑ | Brier ↓ | Best F1 ↑ | Fit | Beats baseline |
+|---|---|---:|---:|---:|---:|---:|---|
+| `random_forest` | trees | 0.1365 | 0.9380 | 0.0027 | 0.2312 | 64s | yes |
+| `catboost` | gbdt | 0.1305 | 0.9341 | 0.0027 | 0.2203 | 35s | yes |
+| `xgboost` | gbdt | 0.1281 | 0.9368 | 0.0027 | 0.2153 | 20s | yes |
+| `extra_trees` | trees | 0.1218 | 0.9362 | 0.0027 | 0.2138 | 23s | yes |
+| `stacked` | ensemble | 0.1200 | 0.9363 | 0.0028 | 0.1902 | 233s | yes |
+| `logistic` | linear | 0.1127 | 0.9204 | 0.0027 | 0.2074 | 11s | yes |
+| `hist_gbm` | gbdt | 0.0881 | 0.9128 | 0.0031 | 0.1896 | 4s | yes |
+| `lightgbm` | gbdt | 0.0357 | 0.9229 | 0.0054 | 0.0926 | 14s | yes |
+| `prior` | baseline | 0.0029 | 0.5000 | 0.0029 | 0.0058 | 0s | *baseline* |
+
+**Best learned family: `random_forest`** at 0.1365, 46.9x the strongest baseline (`prior` at 0.0029).
 
 ## Reproduction
 

model_cards/A3.md

@@ -13,21 +13,21 @@ Will this field capture be rejected at officer review?
 
 ## Results
 
-Ranked by **pr_auc** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable.
-
-| Model | Family | pr_auc | Baseline? | Beats best baseline |
-|---|---|---:|---|---|
-| `xgboost` | gbdt | 0.3088 |  | yes |
-| `stacked` | ensemble | 0.3084 |  | yes |
-| `catboost` | gbdt | 0.2987 |  | yes |
-| `hist_gbm` | gbdt | 0.2955 |  | yes |
-| `random_forest` | trees | 0.2919 |  | yes |
-| `logistic` | linear | 0.2893 |  | yes |
-| `lightgbm` | gbdt | 0.2838 |  | yes |
-| `extra_trees` | trees | 0.2825 |  | yes |
-| `prior` | baseline | 0.2459 | baseline | — |
-
-**Best learned family: `xgboost`** at 0.3088, +25.6% against the strongest baseline (0.2459).
+Ranked by **PR-AUC ↑** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable.
+
+| Model | Family | PR-AUC ↑ | ROC-AUC ↑ | Brier ↓ | Best F1 ↑ | Fit | Beats baseline |
+|---|---|---:|---:|---:|---:|---:|---|
+| `xgboost` | gbdt | 0.3088 | 0.5750 | 0.7122 | 0.3962 | 2s | yes |
+| `stacked` | ensemble | 0.3084 | 0.5796 | 0.4401 | 0.4023 | 48s | yes |
+| `catboost` | gbdt | 0.2987 | 0.5630 | 0.6817 | 0.4033 | 4s | yes |
+| `hist_gbm` | gbdt | 0.2955 | 0.5575 | 0.7166 | 0.3960 | 4s | yes |
+| `random_forest` | trees | 0.2919 | 0.5683 | 0.5619 | 0.4040 | 2s | yes |
+| `logistic` | linear | 0.2893 | 0.5642 | 0.7211 | 0.4068 | 0s | yes |
+| `lightgbm` | gbdt | 0.2838 | 0.5522 | 0.7250 | 0.3954 | 2s | yes |
+| `extra_trees` | trees | 0.2825 | 0.5544 | 0.4573 | 0.4027 | 1s | yes |
+| `prior` | baseline | 0.2459 | 0.5000 | 0.1881 | 0.3947 | 0s | *baseline* |
+
+**Best learned family: `xgboost`** at 0.3088, +25.6% against the strongest baseline (`prior` at 0.2459).
 
 ## Reproduction
 

model_cards/B1.md

@@ -13,23 +13,32 @@ How long until this event is cleared?
 
 ## Results
 
-Ranked by **poisson_deviance** (lower is better). Every family that ran is listed, including those that lost — the comparison is the deliverable.
-
-| Model | Family | poisson_deviance | Baseline? | Beats best baseline |
-|---|---|---:|---|---|
-| `xgboost` | gbdt | 0.4990 |  | yes |
-| `random_forest` | trees | 0.5034 |  | yes |
-| `extra_trees` | trees | 0.5084 |  | yes |
-| `lightgbm` | gbdt | 0.5119 |  | yes |
-| `hist_gbm` | gbdt | 0.5322 |  | yes |
-| `hist_gbm_poisson` | gbdt | 0.5363 |  | yes |
-| `ridge` | linear | 0.5401 |  | yes |
-| `poisson_glm` | linear | 0.5445 |  | yes |
-| `catboost` | gbdt | 0.5468 |  | yes |
-| `mean` | baseline | 0.6904 | baseline | — |
-| `torch_mlp` | deep | 4.4045 |  | **no** |
-
-**Best learned family: `xgboost`** at 0.4990, +27.7% against the strongest baseline (0.6904).
+Ranked by **Deviance ↓** (lower is better). Every family that ran is listed, including those that lost — the comparison is the deliverable. Figures are means over 5 chronological folds, ± one standard deviation.
+
+| Model | Family | Deviance ↓ | MAE min ↓ | RMSE ↓ | R² ↑ | Fit | Beats baseline |
+|---|---|---:|---:|---:|---:|---:|---|
+| `xgboost` | gbdt | 0.4990 ± 0.128 | 538.6806 | 1.4936 | 0.2428 | 1s | yes |
+| `random_forest` | trees | 0.5034 ± 0.134 | 546.6247 | 1.5039 | 0.2350 | 0s | yes |
+| `extra_trees` | trees | 0.5084 ± 0.122 | 551.1427 | 1.5178 | 0.2170 | 0s | yes |
+| `lightgbm` | gbdt | 0.5119 ± 0.133 | 549.3335 | 1.5196 | 0.2177 | 1s | yes |
+| `hist_gbm` | gbdt | 0.5322 ± 0.120 | 549.0341 | 1.5411 | 0.1887 | 4s | yes |
+| `hist_gbm_poisson` | gbdt | 0.5363 ± 0.152 | 552.7476 | 1.5548 | 0.1832 | 1s | yes |
+| `ridge` | linear | 0.5401 ± 0.151 | 554.2421 | 1.5587 | 0.1781 | 0s | yes |
+| `poisson_glm` | linear | 0.5445 ± 0.160 | 555.4239 | 1.5670 | 0.1703 | 0s | yes |
+| `catboost` | gbdt | 0.5468 ± 0.112 | 538.7609 | 1.5571 | 0.1721 | 2s | yes |
+| `mean` | baseline | 0.6904 ± 0.213 | 554.9495 | 1.7510 | -0.0338 | 0s | *baseline* |
+| `torch_mlp` | deep | 4.4045 ± 3.908 | 14098.0595 | 10.6682 | -54.3918 | 1s | **no** |
+
+**Best learned family: `xgboost`** at 0.4990, +27.7% against the strongest baseline (`mean` at 0.6904).
+
+### Scored on their own metric, not on poisson_deviance
+
+These ran and are reported; the target's primary metric simply does not apply to them.
+
+| Model | Family | Metric | Value |
+|---|---|---|---:|
+| `lightgbm_q50` | quantile | Pinball ↓ | 0.5658 ± 0.090 |
+| `lightgbm_q90` | quantile | Pinball ↓ | 0.2343 ± 0.040 |
 
 ## Reproduction
 

model_cards/B2.md

@@ -8,24 +8,24 @@ Will this event require a road closure?
 
 ## Caveats
 
-- 8.3% positive class. Read PR-AUC against that prevalence.
+- 8.727% positive class. Read PR-AUC against that prevalence, not against 1.0 - the best family is a 4.1x lift over it.
 
 ## Results
 
-Ranked by **pr_auc** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable.
+Ranked by **PR-AUC ↑** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable. Figures are means over 5 chronological folds, ± one standard deviation.
 
-| Model | Family | pr_auc | Baseline? | Beats best baseline |
-|---|---|---:|---|---|
-| `catboost` | gbdt | 0.3615 |  | yes |
-| `random_forest` | trees | 0.3578 |  | yes |
-| `lightgbm` | gbdt | 0.3471 |  | yes |
-| `xgboost` | gbdt | 0.3445 |  | yes |
-| `logistic` | linear | 0.3304 |  | yes |
-| `hist_gbm` | gbdt | 0.3269 |  | yes |
-| `extra_trees` | trees | 0.3024 |  | yes |
-| `prior` | baseline | 0.0873 | baseline | — |
+| Model | Family | PR-AUC ↑ | ROC-AUC ↑ | Brier ↓ | Best F1 ↑ | Fit | Beats baseline |
+|---|---|---:|---:|---:|---:|---:|---|
+| `catboost` | gbdt | 0.3615 ± 0.071 | 0.7684 | 0.0688 | 0.4323 | 3s | yes |
+| `random_forest` | trees | 0.3578 ± 0.080 | 0.7723 | 0.0712 | 0.4326 | 1s | yes |
+| `lightgbm` | gbdt | 0.3471 ± 0.060 | 0.7583 | 0.0746 | 0.4144 | 6s | yes |
+| `xgboost` | gbdt | 0.3445 ± 0.081 | 0.7565 | 0.0703 | 0.4093 | 1s | yes |
+| `logistic` | linear | 0.3304 ± 0.065 | 0.7584 | 0.0713 | 0.3818 | 0s | yes |
+| `hist_gbm` | gbdt | 0.3269 ± 0.061 | 0.7366 | 0.0724 | 0.3855 | 7s | yes |
+| `extra_trees` | trees | 0.3024 ± 0.083 | 0.7443 | 0.0734 | 0.3883 | 0s | yes |
+| `prior` | baseline | 0.0873 ± 0.015 | 0.5000 | 0.0798 | 0.1602 | 0s | *baseline* |
 
-**Best learned family: `catboost`** at 0.3615, +314.3% against the strongest baseline (0.0873).
+**Best learned family: `catboost`** at 0.3615, 4.1x the strongest baseline (`prior` at 0.0873).
 
 ## Reproduction
 

model_cards/B3.md

@@ -13,20 +13,20 @@ Is this event High or Low priority?
 
 ## Results
 
-Ranked by **pr_auc** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable.
-
-| Model | Family | pr_auc | Baseline? | Beats best baseline |
-|---|---|---:|---|---|
-| `catboost` | gbdt | 0.9999 |  | yes |
-| `hist_gbm` | gbdt | 0.9998 |  | yes |
-| `lightgbm` | gbdt | 0.9997 |  | yes |
-| `xgboost` | gbdt | 0.9994 |  | yes |
-| `random_forest` | trees | 0.9974 |  | yes |
-| `extra_trees` | trees | 0.9972 |  | yes |
-| `logistic` | linear | 0.9934 |  | yes |
-| `prior` | baseline | 0.6146 | baseline | — |
-
-**Best learned family: `catboost`** at 0.9999, +62.7% against the strongest baseline (0.6146).
+Ranked by **PR-AUC ↑** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable. Figures are means over 5 chronological folds, ± one standard deviation.
+
+| Model | Family | PR-AUC ↑ | ROC-AUC ↑ | Brier ↓ | Best F1 ↑ | Fit | Beats baseline |
+|---|---|---:|---:|---:|---:|---:|---|
+| `catboost` | gbdt | 0.9999 ± 0.000 | 0.9998 | 0.0022 | 0.9990 | 2s | yes |
+| `hist_gbm` | gbdt | 0.9998 ± 0.000 | 0.9997 | 0.0032 | 0.9985 | 2s | yes |
+| `lightgbm` | gbdt | 0.9997 ± 0.001 | 0.9997 | 0.0016 | 0.9992 | 3s | yes |
+| `xgboost` | gbdt | 0.9994 ± 0.001 | 0.9992 | 0.0028 | 0.9982 | 1s | yes |
+| `random_forest` | trees | 0.9974 ± 0.005 | 0.9980 | 0.0394 | 0.9975 | 0s | yes |
+| `extra_trees` | trees | 0.9972 ± 0.005 | 0.9977 | 0.0730 | 0.9975 | 0s | yes |
+| `logistic` | linear | 0.9934 ± 0.008 | 0.9951 | 0.0060 | 0.9975 | 0s | yes |
+| `prior` | baseline | 0.6146 ± 0.024 | 0.5000 | 0.2370 | 0.7611 | 0s | *baseline* |
+
+**Best learned family: `catboost`** at 0.9999, +62.7% against the strongest baseline (`prior` at 0.6146).
 
 ## Reproduction
 

model_cards/B4.md

@@ -13,14 +13,14 @@ What caused this event, from the operator's free-text note?
 
 ## Results
 
-Ranked by **macro_f1** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable.
+Ranked by **Macro F1 ↑** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable. Figures come from a single chronological split, so there is no spread to report.
 
-| Model | Family | macro_f1 | Baseline? | Beats best baseline |
-|---|---|---:|---|---|
-| `tfidf_char_logreg` | nlp | 0.4782 |  | yes |
-| `majority` | baseline | 0.0441 | baseline | — |
+| Model | Family | Macro F1 ↑ | Accuracy ↑ | Macro F1 · en ↑ | Macro F1 · kn ↑ | Fit | Beats baseline |
+|---|---|---:|---:|---:|---:|---:|---|
+| `tfidf_char_logreg` | nlp | 0.4782 | 0.6555 | 0.4833 | 0.4717 | — | yes |
+| `majority` | baseline | 0.0441 | — | — | — | — | *baseline* |
 
-**Best learned family: `tfidf_char_logreg`** at 0.4782, +985.0% against the strongest baseline (0.0441).
+**Best learned family: `tfidf_char_logreg`** at 0.4782, 10.8x the strongest baseline (`majority` at 0.0441).
 
 ## Reproduction