| base_model: "answerdotai/ModernBERT-base" | |
| --- | |
| # HAV classifier (modernbert-base) | |
| Binary: High Analytical Value (HAV) vs Low (LAV) for social posts. Precision-optimised (F0.5). | |
| ## Data preparation | |
| Model has been trained on data with spam removed, spam posts could be classified as HAV. Not trained on tiktok data. | |
| ## Operating point | |
| Apply a **decision threshold of 0.909** on `softmax(logits)[:, 1]`. | |
| Do NOT use argmax@0.5 — this model is precision-tuned and 0.5 gives far lower precision. | |
| ## Metrics | |
| "objective": { | |
| "metric": "F0.5 (HAV)", | |
| "beta": 0.5, | |
| "min_hav_recall": 0.3, | |
| "decision_threshold": 0.9099, | |
| "val_selection": { | |
| "threshold": 0.9099, | |
| "hav_precision": 0.8238, | |
| "hav_recall": 0.5035, | |
| "nonhav_recall": 0.9753, | |
| "fbeta": 0.7308, | |
| "beta": 0.5 | |
| } | |
| }, | |
| "data": { | |
| "train_rows": 10866, | |
| "train_hav": 2033, | |
| "eval_rows": 2366, | |
| "eval_hav": 440 | |
| }, | |
| "test_overall": { | |
| "accuracy": 0.8762, | |
| "hav_f05": 0.6841, | |
| "hav_f1": 0.5936, | |
| "macro_f1": 0.7603, | |
| "weighted_f1": 0.865, | |
| "average_precision": 0.7163 | |
| }, | |
| "test_per_class": { | |
| "non_HAV": { | |
| "precision": 0.8916, | |
| "recall": 0.9652, | |
| "f1-score": 0.927, | |
| "support": 1926.0 | |
| }, | |
| "HAV": { | |
| "precision": 0.7616, | |
| "recall": 0.4864, | |
| "f1-score": 0.5936, | |
| "support": 440.0 | |
| } | |
| } | |
| ## Caveats | |
| Misclassfied negative class still an issue. pos_weight=4.9, max_length=256. |