| # Threshold-conversion artifacts | |
| Question: can any multiclass classifier's threshold curve be converted to a | |
| 0.5-threshold binary rule equivalent to argmax, and can the conversion preserve | |
| maximum F1? | |
| Verdict: the conversion is an exact **reparameterization** of a chosen | |
| threshold (binary), or of each class's chosen threshold (multiclass). It cannot | |
| turn one fixed 0.5 rule into argmax for C >= 3, and it cannot improve F1; it | |
| only moves the selected operating point to 0.5. | |
| ## Layout | |
| - `baseline/` theory, uniform simplex, sklearn digits | |
| - `text/` DistilBERT SST-2, Twitter-RoBERTa sentiment, DistilBERT emotion | |
| - `vision/` ViT-B/16 ImageNet head on CIFAR-10 (1000 classes, OOD) | |
| - `conversion_fit/` fixed C x C linear fit to argmax labels | |
| - `tabular/` 30 tabular dataset/classifier pairs with max-F1 conversion | |
| - `manifest.json` experiment ids, run ids, commits, metrics, links | |
| - `report.md` full synthesis (figures embedded) | |
| - `logbook.ipynb` executable notebook with formulae, tables and every link | |
| ## Headline numbers | |
| | model | C | n | argmax | delta_0.5 | coverage | exact adaptive | | |
| |---|---:|---:|---:|---:|---:|---| | |
| | sklearn digits | 10 | 540 | 97.22% | 0.74% | 99.26% | yes | | |
| | DistilBERT SST-2 | 2 | 872 | 91.06% | 0.00% | 100.00% | yes | | |
| | Twitter-RoBERTa sentiment | 3 | 2000 | 71.50% | 4.65% | 95.35% | yes | | |
| | DistilBERT emotion | 6 | 2000 | 92.70% | 0.20% | 99.80% | yes | | |
| | ViT-B/16 on CIFAR-10 | 1000 | 1000 | n/a | 51.00% | 49.00% | yes | | |
| Tabular max-F1 test: all 30 conversions are exact; `F1(test, tau*)` equals | |
| `F1(test, converted 0.5)` exactly. The selected threshold is not 0.5: binary | |
| tau* ranges 0.482-0.998, and multiclass per-class tau* spans up to 1.0. A | |
| 20-class random forest has delta_0.5 = 99.80% (998/1000 abstain), macro-F1 0.004 | |
| at 0.5, argmax F1 0.5495, and converted-argmax agreement only 87.2%. | |
| ## Links | |
| - Upstream repo: https://github.com/isllabres/python-multiagent-system | |
| - Fork: https://github.com/jomasego/python-multiagent-system | |
| - Bucket: https://huggingface.co/buckets/jomasego/repro-threshold-conversion-artifacts | |
| - Models: https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment-latest , | |
| https://huggingface.co/bhadresh-savani/distilbert-base-uncased-emotion , | |
| https://huggingface.co/google/vit-base-patch16-224 , | |
| https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english | |
| - Datasets: https://huggingface.co/datasets/cardiffnlp/tweet_eval , | |
| https://huggingface.co/datasets/dair-ai/emotion , | |
| https://huggingface.co/datasets/uoft-cs/cifar10 , | |
| https://huggingface.co/datasets/stanfordnlp/sst2 | |
| HF Jobs were attempted for the text node but the account has no pre-paid Jobs | |
| credit: `402: Pre-paid credit balance is insufficient`. No Job URL exists. | |
Xet Storage Details
- Size:
- 2.74 kB
- Xet hash:
- 31ccb370569db026f2bd7e6cc18aa25c0cbaeaf9273c21c3a1d2b7dd9f23f554
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.