Spaces:
Running
Running
| # Reliability, margin ordering & abstention notes | |
| The four models return six-class score vectors, but the score vectors have different interpretations. | |
| 1. Naive Bayes: normalized conditional posterior from a bag-of-words likelihood model. Normalization is **not calibration**. Descriptive reliability bins and ECE are permitted but require appropriate cautions. | |
| 2. Word/character centroid: similarity scores, not class probabilities. Their values cannot be treated as P(correct), and any logistic mapping would require separately fitted and tested calibration. | |
| 3. Hybrid: a fixed mixture of cosine similarities, likewise **not a probability**. | |
| For every model, the raw top-one minus top-two score gap can be sorted **within that model** to make a risk-versus-coverage curve on held-out evaluation examples. `AURC` is the simple average of observed cumulative risks; it is not a generalized deployment-risk integral. AUC under varying distribution or domain shift is unmeasured. | |
| The application never treats a margin threshold as a deployment guarantee or safety certificate. The 54-example challenge has substantial sampling uncertainty and the labels are synthetic-curated. | |