Subramanya97/modernbert-large-unspsc-query

ModernBERT Large UNSPSC Query

Fresh 732-label query classifier fine-tuned from answerdotai/ModernBERT-large on 128,373 query-only examples. It does not continue from the previous 531-label query checkpoint.

Evaluation warning: the metrics below were measured on the same 128,373 examples used for optimization because no held-out validation or test set was supplied. They describe training fit and must not be interpreted as generalization or production accuracy.

Input contract

Format every input exactly as:

<query> copper adapter </query>

The configured template is <query> {query} </query>. No rewrite, part number, company category, label score, or other metadata is part of the model input.

Output contract

This is a multi-label classifier over 732 exact UNSPSC paths. Apply sigmoid to the logits, retain every category at or above the tuned threshold 0.81, and sort by descending score. An empty result is valid.

Training-set fit metrics

Metric	Value
Exact-set accuracy	0.909303
Hamming accuracy	0.999838
Micro precision	0.920636
Micro recall	0.955850
Micro F1	0.937913
Macro F1	0.978951
Weighted F1	0.938904
Top-1 target accuracy (nonempty targets)	0.970259
Label-ranking average precision	0.985520
Recall@3	0.991323
NDCG@5	0.986690
Empty-target accuracy	0.985162

Exact-set accuracy requires the entire predicted label set to match. Hamming accuracy is the fraction of the 732 individual binary label decisions that are correct and can look very high for sparse targets, so micro precision/recall/F1 and ranking metrics should be read alongside it.

Training and provenance

• Fresh base: answerdotai/ModernBERT-large
• Training rows: 128,373
• Label space: 732
• Maximum input length: 256
• W&B run: https://wandb.ai/subramaya1997/unspsc-query-intent/runs/04usxbr4
• Detailed aggregate metrics: decision_threshold.json
• Per-label precision, recall, F1, and average precision: per_label_metrics.json
• Reproducibility metadata: training_provenance.json

Limitations

• There is no held-out evaluation split, so overfitting and generalization error are unmeasured.
• Labels are weak supervision and are not a human-reviewed gold standard.
• Duplicate normalized queries and conflicting weak targets exist in the supplied training file.
• Use this model with search signals and collect a held-out human-reviewed query set before making production-quality claims.

The model uses the Apache-2.0 license, matching the ModernBERT base model. This project is not affiliated with or endorsed by UNSPSC.