Hugging Face's logo

faodl
/
agri-wheat-classifier

Text Classification
Transformers
Joblib
Safetensors
multilingual
binary-classification
amis
agriculture
Model card Files
xet
 Community
agri-wheat-classifier / REPORT.md
lgsilvaesilva's picture
lgsilvaesilva
Upload folder using huggingface_hub
57dcd10 verified
preview code
|
Raw
History Blame Contribute Delete
6.47 kB

AMIS Commodity Classifier Training Report

  • Dataset: faodl/amis-agri-wheat
  • Dataset subset: ``
  • Dataset revision: main
  • Text column: chunk_text
  • Label column: label
  • Transformer: FacebookAI/xlm-roberta-base
  • Generated at: 2026-05-29T18:13:08.384805+00:00

Dataset Summary

Split Rows Label 0 Label 1 Unique groups Mean text length
train 3622 2163 1459 1850 644.8
validation 759 486 273 396 636.7
test 762 470 292 397 643.3

Threshold Comparison on Validation Split

Validation metrics document threshold selection and tuning behavior; test metrics remain the primary estimate of out-of-sample performance.

Model Threshold Accuracy Precision Recall F1 ROC AUC Average precision
logistic_tfidf 0.500 0.818 0.718 0.813 0.763 0.907 0.867
logistic_tfidf 0.470 0.823 0.709 0.864 0.779 0.907 0.867
xgboost_tfidf 0.500 0.868 0.808 0.832 0.819 0.935 0.892
xgboost_tfidf 0.520 0.871 0.816 0.828 0.822 0.935 0.892
embedding-logistic_sentence_embeddings 0.500 0.783 0.658 0.824 0.732 0.862 0.780
embedding-logistic_sentence_embeddings 0.521 0.791 0.673 0.813 0.736 0.862 0.780
embedding-svm_sentence_embeddings 0.500 0.804 0.714 0.758 0.735 0.869 0.792
embedding-svm_sentence_embeddings 0.473 0.805 0.704 0.791 0.745 0.869 0.792
embedding-lightgbm_sentence_embeddings 0.500 0.791 0.694 0.747 0.720 0.868 0.786
embedding-lightgbm_sentence_embeddings 0.433 0.800 0.693 0.795 0.741 0.868 0.786
transformer 0.500 0.925 0.894 0.897 0.896 0.956 0.914
transformer 0.203 0.926 0.883 0.916 0.899 0.956 0.914

Threshold Comparison on Test Split

Model Threshold Accuracy Precision Recall F1 ROC AUC Average precision
logistic_tfidf 0.500 0.803 0.715 0.808 0.759 0.888 0.827
logistic_tfidf 0.470 0.797 0.688 0.860 0.764 0.888 0.827
xgboost_tfidf 0.500 0.835 0.773 0.805 0.789 0.910 0.831
xgboost_tfidf 0.520 0.835 0.777 0.798 0.787 0.910 0.831
embedding-logistic_sentence_embeddings 0.500 0.782 0.699 0.757 0.727 0.877 0.821
embedding-logistic_sentence_embeddings 0.521 0.789 0.713 0.750 0.731 0.877 0.821
embedding-svm_sentence_embeddings 0.500 0.818 0.778 0.733 0.755 0.883 0.824
embedding-svm_sentence_embeddings 0.473 0.812 0.758 0.750 0.754 0.883 0.824
embedding-lightgbm_sentence_embeddings 0.500 0.798 0.740 0.729 0.734 0.892 0.847
embedding-lightgbm_sentence_embeddings 0.433 0.806 0.735 0.771 0.753 0.892 0.847
transformer 0.500 0.885 0.862 0.832 0.847 0.943 0.915
transformer 0.203 0.890 0.854 0.860 0.857 0.943 0.915

Confusion Matrices on Test Split

Rows are true labels and columns are predicted labels.

logistic_tfidf at threshold 0.500

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 376 94
RELEVANT 56 236

logistic_tfidf at threshold 0.470

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 356 114
RELEVANT 41 251

xgboost_tfidf at threshold 0.500

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 401 69
RELEVANT 57 235

xgboost_tfidf at threshold 0.520

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 403 67
RELEVANT 59 233

embedding-logistic_sentence_embeddings at threshold 0.500

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 375 95
RELEVANT 71 221

embedding-logistic_sentence_embeddings at threshold 0.521

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 382 88
RELEVANT 73 219

embedding-svm_sentence_embeddings at threshold 0.500

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 409 61
RELEVANT 78 214

embedding-svm_sentence_embeddings at threshold 0.473

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 400 70
RELEVANT 73 219

embedding-lightgbm_sentence_embeddings at threshold 0.500

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 395 75
RELEVANT 79 213

embedding-lightgbm_sentence_embeddings at threshold 0.433

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 389 81
RELEVANT 67 225

transformer at threshold 0.500

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 431 39
RELEVANT 49 243

transformer at threshold 0.203

True / Predicted NOT_RELEVANT RELEVANT
NOT_RELEVANT 427 43
RELEVANT 41 251

Validation-Tuned Thresholds

  • logistic_tfidf: threshold 0.470 (validation F1 0.779); test F1 change vs 0.5: +0.005.
  • xgboost_tfidf: threshold 0.520 (validation F1 0.822); test F1 change vs 0.5: -0.001.
  • embedding-logistic_sentence_embeddings: threshold 0.521 (validation F1 0.736); test F1 change vs 0.5: +0.004.
  • embedding-svm_sentence_embeddings: threshold 0.473 (validation F1 0.745); test F1 change vs 0.5: -0.001.
  • embedding-lightgbm_sentence_embeddings: threshold 0.433 (validation F1 0.741); test F1 change vs 0.5: +0.018.
  • transformer: threshold 0.203 (validation F1 0.899); test F1 change vs 0.5: +0.010.

Artifacts

  • logistic_tfidf: /content/agri-wheat-classifier/baselines/logistic
  • xgboost_tfidf: /content/agri-wheat-classifier/baselines/xgboost
  • embedding-logistic_sentence_embeddings: /content/agri-wheat-classifier/baselines/embedding-logistic
  • embedding-svm_sentence_embeddings: /content/agri-wheat-classifier/baselines/embedding-svm
  • embedding-lightgbm_sentence_embeddings: /content/agri-wheat-classifier/baselines/embedding-lightgbm
  • transformer: /content/agri-wheat-classifier/transformer