roberta-large-triage

This model is a fine-tuned version of FacebookAI/roberta-large for a 5-class triage classification task. It helps categorize student messages based on how they address specific learning objectives.

Classification Labels

0. ADDR_DIRECT: Message directly addresses the objective.
1. ADDR_PARTIAL: Message partially addresses the objective.
2. NOADDR_OFF: Message does not address the objective (Off-topic).
3. NOADDR_ON: Message does not address the objective (On-topic but irrelevant).
4. NOADDR_TANGENTIAL: Message is tangentially related.

Hyperparameters

{
    "learning_rate": 7.613446028496478e-05,
    "num_train_epochs": 3,
    "seed": 12,
    "per_device_train_batch_size": 32
}

Evaluation Results

The model was optimized for Macro-F1 Score on the test set to ensure balanced performance across unique objectives.

Classification Report (test set)

                   precision    recall  f1-score   support

      ADDR_DIRECT      0.939     0.969     0.954        96
     ADDR_PARTIAL      0.854     0.967     0.907        91
       NOADDR_OFF      0.987     0.902     0.943        82
        NOADDR_ON      0.934     0.944     0.939        90
NOADDR_TANGENTIAL      1.000     0.893     0.943        84

         accuracy                          0.937       443
        macro avg      0.943     0.935     0.937       443
     weighted avg      0.941     0.937     0.937       443

Confusion Matrix (test set)