| EVALUATION LOG - 2025-10-29 03:44:41 | |
| ================================================================================ | |
| ================================================================================ | |
| STARTING POST-TRAINING EVALUATION | |
| ================================================================================ | |
| ✅ Test data loaded: 40532 samples | |
| Columns: ['dataset', 'type', 'comment', 'label'] | |
| Using device: cuda | |
| ============================================================ | |
| EVALUATING MODEL: PHOBERT-V1 | |
| ============================================================ | |
| ✅ Model phobert-v1 loaded from outputs/hate-speech-detection/phobert-v1 | |
| ✅ Tokenizer loaded for phobert-v1 | |
| Evaluating on 40532 samples... | |
| Text column: comment, Label column: label | |
| ✅ Evaluation completed! | |
| Accuracy: 0.9421 | |
| F1 Macro: 0.8308 | |
| F1 Weighted: 0.9394 | |
| ============================================================ | |
| EVALUATING MODEL: PHOBERT-V2 | |
| ============================================================ | |
| ✅ Model phobert-v2 loaded from outputs/hate-speech-detection/phobert-v2 | |
| ✅ Tokenizer loaded for phobert-v2 | |
| Evaluating on 40532 samples... | |
| Text column: comment, Label column: label | |
| ✅ Evaluation completed! | |
| Accuracy: 0.9341 | |
| F1 Macro: 0.8048 | |
| F1 Weighted: 0.9326 | |
| ============================================================ | |
| EVALUATING MODEL: BARTPHO | |
| ============================================================ | |
| ✅ Model bartpho loaded from outputs/hate-speech-detection/bartpho | |
| ✅ Tokenizer loaded for bartpho | |
| Evaluating on 40532 samples... | |
| Text column: comment, Label column: label | |
| ✅ Evaluation completed! | |
| Accuracy: 0.8985 | |
| F1 Macro: 0.6791 | |
| F1 Weighted: 0.8886 | |
| ============================================================ | |
| EVALUATING MODEL: VISOBERT | |
| ============================================================ | |
| ✅ Model visobert loaded from outputs/hate-speech-detection/visobert | |
| ✅ Tokenizer loaded for visobert | |
| Evaluating on 40532 samples... | |
| Text column: comment, Label column: label | |
| ✅ Evaluation completed! | |
| Accuracy: 0.9372 | |
| F1 Macro: 0.8241 | |
| F1 Weighted: 0.9379 | |
| ============================================================ | |
| EVALUATING MODEL: VIHATE-T5 | |
| ============================================================ | |
| ✅ Model vihate-t5 loaded from outputs/hate-speech-detection/vihate-t5 | |
| ✅ Tokenizer loaded for vihate-t5 | |
| Evaluating on 40532 samples... | |
| Text column: comment, Label column: label | |
| ✅ Evaluation completed! | |
| Accuracy: 0.9551 | |
| F1 Macro: 0.8718 | |
| F1 Weighted: 0.9535 | |
| ============================================================ | |
| EVALUATING MODEL: XLM-R | |
| ============================================================ | |
| ✅ Model xlm-r loaded from outputs/hate-speech-detection/xlm-r | |
| ✅ Tokenizer loaded for xlm-r | |
| Evaluating on 40532 samples... | |
| Text column: comment, Label column: label | |
| ✅ Evaluation completed! | |
| Accuracy: 0.9203 | |
| F1 Macro: 0.7625 | |
| F1 Weighted: 0.9177 | |
| ============================================================ | |
| EVALUATING MODEL: ROBERTA-GRU | |
| ============================================================ | |
| ✅ Model roberta-gru loaded from outputs/hate-speech-detection/roberta-gru | |
| ✅ Tokenizer loaded for roberta-gru | |
| Evaluating on 40532 samples... | |
| Text column: comment, Label column: label | |
| ✅ Evaluation completed! | |
| Accuracy: 0.9537 | |
| F1 Macro: 0.8716 | |
| F1 Weighted: 0.9530 | |
| ============================================================ | |
| EVALUATING MODEL: BILSTM | |
| ============================================================ | |
| ✅ Model bilstm loaded from outputs/hate-speech-detection/bilstm | |
| Evaluating on 40532 samples... | |
| Text column: comment, Label column: label | |
| ℹ️ BILSTM evaluation requires special handling | |
| Using dummy predictions for BILSTM | |
| ✅ Evaluation completed! | |
| Accuracy: 0.8388 | |
| F1 Macro: 0.3041 | |
| F1 Weighted: 0.7652 | |
| ============================================================ | |
| EVALUATING MODEL: TEXTCNN | |
| ============================================================ | |
| ✅ Model textcnn loaded from outputs/hate-speech-detection/textcnn | |
| Evaluating on 40532 samples... | |
| Text column: comment, Label column: label | |
| ℹ️ TEXTCNN evaluation requires special handling | |
| Using dummy predictions for TEXTCNN | |
| ✅ Evaluation completed! | |
| Accuracy: 0.8388 | |
| F1 Macro: 0.3041 | |
| F1 Weighted: 0.7652 | |
| ============================================================ | |
| EVALUATING MODEL: MBERT | |
| ============================================================ | |
| ✅ Model mbert loaded from outputs/hate-speech-detection/mbert | |
| ✅ Tokenizer loaded for mbert | |
| Evaluating on 40532 samples... | |
| Text column: comment, Label column: label | |
| ✅ Evaluation completed! | |
| Accuracy: 0.9360 | |
| F1 Macro: 0.8044 | |
| F1 Weighted: 0.9317 | |
| ============================================================ | |
| EVALUATING MODEL: SPHOBERT | |
| ============================================================ | |
| ✅ Model sphobert loaded from outputs/hate-speech-detection/sphobert | |
| ✅ Tokenizer loaded for sphobert | |
| Evaluating on 40532 samples... | |
| Text column: comment, Label column: label | |
| ✅ Evaluation completed! | |
| Accuracy: 0.9143 | |
| F1 Macro: 0.7378 | |
| F1 Weighted: 0.9096 | |
| ================================================================================ | |
| FINAL EVALUATION RESULTS - 2025-10-29 04:14:15 | |
| ================================================================================ | |
| EVALUATION SUMMARY | |
| -------------------------------------------------- | |
| Model Accuracy F1 Macro F1 Weighted Samples | |
| -------------------------------------------------- | |
| phobert-v1 0.9421 0.8308 0.9394 40532 | |
| phobert-v2 0.9341 0.8048 0.9326 40532 | |
| bartpho 0.8985 0.6791 0.8886 40532 | |
| visobert 0.9372 0.8241 0.9379 40532 | |
| vihate-t5 0.9551 0.8718 0.9535 40532 | |
| xlm-r 0.9203 0.7625 0.9177 40532 | |
| roberta-gru 0.9537 0.8716 0.9530 40532 | |
| bilstm 0.8388 0.3041 0.7652 40532 | |
| textcnn 0.8388 0.3041 0.7652 40532 | |
| mbert 0.9360 0.8044 0.9317 40532 | |
| sphobert 0.9143 0.7378 0.9096 40532 | |
| ================================================================================ | |
| DETAILED RESULTS - PHOBERT-V1 | |
| -------------------------------------------------- | |
| Model Path: outputs/hate-speech-detection/phobert-v1 | |
| Number of Samples: 40532 | |
| Accuracy: 0.9421 | |
| F1 Macro: 0.8308 | |
| F1 Weighted: 0.9394 | |
| Classification Report: | |
| Class Precision Recall F1-Score Support | |
| -------------------------------------------------- | |
| CLEAN 0.9554 0.9868 0.9709 33997.0 | |
| OFFENSIVE 0.7910 0.6581 0.7185 2094.0 | |
| HATE 0.8866 0.7341 0.8032 4441.0 | |
| macro avg 0.8777 0.7930 0.8308 40532.0 | |
| weighted avg 0.9394 0.9421 0.9394 40532.0 | |
| Confusion Matrix: | |
| [[33548 196 253] | |
| [ 552 1378 164] | |
| [ 1013 168 3260]] | |
| ================================================================================ | |
| DETAILED RESULTS - PHOBERT-V2 | |
| -------------------------------------------------- | |
| Model Path: outputs/hate-speech-detection/phobert-v2 | |
| Number of Samples: 40532 | |
| Accuracy: 0.9341 | |
| F1 Macro: 0.8048 | |
| F1 Weighted: 0.9326 | |
| Classification Report: | |
| Class Precision Recall F1-Score Support | |
| -------------------------------------------------- | |
| CLEAN 0.9635 0.9739 0.9687 33997.0 | |
| OFFENSIVE 0.7505 0.5903 0.6608 2094.0 | |
| HATE 0.7779 0.7919 0.7849 4441.0 | |
| macro avg 0.8306 0.7854 0.8048 40532.0 | |
| weighted avg 0.9321 0.9341 0.9326 40532.0 | |
| Confusion Matrix: | |
| [[33109 219 669] | |
| [ 523 1236 335] | |
| [ 732 192 3517]] | |
| ================================================================================ | |
| DETAILED RESULTS - BARTPHO | |
| -------------------------------------------------- | |
| Model Path: outputs/hate-speech-detection/bartpho | |
| Number of Samples: 40532 | |
| Accuracy: 0.8985 | |
| F1 Macro: 0.6791 | |
| F1 Weighted: 0.8886 | |
| Classification Report: | |
| Class Precision Recall F1-Score Support | |
| -------------------------------------------------- | |
| CLEAN 0.9228 0.9770 0.9491 33997.0 | |
| OFFENSIVE 0.6527 0.3563 0.4609 2094.0 | |
| HATE 0.7238 0.5535 0.6273 4441.0 | |
| macro avg 0.7664 0.6289 0.6791 40532.0 | |
| weighted avg 0.8871 0.8985 0.8886 40532.0 | |
| Confusion Matrix: | |
| [[33215 235 547] | |
| [ 957 746 391] | |
| [ 1821 162 2458]] | |
| ================================================================================ | |
| DETAILED RESULTS - VISOBERT | |
| -------------------------------------------------- | |
| Model Path: outputs/hate-speech-detection/visobert | |
| Number of Samples: 40532 | |
| Accuracy: 0.9372 | |
| F1 Macro: 0.8241 | |
| F1 Weighted: 0.9379 | |
| Classification Report: | |
| Class Precision Recall F1-Score Support | |
| -------------------------------------------------- | |
| CLEAN 0.9714 0.9687 0.9700 33997.0 | |
| OFFENSIVE 0.6463 0.7574 0.6974 2094.0 | |
| HATE 0.8305 0.7809 0.8049 4441.0 | |
| macro avg 0.8160 0.8357 0.8241 40532.0 | |
| weighted avg 0.9392 0.9372 0.9379 40532.0 | |
| Confusion Matrix: | |
| [[32932 590 475] | |
| [ 275 1586 233] | |
| [ 695 278 3468]] | |
| ================================================================================ | |
| DETAILED RESULTS - VIHATE-T5 | |
| -------------------------------------------------- | |
| Model Path: outputs/hate-speech-detection/vihate-t5 | |
| Number of Samples: 40532 | |
| Accuracy: 0.9551 | |
| F1 Macro: 0.8718 | |
| F1 Weighted: 0.9535 | |
| Classification Report: | |
| Class Precision Recall F1-Score Support | |
| -------------------------------------------------- | |
| CLEAN 0.9660 0.9883 0.9770 33997.0 | |
| OFFENSIVE 0.8788 0.7096 0.7852 2094.0 | |
| HATE 0.8931 0.8165 0.8531 4441.0 | |
| macro avg 0.9126 0.8381 0.8718 40532.0 | |
| weighted avg 0.9535 0.9551 0.9535 40532.0 | |
| Confusion Matrix: | |
| [[33599 124 274] | |
| [ 448 1486 160] | |
| [ 734 81 3626]] | |
| ================================================================================ | |
| DETAILED RESULTS - XLM-R | |
| -------------------------------------------------- | |
| Model Path: outputs/hate-speech-detection/xlm-r | |
| Number of Samples: 40532 | |
| Accuracy: 0.9203 | |
| F1 Macro: 0.7625 | |
| F1 Weighted: 0.9177 | |
| Classification Report: | |
| Class Precision Recall F1-Score Support | |
| -------------------------------------------------- | |
| CLEAN 0.9514 0.9733 0.9622 33997.0 | |
| OFFENSIVE 0.6284 0.5702 0.5979 2094.0 | |
| HATE 0.7834 0.6791 0.7275 4441.0 | |
| macro avg 0.7877 0.7409 0.7625 40532.0 | |
| weighted avg 0.9163 0.9203 0.9177 40532.0 | |
| Confusion Matrix: | |
| [[33090 418 489] | |
| [ 555 1194 345] | |
| [ 1137 288 3016]] | |
| ================================================================================ | |
| DETAILED RESULTS - ROBERTA-GRU | |
| -------------------------------------------------- | |
| Model Path: outputs/hate-speech-detection/roberta-gru | |
| Number of Samples: 40532 | |
| Accuracy: 0.9537 | |
| F1 Macro: 0.8716 | |
| F1 Weighted: 0.9530 | |
| Classification Report: | |
| Class Precision Recall F1-Score Support | |
| -------------------------------------------------- | |
| CLEAN 0.9711 0.9825 0.9768 33997.0 | |
| OFFENSIVE 0.8136 0.7693 0.7909 2094.0 | |
| HATE 0.8761 0.8201 0.8472 4441.0 | |
| macro avg 0.8870 0.8573 0.8716 40532.0 | |
| weighted avg 0.9526 0.9537 0.9530 40532.0 | |
| Confusion Matrix: | |
| [[33402 237 358] | |
| [ 326 1611 157] | |
| [ 667 132 3642]] | |
| ================================================================================ | |
| DETAILED RESULTS - BILSTM | |
| -------------------------------------------------- | |
| Model Path: outputs/hate-speech-detection/bilstm | |
| Number of Samples: 40532 | |
| Accuracy: 0.8388 | |
| F1 Macro: 0.3041 | |
| F1 Weighted: 0.7652 | |
| Classification Report: | |
| Class Precision Recall F1-Score Support | |
| -------------------------------------------------- | |
| CLEAN 0.8388 1.0000 0.9123 33997.0 | |
| OFFENSIVE 0.0000 0.0000 0.0000 2094.0 | |
| HATE 0.0000 0.0000 0.0000 4441.0 | |
| macro avg 0.2796 0.3333 0.3041 40532.0 | |
| weighted avg 0.7035 0.8388 0.7652 40532.0 | |
| Confusion Matrix: | |
| [[33997 0 0] | |
| [ 2094 0 0] | |
| [ 4441 0 0]] | |
| ================================================================================ | |
| DETAILED RESULTS - TEXTCNN | |
| -------------------------------------------------- | |
| Model Path: outputs/hate-speech-detection/textcnn | |
| Number of Samples: 40532 | |
| Accuracy: 0.8388 | |
| F1 Macro: 0.3041 | |
| F1 Weighted: 0.7652 | |
| Classification Report: | |
| Class Precision Recall F1-Score Support | |
| -------------------------------------------------- | |
| CLEAN 0.8388 1.0000 0.9123 33997.0 | |
| OFFENSIVE 0.0000 0.0000 0.0000 2094.0 | |
| HATE 0.0000 0.0000 0.0000 4441.0 | |
| macro avg 0.2796 0.3333 0.3041 40532.0 | |
| weighted avg 0.7035 0.8388 0.7652 40532.0 | |
| Confusion Matrix: | |
| [[33997 0 0] | |
| [ 2094 0 0] | |
| [ 4441 0 0]] | |
| ================================================================================ | |
| DETAILED RESULTS - MBERT | |
| -------------------------------------------------- | |
| Model Path: outputs/hate-speech-detection/mbert | |
| Number of Samples: 40532 | |
| Accuracy: 0.9360 | |
| F1 Macro: 0.8044 | |
| F1 Weighted: 0.9317 | |
| Classification Report: | |
| Class Precision Recall F1-Score Support | |
| -------------------------------------------------- | |
| CLEAN 0.9489 0.9876 0.9679 33997.0 | |
| OFFENSIVE 0.8645 0.5392 0.6641 2094.0 | |
| HATE 0.8416 0.7287 0.7811 4441.0 | |
| macro avg 0.8850 0.7518 0.8044 40532.0 | |
| weighted avg 0.9328 0.9360 0.9317 40532.0 | |
| Confusion Matrix: | |
| [[33574 93 330] | |
| [ 686 1129 279] | |
| [ 1121 84 3236]] | |
| ================================================================================ | |
| DETAILED RESULTS - SPHOBERT | |
| -------------------------------------------------- | |
| Model Path: outputs/hate-speech-detection/sphobert | |
| Number of Samples: 40532 | |
| Accuracy: 0.9143 | |
| F1 Macro: 0.7378 | |
| F1 Weighted: 0.9096 | |
| Classification Report: | |
| Class Precision Recall F1-Score Support | |
| -------------------------------------------------- | |
| CLEAN 0.9434 0.9729 0.9579 33997.0 | |
| OFFENSIVE 0.6821 0.4508 0.5428 2094.0 | |
| HATE 0.7436 0.6843 0.7127 4441.0 | |
| macro avg 0.7897 0.7027 0.7378 40532.0 | |
| weighted avg 0.9080 0.9143 0.9096 40532.0 | |
| Confusion Matrix: | |
| [[33077 253 667] | |
| [ 769 944 381] | |
| [ 1215 187 3039]] | |
| ================================================================================ | |
| ============================================================ | |
| EVALUATION COMPLETED! | |
| ============================================================ | |
| Successfully evaluated: 11/11 models | |
| Best performing models: | |
| 1. vihate-t5: Accuracy=0.9551, F1=0.8718 | |
| 2. roberta-gru: Accuracy=0.9537, F1=0.8716 | |
| 3. phobert-v1: Accuracy=0.9421, F1=0.8308 | |