st2 — EE.py (BIO token classification for subject/object spans) Model: dslim/bert-large-NER Data: combined.csv (train) + News_data/dev.csv + Test_dataset/test.csv Train/Dev/Test: 6791 / 627 / 631 ================================================================ Epoch 1, Loss: 0.15692909639547853 precision recall f1-score support O 0.95 0.97 0.96 71126 B-SUBJ 0.50 0.22 0.31 481 I-SUBJ 0.70 0.54 0.61 4183 B-OBJ 0.46 0.26 0.33 480 I-OBJ 0.57 0.59 0.58 3986 accuracy 0.92 80256 macro avg 0.64 0.52 0.56 80256 weighted avg 0.91 0.92 0.92 80256 dev macro F1 = 0.5571 *** saved *** Epoch 2, Loss: 0.076583794336766 precision recall f1-score support O 0.94 0.98 0.96 71126 B-SUBJ 0.66 0.40 0.50 481 I-SUBJ 0.75 0.42 0.54 4183 B-OBJ 0.56 0.40 0.47 480 I-OBJ 0.62 0.54 0.58 3986 accuracy 0.92 80256 macro avg 0.71 0.55 0.61 80256 weighted avg 0.91 0.92 0.91 80256 dev macro F1 = 0.6084 *** saved *** Epoch 3, Loss: 0.06505040774450582 precision recall f1-score support O 0.99 0.95 0.97 71126 B-SUBJ 0.54 0.57 0.56 481 I-SUBJ 0.63 0.74 0.68 4183 B-OBJ 0.52 0.59 0.55 480 I-OBJ 0.53 0.81 0.64 3986 accuracy 0.92 80256 macro avg 0.64 0.73 0.68 80256 weighted avg 0.94 0.92 0.93 80256 dev macro F1 = 0.6805 *** saved *** Epoch 4, Loss: 0.05100112235557069 precision recall f1-score support O 0.99 0.95 0.97 71126 B-SUBJ 0.60 0.55 0.57 481 I-SUBJ 0.68 0.67 0.67 4183 B-OBJ 0.57 0.53 0.55 480 I-OBJ 0.50 0.88 0.64 3986 accuracy 0.92 80256 macro avg 0.67 0.72 0.68 80256 weighted avg 0.94 0.92 0.93 80256 dev macro F1 = 0.6806 *** saved *** Epoch 5, Loss: 0.04952669816461446 precision recall f1-score support O 0.98 0.96 0.97 71126 B-SUBJ 0.58 0.57 0.57 481 I-SUBJ 0.71 0.58 0.64 4183 B-OBJ 0.57 0.54 0.56 480 I-OBJ 0.54 0.87 0.66 3986 accuracy 0.93 80256 macro avg 0.68 0.70 0.68 80256 weighted avg 0.94 0.93 0.93 80256 dev macro F1 = 0.6800 Epoch 6, Loss: 0.03766294808161226 precision recall f1-score support O 0.98 0.96 0.97 71126 B-SUBJ 0.55 0.64 0.59 481 I-SUBJ 0.64 0.79 0.71 4183 B-OBJ 0.55 0.60 0.58 480 I-OBJ 0.61 0.71 0.66 3986 accuracy 0.93 80256 macro avg 0.67 0.74 0.70 80256 weighted avg 0.94 0.93 0.94 80256 dev macro F1 = 0.7000 *** saved (BEST) *** Epoch 7, Loss: 0.034522786220631506 precision recall f1-score support O 0.97 0.97 0.97 71126 B-SUBJ 0.60 0.54 0.57 481 I-SUBJ 0.63 0.70 0.66 4183 B-OBJ 0.59 0.55 0.57 480 I-OBJ 0.67 0.57 0.62 3986 accuracy 0.93 80256 macro avg 0.69 0.67 0.68 80256 weighted avg 0.93 0.93 0.93 80256 dev macro F1 = 0.6774 Epoch 8, Loss: 0.03382389528087943 precision recall f1-score support O 0.98 0.95 0.97 71126 B-SUBJ 0.57 0.57 0.57 481 I-SUBJ 0.63 0.73 0.67 4183 B-OBJ 0.56 0.57 0.57 480 I-OBJ 0.54 0.79 0.64 3986 accuracy 0.92 80256 macro avg 0.66 0.72 0.68 80256 weighted avg 0.94 0.92 0.93 80256 dev macro F1 = 0.6845 Epoch 9, Loss: 0.03163016356212799 precision recall f1-score support O 0.96 0.97 0.97 71126 B-SUBJ 0.54 0.61 0.58 481 I-SUBJ 0.76 0.48 0.59 4183 B-OBJ 0.55 0.61 0.58 480 I-OBJ 0.58 0.65 0.61 3986 accuracy 0.93 80256 macro avg 0.68 0.67 0.66 80256 weighted avg 0.93 0.93 0.92 80256 dev macro F1 = 0.6644 Epoch 10, Loss: 0.02971732596274438 precision recall f1-score support O 0.98 0.95 0.97 71126 B-SUBJ 0.59 0.58 0.59 481 I-SUBJ 0.66 0.67 0.67 4183 B-OBJ 0.56 0.56 0.56 480 I-OBJ 0.53 0.85 0.65 3986 accuracy 0.92 80256 macro avg 0.67 0.72 0.69 80256 weighted avg 0.94 0.92 0.93 80256 dev macro F1 = 0.6859 ================================================================ Test Set (best model from epoch 6): precision recall f1-score support O 0.99 0.95 0.97 75421 B-SUBJ 0.45 0.42 0.43 445 I-SUBJ 0.46 0.67 0.55 2120 B-OBJ 0.45 0.45 0.45 461 I-OBJ 0.45 0.81 0.58 2321 accuracy 0.93 80768 macro avg 0.56 0.66 0.60 80768 weighted avg 0.95 0.93 0.94 80768