| st2 — EE.py (BIO token classification for subject/object spans) |
| Model: dslim/bert-large-NER |
| Data: combined.csv (train) + News_data/dev.csv + Test_dataset/test.csv |
| Train/Dev/Test: 6791 / 627 / 631 |
|
|
| ================================================================ |
| Epoch 1, Loss: 0.15692909639547853 |
| precision recall f1-score support |
| O 0.95 0.97 0.96 71126 |
| B-SUBJ 0.50 0.22 0.31 481 |
| I-SUBJ 0.70 0.54 0.61 4183 |
| B-OBJ 0.46 0.26 0.33 480 |
| I-OBJ 0.57 0.59 0.58 3986 |
| accuracy 0.92 80256 |
| macro avg 0.64 0.52 0.56 80256 |
| weighted avg 0.91 0.92 0.92 80256 |
| dev macro F1 = 0.5571 *** saved *** |
|
|
| Epoch 2, Loss: 0.076583794336766 |
| precision recall f1-score support |
| O 0.94 0.98 0.96 71126 |
| B-SUBJ 0.66 0.40 0.50 481 |
| I-SUBJ 0.75 0.42 0.54 4183 |
| B-OBJ 0.56 0.40 0.47 480 |
| I-OBJ 0.62 0.54 0.58 3986 |
| accuracy 0.92 80256 |
| macro avg 0.71 0.55 0.61 80256 |
| weighted avg 0.91 0.92 0.91 80256 |
| dev macro F1 = 0.6084 *** saved *** |
|
|
| Epoch 3, Loss: 0.06505040774450582 |
| precision recall f1-score support |
| O 0.99 0.95 0.97 71126 |
| B-SUBJ 0.54 0.57 0.56 481 |
| I-SUBJ 0.63 0.74 0.68 4183 |
| B-OBJ 0.52 0.59 0.55 480 |
| I-OBJ 0.53 0.81 0.64 3986 |
| accuracy 0.92 80256 |
| macro avg 0.64 0.73 0.68 80256 |
| weighted avg 0.94 0.92 0.93 80256 |
| dev macro F1 = 0.6805 *** saved *** |
|
|
| Epoch 4, Loss: 0.05100112235557069 |
| precision recall f1-score support |
| O 0.99 0.95 0.97 71126 |
| B-SUBJ 0.60 0.55 0.57 481 |
| I-SUBJ 0.68 0.67 0.67 4183 |
| B-OBJ 0.57 0.53 0.55 480 |
| I-OBJ 0.50 0.88 0.64 3986 |
| accuracy 0.92 80256 |
| macro avg 0.67 0.72 0.68 80256 |
| weighted avg 0.94 0.92 0.93 80256 |
| dev macro F1 = 0.6806 *** saved *** |
|
|
| Epoch 5, Loss: 0.04952669816461446 |
| precision recall f1-score support |
| O 0.98 0.96 0.97 71126 |
| B-SUBJ 0.58 0.57 0.57 481 |
| I-SUBJ 0.71 0.58 0.64 4183 |
| B-OBJ 0.57 0.54 0.56 480 |
| I-OBJ 0.54 0.87 0.66 3986 |
| accuracy 0.93 80256 |
| macro avg 0.68 0.70 0.68 80256 |
| weighted avg 0.94 0.93 0.93 80256 |
| dev macro F1 = 0.6800 |
|
|
| Epoch 6, Loss: 0.03766294808161226 |
| precision recall f1-score support |
| O 0.98 0.96 0.97 71126 |
| B-SUBJ 0.55 0.64 0.59 481 |
| I-SUBJ 0.64 0.79 0.71 4183 |
| B-OBJ 0.55 0.60 0.58 480 |
| I-OBJ 0.61 0.71 0.66 3986 |
| accuracy 0.93 80256 |
| macro avg 0.67 0.74 0.70 80256 |
| weighted avg 0.94 0.93 0.94 80256 |
| dev macro F1 = 0.7000 *** saved (BEST) *** |
|
|
| Epoch 7, Loss: 0.034522786220631506 |
| precision recall f1-score support |
| O 0.97 0.97 0.97 71126 |
| B-SUBJ 0.60 0.54 0.57 481 |
| I-SUBJ 0.63 0.70 0.66 4183 |
| B-OBJ 0.59 0.55 0.57 480 |
| I-OBJ 0.67 0.57 0.62 3986 |
| accuracy 0.93 80256 |
| macro avg 0.69 0.67 0.68 80256 |
| weighted avg 0.93 0.93 0.93 80256 |
| dev macro F1 = 0.6774 |
|
|
| Epoch 8, Loss: 0.03382389528087943 |
| precision recall f1-score support |
| O 0.98 0.95 0.97 71126 |
| B-SUBJ 0.57 0.57 0.57 481 |
| I-SUBJ 0.63 0.73 0.67 4183 |
| B-OBJ 0.56 0.57 0.57 480 |
| I-OBJ 0.54 0.79 0.64 3986 |
| accuracy 0.92 80256 |
| macro avg 0.66 0.72 0.68 80256 |
| weighted avg 0.94 0.92 0.93 80256 |
| dev macro F1 = 0.6845 |
|
|
| Epoch 9, Loss: 0.03163016356212799 |
| precision recall f1-score support |
| O 0.96 0.97 0.97 71126 |
| B-SUBJ 0.54 0.61 0.58 481 |
| I-SUBJ 0.76 0.48 0.59 4183 |
| B-OBJ 0.55 0.61 0.58 480 |
| I-OBJ 0.58 0.65 0.61 3986 |
| accuracy 0.93 80256 |
| macro avg 0.68 0.67 0.66 80256 |
| weighted avg 0.93 0.93 0.92 80256 |
| dev macro F1 = 0.6644 |
|
|
| Epoch 10, Loss: 0.02971732596274438 |
| precision recall f1-score support |
| O 0.98 0.95 0.97 71126 |
| B-SUBJ 0.59 0.58 0.59 481 |
| I-SUBJ 0.66 0.67 0.67 4183 |
| B-OBJ 0.56 0.56 0.56 480 |
| I-OBJ 0.53 0.85 0.65 3986 |
| accuracy 0.92 80256 |
| macro avg 0.67 0.72 0.69 80256 |
| weighted avg 0.94 0.92 0.93 80256 |
| dev macro F1 = 0.6859 |
|
|
| ================================================================ |
| Test Set (best model from epoch 6): |
| precision recall f1-score support |
| O 0.99 0.95 0.97 75421 |
| B-SUBJ 0.45 0.42 0.43 445 |
| I-SUBJ 0.46 0.67 0.55 2120 |
| B-OBJ 0.45 0.45 0.45 461 |
| I-OBJ 0.45 0.81 0.58 2321 |
| accuracy 0.93 80768 |
| macro avg 0.56 0.66 0.60 80768 |
| weighted avg 0.95 0.93 0.94 80768 |
|
|
