| st0 — Relation_detection.py (binary causal-or-not) |
| Model: roberta-large |
| Data: Combined_dataset_CommonSense+News_Data |
| Train/Dev/Test: 3396 / 627 / 632 |
|
|
| Label distribution |
| train: 1=0.9167, 0=0.0833 |
| dev: 1=0.7879, 0=0.2121 |
| test: 1=0.8418, 0=0.1582 |
|
|
| ================================================================ |
| Epoch 1/10, Train Loss: 0.0438, Validation Loss: 0.4460, Validation Accuracy: 0.9282 |
| precision recall f1-score support |
| 0 0.98 0.68 0.80 133 |
| 1 0.92 1.00 0.96 494 |
| accuracy 0.93 627 |
| macro avg 0.95 0.84 0.88 627 |
| weighted avg 0.93 0.93 0.92 627 |
| Validation F1-score: 0.9563 |
| Saved best model to /content/drive/MyDrive/causalsense/checkpoints/st0_roberta_large (val F1=0.9563) |
|
|
| Epoch 2/10, Train Loss: 0.0246, Validation Loss: 0.9863, Validation Accuracy: 0.8788 |
| precision recall f1-score support |
| 0 1.00 0.43 0.60 133 |
| 1 0.87 1.00 0.93 494 |
| accuracy 0.88 627 |
| macro avg 0.93 0.71 0.76 627 |
| weighted avg 0.89 0.88 0.86 627 |
| Validation F1-score: 0.9286 |
|
|
| Epoch 3/10, Train Loss: 0.0442, Validation Loss: 0.1793, Validation Accuracy: 0.9745 |
| precision recall f1-score support |
| 0 0.98 0.89 0.94 133 |
| 1 0.97 1.00 0.98 494 |
| accuracy 0.97 627 |
| macro avg 0.98 0.95 0.96 627 |
| weighted avg 0.97 0.97 0.97 627 |
| Validation F1-score: 0.9840 |
| Saved best model to /content/drive/MyDrive/causalsense/checkpoints/st0_roberta_large (val F1=0.9840) *** BEST *** |
|
|
| Epoch 4/10, Train Loss: 0.0134, Validation Loss: 0.6935, Validation Accuracy: 0.9187 |
| precision recall f1-score support |
| 0 1.00 0.62 0.76 133 |
| 1 0.91 1.00 0.95 494 |
| accuracy 0.92 627 |
| macro avg 0.95 0.81 0.86 627 |
| weighted avg 0.93 0.92 0.91 627 |
| Validation F1-score: 0.9509 |
|
|
| Epoch 5/10, Train Loss: 0.0058, Validation Loss: 0.7291, Validation Accuracy: 0.9123 |
| precision recall f1-score support |
| 0 1.00 0.59 0.74 133 |
| 1 0.90 1.00 0.95 494 |
| accuracy 0.91 627 |
| macro avg 0.95 0.79 0.84 627 |
| weighted avg 0.92 0.91 0.90 627 |
| Validation F1-score: 0.9473 |
|
|
| Epoch 6/10, Train Loss: 0.0004, Validation Loss: 0.8895, Validation Accuracy: 0.9075 |
| precision recall f1-score support |
| 0 1.00 0.56 0.72 133 |
| 1 0.89 1.00 0.94 494 |
| accuracy 0.91 627 |
| macro avg 0.95 0.78 0.83 627 |
| weighted avg 0.92 0.91 0.90 627 |
| Validation F1-score: 0.9446 |
|
|
| Epoch 7/10, Train Loss: 0.0031, Validation Loss: 0.6923, Validation Accuracy: 0.9155 |
| precision recall f1-score support |
| 0 1.00 0.60 0.75 133 |
| 1 0.90 1.00 0.95 494 |
| accuracy 0.92 627 |
| macro avg 0.95 0.80 0.85 627 |
| weighted avg 0.92 0.92 0.91 627 |
| Validation F1-score: 0.9491 |
|
|
| Epoch 8/10, Train Loss: 0.0003, Validation Loss: 0.6224, Validation Accuracy: 0.9362 |
| precision recall f1-score support |
| 0 1.00 0.70 0.82 133 |
| 1 0.93 1.00 0.96 494 |
| accuracy 0.94 627 |
| macro avg 0.96 0.85 0.89 627 |
| weighted avg 0.94 0.94 0.93 627 |
| Validation F1-score: 0.9611 |
|
|
| Epoch 9/10, Train Loss: 0.0001, Validation Loss: 0.6299, Validation Accuracy: 0.9362 |
| precision recall f1-score support |
| 0 1.00 0.70 0.82 133 |
| 1 0.93 1.00 0.96 494 |
| accuracy 0.94 627 |
| macro avg 0.96 0.85 0.89 627 |
| weighted avg 0.94 0.94 0.93 627 |
| Validation F1-score: 0.9611 |
|
|
| Epoch 10/10, Train Loss: 0.0009, Validation Loss: 0.8330, Validation Accuracy: 0.9091 |
| precision recall f1-score support |
| 0 1.00 0.57 0.73 133 |
| 1 0.90 1.00 0.95 494 |
| accuracy 0.91 627 |
| macro avg 0.95 0.79 0.84 627 |
| weighted avg 0.92 0.91 0.90 627 |
| Validation F1-score: 0.9455 |
|
|
| ================================================================ |
| Best Model - Test Accuracy: 0.9541 |
| Best Model - Test Classification Report: |
| precision recall f1-score support |
| 0 0.90 0.80 0.85 100 |
| 1 0.96 0.98 0.97 532 |
| accuracy 0.95 632 |
| macro avg 0.93 0.89 0.91 632 |
| weighted avg 0.95 0.95 0.95 632 |
|
|
