st0 — Relation_detection.py (binary causal-or-not) Model: roberta-large Data: Combined_dataset_CommonSense+News_Data Train/Dev/Test: 3396 / 627 / 632 Label distribution train: 1=0.9167, 0=0.0833 dev: 1=0.7879, 0=0.2121 test: 1=0.8418, 0=0.1582 ================================================================ Epoch 1/10, Train Loss: 0.0438, Validation Loss: 0.4460, Validation Accuracy: 0.9282 precision recall f1-score support 0 0.98 0.68 0.80 133 1 0.92 1.00 0.96 494 accuracy 0.93 627 macro avg 0.95 0.84 0.88 627 weighted avg 0.93 0.93 0.92 627 Validation F1-score: 0.9563 Saved best model to /content/drive/MyDrive/causalsense/checkpoints/st0_roberta_large (val F1=0.9563) Epoch 2/10, Train Loss: 0.0246, Validation Loss: 0.9863, Validation Accuracy: 0.8788 precision recall f1-score support 0 1.00 0.43 0.60 133 1 0.87 1.00 0.93 494 accuracy 0.88 627 macro avg 0.93 0.71 0.76 627 weighted avg 0.89 0.88 0.86 627 Validation F1-score: 0.9286 Epoch 3/10, Train Loss: 0.0442, Validation Loss: 0.1793, Validation Accuracy: 0.9745 precision recall f1-score support 0 0.98 0.89 0.94 133 1 0.97 1.00 0.98 494 accuracy 0.97 627 macro avg 0.98 0.95 0.96 627 weighted avg 0.97 0.97 0.97 627 Validation F1-score: 0.9840 Saved best model to /content/drive/MyDrive/causalsense/checkpoints/st0_roberta_large (val F1=0.9840) *** BEST *** Epoch 4/10, Train Loss: 0.0134, Validation Loss: 0.6935, Validation Accuracy: 0.9187 precision recall f1-score support 0 1.00 0.62 0.76 133 1 0.91 1.00 0.95 494 accuracy 0.92 627 macro avg 0.95 0.81 0.86 627 weighted avg 0.93 0.92 0.91 627 Validation F1-score: 0.9509 Epoch 5/10, Train Loss: 0.0058, Validation Loss: 0.7291, Validation Accuracy: 0.9123 precision recall f1-score support 0 1.00 0.59 0.74 133 1 0.90 1.00 0.95 494 accuracy 0.91 627 macro avg 0.95 0.79 0.84 627 weighted avg 0.92 0.91 0.90 627 Validation F1-score: 0.9473 Epoch 6/10, Train Loss: 0.0004, Validation Loss: 0.8895, Validation Accuracy: 0.9075 precision recall f1-score support 0 1.00 0.56 0.72 133 1 0.89 1.00 0.94 494 accuracy 0.91 627 macro avg 0.95 0.78 0.83 627 weighted avg 0.92 0.91 0.90 627 Validation F1-score: 0.9446 Epoch 7/10, Train Loss: 0.0031, Validation Loss: 0.6923, Validation Accuracy: 0.9155 precision recall f1-score support 0 1.00 0.60 0.75 133 1 0.90 1.00 0.95 494 accuracy 0.92 627 macro avg 0.95 0.80 0.85 627 weighted avg 0.92 0.92 0.91 627 Validation F1-score: 0.9491 Epoch 8/10, Train Loss: 0.0003, Validation Loss: 0.6224, Validation Accuracy: 0.9362 precision recall f1-score support 0 1.00 0.70 0.82 133 1 0.93 1.00 0.96 494 accuracy 0.94 627 macro avg 0.96 0.85 0.89 627 weighted avg 0.94 0.94 0.93 627 Validation F1-score: 0.9611 Epoch 9/10, Train Loss: 0.0001, Validation Loss: 0.6299, Validation Accuracy: 0.9362 precision recall f1-score support 0 1.00 0.70 0.82 133 1 0.93 1.00 0.96 494 accuracy 0.94 627 macro avg 0.96 0.85 0.89 627 weighted avg 0.94 0.94 0.93 627 Validation F1-score: 0.9611 Epoch 10/10, Train Loss: 0.0009, Validation Loss: 0.8330, Validation Accuracy: 0.9091 precision recall f1-score support 0 1.00 0.57 0.73 133 1 0.90 1.00 0.95 494 accuracy 0.91 627 macro avg 0.95 0.79 0.84 627 weighted avg 0.92 0.91 0.90 627 Validation F1-score: 0.9455 ================================================================ Best Model - Test Accuracy: 0.9541 Best Model - Test Classification Report: precision recall f1-score support 0 0.90 0.80 0.85 100 1 0.96 0.98 0.97 532 accuracy 0.95 632 macro avg 0.93 0.89 0.91 632 weighted avg 0.95 0.95 0.95 632