Anti-overfitting 5-class sentiment model

README.md

@@ -6,7 +6,6 @@ tags:
 - generated_from_trainer
 metrics:
 - accuracy
-- f1
 model-index:
 - name: mca-sentiment-analyzer-v2
   results: []
@@ -19,10 +18,10 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-sentiment-latest](https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment-latest) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.9026
+- Loss: 0.0871
 - Accuracy: 1.0
-- F1: 1.0
 - F1 Macro: 1.0
+- F1 Weighted: 1.0
 
 ## Model description
 
@@ -41,23 +40,25 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 1e-05
-- train_batch_size: 8
-- eval_batch_size: 8
+- learning_rate: 5e-05
+- train_batch_size: 4
+- eval_batch_size: 4
 - seed: 42
-- gradient_accumulation_steps: 2
+- gradient_accumulation_steps: 4
 - total_train_batch_size: 16
 - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
-- lr_scheduler_warmup_steps: 100
-- num_epochs: 5
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 3
 - mixed_precision_training: Native AMP
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1  | F1 Macro |
-|:-------------:|:-----:|:----:|:---------------:|:--------:|:---:|:--------:|
-| 1.284         | 3.88  | 50   | 0.9026          | 1.0      | 1.0 | 1.0      |
+| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 Macro | F1 Weighted |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:-----------:|
+| 1.2197        | 0.8   | 20   | 0.9300          | 0.63     | 0.5436   | 0.5436      |
+| 0.6318        | 1.6   | 40   | 0.3923          | 0.85     | 0.8255   | 0.8255      |
+| 0.3146        | 2.4   | 60   | 0.0871          | 1.0      | 1.0      | 1.0         |
 
 
 ### Framework versions