End of training

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [google/rembert](https://huggingface.co/google/rembert) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.6640
+- Loss: 0.6718
 - Accuracy: 0.9316
 - 1-f1: 0.0
 - 1-recall: 0.0
@@ -48,19 +48,24 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.06
 - num_epochs: 20
 - mixed_precision_training: Native AMP
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Accuracy | 1-f1 | 1-recall | 1-precision | Balanced Acc |
-|:-------------:|:-----:|:----:|:---------------:|:--------:|:----:|:--------:|:-----------:|:------------:|
-| 0.5979        | 1.0   | 95   | 0.6615          | 0.9316   | 0.0  | 0.0      | 0.0         | 0.5          |
-| 0.4411        | 2.0   | 190  | 0.6610          | 0.9316   | 0.0  | 0.0      | 0.0         | 0.5          |
-| 0.6826        | 3.0   | 285  | 0.6614          | 0.9316   | 0.0  | 0.0      | 0.0         | 0.5          |
-| 0.6023        | 4.0   | 380  | 0.6607          | 0.9316   | 0.0  | 0.0      | 0.0         | 0.5          |
-| 0.5391        | 5.0   | 475  | 0.6652          | 0.9316   | 0.0  | 0.0      | 0.0         | 0.5          |
-| 0.6631        | 6.0   | 570  | 0.6640          | 0.9316   | 0.0  | 0.0      | 0.0         | 0.5          |
+| Training Loss | Epoch | Step | Validation Loss | Accuracy | 1-f1   | 1-recall | 1-precision | Balanced Acc |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:--------:|:-----------:|:------------:|
+| 0.5294        | 1.0   | 95   | 0.6951          | 0.4474   | 0.0870 | 0.3846   | 0.0490      | 0.4183       |
+| 0.4281        | 2.0   | 190  | 0.7576          | 0.2395   | 0.0707 | 0.4231   | 0.0386      | 0.3245       |
+| 0.6661        | 3.0   | 285  | 0.8551          | 0.0684   | 0.1281 | 1.0      | 0.0684      | 0.5          |
+| 0.6035        | 4.0   | 380  | 0.6809          | 0.9316   | 0.0    | 0.0      | 0.0         | 0.5          |
+| 0.451         | 5.0   | 475  | 0.6636          | 0.9316   | 0.0    | 0.0      | 0.0         | 0.5          |
+| 0.6474        | 6.0   | 570  | 0.6653          | 0.9316   | 0.0    | 0.0      | 0.0         | 0.5          |
+| 0.8721        | 7.0   | 665  | 0.7759          | 0.9316   | 0.0    | 0.0      | 0.0         | 0.5          |
+| 0.6361        | 8.0   | 760  | 0.6713          | 0.9316   | 0.0    | 0.0      | 0.0         | 0.5          |
+| 0.4965        | 9.0   | 855  | 0.7745          | 0.9316   | 0.0    | 0.0      | 0.0         | 0.5          |
+| 0.8257        | 10.0  | 950  | 0.6718          | 0.9316   | 0.0    | 0.0      | 0.0         | 0.5          |
 
 
 ### Framework versions

config.json

@@ -3,14 +3,14 @@
   "architectures": [
     "RemBertForSequenceClassification"
   ],
-  "attention_probs_dropout_prob": 0,
+  "attention_probs_dropout_prob": 0.2,
   "bos_token_id": 312,
   "classifier_dropout_prob": 0.1,
   "embedding_dropout_prob": 0,
   "embedding_size": 256,
   "eos_token_id": 313,
   "hidden_act": "gelu",
-  "hidden_dropout_prob": 0,
+  "hidden_dropout_prob": 0.2,
   "hidden_size": 1152,
   "initializer_range": 0.02,
   "input_embedding_size": 256,

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:032bb14e5969234528cfccda146ee478e5af6a932c81ddb39010b6de3ec2e44b
+oid sha256:397d3b095cb32cc82b737c9d5c675f25473ae6a755fdc2b184ec336c762ade7e
 size 2303755328

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:de1dde975fd3565a77e0fb7a7f0d5ea452a9820b3144b63f0a82ed2dd9d1a947
+oid sha256:40e7b2ba224b5bcdbadc1c2111cc12143052fc21b191a9f9636d707844f8d520
 size 5432