End of training

README.md

@@ -1,6 +1,4 @@
 ---
-license: mit
-base_model: LIAMF-USP/roberta-large-finetuned-race
 tags:
 - generated_from_trainer
 metrics:
@@ -18,13 +16,13 @@ should probably proofread and complete it, then remove this comment. -->
 
 # roberta-mqa-formrat
 
-This model is a fine-tuned version of [LIAMF-USP/roberta-large-finetuned-race](https://huggingface.co/LIAMF-USP/roberta-large-finetuned-race) on an unknown dataset.
+This model was trained from scratch on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.6094
-- Accuracy: 0.2075
-- F1: 0.1943
-- Precision: 0.2025
-- Recall: 0.2019
+- Loss: 1.1135
+- Accuracy: 0.5671
+- F1: 0.5659
+- Precision: 0.5683
+- Recall: 0.5650
 
 ## Model description
 
@@ -44,9 +42,11 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 2e-05
-- train_batch_size: 8
+- train_batch_size: 4
 - eval_batch_size: 16
 - seed: 42
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 8
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - num_epochs: 3
@@ -54,11 +54,17 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1     | Precision | Recall |
-|:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|:---------:|:------:|
-| 1.6083        | 1.0   | 3712  | 1.6094          | 0.1981   | 0.1925 | 0.1939    | 0.1944 |
-| 1.6124        | 2.0   | 7424  | 1.6094          | 0.2050   | 0.2020 | 0.2033    | 0.2030 |
-| 1.6113        | 3.0   | 11136 | 1.6094          | 0.2075   | 0.1943 | 0.2025    | 0.2019 |
+| Training Loss | Epoch  | Step  | Validation Loss | Accuracy | F1     | Precision | Recall |
+|:-------------:|:------:|:-----:|:---------------:|:--------:|:------:|:---------:|:------:|
+| 1.451         | 0.3233 | 1200  | 1.4125          | 0.4105   | 0.4093 | 0.4151    | 0.4107 |
+| 1.416         | 0.6466 | 2400  | 1.3482          | 0.4412   | 0.4394 | 0.4438    | 0.4385 |
+| 1.3157        | 0.9698 | 3600  | 1.2933          | 0.4788   | 0.4772 | 0.4776    | 0.4773 |
+| 1.2616        | 1.2931 | 4800  | 1.2389          | 0.5032   | 0.5022 | 0.5053    | 0.5011 |
+| 1.221         | 1.6164 | 6000  | 1.2049          | 0.5053   | 0.5039 | 0.5060    | 0.5029 |
+| 1.1556        | 1.9397 | 7200  | 1.1792          | 0.5288   | 0.5276 | 0.5295    | 0.5265 |
+| 1.082         | 2.2629 | 8400  | 1.1593          | 0.5451   | 0.5434 | 0.5487    | 0.5415 |
+| 1.0692        | 2.5862 | 9600  | 1.1153          | 0.5613   | 0.5606 | 0.5641    | 0.5594 |
+| 1.0066        | 2.9095 | 10800 | 1.1135          | 0.5671   | 0.5659 | 0.5683    | 0.5650 |
 
 
 ### Framework versions

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8c8e475a3070772f94fdea14b7c678f5d4c7b77c7bf55fdc99e05abbb517cdfe
+oid sha256:7dd9ac53780af5f8be9aefe9efc425f427a115ce6b44c87c76109eb7bc2bd313
 size 1421491284

tokenizer_config.json

@@ -48,10 +48,17 @@
   "eos_token": "</s>",
   "errors": "replace",
   "mask_token": "<mask>",
+  "max_length": 128,
   "model_max_length": 512,
+  "pad_to_multiple_of": null,
   "pad_token": "<pad>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
   "sep_token": "</s>",
+  "stride": 0,
   "tokenizer_class": "RobertaTokenizer",
   "trim_offsets": true,
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
   "unk_token": "<unk>"
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d1824ef8e236124f54b9356f057c527bb711db5dfbc1105239f1e66ac956410f
+oid sha256:ba3fb9ae00b42cfbf8814d7e9a4a384a5358e732b8de29d5dd573ebcfcd93c92
 size 4920