Alfahluzi/bert2bert-extreme-dropout-0.5-lr-5e-05-batchsize-4-encmaxlen-2048-decmaxlen-512 train 5 epochs with 4 batch size

README.md

@@ -13,18 +13,18 @@ should probably proofread and complete it, then remove this comment. -->
 
 # bert2bert-extreme-dropout-0.5-lr-5e-05-batchsize-4-encmaxlen-2048-decmaxlen-512
 
-This model is a fine-tuned version of [](https://huggingface.co/) on the id_liputan6 dataset.
+This model was trained from scratch on the id_liputan6 dataset.
 It achieves the following results on the evaluation set:
-- Loss: 9.2760
-- R1 Precision: 0.0
-- R1 Recall: 0.0
-- R1 Fmeasure: 0.0
+- Loss: 8.6177
+- R1 Precision: 0.0188
+- R1 Recall: 0.0105
+- R1 Fmeasure: 0.0133
 - R2 Precision: 0.0
 - R2 Recall: 0.0
 - R2 Fmeasure: 0.0
-- Rl Precision: 0.0
-- Rl Recall: 0.0
-- Rl Fmeasure: 0.0
+- Rl Precision: 0.0188
+- Rl Recall: 0.0105
+- Rl Fmeasure: 0.0133
 
 ## Model description
 
@@ -44,8 +44,8 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 5e-05
-- train_batch_size: 1
-- eval_batch_size: 1
+- train_batch_size: 2
+- eval_batch_size: 2
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
@@ -54,18 +54,18 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | R1 Precision | R1 Recall | R1 Fmeasure | R2 Precision | R2 Recall | R2 Fmeasure | Rl Precision | Rl Recall | Rl Fmeasure |
-|:-------------:|:-----:|:----:|:---------------:|:------------:|:---------:|:-----------:|:------------:|:---------:|:-----------:|:------------:|:---------:|:-----------:|
-| No log        | 1.0   | 8    | 9.7200          | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         |
-| No log        | 2.0   | 16   | 9.5455          | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         |
-| No log        | 3.0   | 24   | 9.3678          | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         |
-| No log        | 4.0   | 32   | 9.2887          | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         |
-| No log        | 5.0   | 40   | 9.2760          | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         |
+| Training Loss | Epoch | Step   | Validation Loss | R1 Precision | R1 Recall | R1 Fmeasure | R2 Precision | R2 Recall | R2 Fmeasure | Rl Precision | Rl Recall | Rl Fmeasure |
+|:-------------:|:-----:|:------:|:---------------:|:------------:|:---------:|:-----------:|:------------:|:---------:|:-----------:|:------------:|:---------:|:-----------:|
+| 7.0769        | 1.0   | 96942  | 7.5336          | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         |
+| 7.1014        | 2.0   | 193884 | 7.6800          | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         | 0.0          | 0.0       | 0.0         |
+| 7.0648        | 3.0   | 290826 | 8.1448          | 0.0188       | 0.0105    | 0.0133      | 0.0          | 0.0       | 0.0         | 0.0188       | 0.0105    | 0.0133      |
+| 7.0594        | 4.0   | 387768 | 8.4518          | 0.0188       | 0.0105    | 0.0133      | 0.0          | 0.0       | 0.0         | 0.0188       | 0.0105    | 0.0133      |
+| 7.0322        | 5.0   | 484710 | 8.6177          | 0.0188       | 0.0105    | 0.0133      | 0.0          | 0.0       | 0.0         | 0.0188       | 0.0105    | 0.0133      |
 
 
 ### Framework versions
 
 - Transformers 4.38.2
-- Pytorch 2.2.1+cu121
+- Pytorch 2.2.1
 - Datasets 2.18.0
 - Tokenizers 0.15.2

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b8b9d2b8f016714b23468d3f6427e14287031b2f400d9bdd013b458ff4bf12e1
+oid sha256:082e249bbd0c1c090f7be9e9d81d21b5ab5e269f86208c86e6639ee0b05c3b9c
 size 1002850732

runs/Mar18_09-33-55_c10457f3b6ab/events.out.tfevents.1710754435.c10457f3b6ab.370.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1e97c230d545535269a53a7b2ce0a6227c1b6138a3c450789f5abbb1301de986
-size 220447
+oid sha256:bd5e77b0d2589ca82a01c1aee3f61c09ebeb1ee14c562a86fac9e4b0601f3239
+size 221584