Update README.md
Browse files
README.md
CHANGED
|
@@ -35,39 +35,36 @@ For more information on my dataset, please see the included referenced dataset.
|
|
| 35 |
|
| 36 |
# Hyperparameters
|
| 37 |
|
| 38 |
-
MAX_SOURCE_LENGTH = 256
|
| 39 |
-
MAX_TARGET_LENGTH = 128
|
| 40 |
-
BATCH_SIZE = 16
|
| 41 |
-
NUM_EPOCHS = 35
|
| 42 |
-
LEARNING_RATE = 3e-5
|
| 43 |
-
GRADIENT_ACCUMULATION_STEPS = 4
|
| 44 |
-
EARLY_STOPPING_PATIENCE = 2
|
| 45 |
-
WEIGHT_DECAY = 0.01
|
| 46 |
-
OPTIMIZER = ADAFACTOR
|
| 47 |
-
LR_SCHEDULER = LINEAR
|
| 48 |
|
| 49 |
# Loss
|
| 50 |
|
| 51 |
On the 35th epoch, the model achieved the following loss:
|
| 52 |
|
| 53 |
-
Epoch Training Loss Validation Loss
|
| 54 |
-
26 1.001400 1.288712
|
| 55 |
-
27 0.983600 1.284895
|
| 56 |
-
28 0.961300 1.277680
|
| 57 |
-
29 0.940600 1.275018
|
| 58 |
-
30 0.933600 1.275621
|
| 59 |
-
31 0.918200 1.270074
|
| 60 |
-
32 0.904700 1.268874
|
| 61 |
-
33 0.908800 1.268534
|
| 62 |
-
34 0.900600 1.268240
|
| 63 |
-
*35* *0.894800* *1.268536*
|
| 64 |
|
| 65 |
# BLEU and ROUGE Scores
|
| 66 |
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
BLEU Score: 0.3540
|
| 71 |
-
ROUGE-1: 0.5855
|
| 72 |
-
ROUGE-2: 0.3946
|
| 73 |
ROUGE-L: 0.5243
|
|
|
|
| 35 |
|
| 36 |
# Hyperparameters
|
| 37 |
|
| 38 |
+
MAX_SOURCE_LENGTH = 256 <br>
|
| 39 |
+
MAX_TARGET_LENGTH = 128 <br>
|
| 40 |
+
BATCH_SIZE = 16 <br>
|
| 41 |
+
NUM_EPOCHS = 35 <br>
|
| 42 |
+
LEARNING_RATE = 3e-5 <br>
|
| 43 |
+
GRADIENT_ACCUMULATION_STEPS = 4 <br>
|
| 44 |
+
EARLY_STOPPING_PATIENCE = 2 <br>
|
| 45 |
+
WEIGHT_DECAY = 0.01 <br>
|
| 46 |
+
OPTIMIZER = ADAFACTOR <br>
|
| 47 |
+
LR_SCHEDULER = LINEAR <br>
|
| 48 |
|
| 49 |
# Loss
|
| 50 |
|
| 51 |
On the 35th epoch, the model achieved the following loss:
|
| 52 |
|
| 53 |
+
Epoch Training Loss Validation Loss <br>
|
| 54 |
+
26 1.001400 1.288712 <br>
|
| 55 |
+
27 0.983600 1.284895 <br>
|
| 56 |
+
28 0.961300 1.277680 <br>
|
| 57 |
+
29 0.940600 1.275018 <br>
|
| 58 |
+
30 0.933600 1.275621 <br>
|
| 59 |
+
31 0.918200 1.270074 <br>
|
| 60 |
+
32 0.904700 1.268874 <br>
|
| 61 |
+
33 0.908800 1.268534 <br>
|
| 62 |
+
34 0.900600 1.268240 <br>
|
| 63 |
+
*35* *0.894800* *1.268536* <br>
|
| 64 |
|
| 65 |
# BLEU and ROUGE Scores
|
| 66 |
|
| 67 |
+
BLEU Score: 0.3540 <br>
|
| 68 |
+
ROUGE-1: 0.5855 <br>
|
| 69 |
+
ROUGE-2: 0.3946 <br>
|
|
|
|
|
|
|
|
|
|
| 70 |
ROUGE-L: 0.5243
|