Hugging Face's logo

NasimB
/
bert-dp-second

Fill-Mask
Transformers
PyTorch
bert
Generated from Trainer
Model card Files
xet
Community
1
NasimB commited on
Commit
98c8439
·
1 Parent(s): 0cf9fab

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +100 -0
README.md ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ datasets:
5
+ - generator
6
+ model-index:
7
+ - name: bert-dp-second
8
+ results: []
9
+ ---
10
+
11
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
+ should probably proofread and complete it, then remove this comment. -->
13
+
14
+ # bert-dp-second
15
+
16
+ This model is a fine-tuned version of [](https://huggingface.co/) on the generator dataset.
17
+ It achieves the following results on the evaluation set:
18
+ - Loss: 3.5640
19
+
20
+ ## Model description
21
+
22
+ More information needed
23
+
24
+ ## Intended uses & limitations
25
+
26
+ More information needed
27
+
28
+ ## Training and evaluation data
29
+
30
+ More information needed
31
+
32
+ ## Training procedure
33
+
34
+ ### Training hyperparameters
35
+
36
+ The following hyperparameters were used during training:
37
+ - learning_rate: 0.0005
38
+ - train_batch_size: 64
39
+ - eval_batch_size: 64
40
+ - seed: 42
41
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
+ - lr_scheduler_type: cosine
43
+ - lr_scheduler_warmup_steps: 1000
44
+ - num_epochs: 10
45
+ - mixed_precision_training: Native AMP
46
+
47
+ ### Training results
48
+
49
+ | Training Loss | Epoch | Step | Validation Loss |
50
+ |:-------------:|:-----:|:-----:|:---------------:|
51
+ | 7.3416 | 0.23 | 500 | 6.6532 |
52
+ | 6.5752 | 0.47 | 1000 | 6.5275 |
53
+ | 6.4866 | 0.7 | 1500 | 6.4720 |
54
+ | 6.4273 | 0.93 | 2000 | 6.4540 |
55
+ | 6.4036 | 1.17 | 2500 | 6.4236 |
56
+ | 6.3779 | 1.4 | 3000 | 6.4018 |
57
+ | 6.3528 | 1.63 | 3500 | 6.3768 |
58
+ | 6.3258 | 1.87 | 4000 | 6.3679 |
59
+ | 6.3009 | 2.1 | 4500 | 6.3305 |
60
+ | 6.2646 | 2.33 | 5000 | 6.3142 |
61
+ | 6.2583 | 2.57 | 5500 | 6.3004 |
62
+ | 6.2223 | 2.8 | 6000 | 6.2605 |
63
+ | 6.1941 | 3.03 | 6500 | 6.2353 |
64
+ | 6.1382 | 3.27 | 7000 | 6.2095 |
65
+ | 6.1301 | 3.5 | 7500 | 6.1774 |
66
+ | 6.09 | 3.73 | 8000 | 6.1480 |
67
+ | 6.0624 | 3.97 | 8500 | 6.1061 |
68
+ | 6.0056 | 4.2 | 9000 | 6.0655 |
69
+ | 5.9444 | 4.43 | 9500 | 5.9461 |
70
+ | 5.7101 | 4.67 | 10000 | 5.2594 |
71
+ | 5.005 | 4.9 | 10500 | 4.7348 |
72
+ | 4.6127 | 5.13 | 11000 | 4.4626 |
73
+ | 4.3907 | 5.37 | 11500 | 4.2862 |
74
+ | 4.241 | 5.6 | 12000 | 4.1701 |
75
+ | 4.1286 | 5.83 | 12500 | 4.0673 |
76
+ | 4.0151 | 6.07 | 13000 | 3.9967 |
77
+ | 3.934 | 6.3 | 13500 | 3.9292 |
78
+ | 3.8789 | 6.53 | 14000 | 3.8707 |
79
+ | 3.8231 | 6.77 | 14500 | 3.8222 |
80
+ | 3.7696 | 7.0 | 15000 | 3.7800 |
81
+ | 3.7078 | 7.23 | 15500 | 3.7424 |
82
+ | 3.6671 | 7.47 | 16000 | 3.7093 |
83
+ | 3.6446 | 7.7 | 16500 | 3.6780 |
84
+ | 3.6069 | 7.93 | 17000 | 3.6476 |
85
+ | 3.5782 | 8.17 | 17500 | 3.6283 |
86
+ | 3.5384 | 8.4 | 18000 | 3.6098 |
87
+ | 3.5245 | 8.63 | 18500 | 3.5942 |
88
+ | 3.5209 | 8.87 | 19000 | 3.5841 |
89
+ | 3.4948 | 9.1 | 19500 | 3.5728 |
90
+ | 3.4877 | 9.33 | 20000 | 3.5692 |
91
+ | 3.4818 | 9.57 | 20500 | 3.5641 |
92
+ | 3.4844 | 9.8 | 21000 | 3.5640 |
93
+
94
+
95
+ ### Framework versions
96
+
97
+ - Transformers 4.26.1
98
+ - Pytorch 1.11.0+cu113
99
+ - Datasets 2.13.0
100
+ - Tokenizers 0.13.3