Hugging Face's logo

shubov
/
omop_bert

Fill-Mask
Transformers
Safetensors
bert
Generated from Trainer
Model card Files
xet
Community
shubov commited on
Commit
8c895e2
·
verified ·
1 Parent(s): effd2e8

Model save

Browse files
Files changed (4) hide show
  1. README.md +66 -1
  2. model.safetensors +1 -1
  3. training_args.bin +1 -1
  4. vocab_tets.txt +0 -0
README.md CHANGED
@@ -12,6 +12,9 @@ should probably proofread and complete it, then remove this comment. -->
12
  # omop_bert
13
 
14
  This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
 
 
 
15
 
16
  ## Model description
17
 
@@ -37,10 +40,72 @@ The following hyperparameters were used during training:
37
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
38
  - lr_scheduler_type: linear
39
  - lr_scheduler_warmup_ratio: 0.1
40
- - training_steps: 3
41
 
42
  ### Training results
43
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
 
46
  ### Framework versions
 
12
  # omop_bert
13
 
14
  This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
15
+ It achieves the following results on the evaluation set:
16
+ - Loss: 0.9749
17
+ - Num Input Tokens Seen: 2457600000
18
 
19
  ## Model description
20
 
 
40
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
41
  - lr_scheduler_type: linear
42
  - lr_scheduler_warmup_ratio: 0.1
43
+ - training_steps: 300000
44
 
45
  ### Training results
46
 
47
+ | Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
48
+ |:-------------:|:-----:|:------:|:---------------:|:-----------------:|
49
+ | 5.8177 | 0.25 | 5000 | 5.7533 | 40960000 |
50
+ | 4.9885 | 0.49 | 10000 | 5.0065 | 81920000 |
51
+ | 3.7362 | 0.74 | 15000 | 3.6066 | 122880000 |
52
+ | 3.0498 | 0.98 | 20000 | 2.9935 | 163840000 |
53
+ | 2.0003 | 1.23 | 25000 | 1.8900 | 204800000 |
54
+ | 1.6255 | 1.47 | 30000 | 1.5739 | 245760000 |
55
+ | 1.5021 | 1.72 | 35000 | 1.4226 | 286720000 |
56
+ | 1.3458 | 1.97 | 40000 | 1.3465 | 327680000 |
57
+ | 1.376 | 2.21 | 45000 | 1.2986 | 368640000 |
58
+ | 1.3078 | 2.46 | 50000 | 1.2702 | 409600000 |
59
+ | 1.2738 | 2.7 | 55000 | 1.2256 | 450560000 |
60
+ | 1.2418 | 2.95 | 60000 | 1.2121 | 491520000 |
61
+ | 1.2492 | 3.2 | 65000 | 1.1953 | 532480000 |
62
+ | 1.1849 | 3.44 | 70000 | 1.1750 | 573440000 |
63
+ | 1.1839 | 3.69 | 75000 | 1.1656 | 614400000 |
64
+ | 1.2043 | 3.93 | 80000 | 1.1495 | 655360000 |
65
+ | 1.1442 | 4.18 | 85000 | 1.1444 | 696320000 |
66
+ | 1.1246 | 4.42 | 90000 | 1.1407 | 737280000 |
67
+ | 1.1114 | 4.67 | 95000 | 1.1261 | 778240000 |
68
+ | 1.1513 | 4.92 | 100000 | 1.1163 | 819200000 |
69
+ | 1.1115 | 5.16 | 105000 | 1.1098 | 860160000 |
70
+ | 1.0981 | 5.41 | 110000 | 1.1019 | 901120000 |
71
+ | 1.0474 | 5.65 | 115000 | 1.1050 | 942080000 |
72
+ | 1.1402 | 5.9 | 120000 | 1.0900 | 983040000 |
73
+ | 1.0973 | 6.15 | 125000 | 1.0888 | 1024000000 |
74
+ | 1.085 | 6.39 | 130000 | 1.0842 | 1064960000 |
75
+ | 1.0516 | 6.64 | 135000 | 1.0763 | 1105920000 |
76
+ | 1.1156 | 6.88 | 140000 | 1.0748 | 1146880000 |
77
+ | 1.0714 | 7.13 | 145000 | 1.0624 | 1187840000 |
78
+ | 1.0488 | 7.37 | 150000 | 1.0566 | 1228800000 |
79
+ | 1.0108 | 7.62 | 155000 | 1.0598 | 1269760000 |
80
+ | 1.0904 | 7.87 | 160000 | 1.0542 | 1310720000 |
81
+ | 1.0649 | 8.11 | 165000 | 1.0518 | 1351680000 |
82
+ | 1.081 | 8.36 | 170000 | 1.0457 | 1392640000 |
83
+ | 1.0291 | 8.6 | 175000 | 1.0437 | 1433600000 |
84
+ | 1.0512 | 8.85 | 180000 | 1.0335 | 1474560000 |
85
+ | 1.0435 | 9.09 | 185000 | 1.0322 | 1515520000 |
86
+ | 1.059 | 9.34 | 190000 | 1.0265 | 1556480000 |
87
+ | 1.0154 | 9.59 | 195000 | 1.0278 | 1597440000 |
88
+ | 1.0721 | 9.83 | 200000 | 1.0231 | 1638400000 |
89
+ | 1.053 | 10.08 | 205000 | 1.0138 | 1679360000 |
90
+ | 1.0359 | 10.32 | 210000 | 1.0135 | 1720320000 |
91
+ | 0.9925 | 10.57 | 215000 | 1.0122 | 1761280000 |
92
+ | 0.9817 | 10.82 | 220000 | 1.0084 | 1802240000 |
93
+ | 0.9951 | 11.06 | 225000 | 1.0123 | 1843200000 |
94
+ | 1.0224 | 11.31 | 230000 | 1.0130 | 1884160000 |
95
+ | 1.0071 | 11.55 | 235000 | 1.0054 | 1925120000 |
96
+ | 1.0307 | 11.8 | 240000 | 1.0006 | 1966080000 |
97
+ | 1.0063 | 12.04 | 245000 | 1.0004 | 2007040000 |
98
+ | 0.9985 | 12.29 | 250000 | 0.9925 | 2048000000 |
99
+ | 0.9682 | 12.54 | 255000 | 0.9966 | 2088960000 |
100
+ | 1.0115 | 12.78 | 260000 | 0.9936 | 2129920000 |
101
+ | 0.9699 | 13.03 | 265000 | 0.9891 | 2170880000 |
102
+ | 0.9837 | 13.27 | 270000 | 0.9848 | 2211840000 |
103
+ | 1.006 | 13.52 | 275000 | 0.9847 | 2252800000 |
104
+ | 0.9845 | 13.77 | 280000 | 0.9876 | 2293760000 |
105
+ | 0.9519 | 14.01 | 285000 | 0.9814 | 2334720000 |
106
+ | 0.9565 | 14.26 | 290000 | 0.9787 | 2375680000 |
107
+ | 0.9664 | 14.5 | 295000 | 0.9805 | 2416640000 |
108
+ | 0.9609 | 14.75 | 300000 | 0.9749 | 2457600000 |
109
 
110
 
111
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d96d4d61fe4527bf2ee6fe5cab25b0d943992d02b9ff13509a37fda24757b8e4
3
  size 426395156
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fda778b8ec565f261959863fa074279e0b67b743e0e7904f5c5b7187141f9e42
3
  size 426395156
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e4c71554ca43f4f16c79fe0e73c8e9a680d9fe47d96f7692d7759ebad3251297
3
  size 4728
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee69c44ad8fdf3dacf293697714f310c2481b4664ae24b12b949586c2665fc06
3
  size 4728
vocab_tets.txt ADDED
The diff for this file is too large to render. See raw diff