Hugging Face's logo

Mozart-coder
/
dna_bert_3_2-finetuned

Fill-Mask
Transformers
PyTorch
TensorBoard
bert
Generated from Trainer
Model card Files
xet
Metrics Community
1
Mozart-coder commited on
Commit
163a473
·
1 Parent(s): 58b54d0

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +155 -0
README.md ADDED
@@ -0,0 +1,155 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ model-index:
5
+ - name: dna_bert_3_2-finetuned
6
+ results: []
7
+ ---
8
+
9
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
10
+ should probably proofread and complete it, then remove this comment. -->
11
+
12
+ # dna_bert_3_2-finetuned
13
+
14
+ This model is a fine-tuned version of [armheb/DNA_bert_3](https://huggingface.co/armheb/DNA_bert_3) on an unknown dataset.
15
+ It achieves the following results on the evaluation set:
16
+ - Loss: 0.4668
17
+
18
+ ## Model description
19
+
20
+ More information needed
21
+
22
+ ## Intended uses & limitations
23
+
24
+ More information needed
25
+
26
+ ## Training and evaluation data
27
+
28
+ More information needed
29
+
30
+ ## Training procedure
31
+
32
+ ### Training hyperparameters
33
+
34
+ The following hyperparameters were used during training:
35
+ - learning_rate: 2e-05
36
+ - train_batch_size: 16
37
+ - eval_batch_size: 16
38
+ - seed: 42
39
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
40
+ - lr_scheduler_type: linear
41
+ - num_epochs: 100
42
+ - mixed_precision_training: Native AMP
43
+
44
+ ### Training results
45
+
46
+ | Training Loss | Epoch | Step | Validation Loss |
47
+ |:-------------:|:-----:|:----:|:---------------:|
48
+ | 0.8974 | 1.0 | 62 | 0.6160 |
49
+ | 0.6057 | 2.0 | 124 | 0.6000 |
50
+ | 0.5957 | 3.0 | 186 | 0.5897 |
51
+ | 0.5883 | 4.0 | 248 | 0.5873 |
52
+ | 0.5844 | 5.0 | 310 | 0.5843 |
53
+ | 0.5812 | 6.0 | 372 | 0.5811 |
54
+ | 0.5812 | 7.0 | 434 | 0.5832 |
55
+ | 0.5769 | 8.0 | 496 | 0.5773 |
56
+ | 0.5727 | 9.0 | 558 | 0.5771 |
57
+ | 0.5702 | 10.0 | 620 | 0.5772 |
58
+ | 0.5673 | 11.0 | 682 | 0.5771 |
59
+ | 0.5663 | 12.0 | 744 | 0.5769 |
60
+ | 0.5569 | 13.0 | 806 | 0.5731 |
61
+ | 0.5518 | 14.0 | 868 | 0.5731 |
62
+ | 0.5486 | 15.0 | 930 | 0.5728 |
63
+ | 0.544 | 16.0 | 992 | 0.5683 |
64
+ | 0.5336 | 17.0 | 1054 | 0.5694 |
65
+ | 0.5245 | 18.0 | 1116 | 0.5639 |
66
+ | 0.5162 | 19.0 | 1178 | 0.5641 |
67
+ | 0.5057 | 20.0 | 1240 | 0.5626 |
68
+ | 0.4966 | 21.0 | 1302 | 0.5612 |
69
+ | 0.4859 | 22.0 | 1364 | 0.5492 |
70
+ | 0.4781 | 23.0 | 1426 | 0.5470 |
71
+ | 0.4601 | 24.0 | 1488 | 0.5399 |
72
+ | 0.4523 | 25.0 | 1550 | 0.5424 |
73
+ | 0.4432 | 26.0 | 1612 | 0.5328 |
74
+ | 0.4341 | 27.0 | 1674 | 0.5336 |
75
+ | 0.4183 | 28.0 | 1736 | 0.5315 |
76
+ | 0.4133 | 29.0 | 1798 | 0.5268 |
77
+ | 0.4111 | 30.0 | 1860 | 0.5256 |
78
+ | 0.3919 | 31.0 | 1922 | 0.5155 |
79
+ | 0.3899 | 32.0 | 1984 | 0.5179 |
80
+ | 0.3804 | 33.0 | 2046 | 0.5145 |
81
+ | 0.368 | 34.0 | 2108 | 0.5189 |
82
+ | 0.3603 | 35.0 | 2170 | 0.5081 |
83
+ | 0.3602 | 36.0 | 2232 | 0.5098 |
84
+ | 0.352 | 37.0 | 2294 | 0.5054 |
85
+ | 0.3468 | 38.0 | 2356 | 0.5024 |
86
+ | 0.3359 | 39.0 | 2418 | 0.5053 |
87
+ | 0.3342 | 40.0 | 2480 | 0.5031 |
88
+ | 0.3294 | 41.0 | 2542 | 0.4978 |
89
+ | 0.3158 | 42.0 | 2604 | 0.4923 |
90
+ | 0.3191 | 43.0 | 2666 | 0.4944 |
91
+ | 0.3122 | 44.0 | 2728 | 0.4970 |
92
+ | 0.3084 | 45.0 | 2790 | 0.4910 |
93
+ | 0.2978 | 46.0 | 2852 | 0.4898 |
94
+ | 0.3012 | 47.0 | 2914 | 0.4880 |
95
+ | 0.2938 | 48.0 | 2976 | 0.4924 |
96
+ | 0.2932 | 49.0 | 3038 | 0.4879 |
97
+ | 0.2842 | 50.0 | 3100 | 0.4847 |
98
+ | 0.2828 | 51.0 | 3162 | 0.4849 |
99
+ | 0.2793 | 52.0 | 3224 | 0.4767 |
100
+ | 0.2753 | 53.0 | 3286 | 0.4796 |
101
+ | 0.2725 | 54.0 | 3348 | 0.4829 |
102
+ | 0.2695 | 55.0 | 3410 | 0.4831 |
103
+ | 0.2671 | 56.0 | 3472 | 0.4791 |
104
+ | 0.2664 | 57.0 | 3534 | 0.4791 |
105
+ | 0.2563 | 58.0 | 3596 | 0.4765 |
106
+ | 0.2583 | 59.0 | 3658 | 0.4742 |
107
+ | 0.2535 | 60.0 | 3720 | 0.4766 |
108
+ | 0.2496 | 61.0 | 3782 | 0.4741 |
109
+ | 0.2489 | 62.0 | 3844 | 0.4766 |
110
+ | 0.2444 | 63.0 | 3906 | 0.4748 |
111
+ | 0.2417 | 64.0 | 3968 | 0.4768 |
112
+ | 0.2422 | 65.0 | 4030 | 0.4727 |
113
+ | 0.2404 | 66.0 | 4092 | 0.4729 |
114
+ | 0.2405 | 67.0 | 4154 | 0.4744 |
115
+ | 0.2353 | 68.0 | 4216 | 0.4729 |
116
+ | 0.2307 | 69.0 | 4278 | 0.4705 |
117
+ | 0.2281 | 70.0 | 4340 | 0.4717 |
118
+ | 0.232 | 71.0 | 4402 | 0.4719 |
119
+ | 0.2313 | 72.0 | 4464 | 0.4713 |
120
+ | 0.2266 | 73.0 | 4526 | 0.4726 |
121
+ | 0.2241 | 74.0 | 4588 | 0.4675 |
122
+ | 0.2256 | 75.0 | 4650 | 0.4688 |
123
+ | 0.2299 | 76.0 | 4712 | 0.4713 |
124
+ | 0.2199 | 77.0 | 4774 | 0.4720 |
125
+ | 0.2228 | 78.0 | 4836 | 0.4682 |
126
+ | 0.2261 | 79.0 | 4898 | 0.4676 |
127
+ | 0.2167 | 80.0 | 4960 | 0.4685 |
128
+ | 0.2126 | 81.0 | 5022 | 0.4676 |
129
+ | 0.2217 | 82.0 | 5084 | 0.4672 |
130
+ | 0.216 | 83.0 | 5146 | 0.4672 |
131
+ | 0.2152 | 84.0 | 5208 | 0.4682 |
132
+ | 0.219 | 85.0 | 5270 | 0.4663 |
133
+ | 0.2135 | 86.0 | 5332 | 0.4655 |
134
+ | 0.2046 | 87.0 | 5394 | 0.4644 |
135
+ | 0.2177 | 88.0 | 5456 | 0.4679 |
136
+ | 0.2052 | 89.0 | 5518 | 0.4659 |
137
+ | 0.2147 | 90.0 | 5580 | 0.4665 |
138
+ | 0.211 | 91.0 | 5642 | 0.4668 |
139
+ | 0.2089 | 92.0 | 5704 | 0.4649 |
140
+ | 0.2149 | 93.0 | 5766 | 0.4651 |
141
+ | 0.2034 | 94.0 | 5828 | 0.4689 |
142
+ | 0.2071 | 95.0 | 5890 | 0.4659 |
143
+ | 0.2145 | 96.0 | 5952 | 0.4664 |
144
+ | 0.2036 | 97.0 | 6014 | 0.4661 |
145
+ | 0.2092 | 98.0 | 6076 | 0.4676 |
146
+ | 0.2079 | 99.0 | 6138 | 0.4667 |
147
+ | 0.2081 | 100.0 | 6200 | 0.4668 |
148
+
149
+
150
+ ### Framework versions
151
+
152
+ - Transformers 4.21.1
153
+ - Pytorch 1.12.0+cu113
154
+ - Datasets 2.4.0
155
+ - Tokenizers 0.12.1