| | --- |
| | tags: |
| | - generated_from_trainer |
| | model-index: |
| | - name: dna_bert_3_1000seq-finetuned |
| | results: [] |
| | --- |
| | |
| | <!-- This model card has been generated automatically according to the information the Trainer had access to. You |
| | should probably proofread and complete it, then remove this comment. --> |
| |
|
| | # dna_bert_3_1000seq-finetuned |
| | |
| | This model is a fine-tuned version of [armheb/DNA_bert_3](https://huggingface.co/armheb/DNA_bert_3) on an unknown dataset. |
| | It achieves the following results on the evaluation set: |
| | - Loss: 0.4684 |
| | |
| | ## Model description |
| | |
| | More information needed |
| | |
| | ## Intended uses & limitations |
| | |
| | More information needed |
| | |
| | ## Training and evaluation data |
| | |
| | More information needed |
| | |
| | ## Training procedure |
| | |
| | ### Training hyperparameters |
| | |
| | The following hyperparameters were used during training: |
| | - learning_rate: 2e-05 |
| | - train_batch_size: 16 |
| | - eval_batch_size: 16 |
| | - seed: 42 |
| | - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
| | - lr_scheduler_type: linear |
| | - num_epochs: 100 |
| | - mixed_precision_training: Native AMP |
| | |
| | ### Training results |
| | |
| | | Training Loss | Epoch | Step | Validation Loss | |
| | |:-------------:|:-----:|:----:|:---------------:| |
| | | 0.8607 | 1.0 | 62 | 0.6257 | |
| | | 0.6177 | 2.0 | 124 | 0.6120 | |
| | | 0.6098 | 3.0 | 186 | 0.6062 | |
| | | 0.604 | 4.0 | 248 | 0.6052 | |
| | | 0.5999 | 5.0 | 310 | 0.6040 | |
| | | 0.5982 | 6.0 | 372 | 0.5996 | |
| | | 0.5985 | 7.0 | 434 | 0.5985 | |
| | | 0.5956 | 8.0 | 496 | 0.5968 | |
| | | 0.5936 | 9.0 | 558 | 0.5950 | |
| | | 0.5908 | 10.0 | 620 | 0.5941 | |
| | | 0.5904 | 11.0 | 682 | 0.5932 | |
| | | 0.59 | 12.0 | 744 | 0.5917 | |
| | | 0.5877 | 13.0 | 806 | 0.5921 | |
| | | 0.5847 | 14.0 | 868 | 0.5903 | |
| | | 0.5831 | 15.0 | 930 | 0.5887 | |
| | | 0.5852 | 16.0 | 992 | 0.5878 | |
| | | 0.5805 | 17.0 | 1054 | 0.5872 | |
| | | 0.5795 | 18.0 | 1116 | 0.5853 | |
| | | 0.5754 | 19.0 | 1178 | 0.5869 | |
| | | 0.5757 | 20.0 | 1240 | 0.5839 | |
| | | 0.5722 | 21.0 | 1302 | 0.5831 | |
| | | 0.5693 | 22.0 | 1364 | 0.5811 | |
| | | 0.5667 | 23.0 | 1426 | 0.5802 | |
| | | 0.5652 | 24.0 | 1488 | 0.5775 | |
| | | 0.5608 | 25.0 | 1550 | 0.5788 | |
| | | 0.5591 | 26.0 | 1612 | 0.5724 | |
| | | 0.5538 | 27.0 | 1674 | 0.5736 | |
| | | 0.552 | 28.0 | 1736 | 0.5689 | |
| | | 0.5483 | 29.0 | 1798 | 0.5689 | |
| | | 0.5442 | 30.0 | 1860 | 0.5671 | |
| | | 0.5405 | 31.0 | 1922 | 0.5658 | |
| | | 0.537 | 32.0 | 1984 | 0.5605 | |
| | | 0.5349 | 33.0 | 2046 | 0.5575 | |
| | | 0.5275 | 34.0 | 2108 | 0.5569 | |
| | | 0.5227 | 35.0 | 2170 | 0.5537 | |
| | | 0.52 | 36.0 | 2232 | 0.5509 | |
| | | 0.5173 | 37.0 | 2294 | 0.5504 | |
| | | 0.5123 | 38.0 | 2356 | 0.5435 | |
| | | 0.5088 | 39.0 | 2418 | 0.5472 | |
| | | 0.5037 | 40.0 | 2480 | 0.5383 | |
| | | 0.501 | 41.0 | 2542 | 0.5379 | |
| | | 0.4931 | 42.0 | 2604 | 0.5365 | |
| | | 0.4923 | 43.0 | 2666 | 0.5328 | |
| | | 0.4879 | 44.0 | 2728 | 0.5301 | |
| | | 0.482 | 45.0 | 2790 | 0.5295 | |
| | | 0.4805 | 46.0 | 2852 | 0.5261 | |
| | | 0.4772 | 47.0 | 2914 | 0.5221 | |
| | | 0.4738 | 48.0 | 2976 | 0.5234 | |
| | | 0.4674 | 49.0 | 3038 | 0.5210 | |
| | | 0.4646 | 50.0 | 3100 | 0.5169 | |
| | | 0.4621 | 51.0 | 3162 | 0.5142 | |
| | | 0.4574 | 52.0 | 3224 | 0.5129 | |
| | | 0.4552 | 53.0 | 3286 | 0.5127 | |
| | | 0.4539 | 54.0 | 3348 | 0.5124 | |
| | | 0.4506 | 55.0 | 3410 | 0.5076 | |
| | | 0.4457 | 56.0 | 3472 | 0.5082 | |
| | | 0.4454 | 57.0 | 3534 | 0.5027 | |
| | | 0.4398 | 58.0 | 3596 | 0.5019 | |
| | | 0.4386 | 59.0 | 3658 | 0.4998 | |
| | | 0.4332 | 60.0 | 3720 | 0.4970 | |
| | | 0.4277 | 61.0 | 3782 | 0.4995 | |
| | | 0.4273 | 62.0 | 3844 | 0.4962 | |
| | | 0.4235 | 63.0 | 3906 | 0.4909 | |
| | | 0.4201 | 64.0 | 3968 | 0.4913 | |
| | | 0.4198 | 65.0 | 4030 | 0.4899 | |
| | | 0.4182 | 66.0 | 4092 | 0.4919 | |
| | | 0.4157 | 67.0 | 4154 | 0.4902 | |
| | | 0.4104 | 68.0 | 4216 | 0.4881 | |
| | | 0.4095 | 69.0 | 4278 | 0.4881 | |
| | | 0.4077 | 70.0 | 4340 | 0.4861 | |
| | | 0.4064 | 71.0 | 4402 | 0.4868 | |
| | | 0.4041 | 72.0 | 4464 | 0.4826 | |
| | | 0.4029 | 73.0 | 4526 | 0.4833 | |
| | | 0.3976 | 74.0 | 4588 | 0.4819 | |
| | | 0.3997 | 75.0 | 4650 | 0.4809 | |
| | | 0.3974 | 76.0 | 4712 | 0.4801 | |
| | | 0.3953 | 77.0 | 4774 | 0.4783 | |
| | | 0.3938 | 78.0 | 4836 | 0.4775 | |
| | | 0.3934 | 79.0 | 4898 | 0.4762 | |
| | | 0.3923 | 80.0 | 4960 | 0.4742 | |
| | | 0.3893 | 81.0 | 5022 | 0.4742 | |
| | | 0.3909 | 82.0 | 5084 | 0.4740 | |
| | | 0.3856 | 83.0 | 5146 | 0.4739 | |
| | | 0.3904 | 84.0 | 5208 | 0.4740 | |
| | | 0.3883 | 85.0 | 5270 | 0.4701 | |
| | | 0.3865 | 86.0 | 5332 | 0.4727 | |
| | | 0.3809 | 87.0 | 5394 | 0.4736 | |
| | | 0.3853 | 88.0 | 5456 | 0.4704 | |
| | | 0.3821 | 89.0 | 5518 | 0.4704 | |
| | | 0.3809 | 90.0 | 5580 | 0.4701 | |
| | | 0.3814 | 91.0 | 5642 | 0.4698 | |
| | | 0.3795 | 92.0 | 5704 | 0.4702 | |
| | | 0.3804 | 93.0 | 5766 | 0.4692 | |
| | | 0.377 | 94.0 | 5828 | 0.4683 | |
| | | 0.3812 | 95.0 | 5890 | 0.4692 | |
| | | 0.3806 | 96.0 | 5952 | 0.4683 | |
| | | 0.3745 | 97.0 | 6014 | 0.4690 | |
| | | 0.3825 | 98.0 | 6076 | 0.4684 | |
| | | 0.374 | 99.0 | 6138 | 0.4687 | |
| | | 0.3795 | 100.0 | 6200 | 0.4684 | |
| | |
| | |
| | ### Framework versions |
| | |
| | - Transformers 4.21.1 |
| | - Pytorch 1.12.0+cu113 |
| | - Datasets 2.4.0 |
| | - Tokenizers 0.12.1 |
| | |
