| --- |
| tags: |
| - generated_from_trainer |
| metrics: |
| - accuracy |
| model-index: |
| - name: model_v1_complete_training_wt_init_48_tiny |
| results: [] |
| --- |
| |
| <!-- This model card has been generated automatically according to the information the Trainer had access to. You |
| should probably proofread and complete it, then remove this comment. --> |
|
|
| # model_v1_complete_training_wt_init_48_tiny |
| |
| This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset. |
| It achieves the following results on the evaluation set: |
| - Loss: 3.6497 |
| - Accuracy: 0.3896 |
| |
| ## Model description |
| |
| More information needed |
| |
| ## Intended uses & limitations |
| |
| More information needed |
| |
| ## Training and evaluation data |
| |
| More information needed |
| |
| ## Training procedure |
| |
| ### Training hyperparameters |
| |
| The following hyperparameters were used during training: |
| - learning_rate: 1e-05 |
| - train_batch_size: 64 |
| - eval_batch_size: 64 |
| - seed: 10 |
| - distributed_type: multi-GPU |
| - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
| - lr_scheduler_type: linear |
| - lr_scheduler_warmup_steps: 10000 |
| - num_epochs: 50 |
| |
| ### Training results |
| |
| | Training Loss | Epoch | Step | Validation Loss | Accuracy | |
| |:-------------:|:-----:|:-------:|:---------------:|:--------:| |
| | 6.0224 | 0.33 | 30000 | 5.9447 | 0.1517 | |
| | 5.1853 | 0.66 | 60000 | 4.9635 | 0.2615 | |
| | 4.9483 | 0.98 | 90000 | 4.7016 | 0.2830 | |
| | 4.7679 | 1.31 | 120000 | 4.5154 | 0.2992 | |
| | 4.6448 | 1.64 | 150000 | 4.3884 | 0.3100 | |
| | 4.5688 | 1.97 | 180000 | 4.3095 | 0.3175 | |
| | 4.5102 | 2.29 | 210000 | 4.2511 | 0.3236 | |
| | 4.4662 | 2.62 | 240000 | 4.2038 | 0.3294 | |
| | 4.4269 | 2.95 | 270000 | 4.1677 | 0.3336 | |
| | 4.3982 | 3.28 | 300000 | 4.1367 | 0.3370 | |
| | 4.3714 | 3.6 | 330000 | 4.1103 | 0.3399 | |
| | 4.3493 | 3.93 | 360000 | 4.0869 | 0.3423 | |
| | 4.3303 | 4.26 | 390000 | 4.0680 | 0.3439 | |
| | 4.3131 | 4.59 | 420000 | 4.0467 | 0.3461 | |
| | 4.2875 | 4.92 | 450000 | 4.0292 | 0.3477 | |
| | 4.2629 | 5.24 | 480000 | 4.0109 | 0.3497 | |
| | 4.2413 | 5.57 | 510000 | 3.9931 | 0.3515 | |
| | 4.2282 | 5.9 | 540000 | 3.9759 | 0.3536 | |
| | 4.2003 | 6.23 | 570000 | 3.9608 | 0.3551 | |
| | 4.1867 | 6.55 | 600000 | 3.9445 | 0.3571 | |
| | 4.1607 | 6.88 | 630000 | 3.9273 | 0.3590 | |
| | 4.1511 | 7.21 | 660000 | 3.9130 | 0.3606 | |
| | 4.1335 | 7.54 | 690000 | 3.8971 | 0.3622 | |
| | 4.1158 | 7.87 | 720000 | 3.8798 | 0.3642 | |
| | 4.097 | 8.19 | 750000 | 3.8635 | 0.3663 | |
| | 4.0831 | 8.52 | 780000 | 3.8494 | 0.3679 | |
| | 4.0756 | 8.85 | 810000 | 3.8334 | 0.3696 | |
| | 4.0533 | 9.18 | 840000 | 3.8201 | 0.3712 | |
| | 4.0517 | 9.5 | 870000 | 3.8080 | 0.3724 | |
| | 4.0325 | 9.83 | 900000 | 3.7975 | 0.3734 | |
| | 4.0142 | 10.16 | 930000 | 3.7872 | 0.3748 | |
| | 4.0124 | 10.49 | 960000 | 3.7788 | 0.3759 | |
| | 4.0076 | 10.81 | 990000 | 3.7679 | 0.3767 | |
| | 3.9919 | 11.14 | 1020000 | 3.7609 | 0.3775 | |
| | 3.9888 | 11.47 | 1050000 | 3.7550 | 0.3783 | |
| | 3.9796 | 11.8 | 1080000 | 3.7481 | 0.3789 | |
| | 3.9742 | 12.13 | 1110000 | 3.7414 | 0.3796 | |
| | 3.9667 | 12.45 | 1140000 | 3.7370 | 0.3802 | |
| | 3.9652 | 12.78 | 1170000 | 3.7289 | 0.3810 | |
| | 3.9548 | 13.11 | 1200000 | 3.7278 | 0.3812 | |
| | 3.9556 | 13.44 | 1230000 | 3.7213 | 0.3817 | |
| | 3.9444 | 13.76 | 1260000 | 3.7152 | 0.3825 | |
| | 3.9428 | 14.09 | 1290000 | 3.7120 | 0.3827 | |
| | 3.9424 | 14.42 | 1320000 | 3.7072 | 0.3834 | |
| | 3.9389 | 14.75 | 1350000 | 3.7047 | 0.3836 | |
| | 3.936 | 15.07 | 1380000 | 3.6998 | 0.3844 | |
| | 3.9246 | 15.4 | 1410000 | 3.6968 | 0.3847 | |
| | 3.9281 | 15.73 | 1440000 | 3.6925 | 0.3851 | |
| | 3.9177 | 16.06 | 1470000 | 3.6916 | 0.3849 | |
| | 3.9216 | 16.39 | 1500000 | 3.6870 | 0.3855 | |
| | 3.9141 | 16.71 | 1530000 | 3.6822 | 0.3863 | |
| | 3.9154 | 17.04 | 1560000 | 3.6804 | 0.3864 | |
| | 3.9145 | 17.37 | 1590000 | 3.6795 | 0.3863 | |
| | 3.9103 | 17.7 | 1620000 | 3.6734 | 0.3869 | |
| | 3.9079 | 18.02 | 1650000 | 3.6724 | 0.3873 | |
| | 3.901 | 18.35 | 1680000 | 3.6707 | 0.3872 | |
| | 3.9015 | 18.68 | 1710000 | 3.6695 | 0.3873 | |
| | 3.8987 | 19.01 | 1740000 | 3.6672 | 0.3877 | |
| | 3.8929 | 19.33 | 1770000 | 3.6647 | 0.3878 | |
| | 3.892 | 19.66 | 1800000 | 3.6609 | 0.3884 | |
| | 3.8906 | 19.99 | 1830000 | 3.6595 | 0.3886 | |
| | 3.8923 | 20.32 | 1860000 | 3.6594 | 0.3885 | |
| | 3.8901 | 20.65 | 1890000 | 3.6541 | 0.3893 | |
| | 3.8853 | 20.97 | 1920000 | 3.6539 | 0.3891 | |
| | 3.8808 | 21.3 | 1950000 | 3.6527 | 0.3894 | |
| | 3.8835 | 21.63 | 1980000 | 3.6497 | 0.3896 | |
| |
| |
| ### Framework versions |
| |
| - Transformers 4.30.2 |
| - Pytorch 1.14.0a0+410ce96 |
| - Datasets 2.13.0 |
| - Tokenizers 0.13.3 |
| |
