--- tags: - generated_from_trainer model-index: - name: finetuned_1L_BERT_newcode_5epoch results: [] --- # finetuned_1L_BERT_newcode_5epoch This model is a fine-tuned version of [Youssef320/LSTM-finetuned-50label-15epoch](https://huggingface.co/Youssef320/LSTM-finetuned-50label-15epoch) on the None dataset. It achieves the following results on the evaluation set: - Loss: 3.0134 - Top 1 Macro F1 Score: 0.1420 - Top 1 Weighted F1score: 0.1923 - Top 3 Macro F1 Score: 0.2960 - Top3 3 Weighted F1 Score : 0.3853 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0002 - train_batch_size: 64 - eval_batch_size: 64 - seed: 42 - gradient_accumulation_steps: 32 - total_train_batch_size: 2048 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: constant - num_epochs: 5.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | Top 1 Macro F1 Score | Top 1 Weighted F1score | Top 3 Macro F1 Score | Top3 3 Weighted F1 Score | |:-------------:|:-----:|:----:|:---------------:|:--------------------:|:----------------------:|:--------------------:|:-------------------------:| | 3.2498 | 0.14 | 64 | 3.0549 | 0.1275 | 0.1797 | 0.2717 | 0.3636 | | 3.1662 | 0.27 | 128 | 3.0345 | 0.1311 | 0.1856 | 0.2767 | 0.3674 | | 3.16 | 0.41 | 192 | 3.0158 | 0.1342 | 0.1886 | 0.2792 | 0.3724 | | 3.1154 | 0.54 | 256 | 3.0034 | 0.1343 | 0.1880 | 0.2844 | 0.3770 | | 3.1257 | 0.68 | 320 | 2.9848 | 0.1332 | 0.1876 | 0.2820 | 0.3783 | | 3.1227 | 0.81 | 384 | 2.9721 | 0.1339 | 0.1887 | 0.2844 | 0.3806 | | 3.1058 | 0.95 | 448 | 2.9737 | 0.1369 | 0.1916 | 0.2871 | 0.3815 | | 3.0486 | 1.09 | 512 | 3.0066 | 0.1353 | 0.1873 | 0.2888 | 0.3794 | | 3.0286 | 1.22 | 576 | 3.0036 | 0.1359 | 0.1884 | 0.2874 | 0.3808 | | 3.0181 | 1.36 | 640 | 2.9924 | 0.1369 | 0.1894 | 0.2888 | 0.3823 | | 3.0251 | 1.49 | 704 | 2.9899 | 0.1387 | 0.1916 | 0.2878 | 0.3814 | | 3.0482 | 1.63 | 768 | 2.9787 | 0.1377 | 0.1900 | 0.2903 | 0.3828 | | 3.0565 | 1.77 | 832 | 2.9746 | 0.1397 | 0.1928 | 0.2912 | 0.3847 | | 3.0563 | 1.9 | 896 | 2.9710 | 0.1390 | 0.1936 | 0.2887 | 0.3839 | | 2.9363 | 2.04 | 960 | 3.0022 | 0.1413 | 0.1952 | 0.2902 | 0.3822 | | 2.951 | 2.17 | 1024 | 3.0072 | 0.1381 | 0.1908 | 0.2902 | 0.3822 | | 2.9839 | 2.31 | 1088 | 3.0027 | 0.1413 | 0.1947 | 0.2890 | 0.3811 | | 2.9714 | 2.45 | 1152 | 2.9972 | 0.1411 | 0.1945 | 0.2905 | 0.3829 | | 2.9959 | 2.58 | 1216 | 2.9788 | 0.1387 | 0.1905 | 0.2906 | 0.3845 | | 2.9932 | 2.72 | 1280 | 2.9875 | 0.1412 | 0.1954 | 0.2905 | 0.3840 | | 2.9952 | 2.85 | 1344 | 2.9728 | 0.1403 | 0.1933 | 0.2935 | 0.3862 | | 3.0112 | 2.99 | 1408 | 2.9707 | 0.1399 | 0.1921 | 0.2958 | 0.3879 | | 2.9005 | 3.13 | 1472 | 3.0190 | 0.1398 | 0.1923 | 0.2912 | 0.3825 | | 2.9153 | 3.26 | 1536 | 3.0224 | 0.1400 | 0.1935 | 0.2923 | 0.3814 | | 2.9284 | 3.4 | 1600 | 3.0087 | 0.1387 | 0.1917 | 0.2895 | 0.3819 | | 2.922 | 3.53 | 1664 | 3.0130 | 0.1397 | 0.1918 | 0.2927 | 0.3838 | | 2.947 | 3.67 | 1728 | 3.0023 | 0.1402 | 0.1923 | 0.2916 | 0.3834 | | 2.9538 | 3.8 | 1792 | 2.9948 | 0.1414 | 0.1937 | 0.2944 | 0.3845 | | 2.9478 | 3.94 | 1856 | 2.9876 | 0.1420 | 0.1952 | 0.2926 | 0.3852 | | 2.8445 | 4.08 | 1920 | 3.0409 | 0.1403 | 0.1912 | 0.2961 | 0.3843 | | 2.8498 | 4.21 | 1984 | 3.0378 | 0.1412 | 0.1933 | 0.2942 | 0.3813 | | 2.8519 | 4.35 | 2048 | 3.0405 | 0.1422 | 0.1942 | 0.2952 | 0.3825 | | 2.8715 | 4.48 | 2112 | 3.0341 | 0.1412 | 0.1941 | 0.2922 | 0.3815 | | 2.892 | 4.62 | 2176 | 3.0262 | 0.1405 | 0.1936 | 0.2912 | 0.3813 | | 2.8952 | 4.75 | 2240 | 3.0241 | 0.1428 | 0.1953 | 0.2944 | 0.3840 | | 2.9135 | 4.89 | 2304 | 3.0134 | 0.1420 | 0.1923 | 0.2960 | 0.3853 | ### Framework versions - Transformers 4.20.1 - Pytorch 1.12.1+cu102 - Datasets 2.0.0 - Tokenizers 0.11.0