---
tags:
- generated_from_trainer
model-index:
- name: 1L-BERT-finetuned-newcode
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 1L-BERT-finetuned-newcode

This model is a fine-tuned version of [Youssef320/LSTM-finetuned-50label-15epoch](https://huggingface.co/Youssef320/LSTM-finetuned-50label-15epoch) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 3.0668
- Top 1 Macro F1 Score: 0.1385
- Top 1 Weighted F1score: 0.1902
- Top 3 Macro F1 Score: 0.2868
- Top3 3 Weighted F1 Score : 0.3758

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- gradient_accumulation_steps: 32
- total_train_batch_size: 2048
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Top 1 Macro F1 Score | Top 1 Weighted F1score | Top 3 Macro F1 Score | Top3 3 Weighted F1 Score  |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|:----------------------:|:--------------------:|:-------------------------:|
| 2.8006        | 0.13  | 32   | 3.1069          | 0.1217               | 0.1745                 | 0.2614               | 0.3568                    |
| 2.8197        | 0.26  | 64   | 3.0678          | 0.1271               | 0.1797                 | 0.2715               | 0.3654                    |
| 2.7933        | 0.4   | 96   | 3.0518          | 0.1279               | 0.1819                 | 0.2714               | 0.3661                    |
| 2.7852        | 0.53  | 128  | 3.0463          | 0.1322               | 0.1853                 | 0.2787               | 0.3699                    |
| 2.7847        | 0.66  | 160  | 3.0273          | 0.1294               | 0.1817                 | 0.2773               | 0.3725                    |
| 2.7899        | 0.79  | 192  | 3.0185          | 0.1316               | 0.1877                 | 0.2769               | 0.3717                    |
| 2.793         | 0.93  | 224  | 3.0140          | 0.1332               | 0.1864                 | 0.2794               | 0.3731                    |
| 2.6818        | 1.06  | 256  | 3.0629          | 0.1345               | 0.1879                 | 0.2829               | 0.3739                    |
| 2.6676        | 1.19  | 288  | 3.0798          | 0.1335               | 0.1867                 | 0.2806               | 0.3724                    |
| 2.6859        | 1.32  | 320  | 3.0595          | 0.1320               | 0.1845                 | 0.2787               | 0.3704                    |
| 2.6939        | 1.45  | 352  | 3.0650          | 0.1321               | 0.1841                 | 0.2801               | 0.3710                    |
| 2.7114        | 1.59  | 384  | 3.0594          | 0.1319               | 0.1841                 | 0.2823               | 0.3735                    |
| 2.7414        | 1.72  | 416  | 3.0475          | 0.1340               | 0.1864                 | 0.2788               | 0.3710                    |
| 2.7102        | 1.85  | 448  | 3.0464          | 0.1347               | 0.1883                 | 0.2817               | 0.3741                    |
| 2.7537        | 1.98  | 480  | 3.0270          | 0.1344               | 0.1879                 | 0.2794               | 0.3736                    |
| 2.616         | 2.12  | 512  | 3.0929          | 0.1361               | 0.1883                 | 0.2798               | 0.3706                    |
| 2.621         | 2.25  | 544  | 3.0821          | 0.1347               | 0.1867                 | 0.2785               | 0.3709                    |
| 2.6409        | 2.38  | 576  | 3.0870          | 0.1352               | 0.1873                 | 0.2806               | 0.3705                    |
| 2.6904        | 2.51  | 608  | 3.0735          | 0.1349               | 0.1867                 | 0.2854               | 0.3748                    |
| 2.6531        | 2.64  | 640  | 3.0732          | 0.1357               | 0.1895                 | 0.2820               | 0.3731                    |
| 2.6643        | 2.78  | 672  | 3.0677          | 0.1374               | 0.1896                 | 0.2814               | 0.3729                    |
| 2.6948        | 2.91  | 704  | 3.0668          | 0.1385               | 0.1902                 | 0.2868               | 0.3758                    |


### Framework versions

- Transformers 4.20.1
- Pytorch 1.12.1+cu102
- Datasets 2.0.0
- Tokenizers 0.11.0