rel-cl-transformer-4
This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:
- Loss: 3.8664
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 3052726
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 4.2388 | 0.03 | 76319 | 4.2044 |
| 4.0326 | 0.03 | 152638 | 4.0318 |
| 3.9236 | 0.03 | 228957 | 3.9580 |
| 3.8544 | 0.03 | 305276 | 3.9166 |
| 3.8024 | 0.03 | 381595 | 3.8908 |
| 3.7553 | 0.03 | 457914 | 3.8751 |
| 3.7234 | 0.03 | 534233 | 3.8648 |
| 3.6968 | 1.03 | 610552 | 3.8583 |
| 3.6689 | 0.03 | 686872 | 3.8499 |
| 3.6414 | 1.03 | 763192 | 3.8469 |
| 3.6187 | 0.03 | 839512 | 3.8467 |
| 3.5985 | 1.03 | 915832 | 3.8456 |
| 3.5764 | 0.03 | 992152 | 3.8459 |
| 3.5579 | 1.03 | 1068472 | 3.8476 |
| 3.5454 | 0.03 | 1144792 | 3.8477 |
| 3.5313 | 0.03 | 1221112 | 3.8497 |
| 3.521 | 1.03 | 1297432 | 3.8510 |
| 3.5044 | 0.03 | 1373752 | 3.8522 |
| 3.4925 | 1.03 | 1450072 | 3.8531 |
| 3.4831 | 0.03 | 1526392 | 3.8551 |
| 3.4716 | 1.03 | 1602712 | 3.8569 |
| 3.4625 | 0.03 | 1679032 | 3.8572 |
| 3.456 | 1.03 | 1755352 | 3.8592 |
| 3.4435 | 0.03 | 1831672 | 3.8614 |
| 3.43 | 1.03 | 1907992 | 3.8618 |
| 3.4152 | 0.03 | 1984312 | 3.8625 |
| 3.4052 | 1.03 | 2060632 | 3.8641 |
| 3.3947 | 0.03 | 2136952 | 3.8649 |
| 3.3836 | 1.03 | 2213272 | 3.8653 |
| 3.3684 | 0.03 | 2289592 | 3.8668 |
| 3.3616 | 0.03 | 2365912 | 3.8675 |
| 3.3559 | 1.03 | 2442232 | 3.8683 |
| 3.3476 | 0.03 | 2518552 | 3.8687 |
| 3.3375 | 1.03 | 2594872 | 3.8693 |
| 3.3274 | 0.03 | 2671192 | 3.8691 |
| 3.321 | 1.03 | 2747512 | 3.8689 |
| 3.3125 | 0.03 | 2823832 | 3.8684 |
| 3.3092 | 1.03 | 2900152 | 3.8682 |
| 3.3039 | 0.03 | 2976472 | 3.8670 |
| 3.2951 | 1.02 | 3052726 | 3.8664 |
Framework versions
- Transformers 4.33.3
- Pytorch 2.0.1
- Datasets 2.12.0
- Tokenizers 0.13.3
- Downloads last month
- 6