metadata
library_name: peft
license: llama3
base_model: meta-llama/Meta-Llama-3-8B-Instruct
tags:
- llama-factory
- p-tuning
- generated_from_trainer
model-index:
- name: train_stsb_1754652140
results: []
train_stsb_1754652140
This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the stsb dataset. It achieves the following results on the evaluation set:
- Loss: 2.6397
- Num Input Tokens Seen: 4364240
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 123
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 10.0
Training results
| Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
|---|---|---|---|---|
| 0.7201 | 0.5 | 647 | 0.9210 | 217472 |
| 0.6524 | 1.0 | 1294 | 0.8728 | 435488 |
| 0.5352 | 1.5 | 1941 | 0.7179 | 652480 |
| 0.5722 | 2.0 | 2588 | 0.6825 | 871200 |
| 0.486 | 2.5 | 3235 | 0.6683 | 1089120 |
| 0.6085 | 3.0 | 3882 | 0.5942 | 1307968 |
| 0.5084 | 3.5 | 4529 | 0.5508 | 1529024 |
| 0.3727 | 4.0 | 5176 | 0.5626 | 1745568 |
| 0.4597 | 4.5 | 5823 | 0.5266 | 1965984 |
| 0.7069 | 5.0 | 6470 | 0.5204 | 2182352 |
| 0.4146 | 5.5 | 7117 | 0.5107 | 2399760 |
| 0.5929 | 6.0 | 7764 | 0.5058 | 2619888 |
| 0.5626 | 6.5 | 8411 | 0.5047 | 2837808 |
| 0.3735 | 7.0 | 9058 | 0.5093 | 3057216 |
| 0.3429 | 7.5 | 9705 | 0.5020 | 3275904 |
| 0.4286 | 8.0 | 10352 | 0.5120 | 3493600 |
| 0.4575 | 8.5 | 10999 | 0.5040 | 3712320 |
| 0.3782 | 9.0 | 11646 | 0.5048 | 3928704 |
| 0.3722 | 9.5 | 12293 | 0.5039 | 4147200 |
| 0.4486 | 10.0 | 12940 | 0.5031 | 4364240 |
Framework versions
- PEFT 0.15.2
- Transformers 4.51.3
- Pytorch 2.8.0+cu128
- Datasets 3.6.0
- Tokenizers 0.21.1