metadata
library_name: peft
license: llama3
base_model: meta-llama/Meta-Llama-3-8B-Instruct
tags:
- llama-factory
- p-tuning
- generated_from_trainer
model-index:
- name: train_wsc_1754652156
results: []
train_wsc_1754652156
This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:
- Loss: 2.7778
- Num Input Tokens Seen: 490000
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 123
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 10.0
Training results
| Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
|---|---|---|---|---|
| 2.4086 | 0.504 | 63 | 0.3988 | 25504 |
| 0.5346 | 1.008 | 126 | 0.5207 | 49696 |
| 0.3725 | 1.512 | 189 | 0.4400 | 74112 |
| 0.343 | 2.016 | 252 | 0.3816 | 99136 |
| 0.3615 | 2.52 | 315 | 0.3501 | 123904 |
| 0.5066 | 3.024 | 378 | 0.3483 | 148736 |
| 0.3459 | 3.528 | 441 | 0.3604 | 174432 |
| 0.3261 | 4.032 | 504 | 0.3559 | 198656 |
| 0.3529 | 4.536 | 567 | 0.3538 | 224032 |
| 0.3441 | 5.04 | 630 | 0.3520 | 247424 |
| 0.3572 | 5.5440 | 693 | 0.3479 | 271232 |
| 0.3479 | 6.048 | 756 | 0.3478 | 295728 |
| 0.3533 | 6.552 | 819 | 0.3526 | 320464 |
| 0.3186 | 7.056 | 882 | 0.3628 | 345856 |
| 0.3306 | 7.5600 | 945 | 0.3622 | 371040 |
| 0.3524 | 8.064 | 1008 | 0.3506 | 395216 |
| 0.3447 | 8.568 | 1071 | 0.3489 | 419184 |
| 0.3639 | 9.072 | 1134 | 0.3548 | 444560 |
| 0.3419 | 9.576 | 1197 | 0.3464 | 469104 |
Framework versions
- PEFT 0.15.2
- Transformers 4.51.3
- Pytorch 2.8.0+cu128
- Datasets 3.6.0
- Tokenizers 0.21.1