README.md

metadata

library_name: peft
license: llama3
base_model: meta-llama/Meta-Llama-3-8B-Instruct
tags:
  - llama-factory
  - lora
  - generated_from_trainer
model-index:
  - name: train_multirc_1753094163
    results: []

train_multirc_1753094163

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 0.1342
• Num Input Tokens Seen: 132272272

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3561	0.5	3065	0.1638	6639424
0.1555	1.0	6130	0.1342	13255424
0.0939	1.5	9195	0.1415	19871232
0.1649	2.0	12260	0.1370	26471216
0.0113	2.5	15325	0.1524	33075856
0.1045	3.0	18390	0.1735	39694112
0.0014	3.5	21455	0.1890	46313216
0.0007	4.0	24520	0.2072	52929744
0.1137	4.5	27585	0.2542	59549072
0.053	5.0	30650	0.2518	66152480
0.0003	5.5	33715	0.2611	72765696
0.0002	6.0	36780	0.3484	79389648
0.0	6.5	39845	0.3470	86008784
0.0001	7.0	42910	0.3074	92621824
0.0	7.5	45975	0.3919	99237152
0.0	8.0	49040	0.3871	105830544
0.0001	8.5	52105	0.4637	112458064
0.0	9.0	55170	0.4719	119047920
0.0	9.5	58235	0.4906	125686064
0.0	10.0	61300	0.4911	132272272

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1