README.md

metadata

library_name: peft
license: llama3
base_model: meta-llama/Meta-Llama-3-8B-Instruct
tags:
  - llama-factory
  - prefix-tuning
  - generated_from_trainer
model-index:
  - name: train_boolq_1755694496
    results: []

train_boolq_1755694496

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.6806
• Num Input Tokens Seen: 18160480

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4575	0.5	2121	0.3290	910896
0.2746	1.0	4242	0.3331	1820384
0.345	1.5	6363	0.3276	2725088
0.2623	2.0	8484	0.3277	3635024
0.2687	2.5	10605	0.3219	4551376
0.348	3.0	12726	0.3203	5454288
0.3231	3.5	14847	0.3312	6365376
0.3599	4.0	16968	0.3212	7267408
0.6939	4.5	19089	0.3523	8180384
0.266	5.0	21210	0.3135	9086272
0.7278	5.5	23331	0.3380	9998416
0.2001	6.0	25452	0.3579	10904352
0.2883	6.5	27573	0.3674	11814320
0.0321	7.0	29694	0.4707	12718592
0.0739	7.5	31815	0.4771	13626928
0.0772	8.0	33936	0.4935	14536416
0.0703	8.5	36057	0.6079	15443568
0.1229	9.0	38178	0.6257	16348016
0.2984	9.5	40299	0.6781	17255920
0.009	10.0	42420	0.6806	18160480

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1