README.md

metadata

library_name: peft
license: llama3
base_model: meta-llama/Meta-Llama-3-8B-Instruct
tags:
  - llama-factory
  - prefix-tuning
  - generated_from_trainer
model-index:
  - name: train_boolq_1756729605
    results: []

train_boolq_1756729605

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.3326
• Num Input Tokens Seen: 18160480

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4614	0.5	2121	0.3305	910896
0.2867	1.0	4242	0.3275	1820384
0.3369	1.5	6363	0.3241	2725088
0.2545	2.0	8484	0.3280	3635024
0.2875	2.5	10605	0.3283	4551376
0.3385	3.0	12726	0.3347	5454288
0.3121	3.5	14847	0.3290	6365376
0.3326	4.0	16968	0.3312	7267408
0.3828	4.5	19089	0.3279	8180384
0.3116	5.0	21210	0.3268	9086272
0.4007	5.5	23331	0.3321	9998416
0.3262	6.0	25452	0.3285	10904352
0.2873	6.5	27573	0.3270	11814320
0.2122	7.0	29694	0.3293	12718592
0.3031	7.5	31815	0.3272	13626928
0.3253	8.0	33936	0.3276	14536416
0.2943	8.5	36057	0.3289	15443568
0.3488	9.0	38178	0.3365	16348016
0.3448	9.5	40299	0.3319	17255920
0.3665	10.0	42420	0.3326	18160480

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1