blip2-lora-finetune-2

This model is a fine-tuned version of Salesforce/blip2-opt-2.7b on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7935

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.15
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
102.7678	0.5979	50	13.0764
86.8701	1.1913	100	10.7530
54.3081	1.7892	150	6.4039
18.5161	2.3827	200	2.0638
9.7514	2.9806	250	1.1446
9.7667	3.5740	300	1.0239
7.3572	4.1674	350	0.9157
7.6331	4.7653	400	0.8900
7.8984	5.3587	450	0.8654
7.5447	5.9567	500	0.8642
7.2598	6.5501	550	0.8483
8.3937	7.1435	600	0.8402
6.9849	7.7414	650	0.8205
7.0229	8.3348	700	0.8303
7.4894	8.9327	750	0.8250
8.0766	9.5262	800	0.8261
6.8229	10.1196	850	0.8080
6.3603	10.7175	900	0.8184
6.8538	11.3109	950	0.8257
6.5484	11.9088	1000	0.8114
7.1385	12.5022	1050	0.8148
7.1053	13.0957	1100	0.8309
6.8619	13.6936	1150	0.8110
6.9226	14.2870	1200	0.8279
6.5827	14.8849	1250	0.8123
7.4853	15.4783	1300	0.8072
7.3401	16.0717	1350	0.8103
6.5739	16.6697	1400	0.8020
7.0551	17.2631	1450	0.7967
7.7921	17.8610	1500	0.8073
6.6075	18.4544	1550	0.8066
6.3661	19.0478	1600	0.7947
6.5013	19.6457	1650	0.8012
7.7917	20.2392	1700	0.7953
7.4042	20.8371	1750	0.7989
7.2549	21.4305	1800	0.8095
7.2292	22.0239	1850	0.7950
6.3362	22.6218	1900	0.7930
6.2422	23.2152	1950	0.8145
7.6344	23.8132	2000	0.7909
6.8542	24.4066	2050	0.7892
6.9861	25.0	2100	0.7946
6.5772	25.5979	2150	0.7902
6.5195	26.1913	2200	0.7990
6.7995	26.7892	2250	0.7958
6.1444	27.3827	2300	0.7956
6.8757	27.9806	2350	0.7945
6.9428	28.5740	2400	0.7901
7.4196	29.1674	2450	0.7935
6.5429	29.7653	2500	0.7935

Framework versions

• PEFT 0.18.1
• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.5.0
• Tokenizers 0.22.2