ice_llama2

This model is a fine-tuned version of meta-llama/Llama-2-7b-hf on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.3080

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
2.6464	0.02	10000	2.5455
2.4927	0.03	20000	2.4736
2.5238	0.05	30000	2.4381
2.4711	0.07	40000	2.4124
2.4345	0.08	50000	2.3989
2.3715	0.1	60000	2.3878
2.4212	0.12	70000	2.3786
2.434	0.14	80000	2.3724
2.4798	0.15	90000	2.3670
2.4343	0.17	100000	2.3615
2.4365	0.19	110000	2.3571
2.398	0.2	120000	2.3538
2.4734	0.22	130000	2.3510
2.4356	0.24	140000	2.3499
2.4512	0.25	150000	2.3458
2.4331	0.27	160000	2.3438
2.3749	0.29	170000	2.3409
2.4375	0.3	180000	2.3397
2.3301	0.32	190000	2.3378
2.4341	0.34	200000	2.3364
2.3286	0.36	210000	2.3356
2.3173	0.37	220000	2.3332
2.4105	0.39	230000	2.3333
2.3553	0.41	240000	2.3311
2.386	0.42	250000	2.3307
2.3761	0.44	260000	2.3294
2.42	0.46	270000	2.3277
2.44	0.47	280000	2.3267
2.4169	0.49	290000	2.3276
2.3254	0.51	300000	2.3265
2.4326	0.52	310000	2.3252
2.4203	0.54	320000	2.3244
2.4401	0.56	330000	2.3242
2.4005	0.57	340000	2.3221
2.3734	0.59	350000	2.3223
2.331	0.61	360000	2.3223
2.4652	0.63	370000	2.3220
2.3741	0.64	380000	2.3211
2.3634	0.66	390000	2.3193
2.3688	0.68	400000	2.3201
2.3284	0.69	410000	2.3195
2.4486	0.71	420000	2.3184
2.4113	0.73	430000	2.3183
2.3442	0.74	440000	2.3175
2.3414	0.76	450000	2.3176
2.3534	0.78	460000	2.3166
2.3384	0.79	470000	2.3170
2.4021	0.81	480000	2.3163
2.2645	0.83	490000	2.3161
2.3392	0.85	500000	2.3158
2.3499	0.86	510000	2.3155
2.3111	0.88	520000	2.3149
2.3604	0.9	530000	2.3139
2.4264	0.91	540000	2.3144
2.4964	0.93	550000	2.3138
2.5121	0.95	560000	2.3129
2.3389	0.96	570000	2.3129
2.3755	0.98	580000	2.3131
2.414	1.0	590000	2.3129
2.2656	1.01	600000	2.3127
2.3372	1.03	610000	2.3120
2.3479	1.05	620000	2.3126
2.3752	1.07	630000	2.3121
2.3685	1.08	640000	2.3118
2.4521	1.1	650000	2.3117
2.426	1.12	660000	2.3113
2.4583	1.13	670000	2.3114
2.414	1.15	680000	2.3107
2.3833	1.17	690000	2.3107
2.3053	1.18	700000	2.3107
2.2819	1.2	710000	2.3108
2.4079	1.22	720000	2.3106
2.4529	1.23	730000	2.3101
2.3139	1.25	740000	2.3102
2.3249	1.27	750000	2.3104
2.3575	1.28	760000	2.3102
2.3568	1.3	770000	2.3102
2.3686	1.32	780000	2.3096
2.34	1.34	790000	2.3098
2.4017	1.35	800000	2.3096
2.4174	1.37	810000	2.3096
2.3793	1.39	820000	2.3092
2.4362	1.4	830000	2.3094
2.4015	1.42	840000	2.3091
2.4451	1.44	850000	2.3089
2.369	1.45	860000	2.3091
2.408	1.47	870000	2.3091
2.4213	1.49	880000	2.3088
2.4088	1.5	890000	2.3090
2.3602	1.52	900000	2.3090
2.3525	1.54	910000	2.3087
2.421	1.56	920000	2.3086
2.4586	1.57	930000	2.3083
2.4173	1.59	940000	2.3084
2.3366	1.61	950000	2.3084
2.3604	1.62	960000	2.3084
2.3662	1.64	970000	2.3082
2.3754	1.66	980000	2.3085
2.3966	1.67	990000	2.3083
2.3537	1.69	1000000	2.3083
2.4711	1.71	1010000	2.3084
2.335	1.72	1020000	2.3082
2.3608	1.74	1030000	2.3082
2.3669	1.76	1040000	2.3080
2.463	1.78	1050000	2.3081
2.3369	1.79	1060000	2.3082
2.3314	1.81	1070000	2.3082
2.3681	1.83	1080000	2.3081
2.3867	1.84	1090000	2.3081
2.4012	1.86	1100000	2.3081
2.367	1.88	1110000	2.3080
2.415	1.89	1120000	2.3080
2.3738	1.91	1130000	2.3080
2.4199	1.93	1140000	2.3080
2.3884	1.94	1150000	2.3080
2.3609	1.96	1160000	2.3080
2.3343	1.98	1170000	2.3080
2.3575	1.99	1180000	2.3079
2.4116	2.01	1190000	2.3079
2.3074	2.03	1200000	2.3080
2.4261	2.05	1210000	2.3080
2.3793	2.06	1220000	2.3079
2.3543	2.08	1230000	2.3080
2.404	2.1	1240000	2.3081
2.4234	2.11	1250000	2.3079
2.3582	2.13	1260000	2.3080
2.3922	2.15	1270000	2.3079
2.3461	2.16	1280000	2.3080
2.3395	2.18	1290000	2.3080
2.3755	2.2	1300000	2.3080
2.3718	2.21	1310000	2.3079
2.3894	2.23	1320000	2.3079
2.3955	2.25	1330000	2.3079
2.3445	2.27	1340000	2.3079
2.3894	2.28	1350000	2.3080
2.3648	2.3	1360000	2.3080
2.3999	2.32	1370000	2.3080
2.3102	2.33	1380000	2.3079
2.3311	2.35	1390000	2.3078
2.3408	2.37	1400000	2.3079
2.3844	2.38	1410000	2.3080
2.3825	2.4	1420000	2.3080
2.398	2.42	1430000	2.3080
2.3305	2.43	1440000	2.3080
2.4646	2.45	1450000	2.3078
2.4326	2.47	1460000	2.3080
2.351	2.49	1470000	2.3080
2.4074	2.5	1480000	2.3080
2.3806	2.52	1490000	2.3080
2.4329	2.54	1500000	2.3080
2.421	2.55	1510000	2.3079
2.3322	2.57	1520000	2.3079
2.393	2.59	1530000	2.3079
2.3183	2.6	1540000	2.3080
2.3257	2.62	1550000	2.3080
2.3745	2.64	1560000	2.3080
2.409	2.65	1570000	2.3080
2.4447	2.67	1580000	2.3078
2.3805	2.69	1590000	2.3079
2.4323	2.7	1600000	2.3079
2.3654	2.72	1610000	2.3079
2.4017	2.74	1620000	2.3079
2.3168	2.76	1630000	2.3079
2.3304	2.77	1640000	2.3079
2.4072	2.79	1650000	2.3080
2.3232	2.81	1660000	2.3080
2.3563	2.82	1670000	2.3079
2.4327	2.84	1680000	2.3080
2.3932	2.86	1690000	2.3080
2.3134	2.87	1700000	2.3079
2.3421	2.89	1710000	2.3080
2.4805	2.91	1720000	2.3080
2.4306	2.92	1730000	2.3079
2.3511	2.94	1740000	2.3081
2.3509	2.96	1750000	2.3080
2.3598	2.98	1760000	2.3080
2.4583	2.99	1770000	2.3080

Framework versions

• PEFT 0.8.2
• Transformers 4.38.1
• Pytorch 2.2.0+cu118
• Datasets 2.17.1
• Tokenizers 0.15.2