CeLLaTe-AL-Test_base_adapted_tok

This model is a fine-tuned version of Mardiyyah/cellate-tapt_freeze_llrd_ww_mask-LR_2e-05 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3200
• Precision: 0.7684
• Recall: 0.7479
• F1: 0.7580
• Accuracy: 0.9331

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 3407
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
1.3009	1.0	115	0.6194	0.3310	0.1135	0.1690	0.8242
0.3335	2.0	230	0.2986	0.6149	0.6326	0.6236	0.9083
0.1312	3.0	345	0.2622	0.6834	0.6928	0.6881	0.9200
0.0797	4.0	460	0.2733	0.7287	0.7326	0.7306	0.9265
0.0566	5.0	575	0.2805	0.7384	0.7543	0.7463	0.9306
0.0404	6.0	690	0.3373	0.7501	0.7216	0.7355	0.9278
0.0299	7.0	805	0.3340	0.7543	0.7085	0.7307	0.9269
0.0221	8.0	920	0.3187	0.7684	0.7479	0.7580	0.9331
0.0172	9.0	1035	0.3954	0.7619	0.7057	0.7327	0.9277
0.0142	10.0	1150	0.3873	0.7570	0.7352	0.7459	0.9296
0.0116	11.0	1265	0.3827	0.7515	0.7443	0.7479	0.9310
0.0101	12.0	1380	0.4031	0.7623	0.7344	0.7481	0.9301
0.0074	13.0	1495	0.3884	0.7699	0.7308	0.7499	0.9326
0.0061	14.0	1610	0.4755	0.7410	0.6890	0.7141	0.9248
0.0056	15.0	1725	0.4165	0.7308	0.7503	0.7404	0.9289
0.0052	16.0	1840	0.4325	0.7402	0.7493	0.7447	0.9297
0.0059	17.0	1955	0.4150	0.7535	0.7469	0.7502	0.9313
0.0034	18.0	2070	0.4322	0.7588	0.7421	0.7504	0.9325
0.0037	19.0	2185	0.4424	0.7676	0.7330	0.7499	0.9317
0.0034	20.0	2300	0.4641	0.7462	0.7258	0.7358	0.9285
0.0028	21.0	2415	0.4524	0.7553	0.7453	0.7503	0.9311
0.0023	22.0	2530	0.4675	0.7539	0.7366	0.7452	0.9294
0.0022	23.0	2645	0.4650	0.7628	0.7421	0.7523	0.9308
0.0021	24.0	2760	0.4750	0.7584	0.7358	0.7469	0.9317
0.0022	25.0	2875	0.4700	0.7527	0.7425	0.7476	0.9308
0.0015	26.0	2990	0.4766	0.7618	0.7447	0.7531	0.9325
0.0018	27.0	3105	0.4866	0.7654	0.7300	0.7473	0.9317
0.0016	28.0	3220	0.4792	0.7601	0.7447	0.7523	0.9320
0.0014	29.0	3335	0.4807	0.7609	0.7403	0.7504	0.9320
0.0014	30.0	3450	0.4802	0.7590	0.7429	0.7509	0.9322

Framework versions

• Transformers 4.48.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.2
• Tokenizers 0.21.0