unsmile_classifier_v2

This model is a fine-tuned version of beomi/KcELECTRA-base-v2022 on the smilegate-ai/kor_unsmile dataset. It achieves the following results on the evaluation set:

• Loss: 0.1553
• Toxic Macro F1: 0.8219
• Toxic Macro Precision: 0.8210
• Toxic Macro Recall: 0.8397
• Toxic Micro F1: 0.8200
• Label Macro F1: 0.8187
• Label Weighted F1: 0.8106
• Exact Match: 0.7603
• Binary Precision: 0.9195
• Binary Recall: 0.9347
• Binary F1: 0.9270
• Binary Accuracy: 0.8903
• Fixed05 Toxic Macro F1: 0.8279
• Fixed05 Binary F1: 0.9288

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3.197461365470541e-05
• train_batch_size: 16
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.031198904067240532
• num_epochs: 6
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Toxic Macro F1	Toxic Macro Precision	Toxic Macro Recall	Toxic Micro F1	Label Macro F1	Label Weighted F1	Exact Match	Binary Precision	Binary Recall	Binary F1	Binary Accuracy	Fixed05 Toxic Macro F1	Fixed05 Binary F1
0.2404	1.0	938	0.1281	0.7747	0.7435	0.8116	0.7754	0.7742	0.7750	0.6790	0.8722	0.9648	0.9162	0.8684	0.7347	0.9180
0.1035	2.0	1876	0.1134	0.8129	0.8090	0.8275	0.8071	0.8121	0.8054	0.7196	0.8947	0.9512	0.9221	0.8801	0.7856	0.9204
0.0648	3.0	2814	0.1233	0.8247	0.8068	0.8461	0.8137	0.8220	0.8092	0.7384	0.9023	0.9419	0.9217	0.8807	0.8113	0.9243
0.0388	4.0	3752	0.1367	0.8268	0.8126	0.8446	0.8189	0.8232	0.8120	0.7571	0.9083	0.9454	0.9265	0.8882	0.8235	0.9262
0.0258	5.0	4690	0.1503	0.8293	0.8342	0.8371	0.8225	0.8256	0.8142	0.7662	0.9218	0.9397	0.9307	0.8957	0.8276	0.9295
0.017	6.0	5628	0.1553	0.8219	0.8210	0.8397	0.8200	0.8187	0.8106	0.7603	0.9195	0.9347	0.9270	0.8903	0.8279	0.9288

Framework versions

• Transformers 4.49.0
• Pytorch 2.11.0+cu128
• Datasets 3.3.2
• Tokenizers 0.21.4