84c9d3cc9c2cbcade50a6e0681fd828a

This model is a fine-tuned version of studio-ousia/luke-japanese-base-lite on the contemmcm/hate-speech-and-offensive-language dataset. It achieves the following results on the evaluation set:

• Loss: 0.3314
• Data Size: 1.0
• Epoch Runtime: 57.4905
• Accuracy: 0.8963
• F1 Macro: 0.5965

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Accuracy	F1 Macro
No log	0	0	1.2909	0	4.7830	0.0601	0.0378
No log	1	619	0.6936	0.0078	5.3710	0.7672	0.2894
No log	2	1238	0.6330	0.0156	5.6596	0.7672	0.2894
0.0157	3	1857	0.5222	0.0312	6.8161	0.7675	0.2902
0.0157	4	2476	0.4454	0.0625	8.4137	0.8533	0.5406
0.3886	5	3095	0.3683	0.125	11.7597	0.8685	0.5655
0.03	6	3714	0.3690	0.25	18.3968	0.8825	0.5839
0.3176	7	4333	0.3331	0.5	31.5896	0.8874	0.7210
0.3051	8.0	4952	0.2843	1.0	58.7464	0.9026	0.6057
0.2721	9.0	5571	0.3118	1.0	57.1358	0.8916	0.5909
0.2966	10.0	6190	0.2835	1.0	56.7888	0.9042	0.6206
0.2821	11.0	6809	0.3134	1.0	57.2508	0.9012	0.6729
0.2334	12.0	7428	0.2887	1.0	56.9849	0.9034	0.6067
0.2381	13.0	8047	0.3009	1.0	57.4678	0.9067	0.7410
0.2342	14.0	8666	0.3314	1.0	57.4905	0.8963	0.5965

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.3.0
• Tokenizers 0.22.1