bdbert

This model is a fine-tuned version of csebuetnlp/banglabert on the /kaggle/input/competitions/bangla-code-mixed-sentiment-analysis-banglish-benglish/train.csv dataset. It achieves the following results on the evaluation set:

• Loss: 0.3231
• Accuracy: 0.9549
• F1: 0.9528

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	1.0	51	1.5852	0.6823	0.6170
No log	2.0	102	0.9237	0.8559	0.8517
No log	3.0	153	0.6013	0.9115	0.9061
No log	4.0	204	0.4866	0.9184	0.9146
No log	5.0	255	0.4016	0.9427	0.9405
No log	6.0	306	0.3525	0.9462	0.9441
No log	7.0	357	0.3383	0.9514	0.9493
No log	8.0	408	0.3370	0.9479	0.9458
No log	9.0	459	0.3464	0.9531	0.9509
0.6486	10.0	510	0.3231	0.9549	0.9528

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2