Fine-tuned mBERT for Urdu sentiment classification

README.md

@@ -0,0 +1,69 @@
+---
+library_name: transformers
+license: apache-2.0
+base_model: bert-base-multilingual-cased
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+- f1
+model-index:
+- name: urdu-sentiment-classifier
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# urdu-sentiment-classifier
+
+This model is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.8915
+- Accuracy: 0.81
+- F1: 0.8098
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 64
+- eval_batch_size: 128
+- seed: 42
+- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 100
+- num_epochs: 5
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
+| 0.8954        | 1.0   | 625  | 0.9527          | 0.7696   | 0.7656 |
+| 0.7918        | 2.0   | 1250 | 0.8385          | 0.8      | 0.7999 |
+| 0.6211        | 3.0   | 1875 | 0.8908          | 0.8101   | 0.8099 |
+| 0.5833        | 4.0   | 2500 | 0.9350          | 0.8088   | 0.8087 |
+| 0.4515        | 5.0   | 3125 | 1.0381          | 0.8081   | 0.8081 |
+
+
+### Framework versions
+
+- Transformers 5.0.0
+- Pytorch 2.10.0+cu128
+- Datasets 4.8.5
+- Tokenizers 0.22.2

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e07bac5733c952c5249686ce9f7c77ebebf57b79f7023e9473f176cf844278ec
+oid sha256:5a59bcd26163ad38e5d59321bbb1d986838febdf61bfd727686fc7706d52060e
 size 711443432