metadata
language:
- en
license: mit
base_model: prajjwal1/bert-tiny
tags:
- pytorch
- movie-review-sentiment
- BertForSequenceClassification
- generated_from_trainer
metrics:
- accuracy
- matthews_correlation
model-index:
- name: tiny-imdb
results: []
datasets:
- imdb
library_name: transformers
pipeline_tag: text-classification
tiny-imdb
This model is a fine-tuned version of prajjwal1/bert-tiny on the imdb dataset. It achieves the following results on the evaluation set:
- Loss: 0.2775
- Accuracy: 0.8944
- Matthews Correlation: 0.7888
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 320
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 9
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy | Matthews Correlation |
|---|---|---|---|---|---|
| 0.4927 | 1.0 | 1250 | 0.3557 | 0.8484 | 0.7016 |
| 0.298 | 2.0 | 2500 | 0.2874 | 0.8866 | 0.7732 |
| 0.2555 | 3.0 | 3750 | 0.2799 | 0.8912 | 0.7828 |
| 0.2132 | 4.0 | 5000 | 0.2775 | 0.8944 | 0.7888 |
| 0.1779 | 5.0 | 6250 | 0.3065 | 0.891 | 0.7835 |
| 0.1508 | 6.0 | 7500 | 0.3331 | 0.889 | 0.7811 |
| 0.1304 | 7.0 | 8750 | 0.3451 | 0.8926 | 0.7870 |
| 0.119 | 8.0 | 10000 | 0.3670 | 0.8915 | 0.7852 |
| 0.1118 | 9.0 | 11250 | 0.3655 | 0.891 | 0.7840 |
Framework versions
- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0