srimoyee12/my_awesome_model

This model is a fine-tuned version of distilbert-base-uncased on the Auditor Review Dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.1735
• Validation Loss: 0.3834
• Train Accuracy: 0.8524
• Epoch: 3

Model description

This is a simple classifier model based on DistilBERT. It classifies given data into Negative, Neutral or Positive based on the sentiment.

Intended uses & limitations

Can be used for text classification.

This is created for illustration purposes and might not have the highest accuracy.

Training and evaluation data

Default split from the dataset card

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 1210, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Accuracy	Epoch
0.5919	0.4004	0.8359	0
0.2881	0.3590	0.8473	1
0.1735	0.3834	0.8524	2

Framework versions

• Transformers 4.27.3
• TensorFlow 2.11.0
• Datasets 2.10.1
• Tokenizers 0.13.2