---
language:
  - en
license: mit
library_name: transformers
pipeline_tag: text-classification
tags:
  - bert
  - sentiment-analysis
  - text-classification
  - imdb
  - pytorch
  - transformers
---

# BERT Sentiment Classifier

A fine-tuned **BERT Base Uncased** model for English movie review sentiment classification.

This model classifies movie reviews into two sentiment categories:

- **POSITIVE (1):** The review expresses a positive opinion.
- **NEGATIVE (0):** The review expresses a negative opinion.

The model was fine-tuned using the IMDb Movie Reviews dataset and evaluated on a separate test set containing 2,000 examples.

## Model Details

| Property | Description |
|---|---|
| Model architecture | BERT Base Uncased |
| Base model | `google-bert/bert-base-uncased` |
| Task | Binary Sentiment Classification |
| Number of classes | 2 |
| Positive label | 1 |
| Negative label | 0 |
| Maximum sequence length | 192 tokens |
| Framework | PyTorch |
| Library | Hugging Face Transformers |

## Dataset

The model was fine-tuned using the [IMDb Movie Reviews Dataset](https://huggingface.co/datasets/stanfordnlp/imdb).

The notebook uses the following dataset split:

| Dataset | Number of samples |
|---|---:|
| Training | 8,000 |
| Validation | 1,000 |
| Testing | 2,000 |
| **Total** | **11,000** |

The dataset contains English movie reviews with binary sentiment labels:

- `0`: Negative
- `1`: Positive

The training set is used to fine-tune the model, the validation set is used to monitor training and evaluate checkpoints, and the test set is used for final performance evaluation.

## Training Configuration

The model was fine-tuned for three epochs using the following configuration:

| Parameter | Value |
|---|---|
| Base model | BERT Base Uncased |
| Epochs | 3 |
| Batch size | 16 |
| Learning rate | 2e-5 |
| Weight decay | 0.01 |
| Warmup steps | 100 |
| Maximum sequence length | 192 |
| Random seed | 42 |
| Best checkpoint metric | Validation accuracy |
| Training framework | PyTorch |

A two-class classification head was added to the pretrained BERT encoder and fine-tuned for sentiment prediction.

## Training Results

The following table shows the training progress across all three epochs.

| Epoch | Training Loss | Validation Loss | Accuracy | Precision | Recall | F1-score |
|---|---:|---:|---:|---:|---:|---:|
| 1 | 0.376433 | 0.259453 | 0.896000 | 0.919321 | 0.867735 | 0.892784 |
| 2 | 0.166479 | 0.358983 | 0.884000 | 0.894845 | 0.869739 | 0.882114 |
| 3 | 0.134408 | 0.445110 | 0.894000 | 0.887574 | 0.901804 | 0.894632 |

Training loss decreased throughout training. However, validation loss increased after the first epoch. This may indicate overfitting, even though the validation accuracy remained relatively stable.

## Final Validation Results

The final validation evaluation produced the following results:

| Metric | Result |
|---|---:|
| Validation Loss | 0.259079 |
| Accuracy | **89.70%** |
| Precision | 91.95% |
| Recall | 86.97% |
| F1-score | **89.39%** |

## Final Test Results

The final evaluation was performed on a separate test set containing **2,000 movie reviews**, with 1,000 Negative and 1,000 Positive examples.

| Metric | Result |
|---|---:|
| Test Loss | 0.259910 |
| Accuracy | **88.80%** |
| Precision | 91.54% |
| Recall | 85.50% |
| F1-score | **88.42%** |

The model correctly classified 1,776 out of 2,000 test examples.

### Classification Report

| Class | Precision | Recall | F1-score | Support |
|---|---:|---:|---:|---:|
| Negative | 0.8640 | 0.9210 | 0.8916 | 1,000 |
| Positive | 0.9154 | 0.8550 | 0.8842 | 1,000 |
| **Macro Average** | **0.8897** | **0.8880** | **0.8879** | **2,000** |
| **Weighted Average** | **0.8897** | **0.8880** | **0.8879** | **2,000** |

### Confusion Matrix

The confusion matrix shows the actual and predicted sentiment labels.

| Actual / Predicted | Predicted Negative | Predicted Positive |
|---|---:|---:|
| Actual Negative | 921 | 79 |
| Actual Positive | 145 | 855 |

**Interpretation:**

- 921 Negative reviews were correctly classified.
- 855 Positive reviews were correctly classified.
- 79 Negative reviews were incorrectly classified as Positive.
- 145 Positive reviews were incorrectly classified as Negative.

The model achieved higher recall for Negative reviews (92.10%) than Positive reviews (85.50%).

## Model Architecture

This project uses an **encoder-only transformer architecture**.

BERT reads the input text and creates contextual representations of its tokens. The classification head uses these representations to predict the sentiment of the complete review.

The model follows this general workflow:

1. Receive an English movie review.
2. Tokenize the input text.
3. Pass the tokens through the pretrained BERT encoder.
4. Generate classification logits.
5. Apply softmax to obtain class probabilities.
6. Return the predicted sentiment and confidence score.

## Usage

You can load this model directly using Hugging Face Transformers.

### Install Dependencies

```bash
pip install transformers torch
```

### Load the Model

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

MODEL_NAME = "ullahhafeez321/bert-sentiment-classifier"

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

model = AutoModelForSequenceClassification.from_pretrained(MODEL_NAME)

model.eval()
```

### Make a Prediction

```python
def predict_sentiment(text):

    inputs = tokenizer(
        text,
        return_tensors="pt",
        truncation=True,
        max_length=192
    )

    with torch.inference_mode():
        outputs = model(**inputs)

        probabilities = torch.softmax(
            outputs.logits,
            dim=-1
        )[0]

    predicted_id = int(probabilities.argmax())

    sentiment = model.config.id2label[predicted_id]

    confidence = float(probabilities[predicted_id]) * 100

    return {
        "text": text,
        "sentiment": sentiment,
        "confidence": round(confidence, 2)
    }


result = predict_sentiment(
    "This movie was excellent. I really enjoyed it!"
)

print(result)
```

### Example Predictions

| Input Review | Expected Sentiment |
|---|---|
| This movie was absolutely wonderful! | POSITIVE |
| This movie was terrible and boring. | NEGATIVE |
| I loved every moment of this film. | POSITIVE |
| I hated this movie. It was a waste of time. | NEGATIVE |

These examples are illustrative and are not guaranteed predictions.

## Applications

This model can be integrated into:

- Sentiment analysis websites
- Movie review classification applications
- Text classification projects
- Python applications
- Streamlit applications
- Educational NLP projects

## Limitations

- The model was fine-tuned on English IMDb movie reviews.
- It is primarily intended for movie-review sentiment classification.
- It may not perform equally well on other domains, such as product reviews, news, or social media.
- Sarcasm, irony, ambiguous language, and mixed sentiment can be difficult to classify.
- Confidence scores are model probabilities, not guarantees of correctness.
- Validation loss increased after the first epoch, suggesting possible overfitting.

## Technologies Used

- Python
- PyTorch
- Hugging Face Transformers
- Hugging Face Datasets
- Scikit-learn
- IMDb Movie Reviews Dataset
- Google BERT Base Uncased

## Training Notebook

The training notebook includes:

1. Library installation and environment setup.
2. IMDb dataset loading and splitting.
3. Pretrained BERT tokenizer and model initialization.
4. Dataset tokenization.
5. Model fine-tuning.
6. Validation and test evaluation.
7. Classification report and confusion matrix.
8. Custom sentiment prediction.
9. Saving and uploading the model to Hugging Face.

## Project Integration

This model is also used as the sentiment classification component in my NLP Analysis Studio application, alongside a separate decoder-only Qwen model for Named Entity Recognition.

- **Live Application:** https://sentimentclassifynamedentitymodels.streamlit.app/
- **GitHub Repository:** https://github.com/ullahhafeez321/SentimentClassify_NamedEntityModels
- **BERT Model:** https://huggingface.co/ullahhafeez321/bert-sentiment-classifier
- **Qwen NER Model:** https://huggingface.co/ullahhafeez321/Qwen-NER-Decoder

## License

This project is released under the MIT License.