# DistilBERT Quantized Model for IMDB Sentiment Analysis

This repository contains a quantized DistilBERT model fine-tuned for binary sentiment classification on IMDB movie reviews. Optimized for production deployment, the model achieves high accuracy while maintaining efficiency.

## Model Details

- **Model Architecture:** DistilBERT Base Uncased  
- **Task:** Binary Sentiment Analysis (Positive/Negative)  
- **Dataset:** IMDB Movie Reviews (50K samples)  
- **Quantization:** Dynamic Quantization (INT8)  
- **Framework:** Hugging Face Transformers + PyTorch  

## Usage

### Installation

```sh
pip install transformers torch scikit-learn pandas
```

### Loading the Model

```python
from transformers import DistilBertForSequenceClassification, DistilBertTokenizer
import torch

# Load quantized model
model_path = "./quantized_sentiment_model.pth"
model = DistilBertForSequenceClassification.from_pretrained("./sentiment_model")
model.load_state_dict(torch.load(model_path))
model.eval()

# Load tokenizer
tokenizer = DistilBertTokenizer.from_pretrained("./sentiment_model")

def predict_sentiment(text):
    inputs = tokenizer(text, return_tensors="pt", 
                      padding=True, truncation=True, 
                      max_length=128)
    
    with torch.no_grad():
        outputs = model(**inputs)
    
    prediction = torch.argmax(outputs.logits).item()
    return "Positive" if prediction == 1 else "Negative"

# Example usage
review = "This movie blew me away with its stunning visuals and gripping storyline."
print(predict_sentiment(review))  # Output: Positive
```

## 📊 Performance Metrics

| Metric                   | Value   |
|--------------------------|---------|
| Accuracy                 | 89.1%   |
| F1 Score                 | 89.0%   |
| Inference Latency (CPU) | 12ms    |
| Model Size               | 67MB    |

## 🏋️ Training Details

### Dataset

- 50,000 IMDB movie reviews  
- Balanced binary classes (50% positive, 50% negative)  

### Hyperparameters

- Epochs: 5  
- Batch Size: 24 (Effective 48 with accumulation)  
- Learning Rate: 8e-6  
- Warmup Ratio: 10%  
- Weight Decay: 0.005  
- Optimizer: AdamW with Cosine LR Schedule  

### Quantization

Applied dynamic post-training quantization:

```python
quantized_model = torch.quantization.quantize_dynamic(
    model, {torch.nn.Linear}, dtype=torch.qint8
)
```

## 📁 Repository Structure

```
.
├── sentiment_model/                # Full-precision model files
│   ├── config.json
│   ├── pytorch_model.bin
│   └── tokenizer files...
├── quantized_sentiment_model.pth  # Quantized weights
├── imdb_train.csv                 # Sample training data
├── train.py                       # Training script
└── inference.py                   # Usage examples
```

## ⚠️ Limitations

- Accuracy may drop on reviews with:
  - Sarcasm or nuanced language
  - Domain-specific terminology (non-movie content)
- Maximum sequence length: 128 tokens
- English language only