# Sarcasm Detection with BERT

This repository contains a fine-tuned BERT model for detecting sarcasm in headlines and text. The model achieves high accuracy in distinguishing between sarcastic and non-sarcastic content using natural language processing techniques.

---

## Model Details

- **Model Name:** BERT-Base-Uncased Fine-tuned for Sarcasm Detection
- **Model Architecture:** BERT Base (110M parameters)
- **Task:** Binary Classification (Sarcastic vs Non-Sarcastic)
- **Dataset:** Sarcasm Headlines Dataset
- **Quantization:** Float16 (for optimized deployment)
- **Fine-tuning Framework:** Hugging Face Transformers

---

## Dataset

The model was trained on the **Sarcasm Headlines Dataset** which contains:
- **Total Samples:** 26,709 headlines
- **Features:** 
  - `headline`: The text content to classify
  - `is_sarcastic`: Binary label (1 for sarcastic, 0 for non-sarcastic)
- **Train/Test Split:** 90% training, 10% evaluation

---

## Performance Metrics

| Epoch | Training Loss | Validation Loss | Accuracy |
|-------|---------------|-----------------|----------|
| 1     | 0.2048        | 0.1821          | 92.96%   |
| 2     | 0.1138        | 0.2792          | 91.01%   |
| 3     | 0.0586        | 0.2372          | **93.86%** |

**Final Model Performance:**
- **Best Accuracy:** 93.86%
- **Final Training Loss:** 0.146

---

## Installation

```bash
pip install transformers datasets evaluate scikit-learn torch
```

---

## Usage

### Quick Start

```python
from transformers import pipeline
import torch

# Load the trained model
classifier = pipeline("text-classification", 
                     model="./sarcasm_model", 
                     tokenizer="./sarcasm_model")

# Test examples
test_inputs = [
    "I'm absolutely thrilled to be stuck in traffic again.",
    "The weather is nice and sunny today.",
    "Oh great, another email from the boss with more tasks."
]

for sentence in test_inputs:
    result = classifier(sentence)[0]
    label = "Sarcastic" if result["label"] == "LABEL_1" else "Not Sarcastic"
    print(f"'{sentence}' → {label} (Confidence: {result['score']:.2f})")
```

### Manual Model Loading

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained("./sarcasm_model")
tokenizer = AutoTokenizer.from_pretrained("./sarcasm_model")

# Tokenize input
text = "Oh wonderful, another Monday morning!"
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=128)

# Inference
with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
    predicted_class = outputs.logits.argmax(dim=1).item()

label_mapping = {0: "Not Sarcastic", 1: "Sarcastic"}
confidence = predictions[0][predicted_class].item()
print(f"Prediction: {label_mapping[predicted_class]} (Confidence: {confidence:.2f})")
```

---

## Training Configuration

### Model Parameters
- **Base Model:** `bert-base-uncased`
- **Number of Labels:** 2 (binary classification)
- **Max Sequence Length:** 128 tokens
- **Tokenization:** WordPiece with padding and truncation

### Training Arguments
- **Learning Rate:** 2e-5
- **Batch Size:** 16 (training), 32 (evaluation)
- **Epochs:** 3
- **Weight Decay:** 0.01
- **Evaluation Strategy:** Every epoch
- **Optimizer:** AdamW (default)

### Hardware Requirements
- **GPU:** NVIDIA Tesla T4 (or equivalent)
- **Memory:** ~4GB GPU memory for training
- **Training Time:** ~18 minutes for 3 epochs

---

## Model Architecture

The model uses BERT's transformer architecture with:
- **Encoder Layers:** 12
- **Attention Heads:** 12
- **Hidden Size:** 768
- **Vocabulary Size:** 30,522
- **Classification Head:** Linear layer (768 → 2)

---

## File Structure

```
sarcasm-detection/
├── sarcasm_model/              # Main fine-tuned model
│   ├── config.json
│   ├── model.safetensors
│   ├── tokenizer_config.json
│   ├── special_tokens_map.json
│   ├── vocab.txt
│   └── tokenizer.json
├── quantized-model/            # Float16 quantized version
│   ├── config.json
│   ├── model.safetensors
│   └── tokenizer files...
├── logs/                       # Training logs
├── sarcasm-detection.ipynb     # Training notebook
└── README.md                   # This file
```

---

## Quantization

A quantized version of the model is available for deployment optimization:

```python
# Load quantized model (Float16)
quantized_model = AutoModelForSequenceClassification.from_pretrained("./quantized-model")
quantized_model = quantized_model.to(dtype=torch.float16)
```

**Benefits of Quantization:**
- **Reduced Memory Usage:** ~50% smaller model size
- **Faster Inference:** Improved speed on compatible hardware
- **Minimal Accuracy Loss:** Maintains classification performance

---

## Limitations

- **Domain Specificity:** Trained primarily on headlines; may not generalize perfectly to other text types
- **Context Dependency:** Sarcasm detection can be highly context-dependent and subjective
- **Cultural Nuances:** May not capture sarcasm patterns from different cultural contexts
- **Short Text Focus:** Optimized for headline-length text (typically under 128 tokens)

---

## Potential Improvements

- **Data Augmentation:** Include more diverse sarcasm examples
- **Ensemble Methods:** Combine multiple models for better accuracy
- **Context Integration:** Incorporate additional context beyond the headline
- **Multi-language Support:** Extend to other languages
- **Real-time Processing:** Optimize for streaming applications

---

## Applications

- **Social Media Monitoring:** Detect sarcastic comments and posts
- **Content Moderation:** Identify potentially misleading sarcastic content
- **Sentiment Analysis Enhancement:** Improve sentiment classification accuracy
- **News Analysis:** Analyze editorial tone and bias in headlines
- **Customer Feedback:** Better understand customer sentiment in reviews

---

## Citation

If you use this model in your research, please cite:

```bibtex
@misc{sarcasm_detection_bert,
  title={BERT-based Sarcasm Detection for Headlines},
  author={Your Name},
  year={2025},
  note={Fine-tuned BERT model for binary sarcasm classification}
}
```

---

## Contributing

Contributions are welcome! Please feel free to:
- Report bugs or issues
- Suggest improvements
- Add new features
- Improve documentation

---

## License

This project is licensed under the MIT License. The underlying BERT model follows Google's Apache 2.0 license.

---

## Acknowledgments

- **Hugging Face** for the Transformers library
- **Google Research** for the original BERT model
- **Kaggle** for providing the Sarcasm Headlines Dataset
- **PyTorch** for the deep learning framework