# DistilBERT-Based Intent Detection Model for Banking Customer Queries

This repository contains a fine-tuned **DistilBERT** model for **intent detection** in banking customer support scenarios. It is trained on the **BANKING77 dataset** and designed to accurately classify user queries into 77 distinct banking-related intents.

## Model Details

- **Model Architecture:** DistilBERT Base Uncased  
- **Task:** Intent Detection for Banking Queries  
- **Dataset:** [BANKING77](https://huggingface.co/datasets/banking77)  
- **Fine-tuning Framework:** Hugging Face Transformers  
- **Language:** English  
- **Number of Labels:** 77  
- **Quantization:** *Not applied (full precision)*  

## Usage

### Installation

```bash
pip install transformers torch datasets
```

### Loading the Model

```python
from transformers import DistilBertTokenizer, DistilBertForSequenceClassification
import torch

# Load fine-tuned model
model_path = "./banking77-distilbert"  # Adjust path if different
model = DistilBertForSequenceClassification.from_pretrained(model_path)
tokenizer = DistilBertTokenizer.from_pretrained(model_path)
model.eval()

# Sample query
text = "I need to reset my online banking password."

# Tokenize and predict
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
with torch.no_grad():
    outputs = model(**inputs)
predicted_class = torch.argmax(outputs.logits, dim=1).item()

# Example label map (replace with the full BANKING77 map)
label_map = {0: "activate_my_card", 1: "balance", 2: "card_arrival", ..., 76: "why_was_my_card_declined"}
print(f"Predicted Intent: {label_map[predicted_class]}")
```

## Performance Metrics

- **Accuracy:** ~95% (on the BANKING77 test split)  
- **Loss:** ~0.13 (after fine-tuning for 4 epochs)  

## Fine-Tuning Details

### Dataset

- **Name:** BANKING77  
- **Size:** ~13,000 customer support queries  
- **Intents:** 77 unique labeled banking intents

### Training

- **Epochs:** 4  
- **Batch Size:** 16  
- **Learning Rate:** 2e-5  
- **Optimizer:** AdamW  
- **Evaluation Strategy:** per epoch  
- **Loss Function:** CrossEntropyLoss  

### Hardware

- **GPU Used:** NVIDIA Tesla T4 (via Google Colab)  
- **Training Time:** ~15 minutes  

## Repository Structure

```
.
├── banking77-distilbert/     # Fine-tuned model directory (saved via trainer.save_model)
│   ├── config.json
│   ├── pytorch_model.bin
│   ├── tokenizer_config.json
│   ├── vocab.txt
├── intent_predictor.py       # Script for predicting intents (with preprocessing)
├── README.md                 # Model documentation
```

## Limitations

- The model is trained only on banking-related intents; it may misclassify out-of-domain queries.
- Multilingual support is not included — limited to English.
- Model does not handle multiple intents per query.

## Contributing

Contributions and suggestions are welcome. Please open an issue or pull request for improvements or additional features.