| # DistilBERT-Based Intent Detection Model for Banking Customer Queries | |
| This repository contains a fine-tuned **DistilBERT** model for **intent detection** in banking customer support scenarios. It is trained on the **BANKING77 dataset** and designed to accurately classify user queries into 77 distinct banking-related intents. | |
| ## Model Details | |
| - **Model Architecture:** DistilBERT Base Uncased | |
| - **Task:** Intent Detection for Banking Queries | |
| - **Dataset:** [BANKING77](https://huggingface.co/datasets/banking77) | |
| - **Fine-tuning Framework:** Hugging Face Transformers | |
| - **Language:** English | |
| - **Number of Labels:** 77 | |
| - **Quantization:** *Not applied (full precision)* | |
| ## Usage | |
| ### Installation | |
| ```bash | |
| pip install transformers torch datasets | |
| ``` | |
| ### Loading the Model | |
| ```python | |
| from transformers import DistilBertTokenizer, DistilBertForSequenceClassification | |
| import torch | |
| # Load fine-tuned model | |
| model_path = "./banking77-distilbert" # Adjust path if different | |
| model = DistilBertForSequenceClassification.from_pretrained(model_path) | |
| tokenizer = DistilBertTokenizer.from_pretrained(model_path) | |
| model.eval() | |
| # Sample query | |
| text = "I need to reset my online banking password." | |
| # Tokenize and predict | |
| inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True) | |
| with torch.no_grad(): | |
| outputs = model(**inputs) | |
| predicted_class = torch.argmax(outputs.logits, dim=1).item() | |
| # Example label map (replace with the full BANKING77 map) | |
| label_map = {0: "activate_my_card", 1: "balance", 2: "card_arrival", ..., 76: "why_was_my_card_declined"} | |
| print(f"Predicted Intent: {label_map[predicted_class]}") | |
| ``` | |
| ## Performance Metrics | |
| - **Accuracy:** ~95% (on the BANKING77 test split) | |
| - **Loss:** ~0.13 (after fine-tuning for 4 epochs) | |
| ## Fine-Tuning Details | |
| ### Dataset | |
| - **Name:** BANKING77 | |
| - **Size:** ~13,000 customer support queries | |
| - **Intents:** 77 unique labeled banking intents | |
| ### Training | |
| - **Epochs:** 4 | |
| - **Batch Size:** 16 | |
| - **Learning Rate:** 2e-5 | |
| - **Optimizer:** AdamW | |
| - **Evaluation Strategy:** per epoch | |
| - **Loss Function:** CrossEntropyLoss | |
| ### Hardware | |
| - **GPU Used:** NVIDIA Tesla T4 (via Google Colab) | |
| - **Training Time:** ~15 minutes | |
| ## Repository Structure | |
| ``` | |
| . | |
| ├── banking77-distilbert/ # Fine-tuned model directory (saved via trainer.save_model) | |
| │ ├── config.json | |
| │ ├── pytorch_model.bin | |
| │ ├── tokenizer_config.json | |
| │ ├── vocab.txt | |
| ├── intent_predictor.py # Script for predicting intents (with preprocessing) | |
| ├── README.md # Model documentation | |
| ``` | |
| ## Limitations | |
| - The model is trained only on banking-related intents; it may misclassify out-of-domain queries. | |
| - Multilingual support is not included — limited to English. | |
| - Model does not handle multiple intents per query. | |
| ## Contributing | |
| Contributions and suggestions are welcome. Please open an issue or pull request for improvements or additional features. | |