learn-abc/banking77-intent-classifier

Repo: learn-abc/banking77-intent-classifier

Banking77 Intent Classifier (10-Intent)

Overview

This model is a fine-tuned BERT-based intent classifier designed for banking and financial customer queries. It is trained by mapping the original 77 Banking77 intents into a smaller, production-friendly set of custom intents, making it suitable for real-world conversational systems where simpler intent routing is required.

The model performs single-label text classification and is intended to be used as an intent detection component, not as a conversational or generative model.

---

Model Details

• Base model: bert-base-uncased
• Task: Text Classification (Intent Classification)
• Architecture: BertForSequenceClassification
• Languages: English (robust to informal and conversational phrasing)
• Max sequence length: 64 tokens
• Output: One intent label with confidence score

---

Custom Intent Schema

The original 77 Banking77 intents were mapped and consolidated into the following 12 production intents:

• ACCOUNT_INFO
• ATM_SUPPORT
• CARD_ISSUE
• CARD_MANAGEMENT
• CARD_REPLACEMENT
• CHECK_BALANCE
• EDIT_PERSONAL_DETAILS
• FAILED_TRANSFER
• FEES
• LOST_OR_STOLEN_CARD
• MINI_STATEMENT
• FALLBACK

Any user query that does not clearly belong to one of the supported categories is mapped to FALLBACK.

This design simplifies downstream business logic while retaining strong intent separation.

---

Training Data

• Primary dataset: PolyAI Banking77

• Original training samples: 10,003

• Test samples: 3,080

• After intent mapping and augmentation:

  • Training samples: 19,846
  • Includes: 280 explicitly added FALLBACK examples

Training Intent Distribution (Post-Mapping)

Intent	Samples
ACCOUNT_INFO	1,983
MINI_STATEMENT	1,809
FEES	1,490
FAILED_TRANSFER	1,045
CARD_MANAGEMENT	1,026
CARD_REPLACEMENT	749
ATM_SUPPORT	743
CARD_ISSUE	456
CHECK_BALANCE	352
LOST_OR_STOLEN_CARD	229
EDIT_PERSONAL_DETAILS	121
FALLBACK	280

Class imbalance was handled using class weighting during training.

---

Evaluation Results

Final evaluation on the Banking77 test set:

• Accuracy: 96.04%
• F1 (Micro): 0.960
• F1 (Macro): 0.956

These results indicate strong overall performance with good balance across both high-frequency and low-frequency intents.

---

Usage

Load the model

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "learn-abc/banking77-intent-classifier"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

def predict_intent(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=64)
    with torch.no_grad():
        outputs = model(**inputs)
        probs = torch.softmax(outputs.logits, dim=-1)
        pred_id = probs.argmax(dim=-1).item()
        confidence = probs[0][pred_id].item()

    return model.config.id2label[pred_id], confidence

# Example usage:
if __name__ == "__main__":
    test_texts = [
        "What is my account balance?",
        "Show me my last 10 transactions.",
        "I want to update my address.",
        "How do I apply for a loan?"
    ]

    for text in test_texts:
        intent, confidence = predict_intent(text)
        print(f"Input: {text}\nPredicted Intent: {intent} (Confidence: {confidence:.2f})\n")

---

Intended Use

This model is suitable for:

• Banking chatbots
• Voice assistant intent routing
• Customer support automation
• FAQ classification systems

It is designed to be used together with business rules, confirmation flows, and fallback handling.

---

Limitations and Safety Notes

• The model does not perform authentication or authorization
• It must not directly trigger financial actions
• High-risk intents (e.g. lost or stolen card) should always require explicit user confirmation
• Predictions should be validated with confidence thresholds and fallback logic

This model is not a replacement for human review in sensitive workflows.

---

Notes on Model Warnings

During training, warnings related to missing or unexpected keys were observed. These are expected when fine-tuning a pre-trained BERT checkpoint for a downstream classification task and do not impact inference correctness.

---

Citation

If you use this model, please cite:

• Devlin et al., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• PolyAI Banking77 Dataset

---

Maintainer

Developed and fine-tuned for production-oriented banking intent classification.

---

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors

• Author: Abhishek Singh
• LinkedIn: My LinkedIn Profile
• Portfolio: Abhishek Singh Portfolio

Model Card Contact

[More Information Needed]