| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - text-classification |
| - support-tickets |
| - setfit |
| - nlp |
| - customer-support |
| metrics: |
| - f1 |
| --- |
| |
| # 🎫 Support Ticket Classifier |
|
|
| Automatically classifies customer support tickets by **category** and **urgency** using a fine-tuned SetFit model trained on 30,000+ real support tickets. |
|
|
| ## What It Does |
|
|
| **Input:** Raw support ticket text |
| **Output:** Category + confidence score + urgency level |
|
|
| ```json |
| { |
| "category": "billing", |
| "confidence": 0.79, |
| "urgency": "high" |
| } |
| ``` |
|
|
| ## Categories |
|
|
| | Category | Example ticket | |
| |---|---| |
| | `billing` | "I was charged twice for my subscription" | |
| | `technical` | "My account keeps logging me out" | |
| | `complaint` | "This service is completely unacceptable" | |
| | `refund` | "I want to cancel and get my money back" | |
|
|
| ## Urgency Levels |
|
|
| | Level | When assigned | |
| |---|---| |
| | `high` | Fraud, service down, unauthorized charges, locked out | |
| | `medium` | General issues, standard requests | |
| | `low` | General questions, curiosity, minor changes | |
|
|
| ## Performance |
|
|
| | Metric | Score | |
| |---|---| |
| | Weighted F1 | **82%** | |
| | Complaint F1 | 92% | |
| | Technical F1 | 82% | |
| | Billing F1 | 79% | |
| | Refund F1 | 68% | |
|
|
| Trained on 30,571 labeled tickets from Kaggle + HuggingFace datasets. |
| Evaluated on a held-out test set of 3,058 tickets. |
|
|
| ## Why Use This Instead of an LLM? |
|
|
| - ✅ **100x cheaper per call** than GPT-4 at volume |
| - ✅ **Fast** — under 200ms per ticket |
| - ✅ **Private** — runs on your own server, data never leaves your infrastructure |
| - ✅ **No vendor lock-in** — no API key, no per-token billing |
| - ✅ **GDPR friendly** — fully on-premise capable |
|
|
| ## Quick Start |
|
|
| ### Install dependencies |
| ```bash |
| pip install setfit==1.0.3 sentence-transformers==2.7.0 transformers==4.40.2 huggingface_hub==0.23.5 scikit-learn numpy |
| ``` |
|
|
| ### Run predictions |
| ```python |
| from predict import predict_ticket |
| |
| result = predict_ticket("I was charged twice and need a refund immediately") |
| print(result) |
| # {"category": "billing", "confidence": 0.79, "urgency": "high"} |
| ``` |
|
|
| ## Files in This Repo |
|
|
| | File | Description | |
| |---|---| |
| | `predict.py` | Ready-to-run prediction script | |
| | `requirements.txt` | Pinned dependencies | |
| | `category_model/` | Fine-tuned SetFit classifier | |
| | `calibration.pkl` | Platt scaling confidence calibration | |
| | `label_mappings.pkl` | Label encoders | |
|
|
| ## Tech Stack |
|
|
| - **Model:** SetFit (Sentence Transformers fine-tuning) |
| - **Base model:** `paraphrase-MiniLM-L3-v2` |
| - **Training data:** 30,571 labeled support tickets |
| - **Confidence calibration:** Platt scaling on held-out validation set |
| - **Urgency:** Keyword-rule layer (transparent and auditable) |
|
|
| ## Get the Full Docker API Version |
|
|
| Want a production-ready REST API you can deploy to your own server in minutes? |
|
|
| The **Docker version** includes: |
| - FastAPI wrapper (`POST /predict` endpoint) |
| - Dockerfile — one command to deploy anywhere |
| - Full setup guide |
|
|
| 👉 **[Get the Docker API version on Gumroad](https://faysaldev.gumroad.com/l/support-ticket-classifier)** |
| ## Sample Results |
