---
license: apache-2.0
language:
- en
tags:
- text-classification
- support-tickets
- setfit
- nlp
- customer-support
metrics:
- f1
---

# 🎫 Support Ticket Classifier

Automatically classifies customer support tickets by **category** and **urgency** using a fine-tuned SetFit model trained on 30,000+ real support tickets.

## What It Does

**Input:** Raw support ticket text  
**Output:** Category + confidence score + urgency level

```json
{
  "category": "billing",
  "confidence": 0.79,
  "urgency": "high"
}
```

## Categories

| Category | Example ticket |
|---|---|
| `billing` | "I was charged twice for my subscription" |
| `technical` | "My account keeps logging me out" |
| `complaint` | "This service is completely unacceptable" |
| `refund` | "I want to cancel and get my money back" |

## Urgency Levels

| Level | When assigned |
|---|---|
| `high` | Fraud, service down, unauthorized charges, locked out |
| `medium` | General issues, standard requests |
| `low` | General questions, curiosity, minor changes |

## Performance

| Metric | Score |
|---|---|
| Weighted F1 | **82%** |
| Complaint F1 | 92% |
| Technical F1 | 82% |
| Billing F1 | 79% |
| Refund F1 | 68% |

Trained on 30,571 labeled tickets from Kaggle + HuggingFace datasets.  
Evaluated on a held-out test set of 3,058 tickets.

## Why Use This Instead of an LLM?

- ✅ **100x cheaper per call** than GPT-4 at volume
- ✅ **Fast** — under 200ms per ticket
- ✅ **Private** — runs on your own server, data never leaves your infrastructure
- ✅ **No vendor lock-in** — no API key, no per-token billing
- ✅ **GDPR friendly** — fully on-premise capable

## Quick Start

### Install dependencies
```bash
pip install setfit==1.0.3 sentence-transformers==2.7.0 transformers==4.40.2 huggingface_hub==0.23.5 scikit-learn numpy
```

### Run predictions
```python
from predict import predict_ticket

result = predict_ticket("I was charged twice and need a refund immediately")
print(result)
# {"category": "billing", "confidence": 0.79, "urgency": "high"}
```

## Files in This Repo

| File | Description |
|---|---|
| `predict.py` | Ready-to-run prediction script |
| `requirements.txt` | Pinned dependencies |
| `category_model/` | Fine-tuned SetFit classifier |
| `calibration.pkl` | Platt scaling confidence calibration |
| `label_mappings.pkl` | Label encoders |

## Tech Stack

- **Model:** SetFit (Sentence Transformers fine-tuning)
- **Base model:** `paraphrase-MiniLM-L3-v2`
- **Training data:** 30,571 labeled support tickets
- **Confidence calibration:** Platt scaling on held-out validation set
- **Urgency:** Keyword-rule layer (transparent and auditable)

## Get the Full Docker API Version

Want a production-ready REST API you can deploy to your own server in minutes?

The **Docker version** includes:
- FastAPI wrapper (`POST /predict` endpoint)
- Dockerfile — one command to deploy anywhere
- Full setup guide

👉 **[Get the Docker API version on Gumroad](https://faysaldev.gumroad.com/l/support-ticket-classifier)** 
## Sample Results