README.md

---
license: apache-2.0
language:
  - en
base_model: distilbert-base-uncased
pipeline_tag: text-classification
library_name: transformers
tags:
  - text-classification
  - distilbert
  - it-support
  - ticket-classification
datasets:
  - adisongoh/it-service-ticket-classification-dataset
metrics:
  - accuracy
  - f1
model-index:
  - name: ticket-classification-distilbert
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: IT Service Ticket Classification Dataset
          type: adisongoh/it-service-ticket-classification-dataset
        metrics:
          - type: accuracy
            value: 0.88
            name: Accuracy
          - type: f1
            value: 0.88
            name: Macro F1
---

# Model Card for ticket-classification-distilbert

A fine-tuned DistilBERT model for classifying IT support tickets into topic categories.

## Model Details

### Model Description

This model classifies natural-language IT support ticket descriptions into one of 8 categories: Access, Administrative rights, HR Support, Hardware, Internal Project, Miscellaneous, Purchase, Storage.

- **Developed by:** Vikaash17
- **Model type:** Text classification (fine-tuned transformer)
- **Language(s) (NLP):** English
- **License:** Apache 2.0
- **Finetuned from model:** distilbert-base-uncased

### Model Sources

- **Repository:** [GitHub — IT_Ticket_Classification](https://github.com/Vikaash-17/IT_Ticket_Classification) <!-- replace with your actual link -->

## Uses

### Direct Use

This model can be used to automatically classify IT support ticket text into predefined categories, useful for automated ticket routing/triage in IT service desks.

### Out-of-Scope Use

Not intended for non-English text, tickets outside the IT service-desk domain, or categories not represented in the training data. Not suitable as a general-purpose text classifier.

## Bias, Risks, and Limitations

The model was trained on a single Kaggle dataset and may not generalize well to ticket phrasing, terminology, or categories from other organizations. Class imbalance in the training data (see per-category support counts below) may affect performance on minority classes such as "Administrative rights."

### Recommendations

Users should validate performance on their own ticket data before deploying in production, and monitor predictions for underrepresented categories.

## How to Get Started with the Model

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
import pickle
from huggingface_hub import hf_hub_download

model_id = "Vikaash17/ticket-classification-distilbert"

tokenizer = AutoTokenizer.from_pretrained(model_id, subfolder="ticket_model_final")
model = AutoModelForSequenceClassification.from_pretrained(model_id, subfolder="ticket_model_final")

label_encoder_path = hf_hub_download(repo_id=model_id, filename="ticket_model_final/label_encoder.pkl")
with open(label_encoder_path, "rb") as f:
    label_encoder = pickle.load(f)

text = "My hr made this wrong"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = torch.argmax(logits, dim=1).item()
predicted_label = label_encoder.inverse_transform([predicted_class_id])[0]

print(predicted_label)  # e.g. "HR Support"
```

## Training Details

### Training Data

Fine-tuned on the [IT Service Ticket Classification Dataset](https://www.kaggle.com/datasets/adisongoh/it-service-ticket-classification-dataset) from Kaggle, released under CC0: Public Domain. The dataset is not redistributed in this repository. Columns: `Document` (ticket text) and `Topic_group` (label).

### Training Procedure

- Stratified train/validation/test split: 72% / 8% / 20%
- Class-weighted cross-entropy loss
- Best model selected using macro F1-score on validation set

#### Training Hyperparameters

- **Training regime:** fp32
- **Max sequence length:** 128
- **Epochs:** 3
- **Train batch size:** 32
- **Eval batch size:** 64

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data

Held-out 20% test split from the same Kaggle dataset (9,568 samples).

#### Metrics

Accuracy and macro-averaged F1-score, chosen to account for class imbalance across the 8 ticket categories.

### Results

| Model                   | Accuracy | Macro F1 |
| ------------------------ | -------: | -------: |
| **DistilBERT (this model)** | **0.88** | **0.88** |
| Logistic Regression       |     0.85 |     0.86 |
| Random Forest             |     0.83 |     0.83 |
| Multinomial Naive Bayes   |     0.74 |     0.67 |

#### Per-Category Results (DistilBERT)

| Category              | Precision | Recall | F1-score | Support |
| ---------------------- | --------: | -----: | -------: | ------: |
| Access                 |      0.89 |   0.93 |     0.91 |    1425 |
| Administrative rights   |      0.77 |   0.85 |     0.81 |     352 |
| HR Support              |      0.89 |   0.90 |     0.89 |    2183 |
| Hardware                |      0.90 |   0.83 |     0.86 |    2724 |
| Internal Project        |      0.87 |   0.91 |     0.89 |     424 |
| Miscellaneous           |      0.84 |   0.86 |     0.85 |    1412 |
| Purchase                |      0.92 |   0.92 |     0.92 |     493 |
| Storage                 |      0.89 |   0.94 |     0.91 |     555 |
| **Accuracy**            |           |        | **0.88** | **9568** |
| **Macro avg**           |  **0.87** | **0.89** | **0.88** | **9568** |
| **Weighted avg**        |  **0.88** | **0.88** | **0.88** | **9568** |

#### Summary

The fine-tuned DistilBERT model outperformed all TF-IDF-based baselines (Logistic Regression, Random Forest, Multinomial Naive Bayes), achieving the highest accuracy and macro F1-score on the held-out test set.

## Technical Specifications

### Model Architecture and Objective

DistilBERT (distilbert-base-uncased) with a sequence classification head, fine-tuned for 8-class text classification.

### Compute Infrastructure

#### Software

- PyTorch
- Hugging Face Transformers
- Hugging Face Datasets
- scikit-learn

## Model Card Contact

Vikaash17 — via Hugging Face profile or GitHub repository issues.