Model Card for ticket-classification-distilbert

A fine-tuned DistilBERT model for classifying IT support tickets into topic categories.

Model Details

Model Description

This model classifies natural-language IT support ticket descriptions into one of 8 categories: Access, Administrative rights, HR Support, Hardware, Internal Project, Miscellaneous, Purchase, Storage.

• Developed by: Vikaash17
• Model type: Text classification (fine-tuned transformer)
• Language(s) (NLP): English
• License: Apache 2.0
• Finetuned from model: distilbert-base-uncased

Model Sources

• Repository: GitHub — IT_Ticket_Classification

Uses

Direct Use

This model can be used to automatically classify IT support ticket text into predefined categories, useful for automated ticket routing/triage in IT service desks.

Out-of-Scope Use

Not intended for non-English text, tickets outside the IT service-desk domain, or categories not represented in the training data. Not suitable as a general-purpose text classifier.

Bias, Risks, and Limitations

The model was trained on a single Kaggle dataset and may not generalize well to ticket phrasing, terminology, or categories from other organizations. Class imbalance in the training data (see per-category support counts below) may affect performance on minority classes such as "Administrative rights."

Recommendations

Users should validate performance on their own ticket data before deploying in production, and monitor predictions for underrepresented categories.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
import pickle
from huggingface_hub import hf_hub_download

model_id = "Vikaash17/ticket-classification-distilbert"

tokenizer = AutoTokenizer.from_pretrained(model_id, subfolder="ticket_model_final")
model = AutoModelForSequenceClassification.from_pretrained(model_id, subfolder="ticket_model_final")

label_encoder_path = hf_hub_download(repo_id=model_id, filename="ticket_model_final/label_encoder.pkl")
with open(label_encoder_path, "rb") as f:
    label_encoder = pickle.load(f)

text = "My hr made this wrong"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = torch.argmax(logits, dim=1).item()
predicted_label = label_encoder.inverse_transform([predicted_class_id])[0]

print(predicted_label)  # e.g. "HR Support"

Training Details

Training Data

Fine-tuned on the IT Service Ticket Classification Dataset from Kaggle, released under CC0: Public Domain. The dataset is not redistributed in this repository. Columns: Document (ticket text) and Topic_group (label).

Training Procedure

• Stratified train/validation/test split: 72% / 8% / 20%
• Class-weighted cross-entropy loss
• Best model selected using macro F1-score on validation set

Training Hyperparameters

• Training regime: fp32
• Max sequence length: 128
• Epochs: 3
• Train batch size: 32
• Eval batch size: 64

Evaluation

Testing Data, Factors & Metrics

Testing Data

Held-out 20% test split from the same Kaggle dataset (9,568 samples).

Metrics

Accuracy and macro-averaged F1-score, chosen to account for class imbalance across the 8 ticket categories.

Results

Model	Accuracy	Macro F1
DistilBERT (this model)	0.88	0.88
Logistic Regression	0.85	0.86
Random Forest	0.83	0.83
Multinomial Naive Bayes	0.74	0.67

Per-Category Results (DistilBERT)

Category	Precision	Recall	F1-score	Support
Access	0.89	0.93	0.91	1425
Administrative rights	0.77	0.85	0.81	352
HR Support	0.89	0.90	0.89	2183
Hardware	0.90	0.83	0.86	2724
Internal Project	0.87	0.91	0.89	424
Miscellaneous	0.84	0.86	0.85	1412
Purchase	0.92	0.92	0.92	493
Storage	0.89	0.94	0.91	555
Accuracy			0.88	9568
Macro avg	0.87	0.89	0.88	9568
Weighted avg	0.88	0.88	0.88	9568

Summary

The fine-tuned DistilBERT model outperformed all TF-IDF-based baselines (Logistic Regression, Random Forest, Multinomial Naive Bayes), achieving the highest accuracy and macro F1-score on the held-out test set.

Technical Specifications

Model Architecture and Objective

DistilBERT (distilbert-base-uncased) with a sequence classification head, fine-tuned for 8-class text classification.

Compute Infrastructure

Software

• PyTorch
• Hugging Face Transformers
• Hugging Face Datasets
• scikit-learn

Model Card Contact

Vikaash17 — via Hugging Face profile or GitHub repository issues.