Amirkhan184/Qwen2.5-1.5B-Instruct-Intent-Classifier-QLoRA

Model Details

Specification	Details
Base Model	Qwen2.5-1.5B-Instruct
Fine-tuning Method	QLoRA
Task	Intent Classification
Domain	Customer Support
Framework	Hugging Face Transformers + Unsloth
Language	English

Intended Use

This model is designed for customer support systems that require automatic routing of user requests.

Example use cases:

• FAQ routing
• Account support classification
• Order-related request detection
• Human escalation detection

Example use cases

• FAQ / knowledge-base routing
• Account / billing / order request classification
• Automatic human-escalation detection
• Urgency-based prioritization of tickets

Supported output format

{
  "intent": "order_status | refund_request | account_access | ...",
  "urgency": "low | medium | high"
}

Training

The model was fine-tuned using QLoRA with:

• Unsloth
• Hugging Face Transformers
• TRL SFTTrainer
• PEFT LoRA adapters

Key hyperparameters

Hyperparameter	Value
Quantization	4-bit (QLoRA)
LoRA Rank (r)	32
LoRA Alpha	32
LoRA Dropout	0.1
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Max Sequence Length	1024
Learning Rate	5e-5
Optimizer	adamw_8bit
Effective Batch Size	8
Precision	bf16
Epochs	3

Dataset

The model was fine-tuned on a combination of:

• Bitext Customer Support Dataset
• Custom curated customer-support examples created for this project

Evaluation

The model was evaluated on a held-out test set.

Metric	Score
Valid JSON Rate	100%
Intent Accuracy	86.15%
Urgency Accuracy	92.31%
Exact Match Accuracy	86.15%

Test Samples: 65

Note: The test set is relatively small (65 samples). Results should be interpreted with caution and may not fully represent real-world performance.

Usage

This model is a LoRA adapter and requires the base model:

Qwen/Qwen2.5-1.5B-Instruct

Example:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = "Qwen/Qwen2.5-1.5B-Instruct"
adapter_model = "Amirkhan184/Qwen2.5-1.5B-Instruct-Intent-Classifier-QLoRA"

tokenizer = AutoTokenizer.from_pretrained(base_model)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    device_map="auto"
)

model = PeftModel.from_pretrained(
    model,
    adapter_model
)

Example predictions

User Message	Model Output
"Where is my package? Tracking number is XYZ123"	{"intent": "order_status", "urgency": "medium"}
"I want a full refund, this product is broken"	{"intent": "refund_request", "urgency": "high"}
"How do I change my password?"	{"intent": "account_access", "urgency": "low"}

Limitations

• This adapter is optimized for customer support intent classification.
• It may not generalize well to unrelated domains.
• Multi-intent detection is not currently supported.
• The model performance depends on the quality and coverage of the provided intent categories.

---

Related Project

This LoRA adapter is part of the Smart Support AI project, an AI-powered customer support system featuring:

• LangGraph-based workflow orchestration
• Retrieval-Augmented Generation (RAG) with FAISS
• QLoRA fine-tuned intent classification
• Structured JSON responses
• Automated evaluation pipeline

GitHub Repository:
https://github.com/AmirKhan2400/smart-support-ai