Instructions to use andref218/llama3.2-customer-support-qlora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use andref218/llama3.2-customer-support-qlora with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="andref218/llama3.2-customer-support-qlora")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("andref218/llama3.2-customer-support-qlora", device_map="auto")

PEFT
How to use andref218/llama3.2-customer-support-qlora with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use andref218/llama3.2-customer-support-qlora with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "andref218/llama3.2-customer-support-qlora"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "andref218/llama3.2-customer-support-qlora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/andref218/llama3.2-customer-support-qlora

SGLang

How to use andref218/llama3.2-customer-support-qlora with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "andref218/llama3.2-customer-support-qlora" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "andref218/llama3.2-customer-support-qlora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "andref218/llama3.2-customer-support-qlora" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "andref218/llama3.2-customer-support-qlora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use andref218/llama3.2-customer-support-qlora with Docker Model Runner:
```
docker model run hf.co/andref218/llama3.2-customer-support-qlora
```

Llama 3.2 Customer Support Assistant (QLoRA)

A QLoRA fine-tuned version of Meta Llama 3.2 3B Instruct designed to generate professional customer support responses for common e-commerce scenarios such as refunds, shipping issues, returns, order tracking, and product exchanges.

This model was developed as an educational and portfolio project demonstrating parameter-efficient fine-tuning (PEFT) using QLoRA.

Base Model

Model: Meta Llama 3.2 3B Instruct
Architecture: LlamaForCausalLM
Fine-Tuning Method: QLoRA (PEFT)
Quantization: 4-bit NF4 (BitsAndBytes)

Dataset

Fine-tuned using:

bitext/Bitext-customer-support-llm-chatbot-training-dataset

The dataset contains customer support conversations covering common customer service scenarios.

Training Configuration

Parameter	Value
Fine-Tuning	QLoRA
LoRA Rank	16
Quantization	4-bit NF4
Framework	Hugging Face Transformers
Trainer	TRL SFTTrainer
Monitoring	Weights & Biases

Training Results

Metric	Value
Final Training Loss	~0.53
Validation Loss	~0.535
Mean Token Accuracy	~82.47%
Trainable Parameters	9,175,040
Total Parameters	3,221,924,864
Trainable Percentage	0.2848%

Only 0.2848% of the model parameters were trained while keeping the remaining 99.7% frozen.

Adapter Information

This repository contains only the trained LoRA adapter weights.

To use this model, first load the base model (meta-llama/Llama-3.2-3B-Instruct) and then apply the adapter using the PEFT library.

Example

User

My package hasn't arrived.

Assistant

I'm sorry to hear that your package hasn't arrived. Could you please provide your order number or tracking number so I can check the shipping status for you?

Loading the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_MODEL = "meta-llama/Llama-3.2-3B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(BASE_MODEL)

model = PeftModel.from_pretrained(
    base_model,
    "andref218/llama3.2-customer-support-qlora"
)

Intended Use

This model is intended for:

Customer support chatbots
Educational purposes
Fine-tuning demonstrations
PEFT / QLoRA experimentation

Limitations

The model was fine-tuned on a customer support dataset and is not intended as a general-purpose assistant.
Responses should be reviewed before deployment in production environments.
Performance depends on the quality and scope of the fine-tuning dataset.

Technologies

Transformers
TRL
PEFT
QLoRA
BitsAndBytes
PyTorch
Hugging Face Datasets
Weights & Biases

Author

André Fonseca

Downloads last month: -; Downloads are not tracked for this model. How to track

Model tree for andref218/llama3.2-customer-support-qlora

Base model

meta-llama/Llama-3.2-3B-Instruct

Finetuned

(1898)

this model