ellachang/phishing-detector-14b-lora

phishing-detector-14b — QLoRA LoRA Adapter

QLoRA fine-tuned LoRA adapter for Qwen3-14B on phishing email binary classification.
Base model: unsloth/qwen3-14b-unsloth-bnb-4bit

Performance (400-sample test set)

Metric	Value
Accuracy	88.75%
Precision	0.833
Recall	0.970
F1-Score	0.896

Per-category accuracy:

Category	Baseline	Fine-tuned
Human-Phishing	100%	94%
Human-Legitimate	0%	62%
LLM-Phishing	100%	100%
LLM-Legitimate	0%	99%

Baseline (zero-shot Qwen3) scored 50% overall — the model could not distinguish LLM-generated emails at all.

Training Data

Human-LLM Generated Phishing-Legitimate Emails (Kaggle)
4,000 emails across 4 classes: Human-Phishing, Human-Legitimate, LLM-Phishing, LLM-Legitimate (1,000 each).
Split: 80% train / 10% val / 10% test (stratified).

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/qwen3-14b-unsloth-bnb-4bit",
    load_in_4bit=True,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "ellachang/phishing-detector-14b-lora")
tokenizer = AutoTokenizer.from_pretrained("ellachang/phishing-detector-14b-lora")

prompt = """Analyze the following email and determine if it is a phishing email or a legitimate email. Provide your classification.

### Input:
{email_text}

### Response:"""

inputs = tokenizer(prompt.format(email_text="<your email here>"), return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=20, temperature=0)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

• Method: QLoRA (4-bit quantization + LoRA)
• LoRA rank: r=16, alpha=16
• Target modules: q_proj, k_proj, v_proj, o_proj
• Hardware: NVIDIA RTX 4090 (24GB VRAM)
• Framework: Unsloth + TRL SFTTrainer

Project

Part of a term project on LLM-based phishing detection with RAG and Chain-of-Thought analysis.
Source code: [GitHub repo link]