README.md

---
language:
- en
license: mit
library_name: transformers
tags:
- cybersecurity
- phishing-detection
- nlp
- bert
- social-engineering
datasets:
- phishing-dataset
- email-phishing-detection
metrics:
- accuracy
- precision
- recall
- f1
- roc_auc
pipeline_tag: text-classification
widget:
- text: "URGENT: Your bank account will be suspended in 24 hours. Click here to verify your identity."
  example_title: "High Urgency Phishing Lure"
- text: "Hi team, please find attached the minutes of our quarterly project review meeting."
  example_title: "Benign Communication"
---

# Ph5 Text Classifier: Social Engineering & Email Phishing Detector (BERT-base)

<p align="center">
  <img src="https://img.shields.io/badge/Model-BERT--base-blue.svg" alt="Model">
  <img src="https://img.shields.io/badge/Accuracy-97.22%25-brightgreen.svg" alt="Accuracy">
  <img src="https://img.shields.io/badge/F1--Score-96.67%25-success.svg" alt="F1 Score">
  <img src="https://img.shields.io/badge/ROC--AUC-0.9961-red.svg" alt="ROC-AUC">
  <img src="https://img.shields.io/badge/Latency-34.2ms-orange.svg" alt="Latency">
  <img src="https://img.shields.io/badge/License-MIT-purple.svg" alt="License">
</p>

## Model Summary

The **Ph5 Text Classifier** is a fine-tuned `bert-base-uncased` language model engineered to detect psychological manipulation, urgency cues, coercive authority claims, and credential-harvesting lures across emails, SMS messages, and HTML message payloads.

It functions as the core semantic inspection branch in the **Ph5 Multimodal Cybersecurity Framework**, and is directly coupled with an optical character recognition (OCR) fallback engine to analyze text embedded within attached images and QR-code lures.

---

## Performance & Evaluation Benchmark

Evaluated on an independent, strictly isolated held-out test split of **11,652 deduplicated messages** (6,747 Benign [57.90%], 4,905 Phishing [42.10%]):

| Metric | Measured Score | Standard Deviation (5-Fold CV) |
| :--- | :--- | :--- |
| **Accuracy** | **97.22%** | ± 0.10% |
| **Precision** | **97.51%** | ± 0.08% |
| **Recall** | **95.84%** | ± 0.12% |
| **F1-Score** | **96.67%** | ± 0.09% |
| **ROC-AUC** | **0.9961** | ± 0.0004 |
| **False Positive Rate** | **1.78%** (FP = 120 / 6747) | — |
| **False Negative Rate** | **4.16%** (FN = 204 / 4905) | — |
| **Mean Inference Time** | **34.2 ms** | Batch Size = 1 (RTX 3050 Laptop GPU) |

### Baseline Comparison

| Architecture | Accuracy | Precision | Recall | F1-Score |
| :--- | :--- | :--- | :--- | :--- |
| TF-IDF + Multinomial Naive Bayes | 86.85% | 88.32% | 79.25% | 83.54% |
| TF-IDF + Logistic Regression | 88.27% | 89.24% | 82.02% | 85.48% |
| **Ph5 BERT-base (This Model)** | **97.22%** | **97.51%** | **95.84%** | **96.67%** |

---

## Confusion Matrix (Test Split: N = 11,652)

```text
               Predicted
           Benign   Phishing
Benign      6627      120
Phishing     204     4701
```

---

## Intended Use & Integration

### Python (Transformers & PyTorch)

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

# Load model and tokenizer from Hugging Face Hub
repo_id = "xynaxdev/ph5-text-bert"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
model.eval()

# Sample message text
message = "Urgent: Your account credentials have expired. Verify immediately to avoid permanent lockout."

# Tokenize and run forward pass
inputs = tokenizer(message, return_tensors="pt", max_length=256, truncation=True)
with torch.no_grad():
    logits = model(**inputs).logits
    probabilities = torch.softmax(logits, dim=-1).squeeze().tolist()

p_benign, p_phish = probabilities[0], probabilities[1]
print(f"Benign Probability:   {p_benign:.4f}")
print(f"Phishing Probability: {p_phish:.4f}")
# Output: Phishing Probability: 0.9872
```

---

## Training Configuration & Data Provenance

- **Pretrained Backbone:** `bert-base-uncased` (12 layers, 768 hidden dimensions, 12 attention heads, 110M parameters)
- **Classification Head:** Linear projection (768 -> 2)
- **Training Samples:** 77,677 messages total (54,374 train / 11,651 val / 11,652 test)
  - Sourced from curated open-access corpora on Hugging Face: `ealvaradob/phishing-dataset` and `mrm8488/email-phishing-detection`
- **Optimizer:** AdamW (beta1=0.9, beta2=0.999, weight decay = 0.01)
- **Learning Rate:** 2e-5 with linear learning rate warmup
- **Batch Size:** 16 (Trained with PyTorch Automatic Mixed Precision)
- **Epochs:** 3 epochs (Early stopping checkpoint)

---

## Citation & Framework

```bibtex
@article{devi2026ph5,
  title={Ph5: A Multimodal Framework for Cross-Modality Evasion-Resilient Phishing Detection in Multi-Vector Environments},
  author={Ritu Devi, Deepti Singh and Akash Sah},
  journal={Journal of Computer & Mathematical Sciences / JCMM},
  year={2026}
}
```