xynaxdev/ph5-url-distilbert

Ph5 URL Classifier: Multi-Class Phishing & Malware Detector (DistilBERT)

Model Summary

The Ph5 URL Classifier is a fine-tuned distilbert-base-uncased transformer engineered to detect adversarial phishing and malware distribution URLs with sub-12ms latency. It serves as the primary URL inference branch in the Ph5 Multimodal Cybersecurity Framework.

Rather than relying on static blacklist lookups or hand-engineered lexical heuristics, this model learns tokenized sub-word positional representations of fully qualified domain names (FQDNs), obfuscated paths, homoglyphs, and typosquatting patterns.

---

Performance & Evaluation Benchmark

Evaluated on an independent, strictly isolated held-out test split of 22,182 deduplicated URLs (7,500 Benign, 7,182 Phishing, 7,500 Malware):

Metric	Measured Score	Standard Deviation (5-Fold CV)
Accuracy	99.50%	± 0.03%
Macro Precision	99.50%	± 0.02%
Macro Recall	99.49%	± 0.03%
Macro F1-Score	99.49%	± 0.03%
False Positive Rate	0.51%	—
Mean Inference Time	11.2 ms	Batch Size = 1 (RTX 3050 Laptop GPU)

Baseline Comparison

Architecture	Accuracy	Macro F1	Latency
Handcrafted Lexical + Random Forest (100 Trees)	96.44%	96.44%	0.002 ms
WebPhish Character-CNN	96.30%	95.95%	8.5 ms
Ph5 DistilBERT (This Model)	99.50%	99.49%	11.2 ms

---

Confusion Matrix (Test Split: N = 22,182)

               Predicted
           Benign   Phishing   Malware
Benign      7462       23        15
Phishing     38       7118       26
Malware      11        21       7468

---

Intended Use & Integration

Python (Transformers & PyTorch)

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

# Load model and tokenizer from Hugging Face Hub
repo_id = "xynaxdev/ph5-url-distilbert"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
model.eval()

# Sample URL
url = "https://secure-login.appleid-support.com/auth"

# Tokenize and run forward pass
inputs = tokenizer(url, return_tensors="pt", max_length=64, truncation=True)
with torch.no_grad():
    logits = model(**inputs).logits
    probabilities = torch.softmax(logits, dim=-1).squeeze().tolist()

labels = ["Benign", "Phishing", "Malware"]
results = {label: round(prob, 4) for label, prob in zip(labels, probabilities)}

print(f"URL: {url}")
print(f"Predictions: {results}")
# Predicted: {'Benign': 0.0001, 'Phishing': 0.9984, 'Malware': 0.0015}

---

Training Configuration & Data Provenance

• Pretrained Backbone: distilbert-base-uncased (6 layers, 768 hidden dimensions, 12 attention heads)
• Classification Head: Linear projection (768 -> 3) with Softmax output
• Training Samples: 147,876 URLs total (103,514 train / 22,180 val / 22,182 test)
  • Benign Source: Tranco Top-1M Research List (NDSS 2019)
  • Phishing Source: OpenDNS / Cisco PhishTank verified threat feed
  • Malware Source: abuse.ch URLhaus live malware exchange
• Optimizer: AdamW (beta1=0.9, beta2=0.999, weight decay = 0.01)
• Learning Rate: 1e-5 with linear learning rate warmup
• Batch Size: 16 (Trained with PyTorch Automatic Mixed Precision)
• Token Length Limit: 64 tokens (truncated to prevent trailing dynamic victim tracking parameter leakage)

---

Citation & Framework

@article{devi2026ph5,
  title={Ph5: A Multimodal Framework for Cross-Modality Evasion-Resilient Phishing Detection in Multi-Vector Environments},
  author={Ritu Devi, Deepti Singh and Akash Sah},
  journal={Journal of Computer & Mathematical Sciences / JCMM},
  year={2026}
}