BERT Fine-Tuned for Phishing vs Legitimate URLs

This model is a fine-tuned version of ealvaradob/bert-finetuned-phishing on the Gamortsey/url_defanged dataset.
It classifies URLs (in defanged format) as either phishing-like or legitimate.

Dataset

Source: Gamortsey/url_defanged, Gamortsey/Phishing_Urls

The dataset contains phishing-like and legitimate URLs.
All phishing samples are defanged for safety (e.g., hxxps://example[.]com instead of https://example.com).

Columns

url_defanged (string) → Safe representation of a URL.
label (int) →
- 1 → Phishing-like URL
- 0 → Legitimate URL

Train / Validation Split

After preprocessing and splitting, the dataset was divided into:

DatasetDict({
    train: Dataset({
        features: ['label', 'input_ids', 'attention_mask'],
        num_rows: 1377
    })
    validation: Dataset({
        features: ['label', 'input_ids', 'attention_mask'],
        num_rows: 154
    })
})

🛠️ Training Details

Base model: ealvaradob/bert-finetuned-phishing
Tokenizer: BERT tokenizer (WordPiece)
Framework: Hugging Face Transformers + PyTorch
Hardware: Google Colab (GPU)
Training epochs: 3
Batch size: (default from Trainer with padding)
Optimizer: AdamW

Training Logs

[261/261 09:56, Epoch 3/3] Step Training Loss 50 0.000000 100 0.000000 150 0.000000 200 0.000000 250 0.000000

Final Training Loss: ~2.4e-07
Final Validation Loss: ~1.2e-07
Validation Accuracy: 100% (on held-out set)

📊 Results

Metric	Value
Training Loss	2.4e-07
Validation Loss	1.2e-07
Validation Accuracy	1.0

⚠️ Note: The dataset is relatively small (≈1.6k URLs). While accuracy reached 100% on validation, this may indicate overfitting. It is recommended to evaluate on a larger, more diverse test set for production use.

Second BERT Fine‑Tuned for Phishing URL Detection

Model Name: Gamortsey/bert-finetuned-phishing
Task: Binary classification (Phishing vs. Legitimate URLs)
Framework: Hugging Face Transformers
Base Model: bert-base-uncased (pretrained), further fine-tuned
Dataset: Gamortsey/Phishing_Urls

⭐ Overview

This model is a fine-tuned version of BERT designed to classify whether a given URL, SMS, or email contains a phishing URL or a legitimate (safe) URL. It is optimized for real-world situations such as bank scams, MoMo scams, delivery fraud, and impersonation attacks.

The goal of the fine-tuning was to:

Improve the accuracy of detecting phishing URLs
Handle noisy, real-world URL patterns
Work effectively within SMS, email, and general text messages
Support both short and long URLs

📊 Training Summary

Metric	Value
Epochs	3
Training Loss	0.0665
Validation Loss	0.000002 – 0.000004
Accuracy	100%
F1 Score	1.00

The training metrics show extremely high performance (100% accuracy and F1). This is likely due to dataset repetition, meaning some samples appear in both training and validation sets.

🔎 Notes on Dataset Quality

The dataset used (Gamortsey/Phishing_Urls) contains:

URL samples labeled as is_legitimate ("true" or "false")
Some data duplication or repetition
Some similar URL structures repeated in both splits

Becauseo f this, the evaluation metrics may appear inflated.

Future improvements include:

Deduplication of repeated URLs
Hard negative examples
Mixed SMS + email + URL input
More domain diversity

🛠 Training Details

Optimizer: AdamW
Learning Rate: 5e-5
Batch Size: 16
Max Sequence Length: 256
Hardware: GPU (Google Colab)
Training Library: Hugging Face Transformers Trainer API

Boolean Conversion: "true" → 1 (legitimate), "false" → 0 (phishing)

🧪 Evaluation Behavior

The fine-tuned model has been tested on synthetic phishing and legitimate URLs, including:

Correctly Identified as PHISHING

Correctly Identified as LEGITIMATE

The model generalizes well to real-world scam patterns such as:

Suspicious TLDs (.cn, .tk, .xyz)
Imitation of major brands
URL structures containing "verify", "update", "security", etc.

🚀 Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

model_name = "Gamortsey/bert-finetuned-defanged-url"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

phishing_detector = pipeline("text-classification", model=model, tokenizer=tokenizer)

example = "hxxps://secure-login.bankco[.]com/reset"


print(phishing_detector(example))
[{'label': 'LABEL_1', 'score': 0.9998}]  # LABEL_1 → Phishing

Framework versions

Transformers 4.56.1
Pytorch 2.8.0+cu126
Datasets 4.0.0
Tokenizers 0.22.0

Downloads last month: 5

Safetensors

Model size

0.3B params

Tensor type

F32

Model tree for Gamortsey/bert-finetuned-phishing

Base model

google-bert/bert-large-uncased

Finetuned

ealvaradob/bert-finetuned-phishing